Sat, May 13, 2017
#linux #linuxfest #foss #project-management

How to Start a (Successful?) Open Source Project

This post is for people wanting to start a FOSS project. Specifically a FOSS project which is bigger than a weekend hackfest. Though most of the advice applies there pretty nicely too.

I’m going to assume that you have some idea of what you want to build. If not this post will still have some value as it will give you ideas of how I came up with building Praelatus.

The Why

What makes you want to build something?

Why are you making whatever you’re making?

A large part of my day job right now is supporting a JIRA instance and I hate it with a passion. It’s slow buggy and I spend an inordinate amount of time reverse engineering it to fix it. The support is that bad. But the worst problem in my eyes is that there isn’t an open source alternative that even comes close in functionality. So I started Praelatus to replace it. Note that it’s ok to have multiple reasons, for Praelatus the list looks something like:

We’ll be coming back to these reasons later

The Who

This is the most important question.

Who are you building this for? What is your “target demographic”, it can be as specific or as broad as you want it to be. This is super important to know because it affects decisions all the way down. For Praelatus we’re targeting enterprises and Open Source projects. We have to make all of our decisions with that in mind.

The Success

What defines success for your project?

This really depends on the scope of your project and can be anything you want it to be. The bottom line is none of us are going to be Linus Torvalds. However, we can set out to be “traditionally successful”. A great example of this is Owncloud, they set out to make money off of their software. Praelatus has a very similiar goal though as before you can have many factors for success, and for me if Praelatus functioned to the point it could replace Bugzilla then I’d be happy even if I never got paid. For some of my smaller tools just the fact that it has someone other than me using it is enough or for a library I wrote it got featured in Python Weekly so I considered that as “success” for that project. The key thing to remember is that even when you hit this success milestone you should know that it’s not the end goal but just a step on the journey. If/When Praelatus makes money Praelatus won’t be “done” and I won’t want to stop working on it, I’ll just be proud of it.

The License

Remember the who.

The most dreaded question and one of the most important. The answer largely depends on who your target market its. For small personal tools or stuff you want to share with other developers I recommend the GPLv2 or the Apache license. For stuff you want to give away and not deal with, for instance code you’re throwing up that you don’t know if / when you’ll be able to maintain it, use the MIT or BSD licenses.

For anything you plan on “selling”, support or otherwise, you should seriously consider the Apache license. It’s one of the few that protects your copyright without infringing the rights of everyone using your stuff. Additionally businesses are pretty familiar with it so it doesn’t scare them (thanks Apache Foundation) like the GPL can.

Speaking of selling; something else that you should keep in mind is whether or not you need a CLA. The bottom line is if you plan on selling anything in relation to the software you need a CLA. If you’re not sure if you need a CLA or not then get a CLA. They are pretty low cost and you can adapt any of the multiple existing ones (again the Apache Foundation has your back with an excellent example) to your project. It’s possible that you have something you don’t feel like dealing with the rigmarole of maintaining those records, for example I don’t care about many libraries I put out there, and that’s fine but know that if you ever want to relicense the project or go “commercial” then you will have to consult with everyone who’s ever contributed so you will feel the pain then. Not to mention this process and the requirements are not well defined because copyright law in the US sucks (well it sucks everywhere but especially here).

The Friends

Don’t be scared to beg.

So now you know what you want to build, why you want to build, what success looks like, and what license you want to use. Now it’s almost time to start building. But the problem is, if you’re like me, your eyes are bigger than your stomach. One developer against the full weight of Atlassian? Yeah that’s a recipe for success. So my first goal, and yours should be to, was to make it TWO developers versus the full weight of Atlassian! I’ll take those odds!

Seriously though, if you’re building something non-trivial it’s difficult to do on your own even if you have the time and knowledge. When we develop stuff entirely by ourselves we rarely stop to ask the right questions and this is how spaghetti code gets made. When you have someone or multiple someones that you can ask for feedback as you go along you’ll have a better time.

And don’t be afraid to beg, or bargain. I convinced Mark to help me by pitching him my vision for what Praelatus could be and then said even if that vision never becomes fully realized it’ll still be a great portfolio piece for both of us; and it is, it just got him his first development job ever, and it’s opened opportunities for me that I wouldn’t have had otherwise, including writing this post!. Also remember the more the merrier, never say no to anyone who’s willing to help and don’t be scared to ask for help people generally are pretty cool.

The People

Now that you have people helping you, you have to manage them.

But the inevitable problem with getting people to help you is that like me you’ve never been a manager. Turns out that managers don’t just get to sit around all day and occasionally work. It is its own skill set and takes time to develop. Here are the three major problems while running Praelatus:

  1. Most people coming into the project have no idea what they want to do or if it’s necessary. Make sure you ask to get a feel for how they WANT to contribute but don’t ask if they’ll do something give them a task and a deadline. Deadline’s are super important work fills the space it’s given, obviously you have no bargaining chips to play with here except that you won’t give them work and will assign their tasks to someone else. You’d be surprised at how much people don’t want to “let you down”. (Wes story about email and how it was done in a week)
  2. A lot of people will say they want to learn how to do X but if you start giving them tasks that fit X and you have to teach them and they aren’t “getting it” or working on it on their own then they really don’t want X. Find what they actually enjoy and keep pushing them in that direction. (Ryan story about web dev and bots.)
  3. If someone asks if they can do something never ever say no. Never ever tell someone you don’t have something for them. If you don’t have tasks they can do then you’re failing. OSS is about collaboration if there is nothing find something, because there is always something to do on a project.

The Stack

What technology you’re going to work with.

This is probably the most painful thing I ran into on Praelatus, because I started hacking before finding other people I realized that I was working in a stack that nobody in my area knew and nobody wanted to learn. Mark and I tried Elixir / Phoenix first because we wanted to play with the new shiny and found it wasn’t mature enough. Then we split into backend and frontend so Mark could use React and JS which he was familiar with and wanted to show off. But on the backend I chose Go. I chose it because I knew it really well and had done a bunch of API’s in it but the difficulty was that once we started gathering interest locally and getting developers to sign up to help none of them knew it or cared. Additionally we got a lot of beginners looking for training and a 9,000 line Go code base was just too impenetrable (not only that but there’s no good standard way to do a lot of common webdev stuff in Go which makes it even harder for beginners because it’s not super google-able). So about 4 months ago we rewrote the backend in Python and haven’t looked back. Now I’m not saying that this is correct for everyone because it largely depends on your goals as a project. One of my life’s passions has been getting more people involved in Open Source so if I could restructure my project so I could do that while building something cool I was happy to. For you attracting beginners may not be a goal and that’s fine. This decision is the most personal I think so there’s no right answer (except Python, that’s always the right answer).

The DOCUMENTATION!

No one can read your mind.

It was not long after we got some of these newbies helping us out that I realized, oh crap no one can read my mind. We had people who were like “Ok I want to do backend development where do I start”. To which I’d reply something like “Well, how about you implement stateless sessions so we can remove the dependency on Redis?” and I’d always get a blank stare. The problem I realized was two fold: first, no one should have to ask me what needs done next. That doesn’t scale well and is confusing for anyone who isn’t local and has me as a direct resource. Second, a lot of people wont’ know what you mean or how you want something done. They should have tasks but those tasks should be well documented and structured or else they’ll complete it in some way that you won’t find acceptable and then everyone is unhappy because they wasted effort and you had to review a terrible PR.

So the short of it is there is no such thing as too much documentation. After this situation happened twice I created a Github Issue for EVERYTHING that needed done and applied labels to help beginners find their way. Note that you can’t just create an issue with a subject that says “Do X” or “Implement X” that’s not helpful, every issue or task should clearly describe what needs done and how it needs done. Including links to relevant documentation if you have them.

Beyond just tasks you should also layout the process of contributing, we have documentation on how to download, build, and run the backend including the git commands you need to start a branch and work on a feature. Additionally we have guidelines that layout our expectations for documentation and code quality. These are just as important because again you want to SET EXPECTATIONS so no one is disappointed.

The Blood, The Sweat, The Tears

You’re going to have to put in the majority of the work.

So I mentioned that Mark and I wrote the original prototype in Elixir/Phoenix, and then split to backend / frontend. Yes, well that was just us and we put in long nights and weekends to get anything resembling a working product. The main thing to remember is that NO ONE CARES about your project. You’re the only one who cares and if you can’t show them anything then they are even less likely to start caring. The cool part though is as soon as you have anything (for example I can make JSON show up from a curl call!) people are way more excited. To set your expectations let me show you this:

The Blood, The Sweat, The Tears

Lines of code we’ve written on Praelatus


------------------------------------------------------------------------

Language files blank comment code

------------------------------------------------------------------------

Go 97 2117 557 9097 JavaScript 194 1920 344 8279 Python 57 1048 807 3039
Markdown 16 762 0 2980 Elixir 57 266 183 1146 EEx 19 79 0 407 CSS 2 95
170 197 Bourne Shell 1 50 13 173 JSON 2 0 0 123 Sass 3 18 1 74 INI 1 18
0 73 YAML 4 9 1 55 PO File 1 20 18 55 HTML 2 0 0 18

------------------------------------------------------------------------

SUM: 456 6402 2094 25716

------------------------------------------------------------------------

These are the lines of code we’ve written while working on Praelatus, this includes all of our rewrites. For those who can’t read from that far away that’s TWENTY FIVE AND A HALF THOUSAND lines of code not including comments and blank lines. If you subtract about 100 of those that’s how much is just Mark and I pouring everything into it. This goes back to “No one is as passionate as you are” if you want to make something non-trivial be prepared to put in the work. Just to give some more context here are the cloc’s for our current development version:

The Current Version

Lines of code in the current development version.


------------------------------------------------------------------------

Language files blank comment code

------------------------------------------------------------------------

JavaScript 190 1322 58 6626 Python 57 1048 807 3039 Markdown 7 290 0 713
CSS 2 95 170 197 JSON 1 0 0 100 INI 1 18 0 73 YAML 2 2 0 21 HTML 2 0 0
18

------------------------------------------------------------------------

SUM: 262 2775 1035 10787

------------------------------------------------------------------------

The Marketing

Discoverability is super important.

So now you have answered all of the questions, documented your process, you have people helping you but now you need users and testers. This I think is the hardest part for a lot of us, we’re technical people we aren’t good at marketing speak or salesmanship. But if you want people to use or discover your stuff you need to start learning. I’ve found that Reddit is a great place to start. There’s a subreddit for every programming language in existence and if you post your project there as a github link asking for critique you’re sure to get at least some responses. Additionally start a local group, Meetup.com is a great place to start and is reasonably cheap. Just know that if you don’t find a way tell people about your stuff no one will know about your stuff.

The Shout outs

These guys are my heroes.

These are the first people we got working on Praelatus and they’ve been the flipping bomb. Showing up every week and reliably Getting Shit Done. I’ve put up their github user names here so they get appropriate credit, I’d recommend following them.

The End

In conclusion if you aren’t passionate about what you’re doing no one else will be so don’t do it if you don’t love it.

Mathew Robinson

chasinglogic

@chasinglogic

[email protected]

Download Resume

Linux and FOSS Enthusiast. I do the Dev and the Ops and sometimes both at once.

Home