Skip to main content

A Hybrid World of Open and Closed Source Software

Open Source Was a Lie

Okay, now that I have your attention, let me explain what I mean. Part of the promise of the Open Source movement was that it would produce higher quality software. Looking back, it's clear to see that this isn't always true. It turns out, there are many examples on both sides. Sometimes the open source option is better, and sometimes the closed source option is better. For instance, Apache is far more secure and robust than IIS. Linux is far more reliable than Windows 95. On the other hand, Photoshop is way nicer than the GIMP, and my Java friends have told me that Intelli J IDEA is nicer than Eclipse. I've even heard that Solaris still beats Linux when it comes to NFS robustness.

Stallman Was Right All Along (It's about the freedom, baby! Yeah!)

Stallman actually predicted the above. In the early days, he was pulling his hair out screaming, "It's not about the quality! It's about the freedom!" That's why I now put myself in the Free Software camp.

Unfortunately, at this point, I don't think a Stallman-esque utopia of Free Software is going to happen. Software is a business. There are many business models that work well with Free Software. However, there are many more business models that can work using a hybrid of open and closed source software. Therefore, it should be no surprise that this is what the industy has settled on.

Absolutes Don't Last

Absolutes always seem to mellow in the long term. Communism fell in the Soviet Union in 1991, but one could argue that capitalism fell in the United States back in 1929. The truth of the matter is that the US has never had pure capitalism. It has steadily become more a composite of capitalism and socialism since the time of Franklin Roosevelt.

Similarly, CISC vs. RISC used to be a hot debate. These days, it's an almost irrelevant distinction. What happened is that chips became RISC on the inside and CISC on the outside.

FOSS Stopped the Monopoly

In a sense, the biggest thing we have to thank Free and Open Source Software for is that it stopped the monopoly. To some degree Microsoft had already won the game. FOSS opened it up again.

At the risk of sounding melodramatic, I like to think of it as an epic war between two semi-autistic geniuses. Gates had conquered the major battle for shrink-wrap software. Stallman simply changed the rules of engagement for future encounters by creating the Free Software movement. Suddenly, small companies could hold their own without being crushed the minute Microsoft decided to enter a new market. After all, it's never a good idea to let your enemy be your supplier.

Of course, it remains to be seen how badly patents will hurt all those small companies. As far as I can tell, it's impossible to write any decently-sized software system without violating someone's patents. The bigger the competitor, the more likely they'll hold patents that your code violates.

FOSS Meant an End to the Nightmare

When I was in college, I didn't have access to the source code for my operating system (Windows 95) or a free compiler (I was using Borland's Turbo Pascal). When I learned C++, I was too poor to afford a C++ compiler. It was then that my professor gave me a Linux distro (Slackware, of course). It was a quasi-religious discovery for me. I was no longer limited by money, and I had access to the source code for everything. The only thing that limited me was my own mind. FOSS meant an end to the nightmare of living in a closed source world.

In a sense, I've never fully recovered from that. That is why I have such a hard time accepting software like OS X, TextMate, Perforce, Jira, etc. Don't get me wrong, I've heard that they're all great. However, to me, they represent a return to a nightmare that I thought I had left behind.

Innovating in a Hybrid World

These days, the companies that are doing the best are the ones that are innovating in a hybrid world. For instance, at the surface, LAMP stands for Linux, Apache, MySQL, and PHP. However, at a deeper level, LAMP represents a gold rush of companies using FOSS to build customer-facing, closed source Web sites in the hope of being overnight millionaires. In a few dozen cases, it's even worked.

Apple

One company that's doing really well in a hybrid world is Apple. OS X is a beautiful mix of open and closed source software. I think the FOSS movement played a large role in bringing them back from the dead. It provided a great base for them on which to innovate.

Frustratingly, much of what they did best (e.g. Cocoa and Aqua) remain closed source. In a hybrid world, that's a very reasonable business decision, but it's still frustrating for me. Of course, the little Stallman voice inside my head likes to remind me that this is nothing new. Cocoa is just another in a line of nice, but closed source GUI toolkits such as Motif and QT (although QT became GPL later).

I hate the way Apple treats its customers. However, what can you do? Fork the source code? In a hybrid world, you have access to a lot of their source code, but forking the whole thing is simply not an option. That's critical for Apple's success, of course. A hybrid world presents a mixed bag of pros and cons for all those involved.

Google (Linux for the win!)

Google is another company that's doing extremely well in a hybrid world. Compared to Apple, they've been even more helpful to the FOSS world. It's not just the big things such as Google Code (which includes Summer of Code and project hosting); it's also the little things such as the fact that it lets my local Python interest group host its meetings on campus. They also let Guido van Rossum, the author of Python, spend 50% of his time working on Python with no strings attached.

Like Apple, Google has also strategically withheld the source code for much of its "special sauce." I can understand why they're search algorithms are closed source, but I really do wish they'd open up their implementations of MapReduce, GFS, and BigTable. (To be fair, they did write white papers on them, which was a great help for the rest of the world.)

IBM, Oracle, and Sun

Transitioning to a hybrid world was great for IBM. After they botched the DOS deal with Microsoft and got their lunch eaten with OS/2, they focused on hardware, mainframes, and consulting. When it turned out that it took specialists to make Linux pay off, they stepped up to the plate saying, "Hey, we can provide consultants for that too!"

Oracle jumped onto the bandwagon as well. They embraced Linux, but I'm not under the delusion that it was based on philanthropy. They used Linux as a way to stab their partner Sun Microsystems in the back and try for a bigger piece of the pie.

As for Sun Microsystems, a whole book could be written about their love hate relationship with FOSS. Does anyone remember downloading Linux from SunSITE? They finally opened up Java after years of fighting against it, but the battle rages on. If you go to download OpenSolaris, you'll quickly find out that the "OpenSolaris project does not provide an end-user product or complete distribution." They open sourced a bunch of code, but not enough to run a complete system. I guess living in a hybrid world is what they've always wanted.

My Heroes

In trying to make sense of this whole mess, I've paid close attention to my heroes.

Guido van Rossum, the author of Python, and Alex Martelli, the author of two Python books, both carry Apple laptops.

Bram Moolenaar, the author of Vim, also totes an Apple. I really look up to Bram since Vim is charityware. (I am sympathetic to Bram's sympathy for orphans.) When I asked Bram about his laptop, he said something like, "Open Source is a lot of fun, but everyone has to make money, and closed source software is an easy way to do it."

More than half of the FreeBSD commiters I know (which is somewhere between 7-10) use Apple laptops or desktops these days. Perhaps this should come as no surprise. OS X uses a lot of FreeBSD code. Jordan Hubbard, who co-founded the FreeBSD project, is now the "Director of Engineering of Unix Technologies" at Apple. The little Stallman voice inside my head likes to remind me that the license used by the FreeBSD project is sympathetic to closed source derivatives.

Even Linus Torvalds, the author of Linux, wasn't against using a closed source revision control system, although he later decided to write his own.

Then there's my mentor Mike Cheponis. Mike is an old-school hacker from MIT. He also worked on Unix back when Unix was an actual OS and not just a "style" of OS. Having worked at Apple, he doesn't share my distaste for it. On the other hand, he agrees with me that behind closed doors Apple is just another Microsoft-wannabe. He carries an Apple laptop that also has Windows and NetBSD on it.

Last (but in no way least), there's Stallman. He still has not given up the dream. He started the Free Software movement as a result of the crushing loneliness he faced when all of his friends left to form Symbolics and excluded him. I met Stallman for the first time a couple months ago. I actually gave him a hug. Love him or hate him, it's impossible to disagree with the fact that he's made the software world a better place. Sometimes we need extremists to push us in the right direction. On a personal note, my only regret writing this is that he would be disappointed in me for conceding to a hybrid world. The little Stallman voice in my head likes to chide me saying, "But we've come so far! How can you give up now?"

Picking the Future

Okay, so we're not going to have a world of absolutes. What do we do about it?

Should we abandon the GPL for more commercial-friendly licenses such as the one used by the FreeBSD project? I don't think so. If a company wants to make use of and contribute to FOSS, that's great. However, if I want to write a library in my spare time, and I don't want anyone else producing closed source derivatives of my work, so be it. Linux and FreeBSD use very different licenses, yet they are both a benefit to the world. Anyone who works at a company and complains that they can't make derivative, closed source works of GPLed software should remember that they can't make any derivative works of proprietary software. A hybrid world is still better than a closed source world.

We'll need to choose what we insist on being Open Source and what we'll concede to using closed source. I strongly believe that any programming language that isn't Open Source isn't worth using. Similarly, it's quite unfortunate to write Open Source software that relies on closed source libraries. Let's also give three cheers for Firefox. Closed source software is one thing, but an entire Web that's only viewable using a closed source browser would be a tragedy indeed (which is the primary reason I don't like Flash).

On the other hand, I don't think it's the end of the world that TurboTax is closed source. Writing tax software has got to suck, and I have no wish whatsoever to ever work on it.

What about TextMate, which I mentioned earlier? From what I've heard, Allan Odgaard has done an excellent job in producing a beautiful, friendly, and powerful editor. I don't fault him in the least for wanting to make a living building such an editor. Nor do I fault anyone else for using it. However, for me personally, using a closed source editor 8-10 hours a day sounds like a nightmare. In general, programmers love writing editors; I know that I've contributed several patches to Vim. There's no reason for me to ever settle on a closed source editor when it's such a core part of my life as a coder. I'd sooner write my own from scratch.

Conclusion

What we have now and what we will continue to have is a hybrid world of open and closed source software. To those who think that there's no place for Open Source software in the commercial world: Good luck trying to take on Apple and Google! To those who think the Free Software movement will continue and conquer the world: I sympathize. Don't flame me for believing as I do. Go write more Free Software! After all:

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw

Comments

bobby said…
JJ,

Nice post, but I have to wonder if it's you convincing yourself it's ok to buy a MacBook Pro....

Bobby
> Nice post, but I have to wonder if it's you convincing yourself it's ok to buy a MacBook Pro....

Hahaha, so true ;)
steve said…
I like vim and scribes myself. I can't bring myself to use editors that are closed or need close sources libraries. I've tried before but every time I end up feeling dirty and the marginal niceness they might (or might not) provide is not worth it.

In a hybrid world I really only want Free Software.
mikeal said…
Great post but.... Jira is totally worthless and not worth your time.
ben said…
If there was a little Stallman voice in your head, however did you manage to ignore its continuous screaming the past years you've been spending all day writing closed-source software? ;)

How you can continue to talk about how horrible it is to merely *use* closed source software daily, yet manage to write closed source software all day will always fascinate me. Stallman gave Linus some crap cause he merely used closed source software. I'm sure Stallman would rather having people merely using some closed source software instead of spending all day writing it, no?

You say its unfortunate to write open source software that relies on closed source libraries, what about using open-source software to write closed-source software? ;)

If you're thinking of buying a mac, think of it this way.... closed source money bought a closed source laptop. Your car, your house, the food on your plates, its all been paid for by the closed source software you've written. Having a mac, running a few closed-source programs.... hardly counts as anything against the mass of closed source software you've produced in your life.

So on a happy note, get a mac, enjoy, and write some more open-source!
> In a hybrid world I really only want Free Software.

Haha, I agree. Notice that the only time I really didn't care whether it was open source or not was for tax software ;)
> If there was a little Stallman voice in your head, however did you manage to ignore its continuous screaming the past years you've been spending all day writing closed-source software? ;)

It's easy. My wife scares me more than Stallman ;)

> How you can continue to talk about how horrible it is to merely *use* closed source software daily,

When did I say that?

> yet manage to write closed source software all day will always fascinate me.

Isn't that what this post is about? Did you miss my point?

> You say its unfortunate to write open source software that relies on closed source libraries, what about using open-source software to write closed-source software? ;)

I think you're still missing my point. Writing open source software that relies on closed source libraries sucks because the software isn't really free. Using open-source software to write closed-source software is what this point is about--i.e. that we have a hybrid world whether we like it or not.

> If you're thinking of buying a mac, think of it this way.... closed source money bought a closed source laptop.

That's exactly how I think of it. Didn't I make that clear in the post?

Happy Hacking, Ben!
wavy davy said…
I guess your first point is intentional flame-bait, but it made me think.

For me, FOSS idealogy never said "all open source software will be good software", or "all closed source will be bad software". Software is complex, and building something that's usable and what people actually want is even more complex.

So the first point seems a bit straw man. Plus the phrase "I've heard X is way nicer than Y" is about as subjective as you can get :)

One area it seems to me that FOSS has the technical advantage is security, but YMMV.

I'd like to see some statistics on the distribution of software quality amongst FOSS vs closed projects, but I don't think thats a measurable quantity ('cos, erm, they're closed!).

On the other points, I think I agree with you. For me, the biggest advantage of FOSS the ability to grok the code and change things. The philosophical and technical quality side of things are less important to me. It's been a god-send at work since MS "opened" the source code to big chunks of the .net framework. This didn't provide the philosophical benefits, nor are they accepting contributions to improve the quality of the implementation, but my job just qot a lot easier.
JMC said…
Great article!

I completely agree with your sentiments. I don't condemn anyone for using or writing closed source software. It is, after all, a very good way to make a living. I do this myself. That being said, my true passion is with free software. I would never invest the kind of time into something like my editor (vim) if it were not open source. It's just not worth it.
> Great article!

Thanks!
Kris said…
Interesting comments... I'm kind of pondering the intersection of closed and open right now in fact.

I'm a newcomer to Python and bought my first MacBook over a year ago. Being a full-time windows developer by day, and mac user by night... I find myself wishing I could spend more of my time on the mac, and less on windows. I really like python given it's brevity and easy readability. Anything which allows me to type far less and not live in an increasingly larger IDE like visual studio seems attractive. I estimate a solid hour of each day gets devoted to watching an hour glass spin or VS.NET crash.

I found your blog via my interests in Python, as well as VI. All this being said: I would like to write commercial software that relies on open source tools like Python, etc. I'm considering a web application that users could purchase and host on their own, however; opening up the source code seems a bit scary.

Part of me says that the people who steal your source weren't going to pay for your product anyways, so maybe I'm obsessing over things. In your opinion, is it suicidal to write apps in python that you want to sell for commercial deployments (and yes, many of these could be dropped on to Windows servers)? I don't want to write a hosted service as I don't have the resources, nor do I believe that customers are as attracted to this model since they want their data close to home.

Thanks!
oinopion said…
I totally understand what you feel. But I've chosen freedom. Perhaps Mac is better/nicer than my Kubuntu on my crappy, cheap and ugly laptop, but I'm happy to use software someone wrote mostly for passion and fun.
> I totally understand what you feel. But I've chosen freedom. Perhaps Mac is better/nicer than my Kubuntu on my crappy, cheap and ugly laptop, but I'm happy to use software someone wrote mostly for passion and fun.

Thanks. Glad to hear I'm not the only one who feels that way ;)
> I estimate a solid hour of each day gets devoted to watching an hour glass spin or VS.NET crash.

Wow. I was under the impression that Visual Studio and .NET were a lot nicer than that.

> I found your blog via my interests in Python, as well as VI.

Coool.

> I'm considering a web application that users could purchase and host on their own, however; opening up the source code seems a bit scary.

Selling Web applications as software is something outside my realm of expertise. There aren't that many companies that can make it work, especially if there's an open source competitor. Jira and FogBugz are two examples that come to mind.

What you're talking about isn't open source. Remember that "open souce" means more than just providing access to the source. It also means that it's okay to make modifications and pass it off to your friends.

WingIDE is one interesting company that sells a commercial product. (It's a very nice Python IDE, by the way). They let people have access to the source, but only if they sign an NDA. Perhaps that's slightly less heinous than being completely closed source.

> Part of me says that the people who steal your source weren't going to pay for your product anyways,

What does it mean to "steal your source"? Perhaps you're referring to the fact that Python doesn't get compiled in the same way Java does. Remember, you can still distribute just the .pycs or use py.exe. Using Python doesn't necessarily mean that it has to be open source.

> is it suicidal to write apps in python that you want to sell for commercial deployments

No. Look at how Wing IDE or Komodo IDE do it.

> I don't want to write a hosted service as I don't have the resources, nor do I believe that customers are as attracted to this model since they want their data close to home.

Some aren't. Some are. Personally, I love using Gmail.

Happy Hacking!
Yesterday, I went to a talk by Bruce Perens. He's one of the guys who created the Open Source movement. His view is remarkably different than the Free Software movement. In fact, it matches what I proposed in this post.

He's a strong believer in open sourcing the non-differentiating code in a company, and leaving the "special sauce" proprietary. He thinks that about 95% of the code at a company is non-differentiating and need not be proprietary.

I asked him how to convince the inner Stallman in my head of his view on Open Source. He said that he counted Stallman a friend even though Stallman was the world's hardest person to be friends with. He also said that if I truly believed in Free Software, he would never want to take that away from me. In fact, Open Source is meant to be a baby step along the way to Free Software.

He said there's no shame in writing proprietary software. He said that unless I were a genius with no kids like Stallman, it's not so unreasonable to make a living writing proprietary software. In my mind, Perens' view on Open Source seem more achievable, as I've articulated in this article.

One of the other people at the talk said that he had a friend who made a successful living writing only Free Software.

My takeaway is that both views are acceptable, and you shouldn't feel ashamed of holding either.
By the way, Perens used a Linux laptop ;)
Stuart Ellis said…
Excellent post. I've been wrestling with these issues recently (like lots of people, it seems), and it's good to read a clear and balanced take.
Thanks, Stuart. It's funny how you and I agonize about these things, while so many people completely ignore them.