A Blog Post on Excel, 4x4s, Moonshots, and General Purpose Software
Using Excel is like driving a 4x4. Writing custom software is like building an interstate freeway.
With a modest amount of training, you can do amazing things with a 4x4. You can go to places where there are no roads. If you don't like where you are, you can just as easily drive somewhere new. The downside is that when you're driving off-road, you generally have to drive pretty slowly, and you have to bring your own gas.
You do have to be a specialist to build an interstate freeway. In fact, it requires an amazing amount of work from a large number of people. Yet, once it's built, a 100,000 people a day can fly by at 70mph. It offers them tremendous convenience; they don't even need to bring their own gas! On the other hand, a freeway is only useful if it goes in the direction you need to go.
Hence, Excel offers tremendous flexibility, whereas custom software offers tremendous convenience. Excel is easy to get started with, but grows unwieldy as you scale the size of your models and teams. Custom software is fairly difficult to get started with but is easier to scale in terms of size and usage.
I have to interact with Excel a lot, and yet I've spent most of my career writing custom software. What amazes me about Excel (or, rather, spreadsheets in general) is that it's so amazingly general purpose. Think of how useful spreadsheets are to businesses all over the world. There are few pieces of software that are as general purpose or as widely useful as spreadsheets are.
To some degree, it seems to me that Silicon Valley is drunk with the power of being able to write custom software. Pretty much my whole career has been spent writing it. But what about writing new general purpose software like Excel? As far as I can tell, not many people get to work on general purpose software, and even fewer people get to create new types of general purpose software.
The first commercial spreadsheet was VisiCalc. What an amazing invention! Although VisiCalc was used by far fewer people than, say, Excel, it's notable because it represented a new "type" of general purpose software; it changed the world.
Now, think about all the other types of general purpose software. Most of it was invented a long time ago. Word processors, operating systems, web servers, databases, ERP systems, etc. were all first created decades ago. That's ancient history as far as computing goes! (Although, to be fair, all of these things are incredibly recent in terms of the history of civilization or the history of mankind.)
So, where are the new types of general purpose software? It's actually kind of hard to recognize them because they haven't been around long enough or copied enough times to be recognized as a "type" of software. However, I think that we're far enough along to recognize Facebook is an instance of the type "social network." Similarly, Twitter is a "microblog" (among other things). Both of these things have had a fairly worldwide impact.
Facebook and Twitter also illustrate another thing. You could say that both of them are really just custom software, whereas the truly general pieces of software that they're built on are web browsers, web servers, databases, programming languages, and operating systems. On the other hand, as I said above, they're also instances of new types of software: social networks and microblogs. Innovation is like that. An instance of one type of thing can become its own new type of thing. Innovation is always pushing toward greater specialization.
Now consider programming languages. We've seen a lot of evolutionary improvement, but a lot of the revolutionary ideas first happened decades ago. Think of subroutines, OOP, exceptions, processes, threads, coroutines, types, recursion, closures, compilers, interpreters, virtual machines, etc. These concepts have been around for decades. Sure, we have new programming languages that each have a different philosophy on how these things should work, but most of the really big ideas happened decades ago. I thought software transactional memory might be a counterexample since it seems fairly recent, but even that started about two decades ago.
Imagine, if you were to invent a fundamentally new replacement for subroutines, create a new way to package data and/or code together, or create a new form of concurrency, what would it look like?
Of course the flip side is that it's really hard to get people to understand and accept fundamentally new ideas. It took at least a decade or two before OOP was generally considered a useful technique, and that's still not universally accepted ;) The same is happening now for functional programming. And who knows where we'll go with concurrency in the future!
Truly new ideas just don't come along that often. It takes a while for a new instance of a thing to be recognized as a new type of thing. There aren't that many "moon shots" as Google likes to put it. And when they do come along, they often take a really long time before they gain widespread acceptance. Furthermore, the first mover often isn't the one who reaps the greatest rewards. Excel wasn't the first spreadsheet. Google wasn't the first search engine. Facebook wasn't the first social network.
I'm not really saying anything new. Kevin Kelly did a much better job explaining all of this in his wonderful book, What Technology Wants, but I'll leave you with a quote from another excellent book, The Myths of Innovation, Don't worry about people stealing your ideas. If they're any good, you're going to have to cram them down people's throats!