Avian’s Blog

Software is broken. No, really.

Ask any self respecting software engineer and he'll tell you that software never breaks. It can't wear out in the same way a mathematical equation never degrades over the years of use. The same person is usually quick to add how much he hates the hardware his perfect programs run on. Hard drives always fail when you need them, CPUs overheat and fan bearings seize.

Why is it then that software failure has become so ubiquitous in our lives, that a catastrophic failure in most systems does not even fall under warranty terms, while hardware is guaranteed to work for at least a year without errors or your money back? Why must basically every device today have a little button that says "reset" (or else you curse it because then that same common operation involves removing a battery or pulling the plug). Watchdog timers are common, a mechanism where imperfect hardware helps infallible algorithms do their job. I'm sure the possibility of data loss due to some software bug is several orders of magnitude higher that that of hardware failure.

Photo by Chris Shrigley CC BY 2.5

The software itself may indeed be immune to wear and tear (although even that could be debated), but its human authors are all but perfect, especially when faced with the immense complexity that is common today in software engineering. In contrast to physical products, software is usually equally broken when it's brand new as when it's of a ripe old age.

Complexity is causing all of these problems. Vast majority of production versions of software today should fall under the label of crude prototype. Engineering means understanding what you are doing. Software engineers do not understand their creations. Not with all the layers of abstraction, from high level programming languages, to underlying operating systems and complex CPU instruction sets. Even if you're writing low-level assembly, chances are you can't predict exactly how your code will execute on a user's PC. And given the reliability of embedded software in consumer electronics it looks like that's impossible even when you know exactly what hardware the program will run on.

High-level programming languages have made this problem worse. They give the programmer a false sense of security. It was way too easy for a C program to outgrow its creator's capacity to comprehend all its possible execution paths. It's stupidly easy to do that in Python. Latest trend towards web applications sounds like a bad joke in this respect. Industry that isn't capable of creating reliable consumer software that runs on a single computer wants to move to systems that span thousands of interconnected processes.

Physical systems do not tend to grow that large because production costs rise fast with complexity. Software has no production costs, only design and prototyping. And even then majority of design is usually skipped in favor of getting a semi-working prototype out on the market as soon as possible. The lack of documentation and write-only code is a running joke that comes true way to often.

Code reuse is seen by some as a holy grail that will solve this problem. The theory is that you use a library of well checked, proven code instead of rolling your own, probably flawed solution to a common problem. In practice however, this usually means that such code is used without understanding its full behavior and side effects, even when they are properly documented. It also makes it easier to make a blind assumption that someone else did his homework properly so you don't have to. In short, it makes the software author think he's actually in control.

This is not a technological problem and as such can not be solved purely by technological means. Software is still a novelty. Most users will fall for the shiniest, best advertised product, not for the one that will serve them best. Sadly, shiniest is usually the one with most features and hence the most complex and unreliable. Hopefully this will slowly correct itself when market gets more educated and computers stop being magic dust to most people. It's shockingly apparent that today in a lot of cases the final users are the ones that have the worst ideas about what functionality the product should have.

The software industry should also get its act together. It should have the courage to resist the vocal minority of users demanding thousands of new features and focus on providing simpler software that will work for the silent majority. Bugs should not be seen as an unavoidable problem. The engineering community should learn to respect simple, reliable solutions, not the most clever. Engineers should get a firm grasp of the complexity of the systems they are working on, even beyond the lines of code they themselves have written.

And finally new developer tools that aim to help this situation should focus on revealing the underlying complexity, not hiding it. They should help writing better software, not help writing more software. Rapid application development should become a thing of the past.