elgeeko’s blog

Morgan asked an interesting question, and I thought I would post part of my response. The question is, “Why does software become slower and bulkier as time goes on?”

Jeff’s Law:
1) Users of software always need more features
2) By accommodating users, software always has more features than you need

Bill Gates is famously quoted for his claim that no computer would need more than 640K of memory. In a sense, it wasn’t entirely wrong, but the numbers are more than a thousand-fold for what we use in memory today.

Software always needs more features, support for varied hardware, and languages, and the ability to be backwards compatible with older versions (so in a sense, every new version has tricks to remember the old stuff - like remembering your past lives if you believe in reincarnation).

There is a strong movement of developers in the GNU project who develop free, open-source, targeted software that solves a single problem. However, if the software does not expand to a larger user base, or if it cannot escape its ties to an obsolete standard, it is destined to fail. So, we extend the software, and it gets bigger.

The more interesting question is “why does software speed up more slowly than hardware?”. A couple of thoughts:

First, software theory is academic, and requires incredible thought and research before extending to the next level. Hardware depends on physical techniques, and a “better” or “faster” solution often exists, though it may be cost prohibitive. That is to say that hardware techniques keep the next version in sight. However, Moore’s law is on its last leg, and when we are in the sub-45 nanometer transistor range, we have a new (perhaps academic) problem to solve.

We realize that if we can’t keep bumping up processor speed, we can just add more processors to do two things at once, hence my second thought: software theory is (at the moment) horribly inadequate for concurrent processing. Multicore processors are cool and everything; the Berkeley philosophy is “manycore”, that is collections of hundreds or thousands of processors. Describing a problem in such a way that it can be solved by several thousand sub-problems is immensely difficult. Imagine a model of a nuclear explosion, where every atom is accounted for, and conservation of mass and energy are asserted. How can I solve the position of every atom without knowing of its neighbors? If I partition the explosion into larger regions, how do I account for the interactions at the boundaries? How can I compute the next value of the Fibonacci sequence without knowing the two previous values? It turns out we need a fundamentally different understanding of software and how we describe problems.

… as a math aside, generating the nth value of the Fibonacci sequence does not require the knowledge of any of the previous values in the sequence.

If you pursue programming, you are going to learn threading, which is the worst concurrent model of computation (see The Problem with Threads). There are more advanced models, though somewhat less expressive, that enable powerful concurrent computation that is more easily analyzed and understood. This is largely an electrical engineering venture, and in my opinion, computer scientists have been hesitant to consider models of computation other than threading. So, to be blunt, many computer scientists are stuck in the past with regard to concurrent processing.

So, without regard to the imminent failure of Moore’s law, we can just pack more and more cores into a processor, but we don’t have software that knows how to use it.