Deprecated: Function set_magic_quotes_runtime() is deprecated in /home/raedan/public_html/textpattern/lib/txplib_db.php on line 14
State of Flow: Software Fundamentals::journal

Software Fundamentals

Lance Walton - Tuesday July 25, 2006

Plucking ‘solutions’ from the ether is clearly going to end in tears. Unfortunately, there is a great deal of this going on these days. The signal to noise ratio in ‘advancements’ in software development is, quite frankly, shocking.

In the history of computer programming, there have been a few advancements that represent shining beacons in an otherwise largely undistinguished narrative. For example: the works of Alan Turing, Edsger Dijkstra, Donald Knuth, Charles Hoare, Ted Codd, John McCarthy, etc.

These advancements have three things in common:

This article is not about the advancements themselves. Instead I’d like to talk about why I think the three points above are important.


Today’s practicality is often no more than the accepted form of yesterday’s theory – Kenneth L. Pike

The need for practicality hardly needs discussion; no matter how attractive and deep the theory, if it cannot be effectively operationalised, it has little value to the practising software developer.

Based on Theory

Opinion is the medium between knowledge and ignorance – Plato

The word ‘theory’ unfortunately has several meanings, some of which are certainly not what I mean when I use the word here. The meaning I intend is that pertaining to mathematical theory1.

The importance of this is established by considering alternatives:

The first of these is not at all bad. But experience is simply a mental model which has been derived over time and implies a certain amount of trial and error. Any thinking person will distill principles from this model which they will hopefully be able to communicate with others. As already stated in footnote1, this is a marginally acceptable fallback position in the absence of a mathematical theory. However, we need to remain aware that mental models are frequently excessively robust against contradictory evidence and also that simple invocation of ‘experience’ as an explanatory mechanism is often tantamount to voodoo.

Plucking ‘solutions’ from the ether is clearly going to end in tears. Unfortunately, there is a great deal of this going on these days. The signal to noise ratio in ‘advancements’ in software development is, quite frankly, shocking. There are far too many random, misguided forays and nowhere near enough study. Appalling half-baked ideas are frequently presented as cordon-bleu meals which professionals, who should know better, seem to swallow whole, only pausing to ask the provider ‘Can you supersize that?’

Having a solid theoretical basis to our work allows us to reason soundly. Of course there will always be different opinions about how best to apply the tools we have. What there should not be is vague, inconclusive discussions about what a tool is and what its implications are – an altogether too common experience in software development teams and on various websites. Segue to my next point…

Almost Universally Ignored

We can easily forgive a child who is afraid of the dark; the real tragedy of life is when men are afraid of the light – Plato

We appear to live in an anti-intellectual age. Feelings, intuition and dumbed-down points of view are required to be given as much credence as carefully studied and soundly reasoned arguments. This is utterly bizarre considering that the Western world believes that we are making a transition to a ‘knowledge economy’ in which 70% of workers are considered to be ‘information workers’.

This rejection of genuine intellect is no less present in software development, where senior figures frequently make decisions about languages, tools and infrastructure largely to exclude technologies on the basis that the average developer or development team will get into trouble because they lack the wherewithall to use them correctly.

Having said that, the position of the decision makers in this process is not entirely without basis; the quantity and quality of knowledge of fundamentals demonstrated by a great many developers is minimal2. At the same time, many display a wanton disregard for practices that would allow them to function anywhere near adequately even without those fundamentals. To use a sharp tool in such circumstances is probably asking for trouble.

The solution however, should not be to cripple the competent members of the organisation but to demand greater competence in general. Hire fewer people – 10% as many if Fred Brooks (The Mythical Man Month at The Mythical Man Month at is to be believed – pay them more and let them use the tools they require in order to reward you for your belief in them.

Some may think that I am being unfair. That my characterisation of thinking, feeling people is harsh. That people are doing the best they can, often in difficult circumstances. That in the ‘real world’ this is the way it is.

My response is simple: people are not doing the best they can. The circumstances are difficult and the world is the way it is because they make it so. If they wanted to improve things, they would gain the fundamental knowledge that would allow them to reason through the problems they face and demand better things from their suppliers, thus avoiding the confusion which currently pervades their professional lives.

Some may still think that I am unfair. That maybe people do not seek out the knowledge they are missing because they don’t know that they lack it. This might be true. However, if such people truly think, even in our industry’s infant state, that the pinnacle of software development excellence is the ability to rapidly crank out while loops, if statements and outer joins then the situation is even more dire than I fear.

1 Failing that, we should at least have a set of accepted principles that can be used as the basis for reasoning when the way is not clear.

2 Unfortunately, these senior figures also frequently demonstrate the same lack of fundamental knowledge.