The Waterfall IT Model

Ten years ago, we published an article commenting on a large computer system then being developed for Customs in New Zealand. Here are some excerpts from that article.

Yet another big government IT project has gone off the rails. This time it is something called the Joint Border Management System (JBMS) (...). It was supposed to be finished by the end of 2012 at a cost of $75.9 million. Three years later, the development is still not completed, and the budget has ballooned to $104.1 million.

This is merely the latest in a series of similar debacles. They are practically unavoidable, given the model used by government for IT procurement. It has been described as the “waterfall” model. Mike Bracken of the UK’s Government Digital Services described it as “writing most when you know least.” A group of public servants write massive tender documents attempting to guess the needs of end-users years in advance. The scope of the projects is such that only a few large consultancies can qualify to tender. The selected consultancy then gets on with over-running budgets and missing delivery deadlines. (…)

Does it have to be this way? After all, banks do on-line banking, Amazon sells books, clothing retailers of all sizes sell clothing, Trademe manages an auction site and airlines sell tickets on-line. None of these well-functioning systems were developed using the waterfall model. They were not the product of any grand design and all started as relatively small projects, run by in-house teams or small-scale contractors. Unlike public servants, the owners of those businesses did not have access to lavish amounts of other people’s money. Many of the early versions of these successful applications failed miserably but were soon replaced by others that did not.

Matt Ridley, a member of the British House of Lords, said in an article published in The Times,

[The systems that] succeed allow for plenty of low-cost trial and error and incremental change. It’s the mechanism Charles Darwin discovered that Mother Nature uses. Rather than a grand ‘creationist’ plan or a big leap, natural selection incrementally discovers success through trial and failure. From the English language to an airliner, everything successful has emerged by small steps.

The current Customs system, Cusmod, was developed around the same time as trackstock, the Warehouse Management System that we use at DSL Logistics. We like to say that trackstock is very good indeed, because it incorporates fifteen years of mistakes. Every time something goes wrong, our small in-house team of developers comes up with solutions which improve the functionality of the product. (…)

The idea that a group of public servants can specify the myriad requirements of users, years in advance, is nothing short of arrogance. Yet, New Zealand is awash with very smart developers that deliver solutions to businesses of all sizes, day in and day out. (…) Customs already records details of every transaction in a modern relational database. That provides the mechanism for the collection of statistics and the management of the revenue collection functions. (...) If the tools that they have at their disposal are not good enough, then any of the many local software developers can provide them with query/reporting tools by lunchtime, at a fraction of the IBM costs.