In 1970, Dr. Winston Royce wrote the paper “that started it all” for waterfall project management in IT. His paper, “Managing The Development of Large Software Systems” outlines an incredibly rigorous process that was adopted by the United States Department of Defense. From there, this approach has spread throughout the IT world until it is so entrenched in the IT world that it is almost unavoidable.
Since Dr. Royce’s paper is frequently cited by those who promote agile project management, I felt it was important that I read it. There were several things that stood out for me, points that everyone should understand about waterfall project management:
Dr. Royce admits the waterfall approach is “risky and invites failure.” He then goes on to detail a rigorous approach to waterfall that reduces the risk of failure, but at a substantial cost.
The world of software development has completely and fundamentally changed since 1970. We don’t program on cards anymore. We have user interfaces, servers, the web, desktops, applications, etc. Not to mention, computers do so much more than calculate re-entry paths for primitive spacecraft, which is the the type of capability Dr. Royce’s projects implemented.
The waterfall approach is incredibly expensive. He claims that a true waterfall project, one with adequate rigor to be successful, must have a minimum of 1500 pages of specification documentation for a $5,000,000 software project (1970’s $’s – nearly $27,000,000 today). Those documents would be rigorously reviewed by many people. I haven’t calculated how much that might cost, but you can guess it would be a lot.
The waterfall approach doesn’t result in maintainable software. Remember the 1500 page specification for your IT project? That must be maintained throughout the life of the application if you are to have any hope of keeping the software operational.
Dr. Royce’s assertion of the value of waterfall is not based on scientific rigor of any kind, no matter how rigorous the prescribed approach is. Dr. Royce, clearly an experienced scientist, makes a fundamental error in the rigor of his approach. He states that he believes in the waterfall approach, but never cites his reasoning for this belief. He simply believes it to be true. To this day, there is still no conclusive evidence supporting the claim that IT projects should be managed this way, although most of the IT world behaves as if there is. On the contrary, there is a substantial body of research that, even with the rigor Dr. Royce prescribes, waterfall projects still are risky and invite failure.
For the last couple of years I’ve sort of ridden the fence on where I stand on project management. I’ve conceded that waterfall might have a place in the IT world in the past, that it’s a tool we should have in our toolbox, even if we don’t use it very often. While I still feel that way about agile, and other approaches like spiral, iterative, etc., I’m getting off the fence with waterfall.
The waterfall approach to managing software development projects is one of the most costly mis-diagnoses ever. It structures projects for failure. It creates a cost-change curve that is unacceptably expensive (you know, the cost of a change grows exponentially as the project progresses) and is largely responsible for the high failure rate of IT projects. Waterfall creates a project structure that is rigid and forces you to make decisions and predictions when you know the least, then abide by them no matter what realities you expose in the process.
I am off the fence. Waterfall doesn’t belong in any IT project manager’s toolbox. It is based on and littered with myth. There is not a single shred of scientific evidence, at least as far as software development is concerned, to support the idea that waterfall actually works. It is a cancer between IT professionals and the business they support. It cripples a corporation’s ability to evolve their business, to use technology to meet their goals. It is too expensive and provides too little by way of results.
Thirty-seven years ago Dr. Winston Royce answered the question of how to make waterfall “work,” and it isn’t pretty. Unfortunately, he never bothered to ask whether we should, to question the assumption that software must be developed sequentially. I’m going to cut him a little slack. It’s pretty hard to change software that was written with punch cards. But I’m not extending the same courtesy to the IT professionals who have swallowed that notion since then. If you believe in waterfall as the “best” way to develop software, the one “right” way, you are like a lawyer carrying a claw hammer in his briefcase. A hammer is useful for a carpenter, but lawyers who rely on them don’t do well. Get some real tools.
Before you get ballistic with me, I want to point out one thing. I’ve never seen a project manager who actually manages projects via pure waterfall. I’ve seen many try, but in the end they ALWAYS scrap the “holy process” and become pragmatic, doing only what it takes to deliver their project. Yet, so many of us still talk about waterfall as if it represents some utopic state of software development, as if the achievement of pure waterfall would be something like heaven, streets paved with gold and praises sung to requirements documents. That’s the belief I want to challenge, this theoretical ideal world where everything happens sequentially “because that’s the right way to do things.”
So, take a minute and ask yourself the question Dr. Royce failed to ask. Does software have to be be build sequentially? Is it REALLY like a house, in that you can’t put up walls without first having a detailed blueprint? You can’t frame without a foundation? You can’t shingle a roof if you haven’t installed joists?
I have built many software applications non-sequentially. I’ve build user interfaces without data models. I’ve built data models without requirements. I’ve made significant design changes late in the project. And I’ve done all of this cheaper, and at less risk, than would have been required using waterfall.
One more thought, and then I’ll give up on this rant. It’s too long already. Here’s my final thought, and the biggest fundamental issue I have with waterfall. I believe the obligation of every IT professional is to provide value to the business they serve, well beyond the amount of the salary they receive. Our obligation is to make our companies tougher, more competitive, smarter, and more profitable. That’s why we’re there. To make our employers better. That means we, as project managers, have an obligation to our employer to deliver valuable software at the LOWEST COST POSSIBLE. Whatever quasi-religious beliefs we have about the “right way” to deliver software should be less important than this goal.