Now, it’s interesting to note that my agile experience was a bottom-up implementation in the sense that it wasn’t part of an organization-wide agile implementation. Our CIO simply mentioned Agile in a large meeting, and I took that as implicit permission to mess around with the ideas behind the Agile Manifesto. The fact that he had mentioned it meant that when I was finally open about what I was doing people in my management chain were hesitant to try to stop me because they didn’t want to look stupid to the boss.

This experience made me believe that there is really only one way to make agile stick in corporate IT: to ingrain it into the culture of an entire company (not just IT) so deeply that no single person can root it out. The only way to do that is to make it a fundamental part of the operating model of the company. In other words, the business has to demand it. Not request it, not gradually migrate toward it, not make it an optional approach for the interested, but demand it.

The funny thing is, the business doesn’t really need to know that what they’re demanding is Agile. In fact, demanding Agile explicitly is probably a bad idea because smart CIO’s will immediately bombard them with the common agile-doesn’t-scale or agilists-are-cowboys or some other compelling argument that the business will find hard to ignore.

No, what the business needs to demand must be far more fundamental than that. The demand needs to simultaneously make the agile values imperative AND reject all processes/methodologies that conflict with the agile values. You see, I think it’s not enough to put Agile and Waterfall on equal footing. Waterfall, for software development, must be destroyed.

So… what demand do they make?

I believe it’s simple, and it’s something the business understands. It’s earned value. Earned value is usually calculated as a fraction of the total value of the project, based on progress toward completion. It is the fundamental premise behind Waterfall, and if you can change the way earned value is determined you can put Waterfall behind you and clear the way for Agile and other methodologies like it.

Only working software has value. All of the artifacts leading up to working software have zero value. They are worthless. This is the thing, that above all the other ideas that came out of the Agile Manifesto, matters the most. It’s the driving force behind short iterations, face to face interaction, etc. It’s the thing that started it all.

If the business accepts this, believes it, and establishes it as a guiding principle of the business’s operating model, IT will do what they’ve always done: what the business tells them to.

Better yet, the funding model for IT projects will change dramatically. Instead of stage gates that force the high-risk, unwieldy, and wasteful project phases of waterfall on IT project managers, funding stage gates will be built around the agile concept of earned value. Instead of giving project teams a month to come up with an enterprise design proposal that will be used to award the next round of funding, IT teams will be given a month to implement some portion of the business capabilities needed.

But… we don’t even use earned value in our waterfall projects now. What difference will it make if the definition of earned value changes?

You might not report on earned value, but you still use it. You can’t do waterfall without fundamentally buying into the concept that intermediate artifacts (I refuse to call them deliverables) such as requirements documents have value. If they didn’t have any value, then the risk management portion of the waterfall approach would force you to abandon them. It’s the philosophy behind the earned value formula that matters, and whether you actively calculate EV or not doesn’t really matter.

Earned value is the key to making agile stick. If the business recognizes only working software as having value, IT processes will follow suit.

That is, not until Collaborative Software Initiative and the State of Utah open source UT-NEDSS, their web-based infectious disease reporting and management system later this year. Can you get any more vertical than that?

Here’s a quote from a recent press release about this project that I think highlights the value of applying open source approaches to vertical problems in the specific context of infectious disease management:

“The project is a perfect example of how collaboration in software can have an impact on society – in this case, we can help prevent the spread of disease and improve quality of care for patients by developing a system that works for everyone,” said Stuart Cohen, CEO of CSI. “The State of Utah is taking a necessary leadership role to begin the rollout of an infectious disease reporting and management system for the 21st century. We are very excited to enable that transition and to work with other states to deploy this important system.”

???????? ????? ????????

I think vertical open source solutions like UT-NEDSS are going to have a very important role in the overall IT landscape in the future. Enterprising organizations and individuals who can take open source platforms, apply them to vertical problems, and then release them to the programming world are going to be major players in shaping the software industry. I think it’s really cool. And I’m proud to be involved with this project, even if it’s just a minor role.

This was typical back then, in the good ole days. Work hardware was almost always better than what you could afford at home.

Things have certainly changed since 1997. Check out the system that graces my home office. Sure, I use it for blogging, writing, and the occasional freelance testing gig, but aside from the second monitor it’s not very different from what an average citizen of the IT world has at home.

Now compare it to what you have on your desk at work. Do the words “fully depreciated” come to mind? And where’s your dual monitor? Never mind the decades of research that have proven how much dual monitors increase productivity.

No wonder kids don’t want to study computer science in school anymore. What is IT going to do to attract people who like tech toys if IT doesn’t have tech toys anymore?

The bill, which goes into effect immediately, will make it a criminal offense to use software development lifecycles that divide software projects into serially executed phases distinguished by a particular type of work activity. Violators will be stripped of their project management certifications and could face up to six weeks in jail for their first offense.

“Software projects should be structured in a way that allows them to build value quickly by incrementally rolling out valuable software that works,” said Tate Stuntz of Nimble Consulting and erstwhile contributor to TechDarkSide.com. “This is in stark contrast to projects where all the requirements are delivered up front, then locked down by change management processes that resemble litigation processes.”

Not everyone is pleased with this change. Dave Christiansen, author of Technology Dark Side, a Corporate IT Survival Guide, inquired how this bill would impact Governor Daniels efforts to bring new jobs to his state, prompting a terse response from the Governor. “Bringing new jobs to the state is important, but it’s not appropriate to create jobs by implementing bloated, top heavy project management approaches. Project managers and organizations that can’t adapt won’t survive – in fact, the worst of them will end up in jail.”

Governor Daniels didn’t provide any details about how the law will be enforced at the signing event. Shortly after the event, we were fortunate enough to get some details from a government insider who was willing to speak anonymously about the enforcement process.

“Basically, the whole enforcement process is going to rely on the savvy and judgment of Dog the Bounty Hunter,” she told us. “Dog‘s going to be randomly dropping in on software projects and apprehending project managers, tech leads, and business sponsors who value comprehensive documentation over working software. He’s attending training in San Francisco right now to learn the key ways of identifying a waterfall lifecycle.”

Another government official, who also spoke anonymously, gave a stern warning about trying to hide behind clever lifecycle names that attempt to disguise the true nature of the software development process. “Dog can tell the difference between waterfall and other methodologies, no matter what you call it. He can smell legalistic change management processes a mile away. Plus, there’s an anonymous hotline your colleagues can use to rat you out if you don’t stop making processes more important than people.”

It’s clear that this new law will have widespread, sweeping impact on the lives of project managers in Indiana. “I’m thinking about leaving the state,” one project manager we spoke to said. “I just don’t feel comfortable without a nice complicated Gantt chart showing me the way. I guess I’ll find work in Ohio and hope this thing blows over.”

Other project managers were more resistant to the new law. “We’re going underground,” a group of tough looking PM’s whispered after we offered to buy them Starbucks Coffee. “The government can’t make us give up our Microsoft Project licenses – we’ll just keep two plans. A nice vague, feature-based plan for the witch hunters, and a second secret plan that lays out our detailed phase based plan for the team to use.”

When we asked how the State’s Waterfall SDLC Alert Hotline would impact their plans to secretly continue serial development, the leader of the group spoke angrily. “There are ways of dealing with traitors. Once one or two of ‘em try it, and they see what happens, everyone else will just learn to keep their mouths shut.”

There are still many questions about how this new law will work. How will organizations be held responsible if they force PM’s to use waterfall? Will PM’s who refuse to abandon the approach be eligible for public assistance if they can’t find work? Where does RUP (Rational Unified Process) fit in? Is it waterfall or not?

If you would like to report the illegal use of a waterfall methodology, please call the Waterfall SDLC Alert Hotline at 1-900-41FOOLS.

Hello David,

I am just checking with you to find out what your perspective is on “code reviews” in a Scrum environment? Generally it seems that I see that the discussion on this leans toward paired programming and test driven development to meet this need. But I would be very interested to get your opinion since you have been working in the Scrum environment.

Thank you!

Susan McCoy

Here’s the answer I sent her:

Good question. It’s my opinion that most code reviews are performed too late in a project to do much about serious problems. This seems to be true regardless of PM approach. If you have that problem, you need to fix it or just stop wasting time on code reviews.

I haven’t used a lot of pair programming, but I have done it, particularly with junior people. It seemed to work out well enough, and the two team members who trained each other this way became top-notch coders.

I haven’t really done formal code reviews much at all (except when required to by a process nazi) and don’t value them much. IMO, formal code reviews probably have the least value in agile environments, because the code is changing so quickly and is frequently re-written multiple times (a good thing, I think). I prefer to simply develop a team with one really good coder who also is a great mentor, and several junior people who need mentoring. When structured that way, code reviews tend to happen naturally and early. It’s hard to find a leader like that, but that’s where I would put my focus.

I hope this helps.

Is this the second email in a row where I’ve answered the question without actually answering the question. I hope not.

- Dave

I think it’s worth noting that for most terms there is a difference
between practice and theory in the way they are used. So, while I
call the part of my projects (I’m a PM BTW, for those who don’t know
me) where users come in and test UAT, the purpose of this testing is
to BOTH find bugs and determine whether they accept the product,
never mind the literal meaning of the words behind the abbreviation
UAT. I have found that exploratory testing is extremely effective for
both of these activities. Real users who have been trained in
exploratory testing are able to find bugs very quickly – usually in
the first two or three days of UAT. They also very quickly develop an
opinion of whether or not the product is shippable.

This works for my projects. Here are some reasons why:
- My pool of testers is very friendly towards me. They report to my
business partner, who fully supports my often-controversial approach
to project management.
- I have considerably lengthened UAT to give us time to fix defects
(I get away with this by shortening functional testing).
- UAT testers don’t come out of my project budget. This is an oddity
that may be specific to my situation. As a result, my projects cost
less if real users do the bulk of the testing.
- Finding a bug in UAT is not a bad thing for my projects. Everyone
is on the same page in this regard.

The truth is, my “UAT” is really functional and acceptance testing at
the same time. While this may not meet the typical testing model, it
works very effectively for me. In fact, I have typically taken more
time off of functional testing than I add to UAT, which means
schedule-wise I am better off. This works because I have a rather
large pool of very cooperative UAT testers to draw from, and
typically only have one or two functional testers available to me
(who I have to pay for).

I am convinced that this approach would not work for many projects.
Here are some situations where I think taking my approach would be a
horrible mistake:
- When contractual obligations regarding the software are being tested
- When your UAT testers don’t like you
- When your business partner freaks out if you find bugs in UAT
- When you don’t have time to fix bugs you find in UAT
- When you need to test something very technical about the product
that cannot be verified functionally

In theory, I suppose, what I do may be considered a really bad thing.
But in the context of my world, it has been tremendously effective.
My production launches have been fantastic – we rarely get bug
reports from the field since we started this approach, and my
projects tend to end more quickly.

I guess the point I’m trying to make is that sometimes the
distinctions between terms are not that important. Sometimes they
are. You need to be able to tell the difference and not make
assumptions about the types of activities going on at a particular
point in a project just because it has a particular label or because
it’s using a particular approach. You can argue that UAT is only for
determining whether a product is contractually finished or not, or
that exploratory testing is only for finding bugs, but if that’s not
the way a project is doing it and it’s working for them the academic
distinctions aren’t particularly important if they are limited to a
single implicit context.

I think it helps to look at our IT organizations from the point of view of the business units we support, and I’ve devised a mnemonic to capture a heuristic for measuring how the people we support look at us: COAST. COAST stands for the Cost of Acquiring a Supportive Team. COAST is not about the effort required to complete a project, but the effort expended just to take the first steps necessary to start one. It is the cost of turning a business problem into an actionable project, with resources (people & technology) assigned and prepared to solve the problem.

If there are multiple layers of support organizations involved, COAST extends all the way through to the last layer of support and must include ALL the costs required to get to the point where even the network guy in the data center is committed to the project, if his support is needed.

Let me give you a specific, albeit fictional, scenario for your consideration. Goldilocks is a web project for Hollywood Avenue, a major cosmetology company – the idea being to create a web site that analyses digital photographs of a customer’s hair and recommends hair color products, highlighting patterns, etc. HA’s IT department has three layers of support – application development, infrastructure services, and technology acquisition. The engagement model works like this:

Business => Application Development => Infrastructure Services => Technology Acquisition.

I call this the Dem Bones engagement model. You know the lyrics:
The foot bones’ connected to the ankle bone
The ankle bone’s connected to the shin bone
The shin bone’s connected to the knee bone
etc, until you get to head.

Here’s the point, and I’m going to use caps because I wish I could shout it: COAST doesn’t stop growing UNTIL YOU MAKE IT ALL THE WAY FROM THE FOOT BONE TO THE HEAD BONE!

At HA, this means COAST doesn’t stop accumulating until all resources, from Application Development to Technology Acquisition, have been committed to the delivery of the Goldilocks product. COAST could be a little, or it could be a lot, depending on how much work is required to sign up each of these organizations.

Until all these resources are committed to your project, you haven’t incurred the total Cost of Acquiring Technology & Services.

Why COAST Matters
COAST is a measurement of how you are perceived by the business you support. If COAST is high, you are seen as a hindrance. If it is low, you are a partner. It’s that simple.

Measuring COAST
I’m sure there are numerical ways to measure COAST, but I’m not convinced it matters. My honest opinion is that what matters is not the actual numerical values about COAST, but the way your business partner FEELS about COAST. Specifically, COAST is relevant in three areas:

Economic Cost
How much money does your business partner have to spend just to get an IT project off the ground? Once the project has started, do they feel that money was well spent, or do they consider it wasted money that represents the “cost of doing business?” Did the work leading up to having an executable project make the project less expensive, or more expensive?

Time Cost
How long does it take just to get IT engaged and moving? What is the absolute shortest start cycle for taking any project from concept to the point at which everyone is on board and measurable progress is occurring? How does your business feel about that time? Is it time well spent, making the project more likely to succeed, or does every day the project hasn’t started yet add more risk to the project?

Emotional Cost
Does the process of getting an IT project moving leave your business feeling drained and resentful? Or, are they excited and more committed than ever? How many escalations were involved in getting IT to sign up for the project and assign resource? How many different IT executives have to agree before COAST stops growing? How many meetings, emails, and documents does it take to build consensus?

The Four Rules of the COAST
Rule #1: COAST is cumulative.

The sum of the whole can NEVER be less than the sum of the parts. Take HA for example. The Application Development group can put together a dedicated project team in just a few days, but Infrastructure Services needs six weeks just to assign an analyst to evaluate your needs. When Technology Acquisition is involved, it gets even worse. COAST for TA is eight weeks or more. Even though the Application Development team can get off to a quick start, the total COAST is still high – fifteen weeks at a minimum.

Rule #2: Business Facing IT Groups Must Shield the Business From the Downstream COAST Or Bear the Consequences for COASTs they Can’t Control

It is possible for an IT project manager to absorb much of the COAST produced by the downstream systems. He or she should do their best to structure their projects in ways that take the downstream organizations with high COAST off the critical path and give them time to do their jobs without ecalations. This isn’t always possible though, and when it happens it has a real cost to the business. For example, Goldilocks requires a color analysis package from an outside vendor. This means that AD is going to have to engage IS, which will in turn engage TA (six weeks later), which won’t start working on the purchase of the package for another eight weeks. There may not be much the project can do in the meantime, in which case the project manager is transferring the COAST ($, time, emotion) up to her business partner. But maybe there are things that can be done, like prototyping the user interface, building a data model, or testing a mocked up system on a focus group. These things only shield the business from the COAST of IS and TA if they have tangible, substantial value to the business.

Rule #3: Support Organizations Must Understand and Minimize Their COAST if They Want to Not Suck

The further you are from direct interaction with the business the more responsive you must be as an organization to survive. Even though the opposite is typically true (thanks to Law #2), a support organization like Infrastructure Services or Technology Acquistion in our fictional Goldilocks project needs to build their model around the goal of being at least as responsive, if not more, as the organization they support. In the Goldilocks example, this means IS and TA must be able to commit to projects within a week or less, at minimal cost, and without an act of deity, if they want to be seen as a high value organization.

Rule #4: Analyzing Isn’t Starting

Gathering requirements isn’t starting. Neither is writing use cases, counting function points, or building work breakdown structures. COAST continues to accumulate while you perform these preparatory tasks. COAST continues to grow until you start to deliver something of value.

COAST Defines How the Business Sees IT
Read any business magazine and you’ll find at least one article mentioning the criticality of IT’s ability to respond quickly to the needs of the business. You can’t do that if your COAST is high. You have to lower it if you really want to be a partner in your business enterprise. You can’t lower COAST if you don’t know what it is from the point of view of your business partner. So ask them, and take their answer seriously. Most business think their IT departments suck. COAST is one of the reasons. Lower it.

In its traditional sense as a static master plan of your project I would agree, there is not much value in Gantts. But in its revised dynamic interpretation as a reflection of your current iteration state I still think these charts could still be a useful visualization tool even for agile teams.

You can read the rest of the post here.

You could say it’s a standard ‘paradigm shift’ and that it will just take time for everybody to slowly get on board. It would be nice to think that but, I’m here to tell you that, unfortunately, it won’t be that easy. Why? Because the waterfall methodology still solves problems that Agile doesn’t solve.

“What?” you say. “Impossible!” you say (ahh yes, I can almost hear the rotten tomatoes hurtling toward me now).

Let’s brush up on a little Capitalism 101. People with money frequently want to trade some of their money for something else of value to them. When the something else is raw material or oil or some commodity item, it is relatively easy to work out the deal. “I need 200 more gallons of Foo to add to my inventory so I don’t run out. Who has the best price on Foo this month?” When you get into more specialized items like CPU chips, price is still very important, but there is now another consideration: timing. “I’ll pay $90 per chip, but you have to get 4000 of them in my warehouse by the end of next month. Who can promise me 4000 chips right now?” And if you get into even more specialized things, like acquiring a company or getting plastic surgery or a trial lawyer, there is something even more important still: risk reduction. You’re only getting one and it will be really expensive. All kinds of things can go wrong if you choose poorly. You need the best supplier of whatever it is that you can get.

A real business person who has their own budget authority and who wants to buy (outsource) a software development project fits this mold. Yes, they’d like the developer to be inexpensive, but what they want more than that is to get the value they expect out of the application. There’s a real business need for having this application (which has maybe 40 features) and the sooner they can start realizing that value, the better. And what they desperately need is for the developer to reduce the project risk as much as possible. Perfect for Agile, right?

Yes, except that the business person is not too savvy at evaluating technical talent or organizational capabilities, so the business person is left to try to measure a developer’s ability to reduce risk by evaluating whether the developer is oozing with experience and confidence and talent and provides a beautiful proposal with mature-sounding procedures and cool business cards. You don’t have to use an Agile method to do that.

If you’re proposing an Agile project, what risk are you taking away from the buyer (from her perspective)? What exactly are you promising to do? Are you promising to work efficiently and constantly keep her involved in your project to collaborate and see progress? What if your competition promises to finish all 40 features of the desired system for a fixed price, by a specific date as long as the business person doesn’t change her mind about what she wants. Which sounds more attractive? Which proposal sounds less risky to the un-initiated?

It gets worse. The reality is that business person with her own idea of what she needs and her own budget authority accounts for a small amount of the total dollars spent on projects. In my experience, she’s less than 50% and I’ve worked in companies that specifically target that kind of customer. It’s probably much less than that overall.

What is much more common is to have a business person who has an idea of what they want, but they have to get money approved through a corporate purchasing function. This means that the real root of all this methodology evil will be involved in your project – procurement people.

Procurement people constantly screw up the process of buying a software project because they treat it the same way as buying 200 gallons of Foo. This is especially true in government and manufacturing organizations. They do this in the great hope that it will benefit their organization enough that they can keep their job another year. I’ve heard that some of these people get a commission on how many discount dollars they can wring out of suppliers’ prices. So, price is always supposed to be as cheap as possible and fixed, the date is always supposed to be guaranteed, the scope is…well luckily they don’t know what scope is, so they skip that part, but yes you do need to have customer references and insurance and blah blah blah to prove you’re an excellent supplier of Foo and if you just sign their standard terms and conditions stating that anything of value you might ever achieve in life can be claimed by them to be their own intellectual property, then your proposal will be reviewed alongside the six other submissions for the job.

The business person is still there, but the only thing they have to cling to is their vision of what features the application should have. In fact they probably had to write a spec so the procurement people could put it out for bid. So, if you are going to propose an Agile project on a spec that was written by somebody else, how would you do it? What, exactly would you be promising? You will be required contractually to have a fixed price, you might also have a constraint on the finish date and you have a fixed specification to bid against. How will your waterfall-loving competition bid? Again, ask yourself which proposal is going to sound more convincing to the uninitiated?

When people who run consulting businesses think very hard on this problem, most will come back and say, “Well, if I have to fix the price and the date and deliver that pile of scope in your spec, then I’ll do it for $[x] as long as you don’t change anything in the spec. If you change something then I’ll have to charge you more, but it will be on a case by case basis.” They’ll say this because they have to make payroll and they are trying to keep their staff happy. They’ll say it because they know the customer will change their mind because the spec is crap, so they’ll get to up the price anyway and it will be after they have gotten past the procurement people and are just dealing with the business people.

People define projects and bid them out because they want somebody else to be responsible. Procurement departments try to increase competition, squeeze prices down, and outsource as much risk as possible.

Is it any wonder that services firms developed ‘change management’ procedures in the 80’s and 90’s to deal with government and business procurement problems? Is it any wonder that the idea of dumbing down the roles and responsibilities and using ‘better’ processes and cheaper resources gained ground as boilerplate proposal content (if not real practice)?

The waterfall methodology was not created and does not survive today because it is a good approach to develop software. It was developed because it was a good approach to navigating a procurement minefield and winning software project contracts. Until Agile can find a way to win the procurement battle (or somehow change that game), waterfall methods will stick around.

- Tate Stuntz