Please take a minute and read the open source letter. If you agree about the importance of considering the source of software products in government projects, you can sign the letter by posting a comment with your name, company (if appropriate) and web-site. We’d also appreciate any mention you might make of the open source letter to your friends and colleagues through email, blogs, and twitter.

The letter itself is open-sourced under a creative commons license, which means you can post it, print it, share it, etc with those around you. We hope you will do just that, and help promote the idea that open-source can help bring cost-effective, transparent solutions to government software projects.

Thanks!

Dave

Hacks create technical debt by ignoring principles of good software development in favor of quick, easy fixes

This is the obvious one because it is the one technical teams complain most about. “I didn’t want to do it that way, but my manager said I could only choose from options that took less than a day to implement,” or some variant of this statement, is the most common answer to the question “Why does X suck so bad?”

So how do developers typically fight back against hacks? With reuse anticipation. Now the bad news.

Reuse anticipation creates technical debt by expanding the supported code base without creating a correspondingly timely value proposition

It’s easy to imagine that the service, component, module, whatever, that you’re building might be used by multiple applications in multiple ways. It only takes an extra YY hours to make it universal, you tell the project manager. And then we’ll never have to write this code again. Sometimes, when you already have “universal” demand for a component, this is a good idea. It’s also a good idea when it is a very specific, low-level technical capability, like dumping a message in a queue or searching a string for a value or something like that. Sometimes it’s a really bad idea – over-generalizing a capability can make it so difficult to “reuse” that no one will reuse it. Either way, it increases your technical debt, because whether anyone reuses it or not, the code you have to support is more complex than it would otherwise be.

Time creates technical debt
There’s a great riddle in The Hobbit, by JRR Tolkien:

This thing all things devours:
Birds, beasts, trees, flowers;
Gnaws iron, bites steel;
Grinds hard stones to meal;
Slays king, ruins town,
And beats high mountains down.

Tolkien might have included code in this list had he been a software developer.

Time has a way of dimming our memories of the code we’ve written, what it does, and why it does it. It is for this reason that development teams often turn to comprehensive documentation to save them.

Now for more bad news.

Comprehensive documentation creates technical debt
Every document you write that has more purpose than helping you figure out how to deliver code NOW becomes another artifact you have to maintain. It adds to your code base in that regard. Now, instead of paying people to maintain the code, the business has to pay someone to maintain the code and the documentation. You just fell further behind.

Documentation typically only has ROI in the short term. It is a very rare document that provides value over the long term. Usually it very quickly becomes a cost with little return of any kind.

Innovation creates technical debt
Einstein said we can’t solve our problems with the same thinking we used when we created them. Sadly, when we continuously apply the same thinking to our code base that we used when we created it, the technology world doesn’t do the same. It moves on, finding better ways to do the same thing. Small changes add up and become big changes, and suddenly there are much more attractive technologies that we can’t use because we’ve fallen so far behind.

When we see the world moving on to bigger and better ways of solving our problems, it’s tempting to say something like this:
“There’s no point putting effort into X, we’re just going to move to Y soon anyway.”

Sometimes this is a good idea, if moving to Y is funded, approved, and making progress. But if Y isn’t approved, giving up on X will only increase the amount you owe, and you’ll feel doubly bitter about it when Y never materializes.

When I see this problem, I think of a line from an old song: “If you can’t be with the one you love, well then, love the one you’re with.”

Technical debt creates more technical debt
With credit cards it is sometimes possible to make the minimum payment without reducing your principal a penny. In fact, you can pay your bill and end up with a higher balance because of this.

It’s the same way with your code base. If you don’t invest in it faster than you are accruing technical debt, your technical debt will grow. Why? Because you don’t have time to make it better, so what are you doing? You’re hacking – cutting corners and doing the wrong things just to get done in time with the resources you have.

Technical debt sucks as bad as credit card debt. I suspect that paying off technical debt is very similar to paying off credit card debt. It’s hard, it requires frugality, and it means shedding everything that isn’t really important. More about that some other time.

Have a nice week.

One of the funnest parts of doing this webinar was producing the artwork for the slides. For reasons I explain in the presentation, I wanted three photographs of spaghetti: the first of uncooked spaghetti on a countertop, the second of cooked spaghetti in a bowl, and the third of cooked spaghetti dumped over an angry child’s head. I decided to take the photos myself rather than buy them, and I proceed to try to convince my four-year old son that he would enjoy having spaghetti dumped over his head.

He didn’t buy it. Fortunately, my eight year old daughter did, for a price. One dollar for every year she’d been alive. I paid up, and the resulting photo was excellent, even if Reese wished she had charged me more.

Before long, an Enterprise Integration group was born. It provided two hundred different services, including billing, provisioning, account servicing, and others. It had four hundred employees, all of whom were happily engaged in maintaining and extending the service offering of Enterprise Integration. It was a happy world. The major telecom could roll out new apps quickly and cheaply, because most of their back office capabilities were already exposed. But the greatest asset of enterprise integration wasn’t just the services they owned, but the canonical model it created.

Good times never last forever, and by 2002 this telco discovered it was in trouble. It couldn’t spend money like water anymore, and it started to look for ways to cut back. The billing system was an obvious place to start. It was an ASP, and at $1 per account per month, it was costing this very large telco a heckuva lotta their sweet moolah. Everyone knew it needed to be replaced by a system the telco could own and support in house. The business case was solid. Replace the billing system with one you could own and support in house, and even if it cost $1,000,000 every month you would still net a gain of about $80,000,000 every month. Heck, you could sink $100,000,000, maybe $200,000,000 in the implementation project and still come out way ahead on the ROI.

Fortunately for this telco, they had a solid integration architecture, decoupling the billing system from all the systems it supported. They could just rip it out and put in a new one, support the old interfaces, and move on happily with their lives. Everything would be grand. That’s the way the technologists sold it anyway. “It’s all the same data,” they said. “It won’t matter if it’s structured differently, we’ll just find ways to transform it to match what the new billing system is expecting.”

The sales pitch was bought, and the integration began. The new billing system was installed and loaded with data. Business users started experimenting with the user interface, designing workflows and figuring out how to do their jobs with the new stuff. Everything seemed great, until they started trying to plug the new billing system into the old billing services, of which there were about eighty.

Then, things started to go downhill. It wasn’t the architecture. The architecture was great. It didn’t matter that the new system was java whereas the old system had been exposed through C++ API’s. All EI had to do was create adaptors, transform the canonical model into the new systems API, and off we go. Piece of cake, right?

Wrong. The canonical model wouldn’t map. The new billing system was fundamentally different from the canonical model, and for good reason. It solved problems that hadn’t existed when the canonical model was created. Those problems couldn’t have been anticipated when bright guys defined it. The ground had moved underneath them, and they couldn’t move fast enough to keep up. They couldn’t change the canonical model to meet the new demands of the business. They had too many clients and too many services to move everybody over to a new canonical model. They were being buried under the weight of their own canonical model. They thought they owned the model, but in the end it was clear – the canonical model owned them.

$100,000,000 later, Enterprise Integration finally owned up to the truth. They couldn’t make it happen. The vendor of the new billing system was sent packing. The telco’s stock dropped even further, and eventually the disappointed telco, stuck with their $1 per subscriber per month billing system FOREVER, sent almost their entire IT department to big blue, outsourcing nearly every IT job in the company.

Dave Christiansen is the founder and managing editor of
TechDarkSide.com. He manages projects for a Fortune 100 financial services company and writes and talks about project management. He can be reached at dave@techdarkside.com.