Hacks create technical debt by ignoring principles of good software development in favor of quick, easy fixes

This is the obvious one because it is the one technical teams complain most about. “I didn’t want to do it that way, but my manager said I could only choose from options that took less than a day to implement,” or some variant of this statement, is the most common answer to the question “Why does X suck so bad?”

So how do developers typically fight back against hacks? With reuse anticipation. Now the bad news.

Reuse anticipation creates technical debt by expanding the supported code base without creating a correspondingly timely value proposition

It’s easy to imagine that the service, component, module, whatever, that you’re building might be used by multiple applications in multiple ways. It only takes an extra YY hours to make it universal, you tell the project manager. And then we’ll never have to write this code again. Sometimes, when you already have “universal” demand for a component, this is a good idea. It’s also a good idea when it is a very specific, low-level technical capability, like dumping a message in a queue or searching a string for a value or something like that. Sometimes it’s a really bad idea – over-generalizing a capability can make it so difficult to “reuse” that no one will reuse it. Either way, it increases your technical debt, because whether anyone reuses it or not, the code you have to support is more complex than it would otherwise be.

Time creates technical debt
There’s a great riddle in The Hobbit, by JRR Tolkien:

This thing all things devours:
Birds, beasts, trees, flowers;
Gnaws iron, bites steel;
Grinds hard stones to meal;
Slays king, ruins town,
And beats high mountains down.

Tolkien might have included code in this list had he been a software developer.

Time has a way of dimming our memories of the code we’ve written, what it does, and why it does it. It is for this reason that development teams often turn to comprehensive documentation to save them.

Now for more bad news.

Comprehensive documentation creates technical debt
Every document you write that has more purpose than helping you figure out how to deliver code NOW becomes another artifact you have to maintain. It adds to your code base in that regard. Now, instead of paying people to maintain the code, the business has to pay someone to maintain the code and the documentation. You just fell further behind.

Documentation typically only has ROI in the short term. It is a very rare document that provides value over the long term. Usually it very quickly becomes a cost with little return of any kind.

Innovation creates technical debt
Einstein said we can’t solve our problems with the same thinking we used when we created them. Sadly, when we continuously apply the same thinking to our code base that we used when we created it, the technology world doesn’t do the same. It moves on, finding better ways to do the same thing. Small changes add up and become big changes, and suddenly there are much more attractive technologies that we can’t use because we’ve fallen so far behind.

When we see the world moving on to bigger and better ways of solving our problems, it’s tempting to say something like this:
“There’s no point putting effort into X, we’re just going to move to Y soon anyway.”

Sometimes this is a good idea, if moving to Y is funded, approved, and making progress. But if Y isn’t approved, giving up on X will only increase the amount you owe, and you’ll feel doubly bitter about it when Y never materializes.

When I see this problem, I think of a line from an old song: “If you can’t be with the one you love, well then, love the one you’re with.”

Technical debt creates more technical debt
With credit cards it is sometimes possible to make the minimum payment without reducing your principal a penny. In fact, you can pay your bill and end up with a higher balance because of this.

It’s the same way with your code base. If you don’t invest in it faster than you are accruing technical debt, your technical debt will grow. Why? Because you don’t have time to make it better, so what are you doing? You’re hacking – cutting corners and doing the wrong things just to get done in time with the resources you have.

Technical debt sucks as bad as credit card debt. I suspect that paying off technical debt is very similar to paying off credit card debt. It’s hard, it requires frugality, and it means shedding everything that isn’t really important. More about that some other time.

Have a nice week.

Corporate IT is, by its very nature, a barrier to innovation. This is true at many levels, starting with your management chain. How many layers of buy-in are required to try something new? What if you actually have to buy something to try it? Hierarchical organizations make innovation harder because they impose a decision-making layer of bureaucracy upon the innovators.

Most large organizations are hierarchical, with varying degrees of bureaucracy, but Corporate IT takes it several steps further than just the natural problems of a large organization. We’ve created special processes designed to resist innovation and change. I’m talking about the project management “best practices” (yes, I mean this sarcastically) of change management, upfront requirements, and “contracts” between business and IT.

How many times have you heard an IT manager try to tell the business something like the following: “Don’t tell us how to do it. Just tell us what you want done and we’ll figure out how to do it”? This is a typical response to our business partners doing their best to explain a problem and a potential solution.

Sure, sometimes the business tells us to solve problems in crazy ways that might ultimately be harmful to the organization. I’ve seen that first hand on multiple occasions. But I think there is a better way to avoid the negative outcome of crappy software than telling the business to stay out of IT.

One of the things that often happens when IT managers send this message to business partners is a counter-productive power struggle. Managers on both sides start this escalation process through their reporting chain until they either merge or one of them has to back down because of the eminence to the other. Guess who almost always has to back down? IT.

I think this problem would go away if IT managers would stop seeing business “design” as a bad thing. I prefer to think of it as an attempt of our business partners to move away from the legalistic relationship that plagues IT and collaborate with us. It doesn’t necessarily mean we design it the way they suggest, but it does mean we let them play. Anyone who can contribute should be allowed to. Designs express intent, and often that intent is more important than the requirements the business might write down and throw over the wall. By letting them play in design activities with us, we understand their intent better and are more likely to create a solution that solves the problem we are focused on. In the process, we help the business to have confidence in our skills as technology problem solvers.

When the business is confident in you, innovation becomes easier. They will take risks on you – spend money on the possibility of a big win, and together you will fail or succeed. When the business believes in you, your management will tend to get out of the way, whether they believe in you or not. Everyone in IT knows where the bread is buttered, and it’s not on the IT reporting chain.

Collaboration is the key to innovation. The best collaboration happens when you set aside the restrictions organizations create around roles and let people play. Here are some rules I’ve come up with over the years that I think encourage collaboration and innovation:

Check it out and let me know what you think.

It occurred to me later that the source of the disagreement was very fundamental, and perhaps it was so deeply ingrained in the testers’ various psychologies that they were unaware of the differences between them. I believe the two sides of the debate represented two schools of thought about the purpose of software testing.

School #1: Find bugs.

The best bug-hunters can test anything at a moment’s notice. They don’t need requirements, charters, or test managers to do their job. Just give them an application and they will find bugs.

School #2: Prove the software performs as advertised
This camp wants to demonstrate that the software meets the requirements as defined in the beginning of the project. They are into requirements, traceability, and detailed upfront planning. The extremists among them believe you can test quality into a project, and that the only input for determining whether software works is formal requirements documentation. They are deeply concerned with planning the right tests, based on the requirements, that can demonstrate that the software meets the requirements. They love automated testing, test scripts, and planning. To them, test skills revolve around planning, analyzing requirements, using tools, and executing scripts.

This role is sometimes referred to as quality assurance. I think it’s a stupid, ambiguous term and that nobody knows exactly what is meant by it.

One of the outcomes of this school is specialization. The “good” testers don’t test – they right scripts based on requirements. They do this during the requirements phase of the project. Then, when code starts to ship to the test environment, they become test managers, watching over a horde of script-monkeys who execute scripts and report defects. The test manager gets the job of determining whether the defects are really defects or simply missed requirements, which only get fixed if the business agrees to pony up more moolah.

Three good bug-hunters are worth thirty good script-monkeys and their boss
As a project manager, I don’t really have much use for school #2. There simply isn’t much value in “proving”, on a line-item basis, that the software my teams build meets the requirements. The only thing that really matters to me is whether the business accepts the software as meeting their needs, requirements be damned. My main concern in shipping software is that it 1) works and 2) has value. Oddly enough, the quality assurance school of software testing hasn’t helped me in either respect.

I find bug-hunters are much better at making sure the software works and that it meets the business’s needs. They tend to be good collaborators, and have very curious natures that drive them to interact with business partners and figure out their perspectives on the application being tested. They are capable of using requirements to figure out what the software is supposed to do, but they won’t rely on it exclusively. They are flexible and comfortable with ambiguity. This makes them very valuable team members.

School #2 is crippled by its rigidity
The quality assurance approach only works well under ideal circumstances. My projects are never ideal. They are chaotic, crazy, short-term efforts that move rapidly and don’t rely on precise, complete documentation. They are a lot of fun and emphasize collaboration heavily. QA testers don’t fit in on my teams. They are too uptight, too focused on documentation, and too inflexible. I can’t use them.

Both schools tend to have a near-religious fervor about their beliefs
For the testers who are very deeply rooted in their schools, discussion doesn’t really help much. You can’t talk school #2 into school #1, and vice-versa. The debate between these schools reminded me of biblical scholars with different views on the literal nature of scripture arguing whether Jonah really was swallowed by a whale, or of the fervor with which political leaders once fought the idea that the earth was round. Only when Columbus returned from the new world with coffee, chocolate, and gold, did they accept the truth, in spite of the fact that it was easily observable in the natural world.

Every movement is trying to solve at least one specific problem. This movement is trying to solve two, as far as I can tell:

I don’t believe this movement makes IT a better place.

You can’t beat the cost-change curve by making change MORE expensive
Change control procedures do not reduce the cost of change. They add overhead to change, which makes it more expensive. Legalistic requirements documents add more cost – when “the business” has to commit to a set of documents being what they want, all they want, nothing more, and nothing less, it takes a significant effort to make sure those documents are correct. Especially if it means they have to wait twelve to eighteen months before they can realize any return on their investment, as is typical of serial development.

The cost change curve is one of the first things they teach project managers. “A change of requirements at the end of the project is a bazillion times more expensive than a change of requirements at the beginning.” It makes sense, and it sticks. PMs don’t forget that mantra.

Unfortunately, the cost change curve is also a function of the methodology you follow. The bazillion dollar cost of late change is a feature of serial development (waterfall), a self-fulfilling prophecy that is forced to be true by the project structure itself. Of course late requirements changes are orders-of-magnitude more expensive because you’ve already made all sorts of decisions based on them that would have to be unmade. You’ve built a huge pile of documentation with little or no business value that has to be re-vamped in order to make the change “right”.

You can beat the cost-change curve by delivering incrementally
Let’s say that instead of delivering in phases you decide to deliver in increments. In other words, instead of delivering requirements after three months, then design after another three, and so forth, you instead deliver little chunks of working valuable software every one, two, or three months. You start each increment by working with the business to define that piece of functionality, then you build and deliver it as quickly as possible.

Change doesn’t cost as much when you do it this way. If your business partner decides that feature Y is suddenly critical while your building feature X, you can usually placate them by reminding them that feature X will be complete in a few weeks and then you will start on feature Y.

What if they decide to change feature X, which you’ve already built? Well, you just plan another increment to change it. It’s not unusual for people to change their minds about something they’ve previously only imagined once they see it built?

What if they decide to change feature X while you are building it? Just do it. If you are building in short increments, the impact of a change will be small. Maybe it adds a week, perhaps two. Tell the business what the change means and see how they respond. Big deal.

Building software in small incremental chunks flattens the cost-change curve.

You can reduce the likelihood of change by building in small incremental chunks
The longer the amount of time lapses between the definition of a feature and its delivery to the business, the more likely it is to change. Change could be prompted by many factors such as government changes, market forces, and personnel changes. Heck, it could even be influenced by changes in the prescription medications your overwhelmed business partner consumes.

Change happens – and the longer the window you give it to happen in the more likely it is to occur. This is the great irony of the CDSD movement – it asks for more time for requirements and what-not, ignoring the dramatically increased likelihood of change that it’s approach to resisting change creates.

Laying blame isn’t any easier just because you have contracts
Just ask a lawyer. Contracts don’t necessarily make it any easier to stop your business partner from throwing you under the bus when bad stuff happens. In fact, these contracts are frequently more ammo against you.

“IT said they would deliver this, but they did this instead.”

More importantly, if a technology solution is bad for the business, it doesn’t really matter if the contract specified that solution or not. Companies don’t get a get-out-of-jail-free card when it comes to market forces and competition. If you build the wrong thing and miss a market opportunity as a result, contracts are of little concern. The whole business is going to suffer, and you might have a smug sense of satisfaction because your petty little contract “proves” it’s the business’s fault and not IT’s, but it doesn’t make your project a success.

Perhaps if you hadn’t shoved CDSD down the business throat you might have been able to collaboratively deliver a better solution. Maybe it is your fault after all. That’s what it always boils down to. When these projects fail, IT says it’s the business’s fault. “See, it’s right here – you signed it.” The business sighs and says, “Yeah, we signed it… but if IT was more flexible, more able to respond to change, we would have solved this problem differently.” And then they outsource you.

Reduce the impact of mistakes by delivering incremental chunks of working valuable software
Screwing up doesn’t have to hurt so bad. Mistakes happen, and blame gets apportioned, often unfairly. That blame, however, is typically proportional to the cost of the mistake. Because mistakes are so outrageously costly in CDSD, blame is often a pound of flesh or more. It doesn’t have to be that way.

Incremental delivery reduces the cost of big mistakes by reducing the amount of time it takes to discover them. Since you deliver working software every month or two, the business gets to see the project as it is built. This means they will identify mistakes early and you can correct for them. The worst cost of a mistake is typically one implementation cycle, and if you keep that down to a month or two, it’s not that big of a deal.

Redefine project failure with incremental delivery
I’ve seen CDSD projects shutdown after eighteen months or more, having never delivered a line of code. That hurts. That’s money down the drain. What a demoralizing waste of resources. How sad. There is no hope of picking up such a project where it was left off – requirements documents, design documents, the labor of love of CDSD, have a very limited shelf life. They go bad faster than new bananas.

You can cancel any one of my projects six months in and I won’t be upset. Why? Because we have already delivered valuable working software. Generally speaking, we’ve already delivered the most important features. Move those resources around, assign us to something else, it doesn’t matter. It’s not money down the drain. We delivered something the business wanted, and then we shut it down. Big deal. We shipped software, software that worked and that had business value. We were successful.

Don’t buy into the Contract Driven Serial Development movement. It’s not good for you, for IT as a career, or for the business you signed up to serve. Kick it to the curb as a stupid idea, and then go write some code.

I don’t really find this stuff useful on my projects. I’m not sure there’s any scientific research supporting the effectiveness of this approach, but I’d be surprised if there was any. Of all the “professions” project management seems to be one of the most easily infiltrated by anecdote and myth – I think the discipline of software project management is still so young that not many practitioners have begun questioning it’s foundations. The earth is still flat for IT project management, sadly. There are those who know the earth is round, like Johanna Rothman and the Poppendieck’s, but they aren’t enough to stop the massive institutional brainwashing of young project managers into phased development (aka waterfall).

But I digress. Here’s an alternative way to evaluate project risk that I think is much more useful.

Project risk grows exponentially with the smallest unit of time in which you can regularly deliver valuable working software.

If I didn’t lose you with my completely random outburst in the second paragraph, I probably lost you with the exponential stuff. Here’s what I mean:

If you can deliver valuable working software every two months (start-to-finish) then your overall project risk is pretty low. Why? Because every two months you get to re-adjust, re-plan, and re-prioritize with the people who care about your project, and any mistakes can be fixed in two months or less.

Projects requiring six months to deliver valuable working software should automatically be considered medium to high risk of outright failure.
Sometimes, the delivery of valuable working software takes longer, say six months or so of work. This sucks, but it happens. This is a lot higher risk, a lot more than three times as risky. I’d suggest it’s about ten times as risky. You are ten times as likely to fail at a six month effort than a two month effort. Risk of bad stuff grows exponentially the longer you are exposed to them. Here’s an example, albeit a bit morbid: What are the odds one of your team members will collapse and die tomorrow? Pretty slim, I hope. But, what are the odds that one of your team members will die, quit, get re-assigned, get sick, or go on maternity leave in the next six months? Much higher.

Long odds for long deliveries
Projects that invest twelve months or more before there is a return have extremely high failure rates. I believe they are at least 100 times more likely to fail than projects that can deliver valuable working software in two-month units. These projects should be considered extremely high risk. I have always managed to stay away from projects that have twelve months between start and delivery, and I thank my lucky stars for that.

Long projects don’t have to have long delivery cycles
I just finished an eighteen month project that had a multi-million dollar budget, but not once did I start an iteration that lasted more than two or three months. We never attempted to build anything we couldn’t do in two or three months, and we always had a pretty small team. Sometimes we had multiple small things working on parallel threads, but they always delivered valuable software that worked in a short time frame. It was a weird feeling last month when we shipped the last release and realized we were done. It was like we had moved a mountain, one pebble at a time, and we didn’t really see the mountain had moved until the last pebble was gone. It didn’t feel like we’d just moved a mountain – more like we’d just had a reasonably nice stretch of work, busy but not buried, challenged but not hopeless. It felt good.

Control the delivery cycle and you have controlled the risk
Creative people with decent technical backgrounds and an interest in the needs of their business partners can find ways to divide projects into smaller feature sets that can be delivered in two months. And, if you can manage to structure your project in two month chunks, you have just dramatically reduced the overall risk of your project and created a project structure that is more easily adapted to the potentially changing needs of your biz partners. It also reduces the tendency of team members to create unnecessary work or to argue to long about anything – urgency is something you can feel on two month projects.

You might think I’m full of crap. Go ahead and tell me so, I don’t care. But try it. Now. Don’t wait for your twelve month project to go eleven months without shipping jack to give something like this a shot. What can it hurt you to try it? It’s not like you’re going to look like an idiot for delivering something quickly, and what if it really can reduce the risk of your project by a factor of 100? It’s not like you were planning on shipping anything in month three anyway…

-Dave

I believe Melinda is really asking about a topic I call project design. It is part analysis, part planning, and part problem-solving. I am going to outline a few principles today that I think are important for project design, and in a later post will go into more detail on each of them.

A well designed project doesn’t need Microsoft Project.
I haven’t used Project in over two years, in spite of the fact that I have been managing a multi-million dollar program that spans five different systems at my employer. We just put our last release into production last weekend, and not once did I open MS Project to create or update a project plan. A lot of people write this off by saying that my project is “simple”. It’s not. There are lots of integrations, changes in multiple systems that depend on each other, and several different organizations involved. My plan, on the other hand, is simple, because I’ve designed my project in a way that makes planning simpler.

That said, I do have a Gantt chart, but I make it in Excel, and it really is nothing more than a release plan. It shows which features will ship when, and I use it every couple of months to talk about what we will build next with my business partner.

Focus on features, not phases
Separating projects into distinct phases like initiation, planning, design, construction, testing, implementation, etc makes the planning more difficult, and it increases the risk of the project. Rather than starting with a list of phases, as most PMBOK-trained PM’s do, start with the features list.

Group the features that are similar. I talked about this a few days ago in a post called competence pie. Then, prioritize the groups of features with your business partners. Which are easiest to build? Which features can’t be built until some other feature is built? Which one should be built first?

Build the “best” features first
Try to make a smart choice about the feature you will build first. You’re going to learn a lot in the first iteration, which means you’re going to have mistakes that need to be fixed. Pick a feature that has business value, but also will be a learning experience. Don’t pick the easiest one first, but don’t necessarily go for the toughest one either.

My reasoning for this is simple. Project teams need to gel, and they need to explore the problem space. I’m always amazed at how the productivity of a team increases after the first release and at how much better we understand the remaining work.

Further group the features by what can be delivered in about two months
Long releases suck the life and soul out of project teams and business partners. Short releases build energy for both, because you can see the product evolving rapidly.

Organize your features into sets that your project team can build out in two months or less. If some of the feature sets you identified earlier are too large, split them up. Those that are too small can be combined. Review this with your business partner to make sure you haven’t missed something – sometimes one feature can’t be shipped without the other.

Make a Gantt chart that is based on feature delivery
Take your prioritized list of features and build a Gantt chart. I use Excel for this. Each row is a release, and I list the feature groups in the first cell. Then I make a column for each month and start planning which feature groups go when. I use a color coding system to differentiate between firm dates and tentative dates. I usually don’t have firm dates outside of two or three months ahead.

Collaborate with your business partner
This sort of planning is not what our business partners are used to, although I’ve found they much prefer it to phase-based plans that don’t let them see the product until the very end. If they are used to phase-based planning, you’ve got to get them over the hump, so to speak, to embrace this approach. Develop this plan with them, involving them at every step, and they will support you.

Plan every day
Melinda, I can hear you objecting to this approach right now. “But that’s not a complete project plan,” you are probably saying, and you’re right. I haven’t planned every single task that needs to be done along the way. I don’t do that planning until we start an iteration. Then, the project team and I sit down and quickly list out everything we need to do. We don’t try to plan iterations before we’re ready to start them. I’ve found this to be very beneficial, because it allows me to take into account changes in the team structure, business environment, business needs, etc. along the way. I don’t have to worry about whether my plans will be changed by outside factors because I haven’t set them in stone yet.

Build up to parallel efforts
After a couple of releases, you’ll realize you could increase the pace by having two tracks going at once, offset by a month. This will double the pace and result in monthly deliveries instead of bi-monthly deliveries. If you thought the business was happy before, you should see how they respond to this acceleration.

Please don’t split team members between releases (i.e. no multi-project multi-tasking). Productivity will plummet if you do. To run in parallel, you will need another team.

Measure the past to predict the future
Your first couple of iterations are very important because they will have the most impact on your overall plan/strategy. They will either validate the time-boxed chunks of work as being achievable or force you to re-design the project once you have gotten a sense of the pace your team can achieve. Make sure you involve your business partner in any refactoring you do, because you will need her support.

Build hardware environments in parallel with software development
Sometimes a software project also comes with a hardware project. This might mean you can’t deliver anything to production until the hardware is in place. If that’s the case, run it in parallel, and try to keep your software development team from getting bogged down in the hardware project, especially the procurement process. Build your hardware environment in a similar fashion – don’t do it in phases, but do it by environment. Build test first, then stage (if you need one), then prod. Leverage the things you learn along the way, and don’t build the next environment until you’ve delivered some code to the one you just finished.

Be careful though – sometimes the logistics in hardware procurement is very painful and you might need a lot more time to get hardware than to build software. I recently completed a software project faster than our procurement office could acquire technology, and as a result we had to wait for the technology to ship for over a month. If this happens, use the situation and your business partner to put pressure on your procurement chain to hurry. It generally works – nobody likes to be reason a production release is delayed.

Melinda, this probably isn’t the answer you were looking for. I know I have a fundamentally different approach to planning than many PM’s, but my projects are considerably more flexible than other projects I’ve observed, and I don’t spend a lot of time updating a project plan that is way too detailed. At any rate, it works for me. Please let me know what you think.

Expression is the act of defining (or re-defining) the project vocabulary.
This is the most advanced of the five areas of competence that I described (engagement, fluency, comprehension, implication and nuance, and expression). I struggled to define it well in my original post. I think expression is the ability to effectively define the vocabulary of the project, to give names and categories to things that were chaotic and unorganized. It is a key part of developing a project strategy that allows incremental delivery, to create affinity groups of features and give them names. Here’s an example:

A business person gives you a list of features they want a web app to have, in no particular order and with no particular grouping of the features. It might be something like this:

I’ve seen lists like this that go on for pages – it can be daunting and seem chaotic. Taking this unorganized list of features and making it manageable is an act of expression, because you group those features in affinity sets, which is a fancy way of saying that you take similar features and group them together. Here’s one way you might start grouping the features above:
User Support
Contact customer service by email
Have a live chat with a CSR

User Enrollment
Verify new user’s identities using existing account information

Account Management
Change account informat (address, phone, etc)
Make a payment
View a statement

I didn’t do a complete decomp of the first list – this is just an example to help explain what I mean by expression. One of the things that I’ve seen happen when I am able to use expression well is that it inspires a re-thinking of the requirements so far. For instance, I would expect a business person to look at User Enrollment and say something like “you know, there’s a lot more to user enrollment than just verifying identity.” Then, we would add stuff to that list.

We also prioritize the groups, and start planning iterations to implement the high priority items. This is also an act of expression, because you are starting to turn the list of features into something more – an implementation strategy.

Competence Pie
I like to model concepts visually, and I’ve come up with a model for perceiving and building competence – I call it competence pie.

Others Need to See Your Competence if it’s Going to Benefit You
Being the manager of a project is not just about being competent. You have to be competent – faking it is a short term strategy that will eventually fail if you don’t use it to buy time to build real competence. That said, being perceived as competent is also critical. You need to demonstrate your competence to your business partners, your project team, and your IT management if you want your project to be successful. They need to have confidence in you. Not building that confidence will open the door to a host of troubles, including micro-management, dissent on the technical team, and poor collaboration with the business.

Build real competence, and show it.

Even if you manage to develop comprehension without fluency, it won’t benefit you much. After a brief grace period, people will begin to treat you as incompetent even if you are not, simply because you don’t grasp the terminology of the situation. As an example, I can overlook a new hire saying something really stupid like “Java Real-time Engine” instead of “Java Run-time Engine”, but I wouldn’t overlook that in a job interview with an allegedly “experienced” developer, manager, tester, or analyst.

Fluency can be broken up into several different areas, and it’s important that you come up to speed on the right ones rapidly, depending on your role. Here are some common areas I focus on, in no particular order:

This isn’t an exhaustive list but it’s a good place to start. You should be able to summarize each of these, even if you don’t understand them perfectly, within a week or two of engaging in a new project. If you can’t you risk others thinking of you as incompetent or disengaged.

It’s not that difficult to come up to speed on these things quickly, if you do it consciously. Carry a moleskine with you, and make notes in it as you talk to people. The act of writing stuff down cements it in your brain. If you’re in a meeting and someone is describing features to you, get a marker and draw it on the whiteboard, to make sure you understand. If someone asks you about the project, try to answer them from memory. If you can’t, tell them you will find the information and get back to them. DON’T SEND THEM CHASING THE ANSWER IN DOCUMENTS IF YOU CAN’T ANSWER THE QUESTION. This simply postpones learning for you, and you don’t want to do that. It’s okay to send someone after documents when you do understand, if you have a reason to do so (like lack of time, etc), but only when you are already fluent.

Developing fluency is critical, not only in appearing competent, but actually being competent. I know it’s a bit squirmy feeling to think of oneself as incompetent, but if you or I can’t easily and accurately describe the six aspects of a project mentioned above without referencing notes, documents, or “experts”, we should consider the possibility that we are in fact incompetent to some degree. And others can see it.

I’ve been wondering why this is. Other project managers I know rely heavily on specialists, particularly business analysts, to make their projects successful. One of my colleagues recently observed that I don’t need analysts because I do all the analysis myself. I guess that’s partially true – the abilities to understand business needs, analyze them, design solutions to meet them, refine designs based on feedback, implement them, and then test them effectively are prerequisites to being on my project team. I don’t work with “just” coders, testers, analysts, or other specialists unless I have a specific need for a specific skill. This need must go beyond what I consider fundamental software development, which includes a certain proficiency at analysis, design, coding, and testing.

Are these specialties valuable? Yes, absolutely. There is a craft to testing that only a few have really mastered. I am lucky enough to know a few of them, and it has broadened my perspective. The same might also be true of business analysis in software projects, I just haven’t observed it yet.

That said, I believe the importance of specialists in software projects is largely overstated for all types of specialists, including project managers. Not every project needs a dedicated PM, tester, analyst, architect, etc. You don’t need Magellan to find your way to work, and I don’t need a tester every time code gets written.

To make matters worse, division of labor across a series of specialists doesn’t often make sense, although corporate IT continues to move in that direction in spite of the gross failure of waterfall and other phase-based approaches over the last thirty years. Specialization works when the hand-off between specialists is relatively inexpensive, such as in the case of the hand-off from a framer to a sheet-rocker to a trim carpenter to a painter to a … etc. Framers don’t produce hundred page documents telling the sheet rocker what to do. It’s relatively obvious what he needs to do, and the communications mechanism for that exchange evolved very quickly into standards that are widely accepted across that industry.

This is not generally how software projects work. Instead, analysts have to produce copious documents that aren’t easily turned into design and code and tests and software deliveries. We’ve been trying to establish easy to follow standards for this hand-off for more than a lifetime. I think it’s time we accepted this and relied on upfront analysis only as a last resort.

Another scenario where specialization works is when the work requires so much knowledge that it’s impossible for a generalist to perform. Heart surgery, for instance, is not something a family doctor does. Nor would I recommend you try to restore a hundred year old work of art just because you know where Hobby Lobby is. This is, I believe, what many forms of testing are. Security testing, performance testing, usability testing, and lots of other testing areas are not for the untrained. I don’t think it applies to a lot of the run-of-the-mill software testing I see in projects around me, even though these projects have brought in specialists to do that work.

There are other reasons for bringing in specialists. Sometimes, when a team establishes a rhythm for development that is consistent and sustainable, certain activities reach a threshold where it makes sense to bring in specialists. Maintenance comes to mind, especially if the tickets tend to be fairly self-contained.

I hope nobody thinks I’m trashing the crafts of testing, business analysis, coding, etc. These are valuable and necessary, and I think that’s my point. Specialists make it easy for coders to think they don’t need to be able to test or talk to business users, and I think this is generally harmful to the software development profession as a whole. Everyone needs to be able to wear those hats to a certain degree and should focus some energy on developing their ability to play in every aspect of a project and contribute meaningfully.