I’ve recently discovered a very powerful personal health metric’s usefulness: calories. I first started tracking calories with uladoo.com and twitter, then recently switched to Lose It!, an iPhone app recommended to me by a friend. As I’ve tracked calories, I’ve realized how little I knew about nutrition. It was astounding to discover there are more than 10 times the calories in a tablespoon of Hershey’s syrup than there are in a large asparagus stalk. Cheese, sour cream, ice cream, chocolate, soda, and even macaroni and cheese all have a completely different meaning to me than they once did. Counting calories has really changed my perspective and altered my behavior. But counting calories in itself is not the really powerful part – it’s the calorie budget that really makes a difference. I can eat 2,642 calories every day and lose a pound a week. If I eat more than that, I won’t lose weight. If I eat less, I will lose weight more quickly.

You might think that this experience has changed my viewpoint about software development metrics, and you’d be right, in a way. My position hasn’t changed (still haven’t seen a useful one), but I now understand why well enough to articulate the reasoning behind my position. So, here’s my list of what makes a good metric, along with quotes meant to be good counter-examples:

The vast majority of software development metrics I’ve seen in my career violate most, if not all, of these rules.

Here are some exceptions that I’ve experienced:

Notice that this is a short list, and there are lots and lots of metrics for software development I’ve seen over the users. Why haven’t they succeeded? Because they almost always break one of my premises of metrics that matter.

Think you’ve got a metric I would like? Let’s see it then, in the comments.

Please take a minute and read the open source letter. If you agree about the importance of considering the source of software products in government projects, you can sign the letter by posting a comment with your name, company (if appropriate) and web-site. We’d also appreciate any mention you might make of the open source letter to your friends and colleagues through email, blogs, and twitter.

The letter itself is open-sourced under a creative commons license, which means you can post it, print it, share it, etc with those around you. We hope you will do just that, and help promote the idea that open-source can help bring cost-effective, transparent solutions to government software projects.

Thanks!

Dave

This frustrating tendency of corporate IT to hang on to employees was also a bit of a security blanket for me. I didn’t worry about getting fired. It’s hard to fire people and most managers just don’t have the energy to do it. It’s easier to reassign you to another group, reduce your work assignment to non-critical activities, or just ignore you altogether until layoffs come around and include you in the cut.

Getting fired for performance is not the same as getting fired for cause or getting laid off. Getting fired for cause is bad – it means you don’t get any unemployment benefits, but it’s also reasonably easy to avoid. Don’t steal, don’t forget to call in sick when you can’t make it to work, don’t pinch the secretary’s butt, and don’t use racial/sexual slurs. Getting fired for performance will not result in losing unemployment benefits, and it often comes with a severance package, though generally reduced from what you would get if you were laid off (I often hoped I would get laid off, sometimes secretly but often publicly – I would have received almost six months of pay).

Goodbye Security Blanket!
I don’t have a security blanket anymore, but oddly I’m not worried about getting fired. What I’m worried about is failing. Previously, failing didn’t worry me as much as it does now. Why? Because it didn’t matter as much to the company. What difference did my $2,000,000 mistakes make in a company that had canceled at least $200,000,000 in projects after disastrous failures? Besides, since I used scrum, I had some confidence that we would deliver something of value before we spent the entire budget, reducing the likelihood of total failure substantially.

Now though, failure means much more. The company needs me to be successful. This is not to imply that they would fail without me – I don’t think that’s true. But I impact the bottom line in a way I never have before. Throw my other ventures into the mix – trooptrack.com and uladoo.com – and suddenly the prospect of failure really, really bothers me. It scares me frankly.

Hello Upside
At the same time, the upside potential of the work I’m doing now is staggering. It blows my mind to think about being involved in a product as great as TriSano and to know I have a real stake in its success. We’re taking risks and building an awesome product that benefits the public and the government. The potential consequences of success are buoying and tremendous.

On a smaller scale, the same thing is true of TroopTrack.com and Uladoo.com. Success in these ventures has much more potential than the payoff from successfully delivering a corporate IT project ever did.

Enter the Cycle of Self-Doubt
These two forces, the fear of failure and the hope for success, play off each other in dramatic ways. I find myself cycling weekly through wild optimism and and modest pessimism about the prospects of success and failure. I’m not used to this level of emotional variability, and at first it really bothered me, until I noticed that it was cyclical. Here’s how the cycle works:

First, some sort of positive signals emerge about the prospects of something I’m working on. My enthusiasm builds off of this feedback and I feed off of it.

The enthusiasm grows and I chase after it. I work harder and get more results, until it finally peaks in a somewhat irrational level of optimism.

Finally, the work starts to make me tired. My expectations drop to a more realistic level. Reality points out that progress takes time. I rest a little bit, still pleased with where we’ve gotten to.

Resting makes me nervous about the future. I start to have thoughts about whether I have done my part. Am I pulling my own weight? Am I giving my fair share to the team? Even if I am, is it enough? I start to feel a yearning for progress, for some improvement that makes me feel positive again.

I wade back in, attacking some problem that bothers me. I make progress, and I start to feel optimistic again. Look what a difference you made by doing this, I tell myself.

Some sort of positive signals emerge as a result of these efforts, and the cycle starts all over again.

Recognizing the Cycle Makes It Bother Me Less
A week ago I noticed the cycle and started to joke about it with my friends. When they ask how things are going, I said things like “I’m at the bottom of my self-doubt cycle right at the moment,” or “I’m at the top of the ferris wheel just now”. It usually took a bit of explanation before they knew what I meant, but I found it a useful way to talk about how I’m feeling about my job.

This cycle feels very natural to me, and overall it has a positive effect on motivating me. The bottom of the self-doubt cycle prevents me from resting on my laurels while there is important work to be done, the top of the cycle feeds my imagination and enthusiasm, and the middle states are highly productive times that yield lots of results.

I know this may sound weird, but now that I can see the cycle, I like it. I really do.

Now, it’s worth noting that the problem I’m trying to solve is NOT the problem of hackers getting into your production database. I’m just trying to take a production database, create a database dump file, make that file safe for general use, and then load it in my test database. Simple right?

Yeah, actually it turns out it is. I wrote a simple Ruby script called Obfu that goes through the dump file, finds the fields I’m worried about, and changes them to random (but human readable) values.

Why not just encrypt the sensitive data? Well, sure, that would work, if you want to read X87rkd834k for a first name, I guess. I’m not sure what an encrypted email address, phone number, or street address looks like, but I have doubts about whether it would format properly.

So, here’s how Obfu works: First, you grab the source code from TriSano.org with this link: Obfu

Next, change the method get_obfu_config to include the tables and columns you want obfu’ed. What get_obfu_config is simultaneously complex and simple: it’s complex because it uses an array of hases of arrays of hashes, but it’s simple once you get over that.

Here’s an example from the code that might help you understand how that array/hash/array/hash mess works:
[
{#COPY telephones (id, location_id, country_code, area_code, phone_number, extension, created_at, updated_at, email_address) FROM stdin;
:table_name => 'telephones', :fields => [
{:field_loc => 4, :type => 'value', :value => '555'}, # Area code
{:field_loc => 5, :type => 'num', :digits => 7}, # Phone number
{:field_loc => 6, :type => 'num', :digits => 3}, # Extension
{:field_loc => 9, :type => 'email'} # email
]
},
{#COPY people (id, entity_id, birth_gender_id, ethnicity_id, primary_language_id, first_name, middle_name, last_name, birth_date, date_of_death, created_at, updated_at, food_handler_id, age_type_id, approximate_age_no_birthday, first_name_soundex, last_name_soundex, vector, live, next_ver, previous_ver, disposition_id) FROM stdin;
:table_name => ‘people’, :fields => [
{:field_loc => 6, :type => 'text', :word_count => 1}, # first_name
{:field_loc => 7, :type => 'text', :word_count => 1}, # middle_name
{:field_loc => 8, :type => 'text', :word_count => 1}, # last_name
{:field_loc => 9, :type => 'date', :keep_year => true} # birth_date
]
}
]

Each item in the big array is a table, and it really only has two values: table name and fields. The fields value is an array of the columns you want obfuscated. Each column has a hash of values that tell obfu what to do to the column. You’ll notice I don’t have a field name for the column name. That’s a little lazy on my part and I’d love you to fix it (the TriSano OSS license requires you contribute improvements back, if I understand it correctly). I use :field_loc instead, which is simply the position of the field in the COPY statement in the dump file.

Every column has two values field_loc and type. All the other values depend on the type you specify. Here’s how it breaks down:

type	values
num	:digits
value	:value (I use this to always set area codes to '555')
text	:word_count
email	none
date	:keep_current_year: true or false (useful for birthdates)

Obfu uses lorum ipsum dolor text to randomly generate words for text and email fields. Numbers are all randomly generated, and so are dates, but with some limits. You won’t get day values greater than 28, for example, and you might want to consider the fact that randomly generating certain dates will break validations.

Use Obfu all you like. Tell your friends about it if turns out to be helpful for you. I’m hoping to externalize the configuration and turn it into a gem next time I get some free time (unless you do it first, in which case I’ll just thank you and use yours!).

Obfu takes two arguments: input file and output file, in that order. I haven’t added any error handling, etc., so use at your own risk.

Obfu has only been tested against Postgresql database dump files. You might need to change Obfu up to make it work with other databases.

For a few years, I was able to hold onto a little piece of my soul by blogging and writing articles, but lately I’ve fallen behind even on that, and I’m afraid my soul is completely gone. I may never see it again.

I’ve been coding lately, and that’s been a relief. Sure, it’s not incredibly complex stuff – just RSpec tests. But… still, it’s been oddly invigorating. It reminds of the time, not so long ago, when I was an expert at something. When I could code circles around myself, like the time I wrote an XML marshaller/unmarshaller for an obscure OO language, or when I wrote pattern smoothing algorithms for the high-speed cutting of carbon fiber helicopter parts.

Those were the good ole days. Why’d I ever stop doing that? What made me think management would be more fulfilling?

Java. That’s what it was. I don’t like that language. It’s a nice idea – OO with cross-platform portability, but it’s too wordy. It’s perfect for the quadruple abstraction factory guys, but not for me. I don’t want to have to instantiate four different classes just to read a file. No thank you. I prefer to just open the dang thing. Besides, that whole portability thing didn’t really work out in the end, and it never seemed to matter either. And the frameworks, with all that dang XML. Aargh. I just never liked that girl, I just liked the idea of that girl.

For a while there it seemed Java was going to take over the world. It certainly seemed to take the wind out of the proprietary language I was such an expert at. And the prospect of learning Java made me cringe.

So I turned my back on the code, and checked my soul in at my local chapter of PMI.

But that didn’t help either. The bad taste in my mouth is different now, but it still makes me want to double up on Altoids. Few things have made me want to take up drinking more than IT management.

Then along came Ruby. Well, actually she’d been around for a while, I just hadn’t noticed her. Then one day, in the middle of a ginormous crisis, I needed someone to parse a log file as big as our crisis and turn some specific time stamps into a graph. Mike Kelly did it in about five minutes. I couldn’t believe it. When I asked him how, he said “Ruby”. I didn’t start picking up on her right away, but it registered. Ruby was cute, or at least practical.

A year went by and I needed to parse some log files again. I remembered Mike and his cute little friend Ruby, and I googled her. Ten minutes later, I was looking at the graph I needed. I wasn’t quite as impressed with Mike’s ginormous crisis magic trick now, because it was pretty easy with Ruby. Java had made me expect everything to be hard.

Things blossomed from there. I discovered Rails, and I started getting more serious. Then I found some paying work on the side and suddenly critical mass materilized. Now there was money involved.

At any rate, I’m really digging this Ruby on Rails thing. I know I’m not good enough at it yet to say I have my soul back, but I’m getting there. Thanks Ruby.

If you don’t know, subversion allows you to create “hooks”, actions that are executed when certain events occur in your subversion repository. Some of the events you can use as triggers for your hooks are commits, locks, unlocks, and revision properties changes. You can create pre- and post-hooks, which just means they run before or after the event occurs.

Some examples of how these are used: sending an email to the dev team whenever a commit occurs, kicking off a deployment after a commit, or running a bunch of tests.

I needed some of these hooks for one of my projects, but I couldn’t figure out how to do it at first. Google didn’t help much – most of the examples were really old and looked like really bad ideas. Eventually, I was able to figure it out on my own, but I thought it would be a good idea to post it.

Anyway, it’s pretty easy to set these up with Dreamhost, in spite of what the Google search results will tell you.

The problem with Dreamhost’s subversion install is that all the files are owned by the dhapache user and therefore you can’t make the .tmpl files executable (the .tmpl files are sample hooks provided with the svn install). Normally, you would edit the files you want to use, then use the mv command in ssh to remove the .tmpl files, which is what makes subversion start acting on them. Unfortunately, doing it this way leaves your “hook” with no permission to execute, and you can’t change it.

So, the easiest thing to do is to first copy the hook template you want to use to a new file with no extension (post-commit.tmpl copied to post-commit, for example) and edit it as you like. Then chmod it to 755 and it will work.

That’s all it took. Now, if I exposed some security flaw hopefully one of my readers will know and tell me what a moron I am. I’ll appreciate that.