January 28, 2009

Parkinson's Law - A lesson in estimating

I came across Parkinson's Law about a month ago.
Work expands so as to fill the time available for its completion.
This small but powerful statement originated from a humorous essay(pdf) by Cyril Northcote Parkinson which was published in The Economist, and later reprinted with other essays as a book. Although its original intent was humorous, I think there is quite a bit of truth to it.

Today I finally found a time to share my new found knowledge, which I did in this tweet:
[I] Looked over our last 3 months of work...we've hit our original estimate almost dead on. Good estimates or Parkinson's Law? http://is.gd/hqAH
At the time, I meant it as more of a rhetorical question and a way to brag about my estimates but imply that I've been sandbagging. But after getting this reply from my friend Chris, I thought more about it.
@NateSchneider Definitely a hazard of conservative estimation. Was your estimate conservative or aggressive?
It is a very good and valid question. Sadly, I'm not sure if I know the answer. This project has been a few different "firsts" for me, and without any prior experience, I'm not sure if I know.

This was the first time that I have ever done estimating for a team of people instead of just myself. It was the first time I, alone, have estimated that much work at once. And finally, it was also the first time that I used agile/scrum methodologies for organizing, managing, and estimating tasks. (Instead of making estimates of how long I thought a task would take, I gave it a size value relative to other tasks. I then tracked the total amount of points of all of the tasks that I completed in a two week period, took that number and multiplied it by the number of resources to estimate the how much work could get done in a two week period of time).

So, did I strike gold with my estimates or are we just bums slacking since we have time? The fact that this has been my first go around in several aspects would suggest the latter. But then again, frequently agile is championed for its predictability. Then again, it could just be beginner's luck.

Here is what I think I know.
Agile planning (the use of story points) does provide a reasonable forecast once a team has an established velocity. This seems to help create "realistic" estimates of how much you typically get done over a certain amount of time...and not how much you think you can/will get done (which in my experience has often been a significant difference). This has severed me well since, historically, I tend to be much more of an idealist instead of a realist. I know that I will continue to use it, if for nothing else, to have actual data to help back up the estimates that I make and help show their progress over time.

Here is what I think I think.
Agile planning as described above does a great job of tracking and planning for what typically gets done. But without a strong, active manager helping his/her developers get as much done as possible (maxing out their velocity), Parkinson's law can seep in and hold people back from completing as much as potentially could. That means for poor employees, they lallygag and don't do much else. For better employees, they do things beyond their responsibilities like helping others, doing additional research or doing work "beyond done".

I do feel that my estimates, were realistic and reasonable. But looking back, I can also think of things we did which were a bit more fluff than actual required work.

What do you think? What role do estimates play in limiting Parkinson's law if any? Any suggestions for how I could have done something different or should change in my next go around?


Steve Duitsman said...

My first thought to counter the question of "did we succumb to parkinson's law?" is this: Did you ever exceed your "attempted velocity" for an iteration? The question of whether or not you let your work fill the space is addressed if you ever added a new story to an iteration.

Also, I find Chris comment about aggressive vs conservative estimates intersting. I always thought the agile idea of getting a group consensus should eliminate any one person's propensity to pessimistically or optimisically estimate the size of a story or feature.

The feeling that some work was fluff might come from a less than stellar product owner; it's basically their job to keep the stories done in line with the vision for the project, keeping the team on track.

But to be honest, I'm new to agile concepts myself. Great post, it sounds like you're already doing your post-mortems: reviewing how you're applying agile and changing as you need to. Keep up the good work!

Ross said...

I can't answer your question. All I can say about estimating is from when I was doing hardware development. I don't believe my group ever fell victim to Parkinson's Law because we never ever met our deadlines.