A simple case study for Estimation in IT Projects

It was one fine afternoon in the middle of November, 2011 when I read this email. This email was from one of our delivery managers from Insurance business line.

To be more precise, it was a one liner and it goes like this.

“We’re looking to get an estimate to convert about 250 Crystal reports to SQL Reporting Services.”

By the way, I didn’t even change the font style in the above liner. It’s as is the way it came to us! So, how do you estimate when you have a one liner like this?

One of the options is to tell her that we needed more information. But what information do you need?

The challenge here is that there is ambiguity. If we can’t live with ambiguity, then, what’s to do with the experience we gained by losing all that hair. Thinking like a simple mathematician with a little bit of IT commonsense, I thought we needed to find out two things (even if we have never worked with Crystal Reports).

What are they? Well, I need to know what it takes to convert 1 crystal report to its equivalent SSRS report. Well, let’s just say, for someone who knows both Crystal Reports and SSRS, it takes about X hours. What do you call this X? Well, some guys with little bit of grip on vocabulary, call it “Productivity Factor”. So, once I have productivity factor, all I need to do is multiply X with 250 and bingo…we got the estimate!

Well, well, well…let’s stop…I know what you are thinking. For starters, all 250 reports can’t exactly be the same. Or rather you meant they can’t exactly be at the same level of complexity. I know what you are thinking next. We need to find out how many of them are simple, how many of them medium and how many of them are complex. How would you get that information?

Business wouldn’t know. They don’t know technology. That’s why, they wanted our help. Don’t tell me you are thinking about going through all those 250 reports and figuring out how many of them are simple, medium and complex. That ain’t gonna happen!

Why not...? Because of thousand reasons! First of all, you don’t know Crystal reports. Second of all, even if you know crystal reports and like to go through all of them, it will be few weeks before you can get the access, setup the environment, understand the logic etc…etc..etc…well, that’s only two reasons and not thousand. But still, you got my point, right? If you were to estimate, you can’t expect to get complete understanding of the system and estimate. Then, what’s the point of your experience and wisdom?

So, coming back to that afternoon, when I got this email, I sat back and started thinking what the best course of action is. I have to admit that I didn’t know either crystal reports or SSRS. With that background, so far, I figured it out reasonably okay. I needed to find out the productivity factor and classify the reports into simple, medium and complex. By the way, it would have to be three different productivity factors (for simple, medium and complex).

Where from I can get the productivity factors? Maybe I can Google it… But how do you trust that information? Then, I realized, my friend at my previous company was a master of reporting services. He wrote tons of Proposals and prepared zillions of estimations for those proposals.

I was pretty excited. I suddenly thought, this has become such an easy activity. I can get those numbers in matter of minutes.

So, I called him and asked him how much time it takes to convert a simple Crystal report into SSRS report. He said, “Buddy, slow down…it’s not that plain and simple. The environment of the system makes a big difference”. I said, “What…why? What difference can it make?”

He says “If the report is coming out of one simple database with 3 tables involved, then, its one thing. But if the same report needs to be produced out of the same 3 tables but from 3 separate databases or what even worse, one of the table is accessed through a web service, you would need more effort per one simple report. Because, you need to explore how SSRS functions in such an environment”

I asked him “Then, I would just have to classify that report as complex report and add more effort”.

He goes “Nope! You would classify that report as complex if it has complex logic involved, not because of complex environment. What if all the reports are produced like that? Then, you can’t call all of them to be complex reports. Because, once you understand how to tackle one or two such reports, then, environment is not playing any role in whether it is being simple or medium or complex”

I thought he was making sense, but wasn’t totally convinced. I asked him “Buddy, what’s your point. What are you suggesting?”

He replies “I am merely saying, don’t just take a simple report’s productivity number and multiply. You need to separate out the productivity factor from environment influences. In fact, environment influence can come in at least in two different ways. One is the technical environment which is a result of the factors such as platform complexity, involvement of frameworks, portability needs, complex GUI layer, usage of 3^rd party tools etc… and the second influence is team environment such as application experience of the team, process familiarity, motivation of the team and requirement stability etc… ,  you need to separate out technical complexity from team complexity and then separate both of them from productivity factor”.

He continues “In short, let’s just say, productivity number is 7 hours for a simple conversion in the ideal conditions. Now a team, that is highly motivated and flexible, might take 6.5 hours to produce that report in a simple system that doesn’t have much platform dependency or any 3^rd party tools.  Or the same team might take 7.5 hours to produce that report in a complex technical environment”.

I was totally lost. All I thought was, if I knew productivity factor for one report, then I can multiply it with 250 and get the total effort. And then came complexity factor and I had to break that 250 down into Simple, Medium and Complex. Along the way, I also had to figure out the productivity factor for each of those types as well. This is where I thought I can bank on my friend’s experience and identify the productiviuty factors. But he introduced me to two new factors namely technical complexity and team complexity.

I now realized, estimation isn’t as easy as the way it sounds. So, I asked him “what should I do, now?”

He gave a nice list of few simple steps:

• Break your work (the 250 report conversion in this case) into simple medium and complex category. In short, identify the size of the system. In your case, you have received it as part of the requirement.

• Based on past history, define the productivity factor for each of those categories. Usually industry practice is if productivity factor for a simple item is, let’s just say 5 hours, for medium it could be 7.5 and for complex, it could be 10. Basically a multiplication factor of 1.5 or 2 or 2.5 etc…is used to keep things simple. In which case, you only need to worry about productivity factor for simple reports (of course, we also need to worry about the multiple factors for medium and complex)

• Then, identify all the external influences for this system such as technical complexity and team complexity. In fact, it doesn’t have to be these two only. Let’s say, your system has about 40 different interfaces. Then, you would have to consider that as a separate influence in which case, you would have three influences.

• Once you have all the influences identified, you need to identify the influence factor. This could be any positive number.

• Now, just do the following math:

Estimate for a simple report = Productivity factor for a simple report * Technical complexity factor * Team complexity factor * Interface complexity factor.

As an example, let’s just say, productivity factor for simple report is 7, technical complexity factor is 1.25 (system surroundings are reasonably complex), team complexity is 0.75 (team is a very well experienced and highly charged team) and interface complexity is 1.1 (it has 6 tricky interfaces). Then,

Estimate for a simple report = 7 * 1.25 * 0.75 * 1.1 = 7.22.

He says “I can only help you with that productivity factor for a simple report. So, I can only give you that number whether it is 7 or whatever. You need to identify rest of the factors”!

This kind of looked simple enough for me with bulk load of question marks.

First of all,

1. How do I divide that 250 into simple, medium and complex?
2. How would I calculate the technical complexity factor, team complexity factor and Interface complexity factor?
3. In fact, how do I know what other influences I need to consider?

As if he read my mind, he goes “I know what you are thinking. That’s why we have estimation models such as Function Point, Test case Point, Use Case point etc…! All you need to do is, adopt one of them and they can help you with questions 2 and 3. Sometimes, if the estimation model isn’t capturing all the influences that you think you have in the system, add more buffer to your estimation”

“How about first question? How do I divide that 250 into simple, medium and complex, if I have to?”, I asked.

He goes “Make an assumption. For example, if I were you, I would go with 30% simple, 40% medium and 30% complex. Remember, end of the day, this is an estimate and estimates always come with assumptions. You need to help your business understand that the estimation might change if the assumption goes for a toss. In fact, you always need to revise your estimates after the analysis phase!”

Then, I “Googled” about Function Point Estimation and Use case based estimation and realized that what my friend saying was true. These models simplify lot of our headache! Look at a simple FP based estimation model, I found online. It captures pretty much what my friend was referring to.

Now, I realized, instead of going through so much of research on my own, let me go to my process team in my company and get the model that’s right for us. And then, I will divide the 250 reports into 20% simple, 40% medium and 40% complex (I am being little more pessimistic than my friend).

When I approached my process team, they gave me a template that’s like a combination of use case based methodology for Work Breakdown Structure (component). This perfectly suited my need and I was able to come up with reasonable estimate with after spending few hours.

ఓం నమో ప్రాజెక్ట్ మనేజ్మెంతాయ నమః