Software Estimation
From Zanecorpwiki
Many developers, very much including myself, are terrible at estimating how long development projects are going to take. We're often lucky to off by a factor of two and a factor of 10 is hardly surprising. What's worse, we're always off in the wrong direction so it would seem obvious how to compensate, yet we just can't seem to get it right.
How is it that supposedly smart professionals trained in logic with decades of experience get the question how long is this going to take? so very wrong every single time. As it turns out, there are good reasons.
Contents |
New Work
First of all, we actually don't get it wrong every single time. The first problem with estimation is new work. The problem sometimes seems endemic because many (high end) software developers spend the majority of their time on new work: new implementations, new features, new products. Building a new application is less like building a new house and more like constructing high end architecture. It's not just a change in floor plans, but the application of new techniques and fundamentally new design.
How long will doing something new take? That's a hard question, and it is that question which we're constantly failing to get right. When we roll out the tenth online store built on top the same technology, we do tend to get our estimations right, but that's entirely different work than building the online store platform. Even if one has built e-commerce platforms before, the new stuff in the new one is going to be problematic to predict.
Focus
The best developers have a strong dual focus. In the short term, they see the problems, pitfalls, and obstacles they need to address in order to make solid code. In the long term, however, all they see is the solution. Strategically, one focuses on victory, tactically, one focuses on avoiding failure.
Any project that takes more than a few hours to complete has a strategic component. If three to four hours is the window for a tactical focus, then obviously the vast majority of any project exists on the strategic level for the majority of the project. In other words, at any given time, and especially at the beginning, most of a project is colored by a view of this can be done, and this is how we're going to do it. This unfortunately, though naturally enough, leads to a sense of optimism.
As the Crow Flies
The big difference between 'strategic' and 'tactical' thinking is how one views the distance between two points. Strategically, one says: We need to get here, and there's nothing about the terrain that leads us to believe we can't. The focus is on goal and possibility and leads to an 'as the crow flies' mentality.
The tactical perspective is very different. The shortest path becomes irrelevant. To haul a heavy load, one must take roads. The fact that those roads wind round the mountain rather than go straight up the side, or take meandering switchback routes doesn't matter. With a heavy load, it's quicker to travel 1000 miles over asphalt than a single mile through dense jungle.
At the beginning of the project, one's mindset is naturally strategic. It's easy enough to consider the first steps tactically, but the nature of the kind new work development that we're discussing here is such that the later terrain is only revealed as we reach it. What maps we have are imprecise and often misleading so it's not possible to pre-plan the entire route.
Efficiency
The single most significant advance in the field of software construction in the past ten years is the wide spread understanding that even if one can, it's not a good idea to make a complete tactical level plan for developing a product. The first reason, already discussed, is that pre-planning is hard. In fact, it's harder in some ways to estimate how long it will take to build software than it is to actually build it.
When we build the software, we come across surprises, but when we plan out software we have to figure out what those surprises may be without actually getting there. It's possible, but very time consuming.
The second problem with pre-planning is that it locks us in to development path that is very likely not the best path to be on. No matter how much time and energy we spend/waste on pre-planning, it's impossible to get it just right. It may turn out that it is indeed better to zip rather than zag, yet if the plan says zag, then there's a very strong motivation to zag.
Prototype vs. Production
Doing Better
Scratch
Following is some ideas originally penned in an email; I truncated the email and moved the thoughts here for the moment.
When the application is done isn't the an appropriate question. However much you want the answer is unfortunately irrelevant. It's an unknown quantity of work. There are known elements with unknown and partially implementations: Is the current design for handling designs and produced items sufficient? Is it truly compatible with 'standard stores' that sell non-processed inventory? Are the ideas for process management going to work?
There are also unknown elements. What else will you want me to do? What will I need to do for Srini? What have I missed?
These kinds of questions can be answered, and I've worked on projects where that's done. There are two major problems with that approach. First, it doesn't really change the unknown problem because the time in which you receive the answer is itself unknown. In other words, you can get an answer of how long development might take before development begins, but you don't know when you'll get the answer.
The second problem is that the answer you get is wrong. Thinking about how you're going to code something and actually coding it are two different things. One codes by abstracting, changing levels and focusing on single questions. At one point, you're thinking, How do we model inventory? At another you're thinking, How do I get this data out of the database? It's impossible to consider all the questions at once. It's beyond the capacity of the human mind.
The art of the developer is knowing what levels are appropriate, how to connect up different models, and generally how to shape the abstractions. This is a hands on process, so trying to do it a-priori guarantees some amount of mistakes. There are then two choices: you either change what you thought you were going to do and violate your time goals or you accept your original, sub-optimal (and possibly sub-par) design and violate quality goals.
This is why modern development does away with a-priori estimates. There are certainly problems with this approach. Not the least of which is it's a reality that hasn't caught up with the market or outside thinking so it's tough to deal with communicating to stakeholders. We also need to do much more in making progress real and working with stakeholders in their commitments and realities. But none of those problems change the fact that organic development is better for everyone. It yields better code faster.
The primary cost of losing project-time estimates is itself an advantage because no one ever had useful time estimates in first place. They just thought they did.
Of course I'm talking here of new work. When replicating a process it is possible to give estimates. The more highly developed the process, the tighter the estimates.
It is likely that future techniques will allow us to bound at least some of what currently falls into the new work category, shifting it from new work to process. There are certainly ideas and discussions about how this might be done. This process of estimation and bounding itself can be classified as new work and process. It is even possible that some techniques do exist which would allow me to give better estimates on this kind of work I'm doing, but at the very least these techniques are not mainstream.
