In 1994, three psychologists asked a group of university students a simple question: when will you finish your thesis? The students, on average, predicted 33.9 days. They actually took 55.5 days. Only about 30 percent of them finished by the date they had named (Buehler, Griffin, and Ross, published in the Journal of Personality and Social Psychology).
Read that again, because the detail that matters is easy to miss. These were not strangers guessing about someone else’s work. Each student was predicting their own task, which they understood better than anyone, and they were off by nearly two thirds. The researchers gave the pattern a name that has stuck for thirty years: the planning fallacy.
If you manage people, you commit some version of this error every week. You tell your boss the migration will be done by the end of the month. You promise the client a Friday delivery. You look at your own week on Monday and decide you can absolutely fit in the hiring plan, the budget review, and the two escalations you already know about. Then Friday arrives and half of it is still open. The frustrating part is that you are not careless and you are not lazy. You are running a prediction system that is built to be wrong in one specific direction.
Your estimate is a plan, and the plan is a best case
Here is the mechanism, because understanding it is what lets you fight it. When you estimate how long something will take, you build a story. You picture the task going the way you intend it to go: you sit down, you focus, the pieces fall into place, nobody interrupts you, the vendor responds on time. Daniel Kahneman called this the inside view, and it is the natural way every one of us plans (The Decision Lab’s summary of the planning fallacy).
The inside view is not stupid. It is detailed and specific and it feels like rigor. The problem is that it only ever models the version where things go right. It has no slot for the interruption you cannot predict, the dependency that slips, the thing that turns out to be harder than it looked. And those events are not rare exceptions. Across a career, they are the norm. Any given delay is unpredictable. The existence of some delay is close to certain.
Kahneman told a story on himself that captures this perfectly (recounted in his Edge master class on thinking). He once assembled a team to write a textbook and asked everyone to estimate the finish date. The answers clustered between 18 and 30 months. Then he asked the one member who had actually seen other teams do this kind of work how long those teams had taken. The man went quiet, then admitted that roughly 40 percent of comparable teams never finished at all, and the ones that did had taken seven to ten years. The team pressed on anyway. The book took eight years, and by the time it was done nobody wanted it. The expert had the outside data the whole time. Sitting inside the project, even he had forgotten to use it.
The scale of it should scare you
If this were only a quirk of students and academics, you could shrug it off. It is not. Bent Flyvbjerg spent decades building a database of more than 16,000 large projects, tracking what each one promised at the moment leaders committed to build it against what actually happened. His finding has a name too, the iron law of megaprojects: over budget, over time, under benefits (Flyvbjerg’s own account of the iron law). Only about half a percent of the projects in that database came in on budget, on time, and delivered what they promised. IT projects were among the worst, routinely running far past their estimates.
These were not projects run by amateurs. They were staffed by professionals whose entire job was to plan and deliver at scale, working with formal methodologies and real accountability. The optimism survived all of it. That is the part worth sitting with. Better planning discipline, more detailed Gantt charts, tighter project management: none of it fixes the inside view, because the inside view is generating the numbers those tools organize.
Ask a different question
The fix is almost insultingly simple, which is why so few managers use it. Stop asking how long this will take. Start asking how long it took last time.
Running network operations at a large telecom years ago, I owned maintenance windows: the overnight changes to live systems that customers never see unless they go wrong. Early on, I estimated those windows the way everyone does. I walked through the change step by step, added up the pieces, and landed on a number. Three hours. Every time, three hours. And nearly every time, we were still on the bridge at hour five, working a rollback nobody had modeled, waiting on a callback, chasing a config that behaved differently in production than it had in the lab.
What eventually changed my estimates was not a better method for walking through the steps. It was a spreadsheet. I started logging how long each window actually took, plainly, with no story attached. After about a dozen entries the pattern was undeniable. My three-hour changes were four-and-a-half-hour changes, and the standard deviation was ugly. So I stopped estimating from the plan. When a new window came up, I did not reason about it. I pulled the log, found the three or four most similar changes we had actually run, and used the real distribution of those. My forecasts got dramatically less optimistic and dramatically more accurate at the same time. My credibility with the people who depended on those dates went up, not down, because I stopped promising things I could not deliver.
That move has a formal name, reference class forecasting, but you do not need the jargon. You need the habit. Before you commit to a date, find three to five times you or your team did something genuinely comparable, and look at how long those actually took, start to real finish, including the mess. Then anchor your estimate to that history instead of to your hope. The reason this works is not that history is smarter than you. It is that history already includes the interruptions, the slippage, and the surprises that your plan quietly leaves out.
Building the practice into how you work
You will resist this, and it helps to know why. The outside view feels like giving up. Anchoring to a discouraging past number feels like admitting you are not good enough to beat it. This time will be different, you tell yourself, because this time you will focus, protect the calendar, stay ahead of it. Sometimes that is even true. But you are the least reliable witness to your own future discipline. It is the same self-serving optimism that keeps managers pouring effort into a plan long after the evidence says stop. The estimate that assumes a better version of you is the estimate that misses.
A few things that make the outside view stick:
Keep a boring log. You cannot forecast from a reference class you never recorded. Track how long recurring work actually takes, in plain numbers, with no explanation for why this one ran long. The explanation is where the excuse lives, and the excuse is what lets you discount the data next time.
Estimate ranges, not points. A single number is a promise you will break. “Four to seven days, most likely five” tells the truth about uncertainty and gives the person on the other end something real to plan around.
Watch what you actually commit to. The planning fallacy is not only about individual tasks. It is why your week is overbooked before it starts, the same way a fully utilized team has no capacity to actually ship. If your Monday plan assumes zero interruptions, you have planned for a week that has never once happened to you.
Add the switching cost. Your history captures how long the work takes when you can stay on it. Real weeks are shredded by context changes, which carry their own tax on top of the raw estimate. I have made the case for how much switching quietly steals elsewhere, and it belongs in your padding.
When you genuinely have no history, because the work is new, borrow someone else’s. Find a person who has done something like it and ask what actually happened, not what they meant to happen. Then, because even their number came from an inside view once, add margin on top.
The number you can trust
Twenty-five years of managing has taught me that the confident estimate is almost never the accurate one. The manager who says “three hours, easy” is reasoning from the plan. The manager who says “history says closer to five, so let me build around that” is reasoning from reality, and reality is what your team and your commitments actually run on.
Your estimate is a hope wearing the costume of a fact. Your track record is the fact. When the two disagree, and they will, trust the record. It has been right about you far more often than you have been right about yourself.