Why it always takes longer than expected
In this essay series, I will write down my own thoughts about Eliezer Yudkowsky’s essays on the Rationality: Ai to Zombies –series from the point of view of a historian. My reason for writing these is primarily to organize my own thoughts regarding the use of rationality as a mental tool in my own profession, and as such, I do not presume to even attempt to appeal to a very wide audience. However, if you are reading this disclaimer and find my essays insightful or entertaining, more the power to you and I implore you to go and read the original essays, if you have not already.
The Planning Fallacy is one that is guaranteed to hit most starting academics under the belt. To illustrate it, Yudkowsky gives a few sample results of studies exploring this heuristic.
These are direct quotes from his essay Planning Fallacy¹, where he summarizes the findings. I am including them because the point deserved to be driven home by the anvil.
Buehler et al. asked their students for estimates of when they (the students) thought they would complete their personal academic projects. Specifically, the researchers asked for estimated times by which the students thought it was 50%, 75%, and 99% probable their personal projects would be done. Would you care to guess how many students finished on or before their estimated 50%, 75%, and 99% probability levels? 13% of subjects finished their project by the time they had assigned a 50% probability level; 19% finished by the time assigned a 75% probability level; and only 45% (less than half!) finished by the time of their 99% probability level.
Newby-Clark et al. found that
Asking subjects for their predictions based on realistic “best guess” scenarios; and
Asking subjects for their hoped-for “best case” scenarios . . . . . . produced indistinguishable results.
Likewise, Buehler et al., reporting on a cross-cultural study, found that Japanese students expected to finish their essays ten days before deadline. They actually finished one day before deadline. Asked when they had previously completed similar tasks, they responded, “one day before deadline.” This is the power of the outside view over the inside view.
The planning fallacy has the most impact on the practical side of academia, and its lessons ought to be heeded by especially PhD researches and others who are taking on an expansive research and writing project perhaps for the first time in their lives. Without prior experience on such projects, we tend to over-analyze the project and counter-intuitively this leads to overtly optimistic estimations of the duration it will take us to complete the project. Yudkowksy calls this thinking in the terms of the unique features of the project the ‘inside view’, and recommends switching to the ‘outside view’ instead when organizing projects for oneself.
The outside view is, in all of its simplicity, deliberately avoiding to think about unique features of your current project and just ask how long it took others to finish broadly similar projects in the past.
This should be good news especially to all PhD candidates working on their dissertation, as they have a multitude of peer examples to draw from. And not only that, they also have their advisors, who have completed a dissertation in the past themselves, but have also likely supervised a few of them into completion before you came along. Their estimations should not be brushed off, and one ought not to underestimate other PhD Candidates either – likely, they had their reasons for the project extending beyond what was initially planned. The “inside view,” does not take into account unexpected delays and unforeseen catastrophes.
… And still, I expect my own dissertation project to be finished in the year 2023, maternal leaves in between and all. In my defense, I was faster (around 25% faster) than the average student is during my BA and MA, and I am in a particularly favourable position because I have steady funding until the end of 2022.
Let this blog entry stand as a lesson in humility and the perils of hubris, should 2024 come without me being a PhD.
¹ Yudkowsky, Eliezer. ”Rationality: from AI to Zombies” Berkeley, MIRI (2015). 30–33.
The studies in the quotes are, in the order of appearance:
Roger Buehler, Dale Griffin, and Michael Ross, “Exploring the ‘Planning Fallacy’: Why People Underestimate Their Task Completion Times,” Journal of Personality and Social Psychology 67, no. 3 (1994): 366–381, doi:10.1037/0022-3514.67.3.366; Roger Buehler, Dale Griffin, and Michael Ross, “It’s About Time: Optimistic Predictions in Work and Love,” European Review of Social Psychology 6, no. 1 (1995): 1–32, doi:10.1080/14792779343000112.
Ian R. Newby-Clark et al., “People Focus on Optimistic Scenarios and Disregard Pessimistic Scenarios While Predicting Task Completion Times,” Journal of Experimental Psychology: Applied 6, no. 3 (2000): 171–182, doi:10.1037/1076-898X.6.3.171.
Roger Buehler, Dale Griffin, and Michael Ross, “Inside the Planning Fallacy: The Causes and Consequences of Optimistic Time Predictions,” in Gilovich, Griffin, and Kahneman, Heuristics and Biases, 250–270.]