Mundane Intellectual Honesty
Updated: Oct 30, 2019
In this essay series, I will write down my own thoughts about Eliezer Yudkowsky’s essays on the Rationality: Ai to Zombies –series from the point of view of a historian. My reason for writing these is primarily to organize my own thoughts regarding the use of rationality as a mental tool in my own profession, and as such, I do not presume to even attempt to appeal to a very wide audience. However, if you are reading this disclaimer and find my essays insightful or entertaining, more the power to you and I implore you to go and read the original essays, if you have not already.
Historians have the liberty to interpret their sources with scarcely any limitations. Having an established theoretical framework helps, since the reader can then draw on previous knowledge of the type of reasoning used as they proceed with a book or an article. Nevertheless, as long as we can explain our thought process to whomever who might take issue with our assessments, our interpretations are considered valid, even if people may disagree with us. Consequently, it is quite beneficial to keep track of your thoughts as you do research, preferably by writing down the path your thoughts took to get to a conclusion – including the possible leaps of faith along the way. You might be surprised to find how many of your assumptions are actually based on cached thoughts rather than actual evidence. When pointed out, these reasoning mishaps can cause embarrassment or worse, resistance in yourself to give up your unfounded ideas (because you already became attached to them, and admitting to being wrong is hard). Wherever you may find a mushy step in your reasoning that either equates to ‘…it’s complex’ or ‘Step X emerges from Step Y’ without Y giving any concrete hint of why X would emerge from it, get brutal with yourself and replace the step with “I don’t actually know what happens here”. You can then return it to try to find out what happens there later, or be honest and admit to not knowing everything about the phenomenon you are studying. On the bright side of this self-scrutinizing, you may get novel ideas and perspectives on several points of interest upon reviewing your thought process.
Thinking is such a natural process that society does not put enough emphasis or give credit to those who do it exceptionally well. To think well, you need to meta-think, and through meta-thinking you will be able to find the blind spots of your reasoning and even predict some mental processes before your brain subjects you to them. As Yudkowsky says in his essay “The Lens That Sees Its Own Flaws”:
“The whole idea of Science is, simply, reflective reasoning about a more reliable process for making the contents of your mind mirror the contents of the world.”¹
We want our assessments to be our best possible estimates of the state of reality – everything else is a lie. It is easy to get carried away with ideas that we like – either because it feels like we are offering a novel perspective that will get us attention or because our assessment falls in line with our previous expectations about how the world is. When this happens, we are tempted to not think too hard of the process that our minds went through to reach our conclusions. It is natural to want to be proven right, but a historian, like any other scientist, should be most pleased when they are looking at the world with as few filters as possible.
On a related note, the quest for accuracy can unfortunately take a banal turn at times. Sometimes (quite often), discussions within the field about the true nature of things transform into debates about semantics and what we actually mean when we use certain words. To be sure, it is useful to get on the same page about the use of terms like ‘nationalism’ or ‘commerce’ etc. because of the risk of anachronisms. We also want to avoid discussion participants having a separate idea of what kind of a phenomenon is being discussed, as conversations like these lead nowhere. However, when we delve into discourse about ‘truths’ in history, I am often reminded of this debate example by Yudkowsky:
“Suppose that, after the tree falls, the two walk into the forest together. Will one expect to see the tree fallen to the right, and the other expect to see the tree fallen to the left? Suppose that before the tree falls, the two leave a sound recorder next to the tree. Would one, playing back the recorder, expect to hear something different from the other? Suppose they attach an electroencephalograph to any brain in the world; would one expect to see a different trace than the other? Though the two argue, one saying “No,” and the other saying “Yes,” they do not anticipate any different experiences. The two think they have different models of the world, but they have no difference with respect to what they expect will happen to them.”²
More often than not, semantic discourse between history ends up in this territory of preferred terms, even if the starting point began with a genuine attempt to understand what is actually being talked about. If the expected end result of a discussion is at best the victory of one term over another without either of the debating parties having actually changed their minds about the contents of the phenomenon being discussed, I struggle to find a point in these interactions.
Our beliefs should be our best possible estimates of the nature of reality, and we should avoid using muddled language whenever we can. Be that as it may, getting too wrapped up in semantics over substance only make discourse within the field harder, as well as making us look petty to any outside listener who might be interested in what we have to say about the phenomena themselves.
¹ Yudkowsky, Eliezer. ”Rationality: from AI to Zombies” Berkeley, MIRI (2015). 40–42.
² Yudkowsky, Eliezer. ”Rationality: from AI to Zombies” Berkeley, MIRI (2015). 45–48.