Motivated Cognition and Bias

The last post functioned as a light introduction to the field of heuristics and biases. To reiterate, a cognitive bias is any systematic deviation from optimal reasoning. The human brain is a tangled mess of spaghetti code with a long history of cognitive systems (such as a network of neurons) being re-purposed for some new and unrelated function. Knowing this, we should not be surprised to find a great number of errors and biases in our calculations and judgements. While any deviation from optimal reasoning is cause for some concern, some biases weigh much more heavily on our judgements than others. One especially virulent strain of bias is known as Motivated Cognition, a technical term for any tendency to bias our interpretation of facts to fit a conclusion that we wish to be true. There are several forms of this bias, all intimately connected. They are motivated skepticism, wishful thinking, rationalization and confirmation bias, and learning to recognize and correct for them is one of the most important and potentially life changing challenges to overcome.

-

Cognitive Bias and the Availability Heuristic

A cognitive bias is a pattern of deviation from optimal reasoning or judgment caused by our brain architecture. Our brains evolved under various selective pressures including predation and resource/energy constraints. Genes that most closely approximated the optimal trade-off between accuracy and speed/energy consumption survived and multiplied while the rest produced organisms either too prone to error or too slow and cumbersome.

If we were to take every piece of data and every variable into account before forming a prediction or making a decision we would be effectively paralyzed. However, we must be able to make relatively accurate predictions in order to survive in a dangerous world. One example of a heuristic that our brains use to approximate optimal reasoning is the availability heuristic.

 -

Humans have poor intuitions regarding statistics and probability. When we think of the likelihood of an occurrence we do not perform a statistical analysis; we remember anecdotes, stories and examples that we have recently encountered. This is known as the availability heuristic. The greater our ability to visualize an event, the greater the likelihood we attach to it. David McRaney of You Are Not So Smart explains; 

"School shootings were considered to be a dangerous new phenomenon after Columbine. That event fundamentally changed the way kids are treated in American schools, and hundreds of books, seminars and films have been produced in an attempt to understand the sudden epidemic.
The truth, however, was there hadn’t been an increase of school shootings. During the time when Columbine and other school shootings got major media attention, violence in schools was down over 30 percent. Kids were more likely to get shot in school before Columbine, but the media during that time hadn’t given you many examples.
A typical school kid is three times more likely to get hit by lightning than be shot by a classmate, yet schools continue to guard against it as if it could happen at any second.
When you buy a lottery ticket, you imagine yourself winning like those people on television who get suddenly famous when their numbers are chosen, but you are far more likely to die in a car crash on the way to buy the ticket than you are to win.

-

The availability heuristic does a passable job of approximating statistical analysis; after all, the greater the frequency of an event the more likely an individual is to have experienced or heard examples of it. However, this heuristic sometimes ceases to approximate statistical analysis. Such is the case with news reporting. 

The problem with news reporting is that what is reported does not reflect a representative sample. News is reported because it is interesting, and that generally means unusual. Modern news media now has an entire world of events to choose from. We are exposed to kidnappings in Europe, corruption in China, and terrorism in the middle east. 

There are benefits to being informed with regards to important events around the world, and this is not a general argument against such exposure. However, it is undoubtedly the case that our beliefs about the frequency of rare events are drastically skewed relative to their actual frequency, especially compared to events that aren't considered 'newsworthy'. One need only look at the vaccination scares, terrorism paranoia and other 'epidemics' that flare up regularly for examples of the availability heuristic gone haywire. 
 
-

There is no quick and easy solution to the problems caused by systematic cognitive biases. Scholarship is not enough; learning about cognitive bias will not magically remove it. After all, these heuristics are the results of our particular cognitive machinery, not choices that we knowingly make. However, acknowledging our bias is the first step in improvement. Learning the ways in which our biases affect our judgment and the situations in which they are most prominent is the second step. Noticing when our judgment is likely affected and actually implementing strategies to compensate is one of the most important skills a rationalist can develop. It is only when we can overcome bias that the goal of epistemic rationality, or the map that reflects the territory, can be obtained.


For more information on specific biases and the conditions under which they occur see Kahneman & Tversky "Judgment under Uncertainty: Heuristics and Biases". The relevant fields are Cognitive Psychology, Cognitive Science and Behavioral Economics. For a basic overview read through the archives of You Are Not So Smart. A more technical treatment can be found at LessWrong. 
For more on the topic of evolution I recommend Richard Dawkins "The Selfish Gene" and "The Blind Watchmaker", as well as any introductory Evolutionary Psychology textbook.

The Map that Reflects the Territory: Rationality and Belief

The map-territory relation is a useful pedagogical tool that I shall be employing regularly. It illustrates the distinction between a model or abstraction and the thing in itself. A map is a model of a territory and the amount of detail is necessarily compressed. When the two diverge, when my map shows a bridge or path that does not exist, territory supersedes map. Territory is invariant; I cannot manipulate the territory by altering my map.

-

What does it mean to be a rationalist?

Epistemic rationality is the process of forming and updating beliefs that systematically improves the correspondence between your map and the territory. This correspondence is usually referred to as truth or accuracy.

I will get into specifics regarding the process of belief formation and updating in a later post. What is currently necessary to understand is that the goal of the rationalist is to obtain beliefs that correspond to reality as closely as possible.

-

Why should you care about that?

Instrumental rationality is the process of actually achieving your values.

Try and imagine the many different places you could be a year from now. The smallest decision you make at this instant will influence the opportunities available to you an hour from now, which can themselves influence later opportunities, Ad infinitum. The space of possible future-selves is practically infinite. 

Now imagine where you would like to be a year from now. The number of future selves that you would be happy with given your current set of values, your utility scale, relative to the number of possible future selves is infinitesimally small.

It is our unfortunate condition that we must reach out and pluck the future that we desire from a massive space of possibilities. The odds of success vary based on the strategy employed. Closing our eyes and grasping a future at random, the probability of achieving our desired goal is close to 0%.

Fortunately for us there is a regularity to the system. We can extrapolate probable results of each choice, focusing on those results most likely to converge on our desired outcome. This has the effect of constraining the space of future-selves to a smaller, weighted sample with a much higher desired/undesired ratio.

The extent to which we can accurately predict the results of our choices determines our probability of success. An increased degree of accuracy will increase the probability of success at a greater than linear rate.

To see how this is so, imagine you are walking down a path.

You must start at the bottom and want to reach the destination that is circled in red. The correct sequence of moves is L R L L. If you make a mistake at any point in the sequence you will not reach your destination.

Lets assume that your prediction accuracy is 50%. The odds of you reaching your destination are 1/16 ((1/2)*(1/2)*(1/2)*(1/2)) or 6.25%.

Now lets assume that your accuracy is 60%. The odds of you reaching your destination are 162/1250 ((3/5)*(3/5)*(3/5)*(3/5)) or 12.96%

Finally, lets assume that your accuracy is 70%. The odds of you reaching your destination are 2401/10000 ((7/10)*(7/10)*(7/10)*(7/10)) or 24.01%

The following results continue the pattern:

80% accuracy = 40%
90% accuracy = 65%
95% accuracy = 81%

-

This is how Epistemic Rationality works to enhance our Instrumental Rationality. The more accurate our map of the territory, the greater our ability to navigate the territory.

A Note on my Writing Style

"Of all the sentimental errors that reign and rage in this incomparable Republic, the worst is that which confuses the function of criticism, whether aesthetic, political or social, with the function of reform. Almost invariably it takes the form of a protest: “The fellow condemns without offering anything better. Why tear down without building up?” So snivel the sweet ones: so wags the national tongue. The messianic delusion becomes a sort of universal murrain. It is impossible to get an audience for an idea that is not "constructive"—i.e., that is not glib, and uplifting, and full of hope, and hence capable of tickling the emotions by leaping the intermediate barrier of intelligence."

H.L. Mencken, Prejudices: Second Series, 1920

Mencken continues to be one of my favorite writers in the English language; his humor, propensity for metaphor and clever turns of phrase are second to none. Mencken was a major influence in my literary development. He introduced me to the joy of parsing language.

I have received several comments characterizing my writing style as pretentious, dry and boring. While I bristle at the former, as it makes claims about intentions which are frankly known only to me, the latter are legitimate complaints. I hope in this post to touch on several influences which will explain, if not excuse, my writing.

-

I have always been a reader, although my taste generally encompassed a very narrow range. I primarily read fantasy and Science Fiction, the vast majority of which had engaging plots and were tightly written to that effect. There is something to be said for this kind of writing, and I devoured them at an increasing rate. At one point, this was after my high school graduation, I was reading the works of Philip K Dick and Kurt Vonnegut at a rate of almost a book a day. This amounts to roughly $300 over the course of a few weeks! I still enjoy these genres and the writing style that accompanies them, which is genuinely efficient. Plot is the primary focus and the utilitarian prose reflects that.

I eventually stumbled upon "the classics" and fell in love with Russian literature. Dostoevsky, Gogol, Pushkin and Chekhov were all I read for a great while.

A shift in focus from plot to characterization and theme resulted in a shift in writing style. There was more plot in the first 20 pages of 'The Icarus Hunt' by Timothy Zahn than the entirety of Dostoevsky's 'Notes from Underground'; the former required a strict adherence to action and movement while the latter had recourse to detailed dialogue and description. While neither tool is inherently better than another, the scale of the canvas determining the size of the brush, I did find the styles and themes of classical literature more rewarding.

Finally, I began to read philosophy and popular science books. I started with classical western philosophy such as Plato and David Hume, and progressed to Bertrand Russell, A J Ayers, Rudolf Carnap and Daniel Dennett. At the same time I read as much science as I could, both online and off, mainly focusing around Economics, Cognitive Science, Physics, Probability and Evolutionary Biology.

It is from these that I am mainly influenced; the counter-intuitive nature of much of their conclusions had to be reasoned with a clarity and precision that demanded careful wording. From my study of classical philosophy I learned to be wary of casual, non-rigorous exposition. Too many great minds had fallen pray to vague arguments and intimations that relied on the prevailing intuitions of the day rather than explicit logical inference. 

I came to realize the strengths of using precise, technical vocabulary in addition to enjoying the creative imagery and phrasing of Pushkin and Mencken. Whether or not I am successful at melding the two is up for debate, but I find it difficult to write any other way.

What is this and Why is it here

I have had several experiences over the past few years that have solidified my suspicion that direct, spontaneous conversation is a nearly worthless means of communication.

-

Communication is difficult. The main problem of communication is overcoming the inferential distance between yourself and your target. An inferential gap is the result of a difference in the stock of prior knowledge between two individuals. Imagine being tasked with explaining calculus to a student currently enrolled in remedial algebra.

The process of understanding requires the tackling of single inferential steps in succession, starting from a point which is already understood. The order in which these steps are encountered are of primary importance. It is when you can derive an equation on your own, rather than recite it from memory, that you attain understanding of it.

So what's the problem?

-

It is difficult for us to ascertain someone's current stock of knowledge on any subject in any reasonable amount of time. Too often I have mistaken a social signal for a meaningful proposition, with awkward results. Often, the knowledge of a specific jargon or vocabulary is not enough to determine an adequate understanding of the relevant field.

When Deepak Chopra says that consciousness is a 'non-local, non-unitary quantum phenomenon', he is dressing his actual propositional content or meaning (consciousness is mysterious) in scientific lingo (he understands quantum physics and is therefore credible). This says nothing about his understanding of theoretical physics, or lack thereof.

If we are to truly communicate, we must determine the state of our target's knowledge and begin our chain of inference at a point that is mutually understood.

-

Direct, verbal conversation imposes several constraints on this endeavor.

First, conversations are generally unstructured and informal. It is difficult enough to follow a valid chain of logical reasoning when written down, let alone with all the distraction and imprecision that accompanies spontaneous conversation. A conversational argument fails when it becomes a barrage of unordered inferential steps rather than a structured elucidation of inferential steps in ascending order (from simplest to greatest complexity).

In addition, direct verbal communication is more susceptible to signaling/relationship constraints. I was recently at a lunch with close family members where a potential opportunity to communicate my beliefs was curtailed by a desire to avoid controversy and upset the enjoyable atmosphere.

The final constraint is time, which is often limited.

-

In summary, the constraints of structure, signaling and time are significantly reduced in a written format.

I will be able to write comprehensively enough that people with varying knowledge bases can successfully follow the inferential steps in my arguments.

If I am successful in the above, certain concepts and arguments may require more time than often available in casual conversation. The beauty of the written word is that repeated readings are available and you can stop and begin at your leisure.

I will be able to write honestly without the burden of managing social relationships (please be understanding on this point). I won't disclose any identities but I do plan to write on events that have shaped my understanding of the world, and some of these are personal.

-

And so, finally, what is this blog and why does it exist?

In this blog I will focus on what is possibly the most important and far reaching subject of all - rationality.

This will lead me to eventually explain (in a non-technical, intuitive sense): Bayesian Epistemology, Probability Theory, Solomonoff Induction, Aumann's Agreement Theorem, Kolmogorov Complexity, Causal Decision Theory, Cognitive Bias, and much more.

Written as such, these topics seem intimidating. However a qualitative understanding is not difficult at all and definitely worth the effort. While it may not seem like it, these subjects are both interesting and of profound practical use in most areas of our lives.

I hope you stick around for future posts, and feel free to leave a question or comment.