A Confidence Game

Why do some people remain confident when reality keeps telling them they’re getting it wrong?

Are we less rational than we think? Does our rational faculty systematically fall prey to the stories we tell ourselves to order our lives?

Renowned psychologist Daniel Kahneman has addressed these questions in a lifetime of research. An excerpt from his new book addressed the questions clearly. It appeared in the New York Times Magazine yesterday.

At a time when he was doing his military service in the Israeli army Kahneman and another soldier were tasked with determining which of a group of recruits would make the best officer material.

They were instructed to evaluate the recruits using a set of rules that had been established by the British military during World War II.

To test the recruits Kahneman and his colleague were instructed to set up an artificial task. They observed how each of the recruits functioned and wrote their report. They felt confident that they had identified the best future officers.

A few months later they were called in for a feedback session. They were told that they had gotten it wrong. In fact, their conclusions were so wrong that they might as well have been throwing darts at a list of names.

Having received this glimpse of reality, Kahneman and colleague were asked to evaluate another group of recruits, using the same set of rules.

Apparently, experience had taught them nothing. They made the same mistakes the second time. They were fully confident in their judgments, and again, they got it completely wrong.

How could their confidence, Kahneman asked, remain untouched by the verdict of reality?

In his words: “I thought that what was happening to us was remarkable. The statistical evidence of our failure should have shaken our confidence in our judgments of particular candidates, but it did not. It should also have caused us to moderate our predictions, but it did not. We knew as a general fact that our predictions were little better than random guesses, but we continued to feel and act as if each particular prediction was valid. I was reminded of visual illusions, which remain compelling even when you know that what you see is false. I was so struck by the analogy that I coined a term for our experience: the illusion of validity.”

Kahneman concludes that he was in love with a story in which he was perfectly competent, so much so that he could not give it up even when reality was telling him that it was not true.

In other words, irrational feelings trumped rational judgment.

Let’s look more closely at why they kept getting it wrong. Kahneman explains that the test itself was flawed. A one hour test was too artificial to replicate all the factors that were involved in combat command.

He explains: “We had made up a story from the little we knew but had no way to allow for what we did not know about the individual’s future, which was almost everything that would actually matter. When you know as little as we did, you should not make extreme predictions like ‘He will be a star.’ The stars we saw on the obstacle field were most likely accidental flickers, in which a coincidence of random events — like who was near the wall — largely determined who became a leader. Other events — some of them also random — would determine later success in training and combat.”

They had made up a story, true enough, but I would suggest that they had to do it. They did not make up the game or the rules, thus, they had no experiential basis for judgment.

When we lack experience, we fill in the gaps with stories.

Does this all tell us that we are less rational than we think? Does it prove that our self-confidence never concords with reality?

Perhaps it does. But then again, perhaps it doesn’t.

For all I know Kahneman and his colleague based their confidence on the fact that they had successfully followed the rules. Wouldn’t your confidence get a boost if you fulfill the conditions of an assignment?

Perhaps they should have asked themselves whether the rulebook was inadequate.

If they did not ask the question, the reason may have more to do with the fact that it was above their pay grade. Who were they, two first year soldiers, to question a set of rules established by one of the world’s great military forces?

They were not, in other words, just a couple of human subjects participating in an experiment. They were a couple of soldiers whose identities were defined by their place in the military hierarchy. It was neither their place nor their role to question their assignment.

If you are a low ranking soldier, one who has no real say in policy, you are more likely to fictionalize your experience.

Kahneman is right to say that they had too little information to make an extreme prediction. But could they reject the premise of the game they were told to play?

Here, we need to make an important distinction. As Kahneman says, there is surely a difference between the puffed-up and fragile confidence of a young soldier and the true confidence of someone who has gained his in the crucible of experience.

Two or three experimental tests do not a lifetime of experience make. The less experience you have, the more you will be relying on fictions.

A wealth of experience will give you expertise. In Kahneman’s words: “True intuitive expertise is learned from prolonged experience with good feedback on mistakes.”

If this is true, it appears to contradict another of Kahneman’s views: “The confidence we experience as we make a judgment is not a reasoned evaluation of the probability that it is right.”

If a man plays baseball for years and has an excellent batting average, he has the right to feel confident in his ability to hit a baseball. If he feels confident then he wants to be at bat when the game is on the line. His confidence is not based on a story; it is based on his record.

If he has never played baseball or has a low batting average, and feels confident when he steps up to the plate, we would say that his confidence is an illusion.

If I am to decide which of two young baseball players is going to become a star, I have, if I am a scout, a basis for judgment. If I have, over the years, chosen some talented ballplayers, then I have reason for my confidence. Experience will tell me what to look for and what to look away from.

Confidence is not a constant; it fluctuates according to experience.

When you are dealing with a young person, someone who, by definition, lacks experience, confidence cannot have been formed from experience. Young people are always going to be over or under confident.

If they are, as we would like, overconfident, they will revise and build on their overconfidence through their experiences. They will therefore move from overconfidence to true confidence.

Kahneman’s observations lead to another question. Is it good or bad to feel confident in the face of adversity or in the face of evidence that seems to be telling you that your confidence is misplaced?

What is perseverance if not the ability to maintain your confidence when the evidence seems to suggest that your efforts are futile?

How often do you need to fail before you discover that your confidence is mistaken? Is it rational to give up after one try or after two tries? Is there an optimal number of times that will tell you, for example, that you need to find a better career path.

If something doesn’t work the first time, does that necessarily mean that it will never work? Is it rational to abandon your confidence in the face of a setback? If the British Army has devised a way of doing something, and has devised it over decades and maybe centuries, is it rational or irrational to throw it out because it did not work the one time you tried it?