Overconfidence Goeth Before a Fall

What would we do without psychologists?

Nobel prize winner Daniel Kahneman is arguably the most influential psychologist in the world today. So people grant credence to his views… justifiably so.

In an interview with the Guardian Kahneman explains that if he could rid the human species of any bias, any flaw in the human decision-making process, he would abolish overconfidence.

The Guardian explains:

Not even he believes that the various flaws that bedevil decision-making can be successfully corrected. The most damaging of these is overconfidence: the kind of optimism that leads governments to believe that wars are quickly winnable and capital projects will come in on budget despite statistics predicting exactly the opposite. It is the bias he says he would most like to eliminate if he had a magic wand. But it “is built so deeply into the structure of the mind that you couldn’t change it without changing many other things”.

Naturally, one asks oneself where one might have heard this before. And the answer comes quickly to mind from the book of Proverbs:

Pride goeth before destruction, and an haughty spirit before a fall.

Better it is to be of an humble spirit with the lowly, than to divide the spoil with the proud.

What Kahneman and other psychologists call overconfidence bears a striking resemblance to the deadliest of the seven deadly sins: the sin of what is usually called pride. It might even be considered to be too much of a good thing.

Overconfidence has many faces and many facets. As a rule it refers to states of mind where one’s beliefs about one’s abilities and achievements far surpass one’s actual abilities and achievements.

One suspects that those who are hawking self-esteem, to the point where every child receives a trophy, regardless of ability or achievement, are building overconfidence.

One might note, among the curiosities of human language, that the word overconfidence does not have a correlate: underconfidence. Yet, some people are excessively humble, excessively lacking in confidence, to the point where they are unwilling or unable to take any risk.

Psychologists demonstrate the prevalence of overconfidence by asking students how well they have done on tests. Said students seem invariably to overestimate their scores, by a significant factor. Of course, one should ask whether the same students, had they been suffering from less self-esteem would have done better or worse. A student who lacks sufficient confidence might test poorly... because of a pessimistic attitude. But, an overconfident student might more easily breeze through the exam and not put in enough work. A student who is less confident might take more time and might revise his answers more carefully.

Excessive self-esteem seems to be a way of avoiding a larger expenditure of effort. It is antithetical to the work ethic.

The same applies when a professional asserts expertise. 

In Kahneman’s words:

Overconfident professionals sincerely believe they have expertise, act as experts and look like experts. You will have to struggle to remind yourself that they may be in the grip of an illusion.

Surely, there is truth to the assertion. I am confident that we place far too much credence in the views of so-called experts. At times, we expect that experts can accurately predict the future. Since no one really knows the future, in many cases, when it comes to predicting the GDP or the stock market, they are wrong. And one knows that experts in childhood development often give advice that is driven more by ideology than by what is best for this or that child.

One might count oneself as an expert because of training or experience or degrees. But, do some experts believe so strongly in their expertise that they refuse to accept any outcome that makes them look less expert?

To take an example, how many experts in climate science believe that they know, to a certainty, what the climate will be a century from now?

On the other hand, young professionals often overestimate their expertise. This might be normal when they do not have enough experience. But also, what sometimes looks like arrogance can be a motivating factor or a sales pitch. If you cannot persuade yourself and others that you are better than you are, you will often not get the chance to improve.

How are these young people going to gain experience and expertise if they do not allow themselves to undertake tasks that are slightly beyond their expertise?

Some of Kahenman’s examples involve predicting the future. Regardless of what is and is not built into the structure of the human mind, one’s ability to undertake a difficult task, depends at the least on confidence, and at times on overconfidence. A government may need to fight a war and might know that the public will never support the undertaking if it does not believe that the war can be won, and quickly. If the government's representatives are to communicate effectively, perhaps they need to convince themselves that the war can be won quickly and easily.

As for the capital projects that may or may not come in on budget, the need for the project and the need to motivate people to do the job might cause them to offer too optimistic an appraisal of its cost.

But if a boss exhibits too much confidence, even to the point of seeming to be detached from reality, he will lose the confidence of his team.

And then, if a coach at halftime exhorts his team to go out and win the game because he knows that they can do it, does a loss cause them to think that the coach was lying to them? 

Within the language game that is coaching, it is understood that certain ideas are intended to motivate and to produce an outcome, not to predict it.

I suspect that there is no such thing as confidence that is perfectly attuned to an unknown outcome. Overconfidence might be a psychological strategy to engage one in an activity that might be more necessary than desirable. 

Consider some other examples from the research, summarized by Wikipedia:

If plaintiffs and defendants were prone to believe that they were more deserving, fair, and righteous than their legal opponents, that could help account for the persistence of inefficient enduring legal disputes.^[26] If corporations and unions were prone to believe that they were stronger and more justified than the other side, that could contribute to their willingness to endure labor strikes.^[27] If nations were prone to believe that their militaries were stronger than were those of other nations, that could explain their willingness to go to war.^[28]

Some of these seem to be a reach. When deciding whether or not to sue or to settle a lawsuit, lawyers and their clients often calculate risk and reward. I am not confident that overconfidence is as prevalent as one might think.

On the other hand, some criminal defendants are so overconfident in their cases that they refuse plea deals because they believe that they will be acquitted at trial. The name of Martha Stewart comes to mind.

And of course, there is the case of Jarndyce vs. Jarndyce, from Dickens’ Bleak House. That case seemed to have a mind and momentum of its own... it went on and on and on.

As for the strength or weakness of one’s armies, it is clear that the Estonian army, such as it is, is considerably weaker than the Russian army. But, beyond the obvious gross disparities, how do you know whether your army is weaker or stronger until you have tested it in battle. And also, sometimes stronger armies have had great difficulty fighting against counterinsurgencies led by forces that are considerably weaker? There is a lot more to choosing to fight wars than an estimation of the strength of one's military.

We decided to fight in Iraq in 2003. We decided not to fight in Syria or Iraq after 2011. Which was the better choice? Which produced the better outcome?

And then, there’s the question of how one presents oneself to the public at large. If a politician, Wikipedia suggests, projects confidence, or even overconfidence, people are more likely to vote for him. In this case, it sometimes appears that the more the better:

For instance, those most likely to have the courage to start a new business are those who most overplace their abilities relative to those of other potential entrants. And if voters find confident leaders more credible, then contenders for leadership learn that they should express more confidence than their opponents in order to win election.

Of course, Barack Obama projected and continues to project a confidence in his ability to do his job. By all indications, he cannot. Yet, many people in the country are happy to affirm his overconfidence and protect him from the harsh judgment of reality.

And, Donald Trump, who seems lately to have cornered the market in overconfidence has managed to persuade a large number of people that he can just step into the oval office and do the job of president. You might say it’s a giant bluff and you might say—as I would—that it’s filled the Republican presidential nominating process with some rather juvenile showboating, but for now Trump seems to have attracted a considerable support.

Sometimes one feels that the Obama presidency has produced so much mind numbness that it believes overconfidence is true confidence. Trump and his supporters do not seem to understand that success in business, such as it is, does not mean success in all endeavors, great and small. It's like saying that because you have mastered the art of driving a car you will naturally know how to pilot a plane.

Watch out below.