An Elusive and Judgmental Reality

If scientists are having a problem, what hope is there for the rest of us?

According to Jonathan Lehrer, scientists have a great deal of difficulty dealing with data that does not confirm their theories or hypotheses. They do not embrace it; they ignore it or wish it away.

Link here.

The rest of us, as Lehrer writes, consider ourselves to be empiricists. We pretend that our decisions are guided by facts, not fictions. We believe that trial-and-error is the best approach to life decisions. We consider ourselves rational individuals, shielded from the temptations of emotion, conducting our lives according to the same principles that scientists use when testing their hypotheses.

Apparently this is not the case, either among scientists or among the rest of us.

In Lehrer's words: "The fact is, we carefully edit our reality, searching for data that confirm what we already believe. Although we pretend we're empiricists-- our views dictated by nothing but the facts-- we're actually blinkered, especially when it comes to information that contradicts our theories. The problem with science, then, isn't that most experiments fail-- it's that most failures are ignored."

As I said, we all believe that we think rationally and evaluate our decisions in relation to the evidence of their success or failure. We apply these principles to the way we bring up children, choose lovers, pursue careers, or trade the markets.

Much of the time, however, we are fooling ourselves. We are just as likely, perhaps more likely, to double down on a bad decision than to change our minds.

If scientists are blind to anomalous data, so too are lovers. When humans fall in love they become especially oblivious to any data that contradict their plans, dreams, or feelings. Most especially they are blind to anything that would challenge their idealized judgment of their lover.

Strong feelings seem to trump all other considerations. It's as though we assume once we have been overwhelmed by a strong passion, we are necesarily in touch with a higher truth, one that will always trump facts.

But we do not just fall in love with other people. We also fall in love with our own ideas, our own theories, and our own beliefs. Here again the moment of inspiration feels so good and so right that we allow it to trump any facts that seem to contradict it.

It gets worse. Once we have become convinced of a lover's consummate wonderfulness or an idea's ultimate truth, we invest in them. We invest time and energy in our lovers and ideas, we become publicly identified by our investments.

Once we have altered our life to bring a lover into it or alter our life plan in order to make it conform to a new idea we have a stake in being right. This makes us less willing or able to admit of error, to accept the verdict of reality.

Often enough we become so invested in a bad relationship or a bad idea that we get angry at anyone or anything that would cause us to doubt it.

Among scientists falling in love with an idea seems to be less important than another influence: peer group pressure. As Lehrer explains it, scientists often ignore non-confirming data because they belong to a culture or a department or a fraternity that has invested in a specific theory. When you are part of a group you try to be in mental harmony with other members of the group. Not only do you want to think as they think, but you feel honor bound to uphold the group's values and beliefs.

If you discover data that threatens your theory, you are putting yourself at odds with a group in which you have status and prestige. How many of us, in those circumstances, would sacrifice status and prestige for data that may or may not be conclusive?

Scientists with academic stature come to have what Lehrer calls "a vested interest in the status quo."

I would add that they also have some respect for tradition. While the will to conform inhibits new scientific discovery, it also makes for a more orderly field of study. The truths of past science might be superceded by new work, but they still deserve some respect.

Lehrer offers some advice about how people can become more open to new data and new evidence. After all, it is possible that the theory is wrong, but it is also possible that the data is wrong. How can anyone know?

For academics in a department, the solution involves breaking out of the closed world of their department, and discussing the problem with people from other disciplines. If people in the chemistry department veer toward a form of academic groupthink, they can best solve their research problems by conversing with biologists and medical professionals.

Once the study group has been expanded to include people who are unfamiliar with departmental jargon, everyone will have to reformulate their thoughts and rephrase their issues so that outsiders can understand them.

Such reformulations, occurring in conversation, often clarify issues and advance research. As Lehrer points out, such discussions are far more effective than going it alone.

In some ways this same point applies to the practice of life coaching, especially as it concerns business coaching. How can a business coach help someone when the coach does not have a background in business?

Applying the same principle, we can say that when a businessman has to explain his business to an outsider he will have to articulate details and assumptions that he might have taken for granted or taken on faith, because they are the lingua franca of his business, but which sound a lot less valid once they are presented to an outsider.

Compare this with the practice of psychoanalysis. For those few who still imagine that psychoanalysis has something to do with science, consider this: in psychoanalytic treatment the analyst presents a theory or interpretation. The patient is then required to produce confirming material, and to persuade the analyst that this material has persuaded him of the truth of the analyst's interpretation.

The psychoanalyst's theory is not subject to trial-and-error. If the patient discovers material that seems to contradict the interpretation the patient is labelled resistant, neurotic, and in need of much more treatment.