It’s safe to say we’re paying the price for ignoring the experts, or at least listening only to the wrong ones, a phenomenon I find endlessly interesting. So when I saw Dan Gardner’s Future Babble in Company of Books earlier, I didn’t hesitate to pick it up – Gardner’s Risk, about our pathetic inability to accurately judge what might harm us, is one of the best non-fiction books I’ve ever read, and I can’t wait to see how he dismantles the credibility of well-paid prognosticators.
I wrote a Present Tense column about this phenomenon two years ago, and it got a great reaction – people love seeing phoney expertise being disparaged, I guess. John Gruber linked to it at Daring Fireball, the closest I’ve come to internet royalty, and about the most encouraging praise I’ve received in my writing career, so here it is again. And while the Manchester United – Barcelona reference might seem right up-to-the-minute fresh, it just goes to show how football analysts might be able to recall lots of historical details, but the lessons of the past seem lost on them; there was no way United were capable of giving Barca a good game in either match. There, that’s my predictive skill retrospectively proven.
“UnHumble Opinions”, from The Irish Times, Saturday June 20th, 2009
It is an often-noted but rarely understood phenomenon: the significant number of people who manage to earn a living, often a handsome living, by being wrong. Not always wrong, necessarily, but wrong enough, frequently enough, to warrant the description wrong-headed. We see evidence of it everywhere, from those sporting commentators who consistently struggle to accurately predict a result (“Manchester United have way too much for this Barcelona team” – often heard in May 2009) to the financial advisers whose advice proved very costly indeed (“You can’t go wrong by investing in property and bank shares” – often heard from 1998-2007).
Why, one might reasonably ask, are so many of these people still paid to offer their opinions or advice, despite having a demonstrably flawed track record? Indeed, how did they get their jobs in the first place? Well here’s the science bit – recent research by Prof Don Moore of Carnegie Mellon University in Pittsburgh, Pennsylvania, shows that given the choice between two people offering advice, we’ll tend to go with the guidance offered by the more confident source, even if their track record is less reliable. At a symposium in San Francisco last month, quite brilliantly called “Often in Error, Rarely in Doubt”, Moore gave a paper describing how “When advisers must compete with each other, we find that the confidence they claim escalates over time. This overprecision helps advisers’ sell their advice.”
This goes some way towards explaining why a characteristic shared by many of these wrong-headed “experts” is an overweening self-confidence. All those brass-necked pundits whose self-assurance seems to be inversely proportional to their wisdom are merely the inevitable result of free-markets, because selling certainty is easier than selling nuanced opinions that embrace complexity and doubt.
At first, this quirk of human psychology sounds like one of those interesting nuggets of pop-science, equivalent to the way we like symmetrical faces, say, until you consider the cumulative cost of this cognitive error. If the result of this psychological quirk was restricted to overpaid football commentators embarrassing themselves before a Champions League final, then no harm done. Unfortunately, the implications are rather more profound, and dangerous.
We are, it need hardly be repeated, in the middle of a serious recession, the causes of which are manifold, with plenty of blame to go around. But the ill-fated property bubble that is responsible for most of the current damage was dependent on property market cheerleaders being given more credence by buyers than property market sceptics. Admittedly, experts such as David McWilliams, George Lee and Morgan Kelly, say, were vocal about their doubts, but the voluble ranks of property market supporters, from developers to banking sector economists, from media personalities to leading politicians, were more emphatic, more strident and more confident in their assertion that we had a fundamentally sound property market and economy. They were selling their certainty, while the sceptics were selling their doubt – no contest.
The situation in the US was even more polarised, with the likes of Jim Cramer, the notoriously pugnacious business commentator, relentlessly talking up the markets on his cable TV show. Anybody with an alternative viewpoint, such as Nassim Nicholas Taleb or Nouriel Roubini, weren’t invited to make regular TV appearances to air their opinions.
Far and away the most dangerous example of the over-confident “expert” was readily apparent in the run-up to the Iraq War, when the chorus of voices advocating an attack on Saddam Hussein drowned out the saner voices of fact-based reason. While the line between mistaken and mendacious can be hard to discern, it was clear that the decision-making of everyone from Dubya to Cheney to Wolfowitz to Rumsfeld (whose “known unknowns” speech can be considered a window into the mind of the serially overconfident) was simultaneously egregious and incompetent. The world was faced with the dire consequences of what Prof Moore calls an “excessive confidence in the precision of one’s knowledge”. Eventually, of course, the American people tired of the faulty rhetoric, and voted in a man whose self-confidence appears to be based on actual ability, rather than imagined expertise.
But the administration also relied on an echo chamber of sympathetic voices to make their case for them, voices that also demonstrated an unfailing self-belief. A revealing example is this quote from US neocon Bill Kristol, from an article he penned in March 2003, just before the invasion of Iraq. “The war itself will clarify who was right and who was wrong about weapons of mass destruction…History and reality are about to weigh in, and we are inclined simply to let them render their verdicts.” Given that reality weighed in and history quickly ridiculed Kristol and his hubris, one would think that he might have found himself out of work, like the administration he persistently praised. Instead, Kristol got a gig writing an opinion column for Time Magazine, and then another job writing an opinion column for the New York Times. And not, so far as I can tell, in an ironic way. The triumph of the competent over the confident, it would seem, is far from assured.