Foxes and Hedgehogs

I'd started talking about Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here. He is very controversial right now, immediately before the election, because of his prediction of a much higher chance of an Obama victory than the headline polls indicate. So he will be seen by many in partisan terms.

The book is not partisan, however, so it deserves a fair look by people who might fight over his NY Times articles. After all, Silver argues for skepticism about expert predictions as a general rule.

I've mentioned Philip Tetlock's work on political prediction before. Tetlock found that most prediction by political or international relations experts is terrible. As Silver puts it,

Tetlock’s conclusion was damning. The experts in his survey—regardless of their occupation, experience, or subfield—had done barely any better than random chance, and they had done worse than even rudimentary statistical methods at predicting future political events. They were grossly overconfident and terrible at calculating probabilities: about 15 percent of events that they claimed had no chance of occurring in fact happened, while about 25 percent of those that they said were absolutely sure things in fact failed to occur. It didn’t matter whether the experts were making predictions about economics, domestic politics, or international affairs; their judgment was equally bad across the board.

But Tetlock also found one distinction that pointed to more successful forecasts. Isaiah Berlin had revived the ancient distinction between hedgehogs, who know one big thing, and foxes, who know many small things. I've mentioned it many times. Silver puts it nicely:

Hedgehogs are type A personalities who believe in Big Ideas—in governing principles about the world that behave as though they were physical laws and undergird virtually every interaction in society. Think Karl Marx and class struggle, or Sigmund Freud and the unconscious. Or Malcolm Gladwell and the “tipping point.” Foxes, on the other hand, are scrappy creatures who believe in a plethora of little ideas and in taking a multitude of approaches toward a problem. They tend to be more tolerant of nuance, uncertainty, complexity, and dissenting opinion. If hedgehogs are hunters, always looking out for the big kill, then foxes are gatherers. Foxes, Tetlock found, are considerably better at forecasting than hedgehogs.

Hedgehogs have more trouble seeing what is there without predispositions.

Foxes may have emphatic convictions about the way the world ought to be. But they can usually separate that from their analysis of the way that the world actually is and how it is likely to be in the near future. Hedgehogs, by contrast, have more trouble distinguishing their rooting interest from their analysis. Instead, in Tetlock’s words, they create “a blurry fusion between facts and values all lumped together.” They take a prejudicial view toward the evidence, seeing what they want to see and not what is really there.

Hedgehogs are vey good at coming up with stories and narratives that validate their positions - and turn out to be wrong.

The foxy forecaster recognizes the limitations that human judgment imposes in predicting the world’s course. Knowing those limits can help her to get a few more predictions right.

I am, as you might imagine from reading this wide-ranging blog, a fox to the core. I like to take ideas from different disciplines, and recombine and synthesize. So of course I like this argument. Hedgehogs often suffer from serious blind spots, and are prone to fanaticism.

One issue I haven't seen addressed anywhere, though, is what makes people foxes or hedgehogs. Part of it must be personality, although it's difficult to see a direct link to the big five theories of personality. There is some evidence that liberals tend to have higher openness to experience, but in my experience liberals are if anything more prone to ideological narrowness than conservatives. Conservatives often have a skepticism about systems and experts and theory which might make at least some less prone to hedgehog temptation. But one can find ideologues and zealots across the political spectrum. It is a sensibility rather than a particular political conviction.

Some of it must be a matter of education and styles of learning. And some must be a reflection of the incentives we set up. As Tetlock says, you are likely to be a more successful TV pundit by making overconfident big pronouncements than being nuanced.

In any case, Silver's point is that hedgehogs tend to be worse at prediction, not necessarily worse in general. Some of the most gifted people are hedgehogs by nature. As I've said before, Plato was a hedgehog, Aristotle a fox ( which may explain why I've become so interested in Aristotle). Dante was a hedgehog, Shakespeare a fox. Systems have their place. But we do need a feel as a society for the boundaries and limits and clashes of systems.

People like to believe in more certainty than there really is.