Blaming a whale (and narrative) for defeat

From a David Brooks exchange with Gail Collins in the NYT:

David: This might be a good time for Republicans to redouble their commitment to the reality-based community. Did you see Byron York's reporting from inside the Romney campaign? They apparently had this giant computer model called Orca - named after a whale because it was bigger than anything the Democrats could imagine. It processed huge amounts of data and late in the day was still projecting a Romney victory until its head exploded. Garbage in. Garbage out.

Gail: The Republican Orca - stop me before I fall into a great pile of Moby Dick analogies.

I think that some people are a little transfixed by Nate Silver's accurate projection of the election, despite, as I've noted, his own much more self-awade mature grasp of the limits of models.

I think this is at root a particular problem with journalism.

Or, to put it more precisely, it is a problem with narrative. Human beings have a hardwired fascination with stories, probably dating from sitting around fires in the savannah fifty thousand years ago. It is often the prime way we transmit culture and values and pointers for behavior we should admire.

But stories can be too trite, and this is where journalism becomes very hedgehog-like sometimes. It is all to easy to link stories into an appealing broader narrative, to instinctively frame things in a way which makes for an entertaining read. Stories are much more interesting than data. Once you go looking for "stories", as opposed to drier mechanical reporting of facts, you introduce potential blindspots. Things can fit all too neatly into narrative buckets.

Journalism thrives on stories. Journalists are paid to look for them, rather than dry, ambiguous, complicated truth in isolation.

We've also seen that hedgehogs typically get more media attention, because big striking claims typically make better tv or better stories. Hedgehogs want an audience for their "one big thing." Journalists want a story, And "triumph of models" is itself a good story.

Of course, at the same time as the fivethirtyeight model did well, the Romney and other models performed badly. But why let that get in the way of a good story?

Self-awareness

This is the real explanation of Silver's greater success here. It is not so much a data-driven approach, as using the data to confront your presuppositions. That is why he does better than most journalists and opinion pundits.

It is not, as the story linked above puts it, simply a matter of primitive punditry against spanking new "data driven rationality".

The scientific method is at root about testing hypotheses, not mining data. You test a prediction or explanation against reality, and you revise your views if necessary based on the outcome. You can selectively use data to confirm all kinds of things if you are not careful. So you need to have a falsifiable hypothesis, a situation where at least in principle you may be forced to revise your view.

There's nothing in narrative that compels you to change your view, because you can always add a twist in the story. Indeed, most good stories will have the hero endure many setbacks and misunderstandings before being proved right in the end.

Even in science, as Thomas Kuhn famously pointed out in The Structure of Scientific Revolutions, conflicting data is often left aside as a "puzzle" or "anomaly" in periods of normal science until there is a sudden change of awareness: I.e. the "paradigm shift" which has become so overused a term it is almost a clichė. Even in the hard sciences, it can take a generation for data to settle arguments one way or another. Theories can sprout dozens of ad-hoc adjustments.

So data can be part of this process of self-awareness. But in a world where we have a blizzard of often conflicting and imperfect data, it may equally reinforce entrenched views.

Romney's big Moby Dick model was probably highly sophisticated, but likely being used to mine data more efficiently, not testing against reality.

Innumerate journalists may regard the statistical models as some sort of new semi-deity, partly because they don't understand it, or the many ways in which statistics can be used to obscure. (There is a famous book called How to Lie with Statistics).

But the real message is not statistics as some kind of new rational technique which guarantees prophetic success. Instead, it is self-awareness.


You need devices to help you retain self-awareness, to see things as they really are instead of ever-more elaborate stories or models. And humans typically find that very hard to do. I've got a few more things to say about Silver's book in that light.