Maps, not models, by Mapper

We're looking at Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here.

There's one stray reference in the book, which caught my eye, because I've been interested for over a decade in the difference between maps and models as ways to understand the world. I call myself Mapper on this blog for a reason.

Silver says:

The Finnish scientist Hanna Kokko likens building a statistical or predictive model to drawing a map. It needs to contain enough detail to be helpful and do an honest job of representing the underlying landscape—you don’t want to leave out large cities, prominent rivers and mountain ranges, or major highways. Too much detail, however, can be overwhelming to the traveler, causing him to lose his way. As we saw in chapter 5 these problems are not purely aesthetic. Needlessly complicated models may fit the noise in a problem rather than the signal, doing a poor job of replicating its underlying structure and causing predictions to be worse. But how much detail is too much—or too little? Cartography takes a lifetime to master and combines elements of both art and science. It probably goes too far to describe model building as an art form, but it does require a lot of judgment. Ideally, however, questions like Kokko’s can be answered empirically. Is the model working? If not, it might be time for a different level of resolution.

I hadn't head of Kokko before, but I thought this was interesting. The reason I find the topic so fascinating is that I have sat many times in meetings with senior economic officials - or even more often, their more academic staff - who argue that to think clearly about a situation you need a model in your head, if only to ensure consistency. For most economists, this tends to be linked to some version of Milton Friedman's as-if methodology, which says that the test of a model is not the realism of its assumptions, but the accuracy of its predictions. Simplification and abstraction from reality is what constitutes explanation, and anything else is mere muddle.

Of course, this automatically counts out the value of studying history, which has been largely excluded from the mainstream of education in the discipline for fifty years (despite some good work). Mathematical models have reigned supreme. Optimization subject to constraints is the center of thinking. It is far narrower than even "theory", as it imposes essentially aesthetic constraints on what a theory should be, I.e. mathematically elegant.

It also is dangerously overreliant on consistency. You need to be consistent to be fully rational in your choices, of course, in a narrow sense. But consistency is no guarantee of truth. You can be consistently wrong. And many actual problems are about reconciling different objectives or opposing views. Compromise in this sense is bound to be somewhat inconsistent with someone's basic principles. It is also real life.

However, the most important point is the economic modeling perspective is also a very naive way to think about useful abstraction. A map is an abstraction too, but it tends to be much more useful for most purposes than a mathematical model. Many policy problems are in fact much more like "how to get from A to B" than " maximize X subject to Y".

In particular, maps are much more focused on specific purposes. If I want to get from New York to Albany, I look at a standard road atlas or Google maps. If I want to see where shale gas deposits may lie, I want a geological map of New York State. If I want to sail up the Hudson to Albany, I want a chart which shows sandbanks and shoals and traffic lanes in the river.

In other words, purpose is much more intrinsically present in a map than a mathematical model using aggregate economic statistics or optimization assumptions.

Add to that, as Kokko says, the scale and resolution which are inherent to maps. You abstract away what you don't need. He is right here, but both Silver and Kokko are wrong in a larger sense. The point is not that there is an art to building models so they are , by analogy, a little more like maps with a correct level of detail. It is that maps and models are wholly different ways to understand reality.

Maps are much more suited to seeing risks. The kind of generalized abstraction in a model will not help you avoid the specific rock which is just below the surface as you enter harbor, but the right chart will. A map will show you the massive mountain chain or desert or unfordable river in your way. It will show the paths and cliffs and hazards. A model won't.

So thinking in terms of surveying a specific landscape for specific important features for a specific purpose is just as disciplined an intellectual exercise as developing an abstract model to predict, and much more useful. It can also be a much better predictor of some kinds of problem (eg how long it will take to get to Albany.)


Maps help you see what is actually there, rather than "explain" it in some ultimate universal way. We most often don't need a general theory of roads. We just want to find our way home. Models will predict, but maps will show you the right way. If you want to climb a mountain you are better off bringing a topo map than an abstract mathematical model of paths.

Silver's book is all about prediction, but the more general human problem in decision-making is which way do I go? What is the right direction? Where are the hazards? We need maps more than universalized explanations in most situations. We need a survey of the actual landscape with a particular purpose in mind.

Ways of seeing what is really there are the main way we will develop and make progress. Too much modelling often prevents us from doing that, by distracting people towards the mathematically elegant and tractable, the thin universal rather than the thick description of specifics, and the quantifiable and obvious rather than the danger that lurks in the details.