Happiness versus Meaning

Here's an interesting article in the Atlantic. New research confirms what Victor Frankl famously wrote about following his experience in Nazi concentration camps: a sense of purpose is deeply necessary to people, even more so than immediate happiness.

Research has shown that having purpose and meaning in life increases overall well-being and life satisfaction, improves mental and physical health, enhances resiliency, enhances self-esteem, and decreases the chances of depression. On top of that, the single-minded pursuit of happiness is ironically leaving people less happy, according to recent research. "It is the very pursuit of happiness," Frankl knew, "that thwarts happiness."

This is why some researchers are cautioning against the pursuit of mere happiness. In a new study, which will be published this year in a forthcoming issue of the Journal of Positive Psychology, psychological scientists asked nearly 400 Americans aged 18 to 78 whether they thought their lives were meaningful and/or happy. Examining their self-reported attitudes toward meaning, happiness, and many other variables -- like stress levels, spending patterns, and having children -- over a month-long period, the researchers found that a meaningful life and happy life overlap in certain ways, but are ultimately very different. Leading a happy life, the psychologists found, is associated with being a "taker" while leading a meaningful life corresponds with being a "giver."

"Happiness without meaning characterizes a relatively shallow, self-absorbed or even selfish life, in which things go well, needs and desire are easily satisfied, and difficult or taxing entanglements are avoided," the authors write.

How do the happy life and the meaningful life differ? Happiness, they found, is about feeling good. Specifically, the researchers found that people who are happy tend to think that life is easy, they are in good physical health, and they are able to buy the things that they need and want. While not having enough money decreases how happy and meaningful you consider your life to be, it has a much greater impact on happiness. The happy life is also defined by a lack of stress or worry.

Most importantly from a social perspective, the pursuit of happiness is associated with selfish behavior -- being, as mentioned, a "taker" rather than a "giver."

People want to contribute to something.

Baumeister and his colleagues would agree that the pursuit of meaning is what makes human beings uniquely human. By putting aside our selfish interests to serve someone or something larger than ourselves -- by devoting our lives to "giving" rather than "taking" -- we are not only expressing our fundamental humanity, but are also acknowledging that that there is more to the good life than the pursuit of simple happiness.

It's worth reading the whole article.

 

More signs of a renaissance in psychology

This is an interesting column from David Brooks in the NYT, mostly because he sees many of the main challenges in public policy as a matter of better psychology.

What about the big problems? How do we get people to restrain government commitments now so that debt down the road won’t be so ruinous? How do we calculate the multiplier effects of tax cuts or spending increases among different subgroups of the population, or under different emotional conditions? How do we rig the context of budget negotiations so participants can actually come to a deal? How are people in different cultures likely to react to drone strikes? How do we structure sanctions against Iran to cause the greatest psychic humiliation?

These are the big questions, and most of our policies rely on crude folk psychology from a few politicians. But there’s hope. As Brian Wansink notes in Eldar Shafir’s volume, the 20th century saw great gains in sanitation and public health. The 21st century could be a great period for behavior change.

No mention of economics at all. Of course, economics has some things to say about credibility and commitment, such as the Kydland/Presscott time inconsistency literature. But the rational actor approach appears to be yielding diminishing returns. The focus of the social sciences can shift over time. I've been seeing more and more interesting things in psychology recently.

 

"Sure, Big Data Is Great. But So Is Intuition"

An NYT article is skeptical of inflated claims for Big Data:

The quest to draw useful insights from business measurements is nothing new. Big Data is a descendant of Frederick Winslow Taylor’s “scientific management” of more than a century ago. Taylor’s instrument of measurement was the stopwatch, timing and monitoring a worker’s every movement. Taylor and his acolytes used these time-and-motion studies to redesign work for maximum efficiency. The excesses of this approach would become satirical grist for Charlie Chaplin’s “Modern Times.” The enthusiasm for quantitative methods has waxed and waned ever since.

Big Data proponents point to the Internet for examples of triumphant data businesses, notably Google. But many of the Big Data techniques of math modeling, predictive algorithms and artificial intelligence software were first widely applied on Wall Street.

At the M.I.T. conference, a panel was asked to cite examples of big failures in Big Data. No one could really think of any. Soon after, though, Roberto Rigobon could barely contain himself as he took to the stage. Mr. Rigobon, a professor at M.I.T.’s Sloan School of Management, said that the financial crisis certainly humbled the data hounds. “Hedge funds failed all over the world,” he said.

The problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according to the laws of physics.

In so many Big Data applications, a math model attaches a crisp number to human behavior, interests and preferences. The peril of that approach, as in finance, was the subject of a recent book by Emanuel Derman, a former quant at Goldman Sachs and now a professor at Columbia University. Its title is “Models. Behaving. Badly.”

It really is a matter of proper scope, and consciousness of limits. Big Data is wonderful for finding the Higgs Boson among billions of particle paths, or tracking potential credit card fraud. It is not so good at many other tasks where the data is absent or incomplete or misleading. In those cases, it is little different from ancient farmers looking at the sky and seeing mythical animal patterns.

The Art of Choosing

I read Sheena Iyengar's The Art of Choosing last week. It is a stimulating and rewarding book about recent psychological investigation, often by Iyengar herself, into choice. She is a Professor at Columbia, and her work seems another prime example of the renaissance of psychology has a discipline in the last ten-twenty years. We looked, for example, at Martin Seligman's work here, and Daniel Kahneman has had a major impact on economics.

Iyengar is more interested in choice than happiness or heuristics, however.

Our deep need for choice

First, she emphasizes the profound importance of a feeling of agency, of having some control, to almost all creatures. Even zoo animals exhbit listlessness and distress if they do not have agency. One might think of a zoo as a paradise for wild animals, with food and material needs taken care of. Instead, this "perfect hotel" is anything but.

In spite of the dedication of their human caretakers, animals in zoos may feel caught in a death trap because they exercise minimal control over their lives. .. Due to these physically and psychologically harmful effects, captivity can often result in lower life expectancies despite objectively improved living conditions. Wild African elephants, for example, have an average life span of 56 years as compared to 17 year for zoo-born elephants. p11, 13

There are other examples. Lower ranks of the British civil service have worse health and die younger than the upper ranks, according to a famous piece of research, as they have less control over their daily tasks. Giving residents of an old people's home more control over trivial things - plants, movie screening times, TV - leads to significantly better outcomes in health and happiness.

This is fascinating. We need a minimal sense of control over our surroundings and workflow to be happy. A utopia that takes care of material needs but allows us no sense of control or agency is likely to be make us miserable.

A society built on keeping people in captivity is bound to be unhappy. People need to feel they have choice, or they shrivel. Material abundance alone is not enough to make people happy. It is more likely to be an existence like a captive zoo animal.

Cultural differences

However, cultures differ quite radically in how they think about choice. Most cultures emphasize collective needs and approval over individual needs. The United States, not unexpectedly, is at the individualist extreme.

She contrasts this with the SIkh wedding of her parents in India. All choices were made by the families and tradition. The first time the groom saw the bride was when he lifted her veil after the marriage ceremony.

These cultural predisposotions affects workplace behavior too. She studied worldwide employees of Citibank, carrying out the same tasks. They differed significantly in the amount of choice they perceived they had on their jobs, and how much they preferred management to exert.

 

"Freedom from" versus "freedom to"

She traces such cultural attitudes back to a more fundamental difference over "freedom from" versus "freedom to", which she says Erich Fromm postulated in his 1941 book Escape from Freedom (which I read as a teenager). "Freedom from", said Fromm, is "freedom from the political, economic and spiritual shackles that have bound men,", especially others interfering with the pursuit of our goals. "Freedom to" is an ability, the ability to "attain certain outcomes and realize our full potential."

A system tending too far in either direction can limit people's opportunities, Iyengar says. Capitalism tends towards "freedom from", but not everyone will have access to all the choices available. Communism tends towards "freedom to", including equality of outcomes rather than opportunities. The problem is there is little incentive to work harder or produce more, and it requires a strong government which may become corrupted by power.

She found strong differences still endure between West and East Berliners in her research, for example.

.. people's long-held assumptions about fairness can't simply be swapped for another set of beleifs. I consistently found that West Berliners, like Westerners in general, understand the world through the lens of "freedom from." On the other hand, East Berliners, and in particular the older people, focused on "freedom to", even though communism was now only a memory for them. p65

The American dream is all about "freedom from".

People who see themselves and others as having high personal control tend to favor "freedom from", not only because it provides more opportunities to attain their personal goals but also on grounds of justice - those who put in the most effort will be rewarded, while those who slack off won;t be able to ride on anyone else's coattials. p68

Americans as a whole arguably believe more wholeheartedly in "freedom from" than any other nation.. The basic premise is no one can stand in the way of your highest aspirations, provided that you have the ambition and skills to realize them. p70

This matters because it might mean potentially fundamental and irreconcilable cultural differences in what people believe about the good life, which is a primary interest of this blog. The good life in the Confucian tradtion would be very different from the good life in the Jeffersonian tradition, so perhaps it is better not to think about it at all.

Freedom to what?

I don't think that's true, though. The fact of abundance, the end of the struggle for material survival, is a challenge to every culture, whether one contronts it on an individual or social level. We face "freedom from" the traditional limits of material and social scarcity. That is a massive change from the lot of humanity for all the thousands of generations before we solved the "economic problem".

Since the 1960s, many of the traditional limits in society have been lifted. What have we achieved with it? TV and the welfare office.

We have abundance. Then the question is - what do we actually want? And "Freedom from" in isolation doesn't say anything at all about that. It suffers the traditional liberal blindspot about where preferences and tastes and wants and desires and pleasure actually come from.

How do people find what they want, especially when older cultural institutions like the churches are in decline? What cultural influences or standards influence what people want to do with their lives after it becomes genuinely a matter of choice? Iyengar doesn't go into this.

For example, we looked some time ago at economist Tibor Scitovsky's work. People want the right level of stimulation, he argues, but it has to be genuinely novel stimulation, not simply quantitatively more. How do we achieve a system where there is more genuine choice, rather than people running faster and faster on hamster wheels for more of the same?


But at the same time, simply accepting traditional or collective notions of what "freedom to" means seems alien to our tradition in the West, and at risk from technological and cultural change. There is an impoverished group of choices for "freedom to".

In the past, "freedom to" has often meant spoilt aristocrats fighting futile dramas of status and prestige and honor, or rich trust fund kids subsiding into drugs and nihilism. It has meant religious extremism and monastric austerity and self-denial. At present, it often means crystal meth in small towns and people thinking of the office cubicle as a second home. In practice, it can lead to alcoholism and divorce and loneliness and pursuit of celebrity. As John Stuart Mill once pointed out, much rests on the quality of what we want.


There is, in fact, a longer history to these ideas. "Freedom from" is analagous to "liberty", whereas "freedom to" is more consonant with older ideas of "freedom", where being free has a positive meaning. (This reminds me of one of David Hackett Fisher's books, Liberty and Freedom: A Visual History of America's Founding Ideas (America: A Cultural History)which I must look at again. ) America was at the forefront of liberty in the eighteenth century, with lingering social consequences. It ought to be at the forefront of what freedom means now.

Everyone has to answer the question of "what next?". Perhaps the Woodstock Festival or the ABC Fall Line-up or the Google free snackbar in their Silicon Valley offices is the highest point people will ever achieve. Perhaps not. We do not discuss what we should aspire to any more, beyond egailtarianism.

 

"Freedom to" is not necessarily egalitarian or traditionalist

Iyengar's conception - and the traditional view - of "freedom to" is too limited. "Freedom to" is not necessarily socialist or egalitarian. It is not necessarily about minimal provision for all, or Rawlsian maximin welfarism. After all, we've seen that Aristotle's account of the good life has been criticized since for being so unconcerned with equality.

Instead, I think this is an example of where we have taken such a wrong turn. As soon as any suggestion of material sufficiency or abundance has appeared, the first response has been equal distribution. It seems to set off an egalitarian mania, in terms of welfarism and rights and taxation. Yet egalitarianism is not the only vision of the good life, or indeed, a particularly inspiring or just one.

There are deep potential problems with "freedom to", particulalty if people are coerced into a state vision of what it means. But once "freedom from" has been achieved, the only alternative to more discussion of "freedom to" is an unthinking default socialism or nihilism. We need a new conception of "freedom to" that will also correspond to an answer to the conundrum of "value" in economics which we were discussing recently.

I think that has to be centered on people developing their skills and capacitiies to their full potential, and that has to be connected to a conception of virtue and the good life.

We'll look at more of the book tomorrow.

 

 

 

Walking with Cavemen

I watched the BBC-produced series Walking with Cavemen on Netflix last night. It's an interesting recreation of life as it must have been for early precursors of people, in the style of a wildlife documentary.

It starts with Lucy (on the left here), a young Australopithecus Afarensis female in her twenties. She was discovered 3.5 million years later by a French-American expedition in Ethiopia. It moves up through homo ergaster and homo habilis to the Neanderthals, and the eventual appearance of us.

A few things struck me about the series. The reason some varieties of human survived, it says - us - was adaptability. Our lineage was a jack of all trades, rather than being superbly adjusted to just one particular niche.

In the final episode, the series argues that homo sapiens eventually outdid the Neanderthals and other closely related species because of our capacity for imagination. It was a close-run thing, too. The species almost died out 75-100,000 years ago. There were as few homo sapiens in the whole world then as there are orangutangs now. Only the most adaptable of an inherently adaptable lineage made it through that chokepoint.

There is one scene in which two humans bury an ostrich egg filled with water in the ground, on the off chance that they will be passing that way again and need the water. No other living thing has the foresight to imagine and plan.

In a way, that is the origin of our whole idea of wealth as well. It is a side-artifact of our motivation to plan ahead and have choices in the future. It is probably connected to contingent aids to survival.

Of course, squirrels do the same when they store nuts for the winter, and more systematically. But perhaps the difference is that is predictable behavior for predictable outcomes, and pure instinct. For us, it is more a store of adaptability and choice and material survival, because for four million years has been our distinctive specialization.

One other thing becomes very evident. The reason for the evolution of the massive human brain is not so much dealing with the external environment, but the human environment: such as reading people's intentions, motivations, propensity to cooperate and form alliances. We can cooperate, and also drive each other crazy with office politics.

The physical world is much simpler and predictable than the human social world, and needs less brainpower. Bacteria can move towards opportunities. Cats are extremely good at instantaneously calculating where to pounce. Even we are good at solving the quite complex mathematics of the trajectory of a ball thrown high in the air so we can catch it, without being aware of it. It is hardwired into our brains, even if we would struggle with the formal mechanics of ballistic trajectories.

Understanding people is the most difficult task the brain has. And that is why it evolved to be so large, despite the huge demands in terms of energy.

Against Fairness?

I have often talked about fairness and impartiality on this blog, such as here and here. This philosopher, Stephen T. Asma, boldly argues we overestimate fairness in a new book, and writes in the Chronicle of Higher Education.

Favortism is not the same as prejudice, he says.

I want to argue something counterintuitive here. Contrary to all this received wisdom, open-mindedness is actually compatible with favoritism and bias. ...

These recent findings undermine the old assumption that favoritism automatically entails bigotry toward outgroups. Intergroup relationships and judgments, even among kids, are much more complex than we thought. ..

In short, favoritism or bias toward your group is not intrinsically racist, sexist, or closed-minded. Privileging your tribe does not render you negative or bigoted toward those outside your tribe. And to top it off, we're now beginning to understand the flexible nature of our ingroup favoritism—it doesn't have to be carved along bloodlines, or race lines, or ethnic lines. [..]

Young people in our schools are repeatedly exposed to a bogus association between unbiased equality for all and open-mindedness.

He argues you can favor your own without being inequitable or biased against others.

My favorites are not the best or most accomplished at this or that. They are not virtuoso human beings. It's my sheer affection for them, my ability to relate to them, and my history with them, that raise their status above other people. Love trumps fairness every time. It says: I don't care if other people are more deserving than you, you're mine and that's why I give you more than anyone else. Ethical philosophies of every stripe—egalitarian, utilitarian, Rawlsian, cosmopolitan—have tried to level people with a grid of uniform impartiality, but our favorites cannot be encapsulated in the grid. They loom too large in our moral geography.

So Asma here is arguing here that very basic human motivations, like treating your family or close friends as more deserving, still has some legitimacy. On one level, it is remarkable that one has to argue for it. It is hardly counterintuitive. We have let our ethical universalism get quite distant from common sense.

It is also very striking just how strong the effort to teach against "bias" has become, even in preschool. It seems the education establishment cares about little else.


It is fascinating - and beautifully observed - when he how he talks about the default instinct of kids to argue for what they want, selfishly, in terms of appeals to fairness. Adults are often no different. Invocations of fairness are often used as self-interested weapons.

What happens, as Asma notes, is that many of the people held up as heroes of impartiality, such as Susan B. Anthony or Rosa Parks, have actually been fighting for inclusion or the interests of their own in-group, not general impartiality. It may be justified, but it is rarely impartial. (And affirmative action is of course deliberately about favoring some groups, rather than impartiality. The last thing many on the left want is race-blindness.)

Many of the great achievements of the last few hundred years, such as reducing the prevalence of nepotism, rely on impersonal kinds of impartiality. Modern bureaucracy in essence relies on impartiality. But impartiality is not and cannot be the whole of morality, as I've argued before. An overemphasis on impartiality and neutrality causes much of the rest of our ethical instincts to shrivel up.

As I say, you may need a referee to play a game. But the game is not all just about the referee. You have to have some goals as well.

 

Aristotle: Household, Wealth, Common Property

We're talking about Aristotle's The Politics , starting here.

The household

The first few chapters, on the household, do seem like a relic of a distant 350 BC, and are best hurried through. It is hard to relate to the small farms or city households of ancient Athens or the Troad. This household management, oikonomia , is however the origin of the term "economic."

Most objectionately to modern readers, he believes some people lack the capacity for deliberative reason, so that they are slaves by nature ( although ancient slavery was not racially based). He also takes for granted strict division between the sexes. The head of the household must have moral virtue in its entirety; others, such as women, children, and slaves, in lesser degree.

He was perhaps more enlightened than many of his time , but he was of his time. He took for granted that, as a matter of practical necessity, some had to labor, not least because they were not suited to do anything else.

Of course, for us automation has removed much of the raw mechanical labor from life, although the economy still has an insatiable appetite for cheap labor in many agricultural and service industries. And we more optimistic about people's capacity for reason (although sometimes I wonder, if you see any episode of Access Hollywood or other celebrity glop).

What he says also entails one expects much more from the free, independent head of household in moral terms than children or servants or dependents. We still believe that of children, of course. Perhaps the capacity for moral virtue differs, and that is something we set aside in contemporary debate.

Wealth

He talks about the acquisition of property, and the definition of wealth. Wealth, properly thought of, is a tool:

Solon in one of his poems said "no bound is set on riches for men." But there is a limit, as in the other skills; for none of them have any tools which are unlimited in size or number, and wealth is a collection of tools for use in the administration of a household or state. (P 79 in the Penguin edition).

Of course, one can acquire goods without limit, but the function of the household is to use them.

The reason why some people get this notion into their heads may be that they are eager for life but not the good life; so, desire for life being unlimited, they desire an unlimited amount of what enables life to go on. Others again, while aiming at the good life, seek what is conducive to the pleasures of the body. So, as this too appears to depend on the possession of property, their whole activity centers to business. For where enjoyment consists in excess, men look for that skill which produces the excess that is enjoyed. (p85)

This of course is still a live political issue - is there a point where we have enough? How much is enough? (the title of a book I looked out a few weeks ago). It is also very much linked to questions of environmentalism. There is a very old tradition which sees the answer to human happiness not in prosperity and abundance but in limiting human desires. That ancient ascetic creed can surface today in those who want us to abandon growth and return to a significantly simpler lifestyle.

I think the answer is it is almost impossible to define what "enough" is, or what wealth itself is, without some conception of the good life and flourishing. Seeing wealth primarily as a tool for particular purposes, something to use and activate, rather than a pile of gold or financial assets is productive. It can be hard to transform financial assets into a flourishing, happy, secure life, as many celebrity divorcees or high-profile occupational burnouts know.

Aristotle also somewhat disapproves of trade and charging interest, "since it arises not from nature but from men's gaining from each other", views which were to still resonate late into the modern period, and underpinned nobility looking down on the merchant classes. We have seen this before, (including a quote from the Politics) in the deep suspicion of many religious and philosophical traditions of the institution of money.

But pragmatist that he is, he can have a shrewd appreciation of business. Thales of Miletus, he says, was criticized for making little money from philosophy. So one year he cornered the olive oil presses on his island just before a good harvest.

He made a lot of money, and so demonstrated that it is easy for philosophers to become rich, if they want to; but that is not their object in life. .. the principle can be applied more generally: the way to make money is to get, if you can, monopoly for yourself. (P90).

And that is also why we have to be very careful of monopolies and businsss restrictions and regulation sometimes.

It is also an illustration of how most ideas in the humanities and social sciences are rediscoveries or permutations of much older themes. Michael Porter would advise the way to profitability is to build barriers to entry (Competitive Strategy: Techniques for Analyzing Industries and Competitors). Warren Buffett looks for businesses with a "moat."

Against common property

Aristotle then turns to the state and comparative politics, looking at a number of constitutions including Athens, Sparta, Crete and Carthage.

He has a modern skepticism for Plato's notion of communal ownership or modern socialist property, not to mention sharing wives and children:

The greater the number of owners, the less respect for common property. People are much more careful of their personal possessions than of those owned communally; they exercise care over common property only insofar as they are personally affected. Other reasons apart, the thought that someone else is looking after it tends to make them careless of it. P108

That is also true for organizations, which is why assigning responsibility and accountability is often so important.

What was later called "to each according to his needs" is also met with skepticism by Aristotle.

For if the work done and the benefits accrued are equal, well and good; but if not, there will inevitably be ill-feeling between those who get a good income without doing much work and those who work harder but get no corresponding extra benefit. To live together and share in any human concern is hard enough to achieve at the best of times, and such a state of affairs makes it doubly hard. P114

This has a very contemporary ring about it, no doubt because it is such a timeless human response. So common ownership of property has inherent difficulties. At least, he says, existing laws are strengthened by familiarity.

Far better is the present system - provided that it has the added attraction of being a matter of habit and of being controlled by sound laws.

Even if you could fix a level of common possessions, and achieve absolute material equality,

to fix a moderate amount for all, that would still be no use: for it is more necessary to equalize appetites than possesions, and that can only be done by adequate education under the laws. ... And civil strife is caused by inequality in distinctions no less than inequality in property, though for opposite reasons on each side; that is to say, the many are incensed by the inequality in property, whereas more accomplished people are incensed if honors are shared equally, for then, as the tag has it, 'good and bad are held in equal esteem. p129

This is very important for my interests, as an obvious response to economic abundance is some kind of minimum or basic income. It is an essential illustration of the issues which surround distribution more generally.

The matter of equal esteem and distinctions is also important. It suggests the difficulties of those small slivers of society which have actually achieved abundance in the past. The behavior of aristocracies (or as Aristotle would more likely say, oligarchies) is highly instructive. When landed estates mean they have no immediate material needs, the result has often been a focus on rank and status and courtier affectation, not a higher form of achievement or freedom.

It also is clear that many of our contemporary political issues are claims or conflicts about equality of esteem, more than economic equality.

Moreoever,

Secondly, the depravity of mankind is an insatiable thing. At first they are content with a dole of a mere two obols, then, when that is traditional, they go on asking for more and their demands become unlimited. For there is no natural limit to wants and most people spend their lives trying to satisfy them. p131

Such is the fate of the welfare state as it develops toward fiscal catastrophe, perhaps. And it is a general warning about the complications and difficulties of redistributing wealth.

In general, he is concerned with the immediate psychology of how people will see things and behave in practice, rather than ultimate principle, which is why it is useful wisdom.

We of course instinctively see how what he says helps explain why the USSR and other communist states got into trouble. But it is also a warning against some of our own practices, though that may be harder to see.

 

Change

He is very conservative with a small c.

.. it is clear that there are some occasions which call for change and that there are some laws that need to be changed. But looking at it in another way we must say that there will be need of the very greatest caution. ..A man will receive less benefit from changing a law than damage from becoming accustomed to disobeying authority. .. The law has no power to secure obedience save the power of habit, and that takes a long time to become effective. Hence easy change from established laws to new laws means weakening the power of the law. p138-9

Habituation is a major theme in his ethical approach. There can be significant downside to basing institutions and expectations on the thin and fragile ground of rational choice alone.

What should reformers take from that? Not that reform or major change is impossible, or ought not to be attempted. But that you have to be aware of the practical difficulties and downsides, and do something to avoid or confront or control them. As I said before the election, the left's dreams often turn into darkness , because they look at one shining principle at the expense of daily reality and psychology.

Progress should be measured not by intentions but by actual flourishing lives.

Of course, seeing this as wry, shrewd advice depends on assuming that some elements of human nature are constant, and the human predicament has some timeless elements that unite us with someone who lived so long ago. In international relations, there has been a long debate over Thucydides, and whether his similar observations of power politics and war and history in Ancient Greece (The History of the Peloponnesian War) still have application today.  

I would say that we do advance a little in social understanding over time. But not as much as we think.

I'll look at more tomorrow.


 

 

Mind blindness & Unknown unknowns

I'm now going to conclude an extensive series of posts on Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here. The extra attention is justified by how well-written and fruitful the book is.

I've naturally come across (Nobel Economics Laureate)Thomas Schelling before, especially his ground-breaking The Strategy of Conflict. But Silver uncovers some writings of his I hadn't come across, in a preface to the book Pearl Harbor: Warning and Decisionby Roberta Wohlstetter.

Schelling writes of our propensity to mistake the unfamiliar for the improbable: There is a tendency in our planning to confuse the unfamiliar with the improbable. The contingency we have not considered seriously looks strange; what looks strange is thought improbable; what is improbable need not be considered seriously.

But at least this flawed type of thinking would have involved some thinking. If we had gone through the thought process, perhaps we could have recognized how loose our assumptions were. Schelling suggests that our problems instead run deeper. When a possibility is unfamiliar to us, we do not even think about it. Instead we develop a sort of mind-blindness to it. In medicine this is called anosognosia:part of the physiology of the condition prevents a patient from recognizing that they have the condition. Some Alzheimer’s patients present in this way.

We have an inherent tendency as human beings to develop kinds of mind-blindness. I love that term.

And perhaps the biggest flaw of all is to believe the future is more predicable or controllable than it really is.

There is reason to suspect that of the various cognitive biases that investors suffer from, overconfidence is the most pernicious. Perhaps the central finding of behavioral economics is that most of us are overconfident when we make predictions. The stock market is no exception; a Duke University survey of corporate CFOs,whom you might expect to be fairly sophisticated investors, found that they radically overestimated their ability to forecast the price of the S&P 500.

Heuristics

So what can we do? What we need is heuristics:

We can think of these simplifications as “models,” but heuristics is the preferred term in the study of computer programming and human decision making. It comes from the same Greek root word from which we derive eureka.A heuristic approach to problem solving consists of employing rules of thumb when a deterministic solution to a problem is beyond our practical capacities.

In a complex an confusing world, I think good heuristics may be the best we can hope for.

It really is a very thoughtful and insightful book covering a wide range of issues. Silver clearly has deep talent far beyond baseball statistics and election forecasting.

 

Prediction and Bayesian testing

I still have a little more to say about Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, which I have been looking at starting here.

I argued in this post the other day that what we need for predictive success is not so much big data as self-awareness. Hypothesis testing ought to help us revise our point of view.

Silver rightly emphasizes prediction is largely a means to an end:

The philosophy of this book is that prediction is as much a means as an end. Prediction serves a very central role in hypothesis testing, for instance, and therefore in all of science.As the statistician George E. P. Box wrote, “All models are wrong, but some models are useful.” What he meant by that is that all models are simplifications of the universe, as they must necessarily be. As another mathematician said, “The best model of a cat is a cat.”Everything else is leaving out some sort of detail. How pertinent that detail might be will depend on exactly what problem we’re trying to solve and on how precise an answer we require.

I keep talking about the importance of purpose, for example here about the difference between maps and models.

The potential pitfalls mean we have to know ourselves, says Silver:

This is why it is so crucial to develop a better understanding of ourselves, and the way we distort and interpret the signals we receive, if we want to make better predictions.

Frequentism

However, much recent statistics has run well and truly off the rails by assuming that error arises from our measurements rather than our perception or judgement. Silver criticizes simple-minded statistical "frequentism", which he says mostly stems from nineteenth century English statistician Ronald Fisher.

The idea behind frequentism is that uncertainty in a statistical problem results exclusively from collecting data among just a sample of the population rather than the whole population.

The idea is you can act as if you can repeat an experiment innumerable times. The more random experiments you do, the more accurate the outcome.

Essentially, the frequentist approach toward statistics seeks to wash its hands of the reason that predictions most often go wrong: human error. It views uncertainty as something intrinsic to the experiment rather than something intrinsic to our ability to understand the real world. The frequentist method also implies that, as you collect more data, your error will eventually approach zero: this will be both necessary and sufficient to solve any problems. Many of the more problematic areas of prediction in this book come from fields in which useful data is sparse, and it is indeed usually valuable to collect more of it. However, it is hardly a golden road to statistical perfection if you are not using it in a sensible way. As Ioannidis noted, the era of Big Data only seems to be worsening the problems of false positive findings in the research literature.

 

Frequentism dominated statistics in the twentieth century. Fisher criticized Bayesian statistics (which we will come to in a moment) for beng insufficiently objective. But, says Silver,

Nor is the frequentist method particularly objective, either in theory or in practice. Instead, it relies on a whole host of assumptions. It usually presumes that the underlying uncertainty in a measurement follows a bell-curve or normal distribution. This is often a good assumption, but not in the case of something like the variation in the stock market. The frequentist approach requires defining a sample population, something that is straightforward in the case of a political poll but which is largely arbitrary in many other practical applications. What “sample population” was the September 11 attack drawn from? The bigger problem, however, is that the frequentist methods—in striving for immaculate statistical procedures that can’t be contaminated by the researcher’s bias—keep him hermetically sealed off from the real world. These methods discourage the researcher from considering the underlying context or plausibility of his hypothesis, something that the Bayesian method demands in the form of a prior probability. Thus, you will see apparently serious papers published on how toads can predict earthquakes, or how big-box stores like Target beget racial hate groups,which apply frequentist tests to produce “statistically significant” (but manifestly ridiculous) findings.

Plenty of investors have lost their shirts by having risk models which assume that market events follow a neat normal ( or similar ) distribution.

Bayesian Probability

Instead, Silver strongly advocates the older Bayesian statistics. In essence, one must specify a prior probability of an outcome , based on one's current beliefs. Bayes' formula then specifies how you should alter that probability in response to incoming data and events, which produces a posterior probability. It is about recognizing your current expectations and beliefs, amd learning from new evidence.

The Bayesian viewpoint, instead, regards rationality as a probabilistic matter. In essence, Bayes and Price are telling Hume, don’t blame nature because you are too daft to understand it: if you step out of your skeptical shell and make some predictions about its behavior, perhaps you will get a little closer to the truth.

We took a step backwards when frequentism arose.

As we will see, science may have stumbled later when a different statistical paradigm, which deemphasized the role of prediction and tried to recast uncertainty as resulting from the errors of our measurements rather than the imperfections in our judgments, came to dominate in the twentieth century.

For me, the point about Bayesian probability ( which I haven't ever used professionally) is not so much the math but a procedure which requires you to test and revise your beliefs in response to evidence. I think Silver overdoes Bayesian probability as THE answer, but his main target in his own intellectual world is likely very much the frequentists. He is a statistician. So we can understand his emphasis on an alternative statistical tradition.

Incidentally, I am no statistician, but I have been intrigued in the past by Keynes' arguments in his A Treatise on Probability (Classic Reprint), but I'll leave that for another time.

 

 

Blaming a whale (and narrative) for defeat

From a David Brooks exchange with Gail Collins in the NYT:

David: This might be a good time for Republicans to redouble their commitment to the reality-based community. Did you see Byron York's reporting from inside the Romney campaign? They apparently had this giant computer model called Orca - named after a whale because it was bigger than anything the Democrats could imagine. It processed huge amounts of data and late in the day was still projecting a Romney victory until its head exploded. Garbage in. Garbage out.

Gail: The Republican Orca - stop me before I fall into a great pile of Moby Dick analogies.

I think that some people are a little transfixed by Nate Silver's accurate projection of the election, despite, as I've noted, his own much more self-awade mature grasp of the limits of models.

I think this is at root a particular problem with journalism.

Or, to put it more precisely, it is a problem with narrative. Human beings have a hardwired fascination with stories, probably dating from sitting around fires in the savannah fifty thousand years ago. It is often the prime way we transmit culture and values and pointers for behavior we should admire.

But stories can be too trite, and this is where journalism becomes very hedgehog-like sometimes. It is all to easy to link stories into an appealing broader narrative, to instinctively frame things in a way which makes for an entertaining read. Stories are much more interesting than data. Once you go looking for "stories", as opposed to drier mechanical reporting of facts, you introduce potential blindspots. Things can fit all too neatly into narrative buckets.

Journalism thrives on stories. Journalists are paid to look for them, rather than dry, ambiguous, complicated truth in isolation.

We've also seen that hedgehogs typically get more media attention, because big striking claims typically make better tv or better stories. Hedgehogs want an audience for their "one big thing." Journalists want a story, And "triumph of models" is itself a good story.

Of course, at the same time as the fivethirtyeight model did well, the Romney and other models performed badly. But why let that get in the way of a good story?

Self-awareness

This is the real explanation of Silver's greater success here. It is not so much a data-driven approach, as using the data to confront your presuppositions. That is why he does better than most journalists and opinion pundits.

It is not, as the story linked above puts it, simply a matter of primitive punditry against spanking new "data driven rationality".

The scientific method is at root about testing hypotheses, not mining data. You test a prediction or explanation against reality, and you revise your views if necessary based on the outcome. You can selectively use data to confirm all kinds of things if you are not careful. So you need to have a falsifiable hypothesis, a situation where at least in principle you may be forced to revise your view.

There's nothing in narrative that compels you to change your view, because you can always add a twist in the story. Indeed, most good stories will have the hero endure many setbacks and misunderstandings before being proved right in the end.

Even in science, as Thomas Kuhn famously pointed out in The Structure of Scientific Revolutions, conflicting data is often left aside as a "puzzle" or "anomaly" in periods of normal science until there is a sudden change of awareness: I.e. the "paradigm shift" which has become so overused a term it is almost a clichė. Even in the hard sciences, it can take a generation for data to settle arguments one way or another. Theories can sprout dozens of ad-hoc adjustments.

So data can be part of this process of self-awareness. But in a world where we have a blizzard of often conflicting and imperfect data, it may equally reinforce entrenched views.

Romney's big Moby Dick model was probably highly sophisticated, but likely being used to mine data more efficiently, not testing against reality.

Innumerate journalists may regard the statistical models as some sort of new semi-deity, partly because they don't understand it, or the many ways in which statistics can be used to obscure. (There is a famous book called How to Lie with Statistics).

But the real message is not statistics as some kind of new rational technique which guarantees prophetic success. Instead, it is self-awareness.


You need devices to help you retain self-awareness, to see things as they really are instead of ever-more elaborate stories or models. And humans typically find that very hard to do. I've got a few more things to say about Silver's book in that light.

 

Morality binds and blinds

I was talking about Jonathan Haidt's analysis on Monday. What now, he asks in the NYT this morning. Shared fear and common threats can help overcome partisan blindness, he argues.

A basic principle of moral psychology is that “morality binds and blinds.” In many pre-agricultural societies, groups achieved trust and unity by circling around sacred objects. In modern societies, much larger groups bind themselves together by treating certain books, flags, leaders or ideals as sacred and by symbolically circling around them. But if your team circles too fast, you lose the ability to see clearly or think for yourself. You go blind to evidence that contradicts your group’s moral consensus, and you become enraged at teammates who suggest that the other side is not entirely bad (as New Jersey’s governor, Chris Christie, is now finding out).

Unlike a foreign attack, a problem that threatens only one side’s sacred values can therefore divide us, rather than unite us.

Each side finds it difficult to see the other's sacred values, he says. So conservatives tend to deny climate change, for example. Liberals tend to deny potential problems with entitlement spending.

But there are so many "asteroids" about to hit us in coming years that necessity may force each side to recognize some merit in other views. That may mean more attention to economic inequality and the fact that the family has eroded - 40% of births are to unwed mothers, for example.

I think that is a civilized comment for the morning after the election. At least we can be thankful for the country's sake that there has been no hanging chad dramas and Obama won the popular vote as well as the electoral college.

More reaction from me when I'm a bit less tired and have time to absorb it.

 

 

Negative Feedback loops

 

We're looking at Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here.

Here's another stray reference in the book which reminded me of something else. Silver talks about Larry Summers and feedback loops in the economy.

Usually, in Summers’s view, negative feedbacks predominate in the American economy, behaving as a sort of thermostat that prevents it from going into recession or becoming overheated. Summers thinks one of the most important feedbacks is between what he calls fear and greed.

Summers is a brilliant if famously obnoxious economist, of course, with a lot of practical experience of economic crises in several administrations. Isn't it surprising how little attention there has been to the homeostatic mechanisms in the American economy apart from the price system? A capitalist economy does have some inherent resilience and stability, for all its volatility. But we don't have much real knowledge about confidence and expectations ( for all the attention in economics to rational expectations.)

Another feedback loop is displacement of demand. Recessions produce pent-up demand for houses or cars or consumer durables. Less buying now typically means, other things equal, more buying tomorrow. Choice across time matters. (One of my larger themes, however, is preferences and tastes may not be stable as before, as abundance means many basic needs are satisfied.)

The best account I've seen from this dynamic perspective is Jane Jacob's dialogue about dynamic stability, which we looked at here. Surely someone within the economics profession is doing some systematic work in this vein. Or perhaps not, as it is so heterodox. If someone knows of any such good work, please comment.

In college, one sure way to throw something at an economics problem was to contrast comparative statics with a dynamic view of the economy. But much of the bedrock of economics is still single-country comparative statics, perhaps, for the daring, with some hysteresis - path-dependence - thrown in.

When people try to do dynamics , it necessarily relies much more on assumptions about behavior and psychology. Take, for example, the controversies over CBO dynamic forecasts of tax revenue. Republicans tend to want to see more behavioral changes and higher revenue for a given tax cut because do dynamic effects. Democrats do not.

The upshot is we would do much better thinking about the economy in dynamic terms, with close attention to feedback loops. But we mostly don't.

Maps, not models, by Mapper

We're looking at Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here.

There's one stray reference in the book, which caught my eye, because I've been interested for over a decade in the difference between maps and models as ways to understand the world. I call myself Mapper on this blog for a reason.

Silver says:

The Finnish scientist Hanna Kokko likens building a statistical or predictive model to drawing a map. It needs to contain enough detail to be helpful and do an honest job of representing the underlying landscape—you don’t want to leave out large cities, prominent rivers and mountain ranges, or major highways. Too much detail, however, can be overwhelming to the traveler, causing him to lose his way. As we saw in chapter 5 these problems are not purely aesthetic. Needlessly complicated models may fit the noise in a problem rather than the signal, doing a poor job of replicating its underlying structure and causing predictions to be worse. But how much detail is too much—or too little? Cartography takes a lifetime to master and combines elements of both art and science. It probably goes too far to describe model building as an art form, but it does require a lot of judgment. Ideally, however, questions like Kokko’s can be answered empirically. Is the model working? If not, it might be time for a different level of resolution.

I hadn't head of Kokko before, but I thought this was interesting. The reason I find the topic so fascinating is that I have sat many times in meetings with senior economic officials - or even more often, their more academic staff - who argue that to think clearly about a situation you need a model in your head, if only to ensure consistency. For most economists, this tends to be linked to some version of Milton Friedman's as-if methodology, which says that the test of a model is not the realism of its assumptions, but the accuracy of its predictions. Simplification and abstraction from reality is what constitutes explanation, and anything else is mere muddle.

Of course, this automatically counts out the value of studying history, which has been largely excluded from the mainstream of education in the discipline for fifty years (despite some good work). Mathematical models have reigned supreme. Optimization subject to constraints is the center of thinking. It is far narrower than even "theory", as it imposes essentially aesthetic constraints on what a theory should be, I.e. mathematically elegant.

It also is dangerously overreliant on consistency. You need to be consistent to be fully rational in your choices, of course, in a narrow sense. But consistency is no guarantee of truth. You can be consistently wrong. And many actual problems are about reconciling different objectives or opposing views. Compromise in this sense is bound to be somewhat inconsistent with someone's basic principles. It is also real life.

However, the most important point is the economic modeling perspective is also a very naive way to think about useful abstraction. A map is an abstraction too, but it tends to be much more useful for most purposes than a mathematical model. Many policy problems are in fact much more like "how to get from A to B" than " maximize X subject to Y".

In particular, maps are much more focused on specific purposes. If I want to get from New York to Albany, I look at a standard road atlas or Google maps. If I want to see where shale gas deposits may lie, I want a geological map of New York State. If I want to sail up the Hudson to Albany, I want a chart which shows sandbanks and shoals and traffic lanes in the river.

In other words, purpose is much more intrinsically present in a map than a mathematical model using aggregate economic statistics or optimization assumptions.

Add to that, as Kokko says, the scale and resolution which are inherent to maps. You abstract away what you don't need. He is right here, but both Silver and Kokko are wrong in a larger sense. The point is not that there is an art to building models so they are , by analogy, a little more like maps with a correct level of detail. It is that maps and models are wholly different ways to understand reality.

Maps are much more suited to seeing risks. The kind of generalized abstraction in a model will not help you avoid the specific rock which is just below the surface as you enter harbor, but the right chart will. A map will show you the massive mountain chain or desert or unfordable river in your way. It will show the paths and cliffs and hazards. A model won't.

So thinking in terms of surveying a specific landscape for specific important features for a specific purpose is just as disciplined an intellectual exercise as developing an abstract model to predict, and much more useful. It can also be a much better predictor of some kinds of problem (eg how long it will take to get to Albany.)


Maps help you see what is actually there, rather than "explain" it in some ultimate universal way. We most often don't need a general theory of roads. We just want to find our way home. Models will predict, but maps will show you the right way. If you want to climb a mountain you are better off bringing a topo map than an abstract mathematical model of paths.

Silver's book is all about prediction, but the more general human problem in decision-making is which way do I go? What is the right direction? Where are the hazards? We need maps more than universalized explanations in most situations. We need a survey of the actual landscape with a particular purpose in mind.

Ways of seeing what is really there are the main way we will develop and make progress. Too much modelling often prevents us from doing that, by distracting people towards the mathematically elegant and tractable, the thin universal rather than the thick description of specifics, and the quantifiable and obvious rather than the danger that lurks in the details.

 

Overheard in a New York Coffee Shop

Loud declamation in a thick Brooklyn accent:

"Maybe those MAYANS were RIGHT!!"

 

Economic forecasts

We're looking at Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here.

One other area which has experienced consistent failure is economic forecasting. I cite what Silver says here with a certain amount of glee. Of course, I know this as background information, but to see the hard facts marshalled together is striking. Take a survey economic forecasts in 2008, for instance.

As I mentioned, the economists in this survey thought that GDP would end up at about 2.4 percent in 2008, slightly below its long-term trend. This was a very bad forecast: GDP actually shrank by 3.3 percent once the financial crisis hit. What may be worse is that the economists were extremely confident in their bad prediction. They assigned only a 3 percent chance to the economy’s shrinking by any margin over the whole of 2008.15 And they gave it only about a 1-in-500 chance of shrinking by at least 2 percent, as it did.

Nor was this a once-off occurrence because of a freak once-in-a-lifetime crisis.

In fact, the actual value for GDP fell outside the economists’ prediction interval six times in eighteen years, or fully one-third of the time. Another study,18 which ran these numbers back to the beginnings of the Survey of Professional Forecasters in 1968, found even worse results: the actual figure for GDP fell outside the prediction interval almost half the time. There is almost no chance that the economists have simply been unlucky; they fundamentally overstate the reliability of their predictions.

Aggregate forecasts tend to be more reliable than individual forecasts, however. This has been bad news for in-house corporate economists, who were mostly eliminated in the 1990s. Bluechip or Consensus Forecasts are better.

My research into the Survey of Professional Forecasters suggests that these aggregate forecasts are about 20 percent more accurate than the typical individual’s forecast at predicting GDP, 10 percent better at predicting unemployment, and 30 percent better at predicting inflation. This property—group forecasts beat individual ones—has been found to be true in almost every field in which it has been studied.

Perhaps the new availability of computers made forecasters particularly overconfident in the 1960s and 1970s, he says - the age of the massive economic forecasting model. But ultimately you have to have some theoretical understanding or you will sink into mere data mining, he says.

The idea that a statistical model would be able to “solve” the problem of economic forecasting was somewhat in vogue during the 1970s and 1980s when computers came into wider use. But as was the case in other fields, like earthquake forecasting during that time period, improved technology did not cover for the lack of theoretical understanding about the economy; it only gave economists faster and more elaborate ways to mistake noise for a signal. Promising-seeming models failed badly at some point or another and were consigned to the dustbin.

Economics has inherent limitations on theory, however. One of the decisive intellectual impacts on me in college was learning about the Lucas Critique, which says people's behavior may change when policy changes, so you cannot rely on large-scale econometric relationships. I lost interest in econometrics and forecasting.

The economics profession has mostly responded to this problem by searching for policy-invariant microfoundations. It tries to model individial choice, far below the level of economic aggregates. In practice, this mostly entrenches naive rational-choice mathematical optimization even further.

A better answer to this is deeper knowledge of history. At least some people in the central banks note that we are fortunate that Ben Bernanke was an acknowledged expert in the history of the Great Depression, rather than, say, real business cycle models.

 

Overfitting models

We're looking at Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here. Silver discusses some of the inherent problems and mistakes people make with statistical models.

One of the most important is overfitting.

The name overfitting comes from the way that statistical models are “fit” to match past observations. The fit can be too loose—this is called underfitting—in which case you will not be capturing as much of the signal as you could. Or it can be too tight—an overfit model—which means that you’re fitting the noise in the data rather than discovering its underlying structure. The latter error is much more common in practice.

It can lead to serious problems.

 

As obvious as this might seem when explained in this way, many forecasters completely ignore this problem. The wide array of statistical methods available to researchers enables them to be no less fanciful—and no more scientific—than a child finding animal patterns in clouds.* “With four parameters I can fit an elephant,” the mathematician John von Neumann once said of this problem. “And with five I can make him wiggle his trunk.” Overfitting represents a double whammy: it makes our model look better on paper but perform worse in the real world. Because of the latter trait, an overfit model eventually will get its comeuppance if and when it is used to make real predictions.

This is one of the great stories of financial markets. People are forever trying to come up with the equivalent of quantitative alchemy to transform historical data into gold. It is quite easy to tune a model so it performs very well on past data, and marches undulations with surprising precision. And it is amazingly easy to lose your shirt when the model goes awry when used to predict where the market will go next.

 

Foxes and Hedgehogs

I'd started talking about Nate Silver's The Signal and the Noise: Why So Many Predictions Fail-but Some Don't, starting here. He is very controversial right now, immediately before the election, because of his prediction of a much higher chance of an Obama victory than the headline polls indicate. So he will be seen by many in partisan terms.

The book is not partisan, however, so it deserves a fair look by people who might fight over his NY Times articles. After all, Silver argues for skepticism about expert predictions as a general rule.

I've mentioned Philip Tetlock's work on political prediction before. Tetlock found that most prediction by political or international relations experts is terrible. As Silver puts it,

Tetlock’s conclusion was damning. The experts in his survey—regardless of their occupation, experience, or subfield—had done barely any better than random chance, and they had done worse than even rudimentary statistical methods at predicting future political events. They were grossly overconfident and terrible at calculating probabilities: about 15 percent of events that they claimed had no chance of occurring in fact happened, while about 25 percent of those that they said were absolutely sure things in fact failed to occur. It didn’t matter whether the experts were making predictions about economics, domestic politics, or international affairs; their judgment was equally bad across the board.

But Tetlock also found one distinction that pointed to more successful forecasts. Isaiah Berlin had revived the ancient distinction between hedgehogs, who know one big thing, and foxes, who know many small things. I've mentioned it many times. Silver puts it nicely:

Hedgehogs are type A personalities who believe in Big Ideas—in governing principles about the world that behave as though they were physical laws and undergird virtually every interaction in society. Think Karl Marx and class struggle, or Sigmund Freud and the unconscious. Or Malcolm Gladwell and the “tipping point.” Foxes, on the other hand, are scrappy creatures who believe in a plethora of little ideas and in taking a multitude of approaches toward a problem. They tend to be more tolerant of nuance, uncertainty, complexity, and dissenting opinion. If hedgehogs are hunters, always looking out for the big kill, then foxes are gatherers. Foxes, Tetlock found, are considerably better at forecasting than hedgehogs.

Hedgehogs have more trouble seeing what is there without predispositions.

Foxes may have emphatic convictions about the way the world ought to be. But they can usually separate that from their analysis of the way that the world actually is and how it is likely to be in the near future. Hedgehogs, by contrast, have more trouble distinguishing their rooting interest from their analysis. Instead, in Tetlock’s words, they create “a blurry fusion between facts and values all lumped together.” They take a prejudicial view toward the evidence, seeing what they want to see and not what is really there.

Hedgehogs are vey good at coming up with stories and narratives that validate their positions - and turn out to be wrong.

The foxy forecaster recognizes the limitations that human judgment imposes in predicting the world’s course. Knowing those limits can help her to get a few more predictions right.

I am, as you might imagine from reading this wide-ranging blog, a fox to the core. I like to take ideas from different disciplines, and recombine and synthesize. So of course I like this argument. Hedgehogs often suffer from serious blind spots, and are prone to fanaticism.

One issue I haven't seen addressed anywhere, though, is what makes people foxes or hedgehogs. Part of it must be personality, although it's difficult to see a direct link to the big five theories of personality. There is some evidence that liberals tend to have higher openness to experience, but in my experience liberals are if anything more prone to ideological narrowness than conservatives. Conservatives often have a skepticism about systems and experts and theory which might make at least some less prone to hedgehog temptation. But one can find ideologues and zealots across the political spectrum. It is a sensibility rather than a particular political conviction.

Some of it must be a matter of education and styles of learning. And some must be a reflection of the incentives we set up. As Tetlock says, you are likely to be a more successful TV pundit by making overconfident big pronouncements than being nuanced.

In any case, Silver's point is that hedgehogs tend to be worse at prediction, not necessarily worse in general. Some of the most gifted people are hedgehogs by nature. As I've said before, Plato was a hedgehog, Aristotle a fox ( which may explain why I've become so interested in Aristotle). Dante was a hedgehog, Shakespeare a fox. Systems have their place. But we do need a feel as a society for the boundaries and limits and clashes of systems.

People like to believe in more certainty than there really is.

Loneliness and Marriage

This column in the UK Telegraph notes there has been a huge increase in the number of people living alone. It isn't just more single young people, either.

As the ONS makes clear, the largest increase in solitary living is down to the 45-64 age group. Almost two and a half million Britons in that age category have no one with whom to share their home, an increase of more than 800,000 households since the mid-Nineties. Even allowing for the increase in total population size, that’s still a noticeable change, and they don’t all enjoy the experience.

In US terms, that would be an increase of around four million people living alone. Many might choose it, of course. Many more would not.

Much of this blog is about what leads to actual happiness or flourishing, rather than simply measuring welfare as GDP or material standards of living. And this is a sign of social deterioration. Loneliness is not part of flourishing, and it deserves far more attention than it gets in general public discussion. Gross National Loneliness is as important as GDP once you have a threshold of acceptable material living standards, as I've argued before.

Marriage, the author argues here, is the most important bulwark against loneliness, and the government should promote it.

Michael Howard deployed a powerful phrase in defence of his criminal justice policy: prison works. It’s time we used a similar phrase, in defence of social justice: marriage “works” too. It works for most people and definitely for civic society, yet we find it hard to say this, and shy away from its political implications. What started as a desire not to judge “lifestyle choices” has bred a generation living in lonely, quiet despair. Loneliness is a much harder political issue to tackle than, say, house-building, but – if we believe in “society” at all – hardly one of lesser significance.

Socially conservative? Yes. But then it's often the most vulnerable who are left worst off by declining social ties. More choice for some often has consequences. A libertarian or highly liberal approach to relationships and families leaves a lot of shattered lives. Liberal neutrality often produces meaninglessness and emptiness.

 

Signal, Noise and Prediction

I'm now going to turn to Nate Silver's new book, The Signal and the Noise: Why So Many Predictions Fail-but Some Don't. Silver is the well-known political forecaster who parlayed his blog FiveThirtyEight into a prominent spot in the New York Times.

I didn't expect much when I bought it. I thought it would be one of those "my quantitative model explains the universe" books (and investment funds) which are so tiresome and common. The world, and especially the markets, are filled with quants who think all you need is Mathematica and some back issues of Econometrica to explain everything. They are usually overconfident, expert on code rather than decisions,and tend to blow up spectacularly like LTCM given time.

Nothing could be further from the truth in this case. The book massively exceeded expectations and turns out to be a thoughtful, mature and reflective. It is consistent with much of my experience and thinking, but I still learned a lot of things I didn't know. It's also fluent and well-written. I'd recommend it without hesitation, and I'll look at it in some detail.

The crux of the book, from someone known for his number-crunching models, is that there is no such thing as objective data-driven models, at least in human affairs.

The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning. Like Caesar, we may construe them in self-serving ways that are detached from their objective reality. Data-driven predictions can succeed—and they can fail. It is when we deny our role in the process that the odds of failure rise. Before we demand more of our data, we need to demand more of ourselves.

The more information we have, the more we tend to screen out that which does not match our preconceptions. More information most often makes us narrower rather than wiser.

Alvin Toffler, writing in the book Future Shock in 1970, predicted some of the consequences of what he called “information overload.” He thought our defense mechanism would be to simplify the world in ways that confirmed our biases, even as the world itself was growing more diverse and more complex.

Information is no longer scarce, but much of it is not very useful.

Our biological instincts are not always very well adapted to the information-rich modern world. Unless we work actively to become aware of the biases we introduce, the returns to additional information may be minimal—or diminishing.

This does not mean we should just give up, or adopt lazy relativism. Instead, everything is approximate.

Some of you may be uncomfortable with a premise that I have been hinting at and will now state explicitly: we can never make perfectly objective predictions. They will always be tainted by our subjective point of view. But this book is emphatically against the nihilistic viewpoint that there is no objective truth. It asserts, rather, that a belief in the objective truth—and a commitment to pursuing it—is the first prerequisite of making better predictions. The forecaster’s next commitment is to realize that she perceives it imperfectly.

So what are the causes of failure to predict outcomes?

The most calamitous failures of prediction usually have a lot in common. We focus on those signals that tell a story about the world as we would like it to be, not how it really is. We ignore the risks that are hardest to measure, even when they pose the greatest threats to our well-being. We make approximations and assumptions about the world that are much cruder than we realize. We abhor uncertainty, even when it is an irreducible part of the problem we are trying to solve.

Indeed, experts have a particular tendency to ignore threats to their expertise. The rating agencies, for example, did not think through the possibility that default risk of various CDOs and CDO tranches might not be independent and uncorrelated.

The possibility of a housing bubble, and that it might burst, thus represented a threat to the ratings agencies’ gravy train. Human beings have an extraordinary capacity to ignore risks that threaten their livelihood, as though this will make them go away.

Our expectations about the future are riddled with blind spots, as anyone who has ever really thought about the policy process or had to predict events for a living - and been held accountable for it - knows.

We'll look at some other aspects of the book in more detail.

 

Why are states so red and blue?

Steven Pinker looks at the evidence.

If this history is right, the American political divide may have arisen not so much from different conceptions of human nature as from differences in how best to tame it. The North and coasts are extensions of Europe and continued the government-driven civilizing process that had been gathering momentum since the Middle Ages. The South and West preserved the culture of honor that emerged in the anarchic territories of the growing country, tempered by their own civilizing forces of churches, families and temperance.

I looked at Pinker's excellent (and very long) last book here.