Siren Servers

The stories about NSA surveillance of email and phone metadata are one more boost to skepticism about the new digital economy.

Here's a separate article in the NYT about digital concentration of influence and wealth.

DISSECT almost any ascendant center of power, and you’ll find a giant computer at the core. In the past, power and influence were gained by controlling something that people needed, like oil or transportation routes. Now to be powerful can mean having the most effective computer on a network. In most cases, this means the biggest and most connected computer, though very occasionally a well-operated small computer can play the game, as is the case with WikiLeaks. Those cases are so rare, however, that we shouldn’t fall into the illusion of thinking of computers as great equalizers, like guns in the Wild West.

The new class of ultra-influential computers come in many guises. Some run financial schemes, like high-frequency trading, and others run insurance companies. Some run elections, and others run giant online stores. Some run social network or search services, while others run national intelligence services. The differences are only skin deep. I call this kind of operation a “Siren Server.”

The irony is the original image of the personal computer was of liberation, such as Apple's famous Superbowl ad in 1984. The Silicon Valley political vision of the world - a kind of libertarian paradise - is encountering increasing resistance and skepticism.

So the author of the NYT article suggests we should move towards micropayments for our personal information, to reduce the advantage of the "siren servers."

Of course, the counterargument is today's titans - Google, Facebook, Amazon, Apple - are likely to be tomorrow's Dell, Compaq, MySpace, or AOL, let alone earlier titans such as Bethlehem Steel or Pennsylvania Railroad. I doubt there are natural monopolies or network effects sufficiently powerful to entrench the current digital elite for long.

Algorithms can be replicated. That kind of information wants to be free, too. And if people find their personal data is being over-exploited, they will find alternative ways to communicate.

There can often be something of an arms race with technology. Take airline fares, for example. Large centralized computing power originally enabled the airlines to segment their customers in the 1970s and 1980s, and extract the maximum amount people would pay for a certain route. But the consumer caught up. Technology also brought transparency before long, including sites like Expedia or Kayak. Large centralized computers still price airline fares, but the airlines barely stay out of bankruptcy most of the time, and their only way to get pricing power is to cut capacity. Their industry became commoditized.

People have also feared the use of technology for surveillance for decades, but it usually enables transparency and less power for gatekeepers too.

The whole economy is changing in deeper ways, and that is irreversible. But I doubt that technology will enable any enduring center of power.

 

 

 

 

 

Morality unhinged from Wisdom

Walter Russell Mead writes about the situation in Syria, as Samantha Power, who urged humanitarian interventions, leaves the Obama Administration:

As a general rule, sad to say, the good guys and the smart guys often play on different teams. For too many foreign policy humanitarians, it is more important to have good intentions than to understand the crooked and wicked ways of the world you want to change. This instinct for the ideal over the real was a hallmark of humanitarian policy failures all during the 20th century and on the evidence to date the deadly mixture of political amateurism with ambitious humanitarian international agendas has persisted into the 21st. America’s university campuses are packed with people who believe that the flaws in our foreign policy are failures of morality rather than failures of forethought and execution, but morality unhinged from wisdom is one of the most destructive forces known to man.

 

Underperforming hedge funds (again)

More bad news for hedge fund returns, according to this NYT piece. A standard 60% equity/40% bond portfolio returned 90% over the last decade, compared with 17% after fees for a common hedge fund index.

The author finds some people just seem to want to pay higher fees for lack of transparency and hints about secret rocket science under the hood. People - and also pension funds. Two-thirds of the industry is now institutional money. But how long can that go on, if performance is so bad on average?

 

The "Semantic Web" is next

This is an interesting piece on where Google Search is going in the future.

Until now, Google has been an unprecedented signposter of knowledge. It has not "known" the answer to anything itself but it has had an awfully clever way of directing you to exactly the place you can find out. In some senses, that attribute is in the process of changing. This year, Google will roll out what it calls its Knowledge Graph, the closest any system has yet come to creating what Tim Berners-Lee, originator of the web itself, called "the semantic web", the version that had understanding as well as data, that could itself provide answers, not links

 

"History suggests the entitlement era is winding down"

Michael Barone says major changes in American life have often happened at 76-year intervals. The entitlement state may be come to an end, as one of those 76-year intervals is almost up.

The original arrangements in each 76-year period became unworkable and unraveled toward its end. Eighteenth-century Americans rejected the colonial status quo and launched a revolution and established a constitutional republic.

Nineteenth-century Americans went to war over expansion of slavery. Early 20th-century Americans grappled with the collapse of the private sector economy in the Depression of the 1930s.

We are seeing something like this again today. The welfare state arrangements that once seemed solid are on the path to unsustainability.

Of course, 76 year intervals are just a rhetorical hook. The broader point is that underlying social attitudes and institutions need to adjust and shift at least every three generations or so, as the world changes.

 

More signs of a renaissance in psychology

This is an interesting column from David Brooks in the NYT, mostly because he sees many of the main challenges in public policy as a matter of better psychology.

What about the big problems? How do we get people to restrain government commitments now so that debt down the road won’t be so ruinous? How do we calculate the multiplier effects of tax cuts or spending increases among different subgroups of the population, or under different emotional conditions? How do we rig the context of budget negotiations so participants can actually come to a deal? How are people in different cultures likely to react to drone strikes? How do we structure sanctions against Iran to cause the greatest psychic humiliation?

These are the big questions, and most of our policies rely on crude folk psychology from a few politicians. But there’s hope. As Brian Wansink notes in Eldar Shafir’s volume, the 20th century saw great gains in sanitation and public health. The 21st century could be a great period for behavior change.

No mention of economics at all. Of course, economics has some things to say about credibility and commitment, such as the Kydland/Presscott time inconsistency literature. But the rational actor approach appears to be yielding diminishing returns. The focus of the social sciences can shift over time. I've been seeing more and more interesting things in psychology recently.

 

"Sure, Big Data Is Great. But So Is Intuition"

An NYT article is skeptical of inflated claims for Big Data:

The quest to draw useful insights from business measurements is nothing new. Big Data is a descendant of Frederick Winslow Taylor’s “scientific management” of more than a century ago. Taylor’s instrument of measurement was the stopwatch, timing and monitoring a worker’s every movement. Taylor and his acolytes used these time-and-motion studies to redesign work for maximum efficiency. The excesses of this approach would become satirical grist for Charlie Chaplin’s “Modern Times.” The enthusiasm for quantitative methods has waxed and waned ever since.

Big Data proponents point to the Internet for examples of triumphant data businesses, notably Google. But many of the Big Data techniques of math modeling, predictive algorithms and artificial intelligence software were first widely applied on Wall Street.

At the M.I.T. conference, a panel was asked to cite examples of big failures in Big Data. No one could really think of any. Soon after, though, Roberto Rigobon could barely contain himself as he took to the stage. Mr. Rigobon, a professor at M.I.T.’s Sloan School of Management, said that the financial crisis certainly humbled the data hounds. “Hedge funds failed all over the world,” he said.

The problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according to the laws of physics.

In so many Big Data applications, a math model attaches a crisp number to human behavior, interests and preferences. The peril of that approach, as in finance, was the subject of a recent book by Emanuel Derman, a former quant at Goldman Sachs and now a professor at Columbia University. Its title is “Models. Behaving. Badly.”

It really is a matter of proper scope, and consciousness of limits. Big Data is wonderful for finding the Higgs Boson among billions of particle paths, or tracking potential credit card fraud. It is not so good at many other tasks where the data is absent or incomplete or misleading. In those cases, it is little different from ancient farmers looking at the sky and seeing mythical animal patterns.

Krugman, Robots and Redistribution

There's a lot of attention to this post by Paul Krugman on long-term growth.

And this means that in a sense we are moving toward something like my intelligent-robots world; many, many tasks are becoming machine-friendly. This in turn means that Gordon is probably wrong about diminishing returns to technology.

Ah, you ask, but what about the people? Very good question. Smart machines may make higher GDP possible, but also reduce the demand for people — including smart people. So we could be looking at a society that grows ever richer, but in which all the gains in wealth accrue to whoever owns the robots.

Perhaps at some stage there will be a deeper discussion of these issues. He says in a related column he will talk more about long-term growth in due course. But, he concedes, economists know very little about long-run prospects.

The great bulk of the economic commentary you read in the papers is focused on the short run: the effects of the “fiscal cliff” on U.S. recovery, the stresses on the euro, Japan’s latest attempt to break out of deflation. This focus is understandable, since one global depression can ruin your whole day. But our current travails will eventually end. What do we know about the prospects for long-run prosperity?

The answer is: less than we think.

The long-term projections produced by official agencies, like the Congressional Budget Office, generally make two big assumptions. One is that economic growth over the next few decades will resemble growth over the past few decades. In particular, productivity — the key driver of growth — is projected to rise at a rate not too different from its average growth since the 1970s. On the other side, however, these projections generally assume that income inequality, which soared over the past three decades, will increase only modestly looking forward.

It’s not hard to understand why agencies make these assumptions. Given how little we know about long-run growth, simply assuming that the future will resemble the past is a natural guess. On the other hand, if income inequality continues to soar, we’re looking at a dystopian, class-warfare future — not the kind of thing government agencies want to contemplate.

Yet this conventional wisdom is very likely to be wrong on one or both dimensions.

An additional problem is economics has very little understanding of the drivers of productivity. I don't know how many times senior officials have told me that it is a "residual of a residual" in quantitative terms.

 

Redistribution

We've looked at these issues before, such as here and here.

In particular, we looked at an excellent survey of technological developments here. Diamandis and Kotler conclude their book Abundance: The Future Is Better Than You Think with this:

Our problem is not that we don’t have enough stuff—it’s that we don’t have enough ways for people to work and prove that they deserve this stuff.

However, simply transferring money from "the rich" to the middle class or the poor is ultimately self-defeating. It undermines self-reliance and self-respect and the fact that living is more than just having stuff.

Redistribution is going to be the core issue in coming years - but even the word itself is tainted with leftist welfarism.

Instead, we need a better conception of how people earn a living, especially if the labor market is in trouble. And that means a better idea of value. And that entails some idea of the good life for people, instead of liberal neutrality whose whole point is to avoid any discussion of value.

Economics has deep difficulty with this, because value is primarily an ethical issue which is not reducible to models. Indeed, we saw recently economics as a discipline sidestepped the problem of value in the marginalist revolution by replacing it with a poorly conceived mathematical treatment of utility.

The whole apparatus of pareto-equality and welfare economics is flawed. Liberal ethics is flawed. Both lack any conception of human flourishing. Automatic equality of respect destroys any prospect of a better life, because it corrodes value. If everything is equal, there is no value.

 

Merry Christmas to Everybody

Nothing like a break for the holidays with family.

Shaking the British Establishment

It's interesting that Canadian Mark Carney has been appointed Governor of the Bank of England. Carney is very able. But it is a huge blow to BoE insiders like Paul Tucker, who had spent twenty-five years specifically working very are to get the top job someday.

Interestingly, it is a sign of much deeper disarray and loss of confidence in the British economic establishment. From the Guardian:

Rachel Lomax is practically the definition of establishment: Cheltenham Ladies' College followed by Cambridge and the LSE; principal private secretary to then-chancellor Nigel Lawson; deputy governor of the Bank of England for five years until 2008. Which makes what she said on Friday evening all the more startling.

This being a debate on the future of capitalism in the People's Republic of Bristol, the audience were satisfyingly radical – but Lomax was just as bluntly and disarmingly political. The former Treasury mandarin made no bones about admitting that she had been part of a project of "dismantling a version of capitalism" and replacing it with "Anglo-American neo-liberalism". You'd struggle to get scholars of Thatcherism to speak with such straightforwardness, but here it was coming from one of the era's key backroom players.

And now this co-architect of Britain's economic model as good as admitted that the system she had helped create was broken. But Lomax had one question: "Where is the revolutionary thinking?"

You surely couldn't ask for a better measure of the economic mess we're in, that even members of the establishment are now calling for revolution.

Striking as it is, such despair isn't exceptional. Indeed, it now appears endemic among the policy-making elite. Whether you look at Westminster or Threadneedle Street, Britain's economic officials reek of policy fatigue – of having riffled through all the pages in their textbooks without getting a good answer.

That also exists here in the US.

 

Food breakthrough?

This is pretty interesting. A new company is successfully growing food using just sea water and Australian scrubland.

Indeed, the work that Sundrop Farms, as they call themselves, are doing in South Australia, and just starting up in Qatar, is beyond the experimental stage. They appear to have pulled off the ultimate something-from-nothing agricultural feat – using the sun to desalinate seawater for irrigation and to heat and cool greenhouses as required, and thence cheaply grow high-quality, pesticide-free vegetables year-round in commercial quantities.

So far, the company has grown tomatoes, peppers and cucumbers by the tonne, but the same, proven technology is now almost ready to be extended to magic out, as if from thin air, unlimited quantities of many more crops – and even protein foods such as fish and chicken – but still using no fresh water and close to zero fossil fuels. Salty seawater, it hardly needs explaining, is free in every way and abundant – rather too abundant these days, as our ice caps melt away.

It could dramatically change world water usage.

It's an inspiring project, more important, it could be argued, than anything else going on in the world. Agriculture uses 60-80% of the planet's scarce fresh water, so food production that uses none at all is nothing short of miraculous.

I can't help wondering if there is a catch, or long-term problem that isn't evident yet. But a combination of hydroponic techniques and more efficient solar power may be durably effective.

It's bearing out some of the ideas and arguments and new technologies we discussed here.

 

"The Curse of Warholism"

The New Republic has a swashbucklingly irate review of the big Andy Warhol exhibition at the Met. According to Jed Perl,

Warhol has become his own ism. Warholism is the dominant ism of our day, grounded as it is in the assumption that popular culture trumps all other culture, and that all culture must become popular culture in order to succeed, and that this new high-plus-pop synergy relieves everybody of the responsibility to experience works of art one on one. The belligerent knowingness of Warholism is what fuels “Regarding Warhol: Sixty Artists, Fifty Years,” the extraordinarily elaborate exhibition now at the Metropolitan Museum of Art. Nearly everybody agrees that the show is a mess, although few seem to have stopped to wonder if Warholism is the reason why.

G and I saw the show earlier this week. "Mixed", we decided. "Variable." "It reminds me a lot of the more posing and pompous people I have to deal with", said G.

 

It certainly taps into something in the culture, but perhaps the most annoying and trivial politicized people in the culture.

The actual Warhol pieces themselves were satisfying , although it is hard to know of this is just because they have an aura of familiarity because of their near-iconic status. But the curation was all over the place, and annoyingly hagiographical rather than analytical.

Perhaps this is at root about fear of not being in the in-crowd, or manoevering to get there.

So Warholism began with an anxiety—the anxiety of philistinism, or the fear of an allegation of philistinism—to which a few writers proposed a few tentative solutions. For the educated public that even back then was beginning to fear cultural ostracism, Warholism offered the assurance that anybody who climbed on the Pop Art bandwagon could have the social cachet of an avant-gardist. Steinberg in particular brooded about what he saw as the inability of even some legendary avant-gardists to accept a new avant-garde, recalling Signac’s difficulties with Matisse’s most simplified work, and Matisse’s difficulties with Picasso’s Demoiselles d’Avignon and Braque’s first Cubist compositions. Steinberg is understanding about their equivocations, concluding not “that only academic painters spurn the new,” but that in fact “any man becomes academic by virtue of, or with respect to, what he rejects.”

Perl notes the critic Arthur Danto could see where this was going back in 1964.

What the work itself elicits is far less important than the task of locating whatever happens to have turned up in the art galleries in some historical scheme. “To see something as art requires something the eye cannot descry—an atmosphere of artistic theory, a knowledge of the history of art; an artworld.” This statement bears close examination. The art world has trumped the art—and Warholism is born. “What in the end makes the difference between a Brillo box and a work of art consisting of a Brillo Box is a certain theory of art.... The world has to be ready for certain things, the artworld no less than the real one. It is the role of artistic theories, these days as always, to make the artworld, and art, possible.”

It means we do not have confidence in the experience of the eye itself, says Perl.

I'm not sure. Something seems confused and wrong about the show. Perhaps the basic point is that art should not simply be about cliques. Warhol democratized the avant-garde, in a sense, but also reinforces it. New cliques establish themselves by undermining the old cliques. Ideas are used simply as political weapons. And that cynicism seems deeply false.

Perhaps humans do yearn for pointers to the real, to deeper awareness, rather than simply signifiers of social inclusion and exclusion.

 

The Internet of Things

GE is building up a huge new research center in the East Bay, near Silicon Valley, says this NYT article, because the same thing that powers sharing trivia on Facebook could also share much more significant data in the real economy. The industrial Internet is being born.

.. G.E.’s effort, analysts say, shows that Internet-era technology is ready to sweep through the industrial economy much as the consumer Internet has transformed media, communications and advertising over the last decade.

The key is adding sensors to machines, to help make them more intelligent.

Today, G.E. is putting sensors on everything, be it a gas turbine or a hospital bed. The mission of the engineers in San Ramon is to design the software for gathering data, and the clever algorithms for sifting through it for cost savings and productivity gains. Across the industries it covers, G.E. estimates such efficiency opportunities at as much as $150 billion.

I think this is very interesting, but the question is whether it will prove to be about transformation and new business models, not simply efficiency. There is a certain amount of big data "me-too-ism" about it.

I was watching an interview between Jeff Bezos and Charlie Rose the other day. Bezos said they never liked to enter a market just because they could. There isn't much money to be made doing the same thing as everyone else, entering a market late a in "me-too" way.

GE may be very good at efficiency, but I wonder if they can innovate as well. We've seen before that the rise of the modern world cannot be explained by greater efficiency, but by doing things in new ways. And I say that as a GE shareholder. Their share price is still near half what it was before the crisis.

They have such vast scale that more efficiency means millions of dollars for even tiny gains, and that could ripple across the economy in power and transportation sectors. But we need new possibilities as well.


 

Leftovers now in freezer

Coming out of a turkey and cranberry and sweet potato zone this morning, after several days cooking and eating and watching movies together on the couch.

We did a variation on our usual recipes this year, using the Thanksgiving section in Bobby Flay Cooks American: Great Regional Recipes with Sizzling New Flavors for a change - turkey with pomegranate molasses glaze, gravy with a bit of chipotle. It worked really well.

Now we just have to motivate ourselves to get to the gym in recompense.

 

Futility and stock-picking

Active managers have had another bad year, says CNBC. Traditional stock-picking and bottom-up analysis doesn't work any more.

 

Just as in 2011, only about 1 in 5 active managers are beating their benchmarks in a year marked by the same type of headline volatility caused by events in Europe and fiscal concerns closer to home.

While the advantage of passive over active is nothing new, the near-record level of futility is, and the cracks are beginning to show.

[...]

"The market is being driven by macro factors," Flam said. "So most professional advisors have a background in evaluating companies, industries, economies. It's not in politics, and politics is what dominating the markets over the last couple of years."

Political factors are about decisions and perception, not ratios.

What is good for the individual...

Robert Samuelson says we need to recognize America has a massive welfare state.

The welfare state's great contradiction -- the reason its politics are so messy -- is that what seems good for the individual is not, when multiplied by thousands or millions of cases, always good for society. Politicians appeal to individuals who vote, but in doing so may shortchange the nation. Most obviously: The welfare state's costs may depress economic growth.

The need is not to dismantle the welfare state but to modernize it gradually, preserving its virtues, minimizing its vices and not doing it abruptly so as to derail the recovery. But first we need to admit it exists.

What seems good for the individual is not always good for society. That is a very difficult issue we like to avoid.

 

Mobs and Marvels

The other end of the scale from medieval mobs burning schools in Pakistan: Microsoft demonstrates something just like a Star Trek Voice Replicator.

It could be the next best thing to learning a new language. Microsoft researchers have demonstrated software that translates spoken English into spoken Chinese almost instantly, while preserving the unique cadence of the speaker's voice—a trick that could make conversation more effective and personal.

 

Pakistan slowly melts

Meanwhile, grim signs in Pakistan. A family spent thirty years building one of the best girls schools in Lahore. The son of the family,a teacher in his thirties, was particlarly passionate about talking to many groups about astronomy.

But on 31 October the school was burned to the ground by a crowd who had heard it was accused of blasphemy. Lab equipment and computers were looted. Hundreds of library books – obviously with little use to the mob – tossed into the fire. Some even tried to pull the marble tiles off the floor

A teacher had inadvertently missed a page while photocopying, linking a sentence about the Prophet Mohammed to a chapter on begging.

This blasphemy law is devouring Pakistani society from within. It is an all-purpose tool in the service of intolerance. It has often been used against religious minorities, but Muslims are paying the price as well. The repeal of the law, unfortunately, is unlikely. Some voices critical of the law have already been silenced by intimidation and violence, such as the assassination of the governor of Punjab, Salmaan Taseer in 2011.

What a terrible story. A country of 190 million is slowly sliding into a civilizational abyss. I was in Islamabad, Rawalpindi and some of the northern areas about fifteen years ago, and even then travel to Karachi or even main roads toward the south was not advised.

The Islamist methods used against the west, such as suicide bombings and oversensitivity, are now having their worst effects at home.

Demography and the election

I'm still working through some of the implications of the election, although cautious about drawing conclusions too fast. Perhaps the most widespread immediate explanation is that Republicans can't win now because of changing demographics, especially more Latinos.

It's not actually clear that this explains this election, at any rate. According to RealClearPolitics, the absolute numbers of minority votes is likely to have hardly risen at all once postal ballots are counted and final results are in. The big difference this time is five million fewer white voters turned up at the polls. Obama did not suffer as much disaffection from disappointed liberal constituencies like students, but many whites seemed disaffected with Romney but unwilling to vote for Obama. I think this is not yet proven yet , but the data bears watching. And in any case, I think immigration reform is exaggerated as a means to Latino votes. Poorer working-class Latinos are likely to vote Democratic for a generation or two, no matter what Republicans do.

Megan McCardle doubts talk of the emerging democratic majority is justified, partly because we've been hearing it since the 1990s and it has not stopped GOP blowouts like 2010, and partly because the Democratic coalition itself is likely to fracture. Most Latinos are white, and could behave like the many Irish and Italian Catholics who fled the party since Reagan. There could also be massive conflicts between unions and other parts of the Democratic coalition.

Don't take this for a "Hey, GOP, everything's fine! Don't you go changing!" I've been saying for years that the GOP has run tax cuts out as a campaign plank--indeed, they're now over the cliff and about to plunge while Roadrunner chortles. .. . And they've now nominated two candidates who have put forward almost nothing that couldn't be found in Reagan's 1980 platform. The party desperately needs some new ideas to sell to the American public.

But I am highly skeptical that last night means they've gone into some sort of permanent decline. It was a close election in which Obama lost states that he carried in 2012. The Democratic bench is very weak--the current leading candidates to succeed Obama, Hillary Clinton and Joe Biden, will be 69 and 74 in 2016. And Obama is going to have to preside over some very, very tough choices. We can't borrow a trillion dollars a year for another four years. Nor can we get all the money from Republican constituencies; they just don't have enough of the stuff. Whoever's ox Obama chooses to gore will probably be a considerably less enthusiastic coalition member come 2016.

This is probably true.

You could have argued the traditional Yankee Calvinists were supplanted in Massachusetts in the nineteenth century, and much of the American Protestant mainstream by Catholic and Jewish immigration in the early twentieth century. All are now lumped together as "white" by the liberal press. This has been going on a long time, and assimmilation tends to grind away the differences over time. The Democratic Party tends to cater to the less assimilated, but that is historically a moving target.

There is a much more worrying possibility, though, as well. John O'Sullivan writes in National Review:

But it would be false comfort (and the kind of irresponsible optimism I detest) not to mention a darker possibility. That possibility is that whites will develop a defensive minority consciousness in response both to their statistically weaker position.

That has happened before where majorities have become minorities, and it is a “rational” response (so to speak) to this change in their condition. When their collective power was numerically unassailable, they felt able to extend generous concessions to other groups. When they feel threatened, they defend every item of privilege and resent every loss. [..]

The late Sam Huntington warned in his fine book Who Are We? that a racial concept of American identity might gain ground in the circumstances of America’s whites losing their majority status. I didn’t buy this explanation at the time. I still think it is somewhere down in the low teens of possibility. And the spread of intermarriage is one hopeful defense against it. But it cannot be dismissed entirely and thus deserves a mention alongside the sunnier prospects.

America works as a multiethnic society precisely because in principle we believe that ethnicity is less important than what unites us. E pluribus unum. An alignment along largely ethnic lines would be potentially disastrous. Look at Yugoslavia or Nigeria or Northern Ireland or the collapse of the Austro-Hungarian empire or the current Middle East , such as Syria, for where that leads. Diversity can quickly turn into conflict in the wrong circumstances.

The melting pot made ethnicity less important in the past. A racial spoils system could enflame differences if the stakes - i.e. the whole future of the country - are as high as the liberal press claims.

I said before the election the left often prefers the romantic dream that turns to darkness. History offers many examples of instances where changing demographics produces resistance and conflict, not a liberal paradise. Liberals would be much better off emphasizing differences of ideas as the basis for political coalitions, not ethnicity or race. And everyone would be better off thinking about what is good for the country as a whole, not particular sectional interests.

 

 

Blaming a whale (and narrative) for defeat

From a David Brooks exchange with Gail Collins in the NYT:

David: This might be a good time for Republicans to redouble their commitment to the reality-based community. Did you see Byron York's reporting from inside the Romney campaign? They apparently had this giant computer model called Orca - named after a whale because it was bigger than anything the Democrats could imagine. It processed huge amounts of data and late in the day was still projecting a Romney victory until its head exploded. Garbage in. Garbage out.

Gail: The Republican Orca - stop me before I fall into a great pile of Moby Dick analogies.

I think that some people are a little transfixed by Nate Silver's accurate projection of the election, despite, as I've noted, his own much more self-awade mature grasp of the limits of models.

I think this is at root a particular problem with journalism.

Or, to put it more precisely, it is a problem with narrative. Human beings have a hardwired fascination with stories, probably dating from sitting around fires in the savannah fifty thousand years ago. It is often the prime way we transmit culture and values and pointers for behavior we should admire.

But stories can be too trite, and this is where journalism becomes very hedgehog-like sometimes. It is all to easy to link stories into an appealing broader narrative, to instinctively frame things in a way which makes for an entertaining read. Stories are much more interesting than data. Once you go looking for "stories", as opposed to drier mechanical reporting of facts, you introduce potential blindspots. Things can fit all too neatly into narrative buckets.

Journalism thrives on stories. Journalists are paid to look for them, rather than dry, ambiguous, complicated truth in isolation.

We've also seen that hedgehogs typically get more media attention, because big striking claims typically make better tv or better stories. Hedgehogs want an audience for their "one big thing." Journalists want a story, And "triumph of models" is itself a good story.

Of course, at the same time as the fivethirtyeight model did well, the Romney and other models performed badly. But why let that get in the way of a good story?

Self-awareness

This is the real explanation of Silver's greater success here. It is not so much a data-driven approach, as using the data to confront your presuppositions. That is why he does better than most journalists and opinion pundits.

It is not, as the story linked above puts it, simply a matter of primitive punditry against spanking new "data driven rationality".

The scientific method is at root about testing hypotheses, not mining data. You test a prediction or explanation against reality, and you revise your views if necessary based on the outcome. You can selectively use data to confirm all kinds of things if you are not careful. So you need to have a falsifiable hypothesis, a situation where at least in principle you may be forced to revise your view.

There's nothing in narrative that compels you to change your view, because you can always add a twist in the story. Indeed, most good stories will have the hero endure many setbacks and misunderstandings before being proved right in the end.

Even in science, as Thomas Kuhn famously pointed out in The Structure of Scientific Revolutions, conflicting data is often left aside as a "puzzle" or "anomaly" in periods of normal science until there is a sudden change of awareness: I.e. the "paradigm shift" which has become so overused a term it is almost a clichė. Even in the hard sciences, it can take a generation for data to settle arguments one way or another. Theories can sprout dozens of ad-hoc adjustments.

So data can be part of this process of self-awareness. But in a world where we have a blizzard of often conflicting and imperfect data, it may equally reinforce entrenched views.

Romney's big Moby Dick model was probably highly sophisticated, but likely being used to mine data more efficiently, not testing against reality.

Innumerate journalists may regard the statistical models as some sort of new semi-deity, partly because they don't understand it, or the many ways in which statistics can be used to obscure. (There is a famous book called How to Lie with Statistics).

But the real message is not statistics as some kind of new rational technique which guarantees prophetic success. Instead, it is self-awareness.


You need devices to help you retain self-awareness, to see things as they really are instead of ever-more elaborate stories or models. And humans typically find that very hard to do. I've got a few more things to say about Silver's book in that light.