Facebook data could lead to new insights on human behavior?

I'm not sure whether this is creepy or exciting. Probably creepy, given it's Facebook.

Private conversations, family photos, and records of road trips, births, marriages, and deaths all stream into the company's servers and lodge there. Facebook has collected the most extensive data set ever assembled on human social behavior. Some of your personal information is probably part of it.

And yet, even as Facebook has embedded itself into modern life, it hasn't actually done that much with what it knows about us. Now that the company has gone public, the pressure to develop new sources of profit (see "The Facebook Fallacy") is likely to force it to do more with its hoard of information. That stash of data looms like an oversize shadow over what today is a modest online advertising business, worrying privacy-conscious Web users (see "Few Privacy Regulations Inhibit Facebook") and rivals such as Google. Everyone has a feeling that this unprecedented resource will yield something big, but nobody knows quite what.

So they have a 12-strong social science team trying to figure out just that.

Not all the possibilities are bad. For example, this is quite amazing if true:

In April, influenced in part by conversations over dinner with his med-student girlfriend (now his wife), Zuckerberg decided that he should use social influence within Facebook to increase organ donor registrations. Users were given an opportunity to click a box on their Timeline pages to signal that they were registered donors, which triggered a notification to their friends. The new feature started a cascade of social pressure, and organ donor enrollment increased by a factor of 23 across 44 states.

But I'm also a little skeptical that "big data" will live up to the hype surrounding it. I doubt it will generate genuinely new or useful insights into people's behavior in general, or allow social influence which is durably different to the real world. At most, it will give more information on weak ties and how they impact us.

It might predict given enough data when people are going to buy a car or announce an engagement, or the birth of a child, however. Although the more it targets people accurately in commercial terms, the less it may be trusted.

It reminds me of this fascinating NYT piece a few months ago :

 

As the marketers explained to Pole — and as Pole later explained to me, back when we were still speaking and before Target told him to stop — new parents are a retailer’s holy grail. Most shoppers don’t buy everything they need at one store. Instead, they buy groceries at the grocery store and toys at the toy store, and they visit Target only when they need certain items they associate with Target — cleaning supplies, say, or new socks or a six-month supply of toilet paper. But Target sells everything from milk to stuffed animals to lawn furniture to electronics, so one of the company’s primary goals is convincing customers that the only store they need is Target. But it’s a tough message to get across, even with the most ingenious ad campaigns, because once consumers’ shopping habits are ingrained, it’s incredibly difficult to change them.

 

There are, however, some brief periods in a person’s life when old routines fall apart and buying habits are suddenly in flux. One of those moments — the moment, really — is right around the birth of a child, when parents are exhausted and overwhelmed and their shopping patterns and brand loyalties are up for grabs. But as Target’s marketers explained to Pole, timing is everything. Because birth records are usually public, the moment a couple have a new baby, they are almost instantaneously barraged with offers and incentives and advertisements from all sorts of companies. Which means that the key is to reach them earlier, before any other retailers know a baby is on the way. Specifically, the marketers said they wanted to send specially designed ads to women in their second trimester, which is when most expectant mothers begin buying all sorts of new things, like prenatal vitamins and maternity clothing. “Can you give us a list?” the marketers asked.

So this kind of short-run prediction can still be valuable in dollar terms, even if it does not produce a revolution in understanding human nature that eluded Shakespeare or Dostoyevsky.

 

Learning to change

We were just discussing online education in the last post. And this morning in the NYT Thomas Friedman talks about the education revolution.

Andrew Ng is a Stanford Professor the founder of Coursera, a new online education company that is offering certificates taught by professors from the leading schoools .

 

I normally teach 400 students,” Ng explained, but last semester he taught 100,000 in an online course on machine learning. “To reach that many students before,” he said, “I would have had to teach my normal Stanford class for 250 years.”

 

That is one heck of a leap in productivity.

Of course, it ought to mean that teachers have more time to do the higher value activity like small classes or one-to-one discussion. But it may also mean something like the kind of process Clayton Christensen describes in other industries , as we were discussing in a recent post. A relatively inferior product appears at the bottom end of a market, and then slowly improves and chews its way up through the rest of the market.

Serious change in third level education is probably still at least ten years off, however. This is going to follow the hype cycle of expecting too much too soon. For comparison, it ought to have been easier to transform academic publishing, with its absurdly high prices for journals. But the institutional momentum of existing reputations and brands is very strong. Most serious researchers still want to publish in the top referreed journals in their field.

The soaring cost of college education will be an immense force for change. But until there is solid evidence over many years that a Coursera certificate counts for something in the labor market, people will still choose to go to four year colleges if they can. It takes much longer to alter human institutions than to create technological possibilities.

And it could actually reinforce the position of the top schools. Recall the way technology meant that financial markets actually consolidated into the key centres of New York, London, and Tokyo at the expense of many small markets. If everyone has access to most of the same information, every last little edge that you get from being near the center of the action becomes ever more important.

However, it also means the basic content of the best education will be vastly more widely available as well. The courses, exams and access to the university library could be just as easily offered in a town in rural Kenya as Palo Alto.

 

Abundance and what it means

And now it's time for a quick look at another book. Abundance: The Future Is Better Than You Thinkby Peter Diamandis and Steven Kotler. It is a startling look at the potential impact of new technologies, many of which I hadn't even heard of. We'll look at some of those in the next post.

They start, though, with a bold claim:

Humanity is now entering a period of radical transformation in which technology has the potential to significantly raise the basic standards of living for every man, woman, and child on the planet. Within a generation, we will be able to provide goods and services, once reserved for the wealthy few, to any and all who need them. Or desire them. Abundance for all is actually within our grasp.

This is a remarkable claim, if true. It means that the economic problem, as Keynes put it in the essay that served as one of the starting points of this blog, will be solved within the next few years.

How can they say this? Well, they are highly experienced technologists. One is the innovator between the X-prizes for new technology, for example, and the other is a highly connected science journalist.

And from their standpoint the world is changing even faster than most people believe. Contrary to the pessimism many people habitually like to wallow in, much of the change is positive. Consider the impact that mobile phones are having in Africa:

Right now a Masai warrior with a cell phone has better mobile phone capabilities than the president of the United States did twenty-five years ago.

And if he has rudimentary access to Google on the cellphone, he has much of the world's information and knowledge, too.

This claim that the whole world is a generation or so away from solving basic needs is much more radical than anything I've claimed in this blog. Of course, I've argued that the developed countries are approaching the point where many of our problems have to do with abundance. We have satisfied most basic needs, and higher-order needs on Maslow's hierarchy aren't as easy to convert into tradable goods or services.

But I acknowledge that many poorer countries may be far off that point, and environmental or energy constraints could create addiitonal headwinds.

Diamandis and Kotler, by contrast, think that within a generation or so, all basic needs problems will more or less be solved.Technological advances will have a transformative effect on healthcare, agriculture and services. For example, electric cookstoves could reduce ecological damage, free up time spent gathering firewood and signficantly improve mulitple health problems which arise from indoor air pollution.

For example, the two-burner electric cookstove is a simple device, but it would bring magnificent change to the 3.5 billion people who now cook food and get light and heat by burning biomass: wood, dung, and crop residue.

Luxury versus possibility

That is not to say everyone will live like the inhabitants of Bel Air or South Kensington.

Abundance is not about providing everyone on this planet with a life of luxury—rather it’s about providing all with a life of possibility. To be able to live such a life requires having the basics covered and then some.

That is a definition I can easily live with, indeed strongly support. But, as we will see, the question is what to do next. And here is where the book leaves many questions unanswered. What use should we make of possibility? Will it mean mass unemployment and drift, or genuinely freeing people to flourish and live a fulfilled life?

The same applies to another very interesting idea they adopt from Matt Ridley (whose book The Rational Optimist: How Prosperity Evolves (P.S.)I strongly recommend). Prosperity is saved time.

This process, Ridley feels, creates a further feedback loop of positive gain: “Specialization encouraged innovation, because it encouraged the investment of time in a tool-making tool. That saved time, and prosperity is simply time saved, is proportional to the division of labor. The more human beings diversified as consumers and specialized as producers, and the more they then exchanged, the better off they have been, are and will be.”

Again, the question is time is saved for what. When you solve problems of basic abundance, then questions of purpose become much more important.

The Hype Cycle

Finally, one other basic idea before we turn to a few specifics about technology. One eternal problem in discussing technological change is a feeling that much o f it is "gee-whiz" speculation, just silly dreams of flying cars and martin colonies and Star Trek warp drive. What we actually got was more prosaic. Instead of interstellar travel, we got the iPod and Angry Birds. At least so far.

We are often not good at understanding the pace of change. We expect both too much and too little of technology. Diamandis and Kotler note that the research firm Gartner have developed a graphical representation of this.

.. we fall prey to what’s become known as the “hype cycle.” We have inflated expectations when a novel technology is first introduced, followed by short-term disappointment when it doesn’t live up to the hype. But this is the important part: we also consistently fail to recognize the post-hype, massively transformative nature of exponential technologies—meaning that we literally have a blind spot for the technological possibilities underlying our vision of abundance.

A 2011 edition of the Gartner chart is here.

Added to that, many technologies are exponential in character, as we've discussed before on this blog. Exponential change starts small, and looks linear. But the curve can suddenly break out.

And it’s this kind of explosion, from meager to massive and nearly overnight, that makes exponential growth so powerful. But with our local and linear brains, it’s also why such growth can be so shocking. 

Short-term disappointment and long-term transformation often tends to be the pattern of change. But the technological developments they discuss do look like they will be genuinely transformative. We'll discuss those in the next post.

 (edited to fix formatting, 5/4)