Flawedonomics

January 26, 2012

From Freakonomics, Cold Fusion, 2010

From American Scientist:

The nonfiction publishing phenomenon known as Freakonomics has passed its sixth anniversary. The original book, which used ideas from statistics and economics to explore real-world problems, was an instant bestseller. By 2011, it had sold more than four million copies worldwide, and it has sprouted a franchise, which includes a bestselling sequel, SuperFreakonomics; an occasional column in the New York Times Magazine; a popular blog; and a documentary film. The word “freakonomics” has come to stand for a light-hearted and contrarian, yet rigorous and quantitative, way of looking at the world.

The faces of Freakonomics are Steven D. Levitt, an award-winning professor of economics at the University of Chicago, and Stephen J. Dubner, a widely published New York–based journalist. Levitt is celebrated for using data and statistics to solve an array of problems not typically associated with economics. Dubner has perfected the formula for conveying the excitement of Levitt’s research—and of the growing body of work by his collaborators and followers. On the heels of Freakonomics, the pop-economics or pop-statistics genre has attracted a surge of interest, with more authors adopting an anecdotal, narrative style.

As the authors of statistics-themed books for general audiences, we can attest that Levitt and Dubner’s success is not easily attained. And as teachers of statistics, we recognize the challenge of creating interest in the subject without resorting to clichéd examples such as baseball averages, movie grosses and political polls. The other side of this challenge, though, is presenting ideas in interesting ways without oversimplifying them or misleading readers. We and others have noted a discouraging tendency in the Freakonomics body of work to present speculative or even erroneous claims with an air of certainty. Considering such problems yields useful lessons for those who wish to popularize statistical ideas.

…

The climate-change dustup: Rendering research conducted by others is much more challenging than explaining your own work, especially if the topic lies outside your domain of expertise. The climate-change chapter in SuperFreakonomics is a case in point. In it, Levitt and Dubner throw their weight behind geoengineering, a climate-remediation concept championed at the time by Nathan Myhrvold, a billionaire and former chief technology officer of Microsoft. Unfortunately, having moved outside the comfort zone of his own research, Levitt is in no better a position to evaluate Myhrvold’s proposal than we are.

When an actual expert, University of Chicago climate scientist Raymond Pierrehumbert, questioned the claims in Levitt and Dubner’s writing on climate, Levitt retorted that he enjoyed Pierrehumbert’s “intentional misreading” of the chapter. Referring to his own writings on the subject, Levitt wrote, “I’m not sure why that is blasphemy.” We’re not sure on this point either—we could not find a place where Pierrehumbert described Levitt’s writings in those terms. It is easy to be preemptively defensive of one’s own work, or of researchers whose work one has covered. Viewing alternative points of view as useful rather than threatening can help take the sting out of critiques. And if you’re covering subject matter outside your expertise, it pays to get second—and third and fourth—opinions.