Economics fail: the consequences of dodgy data

If we needed more evidence that economics is not a science, we have it.

A shockwave hit the economics world this week when two of its most famous practitioners - Kenneth Rogoff and Carmen Reinhart - were found to have produced some very dodgy data to support their claims about the consequences of high government debt.

It comes back to a research paper of theirs, Growth in a Time of Debt, widely quoted since it was published in 2010.

The paper says that if government debt becomes too high - say, about 90 per cent of gross domestic product - then economic growth will almost always suffer.

Global policymakers have taken it to mean that if countries with too-high debt levels want to kick-start their flagging economies, then they ought to begin the resuscitation process by reducing their debt levels first.

It has been repackaged into a simple message: reduce your debt levels and economic growth will begin to pick up.

But the corollary of that message is that highly indebted governments should not try to spend their way out of economic stagnation because spending more money will only make things worse.

It has helped to provide the intellectual justification that the proponents of austerity wanted; thus the wave of austerity policies washing around the world since 2010.

Millions of people have suffered because of it.

But the intellectual edifice for the global austerity movement was severely weakened this week after it emerged that Reinhart and Rogoff, both professors, had made some basic errors in their interpretation of data that supported their research.

The errors were discovered by Thomas Herndon, a student in the University of Massachusetts Amherst's doctoral program in economics. He published a paper explaining what he had discovered, with help from two of his teachers, Michael Ash and Robert Pollin.

The paper shows that Reinhart and Rogoff had omitted data, made a simple mistake in their Excel spreadsheet, and used a bizarre statistical methodology, all of which skewed their results.

It sent the academic world ablaze.

As Nobel laureate Paul Krugman wrote in response: "In this age of information, maths errors can lead to disaster. NASA's Mars Orbiter crashed because engineers forgot to convert to metric measurements; JP Morgan Chase's 'London Whale' venture went bad because modellers divided a sum instead of an average.

"So, did an Excel coding error destroy the economies of the Western world?"

Reinhart and Rogoff, in a joint statement, have acknowledged that they made a spreadsheet error, but they also said it didn't affect their result much.

"It is sobering that such an error slipped into one of our papers despite our best efforts to be consistently careful," they said. "We do not, however, believe this regrettable slip affects in any significant way the central message of the paper or that in our subsequent work."

But in the brouhaha that has followed, a few people have been asking a very good question: why has it taken so long for Reinhart and Rogoff's research to be tested?

Imagine you've handed your assignment in at school. You make some wonderful claims in it about the way the world works.

Your research - based on an analysis of data of 44 countries spanning 200 years - has led you to discover that high government debt to GDP ratios above a "90 per cent threshold" almost always lead to a slowdown in economic growth.

It's a simple law that seems to hold no matter what you throw at it. You can compare different countries in disparate regions, and once you try to take account of the fact that a country's political and financial systems evolve over time, you can mix and match these things across centuries of data and the law remains the same.

It's a striking thesis. And luckily for you, you're not expected to hand your data in with your assignment so that your work can be checked. Your teacher simply takes your word for it.

That's not how the scientific method is supposed to work.

Some economists, such as L.Randall Wray of the Levy Institute, say they have written to Reinhart and Rogoff in the past to ask for their data, but they have been rebuffed.

"They ignored our request. I have heard from some several other researchers that Reinhart and Rogoff also ignored their repeated requests for the data," Wray wrote this week.

It is sobering to be reminded that economic analyses, produced in this way, can have such influence in the real world.

It's worth remembering next time we hear some politician referring to "economic modelling" that supports his or her claim.