The end of the 'old school' weatherman?

After years of tinkering, applied mathematicians say their weather algorithms based on historical data are now better at forecasting than traditional meteorologists, who rely more on current patterns.


Computer scientists are picking a new fight with old school meteorologists, claiming finally to have cracked the code on weather forecasting at a pivotal, profitable moment for the field, as climate change roils commodities markets and industries.

Banks and traders are reporting outsized profits, and losses, on everything from natural gas to grains as severe weather causes extra price volatility; power grid operators are struggling with bouts of extreme cold or droughts that crimp supplies while demand spikes; and more and more retailers and manufacturers are using forecasts to manage inventories.

Traditional meteorologists, who look at current weather patterns to make forecasts, have long derided examining historical temperatures as "climatology", of limited use, at best, when trying to predict the future.

But applied mathematicians, some of whom once worked on Wall Street as market-predicting "quants," see the future in patterns of historical data. After years of tinkering, they say their weather algorithms can blow away traditional forecasting.

"It has taken me two solid decades to get something useful," said data miner Ria Persad, the president of StatWeather.

"Weathermen are looking at what's happening now – they are looking at current data to get to the future," said Persad. "They aren't actually studying this 120 years of data log to extract patterns like we are to draw statistical lessons."

Persad looks far ahead: she sees the California drought persisting until late 2015, so far into the future as to draw scoffs from some practitioners.

Traditional meteorologists use computer models as well, and some see value in mixing historical data with what is happening outside their window, but they are sceptical of relying too heavily on the past.

"We only have data for the last 100 years, which is 100 winters, which is a really small sample size. It would work if we had 1000 years or 10,000 years of data, but we don't," said Mike Halpert of the US National Oceanic and Atmospheric Administration's Climate Prediction Center.

"This is kind of like being a gambler in Las Vegas, on any one hand you may lose," he added, declining to discuss StatWeather specifically.

Halpert, however, had predicted this past winter was going to be warmer than normal. Instead, it was unusually cold – just as StatWeather predicted. Only about 20 percent of commercial forecasters saw the colder winter coming, Persad said.

StatWeather nailed calls on a cold snap in late 2013 and a string of frigid temperatures through March, surprising some in the forecasting community and even Persad herself. She attributed improving accuracy to her software training itself.

Another company, Global Weather Oscillations, uses historical data to predict where hurricanes will strike land and correctly predicted a weak hurricane season last year, unlike many rivals.

"We don't have to wait four days before a hurricane hits to do this. We can do it eight months into the future," said Chief Executive David Dilley, whose company sells its forecasts to insurance firms and big retailers.

Managing risks

Climate change is already causing drier droughts, more intense floods and wilder temperature swings across the United States, the National Climate Assessment said in May.

The winter of 2014, when frigid temperatures roiled natural gas markets as heating needs rose, may be a glimpse of what lies ahead.

Major trading houses, including Morgan Stanley and Goldman Sachs, hedge funds and energy producers made and lost hundreds of millions of dollars as gas futures prices spiked by more than 50 percent to a five-year high of $6.49 per million British thermal units (mmBtu) in late February. At a delivery point into New York City, spot prices rose 20-fold.

Oil giant ConocoPhillips posted some $200 million in profit during the quarter from natural gas. Texas-based hedge funds e360 and Goldfinch reportedly had gains of 14 percent and 21 percent in January, respectively, when gas spiked.

"It was a very unusual quarter because of weather," ConocoPhillips CFO Jeff Sheets told Reuters in May, describing a successful winter of gas trading. He warned the results might not be repeatable.

Commodities giant Cargill Inc's quarterly earnings fell 28 percent on market disruptions that it blamed in part on extreme weather.

Most firms active in energy markets have contracts with several forecasting companies, paying them tens of thousands of dollars a year.

StatWeather, which just moved to Houston from Florida to be closer to clients, declined to detail its roster of users and several trading houses consulted by Reuters would not identify their suppliers.

Air Liquide, which produces and buys power to distill specialised gases, said it relies on half a dozen suppliers – like StatWeather, Planalytics, DTN, Wilkens and Vaisala – that track not just temperature but also wind and in one case lightning.

The forecasts help it monitor pipeline safety, calibrate its plants based on the price and availability of power, and gauge when the Texas grid might suffer supply disruptions.

The suppliers distinguish themselves by forecast time frame; each is better at viewing a particular slice of the future, said Charles Harper, Air Liquide's global head of smart manufacturing.

As the forecast battle continues, there's one fundamental disagreement – whether human instinct plays a role in the science.

"You live by the model, you die by the model," said Marshall Wickman, senior meteorologist at Wilkens Weather, a unit of Rockwell Collins. The wise forecaster doesn't wager everything on the computer, he believes.

"That's where the meteorologist comes in. The model doesn't do everything. It's a guide," he said.

Originally published by Reuters. Reproduced with permission. 

InvestSMART FORUM: Come and meet the team

We're loading up the van and going on tour from April to June, with events on the NSW central & north coast, the QLD mid-north coast and in Perth, Adelaide, Melbourne, Sydney and Canberra. Come and meet the team and take home simple strategies that you can use to build an investment portfolio to weather any storm. Book your spot here.

Want access to our latest research and new buy ideas?

Start a free 15 day trial and gain access to our research, recommendations and market-beating model portfolios.

Sign up for free

Related Articles