Mining a data glut

Big data can be big trouble if you don't know how to manage it. It's an issue that's causing more than a few headaches for Australian companies sitting on a mountain of information but still coming to grips with how to use it to their advantage.

This disconnect is not only putting the brakes on productivity but hurting the economy to the tune of up to four billion dollars a year according to storage vendor Hitachi Data Systems' (HDS) second bi-annual Great Information Glut report. What's alarming is that our resources sector isn't exactly immune from that malady either. If anything the nature of the industry and the commodities boom actually poses a unique set of problems for companies.

One key driver underpinning the information overload is the rapid digitisation of society which in turn has led to companies deploying more technology. As more information is generated, the need for effective data storage and management solutions grows.

According to HDS Australia and New Zealand chief technology officer, Adrian De Luca, this is a problem which is growing at a faster rate than expected.

“We are actually struggling to put sufficient strategies in place to manage the information, let alone actually harness the valuable data that is being collected.”

Right now up to 70 per cent of enterprise employees access data or information systems to do their job on a daily basis and Deloitte research released earlier this year pointed out that an efficiently implemented IT system can lift productivity by up to 21 per cent.

According to De Luca, the issues were first highlighted in a 2009 report, which showed that 34 per cent of Australian organisations were suffering from an information glut. Unfortunately, two years down the line the numbers are pretty grim with 95 per cent of the respondents in the current survey saying that they are suffering from information glut and 25 per cent saying that it was having a significant effect on organisational performance.

De Luca says things have actually gotten worse in the last two years with the number of companies affected rising five per cent.

“Companies are still grappling with the issue of truly identifying the problem and at a CFO, CIO level in some cases it takes a significant event for them to see that,” De Luca says.

“Traditionally CFOs have seen IT infrastructure as a  commodity, especially during the last couple of years of economic uncertainty, so they are more focused on reducing the dollar cost per unit of acquisition as opposed to seeing it as a strategic differentiator.”

Driving blind

The impact of this at a management level is that decision makers are often flying blind. According to the Business Analytics and Optimisation for Intelligent Enterprise study from IBM, one in three business leaders frequently make critical decisions without the information they need.

Recognising the benefit of effectively managing data is an important step for a company to gain a competitive edge. The second critical step, which is particularly important to the resources sector, is to build efficiencies into the way companies collect and analyse the data through business analytics and optimisation (BAO) technology tools.   

Resource companies produce vast streams of data and it's a combination of complex technical and qualitative data. Much of it is unstructured and growing at an exponential rate and IBM's associate partnerfor business analytics & optimisation, Fergal Murphy, says that resource companies have a greater onus when it comes to managing information effectively because the nature of resource exploitation will throw new challenges their way.

According to the Business Analytics and Optimisation for Natural Resources whitepaper, there are several key elements to effectively leverage BAO capabilities.

Organisations need to develop a strategy that delivers operational intelligence and shows performance across the organisation in real time. This in turn allows them to tether together individual islands of performance data and optimise production performance and asset maintenance.

The whitepaper also points out that budget forecasting and planning is often decentralised and hard to consolidate in resources companies. This makes it hard for managers to have a single unified view of the capacity constraints and drivers.

Smarter thinking, intelligent decision making

The commodities boom has led to a huge wave of capital projects, from the massive LNG projects underway in Gladstone to the coal exploration in the Galilee Basin, and organisations are scrambling to deal with critical  issues of cost blowouts and a shortage of skilled staff.

According to Murphy, the information challenge often manifests itself into business challenges leaving management unable to stay abreast of on the ground challenges.

“If you are not using effective analytical technology you may not be optimising the potential for the underground resources, and how you make decisions about where to place facilities, how to do the calculations around volumes and productivity,” he says.

These are complex challenges and Murphy adds that intelligent decision making is predicated on bringing smarter thinking to the table and its one of the key lessons that IBM has been teaching at its Natural resources Solution Centre in Perth.

“The objective is to develop a collaboration centre which brings business leaders in the sector together and connects them with global analytical capabilities,” Murphy says.

The basics of the resources business may be straightforward  in some respects, but finding success requires very tight management and Murphy says the real trick is to understand the “value of information” to make the volumes of data mean something.