Time for a Big Data diet

The current buzz around ‘Big Data' is almost big enough to give the ‘Cloud' a run for its money but amid the grand promises that seem to accompany every piece of technology a clear definition of just what makes data ‘Big' can be hard to find.

Well, the simplest description would be that if the amount of data that you need to process exceeds the capability of your database systems, you've got Big Data. This may seem a tad simplistic but it is accurate. Volume is really the name of the game given the massive amount of data generated and collected by organisations every day.

This volume is closely aligned to the other two Vs in this equation– velocity and variety. Velocity refers to the speed at which the data can be analysed and variety points to the multitude of data points from where the information is flowing in.

However, there are a couple of other Vs that could just as easily be inserted into the scenario. Firstly, the vendors, who are increasingly playing a crucial role in driving the big data narrative and perhaps more importantly value.

Big appetite but what's next?

The promised benefits of big data and analytics are manifold but can only be truly realised once the time travelled between the collection of the data and the delivery of insight is reduced to a minimum. The journey of creating this value can often be a difficult one for organisations and according to the ANZ head of IT consulting firm Accenture's marketing & analytics practice, Jason Juma Ross, many Australian organisations are headed down the wrong road.

With 15 years of experience in the analytics space Ross has seen the big data story shift in an out of public attention before but he reckons that the capabilities available to organisations today have never been better.

“We have had a couple of goes at this, there was a big push eight years ago when everyone was building data centres, and we had another go five or six years ago with the series of books by Tom Davenport,” says Ross.

Of course some, let's just bear in mind that the current momentum behind the trend does come, to some extent, on the back of the significant marketing efforts of vendors. Nevertheless, there is an undeniable appetite for analytics in Australia and organisations are starting to ask the really important question: we have the capability so what's next?

Time for a data diet

It's a pertinent question because as the flow of data continues to grow it puts a greater burden on organisations. The problem isn't just how do you crunch the data but more importantly how do you make sense of it. Effective analytics is really about separating the data that needs to ignored and data that's useful. Unfortunately this quandary can end up consuming vast resources and Ross recommends that many organisations could do with going on a data diet.

The default position of collecting all the data just to be on the safe side is usually a reflection of a misguided strategy. Hoarding all the data under the sun amounts to very little unless an organisation can derive some value from it.

“What organisations need to do is take that next step and figure out what's useful from an analytics perspective and where they should be focusing their investment.” Ross says.

One major issue, according to Ross, is that executives often spend more time perfecting techniques and diagnosing current problems rather than getting analytics to achieve something in the real world.

“If an executive can address the analytics investment against a tangible customer or a business need then they are on the right track.”

“If it's just for the sake of having a beautiful model, then there is a big question mark.”

Getting the timing right

Another key thing to keep in mind is that there is no one size fits all approach when it comes to big data and analytics. Different verticals have their individual needs and devising a strategy that encourages the implementation of targeted solutions is critical.

For a telco or a financial service provider with virtualised product sets then the focus can often be narrowed down to an individual need, like monitoring churn or the efficiency of online channels. For a retailer, the analytics can deliver benefits with regards to controlling inventory and product innovation. Again the achievable benefits are very real but organisations need to ensure that the right decision is made at the right time.

According to Ross, analytics isn't always about making the most accurate decision but rather about making the best possible decision faster than the competition.

“If you take 12 months to make a decision, it might be perfectly right, but it might also be too late.”

For a lot of organisation working on their own platforms and solutions this can mean a lot of wasted time and money. This is where the cloud can play a role in cutting this cycle short with organisations using on tap cloud analytics as a stop gap measure until their purpose-built analytics platform is built.

Finding the right answer faster 

The big data story in Australia is taking shape rapidly with some sectors (telco, financial services) already quite mature in their approach and capability. On the other end of the spectrum the retail sector is to some extent still stuck in the old market-research model of analytics.

Given how much real time analytics has to offer to the retail sector, especially in the current environment, you would think this would be a no-brainer but there are evidently some organisations with analytics departments out there that are still trying to get their heads around the trend and finding the right solution.

As Ross points out analytics folks tend to be an academic lot, more eager to find the most accurate answer rather than looking at the watch.

Unfortunately in a world dictated by speed and agility the onus now is on finding the right answer faster.