The next 'Big Data' shift

The “Big Data” trend isn't going anywhere in a hurry and while there is hardly a business out there that doesn't recognise the value of tapping into this well of information to get a snapshot of what the market is thinking, it isn't exactly the simplest thing to do.

The first impediment for businesses is that there is simply too much data out there and the pile is only getting bigger. We are at a point where data is doubling every 72 hours and it is coming from multiple sources, so making sense of all of this data requires a company to bring in the likes of SAP, Oracle, SAS Institute IBM and Microsoft to start crunching the numbers and deliver insight.

The other big issue is that a lot of the data out there is unstructured and uncertain. Speaking at the IBM Business Analytics Forum 2012 in Melbourne last month the senior manager of the IBM Research and Development – Australia lab, Dr Jürg von Känel pointed out that the hardest thing is often figuring out what data is useable and what's not.

According to Dr Känel, this uncertainty comes not only from the hardware side of things (sensors) but also on the input side - the difference between what you want to type and what you actually type. Apart from these inaccuracies, not everyone is always happy to reveal all the information.  Even if a company has the perfect set of customers data, it will deteriorate at a rate of two per cent a year.

“If you don't do anything with the data it actually starts to become more and more uncertain.” Dr Känel says.

Given these complexities, it's never been a better time to be in the business intelligence, analytics and performance management business.

According to Gartner, worldwide business intelligence (BI) platform, analytic applications and performance management (PM) software revenue ticked past  $US12.2 billion in 2011. The sector was the second-fastest growing segment in the overall worldwide enterprise software market last year.

When it comes to BI and analytics, CIOs are still more than happy to splash the cash despite tighter budgets. That's a good indicator of where they think the competitive advantage lies and the push by vendors to demystify the analytics process and give customers access to self-service data discovery tools is certainly hitting the sweet spot at the moment.

According to Gartner principal analyst Dan Sommer, customers are starting to take a more balanced approach to using the technology mix available to them with all three sub-segments of the market exhibiting even growth.

Worldwide BI, Analytics and PM Revenue Estimates for 2011 by Sub-segment (Millions of USD)

Source: Gartner (March 2012)

 "This goes to show that clients prefer a balanced approach to sourcing across a portfolio of technologies, rather than focusing on just one sub-segment," Sommer says.

 "It's not a build or buy decision; it's both."

This shift in behaviour has not gone unnoticed by vendors who are starting to develop products in tune with the mobility and cloud phenomenon. While the market is dominated by the five major players, it's by no means a closed shop and there are hundreds of innovative outfits all trying to carve out a niche for themselves.

Market leader SAP overnight unveiled its latest offering SAP BusinessObjects Predictive Analytics and the product is a god example of how vendors are adapting to the changing conditions.

Predictive analytics, as the name suggests, is all about using statistical analysis to extract information and predict future trends and behaviour patterns. It's already being touted as the next big thing in BI and analytics market and SAP has certainly decided to strike while the iron is hot.

The segment has traditionally been the domain of statisticians and data scientists, which is just about as far as you can get from the boardroom, but SAP is aiming to deliver the same sort of functionality to the hands of the average business user.

According to Sanjiv Bansal from the SAP ANZ Centre of Excellence and Business Intelligence, the combination of dashboard visualisation features and the ability to process data in real time (through SAP HANA platform) is just the sort of tool that businesses are increasingly gravitating towards.

“With the software, analysts can create models either by using the predictive algorithms or integrate and use algorithms from the widely-used R open-source data analysis and statistical language,” Bansal says.

The sentiment is echoed by SAP's vice president and general manager of business intelligence solutions Mani Gill who said in a blog post that the layering of BusinessObjects on top of SAP HANA is the “one-two punch”  that gives users the speed and power to build their predictive strategy and use “Big Data” in real-time.

SAP is clearly hoping that jumping on the predictive analytics bandwagon will pay off and making the technology a little more accessible to the average user seems a sound strategy.  While predictive analytics is increasingly shaping up as the new vendor battleground, Bansal says that the next big shift in the sector is going to be around melding real-time analytics with mobility  with applications powered by In-memory computing that are lighting fast and capable of handling massive data sets.

 There is no denying the potential benefits of BI and analytics and the fact that the volume of data is only going to get bigger and more diverse. However, just how much advantage businesses can actually derive from the technology still depends on how much trust decision makers are willing to put on the extracted results.

All the signs right now are pointing towards a greater sense of urgency and a concerted strategy within the boardrooms. The addition of greater accessibility should only make the message clearer.