Making sense of the data deluge

As the volume of data available to businesses continues to grow, effective content management strategies must provide organisations with the ability to rapidly analyse data and identify what's important and what's not.

As the volume of data available to businesses continues to grow rapidly making the most of this data explosion requires careful planning. Effective enterprise content management (ECM) strategies must provide organisations with the ability to rapidly analyse data and identify what does and does not need to be stored. 

The data explosion is being propelled by three main drivers. Firstly, we now gather and store content from a vastly expanded number of sources, ranging from the boom in mobile devices and social media to ubiquitous tracking and sensor data derived from core business processes. Second, each of these sources generates far more content than ever before, thanks to steady declines in storage and transfer costs. Finally, growing legislation and policy around information management renders businesses increasingly unwilling to discard historical data. 

As businesses generate more and more unstructured content, they will require more intelligent and consolidated infrastructure to capture and access it in all forms. However, this infrastructure must be complemented by a structured approach to content governance to reach its full potential. 

Capture everything, don’t keep it all

First of all, businesses must take steps to capture all content which is relevant to their operations and outcomes. This content now comes in many forms at many speeds, which can result in organisational overload if not correctly managed; conversely, businesses which fail to capture the entirety of this content deprive themselves of a key strategic resource. A strong ECM system analyses content from the full range of sources related to the organisation, including both unstructured (like social media chatter or help-desk calls) and structured content (like back-end process data). This content should ideally  be stored in a system which allows users to access, filter and analyse all information relevant to their particular goals.

However, traditional content management approaches which simply store all captured data can no longer cope with greatly increased data volumes. Simply adding storage media or increasing data compression only exacerbates the problem: the more content a business stores, the harder it becomes to unlock the insights it contains.

The resulting siloed approaches can lead to greatly reduced clarity from both analytic and compliance perspectives, negating the very value of storing content in the first place. Most businesses already experience 30 to 40 per cent year-on-year storage growth, and simply adhering to business-as-usual ECM will only see this sustained burden grow in coming years.The challenge for businesses is twofold: store useful content in an easily-accessible manner, while reducing overall volume by disposing of excessive data.

This disposal process must be strategic so as not to lose potentially useful business insights; it must also be defensible so as not to risk legal action for non-compliance (particularly in areas of communication like email archives or customer call data).

Best-practice content governance

Deciding whether to keep or dispose of content should ultimately stem from the business’ goals. Building customer retention and reducing churn may be best served by storing call detail records; marketing departments may turn insights gleaned from social media content into more targeted, effective campaigns. Core to well-governed ECM is an understanding of what content is relevant to current and future business goals. By then prioritising certain data over others, organisations can avoid uncontrolled expansion in their stored content, reducing policy risks and operational costs in the process.

This is relatively simple when dealing with structured data, but what about the far greater volume of content which is unstructured? Almost 80 per cent of information now falls into this category, and with 15 petabytes of information generated daily, bringing unstructured content into decision-making constitutes a growing challenge for most organisations. New technologies such as natural-language processing (NLP), sentiment analysis and intelligent search mechanisms offer powerful complements to core ECM systems. However, their use must be governed by a clear focus on business priorities and adding value to existing processes.

As big data continues to grow, effective enterprise content management must focus first on well-defined policies around content capture, storage and disposal. Effective analysis tools help reduce the volume of information which the business stores, while improving how this information is used for business outcomes. This also dramatically reduces the legal and regulatory risks which can easily grow alongside the volume of data. A well-governed ECM approach to storage not only reduces costs and inefficiencies, but also allows businesses to focus on generating meaningful insights out of available information. 

Tass Melissinos is the Enterprise Content Management (ECM) solutions manager at IBM Australia and New Zealand.