Big Data's hidden hurdles
Most consider the key problem around managing Big Data to be its size, resulting in complex management strategies for organisations as they seek to process this rising volume of information at hand. But its not the only hurdle companies need to overcome to get the most out of their data.
A less talked about problem is the diversity of data. Customers interface at multiple touch points and in multiple ways. As a result this data that streams into the enterprise can be a mix of clickstreams and weblogs, call recordings, chat transcripts, and survey results.
It isn't generated as nice rows and columns that can be managed by conventional database technologies. The traditional analytics and data science developed to deal with structured rows and columns of customer data is no longer enough. New science that lies in the intersection of statistics, mathematics, computer science and linguistics has to be developed to process and derive intelligence and predictive power out of this data.
Currency, or timeliness, is another issue. All of this cumbersome data needs to be mined for decisions and insights instantly if an organisation wants to use it to improve the customer experience. Today, we are very familiar with this concept of timeliness in the world of search and information retrieval. When a person enters a term into a search engine, they expect the result to come up within seconds. Taking the request offline for analysis or delivering the results within days would render the result meaningless, as the original need would have long since passed. Organisations need to be able to act on their Big Data in real time and doing this in customer service scenarios delivers a truly intuitive customer experience.
Therefore, the first step in turning Big Data into a usable commodity is to integrate and “fuse” all the multiple sources of customer data within the enterprise. The next – and in some ways the most difficult step to come to grips with – is to leverage the different types of data in ways that allow you to reshape the customer experience.
Therefore, the key to leveraging Big Data is to be able to derive a single view of the customer that takes into account signals that are obtained from multiple sources and data types.
It can be done
Good e-retailers are a prime example of how Big Data can work to improve customer relationships and add value to a business.
Consider the seamless experience you receive from a book e-retailer such as Amazon. You log into your account and Amazon immediately recommends books that you might like based on your past purchases. You commit to buying a new e-book and as you go through the checkout process, Amazon makes it easy. Amazon knows your credit card number, whether you want the item shipped or where you want to download it and all your other preferences. The moment the transaction is completed you turn on your Kindle and the book is there.
The experience hinges on intent. From the recommended reading list all the way to checkout, Amazon uses your sales history and the history of similar customers to guess your intent and anticipate your next move. When they get it right, the transaction is swift and you, the customer, go away satisfied.
Understanding intent
Typically, intent depends on four factors:
- Identity of the customer
- The current journey and interaction
- Current Location
- Context based on previous interactions
In the case of a web retailer a key signal could be how the customer arrived on the site - directly, by clicking on an advertisement or courtesy of a search engine. If the latter, what was the search term used? The person's IP address or mobile device provides a usable guide as to location. If the customer logs in and authenticates themselves then the accuracy of the predictions go up dramatically since you have the actual customer identity and information. Has the person been to your site before, or have they had discussions with customer service agents via other channels – provides more context to what their likely intent is going to be? As the customer moves through the site, time spent on each page provides further indications of intent and at times evolving intent.
With real time analysis an organisation can soon narrow down the likely requirements and try to engage the customer with an offer of a webchat where intent will hopefully be confirmed and met.
On conclusion of the interaction, transcripts of the chat should be mined across multiple attributes such as response time, engagement level, actual intent and outcome (did the customer buy?). All of this data helps to evolve and mature the system by building a more accurate profile about the type of customer likely to purchase through chat and adding to the system's predictive capabilities.
Intent evolves
Intent is rarely expressed accurately and/or fully in search terms. When looking for a toy for a child, I might type “Toys R Us” into a search engine. Obviously, my intent is not the company but it's where I choose to start. What's more, intent evolves as the online journey progresses and it can be shaped.
Our experience of mining data across many companies suggests that around 70 per cent of visitors specify intent generically, not specifically, when they visit a website. For example, they may type “computer” but rarely their true requirement i.e. “a laptop for day to day personal browsing” or a “high-end server with lot of ram and storage”. Of the 30 per cent who do conduct a specific product search, research shows that one-quarter still don't buy the product they started out looking for. Chat on the other hand is a true expression of intent since the user is not constrained by word limit unlike search. In addition, the user is being asked to reveal their intent by customer service representative through probing questions. Marrying chat transcripts to customers' online journey provides a much deeper understanding of the relationships between search, online journey and intent.
For example, in the case of a computer manufacturer and retailer, mining online webchats and integrating them with the corresponding online journey revealed that 27 per cent of people visiting a particular product page were only looking for accessories. The first lesson learned from this was that the accessories page was not prominent enough for visitors. Data mining helped to glean another lesson. It identified among other attributes, that speed of browsing of web pages was a key attribute determining if a person seeks to buy a given product or a related accessory. This has allowed the company's webchat agents to better understand when they should intervene to add value and how to respond.
When webchat adds most value
Analysis of search data shows that people who arrive at a site through an organic search (one based on relevance rather than sponsorship or advertising) are over 60% per cent more likely to respond positively to an invitation to a webchat. On top of that, people who do engage in webchat are 55% more likely to buy.
The organic search is likely to be carried out by customers who are still researching their product or service choice. They welcome webchat as a way of gaining more information about the options. Conversely, people arriving due to an ad are more likely to already be interested in the brand or product category. They will have an idea of the information they are after and a chat is unlikely to add a great deal of incremental value.
Casual browsers may often chat but not during that web session. However, tracking these visitors shows that those who end up chatting have a higher propensity to return to the site (as a repeat visitor), and a significantly increased probability of conversion/purchase.
The responses of customers on your own site will form their own patterns, but with good data mining it becomes easier to work out who wants help, who you can successfully engage, when is the best time to engage and which customers you should leave alone.
In summary the best approach is predict, experiment, measure and learn. The key to success in mining Big Data is in being able to learn at scale and in continuously adapting your intent models and real time engagements with the customer to match these learnings. Key responses or variables to optimise are the ones that business truly cares about - revenues generated and measures of customer experience such as customer satisfaction and Net Promoter Score.
Ananth Siva is the MD of [24]7's Asia Pacific division.