Big Data is broken - until you see its value

When it comes to 'Big Data' one size definitely does not fit all. There's a right and a wrong way to do data analytics, according to Laurence Armiger from Zizo, who here explains why the term 'Big Data' is so over-used and vague that it has become almost meaningless.

Companies are taking existing technologies and trying to convert that technology into delivering analytics and end up using the wrong tool for the job. We live in a dynamic and rapidly changing world and people want to use analytics to improve how they're doing business so a data tool needs to be able to adapt and change quickly.

The key to a successful big data project is for a business, regardless of how big or small it is, to start from the desired business outcome. Once you know what you're trying to get out of the data, you can then decide what's the best analytical tool for the job.

Big Data, big hype?
Everyone in business seems to be talking about big data now. Many of us have sat through bombastic and over-long PowerPoint presentations about it. But relatively few businesses seem to know how to use it effectively, or be making a decent return on collecting and analysing vast amounts of sales receipts, ERP and other back-office data; plus tweets, Facebook 'likes' of brands etc.

Despite big data's patchy track record, IT directors and marketing directors face growing corporate peer pressure to invest in it. Some IT directors are dabbling with big data because they want to have it on their CV - 'I've put 2 million records into a Hadoop database' etc. - not because they think it'll help their business. Marketing is also being dragged along somewhat unwillingly because the cost and benefits of many big data projects are often vague.

Sales directors tell us that they don't care about big data because existing data warehouse and reporting structures can't give them the data they need - daily or 'real time' information on sales and stock levels.

So, why is big data falling so far short from its hype? And how can companies get better value from their data?

Business benefits
Too many big data projects we see start by searching for the right technology, dump a mass of data into it and then try to work out how it can help the business. That's the wrong way round. It's like buying land in a distant country without getting it surveyed first and then trying to work out what to do with the land.

Some retailers are trying to track customers moving round their stores via their mobile phones and RFID tags. They're putting 'beacons' on products and shelves and using their RFID tags to pick up a mobile phone signal. Consequently, one of the excuses for failure of RFID projects was that they were swamped by the data.

But this is a poor excuse. These projects are still failing even though retailers have numerous big data tools to choose from that purport to be perfectly capable of handling the data volumes. The problem lies in assigning meaning to data. It's both an intellectual and a technical problem.

Skills shortage
In a recent report on Hadoop, one of the biggest suppliers of big data tools (its open-source software lets companies use cluster of computers to analyse a vast amount of data) Gartner said future demand for the product would probably be "fairly anaemic" over the next two years. The research company said this was partly due to a shortage of people who know how to use Hadoop and sluggish adoption by business.

The human factor can also complicate big data. Analysing text in CRM and social media can be difficult because it involves interpreting customer opinions and deciding whether they are positive or negative about a brand. Computers don't do irony.

Getting value from Data Analysis
So, how can companies get bigger benefits from their data analysis? Here are five ideas:

1. The business has to say what it wants to achieve from collating and analysing data. It should lead the project; not IT whose staff can feel threatened by big data.
    
2. Don't ask questions that can't be answered by the data. Retailers may want to know how many repeat visits customers make. But if, say, 40% of their sales are cash sales rather than those made on debit or credit cards that's a big gap in sales data. Some retailers will struggle to identify individual customers in stores or customers with different cards.
    
3. Start small and then get bigger. Cut irrelevant data. If you're trying to understand all customers, do you need demographic data on them? It may not be accurate. Do you need your trial to use data on all customers or just use data from 5 or 6 stores? Bear in mind that Tesco trialled its loyalty card in a handful of stores. When the value of the data was clear it rolled out the system across its business.
    
4. Manage your costs. Pick suppliers who allow you to trial technology and will help you understand the technology.
    
5. Understand what data really matters to your business. For retailers the most common objective is to sell more products, at a better price, more often. Another common goal will be a more efficient supply chain. Be specific. The aim may be, for example, to use your data to work out how to get stock on your shelves faster; or get more short-sleeved shirts on shelves rather than scarves on a hot day.

"Having a clear business outcome makes it easier to measure the value of your data project," concluded Armiger. "Yes, many of these ideas are simply common sense but, in the rush to use big data, companies are being dazzled by technology and forgetting that it should serve the business, not the other way around."