Think Big Data: Act Small Data
Business is obsessed with data - data is the new raw material to crunch. Beliefs are being strengthened that data beats emotions. Data really powers everything we do. Being a statistician is supposed to be the sexiest job over next few decades.
And Big Data is the new buzz. Without big data analytics, companies are blind and deaf, wandering out onto the web like deer on a freeway, exclaims Moore. Peter Sondergaard asserts that while information is the oil of the 21st century, analytics is the combustion engine. Even Larry Page of Google admits his surrendering to big data, as it also sounds cooler.
According to dictionary.com, Big Data is "extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions." Big data has changed the way the companies analyse their consumers' behaviour. And it has changed our approach to target consumers and position products and services. It's the tool to reach out and influence the consumers' mind. Big Data enables marketers to keep a bird's eye view with a high level of dependency on analytics.
But then, there have been critic and cynic and pragmatic perspectives too. Ariely speaks Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it. Coase mocks, "Torture the data, and it will confess to anything". The infamous ridicule of Lang, "He uses statistics as a drunken man uses lamp posts-for support rather than for illumination", has some solid merit. Fiorina, the balanced CEO, enlightens - The goal is to turn data into information, and information into insight. 
Today we heavily rely on big data, about how becoming the number 1 Cricket world cup team by applying analytics to analyse data about the opponent players (how he get out often etc). In the same spirit, big data augments or even overrule the human judgment. Analytics based on statistical analysis of data forces marketers to reconsider their instincts. Influence of the subject matter experts is reducing in many areas, one such example is that of Amazon, where Jeff Bezos got replaced in-house book reviewers with algorithmic recommendations.
The Internet has a glut of data - finding information is like drinking from a fire hose. Big data is a massive collection of data from various sources (variety) coming at various speed (velocity), and in various quantity (volume), and social media websites are adding the huge amount of data everyday-every second. Gartner Analyst, Doug Laney estimated that the worldwide information is growing by 59% and proposed the 3Vs of Big Data - Volume, Variety and Velocity.
Big data on social media is created with all the pictures that you share, all the smileys that you post, all the videos that you share, all your likes, comments, and shares, tweets, blogs, videos, posts, links, data from GPS signals, etc., adds up to 2.5 quintillion bytes of data every day. Here are some facts about social media and big data:
1. Facebook takes 500 times more data each day than the New York Stock Exchange.
2. Twitter produce 12 times more data each day than the New York Stock Exchange.
3. Facebook tracks every computer IP a user has logged in from, and other users who have logged before and after you from the same computer up to 800 pages size of data per users.
4. More than 4 Billion hours of videos are watched on YouTube each month.
5. More than 30 billion pieces of content are shared on Facebook each day.
According to Viktor Mayer & Kenneth Cukier, big data will be a source of new economic value and innovation. In their book, "Big Data" they have identified three shifts in the way we analyse information; first we are able to analyse far more data, secondly, it's a tradeoff: with less error from sampling we can accept more measurement error. Thirdly, we will move from causality to discover patterns and correlations in the big data. While big data can tell us what's working, it isn't always able to tell us why it's working. Big data alone cannot tell us the need gap between consumers and their deep emotional connect with the brands.
Big Data is available to all and therefore the same data will give everyone the same generic insights. It's a race to the middle that can dilute brand perceptions and value. The problem with big data is manifolds for example:
1. Big Data only provides us with averages and medians
2. It emphasizes correlation, rather than causation
3. Individual customers defy the norms set by big data
4. While big data can tell us what's working, it isn't always able to tell us why
Therefore, brands that want to develop competitive advantage through differentiation can't do so with big data alone. That's where small data comes in. Martin Lindstorm in his book "Small Data" championed the concept and brought to attention the importance of small insights matter to discover consumer's motivation to buy and consume. He opinions that big data clubbed with small data are an ideal combination of correlation and causation leading to business transformation. To solve the puzzle we need to find the missing pieces, the small data found in little observations has a big impact. The observations made in consumers' homes, ?in the workplace, in shopping malls, in Airports, in hotels, or while driving or using mobile phones. These seemingly insignificant observations of consumer behaviour have vital information about how consumer buy and consume. We are obsessed with big data to drive innovation, by analysing billions of data points. However, sometimes the key to innovation is often a serendipitous observation.
For instance, Snapchat - the social media app would not have existed if its founder had not thought of it while searching for his photograph in a message. Similarly, Post-it notes would not have made our life simpler, if a Priest would not have dropped off his Bible on the floor spilling all his bookmarks. There are much more such product inventions originating from small data insight, for example, Band-Aid would not have been invented if Josephine Dickson had not thought of it while cutting his finger in the kitchen while preparing food. Similarly, the Kellogg brothers, John & Will, discovered cornflakes while they accidentally left some grains on the stove for several days, the mixture turned mouldy creating cornflakes. The famed practitioner Steve Lohr guides: "Listening to the data is important… but so is experience and intuition. After all, what is intuition at its best but large amounts of data of all kinds filtered through a human brain rather than a math model?"
Therefore as marketers, we should be curious to know that if the consumers cook Maggie with or without vegetables? Where do you put your fridge magnet? Do you have cornflakes with cold or hot milk? The answers to these questions can only come from small data. Well in today's competitive world, companies cannot just rely on big data because seeking correlation (Big Data) without causation (Small Data) is no win-win. Marketers need to think like the modern day Sherlock Holmes and harness the power of "Small Data" in order to create disruptive innovative ideas. And as Sherlock Holmes says, "The world is full of obvious things which nobody by any chance ever observes". Holmes again scores the final punch - It is a capital mistake to theorize before one has data. Insensibly, one begins to twist the facts to suit theories, instead of theories to suit facts. Intelligence and talent still dominate the leviathan stacks of big data.
