Big Data in Retail - Impacts Now and in the Future

Connor Cameron

A now heavily discussed topic with regard to innovation and technology, big data has become a focus for some of the world’s largest retailers, proving to be a viable solution to margin pressures and ever-increasing retail competition.

Big data is a relatively broad phrase to describe large data sets that are too large, fast or complex to be processed using traditional methods. But the phrase has come to be synonymous with the field of systematically processing and analyzing the information to reveal patterns and trends, particularly relating to human behaviour.

What is important is what organizations can do with this data, optimizing for cost efficiency and revenue increases in efforts to increase margins. 

While the phrase ‘big data’ was coined in the early 2000’s by industry analyst Doug Laney – who attributed to big data three main attributes, volume, velocity and variety – the field has really hit its stride in conjunction with advancements in machine learning.

The Role of Big Data in Retail

For the retail industry specifically, Big Data has the capacity to track consumer trends in order to attract new customers and, arguably more importantly, increase the lifetime value of existing customers through targeted offers and loyalty programs.

However, critical to a retailers ability to utilise the benefits is “a robust and methodical way of collecting, managing and interpreting data, then linking that insight to the overarching business strategy,” according to KPMG. With this capacity, a retailer has numerous opportunities.


Targeted Recommendations and Advertising

The extensive analysis of a large consumer dataset allows a retailer to predict what a customer is likely to purchase next. A retailer can study a consumer’s previous purchases, purchases of similar customer segments, how they interact with online stores/apps/social media accounts, and even significant other factors like the weather. Using this data, they can personalise recommendations and advertising to ensure the greatest rate of return on marketing expenditure possible.

Brands like Walgreens and Pantene in the United States partnered with the Weather Channel to anticipate weather patterns and shift marketing strategies accordingly. Seeing a period of high humidity coming, Walgreens began to market anti-frizz products more heavily to women due to the increased demand during these conditions, resulting in a 4% increase in hair product sales over a 2 month period.

Similarly, with regard to the point about studying purchases of similar customer segments, Amazon attributes 29% of sales to their recommendation engine. The engine studies the data sets of 150 million customers to recommend products to customers throughout their website.

Pantene's Targeted Advertisements

Merchandising and Stock Levels

In a similar manner to targeted recommendations and offers, a retailer can study data extensively to optimize stock variety and the availability/presentation of stock between outlets.

At the most basic level, a retailer can study data down to the day of the week to determine their allocation of stock. By optimizing stock allocation, the retailer can simultaneously avoid lost revenue as a result of a lack of in-demand products and reduce the costs associated with holding stock, such as storage costs (utilities, warehousing etc.) and shrinkage.

Walmart, the world’s largest brick and mortar retailer by both revenue and number of stores, is investing heavily in what will be the world’s largest private cloud system, capable of managing 2.5 petabytes of data an hour. One of the major focuses of the Arkansas based analytics hub analyzing and optimizing stock levels.

Turning Trends into Revenue

The analysis and response to consumer trends has proved to be a lucrative option for industries globally. A study of the car industry and Tesla’s rapid growth as a result of a growing number of drivers turning to eco-friendly options, provides ample proof of this fact.

The issue then is the reactive stance retailers have often taken; adapting to trends too late and missing out on revenue opportunities. Big Data provides a retailer with the capacity to be proactive with regard to trends. Studying large datasets gathered from social media, forums and existing customers, can reveal growing trends before they really accelerate, unlocking vast revenue opportunities.

The heavily publicized Dollar Shave Club capitalized on the consumer’s increased engagement with subscription boxes for items they would otherwise have to purchase regularly. Combined with a remarkably effective satirical marketing approach, the company has managed to acquire 3.2 million subscribers, including 12,000 customers in its first 24 hours. In 2016 the company was acquired by Unilever for an estimated $1 Billion USD.

Dollar Shave Club - Photo by darren&curtis


The future of big data in retail

The field of Big Data is still relatively new to the retail sector; but is becoming exponentially more important. The revenue and cost reduction potential of Big Data is growing exponentially, and numerous retailers and startups are running extensive experiments in hopes of utilizing Big Data for new benefits. This is what we can expect to see from Big Data in the future.


Natural Language Processing (Unstructured Data)

Retailers have made significant inroads to the study of unstructured data such as that which comes from social media; however, there is still a way to go.

Natural Language Processing (NLP), is a branch of artificial intelligence with the objective of understanding how humans communicate; from reading to understanding. It becomes remarkably difficult for computers to understand human communication due to numerous abstract tendencies humans undertake in order to pass information. One such issue is sarcasm.

A comprehensive understanding of the human language involves connecting the words spoken/written and associated concepts.

As major developments are made in the area of artificial intelligence, computers may be able to use syntax (how a sentence is structured), and/or semantics (the meaning conveyed by a text), to process natural language. 

This will result in a significant improvement to the capacity for computers to make sense of unstructured data, as previously mentioned, providing more accurate datasets from information-rich environments like social media, linking closely to trend analysis and targeted recommendations.


Machine Learning and Predictive Analytics

One of the most important benefits of the study of large data sets is predictive analytics. Retailers of all sizes already do this – with or without the help of computers – when conducting general tasks like financial planning or stock purchasing. Predictive analytics links directly to the benefits previously established.

However, the future of Big Data holds large potential for predictive analytics to improve its accuracy, predominantly due to the impact of machine learning. The nature of effective machine learning algorithms dictates that they will only become more influential in retail planning, particularly in areas like merchandising. Greater accuracy in predictions will result in more effective optimization for revenue and cost reductions.


Customer Identification

Similar to the use of Machine Learning and Predictive Analytics, in store customer identification has the capacity to enhance how data can be used to drive revenue. Retailers can combine targeted offers and advertising in stores with customer identification to drive in store sales.

The highly publicized Amazon Go concept has drawn attention toward in-store customer identification; using body mapping technology to track dozens of customers and recognising what a customer picks off the shelf and purchases.

If retailers can link this customer identification with existing customer databases they can link online and offline activities, they can increase average customer spend; similar to Amazon’s remarkably successful recommendation engine.  

Amazon Go San Francisco - Photo by Nick Statt / The Verge


McKinsey, in their report 'Big data: The next frontier for innovation, competition, and productivity,' found that, "a retailer using big data to the full could increase its operating margin by more than 60 percent". Therefore, while it is important to consider that Big Data may have limits in terms of its effectiveness, on its current trajectory, it will have numerous benefits in the near future; from highly targeted offers/advertisements to predictive analytics.

Retailers who capitalize on Big Data may be able to stay ahead of competitors, fighting off ever increasing margin pressure by lowering costs and improving revenue. Big Data may soon not become an option for retailers, but rather a necessity.

Also On The Blog