Digital technology arms consumers with more knowledge than ever before…about what their choices are, prices for alternatives, and which retailers sell what goods. They no longer have to settle for some product they preferred less, or to buy at a time that is inconvenient for them…and nearly all users who are always on take advantage of this (AOL Buying at Speed study, 2014).

While digital empowers shoppers, it is also a retailing game changer. Every click offers a clue about what that shopper wants.  Retailers have an advantage than advertisers do not have; these clues come to retailers from customers where they can link together their transactions with clickstream data if they encourage frequent shopper log-in…even tying together behaviors across screens, and purchase results. Shoppers can be segmented and it is even possible to match shoppers to their social media profiles to understand better how each segment talks about their needs, lifestyles, and feelings about various retailers and who the influencers are.

Amazon is a data master.  I get e-mails that are completely personalized to my recent purchases or to things I might like.  So is the landing page when I go to And mastering their data opens up windows into new business opportunities. Amazon is now exploring another route to monetizing data…they are going in the targeted ad business, and going after the $50 billion that Google brings in each year.  By the way, as many or more product searches are done with Amazon vs. Google. The online retailer is building a new ad platform similar to Google’s, which could open a huge new business. Amazon Sponsored Links…Someone who types in the search term “bicycle,” for instance, sees a text ad from a bicycle retailer…But the upside for Amazon is undeniable. And Amazon’s treasure trove of customer data could be used to make ads more “click-worthy” to users — even more than Google’s efforts…*

But all of this competitive edge, these unanticipated business opportunities only come to those who tame their big data. Retailers are at a crossroads…will you master your data and compete in a data driven way or leave these potential data gifts on the table and practice pre-digital marketing?

Consider the case of a fast growing retailer, Etsy which sells artisan products. The person leading their big data efforts is Chris “CB” Bohn who was interviewed by HP Vertica, a leading big data solution provider who Etsy works with. The interview began by HP sharing several data-driven marketing principles:

  • Log everything
  • Analyze everything
  • Sample size = infinity
  • Segment = me
  • Analyze what didn’t happen

And by practicing these principles retailers will improve in several important areas:

  • Customer personalization
  • Dynamic pricing
  • Customer service
  • Supply chain
  • Fraud prevention
  • Revenue forecasting

…will you master your data and compete in a data driven way or leave these potential data gifts on the table and practice pre-digital marketing?

Now back to Etsy…they are growing rapidly…sales at $1.3 billion, 45 million registered users, 4.5 million shops, and global with transactions across 90% of countries. CB at Etsy was focused on solving the problem that their BI data architecture no longer scaled but in migrating to a more robust system, they needed to continue to leverage their SQL skills and query assets. In searching for a solution, they had the following BI use cases in mind:

  • Repository for their massive clickstream data
  • Data analysis for ad hoc requests
  • A/B testing (different page set ups, offers, etc.)
  • Financial
  • Monitoring dashboards

Etsy settled on HP Vertica as their solution meeting all of their needs above and working well with Hadoop and advanced analytic packages.  This positioned Etsy for data-driven retailing:

  • Understand customer behavior and appeal to personal interests via affinity and market basket analysis.
  • Combine loyalty programs, customer tracking, and market research with sales and inventory data to drive insights and decision making.
  • Optimize online experiences via true personalization.
  • Analyze social media to gain actionable insights into customer opinions, attitudes and trends.

So, let me ask you the reader, a simple question: “Why do shoppers search?”  The answer is simple: “To find something they are looking for but are not quite sure what it is yet.”  So retailers, listen up…give shoppers what they want…help them find things even before they know they want it! To do this, you will need to master big data…to integrate transaction, clickstream, social, survey and other data streams into a predictive engine so you can delight shoppers with relevant offers, content, experiences, and to offer the right products at a fair price. Anything less is failing the shopper and failing the retail enterprise.

Written by: Joel Rubinson

*Fortune 8/22/14