is_marketing_data_big_data.pngAccording to Wikipedia, Big Data "consists of datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing." Does Marketing data really fall into this category?

Marketing data adds up quickly if you store:

Every click to the website

Every page view on the website

Every ad impression purchased/served

Every email ever sent

Every direct mail piece ever sent

Every order ever taken

For one large retailer, the marketing touchpoints alone – not including the orders taken or website page views – contained billions of records in just one year. How big is a billion? If you were to stack a billion pennies in a single pile, the stack would reach nearly one thousand miles high. It’s pretty safe to say that this is Big Data.

At MarketShare we’ve confronted three major Big Data challenges:

1. Statisticians adapted techniques to model event data of this size

2. Engineers and statisticians had to collaborate to shape data into modeling use

3. Engineers had to design a scoring system that could handle dozens of models per client

We’ve repeatedly asked ourselves: Is there a business payoff for all this complexity? You can guess the answer if we’re writing about it. :)

What are you doing with your big data?