How to Avoid Data Analysis Paralysis: Separating the Wheat From the Chaff

Consider the following scenario:
You run an integrated marketing campaign promoting your new product. The campaign includes all possible online marketing channels – PPC, display, email, content syndication, PR and of course social media and SEO. You also throw in mobile ads and some in-app advertisement. You have a big budget and since this is a crucial campaign for the company, you run print ads that direct people to a unique landing page and invite your house list to a local meet up and a launch party.

Each URL is tagged and every campaign is tracked. Over the next several weeks leading up to and following the launch you collect all the data in your digital marketing solution and when the campaign ends, you have amassed mountains of data. Now what? It will take days — if not weeks — to analyze the data and come up with a list of insights and recommendations.

pile_of_data

While this scenario is unique to marketing organizations with big budgets that run integrated campaigns, it’s not uncommon even for small companies to face the analysis challenge created by the luxury of being able to track EVERYTHING.

Digital marketing enables marketers to track and measure almost every single pixel they put out on the World Wide Web. But the ability to collect all this data creates challenges for marketers who don’t know how to handle the slew of data and get overwhelmed by the terabytes of data looking down on them from their marketing software solution.

This is what makes the promise that Big Data applications hold so enticing; instead of a human being analyzing the data and coming up with the insight, a machine, armed with ridiculous processing power and designed to look at millions of rows of data simultaneously, will analyze these mountains of data and will spew out insight and action items that will free us to focus on the fun parts of marketing – creating interesting content and creative campaigns… right?

Sadly, the answer is no.

Even if such a machine existed, human beings will still have to look and determine what’s valuable and what isn’t. They have to decide what the important KPIs are and what is just noise, and come up with a prioritized list of action items that will lead to better performance on the next campaign. And as marketing moves more towards digital and inherently requires the aspect of data and analysis, marketers who previously focused only on the creative sides of marketing will soon have to “adapt or die.”

Separating the wheat from the chaff

dilbert-analysis-bad-assumptions
I learned this phrase early in my career when I was suddenly overwhelmed with data, tasks and goals. Not to be confused with simple prioritization, the process of separating the wheat from the chaff requires clear understanding of what is of value to you. Knowing how to recognize the valuable elements of a system will allow you to quickly get rid of the noise and minimize the variables to a manageable number. At that point you will be able to apply some prioritization and come up with a plan. Here are a few tips on how to use this method for data analysis:

  1. Identify the valuable data
    What’s really important for your business? Is it leads, qualified leads or opportunities? Are you running a lead generation campaign or is your campaign directed to generate social media follows or shares? The valuable data should be tightly related to – or actually be – the goal of the campaign.
  2. Identify the critical paths
    Work your way back from the valuable data (goals) to identify the critical paths. This process will allow you to identify other important data points as well as start to gain insight into what is important (wheat) and what isn’t (chaff).

    Critical Path
    Critical Path, source: Wikipedia
  3. List the KPIs
    Along the critical path you will find points of control that will allow you to impact the final result. Those are KPIs – Key Performance Indicators. In most cases, if you want to have a real impact on your goal, you will have to address the KPIs. This is also where you should spend your analysis time.
  4. Identify the “noise”
    Before you can use broad strokes to eliminate all the unnecessary data, you should try to identify the noise first. This exercise will allow you to apply some judgment on what you think is valuable and what isn’t. Beware – this is where most people fall into the analysis paralysis trap. Remember, just because you can measure it, it doesn’t mean that you need to.
    Read Avinash Kaushik’s post “Eight Silly Data Things Marketing People Believe That Get Them Fired” to get some idea on what could be considered as noise.
  5. Ignore the rest
    After you identified the valuable data, highlighted the KPIs and marked the clear noise, for your analysis purposes, just ignore the rest of the data. Don’t get rid of it, delete it or take it out of your data set, just ignore it for now.
  6. Present only what matters, and always remember your valuable data
    You will get distracted, especially when it’s time to present your findings. People will ask you things like “why is this metric going down?” or, “If our lead numbers are going up why is our bounce rate still so high?” They will try to concentrate on the noise, on the irrelevant, on the one metric that has nothing to do with your goal but can throw your entire analysis off and send you in the wrong direction. They would do that not because they have bad intentions, but because you show them data and because it’s human behavior. You can avoid that by controlling the story and only present what is relevant and significant. And when questions about the data that isn’t there come up – and they will come up – stay true to your working assumptions and clearly say, “Our main goal was to generate leads and this metric that you are inquiring about is not in the critical path and has no impact on that goal.”

Happy analyzing!