Modern marketing is all about data and however hard you might try, you can't spend any time around marketers online without being subjected to endless think pieces, how-to guides, ebooks or other dreck about how we need to track and measure and count every little thing.
We've got click rates, impressions, conversion rates, open rates, ROAS, pageviews, bounces rates, ROI, CPM, CPC, impression share, average position, sessions, channels, landing pages, KPI after never ending KPI.
That'd be fine if all this shit meant something and we knew how to interpret it. But it doesn't and we don't.
The reality is much simpler, and therefore much more complex. Most of us don't understand how data is collected, how these mechanisms work and most importantly where and how they don't work.
- Email open rates don't actually indicate that an email was opened, merely that a request was made to a server
- The black boxes inside Facebook and other ad exchanges give you flat out wrong data about how your ads are performing
- The audiences you're targeting on Google, Bing, etc are fraudulent and don't even exist
- The exchanges you're purchasing media space from are cheating you
And even if we know how the data is collected, what it means and what it's actually tracking, most of us don't have the technical chops to analyse the data we've collected1. I don't mean to rag on anyone by saying this, but we do need a reality check.
And look. I get it. Having tangible data allows us to demonstrate that we're doing our job and we're trying to measure and improve what we're doing. But as Bob Hoffman rightly points out - that's not how brands are built.
The numbers are often all we have to prove our case, to get more budget and in extreme cases, to continue to stay employed. We'll remain in this mess until we can separate marketing from short sighted and poorly informed decision making. Until leaders can lead on the strength of their conviction and experience instead of second guessing themselves and their staff based on the inadequacy of data.
I don't know what the way out of this mess is, or what the path to success looks like. All I know is this.
We're addicted to bad data.
Go ahead, ask the average marketer how to properly calculate the sample size for an A/B test, or what a Chi-Squared test is, or even what power is and why it's important. ↩