Ellen Pao certainly put the cat among the pigeons with her recent comment that “It's all true: Everything is fake”.
She was referring to social media traffic and engagement statistics, and confirming what had been exposed by Max Read in the New York magazine some days earlier: that an awful lot of these statistics are ‘creative’ interpretations of numbers that are often generated by bots in the first place.
So is this the death of management by metrics? Not by a long shot. Here’s why, starting with the bad news.
Why You Should Never Blindly Trust Metrics
To say it as simply as possible: large numbers of people are looking at numbers without any real understanding as to why they are meaningful or valuable.
As a consequence, many of the numbers they look at are neither of these things.
There are a few reasons why this happens, here’s three.
Agencies, media companies and anyone selling you any kind of a service are incentivised to do whatever it takes to sell to you. That means ‘forgetting’ to point out that 40% of traffic is bot traffic, or creatively manipulating numbers until they are ‘technically’ true rather than meaningfully true. Welcome to capitalism.
Everyone is taught that numbers and metrics are important, so from the top down we ask to see them. We like people who show us numbers, and we LOVE people who show us numbers that keep going upwards. As a result, we don’t ask too many questions. Whether it’s brand awareness numbers based on dubious research methods or clicks that come straight from a farm in China, we end up complicit in the lie.
Nobody likes to admit they’re wrong, and if your ideas come from the top nobody will tell you either. Think about it. If your CMO decides that all-in on Twitter is the right strategy for the business, what happens to the wise-girl who runs the numbers and discovers the ROI is lower than Theresa May’s current stock in Brussels? What happens when she raises question marks about the ‘engagement’ levels the company are seeing? We all know that the answer in theory and practice don’t always match up.
All of this was a rather long-winded way of saying that it doesn’t require ‘fake news’ and some malign outside influence to create a situation in which fake numbers are used every day to justify everyone’s existence in a business. It comes all too naturally.
So what can we do about it?
Making Metrics Great Again
There’s one metric that isn’t fake: cold hard cash in the bank. And when we think about metrics, we would do well to start from there and move outwards.
What that means is relatively straightforward. Focus above all on the metrics with a very clear relationship with new business.
Conversely, if you can’t see how a given metric has a clear and unambiguous effect on the bottom line (even if a few steps removed), then it’s worth regarding it with some suspicion.
If it is reported by a third party, and there is no clear way to measure the quality behind the quantity, my scepticism increases.
Examples of ‘dubious’ metrics like these might include:
Clicks. For most businesses, clicks are not a metric that necessarily means a huge amount. In the context of paid media advertising, they can be actively dangerous. We pay per click, but if we are bringing the wrong traffic we won’t see any ROI on that spend. If I am measuring the effectiveness of this type of campaign, I’ll look at conversions (how many qualified prospects it delivers)
Impressions. The fact that many of these impressions are bots tells its own story. If we can’t even distinguish between people and machines, how can we say with any confidence that the right kind of people are behind impressions? Sometimes I think that impressions are the last refuge of a scoundrel - if we can’t show any real impact from a campaign, just pull a big number out of the air….
Followers. Again, “who are they?” and “why do I care?” are two questions that spring to mind in this context. In a world in which followers can be bought by the thousand there is no clear link between how many you have and how much business they generate.
I’m going to stop after three examples, but I hope the point is clear. I could certainly go on.
To be clear, metrics like these can in some circumstances be useful but they should be used and reported with great care. If they ever become an end in themselves, you have a serious problem, because it is desperately easy to artificially inflate numbers like these.
It’s worth repeating something else: these are not the right metrics to use when judging campaigns and marketing spend, or comparing the effectiveness of competing creative. It is all too common for a creative to deliver more clicks - but the sort of clicks that die in the conversion funnel long before they become new business.
So what metrics should we trust?
Well, we should be using metrics like “new qualified leads”, “new qualified opportunities” and ultimately “closed won new business”.
All these metrics all incorporate some measure of both quality and quantity. If they increase, then - all other things being equal - our revenues must also increase.
For some, introducing any form of subjectivity into measurement is dangerous. I would say as long as the process for qualification is itself as objective as possible, that danger is infinitely less significant than looking at supposedly objective numbers that don’t mean anything.
So not all metrics are bogus. Only some. If we want to make metrics great again, let’s focus on those that have a direct relationship with the bottom line.
Comments