Having spoken recently about focus in marketing and attribution, I felt I might take a step back a little this week, and talk about a subject that lurks in the background of all these conversations: return on investment (ROI).
Most people think they understand ROI, or at least they make sure they pretend to understand ROI. And yes, at heart it isn’t a particularly complicated matter. But in the real world, how ROI is mis-understood and mis-used is a subject that could be discussed for hours (don’t worry, I’ll limit this post to 1,000 words maximum*).
And as we live in the real world, that’s what I am going to write about.
Defining ROI
I suspect most of us know the equation. It looks something like this:
(Revenue — Investment) / Investment x 100
Or in other words, if I spend $500 on a marketing campaign and deliver $2,000 or revenue as a result, my ROI is going to be 300%.
This is what you will read in a textbook, and just to be crystal clear there is nothing whatsoever wrong with it. It’s just that if we don’t use it without understanding real-world context, and without understanding how complicated the words ‘revenue’ and ‘investment’ actually are, then we can end up in all sorts of trouble.
For the rest of this piece I’m going to present a list of ROI challenges in no particular order. None of these are intended to suggest you shouldn’t be trying to measure ROI. They are instead supposed to help you get better at it. Good luck!
Correlation and causation
We have to start here, but I’ll be very brief on this particular subject. The principle is simple: just because you did one thing (spend money on a TV advertising campaign, for example), doesn’t mean that another thing that happened (an increase in new customers) happened because of that campaign.
We all know this, but it gets ignored a lot of the time. Post hoc rationalisation is rife in marketing teams. It is common practice to go looking after the event for numbers that have improved, ignore those that haven’t (or have got worse) and claim success and ROI as a result. Not really good enough.
This, of course, is precisely why attribution is considered important. But as we saw last week, there are real limits to how successful that approach can be. Sometimes it will help, sometimes it isn’t appropriate.
At the very least, before running any campaign document what you expect to happen and which metrics you expect to see change. Any arguments drawn up after the fact should be judged close to worthless.
Externalities
Economists love externalities. You should too. An externality is cost or benefit that isn’t suffered or enjoyed by the producer. Think pollution, to give the obvious example. Without wanting to bore you senseless, externalities are causes of ‘market failure’, which is fancy economist speak for “things not working as well as they should do, even though everyone is doing what they think is the right thing”.
I hope the implications for marketing campaigns are clear. When you deliver a campaign, you tend to look at the costs and benefits directly attributable to that campaign. But what about costs and benefits that fall elsewhere?
Here’s one example: if I instigate a twice-weekly email campaign to my entire database promoting a specific new product line, then new sales of that line attributed to my campaign would certainly be a benefit. But if I failed to account for the cost of increased unsubscribe rates, I might see great ROI today and a nasty surprise tomorrow.
The truth is externalities are almost everywhere we look in marketing. And the bad news is they can be almost impossible to quantify. But we have to be aware of them and make an attempt.
In the example above, you would need your wits to understand unsubscribes were a potential cost, an estimate of what an email opt-in is worth to your business - not just to me, but to everyone else in the business - and the data to track the impact of your campaign. And of course, you would want to spend plenty of time considering all the possible externalities - including (or more accurately especially) those that don’t impact you specifically.
Only then will you be estimating ROI in a meaningful way.
Time
How long have you got? When it comes to time, measuring ROI is a kind of ongoing tussle between two opposite extremes, namely:
Wait for it all to come out in the wash, for leads to turn into closed deals, which could be measured in years with some sales cycles, so the data is worthless, or
Return ROI calculations within days, but based on assumptions that may or may not be accurate
There is no right answer to this argument. Obviously option 1 is the most ‘accurate’, but it is also useless in practice.
Option 2 is usually closer to where we end up, but we absolutely MUST understand the limitations of this approach. To show ROI in this way, we usually do so by assigning a value to stages in the marketing and sales funnel. A new customer is worth x. An opportunity is worth x/3, a marketing qualified lead (MQL) is worth x/10 etc (I might write about this process next week).
But that is assuming that all MQLs, for example, are created equal. And this is not the case.
To be clear, we should be trying to ensure it is the case as much as possible - but it never really is. We need to do at least three things to control for time in ROI analysis:
Agree how long we give a campaign before performing the analysis, and be consistent across campaigns.
Watch for atypical data like a hawk. If something appears ‘wrong’ (like MQLs from a specific campaign not converting into opps at a rate we expect) we need to go back and check our assumptions and establish if something about this particular campaign is challenging them.
(related to point 2 above) regularly revisit our assumptions. You would be amazed how many businesses operate on the basis that a lead is worth x, with x being a number passed down through time as if Moses himself brought it down from the top of Mount Sinai.
Opportunity Cost
Anyone who opens up the big book of marketing cliches will be familiar with the phrase “washes its face” which means “this activity breaks even” and is usually wheeled out as the last refuge of a marketing scoundrel desperately trying to shore up support for his or her pet project (see below).
As a stand-alone statement of fact it is fair enough, but if it doesn’t prompt the question “should we be doing something else then?” you are probably in the wrong job.
Which is all a long-winded way of saying that ROI in itself is only really good for killing off things that definitely don’t work. Just because something has a positive ROI doesn’t mean we should be doing it. One of the dangers of ROI is that we keep doing things that are just about worthwhile because they deliver “proven ROI”. Make sure it is the start of the conversation - not the end.
Asymmetric Scrutiny
I don’t know if I am coining that term, but it’s probably the simplest way to express the concept I have in mind, which is endemic in most businesses and roughly translates as “ideas I don’t like have to jump through hoops to prove their worth, ideas I do like get a free pass”.
Or in other words, ROI analysis is a great idea, but if you apply it selectively then you may as well not bother.
Marketing veterans will recognise a number of behaviours that fall under this heading, but the following four probably cover most of it:
Exempting some activity from ROI analysis altogether, just because it is so obviously a good idea (tends to be highly correlated with the idea coming from somebody with the letter C at the start of their job title)
Loading positive externalities onto ideas we like, and negative ones onto those we don’t, until we get the result we want.
Inventing increasingly absurd anecdotes to justify why doing something is actually a good idea, even if the numbers are struggling to show that is the case. These tend to become slightly insane just-so stories, that usually begin with the words “but what if” and should probably end there.
“Intangibles”. I usually start polishing my CV when I hear this word.
The truth, as I mentioned above, is that ROI is far more often used to kill an idea someone doesn’t like than prove the value of an idea that somebody hates. Keep that in mind.
In conclusion, ROI is complicated. Sometimes you can calculate it, sometimes you can’t. When you can calculate it, it is incredibly easy to do so in the wrong way, or ignore factors that should be considered. You have to compromise when it comes to when you choose to gather your results. You will have to deal with prejudices and biases. Sometimes you’ll do the right thing whilst others ignore the rules.
But it’s worth trying nevertheless. Good luck!
*this turned out to be inaccurate
Comments