To Test or to Target? Where to Start for Best ROI?

The previous post had concrete recommendations for proving the ROI of behavioral targeting. Several smart reader comments brought together a pretty clear picture.

However, when I was meeting with a number of experienced online bankers in Europe recently, the question that I received was more difficult to answer than just proving the ROI of targeting.

Namely, the question was whether one can expect greater ROI from testing or targeting? Whichever promises greater ROI, shouldn’t that be where you may want to start?

Thinking about it in terms of anecdotal results

Testing tends to be much more black and white in outcomes when you first begin to optimize a website. After all, no perfect website falls from the sky. At first you can get a big impact from making changes to messaging, calls-to-acton, layout, etc.

There are case studies, e.g. from Unica web analytics customers, where website conversions were improved by as much as 1900% over time with the help of continuous testing, measurement, and improvement.

Now, I have seen many great case studies of behavioral targeting online. But I haven’t seen one where a better targeted message (via email or website or ad network) drove a 1900% increase in conversions.

Should you count on a 1900% improvement from your testing efforts?

Most web marketers when they hear these great testing case studies can be very disappointed when their own testing efforts only yield a percentage point or two in improvment. That happens a lot too.

And depending on the size of your web business, a 1 percent improvement from testing can still be a highly profitable outcome.

But, it goes to show that a mere comparison of anecdotal results that other people achieved from testing vs. targeting is not going to be enough to help you decide whether you should start with one or the other.

A better way to think about it

Testing is a key competency for targeting. As the previous post suggested, you would probably have a hard time proving the value of targeting if your organization didn’t know how to do (hold-out) testing.

Likewise, reporting is a key competency for testing. If you didn’t know how to measure and report on key metrics, how could you prove which test candidate truly drove better results?

Growth path

That is why it does make sense to follow a maturity model where the organization grows from reporting to site optimization (with testing) and then targeting similar to the diagram below from one of Unica’s whitepapers.

5 step maturity model from web analytics reporting to behavioral targeting

Just don’t get stuck in the stage of reporting and testing but make sure you continue to climb up the maturity curve when the business value that you are generating in Stage 2 begins to level off.

Behavioral analysis and targeting (for interactive marketing) has helped companies who do it well push up business value beyond the plateau that they reached from testing alone (see previous post).

2 Responses to “To Test or to Target? Where to Start for Best ROI?”

  1. There is a very good book around at the moment, “How to measure anything” by Edward Hubbard, one section is about how to value information by comparison of the levels of uncertainty reduction acheived by different stages.

    In my own experience this approach can act as a drive along the development path you discuss, so initially the ROI will be large for testing, but tthere will be diminishing retruns from this so one the onging value has reduced the ROI benefit from Targeting will increase a this allows more focus.

    So I don’t think this is an either/or situation but rather a sequential things, in order testing is a good place to start but whilst its benefits can be very large the can be short lived, because the solutions once identified are lower cost but high impact and enduring. Once implemented the ROI from Targeting gets more focus which can be more involved and have a longer fail as extra niches are identified with more experience of a market.

    I hope this chimes with others experience but I would heartily recommend the book

  2. Thank you Tim,

    I am intrigued by the book suggestion and have ordered my copy.

    I frankly couldn’t even follow this sentence of yours: “ne section is about how to value information by comparison of the levels of uncertainty reduction acheived by different stages.”.

    Let’s see what the book says.

    Thanks much again. And I agree with you that it isn’t either – or.

    Akin