Are Web Analytics Easy or Hard?

If you are not an insider in this little niche industry, you may be surprised to learn that this question has been a heated debate.

———-
This post is on the occasion of an interview to which I was kindly invited for Aug 26 at wsRadio, in Online Marketing with RSS Ray. From the show music to the ad break – this was a fun experience. Browse RSS Ray’s archives and look for podcasts from Gary Angel, Jim Sterne, John Squire, and many more fun people.
———-

Looking from the outside, you might think that web analysts just breed over web site usage reports all day long to dream up ways for increasing usability, conversion rates, or sales.

But you need to know the following fact:

While outsiders often feel that the topic of web analytics is so boring that it could cure insomnia, for us on the inside, there is deep passion.

The fire of passion is burning:

So, it isn’t surprising then maybe that all this passion has led to a bitter debate among the best minds in our field

The debate

Are analytics “easy” and can even be handled on the side to certain degrees? Or are they “hard”, with lots of obstacles to overcome, and require closest attention to get anything right?

Some of our brightest are on a crusade to help by making analytics intuitive and spreading their adoption to the masses. Others of our brightest are on a crusade to help by uncovering all the pitfalls that exist and that have prevented too many companies from generating ROI from web analytics.

It is a good thing the world has me to judge the outcome and reveal the answer to this epic debate now!

The answer is, of course, that web analytics are both easy AND hard.

There are aspects of analytics that are easy or at least straight forward. For example:

  • If you measure that visitors coming to you from search keyword XYZ have a high bounce rate, i.e. they are arriving at the landing page and them immediately leaving, chances are that either the landing page doesn’t fit their expectations or the keyword isn’t a good one for your offering.
  • If you create two test versions of the landing page with essentially the same content but different layout, design, etc. and you find that one leads to higher engagement and conversion rates, chances are you should keep the better performing page.
  • If you measure that visitors coming to you from search keyword ABC have a great conversion rate but there are only few people reaching you via this keyword, you probably want to check whether you should try to rank higher for that keyword ABC.
  • If you measure that visitors buying from you are all shopaholic until they reach your page where you reveal exorbitant shipment costs or a long form that they must complete, chances are that improving these items will decrease leaks from your funnel

If you did nothing but the above, you’d likely create very respectable ROI from analytics.

But there are other valuable aspects of analytics that are far from easy.

In fact, the harder you look at any individual metric the less it seems to say.

Could also add that the more you know about analytics, the less sure you become what any individual report really means.

Huh?

Well remind yourself of the following:

  • If search keyword ABC has great conversion rates, is that because of only the keyword itself or have visitors been exposed to other ads or emails of yours that led them to search for ABC in the first place? Most obviously, anyone searching for your brand or product names must have heard them elsewhere.
  • One of Unica’s clients, Braden Hoepner from Coastal Contacts was pointing out the following gotcha at eMetrics this year: If you create two versions of a landing page with different offers and you pick the one that performs better for conversion rates, you may still find that you have just hurt your company. How so? By producing lower sales or profits. That happens if you accidentally lead people towards products that are cheaper or less profitable.
  • If people leak at a particular page in your funnel is it because of something you said? Or is it the point where they have learned enough from you to stop and check first what the competition has to offer?

So given both easy and hard options to choose from, which would you pick?

Pick the deeper questions first

Tackling the more difficult questions is often critical for working towards the ultimate optimization summit whereas the easier questions may leave you working towards a local optimum.

Pick the easy questions first

But the easy questions have potentially higher % ROI because you put less effort into them and can still get great improvements. So you might be inclined to start with the easier tasks and work yourself to the more difficult questions over time.

Pick the deeper questions first

But that approach doesn’t take time into account. If you waste time on achieving a local optimum you delay the overall optimum, I.e. you incur opportunity costs. So, if you know that there is a bigger optimum to be achieved you could make your company richer by reaching it sooner than later.

Pick the easy questions first

But what if it takes a really long time to tackle the harder questions and outcomes are uncertain?

Arggghhh…!^%$^%@!!!

See what I mean. Analytics are both easy and hard. And the more you think about them the worse it can get!

8 Responses to “Are Web Analytics Easy or Hard?”

  1. For me one of the biggest spanners in the works is the fact that web analytics tools make it all ‘seem’ so easy. If you want to [properly] analyse a customer database, you need a statistician with 5+ years SAS experience – but any idiot can use Google Analytics; as long as you know how to use the internet you know how to use web analysis tools!

    So the problem is that it IS easy to look at the data, but this makes people forget what analysis actually is.

  2. Great post about a crucial debate.
    I really think that web analytics can’t be easy.

    How to be totally sure of what I get from my analysis?
    Is it the real cause of my website problem? Or do I miss the right insight?

    I’m afraid that only tackling easy questions drives to misinterpretations.
    It’s essential to go further and analyse all available indicators about an element to get a holistic view.

    Take the example of a suspicion of a landing page problem.

    You need to gather all available and useful informations about this landing page.
    It means that you know which informations are essential (hard).
    Then, you have to gather all these informations (hard), to display them in a single page (hard) and to analyze them to reveal insights (hard).
    It reduces risks of mistakes but it takes time and it’s hard.

    But when your insights are taken in account and they improve outcomes, you feel like a world champion!

  3. I think the whole question is kind of insane. I think it’s very biased so it depends a lot who you are talking with. And as Akin clearly pointed out you can talk same things and metrics in a very different level. Good post, thanks.

  4. Hi Akin,

    Good post and yes the debate is still open 😉 I remember two years and a half ago at eMetrics, Aurélie and I were sitting at lunch with some of the most renowned Bloggers in Web Analytics and we had a discussion around this debate, Eric was already stating back then that Web Analytics was hard (he’s done a few presentations on that topic these past years) while Avinash was disagreeing slightly, not saying that Web Analytics was easy but stating that it was complex.

    My two cents are that Web analytics can seem easy and it’s easy to get started with tools like Google Analytics, but it gets more complex further down the road and yes it can get really hard, but this as all disciplines. I like to compare Web Analytics to cooking, where the tools are the kitchens and the analysts are the cooks. Well, for some people it’s easy to cook in any kitchen, while others struggle just to understand how appliance X or Y works… So if you have the correct mindset (I don’t believe that anybody can be a Web Analyst) it can be easier, but if you want to make a soufflé, well it’s not going to be easy, specially the first times 😉 To end this analogy I always state that I’d rather have a good cook in a simple kitchen than a bad cook in a very sophisticated one. And let’s be honest Web Analytics tools aren’t as easy as vendors (and now I’m one so I measure my words ;-)) try to present them. As everything good in life, it takes time to be good at it and use it properly. With time you’ll find it easier but if you’re trying to integrate your online data with your offline data for instance it can be hard or complex (pick the one you prefer). Of course tools are important if you’re trying to do things like that and you have tools that make your life easier than others (and Unica is in the first group in this example).

    Cheers from Sunny Madrid and please pay us a visit next time you come to Spain.

    Cheers,

    René

  5. IMHO, Web Analytics requires a lot of thought. Thinking is not easy. No, i’m not kidding.

    And if you need a statistician to think for you, well you’re in real trouble then.

    Google Analytics is a tool, nothing more and nothing less. Its a LOT better than the way I first learned to do web analytics (AWK, PERL, Excel, Powerpoint, rinse, repeat, etc…). But GA is never going to think for you.

    One approach is to put a few key questions into a bucket – some deep, some easy . Then take that bucket and pitch it to your boss/client/community as a project. Answer all the questions and come up with recommendations. Only give your boss/client/community the answers and recommendations. I mean, show them a report if they ask or demand one…but ultimately people want answers not reports.

    Your boss/client/community wants someone to think through the deep questions for him/her. Its Akin’s job to make it look easy, which he clearly does 🙂

  6. […] link is being shared on Twitter right now. @jonnylongden, an influential author, said Is #measure easy […]

  7. Man! I’ve been doing this for tooo long to find it easy. You know me, the more I think about even what’s supposed to be the easy, simple stuff, the more I challenge its validity.

  8. Thanks everyone for the well taken comments!

    Jonny is certainly right to point out that many buyers of web analytics systems expect answers out of the tool instead of reports. Let alone doing analysis! The difference between reporting and analysis is only an insider topic anyway, as far as I know. Not sure whether somebody out of business school would necessarily know what the difference is.

    Benoit’s comment speaks to the passion that we all see of trying to make our work really matter. Creating anything that matters seems really hard work and rarely seems to fit in a 40 hours work week. As for the potential “analytics mirages”, I ran into them too while writing the “easy” bullets in the post and had to work to constrain the points.

    Petri says we may all be in danger of going insane. What is happening here though because this question is clearly drawing our attention?

    Rene,
    Wow, excellent analogy! I gotta remember that one. During the radio show the host had another good analogy: Pen and paper are easy to use too but that doesn’t make everyone a good author. Smart guy, he was getting at the same point. Can’t wait to see the home warming party reviews for your new place in Madrid!
    ….
    Shwe,
    raises another great point about speaking with execs which is definitely not easy but hard. These guys are not dumb. They may not know the inner workings of the WWW bu sure want their information concise and we must be able to substantiate and defend our advice. Not easy!

    Jacques,
    Good point. But why actually? Is every web site different? Would it be possible to create an expert system for the 4 – 5 site types out there that recommends how to trouble shoot each KPI?

    Thanks again
    Akin