I'm Split Testing ... Why Haven't I Doubled My Revenue Yet?

I’m Split Testing … Why Haven’t I Doubled My Revenue Yet?

The Challenge

Perhaps you have been doing some split testing with your landing pages, particularly ones close to purchase.

Maybe you’ve done nifty things like change call-to-action buttons around and test the impact and WOW you are getting a nice 50% bump in people clicking that Buy Now! button.

Here’s a real test running in Performable. This is a late-in-the-funnel page seen only by people who are at least remotely interested in buying a particular product.

The first column of stats is visitors for each test branch, the second is the conversion for clicking the Buy Now! button, (ignore the next column of 0’s) and the final column is purchase conversion (‘remote conversion’ in Performable).

WE DOUBLED OUR BUTTON CLICKS AND ONLY RAISED REVENUE BY 12% … WHO DO I FIRE?!?

Chill out there cowboy, it makes sense … dig this:

Let’s think for a moment about ways of looking at the population that sees this page. I like the idea of looking at the level of interest, or how qualified each tier of the population is. Here’s how I propose to think about the visitors:

You with me? What this is saying is that on the left end of the X-axis we will have the 1% of people with a very high level of interest, and as we go right along the X-axis 0- widening this segment of the population - we can basically say that the minimum level of interest held by that pool of people drops.

If your eyes are glazing over, chill out, it’s pretty simple.

Think about certain consumer products coming out of Cupertino; how many people do you know who will buy anything that Apple makes? Probably 5% of my friends fall into that category (and you know who you are!).

That population has a pretty darn high minimum level of interest, right? Specifically, they have a get-the-eff-out-of-my-way-I-need-to-set-up-my-tent-outside-the-store-the-night-before-product-launch level of interest. These folks are not hard to sell to, you’ve established credibility with them and they are hooked. How you do that is a good topic for many other posts … by Seth Godin. Read those.

So the next segment of the population is really the interesting part. This is where your bread is buttered, and where your optimizations can pay off. In the Apple example, I would fall into this category: I dig Apple stuff, maybe have a couple bucks to drop on a toy, and have had good experience with Apple products, BUT I’m not going to blindly buy anything that they put out.

By now, I’m hoping you sort of see the trend here, and the overall idea. So how do we classify the ‘long tail’ here, as we approach the minimum level of interest for the entire population (which ends up being at or close to 0% - everyone in the world has at least 0% interest in your stuff).

Let’s go with that.

Ok, so back to the test we’re running. Here’s an overlay (not to scale - you think I have all day to play in Photoshop? I have products to make!) of the test report data on our interest/population curve. The dark gray area is the baseline Buy Now! button-clicking population. We’re assuming it is the most qualified part of the population because, well, it is.

Here’s the punchline, good people. And thanks for bearing with me, though in all likelihood you just scanned all the graphs, either got what I’m saying, or didn’t and left. But no, you - you - are reading this, and I like you.

Punchline: you can double the number of people who convert in this middle stage (clicking the Buy Now! button) without doubling your revenue. Why? Because you’ve already captured the most qualified lead base, and now you are extending into tougher territory sales-wise. The area of the curve that you are capturing represents (again, not to scale) your revenue, which is ultimately the only thing that matters.