Ask us a question!

Web Moves Blog

Web Moves News and Information

Posts Tagged 'ab testing'

In Part I of this series, we examined what to measure, The Funnel, and landing pages. In Part II we will put this preparation into action.

Step 4: Test

SEO Moves - AB TestingAfter all the effort people put into setting up programs and campaigns, nothing amazes us more than how seldom site owners or managers check to actually see if their whole process works. Emailed form results that disappear into cyber oblivion, missing images, or simple broken links are among the most common things we witness.

But these mistakes can be far worse.

I recall a client coming to us several years ago with a very nice healthcare supplement website he just had built. It was well designed and e-commerce equipped and he had product ready for fulfillment. He asked us to see if we could strengthen some of the marketing copy on his order pages as his orders just seemed to be “way down.”

We dove in to take a look. The site was organized and some of his product names were very clever. After spooling through several categories and pages, we decided to see how easy his purchase process was. And we liked his site so well, we thought, What the heck, even if we end up buying something just to test it out, it’s worth it. Some of us could certainly use that Ear Nibbler Female Pheromone! We zeroed in the product, chose our quantity, resisted the add-ons and upsell items, broke out the credit card, armed ourselves for purchase, and hit BUY NOW.

Only to stare in disbelief as an error page came up. FAIL.

Can you imagine? All the time and money that went into gathering that sale ended in error. And he didn’t just have one page like that, he had many. Needless to say, all the slick copy in the world wasn’t going to improve his sales. But the larger point is, while we’re not saying you have to be perfect, if you don’t test to see if your setup works, all you’re setting yourself up for is failure.

(more…)

Like any other art form, web design is completely subjective. A web site might look like a thing of beauty to one person, and a complete mess to another person. There is, after all, no accounting for taste, and everyone’s tastes are different. However, there’s more to a web site’s design than merely its appearance. A web site design can have an enormous impact on conversions and even the most subtle design decisions can have a big effect. For example, a user might be more inclined to click on a green “Buy Now!” button more so than a red one. Finding a good balance between a site that looks good and a site that performs well in terms of conversions can be a real challenge.

How then can something as subjective as web design be analyzed in an objective manner to find the most effective design? One widely technique is A/B testing. In a nut shell, A/B testing sets up two or more control groups: Group A will see one version of the site, while Group B will see another version. This way various design elements can be tested and compared.

A/B Testing Representation

But is A/B testing really the best way to determine the most effective web design? Perhaps not. This excellent blog post by Steve Hanov suggests another method for finding the best design. Best of all its fully automated. Set it, forget it, and the page will “learn” which elements result in the most conversions.

In his post, Steve outlines the epsilon-greedy algorithm, also known as the multi-arm banded problem. Given a set of variations for a particular page element, the algorithm can make an ‘educated’ decision on which element to show based on its past performance. The best performing page elements are displayed the most frequently.

The algorithm records of the number of times a particular page element was displayed, and the number of times the element resulted in a conversion. However, the algorithm will also adapt to change. If a page element’s conversions begin to decrease, the algorithm will start to adapt and display different variations. The best part of this is that you can set up different variations of page elements one time, and let the computer do the work of figuring out which variations are the most successful. Pretty neat stuff!

Armed with this knowledge, I set out to try a few experiments with it, the result of which is Robo_AB_Tester, a small PHP class library I created which implements the epsilon-greedy algorithm. You can give it a try here.

 

Robo_AB_Tester tries to abstract away as many implementation details as possible and create a simple interface that is, hopefully, easy to integrate into a PHP based website. Once it is set up, it will:

  • Allow you to test multiple elements per page
  • Allow you to specify any number (A/B/C/D/E…) variations of each element.
  • Detect on page events for the tested elements (e.g clicks, form submits, etc)
  • Handle all ajax communication between your web page and Robo_AB_Tester
  • Keep track of how many times the elements were displayed
  • Keep track of how many times a user interacted with the element
  • Autonomously determine the best performing elements

For more details, see the demo page.