Test, Measure, Optimize – The power of A/B testing


Are you an entrepreneur busy building a webstartup? Or are you involved in a startup that has already grown to a fair size? If the answer is yes, you have to make important decisions every day. Some of them are about your website and how it should look like. When you decide about the content and look you want to present visitors, do you ask yourself: What images should I include? What should my “Call to Action” say? Which fields should my sign-up field include?

You probably do. But the more important question is:

What do you base your decisions on?

Often your decisions are probably based on gut feeling, maybe experience from other sites that you have seen and things other people that work on your startup will tell you. But do you ever use data to inform these design decisions?

Find out what works on your website

“Build, measure, learn” is the motto of “The Lean Startup”. Most startups probably adhere to this mantra religiously. The building part is very often the one that most entrepreneurs focus on, which seems to make perfect sense. The measuring part is, however, often neglected. Maybe this is because it seems hard for many young companies. The same goes for big companies too, so don’t worry. The reason tracking and measuring is particularly hard for younger stage startups is that you do not really know who your customer is – tracking the right metric and knowing what people want to find on your website becomes extremely difficult. Measuring is a pain.

But even when you are a young company with little traffic, there is ways to make your website better. One of them is A/B testing. It is a great help in answering the question: What do visitors need/want to see on my website? How can I optimize my site, for example the homepage, so that I turn more visitors into customers?

The power of testing your website

A/B testing? So I build a new version of my website and then test it against the old one? This is luckily a thing of the past, as A/B testing has become easier than ever. Also people that know nothing about web development, mere marketing mortals, can do A/B testing. Optimizely, for example, allows you to load the URL of your website into an editor where you can make changes to your website. It is as easy as drag and drop and you have created a new version of your website. Let’s say that you changed the menu from the top to the left side of your page.

This one we could call version B. Then you can test this version against the original, version A. When people go to your website, half of the visitors see the original, the other half the version you modified using the Optimizely editor. Then you compare which one worked better. Did more people sign up in the modified version? Did you get more clicks? Did you sell more? No more hour-long meetings about the color of a button or if you should use an image or a video to present a particular product – the answer to your questions is in the data. (Here more infos on how A/B testing works.)

Why you just can’t lose with A/B tests

The best thing about A/B testing is that you simply can’t lose. Let’s imagine you run a test and you get a clear winner in the modified version. In version B where you removed elements, you see an increase of 50% in clicks, for example. Then you know that customers prefer that version, so you should go with it and implement these changes. If you run a test and you find that there is simply no difference in how people responded to the changes you made, then you also won – you know that you can stick with the original. The same goes for when you run a test and you see that the changes you made actually perform worse than the original – stick with the original, since data tells you that people did not react positively to the new version.

A lot of talk, let me see some results! Here a small example from Spreadshirt that wanted to test its “Start Selling” section on the Homepage. It created a drastically changed version and then ran a test.

Version A: The original rather complicated graphic explaining how to sign up for the service.

optAVersion B:

Screen Shot 2013-11-11 at 1.14.11 PM

Maybe you guessed it. When Spreadshirt ran a test on this page, version B, the more streamlined infographic, got 606% more clicks. Not bad, right?

Paul Graham from Y-Combinator famously said: Make things people want.

A/B testing helps you do exactly that.

Find more infos about Optimizely and A/B testing our website and see you at Startupweekend in Berlin!