Advantages and Disadvantages of A/B Split Testing for Landing Pages

Marketers can increase their conversion rates dramatically — and in a short period of time — by testing alternative landing page designs.

There are two types of testing marketers can do: multivariate and A/B split testing.

With multivariate testing, marketers mix and match several page elements at once — such as headline, layout, images and call-to-action messaging — in an effort to determine which combination produces the best results. But multivariate tests can be complex to set up and analyze.

A/B split testing, on the other hand, is the most basic tuning method. The name comes from the fact that two versions of a landing page — “A” and “B” — are tested. “Split testing” then refers to the random assignment of new visitors to the version of the page they see. In other words, the traffic is split, and both versions are shown in parallel throughout the data collection period in equal proportions.

A/B split testing advantages
A/B split tests have several advantages, including these:

• Useful in low-data rate tests. If your landing page has only a few conversions per day, you simply can’t use a more advanced tuning method.
• Ease of implementation. Many software packages support simple split tests. You even may be able to collect the data you need with your existing Web analytics tools.
• Ease of test design. Split tests don't have to be carefully designed or balanced. You simply decide how many versions you want to test and then split the available traffic evenly among them.
• Ease of analysis. Only very simple statistical tests are needed to determine the winners. All you have to do is compare the baseline version to each challenger to see if you’ve reached your desired statistical confidence level.
• Flexibility in defining the variable values. The ability to mix and match allows you to test a range of evolutionary and revolutionary alternatives in one test, without being constrained by the more granular definition of variables in a multivariate test.

A/B split testing disadvantages
A/B split tests also have their drawbacks, including these:

• Limited number of recipes. While you may want to test dozens of elements on your landing pages, because of the limited scope of split testing, you have to test your ideas one at a time.
• Inefficient data collection. Conducting multiple split tests back to back is the most wasteful kind of data collection. None of the information from a previous test can be reused to draw conclusions about the other variables you may want to test in the future.

Despite these drawbacks, if you’re new to landing page optimization, A/B split testing is definitely the way to start. Because of its simplicity and the availability of free tools, it’s easy to begin down the road of improving your conversion rate.

• MORE ON
• E-COMMERCE