Connect
Follow us on
Advertisement
 
 

Advantages and Disadvantages of A/B Split Testing for Landing Pages

February 5, 2009 By Tim Ash
1
Get the Flash Player to see this rotator.
 

Marketers can increase their conversion rates dramatically — and in a short period of time — by testing alternative landing page designs.

There are two types of testing marketers can do: multivariate and A/B split testing.

With multivariate testing, marketers mix and match several page elements at once — such as headline, layout, images and call-to-action messaging — in an effort to determine which combination produces the best results. But multivariate tests can be complex to set up and analyze.

A/B split testing, on the other hand, is the most basic tuning method. The name comes from the fact that two versions of a landing page — “A” and “B” — are tested. “Split testing” then refers to the random assignment of new visitors to the version of the page they see. In other words, the traffic is split, and both versions are shown in parallel throughout the data collection period in equal proportions.

A/B split testing advantages
A/B split tests have several advantages, including these:

  • Useful in low-data rate tests. If your landing page has only a few conversions per day, you simply can’t use a more advanced tuning method.
  • Ease of implementation. Many software packages support simple split tests. You even may be able to collect the data you need with your existing Web analytics tools.
  • Ease of test design. Split tests don't have to be carefully designed or balanced. You simply decide how many versions you want to test and then split the available traffic evenly among them.
  • Ease of analysis. Only very simple statistical tests are needed to determine the winners. All you have to do is compare the baseline version to each challenger to see if you’ve reached your desired statistical confidence level.
  • Flexibility in defining the variable values. The ability to mix and match allows you to test a range of evolutionary and revolutionary alternatives in one test, without being constrained by the more granular definition of variables in a multivariate test.

A/B split testing disadvantages
A/B split tests also have their drawbacks, including these:

  • Limited number of recipes. While you may want to test dozens of elements on your landing pages, because of the limited scope of split testing, you have to test your ideas one at a time.
  • Inefficient data collection. Conducting multiple split tests back to back is the most wasteful kind of data collection. None of the information from a previous test can be reused to draw conclusions about the other variables you may want to test in the future.

Despite these drawbacks, if you’re new to landing page optimization, A/B split testing is definitely the way to start. Because of its simplicity and the availability of free tools, it’s easy to begin down the road of improving your conversion rate.

 
1

SPONSORED CONTENT

MORE ON E-COMMERCE >>

FROM THE BOOKSTORE

A guide to delivering a better user experience through A/B testing: Offers best practices and lessons learned from over 100,000 experiments run by over 2,000 Optimizely customersDetails a roadmap for how to use A/B testing to personalize your customer's web experience and a practical guide to start A/B testing todayAuthors Dan Siroker and Pete Koomen are cofounders of Optimizely.com, an user-friendly testing system that more than 2,000 organizations use A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

A guide to delivering a better user experience through A/B testing:
Offers best practices and lessons learned from over 100,000 experiments run by over 2,000 Optimizely customersDetails a roadmap for how to use A/B testing to personalize your customer's web experience and a practical guide to start A/B testing todayAuthors Dan...

ORDER NOW

Is social media baffling to you?Fundraising coach Marc A. Pitman thinks we live in the most exciting time for nonprofits! In Nonprofit Social Media: The Fundraising Coach’s guide to nurturing relationships from your desk, he sets out to prove it. Nonprofit Social Media

Is social media baffling to you?Fundraising coach Marc A. Pitman thinks we live in the most exciting time for nonprofits! In Nonprofit Social Media: The Fundraising Coach’s guide to nurturing relationships from your desk, he sets out to prove it....

ORDER NOW

 

COMMENTS

Click here to leave a comment...
Comment *
Most Recent Comments:
Rex Dixon - Posted on January 22, 2010
Have you checked out Performable yet? They are in closed beta - http://www.performable.com/ but are working together with another A/B Testing site - http://www.abtests.com/ ?
Click here to view archived comments...
Archived Comments:
Rex Dixon - Posted on January 22, 2010
Have you checked out Performable yet? They are in closed beta - http://www.performable.com/ but are working together with another A/B Testing site - http://www.abtests.com/ ?