Your audience isn't homogenous or uniform. Streams of diverse people visit your site as a result of your current and past marketing activities. None of these streams fit together very well, so it’s dangerous and misleading to stitch them together into a unified picture.
Your audience also acts differently under certain circumstances and conditions. Time of day can have very strong behavioral effects, for example. People browsing surreptitiously at work spend less time on your site than people on their home computers in the evening. Time zones can cause a shift in your audience at different hours of the day, as more international visitors have their daytimes.
Such effects can be mitigated by running conversion tests in week-long units. But longer-term time variations, such as external event-driven behavior or seasonality, also can play a strong role and are much harder to deal with.
When you look at the suitability of the traffic mix for conversion testing, watch for three important characteristics: recurring, controllable and stable.
Recurring. The traffic must come from a replenishable resource. For example, pay-per-click or banner ad traffic is essentially endless — you can get more of it as long as you're willing to pay. This supply of “fresh meat” is important because it enables you to run conversion tests on new people who haven't been exposed to your company or Web site before. It’s OK to have a high percentage of repeat visitors in your test, as long as the mix of visitors doesn't change and it represents a roughly constant percentage of your traffic.
Controllable. It’s easy to control paid search and other online media buys. Unfortunately, other traffic sources aren't under your command. For example, organic SEO depends on changes in the ranking algorithms of the search engines, and you can't predict what mix of currently high-ranking keywords your traffic will arrive from. You also can’t control the context in which your site was seen, such as the text of the search result that was shown alongside your link. Nor can you directly specify the pages on which the traffic will land. However, SEO traffic still can be used for tests if it has a record of being historically stable in terms of the volume and mix of landing pages.
Stable. Even if your traffic is recurring and controllable, it may not be stable. You may see a periodic traffic spike if one of your marketing partners pushes a special recurring campaign that drives visitors to your site, for example. Or the turnover in the composition of your affiliate program results in a rapidly changing traffic mix. SEO traffic also can disappear overnight as the ranking algorithms are adjusted.
There's an old computer acronym, GIGO — “Garbage In, Garbage Out." The same holds true for landing page testing. If you don't have clean traffic sources for your landing page test, the results are highly suspect or downright wrong.
Tim Ash is the president/CEO of San Diego-based SiteTuners.com, a landing page optimization company. Tim's a frequent columnist and writer on conversion improvement, and is the author of the best-seller book “Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions.” You can reach him at tim@sitetuners.com.




Social Media ROI
Email Marketing that Works (2nd Edition)