Common SEO Split Testing Mistakes

For any SEO who wants to call themselves data-driven, SEO split testing is a relatively new concept and an essential tool. The public has been familiar with A/B testing in the context of Conversion Rate Optimization (CRO) for a long time, and applying those concepts to SEO is a logical next step if you want to be confident that what you’re spending your time on is actually going to lead to more traffic.


Before diving into how it’s done wrong (and right), it’s worth stopping for a minute to explain what SEO split testing actually is.

During SEO split testing, we’re trying to ascertain which version of a page will perform better in terms of organic search traffic. If we were to take a CRO-like approach of bucketing users, we would not be able to test the effect, as there’s only one version of Googlebot, which would only ever see one version of the page.

To acquire around this, SEO split tests bucket pages instead. We take a section of a website in which all of the pages follow a similar templateand make a change to half the pages in that section. That way we can measure the traffic impact of the change across the variant pages, compared to a forecast based on the performance of the control pages.

Also Read:- Facebook Is Going To Shrink Ad Space On Mobile News Feed

Testing Mistakes

  • Running split testing for a short time only

As SEOs, we know that it can take a while for the changes we make to take effect in the rankings. While we running an SEO split test, this is borne out in the data. SEO split test often takes a couple of weeks for the uplift to show.

The other factor to bear in mind here is that the longer you leave it after this initial flat period, the more likely it is that your results will be significant, so you’ll have more certainty in the result you find.

·  Testing those pages that don’t have enough traffic

As we need to follow the rule of thumb while split testing, according to this, if a site section of similar pages doesn’t receive at least 1,000 organic sessions per day in total, it’s going to be very hard to measure any uplift from your split test. If you have less traffic than that to the pages you’re testing, any signal of a positive or negative test result would be overtaken by the level of uncertainty involved.

  • Bucketing pages arbitrarily

In an SEO split test, we need to apply more nuance to this approach. For site sections with a very large number of pages, where the traffic is well distributed across them, the purely random approach may well lead to a fair bucketing, however, most websites have some pages that get more traffic, and some that get less. As well as that, some pages may have different trends and spikes in traffic, especially if they serve a particular seasonal purpose.

·  Doing pre/posttests instead of A/B tests

When people talk colloquially about SEO testing, often what they mean is making a change to an individual page (or across an entire site) and seeing whether their traffic or rankings improve. This is not a split test. If you’re just making a change and seeing what happens, your analysis is vulnerable to any external factors, including Seasonal variations, Algorithm updates, Competitor activity, your site losing or gaining backlinks, any other changes you make to your site during this time