DON’T MISS OUT

Get updates on new articles, webinars and other opportunities:

Your Step-by-Step Guide to A/B Testing with Google Analytics

by Shane Barker

A/B testing can be as simple as reciting the alphabet…

You design two versions of a web page (A & B), divide the traffic between the two, and choose the one that gives you the maximum conversions.

Simple, right? Wrong.

Most newbies to A/B testing struggle with which tools to use, how to set up their test, and how to know when it’s done. In this article, I’ll show you a free tool, readily available to every website owner. I’ll also give you some guidance on setting up your test and knowing when to call it done.

Before you start

There are many things you can learn from an A/B test. It can be used to determine whether you should focus on single conversion goal or strive for multiple conversion goals. It can help you know which design elements and messaging are most persuasive for your audience.

No matter what you’re testing, keep your priorities straight. The end goal of any CRO (conversion rate optimization) process should be to increase your total revenue.

Imagine you’ve set up an A/B test to choose the best page design for increasing your subscriber rate. It works. Your subscription rate goes through the roof—but the design somehow hurts your sales rate and results in lower revenue.

This might make you insane. Do you keep your winning design? No. Always choose the page that will increase your bottom line, not just your conversions. Remember, companies run on revenue, not on conversion rates.

A conversion for a business could be anything.

  • For bloggers, a single subscription could be considered a conversion.
  • For eCommerce, a conversion might be a sale, subscription, newsletter sign-up, product carting, or even an event click.
Before you can start conversion testing, you’ll first need to define what your goal is for each test, so you can accurately identify the winning page design.

a/b testing Google analytics

There are many conversion testing tools on the market, but the best are usually paid and add to your marketing expenses. The one exception to this rule is Google Analytics.

It’s very simple to use Google Analytics for A/B testing or split testing with two or multiple variations in the website design. Now, I’m going to provide you with a step-by-step guide for easily testing your web page through a Google Analytics content experiment.

  • Choose an Experimental Objective

Google has combined A/B testing and split testing into one term—content experiment.

You’ll find it in Google Analytics under “Behavior” and “Experiments.” On that page, simply click the button for “Create Experiment.”

behavior -experiment

You’ll be take to a screen (pictured below) where you can set up your experiment.

First, add in the experiment name. For instance, if you want to perform A/B testing for a sign-up form or product selling button, then create a descriptive name that makes it easy to identify the experiment.

Under “Objective for this experiment,” you’ll define the metric you’ll use to evaluate the results from your test. Metrics can be chosen from Adsense, Ecommerce, Goals, Site Usage, etc.

  • If you’re looking to improve ad clicks or impressions, then choose the Adsense option.
  • If you want to boost revenue or the number of transactions, select eCommerce.
  • If you have predefined goals like session duration, event attendances, or destination page clicks, then opt for the goal metric.
  • Lastly, if you’re looking to better user experience through average page views or time on site, go for site usage.

The best part is you can set multiple metrics at one time.

set objective of a/b testing
  • Divide Your Web Traffic

Once you’ve set the objective, you can divide the percentage of web traffic for the content experiment. This will control how many people visiting your website will see one of your test pages as opposed to your original page.

For quick results, you may want to include a high percentage of visitors in the experiment. However, if your experiment is rather drastic or risky, include only a small percentage of your website’s traffic. It’s also smart to turn on the email notification to stay updated on any changes occurring in the experiment.

In the “Advanced Options” tool, you can control how to divide the traffic by turning on the “Distribute traffic” toggle button. Enable this option to assign an equal amount of traffic to each variation for the life of the experiment.

If this button is left disabled, content experiments will follow the default behavior by adjusting traffic dynamically based on variation performance.

From there, you should set the minimum experiment time at three weeks for the best results.

Google Analytics also allows you to fix the confidence threshold for your content experiment to determine the minimum confidence level that must be achieved before a winner can be declared.

The higher the threshold, the more confident you can be that the winning web page has competed well against the other design. Keep in mind that higher thresholds can make your content experiment considerably longer as Analytics waits to crown a champion.

Use this confidence calculator to determine your confidence threshold.
  • Configure Your Experiment and Code

The next step is to configure the experiment by adding in your original web page and your test pages.

As you can see in the image below, you simply need to enter the URL of your current page and all variation web pages.

Once you add the original and test pages, look over the preview image to be sure you’ve entered the right URL. Hit the “Save Changes” button after you check for completeness to head to the next experiment section.
configure the experiment

Now, you’ll need the experiment code for your testing project.

If Google Analytics tracking codes are properly installed on your original and variation pages, an experiment code will be immediately visible in the box.

Place this code immediately after the opening head tag at the top of your original web page. Once it’s added, hit the “Save Changes” button again to progress to the final step.

setup the a/b testing code
You can use the Google Content Experiments plugin to enter the code on your page.
  • Review and Get Started

When you’ve added the code, Google Analytics will validate it and show any errors that have been encountered if applicable.

Sometimes Analytics isn’t able to find the code. In this case, you can skip the validation phase as long as you’re sure the code was properly added. Google recommends skipped validation as only a last resort move. Instead, check your page for any errors that may have been introduced.

Otherwise, you’ll be given the green flag to start your content experiment. Your experiment will launch and you’ll start seeing reported data within one to two days.

review the experiment
  • Check Your Results

After your experiment has run its course, Google Analytics will declare the winner based on your previously defined metrics and confidence threshold. It will take at least three weeks to reach this step.

By reviewing your results, you’ll identify the page that performs the best. You can then publish this as the page you want viewed by all website visitors.

Easy. But does it work?

Kapitall increased conversions by 44 percent through A/B testing from a Google Analytics content experiment. So clearly it does.

Overall, Google Analytics is a free tool that’s very easy to configure for running testing experiments because the search engine handles all of the dirty work.

There’s one downside though. Analytics doesn’t support multivariate testing, which is a well-known technique for testing multiple variables like color, text size, and buttons all at once. Google Analytics can’t be used for an email campaign either.

That said, it’s still the best option for running A/B testing on your landing page and conducting a content experiment at the very low price of free.

Have you used Google Content Experiments? What was your biggest win?

14 Comments

DON’T MISS OUT

Get updates on new articles, webinars and other opportunities:

Shane Barker

Shane Barker is a digital marketing consultant, named the #1 social media consultant in the nation by PROskore Power Rankings. He has expertise in business development, online marketing and is an SEO specialist who has consulted with Fortune 500 companies, government agencies, and a number of A-list celebrities.

14 COMMENTS

Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.

SPEAK YOUR MIND

Your email address will not be published.

  1. amar says:
    June 22, 2016 at 5:51 am

    Hi,

    I have two landing pages in the AB Experiment under Google analytics. Under the advanced option in the settings, I have also selected the option of equally distributing the traffic to both the landing/ testing pages. Here is my question:

    Suppose, at the same time, I am running Google ads & other social media ads, And all of the ads have the landing. final URL to one of the pages (the original page) in the AB experiment. I want to know that is google going to split the traffic generated from different sources like Adwords and social media as well to the two different landing pages which are part of the AB testing?

  2. Anonymous says:
    March 11, 2016 at 4:58 am

    Hey, Thanks for your share !!

    I have quick question about this A/B testing. I have word press blog with 500+ posts, here i want to check with homepage adsense ads placement clicks + impression with new design, so my question was, how to create a testing page for my homepage with new design & in what URL i want to get publish in my server ??

    If homepage seems: http:mydomain.com
    Variation page url should be: ??

    Please suggest me to start a quick testing

    Thanks

  3. Chloe says:
    February 2, 2016 at 6:20 am

    Hello,

    Thanks for this post! Super clear and easy to follow. My test has been running for about 9 days now and I just went to check on it. I’m worried as there seems to be no data (graph is a horizontal line and All Sessions reads 0.00%).

    I clicked the Re-validate button and to check my pages for working code and both pages pass the tests. At the bottom, it also says “Note: Two experiment variations do not appear in the table.” Do you know what I’m doing wrong?

    Thanks!
    Chloe

  4. Jim Parris says:
    January 6, 2016 at 10:27 am

    Since I don’t think anyone mentioned this yet, it is important to keep in mind that this is a javascript-based redirect.

    It has to read the javascript, execute, confer with Google about which segment the user is in (to show experimental or original page), and then redirect the user.

    This causes a different experience.

    Also, the url will change to the experimental version (obviously).

    For beginners, and if nothing else is possible, this is a very convenient solution for quick A/B testing.

    However, for serious testing stick with logic implemented on the webserver or load balancing level that uses server-side redirect logic to avoid the bad experience of a page-half-loaded-redirect and to hide the different URL.

    • Daniel says:
      January 19, 2016 at 2:55 pm

      @JIM, is it really possible to hide the different url? Do you use some proxy redirect? I guess you cannot use a regular temporary redirect (302).

      The site I would like to a/b test is accessed through a cdn. I guess this makes it even harder to hide the url. As soon as the html is cached every user will get exactly the same html. Or?

  5. Ivaylo says:
    November 18, 2015 at 3:43 am

    Nice post. Very informative. I have a quick question though: If one have several pages (variations) to test is it better to test pages 1 vs 1 at a time (A vs B) for 3 weeks lets say and then the next 3 weeks to test another 1 vs 1 (B vs C) and so on…
    OR is better to run the test all at once testing 1 page vs. all the other variations (A vs B,C,D,E…) at once.

    In other words which is more accurate: – adding multiple variation pages or testing one variation page at a time?

    ty

    • Shane Barker says:
      November 20, 2015 at 12:23 pm

      Ivaylo,

      Google Analytics content experiment is based on A/B/N model where you can check multiple pages at a time. In your case, you can start experiment all pages at a the same time, but make sure to get the best page result you need to run the test for at least 2 months (More is good) so that you can analyze data and get the best page with the highest conversion rate.

  6. Alexander says:
    October 30, 2015 at 6:06 am

    Thanks for publishing this post. I found it through BuzzSumo. As I’m workinh in CRO area I found this guide useful for the beginners. And I have to say it’s quite difficult to use Goggle Content Experiment as it demands participation of IT team.
    Thee is a couple of tools for A/B testing but the best known of them have quite poor GA integration.
    There is an A/B testing tool with visual editor and full GA integration changeagain[dot]me. It’s for marketers without coding skills and suites for the beginners.

  7. Jeffrey says:
    September 25, 2015 at 1:48 am

    Great post.
    Is it possible to add the code via Google Tag Manager?
    I’ve been looking for this feature, but haven’t found it…

  8. sam vanderbempt says:
    September 18, 2015 at 3:39 am

    Hi Shane,

    I was wondering if there’s a way in GA to perform an A/B test on a group of pages. For example, let’s say I have an ecommerce site where you can buy a certain product. I want to test the layout of my product detail page, to try to get more people to click the buy-button. Of course, my website has hundreds of toys, and testing it on one specific product page would not be enough.

    Is there a way to test all the pages in http://mywebsite.com/toys/… versus http://mywebsite.com/toys-test/… if all product URLs are identical besides the /toys/ or /toys-test/ directory? It’s easy to change the layout for all my product detail pages at once, but can I also test all of these pages?

    Thanks!

  9. Shalin says:
    August 23, 2015 at 9:24 pm

    Does the second page get indexed and get traffic seperately as well? In that case wouldn’t that be mixed data?

    • Shane Barker says:
      September 26, 2015 at 1:04 am

      Shalin,

      Yes, it could get indexed, but you need to place the meta noindex tag to the test page to keep the data clear and get precise results.

  10. June 3, 2015 at 8:30 am

    We’ve used Google Analytics Content Experiments for all our testing thus far. It’s an easy to use tool, but for advanced testing you’ll want to go with a dedicated testing platform. The nice thing about testing through GA is having all your data in one place. You can also create segments within your test results to see how your A/B test stacks up on mobile, with new visitors, etc.

    Great post Shane!

    • Shane Barker says:
      June 6, 2015 at 5:41 pm

      Indeed, we can use segmentation within A/B testing for better insights. I would surely write a further post on it 😉

      Thanks Man!!!

Show Me My Heatmap

A3: @CrazyEgg was one of the best tools I have used to understand what and where users are engaging with a page. #seochat

Matthew Diehl

@matthewdiehl