When a visitor is browsing your website, what impacts their willingness to start a relationship with your organization? The decisions you have made around your site's design, messaging, content, and layout. How confident are you in these decisions? As you update your site, how can you be sure the changes improve your visitors' experience?
A/B Split Tests are a tool for increasing confidence in the decisions you make about your website. They allow you to scientifically test and validate hypotheses around your goals. Google Analytics' "Experiments" tool makes it simple to setup A/B Split Tests on any website using Google Analytics. In this post, we'll walk through each of the steps.
1. Define your Goal
Some visitors' actions tied directly to your organization's website goals are important to setup as Goals in Google Analytics. Non-profit websites should track supporter engagement and fund raising. Political websites should track donations and volunteer registrations. Business websites should track leads and sales of their products and services.
Goals are a big topic of on their own. If this is your first time setting one up, check out this Guide on Setting up Goals in Google Analytics.
2. Form your A/B Test Hypotheses
With your goal of increasing engagement, or leads, or donations established, it's time to form your hypotheses: what design, messaging, content, or layout decisions do you think could achieve this goal more effectively?
With Google Analytics Experiments, you're not limited to just two, A and B, hypotheses. You can setup as many as you'd like. The algorithm Google uses is called a multi-armed bandit experiment. The statistics behind it are complex, but the idea is not. The gist is, as time goes on and hypotheses start showing promise Google will send more visitors to the leaders. It has two big advantages over a traditional A/B test that distributes traffic evenly until completion. First, by sending more traffic to winning variations while the experiment is running, you start to benefit from the leaders earlier. Fewer visitors are sent to poor performers before the experiment ends. A side effect of this is the other key benefit: deciding the best of multiple good choices happens more quickly.
Each hypothesis must be setup as its own page, with a unique URL, on your website. How this is done will depend on the content management system (CMS) your website uses. With a developer-friendly CMS, like HiFi, this is a simple task of cloning a page and changing its content or using an HTML template to target the structure of the hypothesis page in isolation.
3. Setup your Google Analytics A/B Split Test Experiment
With a goal and hypotheses in place, it's time to setup your experiment in Google Analytics. This is the easy part!
3a. Create Experiment
You can reach your experiments in Google Analytics from the reporting sidebar. Select Behavior, then Experiments. This is how you'll get to your experiments after you've set them up. Select Create Experiment.
Experiments only require a Name and the Goal you're evaluating success with. There are other options, too, like how much of your traffic should be experimented with, e-mail notifications, the confidence level with which a winner is determined, and so on. For your first experiments, you can safely use the defaults, but consider enabling e-mail notifications.
3b. Drop-in Original URL and Hypothesis Variation URLs
The next step is to add all of your hypotheses to the experiment. This is just a matter of copy and pasting the URLs you setup in. If you are testing changes to a page that already exists, use it as the Original. For entirely new content, pick the URL you will eventually want to use for the winner. The Original Page is where the code snippet will be installed in step 3c.
3c. Insert the experiment Testing Code
Once your hypotheses URLs are setup, the next step is to select "Manually insert the code" and install the experiment code snippet just inside the <head> tag of your Original Page.
After you have saved the template, click Next Step and Google Analytics will verify that your experiment code is properly in place.
All that is left now is to Start Experiment and Google Analytics will take care of routing visitors to your hypotheses and recording their effectiveness.
Deciding the Winner
The winning hypothesis will be decided algorithmically over some period of time, usually on the order of 2 to 4 weeks. This period will vary in length depending on the volume of traffic to your site, the relative performance of hypotheses, the rate at which the leading hypotheses converts, and so on.
For experiments that end with a clear winner, if it is not the original, there are just a few more steps to take. First, replace the original's content or template with the winning variation's. Second, remove the experiment code from the template.
For some experiments, you may be able to decide to move forward with a hypothesis before Google's algorithm statistically can. An example of this is a desirable change to your website that is performing the same as the original page it intends to replace and has a low probability of outperforming the original. In these cases, you can manually select Stop Experiment to end it, replace the original's content with the hypothesis', and remove the experiment code.
Google Analytics Experiments is a free A/B Split testing tool you can easily setup to help you make confident decisions about your website's design, messaging, content, and layout. It is a powerful tool you can use to make your website better at achieving your organization's goals.