An A/B test allows you to create multiple versions of a campaign – for example with different subject lines, calls to action, or other content – and then automatically send those different versions to a subset of your total recipients.
The version that results in better metrics based on the criteria you define will then be sent to the remainder of your recipients after a designated period of time. Or if you'd rather manually select the winner, you can also stop the test and have it sent to the remaining recipients immediately.
Create A/B Test
- Go to the Content step of an in progress campaign.
- Just below the subject line, click the Show text link next to the "Advanced" heading.
- Click the Create A/B Test text link.
- Click OK, and a second version of your campaign will automatically be created as an exact copy of the first. You might want to rename the tabs to better represent the changes you're making.
- Modify the content of the second version as desired such changing the subject name. Click Add Version text link to add additional versions. You can also rename the default Version 1 and Version 2 by clicking the down arrow icon next to the version name.
- Click the A/B Test Settings text link.
- By default, the different versions of your campaign will be setup to be sent to equal segments of random recipients representing 20% of your total recipients. Drag the arrow icon to adjust the sample size. A minimum sample size totaling at least 1,000 recipients is recommended for optimum results.
- Select the criteria on which the winning version should be selected: unique view rate, unique click rate, or unique clicks on a specific link.
- By default, the A/B test will run for 4 hours after the campaign has been sent. You can enter a different duration if desired, and/or you can manually select a winner and stop the A/B test (see step 13 below).
- When using the A/B testing feature, you can now choose which version of your Campaign to send as a test. This means you can thoroughly test all versions of your A/B test Campaign before pressing send to all of your recipients.
Just like the normal testing feature, you will be able to send a test Campaign of your Version to up to five recipients.
- After your campaign has been sent, you can view the results in real time via reports.
- Click the Reports tab in the main navigation menu.
- Click on the name of the campaign to view its details.
- In the upper right you will see "A/B Test" and the amount of time remaining as well as the "Select Winner" link.
- Click the A/B Test tab at the far right to view the metrics over time.
- At the end of the A/B test, the winning version will be sent to the remainder of your recipients.
- Or if you would prefer to suspend the test and immediately send a selected version to the remaining recipients, go to the campaign's reports and click the Select Winner link.
Please note: When copying over campaigns, they'll keep their settings. So be sure to make sure the A/B test is turned off when copying to an account without the functionality.