Email Testing: How and Why

Email testing options

What do we mean by testing? Assessing the quality of an email, be it in its concrete or more abstract elements e.g. the words/format or its readability, and seeking to improve them. Of course, improvement is determined by your own set of metrics e.g. the open rate and/or click-through rate. However the choice of metrics may also change depending on the type of test, (explained below). It is crucial to set out before an email is sent, which metrics you would use to calculate the Campaign’s success.

Email testing is still being neglected by many email marketers and only a small amount of companies conduct regular testing; The reasons why include: tighter budgets reducing its priority, time constraints, and perceived complexity.

You can significantly improve your emails without falling into these traps. For example, a simple subject line test can alert you to which subject line achieves 10% more opens. For an email sent to 1,000 people, that’s an extra 100 viewers, which could potentially be an extra 100 conversions. In email marketing, incremental changes can have huge consequences.

So, how do you test? There are three options:

A/B testing

Email version A is deployed with a version B, usually with one contrasting variable e.g. subject line, both sent to a different small selection of the total recipient volume (commonly 5%). The winner of the two is determined, and sent as version C to the remaining majority (commonly 90%).

For more information on A/B Testing, click here.

Full Factorial testing

A multivariate system, you test several different elements - or factors - in an email. A considerable number of email versions are required, since every combination is tested. Success or failure of each email version is determined by their overall results.

Fractional Factorial testing

A multivariate system similar to Full Factorial, instead of the winning email version being dependent on overall results, the individual elements across tests are analysed. The final email is built to incorporate the consistently strongest individual elements.

Multivariate testing does require a thoroughness which takes time, including the creation of many email versions. It is therefore very difficult to advocate for short-term projects. This is where A/B testing is most useful.

 What to test in an Email

If it’s in the email, you can test it. Here is a list of the most essential elements to consider, and why you should seek to improve these:

1. Subject line

A good one can be the difference between the email being viewed or deleted.

2. From line

It’s vital in first communicating your business brand.

3. Landing page optimisation

A clickthrough to an unsuitable page could greatly undermine the user experience.

4. Personalisation

Now more frequent at a basic level, it’s important to reach out directly.

5. Drop day

A send on your optimal day will see the best chance of immediate response.

6. Drop time

Similarly, a send at your optimal time will have the same effect.

7. Creative

Minor improvements in design can dramatically affect engagement.

8. Content

Exactly what is communicated is obviously paramount and must be relevant.

9. Link placement

The positioning of key links and buttons strongly influences clickthroughs.

10. Link design

If you get it right, you will entice interaction which may convert.

11. Link frequency

Too many or too little can cause confusion, so this should be balanced.

12. Social media integration

The right exposure may lead to greater email readership.