- Business Management Tips
- Industry Insights
All Skill Levels
Join hundreds of other operators and REGISTER NOW for Spark 2024 New Orleans October 13-15th!
You’re probably familiar with the saying “if at first you don’t succeed, try, try again.” That’s A/B testing in a nutshell – evaluating two versions of something like an email subject line to determine which one is more successful.
A/B testing lets you choose the elements of your email that work and tweak the ones that don’t. That could look like choosing the subject line that generates a higher open rate, for example. Every element, from headlines and CTAs to graphics and email templates, has room for improvement. Skip the guesswork and use data to understand exactly what content your customers respond to.
Need a refresher on the benefits of A/B testing? Start with this guide, then follow these steps to run your first test!
There are many elements in an email campaign and, therefore, many possible combinations to test. To effectively compare campaigns, stick to one element at a time to get the most accurate results. For example, play around with various subject lines before moving onto message design. Here are some email elements you should consider A/B testing.
When choosing what to test first, think about which part of the conversion process needs improvement. For example, if you have a low open rate, start by testing subject lines. If your unsubscribe rate is high, focus on creating new content that better engages readers.
Think about which tests are likely to have the biggest impact and which ones you have the most confidence in. Some changes will be more obvious, such as switching a hyperlink to a button. Others aren’t as clear, like determining which email template will be best received. Start with tests for which you’ve already formed a hypothesis, as this will help you learn more from the process. Another factor to keep in mind is ease of implementation. Changing a subject line requires minimal effort, but redesigning email graphics is a bigger undertaking.
It can feel tempting to tweak a few elements at once. Just keep in mind that A/B testing is all about making small changes that lead to big improvements. Take Hubspot for example: They tested using a personalized sender name as opposed to a company name and generated 131 new leads from simply changing one small part of their emails.
Once you’ve figured out what you’re testing and the specific changes you’ll make, you need to consider your testing pool and the duration of the test.
The size of your testing pool depends on the size of your email list and how confident you are in the result. The most common confidence level used in A/B testing is 95%, meaning 95 out of 100 future emails you send should deliver the expected result. Use a sample size calculator to generate your ideal sample size. All you need to do is input the total size of your email list, your confidence level, and the margin of error you want (3% is common).
With A/B testing, it pays to be patient. Run your test for at least 24 hours to get significant results.
Are you familiar with the most important email marketing benchmarks? These metrics are key to understanding the results of A/B testing, as they can correlate to specific email elements and give you insight into what’s working – and what isn’t. Here’s an idea of which elements affect which metrics.
Ready to give A/B testing a try? Think of it as a way to get a leg up on other businesses in your industry. After all, only 31% of companies do A/B testing in their email marketing (Yesware).
To get started, see what A/B testing services your email service provider offers. Many of these platforms, like Mailchimp, have built-in tools that make it easy to test various elements, set a testing duration, and automatically send the winning version to the rest of your email list. Get in touch with your email marketing platform to see what they offer.
Looking for more email tips? Explore our email marketing guides.