Step #1: Set your goals. Step #2: Set your test. The biggest mistake marketers make is testing when they shouldn't - Set aside a random segment of your audience. This will be your "test" without meaningful or clearly defined goals in place. group who will see the variants you're testing and allow you to gauge audience response to your messages. To avoid this aimless limbo we recommend starting testing with your subject lines. Numerous sources of research - which you can find in The most important thing about your test group is making sure it's large our blog on the subject - show that more opens means more clicks enough so that the test results you get are statistically significant (see and conversions. Plus, you can get data from subject line testing that our definition below), but small enough so that there's still plenty of you can then apply to the rest of your email content later. room for the winner to go out to the control group at the end of the test. So if you're testing email subject lines, your goal - the KPI that you're Using 30% of your total database for split tests is a reasonable trying to move the needle on - is your engagement. Engagement is assumption for average-sized datasets, but for particularly large measured by open, click and conversion rates, so it’s up to you which of datasets, you might consider decreasing the percentage to optimize these metrics you choose. potential revenue once you’ve identified your highest-performing subject line. It’s worth remembering that Apple’s Mail Privacy Protection measures have introduced some statistical noise to open rates, so some marketers For a shortcut to setting your test group based on your audience have shifted their focus towards clicks and conversions. size, head to our split test calculator! Check out our Orange Paper on what you can do to mitigate the impact of Mail Privacy Protection. What is statistical significance? Statistical significance (or "stat sig") is the likelihood that a relationship between two or more variables is caused by something other than chance. Subject line split testing with a large enough group of recipients will get you results that are reliable enough to draw conclusions from. Test with small groups at your peril! All you'll get is a few random, largely meaningless data points. www.phrasee.co 5