An email A/B test (also called a split test) sends two or more versions of a message to small, random slices of your audience. After a set waiting period, the version that performed better on the metric you chose is sent to everyone who hasn’t received anything yet. The rest of the list only ever sees the winner, so the test costs you very little.
The golden rule is to change one thing at a time. If version B has a new subject line, a new sender name and a different hero image, you can’t tell which change made the difference. Common variables are the subject line, the preheader, the sender name, the main offer, the call-to-action wording and the send time. Subject lines are the easiest place to start because the effect shows up quickly.
Sample size matters more than most people expect. With a list of 3,000 and a 10% test group per version, each version reaches only 300 people; a gap of two or three opens between them is noise, not a result. Small lists get more reliable answers by testing bigger differences (a question versus a statement, a discount versus no discount) and by repeating the same kind of test over several campaigns.
Choose the winning metric with care. Since Apple Mail Privacy Protection, opens are inflated for many audiences, so a test decided on opens can pick the wrong version. Clicks are a cleaner signal for content and offer tests. Leave the test running long enough too: two hours is fine for a daily-deals list in Mumbai, but a B2B list may need most of a working day before the result settles.
A saree shop in Surat sends a Diwali campaign to 12,000 subscribers. Version A says “Diwali sale: up to 40% off silk sarees”; version B says “Your Diwali silk saree is waiting”. Each goes to 10% of the list. After four hours B has the higher click rate, so the remaining 80% receive B automatically.
MailNesto lets you test subject lines, sender name, whole content or send time, with a test group you size yourself; the winner is picked by opens or clicks and sent to the rest automatically.