Unlimited Testing: Let’s Test Your Emails with AI

Testing email marketing effectiveness is an active research area because email remains an important channel for customer acquisition and retention. Email open rates are a key measure of campaign effectiveness. Scholars identify three predictors of open rates: recipients’ characteristics, headline characteristics, and sending time. The industry-favored A/B testing has three drawbacks: it takes hours, depletes lists available for main campaigns, and limits testable email versions because of sample size and power requirements. These limitations continue to motivate researchers to build and improve open rate prediction models. Although they reduce testing time, models developed in marketing use only recipients’ past open rates as predictors. By contrast, models in computer science typically use only email headline characteristics as predictors. Consequently, current models’ open rate prediction errors are high. The authors address the limitations of both literature streams and use all three predictors and machine learning to build an email open rate predictor (EMOP) based on their universal emotion detector (UED). They test EMOP on data from four brands and set state-of-the-art prediction results. Experimental validation shows that EMOP can pick the best headline from a set of professionally generated headlines. Also, UED ranked second at the SemEval 2018 Task 1 E-c competition as of January 5, 2023.