Does email personalization have an effect on open rates?
Research released earlier this year MailerMailer suggested that personalized emails tend to see higher open and click rates, although the same study last year found the opposite result. Now, new data from MailChimp supports the case that personalization can drive higher email open rates. Measuring the impact on open rate by standard deviations from the norm, MailChimp found that personalizing both the first and last name has the biggest impact, followed by personalization of last-name only and first-name only. The results varied considerably by industry.
A quick word on methodology: MailChimp cautions that “it’s important to know that a standard deviation is a standardized measurement of how much something deviates from the average value. One standard deviation for a user who tends to see large swings in open rates will be a higher percentage than it will be for someone with consistent open rates. That means choosing words wisely will have a larger impact on open rates for people with a higher standard deviation, while users with very consistent open rates can expect to see smaller changes.”
The figures represent “standard deviations from the mean open rate fora user/list”.
That said, given that first name personalization is much more common than the other use cases, MailChimp delved further into this area, looking specifically at industries where the impact was statistically significant. The researchers discovered that the impact was particularly significant in a few industries, namely government (standard deviation of 0.92), creative/services agency (0.45), politics (0.3), computer and electronics (0.28) and hobbies (0.26). Personalization appeared to have had less of an influence on open rates in marketing and advertising (0.13) and business and finance (0.11), and actually was found to have a negative effect in the legal industry (-0.31).
Other interesting findings:
- Free vs. Freebie
The data suggests that the term “freebie” has a significant impact on open rates (0.26 standard deviations), much more so than the term “free” (0.02). Again, though, the impact of the term “free” varies substantially by industry: in those where it was significant, it ranged from the positive for recruitment and staffing (0.45), restaurants and venues (0.11) and e-commerce (0.06) to the negative for retail (-0.04), business and finance (-0.09) and travel and transportation (-0.25).
Use of the term “free” has also seen mixed results by industry, per subject line research from Adestra.
- Time Sensitivity
It looks like adding some urgency to email subject lines can have the desired effect. Emails using the terms “urgent” (0.79) and “breaking” (0.68) enjoyed much higher open rates than the norm, with “important” (0.55) and “alert” (0.31) also seeing above-average results.
- Announcements and Reminders
The research suggests that recipients are interested in reading announcements, but not cancellations. Subject lines with the term “announcement” sported an above-average open rate (0.46 standard deviations from the norm), although other iterations of the term – “announcing” (0.32), “announced” (0.26), “announces” (0.21), “announce” (0.21) and “announcements” (0.10) seemed to have less of an impact. As for cancellations (the term “cancelled” had a standard deviation of -0.4), it may well be that recipients simply browse the subject line to get all the information they need.
- Charity-Related Terms
Interestingly, each of the terms associated with charity appeared to have a negative impact on open rates, with the term “donate” (-0.56) the worst offender. That contrasts with the Adestra results – which had found an above-average open rate for charities using the term “donate.” As is always the case with these types of email studies, the data is interesting and a good starting point, but should not be considered prescriptive, and should definitely not replace readers’ own testing.
About the Data: The researchers described their methodology in the following way:
We studied approximately 24 billion delivered emails with subject lines composed of approximately 22,000 distinct words. If you think that sounds like a lot of data, you’re right. We looked at subject lines both in general and within specific industries. Here’s a quick rundown of our criteria and approach:
1) Investigate campaigns sent by users from the United States with tracking turned on in the past year. Only consider campaigns that were sent to 500 recipients or more, and only consider campaigns sent by users who have sent 10 or more campaigns before.
2) For each campaign, calculate the open rate and standardize it using the user/list average open rate and standard deviation.
3) Remove special symbols and convert subject lines to lowercase. For any given word, average the performance of all related subject lines and perform t-tests to identify high impact words.
4) For every subject line being tested, create flags for the presence of high impact words. Perform a correlation analysis on word presence to determine which words are frequently used together. Create additional flags for frequently used word combinations.
5) Perform a linear regression analysis to estimate the impact each word has on standardized campaign open rates when accounting for all other tested words. Repeat this process on industry-specific data sets.