We recently had an interesting experience with a news publisher that had been trying Echobox Email on a trial basis and was comparing the product to its existing solution, gauging the performance of its newsletters on an identical list.
However, its calculations differed quite significantly from ours and for a very good reason.
We know there are many tactics that can affect open and click rates. Some of these are in the hands of publishers, such as effective subject lines and preview text, for example. Some are due to the tools that solutions place at their service — personalised send times, for instance.
But an e-mail solution itself can also have a significant impact on open and click rates, to the extent that two identical e-mails sent to identical lists can produce different results.
Not all e-mail solutions are created equally, and publishers should understand exactly how they work and what they measure to ensure the selected tool meets a company’s needs.
The exact mechanics that can produce this divergence are hidden under the hood of the solution. This means the publishers themselves are very often unaware of what’s going on. Without knowing this, and without being able to compare kind with kind, it can lead to flawed decision-making.
So, how can an e-mail solution itself affect open and click rates? And, how can you remove bias when comparing solutions?
Open rates and deliverability
Maintaining good deliverability is central to a strong newsletter strategy. E-mail services like Gmail collect data on your e-mails. Repeated bounces can lead it to conclude that a publisher’s e-mails are spam and treat them accordingly. (Incidentally, this is also the reason why making it easy to unsubscribe is good practice. Subscribers who can’t quickly see how to unsubscribe from unwanted newsletters are far more likely to simply send it to spam and damage overall deliverability.)
It’s for the sake of deliverability that e-mail solutions themselves actively only send to a certain proportion of a total list. There is a delicate balance to be struck between maintaining deliverability and maximising sends, and each solution will prioritise differently.
A solution that is more aggressive in maintaining deliverability will be more proactive in removing bounces both hard and soft. A solution favouring sends will filter out a smaller number of subscribers who have previously soft bounced. For instance, if a publisher has 100 names on its subscriber list, one solution may send to only 98 while another might send to 96.
This produces a knock-on effect in other metrics. Given the same number of opens, a solution that is more aggressive in excluding subscribers and therefore sends to a smaller proportion of that list will produce artificially high open rates compared to one sending to a larger proportion.
The numbers involved may seem small, but over a large number of sends, they can add up. Where a newsletter’s subscriber list numbers in the tens or hundreds of thousands, a publisher can be missing out on thousands of opens and clicks despite a higher headline figure.
Calculating true open rates
If there’s one lesson to be learned it’s to always look below the headline figure.
To really compare the effectiveness of different solutions in terms of open and click rate, it’s important to calculate from overall list size. Typically, metrics displayed in the analytics of an e-mail solution show open rate as the ratio of opens to deliveries, masking the extent to which they filter soft bounces.
Comparing these stated open and click rates, therefore, won’t provide an accurate reflection of performance.
Open and click rates aside, each e-mail solution finds a different balance of deliverability and sends. There are no industry standards. Therefore, it is important for publishers to pay close attention to the proportion of a list that is not sent to and decide for themselves whether that balance is right for the business.
YOU MUST BE A MEMBER TO ACCESS THIS CONTENT
Become An INMA Member or Log In
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
