Inbox Inspector: review

By Mark Brownlow, October, 2007

MailChimp's Inbox Inspector is a standalone email design and delivery testing tool. You let it loose on an email and it reveals potential problems with how that email will display at various webmail services (like Gmail) and in various email software environments (like Outlook 2007).

It also examines how common anti-spam filters and technologies will likely treat the email. And highlights coding and content problems. The tool draws on the expertise provided by ReturnPath, giving smaller businesses and organizations access to the kind of design and delivery intelligence traditionally reserved for large corporations.

All well and good. But how does it shape up in practice? Here the results of my test drive:

The Inbox Inspector package

MailChimp is a well-known email service provider (ESP). So their main business is email list hosting and distribution. The Inbox Inspector is an add-on to their standard ESP service, but you can also buy it independently. You don't have to use MailChimp as your ESP to use the tool.

But you do need to sign-up for a MailChimp account (free), and then purchase credits for Inbox Inspector reports (US$29 for three credits).

To use the tool, you first set up a dummy email campaign using whatever email you want to test. Then you simply click on an "Inbox Inspector" button to run that campaign through the tool.

The resultant report is split into three sections:

1. Campaign preview

This gives you thumbnails of how your email looks in various webmail and software environments. At the moment these include:

AOL 9, AOL (web), Comcast, Earthlink, Gmail, Hotmail, Windows Live Hotmail, Lotus Notes 6.5.4, Mail.com, Outlook 2003, Outlook 2007, Outlook XP, Outlook Express 6, Thunderbird, Windows Mail and Yahoo (old and new versions)

If you click on a thumbnail to get the full screenshot, you often have options to see the email display as if images had been blocked or as it appears in a typical horizontal or vertical preview pane.

Since preview panes and image blocking are pretty important issues these days, this functionality is very welcome.

2. Spam filter check

This part of the report lets you know how your email is likely to fare when it encounters some common email filter software, firewalls and similar technologies. These include desktop filters (like McAfee), server-level filters (like SpamAssasin) and enterprise-level technologies (Ironport).

As well as indicating whether each potential barrier would let your email through, the tool also provides information on the "problems" picked up by each filter. Here's the feedback I got on how SpamAssasin saw one test email:

The spam check is much more comprehensive than the typical freebie spam checkers, but still only a guide of course. Most of these filters and firewalls have customizable settings, and the tool cannot account for that.

Still, the feedback gives you a feel for your likely delivery success. And the specific details on "problems" identifies areas where you need to make modifications (which may be as little as changing a word or two).

3. Content assessment

The final section highlights errors and issues in the code and words your design uses. This includes checks on whether links in your email work, whether any of these links are on domain blacklists that might cause the email to get blocked, spelling, code integrity and standards compliance, and spammy words.

Pros

The result is a well-rounded package of information which shows you where your email has problems.

The preview images cover all the common places your email might be read, and the screenshot quality is very high. So you can pick up on any little design incongruities that need further investigation.

The spam and content/code feedback lets you pinpoint the coding and content changes you should consider to improve your chances of making it into the inbox of your readers.

At $13 a throw, it's a no-brainer. For example, I was relieved to see my standard email template looks fine everywhere, as was the intention. Almost everywhere.

Now I know what I need to take care of. I also learnt that a specialist search site I always link to from every email, though reputable in itself, might cause delivery problems because it has a .info domain name. A problem easily solved in the next email campaign.

So...it gets my stamp of approval.

Issues

While available as a standalone application, the user interface is really built for customers using the company's ESP services.

As a result, it can take a bit of getting used to. For example, I got an error message when trying to save my dummy "campaign" as a draft, until I realized I first needed to create an email list to theoretically send to. All test emails must also include the MailChimp unsubscribe code, which obviously protects their normal customers from sending out emails that are not legal. It took a good few minutes to set up the system and actually use the tool.

Once you're over that hurdle, though, it's easy.

I turned up one or two minor bugs. On one of my two tests, for example, the Gmail screenshot only captured the right-hand side of the screen.

While nearly all information appeared within a few minutes, some preview screenshots still displayed a "come back later" message two to three hours after the test. So don't leave the test until the last minute...give the tool time to collect all the results.

The results also refresh at regular intervals, taking a few seconds to do so. I'd prefer a manual "refresh/update" report button.

These are all minor issues, though.

More documentation would be helpful in the future, too, such as explaining the display settings listed under each preview screenshot. The results you get are very interesting, but interpretation is often left up to the user. So while problems are identified (itself justifying the price of the service), you're on your own for finding some of the solutions.

This is not a criticism of Inbox Inspector per se, more of a warning that you can't expect an automated tool to do all the work for you. But it will do much of that work for you...and at a price that makes it easy to recommend.

My summary: a great tool for those serious about email design and deliverability.

Recent E-mail Marketing Blog Articles