The web is full of different types of tools for testing and optimisation. The use of these should pe a part of the process when doing digital stuff (web, email, mobile), it should be mandatory but it is not. Why? It is a help for everyone: coders get technical feedback, project managers get documentation and ease of mind and designers can react if something is not working. Still we are not using them, and I believe the reason is that we are lazy and stuck to our processes. Especially in the advertising business the schedules are super tight – so tight that something always has to be minimized (or even cut out) from the process. That part is usually the testing, it is made social (let’s launch and then fix it when people stumble onto errors).
I’m as guilty here as the rest of us, but finally I took the step from just thinking to actually doing stuff. Change the process and force in testing, but in a smart non obtrusive way…then learn from doing and fine tune it. As always there is 2 ways to go:
- planning to start testing and thinking about what elements to really test and and and…
- just pick one tool that looks good and try it out
We did the latter…
The case
Super simple but then that’s a great way to start: an email template for Savonlinna Opera Festival. As a tool we chose Litmus as I had bookmarked it in the past and it looked like it had the things we needed for the right price. Here is a screenshot with the summary of all email clients that were tested:
The technical stuff (the coding of the html email) was done by Arttu Manninen, and as a bonus he made an email testing tool for Midgard ). I asked Arttu a couple of questions about the findings and about the tool.
- The overall feeling of Litmus?
- the learning curve was almost non-existant, it is an easy tool that seems to be build right
- one great feature is version control, you can revert to older versions
- Is it customizable (how does a text version of the mail look like, what if the images are blocked, …)?
- they seem to have stripped down the features for ease of use, there is almost no room for customization in Litmus except for choosing mail clients
- Anything annoying?
- one small thing is that you have to change the recipient for each test to a Litmus-generated email. I’m sure there is a way to skip this step (think of support email etc).
Summary and ideas for improvement
Overall Arttu was very pleased with Litmus, especially for the email part. I found that having a tool that does the basic testing and documents it is something that I will demand for all projects in the future – especially the email projects. Some features I’d like to see in a near future (there might be other tools out there that have these features):
- mobile email testing
- testing with settings (text based, html without images, …)
The browser test is not as useful as the email test at the moment. Sure one can test that the layouts work in all the browsers (and this is really relevant stuff), but browsers are dynamic and email is static. Here are some ideas that would make the tool even better:
- click path
- define a path that will be clicked that all browsers go through and capture it with screenshots
- measure the time it takes to go through the path
- social media
- login with facebook/twitter/google and see what happens with the click path
- alternative interfaces
- how does the site look like with mobile browsers
- print views
- accessibility browsing
- …
If you have tried other test tools, please leave a comment.
