To a passionate member of the team there is little worse feeling than finding a bug related to something you verified yourself and missed. Maybe it seemed fine or maybe you just got used to the going through the motions and thought the functionality and user interface were OK at first glance. But…they weren’t. You missed an issue and this resulted in a poor experience for the customers.
After some time you get accustomed to the product and you become change blind to small things being moved/adjusted/replaced/etc. In these cases even standard Automation tests can run fine but what about everything you did not see, the elements that were missed because they aren’t part of the test, the parts you don’t see because you did not automate it? “How do I fix this? We need more eyes on it, better eyes.”
That’s when the lightbulb went on. We decided to compare the product against itself for differences even more sharply than can be seen with the naked eye.
Over the last few months our engineering team has decided to tackle this challenge via “Screenshot Testing” using Blink-diff(https://github.com/yahoo/blink-diff) as an NPM package within Selenium to capture the screenshots. Just what is Screenshot Testing and why should it matter to me? We have taken our test automation to a new level by capturing full page screenshots of our production and staging UI/UX then comparing the two to ensure the HTML, CSS and all visible elements appearing as expected. This gives us the best chance to catch any visual defects before they ever hit production. This kind of testing is fast and easy to use resulting in a cleaner product for your customers, less time manually thumbing through the testing, more time for fixes, and lots of customer satisfaction since there is less developer time going towards bug-fixing, and more towards feature development.
We run Screenshot testing in conjunction with our functional automation tests. Screenshot Testing has been a huge win for the MindTouch team, and there is no reason it should not be one for you as well. This is a truly powerful tool to help deliver a clean product to your customers, and fewer worries for your developers and UI/UX teams.
Above is an example problem of what we were able to find in Screenshot testing. The back arrow class broke. We were able to fix this change and it never went customer facing.
We hope you you can see our success here and we also hope this has contributed to how you search for UI inconsistencies. Please keep checking back on our main Blog to see more articles like this.
All the best,