Making the most of Screenshot Comparison Heading

Making the most of Screenshot Comparison

Ghost Inspector offers a limited implementation of Screenshot Comparison. This gives us the opportunity to implement some form of visual regression testing without using a specialist tool (e.g. BackstopJS, Percy). In this post, we look at the merits of the functionality and consider some examples of potential usage.

Recommendations

Expect the tests to require some oversight

The baseline screenshot is critical to the value of the comparison, and may need to be updated fairly frequently. Every failure should be scrutinised – decide whether there is a genuine failure (unexpected change) or whether the the baseline screenshot can just be updated. However, if you find you are having to do this frequently, consider the following advice or whether the test is yielding any value.

Check common elements separately

If there are elements that are present on most pages (for example, the header and footer), it makes sense to check these in isolation. You can then exclude these elements from the other comparisons so a failure of a common elements doesn’t also fail all other tests. As with all automation, we should always be looking to limit maintenance and the level of manual intervention wherever possible.

Exclude dynamic content from the comparison

You’re going to give yourself a serious maintenance overhead if the page/area you’re comparing is constantly changing. If dynamic content is a constant cause of comparison failure, then you need to accept that it’s not helpful to have that content included in the comparison. Thankfully, GI lets you exclude a list of selectors from the comparison (it will make them invisible, retaining the page layout).

Keep the tolerance as low as possible

What’s the value in a test if the tolerance is set at a level where small changes are ignored? Ideally, you should aim for a tolerance of as close to zero percent as possible. If your tests require a tolerance approaching or greater than 5%, you should question whether the comparison is valuable – that 5% change could potentially be critical. Sometimes, the page inexplicably shifts a few pixels horizontally, which can cause a false indication of failure that it can be tempting to try to bypass by increasing the tolerance. Sometimes using a different browser for the comparison (Firefox, for example) can minimise this.

Don’t just check the desktop breakpoint

It’s probably fairly obvious, but there’s no need to just check a single breakpoint. Generally, it’s best to prioritise targeting the primary differences between the different breakpoints (navigation changes, for example). While Ghost Inspector stores baseline screenshots for each resolution the test is run at, it’s generally cleaner to create a new suite for the different breakpoints, setting the capture resolution at the suite level. In many cases, you’ll be able to start the new suite as a duplicate of the first to save some time.

Consider specialist tools

While Ghost Inspector provides a great entry point to figuring out how to get value from visual comparison, it’s ultimately quite a limited implementation. Once you’ve decided you’re getting value from the tests, and would like to invest further, perhaps look at the numerous script- and configuration-based tools available specifically to handle visual regression testing. This article from softwaretestinghelp.com covers a variety of options.

Worked Example

Finding common elements

Let’s use mancity.com as our example. As with many sites, the header and footer appear on all pages. There is a slight variation in the header – the current section is highlighted, but let’s decide we aren’t concerned with this.

mancity.com Header Screenshot
mancity.com site header

mancity.com Footer Screenshot
mancity.com site footer (2 sections)

We could just focus on the content area of the header, but let’s use the full container to make sure we also compare alignment. We could use the whole header element, but the ad at the top is dynamic, and doesn’t always appear, so it’s better to rule it out.

mancity.com Header Source Screenshot
.mc-header__primary-wrapper is the CSS selector we require

For the footer, there are two sections we’re interested in. Both have static content, so we can be general with our targeting.

mancity.com Footer Source Screenshot
section.partners-section and .main-footer are the selectors we’ll use

Finding dynamic content to exclude

Let’s take a look at an article to see how we might want to approach the content. First of all, we want to make sure we’re excluding the header and footer, as we already cover those separately. We can see there’s a container for the article container, so let’s focus on that.

mancity.com Article Container Screenshot
.js-news-article is the selector we’ll use to target the screenshot

While we now have our target, there’s most likely a large amount of dynamic content we need to exclude from the comparison to avoid continual failure. While there are many examples in a mancity.com article, let’s just focus on a few here as examples. The first example is common on content sites – an ad served by Google in an iframe. We know that this is going to be subject to frequent change. There’s a lot of options for a selector to exclude the ad, but let’s just pick an easy one.

mancity.com Article Ad Screenshot
#article_page_slot_1 will do nicely

There are a couple of blocks at the base of an article designed to display related and popular content. Both of these are likely to be dynamic. In the case of the Further Reading section, we might as well keep the section heading in the comparison, as it’s unlikely to change, but the article link tiles need to be excluded.

mancity.com Article Further Reading Block Screenshot
.further-news will exclude the tiles, but keep the heading and horizontal rules in the comparison

Looking for key differences between breakpoints

The site navigation and footer are always likely candidates for layout changes in most sites. mancity.com is no different. There are other examples in the article of layout changes, but let’s just choose to focus on these two elements for this example.

mancity.com Mobile Header Screenshot
The Mobile breakpoint header/navigation variant
mancity.com Mobile Footer Screenshot
The mobile breakpoint footer variant

Configuring the tests in Ghost Inspector

Suite settings

Ghost Inspector Suite Configuration Screenshot
The choice here is to use Firefox at a desktop breakpoint with zero tolerance

Header/navigation configuration

Ghost Inspector Navigation Target Screenshot
We only need to specify a target

Header/navigation steps

Ghost Inspector Navigation Test Steps Screenshot

Header/navigation mobile variation

Ghost Inspector Mobile Configuration Screenshot
Our selector is fine for Mobile, too, so all we change is the script setting for screen size
Ghost Inspector Partner Footer Configuration Screenshot
We aren’t able to specify two targets. therefore we require two scripts for the two footer sections
Ghost Inspector Footer Configuration Screenshot
Ghost Inspector Footer Steps Screenshot
We can use the same step in both scripts
Ghost Inspector Mobile Configuration Screenshot
As with the header, the footer is fine without any changes other than screen size
Ghost Inspector Footer Configuration Screenshot 2
However, the Firefox browser fails to render the footer correctly. Using the Headless Chrome version gives us the expected baseline screenshot

Article configuration

Ghost Inspector Article Configuration Screenshot
Our target and exclusions are set (with more areas of dynamic content to the exclusions)

Article steps

Ghost Inspector Article Steps Screenshot

Article baseline screenshot with exclusions

Article Baseline Image Screenshot
We can see that only the article content was targeted, and all dynamic content excluded from the comparison

Final suite setup

Ghost Inspector Suite Screenshot

Leave a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.