All Posts
2026-02-18

Visual Regression Testing with AWS CloudWatch Canaries

AWSCloudWatchTestingDevOpsVisual Regression
V

Visual regression testing is a critical process for detecting unintended changes in an application's user interface (UI). While traditional functional tests can check if buttons work, they can't always tell if a button has disappeared, is the wrong color, or is obscured by another element. This is where AWS CloudWatch Synthetics Canaries come into play.

What is Visual Regression Testing?

Visual regression testing captures screenshots of your application and compares them against previously approved "baseline" screenshots. If there is a difference between the two images above a certain threshold, the test fails. This is perfect for catching CSS changes, broken layouts, or missing images.

How to Do It with AWS CloudWatch Canaries?

AWS CloudWatch Canaries are configurable scripts (Node.js or Python) that simulate user traffic. They interact with your web application using Puppeteer or Selenium. For visual testing, the syn-nodejs-puppeteer runtime is typically preferred.

Step 1: Create a Canary

Go to the CloudWatch service via the AWS Console and click the "Create Canary" button under the "Synthetics Canaries" section. You can choose the "Visual Monitoring" blueprint or write your own script.

Step 2: Prepare the Script

Below is an example Node.js (Puppeteer) Canary script that performs a simple visual comparison:

const synthetics = require('Synthetics');
const log = require('SyntheticsLogger');

const pageLoadBlueprint = async function () {
    // Set the URL
    const url = "https://example.com";

    let page = await synthetics.getPage();
    
    // Navigate to the page
    await synthetics.executeStep('GoToUrl', async function (timeoutInMillis = 30000) {
        await page.goto(url, {waitUntil: ['domcontentloaded', 'networkidle0'], timeout: timeoutInMillis});
    });

    // Take Screenshot and Compare
    // Takes a screenshot named 'visual-test' and compares it with the previous baseline
    await synthetics.executeStep('VisualCheck', async function () {
        await synthetics.takeScreenshot('visual-test', 'main-page');
    });
};

exports.handler = async () => {
    return await pageLoadBlueprint();
};

When this script runs, CloudWatch saves the screenshot taken during the first run as the "baseline". Subsequent runs compare the new screenshot against this baseline.

Step 3: Examining Differences

If a visual difference is detected, the test is marked as failed in the CloudWatch Synthetics console. When you drill down into the details, you can see the Baseline, Current, and Diff images side-by-side. Differences are usually highlighted in red, allowing you to immediately understand where the problem lies.

Tips and Best Practices

  • Dynamic Content: If your page has constantly changing content (ads, date/time, exchange rates, etc.), perform DOM manipulation in your Puppeteer script to hide or mask these elements during testing.
  • Thresholds: sometimes, due to rendering differences, a 100% match might not occur. You can set a threshold value in CloudWatch settings to ignore minor differences.
  • Different Devices: Configure your Canary to run with different viewport sizes (mobile, desktop) to test responsive design as well.

Conclusion

AWS CloudWatch Canaries allows you to perform powerful visual regression tests without requiring infrastructure setup. This way, you can catch UI bugs before moving your code to production and always provide a flawless experience to your users.

<div class="toc"> <ul> <li><a href="#what-is-visual-regression-testing">What is Visual Regression Testing?</a></li> <li><a href="#how-to-do-it-with-aws-cloudwatch-canaries">How to Do It with AWS CloudWatch Canaries?</a></li> <li><a href="#conclusion">Conclusion</a></li> </ul> </div>

Kaynak / Source: https://awsfundamentals.com/blog/visual-regression-testing-with-aws-cloudwatch-canaries