Screenshot API for Testing: Automated Visual Regression Testing

What is Visual Regression Testing?

Visual regression testing is an automated quality assurance process that compares screenshots of your application before and after code changes to detect unintended visual differences. Unlike functional testing that validates behavior, visual regression testing ensures your UI looks correct across all browsers and devices.

The process involves capturing baseline screenshots of your application, making code changes, capturing new screenshots, and comparing them pixel-by-pixel to identify differences. Any detected changes are flagged for review, allowing developers to catch CSS bugs, layout issues, and rendering inconsistencies before they reach production.

Common use cases include testing responsive designs, validating cross-browser compatibility, detecting CSS regression bugs, verifying component library changes, and ensuring consistent branding across pages.

Why Use a Screenshot API for Testing?

While tools like Puppeteer and Playwright offer screenshot capabilities, using a dedicated screenshot API provides significant advantages for testing workflows:

• No infrastructure management: No need to maintain browser instances, handle updates, or manage server resources.
• Cross-browser testing: Test across Chrome, Firefox, and Safari without setting up multiple environments.
• Consistent rendering: Guaranteed consistent screenshot output across different machines and CI environments.
• Scalability: Handle parallel screenshot requests without worrying about resource limits.
• Easy integration: Simple REST API that works with any testing framework or CI/CD pipeline.

Basic Screenshot API Usage for Testing

Here's how to capture screenshots using the CustomJS API for visual regression testing. This example shows the basic setup that can be integrated into any testing framework.

Simple Screenshot Capture

const axios = require('axios');
const fs = require('fs');

async function captureScreenshot(url, filename) {
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: url,
        commands: [],
        box: {
          x: 0,
          y: 0,
          width: 1920,
          height: 1080
        }
      }
    },
    {
      headers: {
        'x-api-key': 'YOUR_API_KEY',
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  fs.writeFileSync(filename, response.data);
  console.log(`Screenshot saved: ${filename}`);
}

// Capture baseline screenshot
await captureScreenshot('https://example.com', 'baseline.png');

Python Example for Testing

import requests
import os

def capture_screenshot(url, filename):
    api_url = 'https://e.customjs.io/screenshot'
    headers = {
        'x-api-key': 'YOUR_API_KEY',
        'Content-Type': 'application/json'
    }
    payload = {
        'url': url,
        'viewport': {
            'width': 1920,
            'height': 1080
        },
        'fullPage': True,
        'waitUntil': 'networkidle0'
    }
    
    response = requests.post(api_url, json=payload, headers=headers)
    
    with open(filename, 'wb') as f:
        f.write(response.content)
    
    print(f'Screenshot saved: {filename}')

# Capture baseline
capture_screenshot('https://example.com', 'baseline.png')

Visual Regression Testing Setup

Setting up visual regression testing involves capturing baseline screenshots, implementing comparison logic, and integrating with your testing workflow. Here's a complete example using Node.js and the Pixelmatch library for image comparison.

Install Required Dependencies

npm install axios pixelmatch pngjs

Complete Visual Regression Test

const axios = require('axios');
const fs = require('fs');
const PNG = require('pngjs').PNG;
const pixelmatch = require('pixelmatch');

async function captureScreenshot(url, width = 1920, height = 1080) {
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: url,
        commands: [],
        box: {
          x: 0,
          y: 0,
          width: width,
          height: height
        }
      }
    },
    {
      headers: {
        'x-api-key': process.env.CUSTOMJS_API_KEY,
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  return Buffer.from(response.data);
}

function compareScreenshots(baseline, current, diffOutput) {
  const img1 = PNG.sync.read(baseline);
  const img2 = PNG.sync.read(current);
  const { width, height } = img1;
  const diff = new PNG({ width, height });

  const numDiffPixels = pixelmatch(
    img1.data,
    img2.data,
    diff.data,
    width,
    height,
    { threshold: 0.1 }
  );

  fs.writeFileSync(diffOutput, PNG.sync.write(diff));
  
  const diffPercentage = (numDiffPixels / (width * height)) * 100;
  
  return {
    numDiffPixels,
    diffPercentage: diffPercentage.toFixed(2),
    passed: diffPercentage < 0.5 // 0.5% threshold
  };
}

async function runVisualRegressionTest(url) {
  console.log('Starting visual regression test...');
  
  // Capture current screenshot
  const currentScreenshot = await captureScreenshot(url);
  fs.writeFileSync('current.png', currentScreenshot);
  
  // Load baseline
  if (!fs.existsSync('baseline.png')) {
    fs.writeFileSync('baseline.png', currentScreenshot);
    console.log('Baseline created. Run test again to compare.');
    return;
  }
  
  const baseline = fs.readFileSync('baseline.png');
  
  // Compare screenshots
  const result = compareScreenshots(
    baseline,
    currentScreenshot,
    'diff.png'
  );
  
  console.log(`Diff pixels: ${result.numDiffPixels}`);
  console.log(`Diff percentage: ${result.diffPercentage}%`);
  console.log(`Test ${result.passed ? 'PASSED' : 'FAILED'}`);
  
  if (!result.passed) {
    throw new Error(`Visual regression detected: ${result.diffPercentage}% difference`);
  }
}

// Run test
runVisualRegressionTest('https://example.com')
  .catch(error => {
    console.error(error);
    process.exit(1);
  });

CI/CD Integration for Automated Testing

Integrating visual regression testing into your CI/CD pipeline ensures every code change is automatically tested for visual regressions. Here are examples for popular CI platforms.

GitHub Actions Integration

name: Visual Regression Tests

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  visual-regression:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Download baseline screenshots
        uses: actions/download-artifact@v3
        with:
          name: baseline-screenshots
          path: ./screenshots/baseline
        continue-on-error: true
      
      - name: Run visual regression tests
        env:
          CUSTOMJS_API_KEY: ${{ secrets.CUSTOMJS_API_KEY }}
        run: npm run test:visual
      
      - name: Upload diff screenshots
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: visual-diff-${{ github.sha }}
          path: ./screenshots/diff
      
      - name: Upload baseline screenshots
        if: github.ref == 'refs/heads/main'
        uses: actions/upload-artifact@v3
        with:
          name: baseline-screenshots
          path: ./screenshots/baseline

GitLab CI Integration

visual_regression_test:
  stage: test
  image: node:18
  
  variables:
    CUSTOMJS_API_KEY: $CUSTOMJS_API_KEY
  
  script:
    - npm ci
    - npm run test:visual
  
  artifacts:
    when: on_failure
    paths:
      - screenshots/diff/
    expire_in: 7 days
  
  only:
    - merge_requests
    - main

Jenkins Pipeline

pipeline {
    agent any
    
    environment {
        CUSTOMJS_API_KEY = credentials('customjs-api-key')
    }
    
    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }
        
        stage('Visual Regression Tests') {
            steps {
                sh 'npm run test:visual'
            }
        }
    }
    
    post {
        failure {
            archiveArtifacts artifacts: 'screenshots/diff/**/*.png'
        }
    }
}

Advanced Screenshot Testing Techniques

Element-Specific Screenshots

Instead of capturing full-page screenshots, you can capture specific elements to reduce noise and focus on critical UI components.

async function captureElementScreenshot(url, selector) {
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: url,
        commands: [
          { type: 'waitForSelector', value: selector },
          { type: 'screenshotElement', value: selector }
        ],
        box: {
          x: 0,
          y: 0,
          width: 1920,
          height: 1080
        }
      }
    },
    {
      headers: {
        'x-api-key': process.env.CUSTOMJS_API_KEY,
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  return response.data;
}

// Capture only the header
await captureElementScreenshot('https://example.com', 'header.main-header');

Screenshot Cropping

Use the box parameter to crop screenshots to specific regions. This is useful for capturing specific areas of a page without the full viewport.

async function captureRegion(url, region) {
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: url,
        commands: [],
        box: {
          x: region.x,
          y: region.y,
          width: region.width,
          height: region.height
        }
      }
    },
    {
      headers: {
        'x-api-key': process.env.CUSTOMJS_API_KEY,
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  return response.data;
}

// Capture header region only (top 200px)
await captureRegion('https://example.com', {
  x: 0,
  y: 0,
  width: 1920,
  height: 200
});

// Capture center content area
await captureRegion('https://example.com', {
  x: 200,
  y: 100,
  width: 1200,
  height: 800
});

Wait Strategies for Dynamic Content

For pages with dynamic content, animations, or lazy loading, use appropriate wait strategies to ensure consistent screenshots.

async function captureWithWaitStrategy(url, strategy) {
  const commands = [];
  
  // Different wait strategies
  switch(strategy) {
    case 'networkidle':
      commands.push({ type: 'waitForNetworkIdle' });
      break;
    case 'selector':
      commands.push({ type: 'waitForSelector', value: '.content-loaded' });
      break;
    case 'timeout':
      commands.push({ type: 'wait', value: 3000 }); // Wait 3 seconds
      break;
    case 'custom':
      commands.push({ type: 'waitForFunction', value: 'window.dataLoaded === true' });
      break;
  }
  
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: url,
        commands: commands,
        box: {
          x: 0,
          y: 0,
          width: 1920,
          height: 1080
        }
      }
    },
    {
      headers: {
        'x-api-key': process.env.CUSTOMJS_API_KEY,
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  return response.data;
}

Diff Detection Strategies

Different scenarios require different approaches to detecting and handling visual differences. Here are common strategies for various testing needs.

Pixel-Perfect Comparison

Best for static content where exact pixel matching is required.

const result = pixelmatch(
  baseline.data,
  current.data,
  diff.data,
  width,
  height,
  {
    threshold: 0.0, // Exact match required
    includeAA: false // Ignore anti-aliasing differences
  }
);

Threshold-Based Comparison

Allows minor differences, useful for dynamic content or font rendering variations.

const result = pixelmatch(
  baseline.data,
  current.data,
  diff.data,
  width,
  height,
  {
    threshold: 0.1, // Allow 10% color difference
    includeAA: true // Include anti-aliasing in comparison
  }
);

Ignore Regions

Mask dynamic areas like timestamps, ads, or user-specific content before comparison.

const sharp = require('sharp');

async function maskDynamicRegions(imagePath, regions) {
  const image = sharp(imagePath);
  const metadata = await image.metadata();
  
  // Create mask overlay
  const mask = Buffer.from(
    ``
  );
  
  return await image
    .composite([{ input: mask, blend: 'over' }])
    .toBuffer();
}

// Mask timestamp and ad regions
const maskedBaseline = await maskDynamicRegions('baseline.png', [
  { x: 10, y: 10, width: 200, height: 50 }, // Timestamp
  { x: 800, y: 100, width: 300, height: 250 } // Ad banner
]);

Integration with Testing Frameworks

Jest Integration

// visual-regression.test.js
const { captureScreenshot, compareScreenshots } = require('./screenshot-utils');

describe('Visual Regression Tests', () => {
  test('Homepage should match baseline', async () => {
    const current = await captureScreenshot('https://example.com');
    const baseline = fs.readFileSync('baselines/homepage.png');
    
    const result = compareScreenshots(baseline, current, 'diff/homepage.png');
    
    expect(result.diffPercentage).toBeLessThan(0.5);
  });
  
  test('Product page should match baseline', async () => {
    const current = await captureScreenshot('https://example.com/product');
    const baseline = fs.readFileSync('baselines/product.png');
    
    const result = compareScreenshots(baseline, current, 'diff/product.png');
    
    expect(result.diffPercentage).toBeLessThan(0.5);
  });
});

Playwright Integration

const { test, expect } = require('@playwright/test');
const axios = require('axios');

test('visual regression with CustomJS API', async ({ page }) => {
  await page.goto('https://example.com');
  
  // Trigger any interactions
  await page.click('button.show-modal');
  await page.waitForSelector('.modal');
  
  // Capture screenshot via API
  const response = await axios.post(
    'https://e.customjs.io/screenshot',
    {
      input: {
        url: page.url(),
        commands: [],
        box: {
          x: 0,
          y: 0,
          width: 1920,
          height: 1080
        }
      }
    },
    {
      headers: {
        'x-api-key': process.env.CUSTOMJS_API_KEY,
        'Content-Type': 'application/json'
      },
      responseType: 'arraybuffer'
    }
  );
  
  // Compare with baseline
  const baseline = fs.readFileSync('baseline.png');
  const result = compareScreenshots(baseline, response.data, 'diff.png');
  
  expect(result.diffPercentage).toBeLessThan(0.5);
});

Cypress Integration

// cypress/e2e/visual-regression.cy.js
describe('Visual Regression', () => {
  it('should match homepage baseline', () => {
    cy.visit('https://example.com');
    
    // Use custom command to capture via API
    cy.captureScreenshot('homepage').then((screenshot) => {
      cy.readFile('cypress/baselines/homepage.png', 'base64').then((baseline) => {
        cy.compareScreenshots(baseline, screenshot, 'homepage-diff.png')
          .then((result) => {
            expect(result.diffPercentage).to.be.lessThan(0.5);
          });
      });
    });
  });
});

Best Practices for Visual Regression Testing

1. Establish Clear Baselines

• Create baselines from a known-good state of your application
• Store baselines in version control or artifact storage
• Update baselines intentionally when UI changes are approved
• Document when and why baselines were updated

2. Handle Dynamic Content

• Mask or stub dynamic content (timestamps, user data, ads)
• Use consistent test data for reproducible results
• Disable animations and transitions during testing
• Mock external API calls to ensure consistent content

3. Optimize Test Performance

• Run tests in parallel to reduce execution time
• Test critical pages first, comprehensive pages later
• Use element-specific screenshots when possible
• Cache baselines to avoid repeated downloads

4. Set Appropriate Thresholds

• Start with strict thresholds (0.1%) and adjust based on false positives
• Use different thresholds for different page types
• Consider font rendering differences across environments
• Account for browser-specific rendering variations

5. Review and Approve Changes

• Always review diff images before approving changes
• Implement approval workflows for baseline updates
• Store diff images as CI artifacts for easy review
• Document visual changes in pull request descriptions

Integration with Make.com and n8n

For no-code visual regression testing workflows, integrate the CustomJS Screenshot API with Make.com or n8n.

Make.com Workflow

1. Add CustomJS module to your scenario
2. Configure screenshot parameters (URL, viewport, browser)
3. Store screenshots in Google Drive or Dropbox
4. Use image comparison service or custom webhook
5. Send notifications on visual differences detected

Learn more in our Make.com integration guide.

n8n Workflow

1. Install CustomJS n8n node
2. Create workflow triggered by schedule or webhook
3. Capture screenshots using CustomJS node
4. Compare with baseline using custom function node
5. Send Slack/email alerts on regression detection

Check out our n8n integration documentation.

Pricing and Limits

CustomJS offers transparent pricing for screenshot API usage:

• Free Tier: 600 screenshots per month (no credit card required)
• Starter Plan: $9/month for 3,000 screenshots
• Growth Plan: $29/month for 15,000 screenshots
• Enterprise: Custom pricing for high-volume testing needs

Each screenshot request counts as one API call, regardless of viewport size or browser. Compare this to self-hosting Puppeteer clusters or using alternatives like Percy ($299/month) or Chromatic ($149/month).

Start free with 600 screenshots per month

Frequently Asked Questions

1. What is visual regression testing?

Visual regression testing is an automated process that compares screenshots of your application before and after changes to detect unintended visual differences. It helps catch CSS bugs, layout issues, and rendering inconsistencies that functional tests might miss.

2. How accurate is screenshot comparison?

Screenshot comparison accuracy depends on your threshold settings and comparison algorithm. Pixel-perfect comparison (0% threshold) detects every pixel difference, while threshold-based comparison (0.1-1%) allows minor variations from font rendering or anti-aliasing. Most teams use 0.1-0.5% thresholds for production testing.

3. Can I test across different browsers?

Yes! The CustomJS Screenshot API supports Chrome (Chromium), Firefox, and WebKit (Safari) engines. You can capture screenshots from all three browsers with a single API and compare them to detect cross-browser rendering differences.

4. How do I handle dynamic content in visual tests?

Mask dynamic regions (timestamps, ads, user data) before comparison, use consistent test data, disable animations, and mock external APIs. You can also use the screenshot API's wait strategies to ensure content is fully loaded before capture.

5. What's the difference between screenshot API and Puppeteer?

Puppeteer requires managing browser instances, handling updates, and maintaining infrastructure. A screenshot API provides managed browsers, cross-browser support, consistent rendering, and easy scalability without infrastructure overhead. It's ideal for CI/CD integration and testing workflows.

6. How do I integrate visual regression tests into CI/CD?

Add a test step to your CI pipeline that captures screenshots via API, compares them with baselines, and fails the build if differences exceed your threshold. Store baselines as artifacts and upload diff images on failure for review. See our GitHub Actions, GitLab CI, and Jenkins examples above.

7. What screenshot dimensions should I use?

Use standard desktop dimensions like 1920x1080 or 1280x720 for full-page screenshots. For cropped regions, adjust the box parameters (x, y, width, height) to capture specific areas like headers, footers, or content sections. The box parameter crops the screenshot, it doesn't change the browser viewport.

8. How do I update baselines when UI changes are intentional?

Capture new screenshots after verifying the changes are correct, review the diffs to ensure they match expectations, replace old baselines with new screenshots, commit updated baselines to version control, and document the change in your commit message or pull request.

Conclusion

Visual regression testing with a screenshot API provides automated, reliable detection of visual bugs in your web applications. By integrating screenshot comparison into your CI/CD pipeline, you can catch CSS regressions, layout issues, and rendering inconsistencies before they reach production.

The CustomJS Screenshot API simplifies visual testing by providing managed Chromium browsers, screenshot cropping capabilities, and easy integration with existing testing frameworks. With 600 free screenshots per month and native Make.com and n8n integration, it's an accessible solution for teams of all sizes.

Whether you're testing a small website or a large-scale application, automated visual regression testing ensures consistent user experiences and reduces the risk of visual bugs escaping to production.

Get started with visual regression testing today