Skip to main content

Use TestOps Visual Testing

You can compare images captured during test executions with TestOps Visual Testing.

Important:

Set up visual testing

Follow these steps:

  1. Enable screenshot capture in Katalon Studio. See Capture Screenshots.

    Note:
  2. Run a Test Suite in Katalon Studio.

    Note:
    • If you have enabled Katalon Studio Integration, Katalon Studio automatically uploads Test Results to Katalon TestOps.

  3. Sign in to Katalon TestOps and go to your Project.

  4. Select Visual Testing on the navigation bar.

  5. Click on the ID of a Test Run.

  6. Select the Checkpoints tab.

    test run #1 checkpoints page

    You can see the screenshots captured during a test execution.

  7. Select one of the screenshots to see the details.

    Note:
    • If you run a Test Suite for the first time, there is no baseline image.

    • If you have existing visual data, it is stored in the System-generated baseline collection folder.

Configure a baseline image

You can set up a baseline image to compare it with the screenshot of the next Test Run.

Follow these steps:

  1. Select a screenshot to see the details.

    visual testing image passed status

  2. Select Mark as Passed at the top right corner of a screenshot, then close the screenshot.

    An Unsaved label now appears on the Checkpoints page.

  3. Go to the Results page.

    unsaved label save to baseline button

  4. Click Save to baseline.

Note:
  • If you run a Test Suite for the first time and the Test Suite has passed, the screenshots are automatically marked as Passed and there is no Save to baseline button. You can open a screenshot and click Failed, then switch back to Passed so that the Save to baseline button appears.

Unresolved images

If you run the Test Suite again and the new screenshots of this test execution are different from the baseline images, the status of this Test Run is then marked as Unresolved.

To resolve the issues, follow these steps:

  1. Click on the ID of the unresolved Test Run.

  2. Select and open a screenshot.

    mismatch image comparison method box

    You see a drop-down list in the Comparison Method section. The three options include: Pixel, Layout, Text Content.

    Note:
  3. Select one of the three options to start comparing the new screenshot (on the right) with the baseline image (on the left).

    Note:
    • Comparison Method is enabled only if the status of the screenshot is mismatched.
    • If you select Pixel, check the Diffs between two images (the differences are highlighted in red). See: Pixel-based comparison.
    • If you select Layout, check the Identical and Distinguishable and Missing/New boxes to see the differences highlighted in green/red/pink color respectively. See: Layout-based comparison.
    • If you select Text Content, check the Identical and Shifted and Missing/New boxes to see the differences highlighted in green/blue/pink color respectively. See: Content-based comparison.
  4. Click Mark as Passed or Marked as Failed to resolve.

    Note:
    • The Unsaved label appears whenever you change the status (Passed or Failed) of a screenshot. If you click Save to baseline, the new Passed screenshots replace the older baseline images. Katalon TestOps then compares between the new baseline images and screenshots of the next Test Runs.

Baseline collection

A baseline collection is a collection of all baseline images in one and/or several related visual test run(s).

In TestOps, the visual baseline collection stores every version of all baseline images that you have saved each time. Each baseline collection has an ID (Baseline Collection ID). To access a baseline collection, follow these steps:

  1. Sign in to Katalon TestOps and go to your project.
  2. Go to Visual Testing > Visual Baseline Collections.
    You will see all baseline collection IDs and their details (latest version, number of the baseline images in each baseline collection, and the test schedule that links to them).
    visual baseline collections page

  3. Click on the Baseline Collection ID you want to check.
    You then can see the latest version of all baseline images in this baseline collection.
    Layer 1Click History to see the older versions of this baseline collection. Click on the Pencil icon to edit the name of the baseline collection. You can also edit or delete each baseline image of this baseline collection.

Ignored zones

You can configure ignored zones in the baseline image. Visual testing then ignores these zones during image comparison.

Specifying ignored zones is especially useful when testing a website with some dynamic and changing content (such as pop-up ads on specific areas). TestOps Visual Testing can ignore such content while comparing captured images with the baseline one. To configure the ignored zone, follow these steps:
  1. Sign in to Katalon TestOps and go to your project.
  2. Go to Visual Testing > Visual Baseline Collections.
  3. Select the Baseline Collection ID you want to check.
    The list of baseline images appears.

    Layer 1

  4. Click on the Edit image icon of the baseline image (e.g., image10) you want to configure.
    The page displays as below.
    configure ignored zone in image10

  5. Click on the Configure Ignored Zones button.
    The button is then highlighted in blue, which indicates the edit mode.
    configure ignored zones button highlighted

  6. Draw (in rectangles) to specify the area you want to ignore.
    The ignored zones are highlighted in purple. You can draw up to 20 ignored zones (i.e. 20 rectangles).
  7. Click Save.
    You have temporarily saved the ignored zones on this baseline image.
    save baseline image10

    Note:
    • You have not permanently saved the configuration of ignored zones at this stage.
  8. Click Save Changes on the top right corner to permanently save the ignored zones you have configured.
    save changes button for baseline collection

The baseline image has now had ignored zones. Visual testing then ignores these zones when comparing captured images with this baseline image.

When comparing the checkpoint image with the baseline image, you can show or hide the ignored zones by ticking/un-ticking the Ignored Zones checkbox.

Layer 1