Visual Testing overview
This document covers TestOps Legacy version only.
What is Visual Testing?
Visual testing is an automated process that verifies your application's user interface (UI) appears correctly to end users across web and mobile platforms. This testing method ensures your software displays exactly as you designed it, maintaining visual consistency and functionality.
AI-Powered Image Comparison
When you implement visual testing with artificial intelligence, you gain access to advanced image comparison capabilities. The AI engine compares your test outputs against baseline images, automatically detecting discrepancies that might indicate visual bugs or regressions.
Automated Detection Capabilities
Visual testing eliminates the limitations of manual image review by providing automated comparison through AI-powered detection methods:
- Pixel-by-pixel detection identifies precise changes in individual pixels, catching even subtle color variations or minor shifts in visual elements.
- Text-change detection monitors typography modifications, including font changes, text positioning, or content alterations that could affect user experience.
- Layout-change detection tracks structural modifications in your interface, ensuring elements maintain their intended positioning and spacing.
Benefits of Automated Visual Testing
You can significantly reduce human error in your testing process because visual testing captures minute differences that you might overlook during manual inspection. This automated approach ensures comprehensive coverage of visual elements while maintaining consistent quality standards across your application's interface. The precision of AI-powered visual testing allows you to maintain visual integrity throughout your development cycle, catching issues before they reach production environments.
A baseline image is a reference image initially configured to compare with other image output from visual testing.
Insights
Visual defects represent a critical blind spot in traditional functional testing. While functional tests excel at validating software behavior and logic, they cannot detect visual regressions such as layout shifts, color changes, font inconsistencies, or misaligned elements.
Consider a scenario where automated functional tests pass successfully, yet users encounter a broken layout, incorrect styling, or displaced UI components. Traditional testing approaches would miss these visual issues entirely, potentially allowing them to reach production and impact user experience.
The challenge becomes even more complex in today's multi-platform environment. With users accessing applications across diverse combinations of operating systems, browsers, and devices, ensuring visual consistency everywhere is paramount. A website may render perfectly in Chrome on Windows but display incorrectly in Safari on macOS. Manual cross-platform visual validation is not only time-intensive but also prone to human oversight, especially when dealing with subtle changes in complex interfaces.
Visual testing addresses these challenges by automatically detecting visual regressions faster and more reliably than manual inspection. By catching visual bugs before they reach production, visual testing enhances development productivity, builds product confidence, and delivers superior user experiences.
TestOps Visual Testing
Katalon TestOps offers visual testing with three image comparison methods: pixel-based, layout-based, and content-based comparison. Since AI powers the layout-based and content-based methods, TestOps Visual Testing could become an integral part of your visual testing infrastructure.
Pixel-based comparison
This method compares the pixel resolution of two images to figure out the pixel-by-pixel differences between them.
The advantage of this method is that it is a popular function and easy to understand and implement. This method identifies pixel differences and picks up minor changes that could be trivial or ignorable to the human eyes.
Layout-based comparison
This method identifies and maps the similar zones between the baseline and checkpoint images, highlighting the layout differences between the two images.
In Katalon TestOps, the zones are categorized into 3 types:
The AI engine will identify the zones on the images.
| Label | Color | Description |
|---|---|---|
| Identical | Green border | UI zones identically rendered in both images. |
| Distinguishable | Red border | UI zones that look identical for the most parts but have some recognizable distinctions. |
| Missing/New | Pink border | UI zones that exist in one image but not in the other. |
This method is helpful to review changes identified by the pixel-based comparison, and it spots the changes in layout, color, font, and element location.
Content-based comparison
This method identifies the text content differences between two images.
The AI engine will identify the zones on the images.
In Katalon TestOps, the zones are categorized into 3 types:
| Label | Color | Description |
|---|---|---|
| Identical | Green border | Shows text-look-like zones identically rendered in both images |
| Distinguishable | Red border | Shows text-look-like zones with identical but shifted positions |
| Missing/New | Pink border | Shows text-look-like zones in one image but not in the other. |
This method is useful when there is a lot of text content on the pages. It helps users review and prioritize which text changes are critical and need immediate attention.