Accessibility test automation in Katalon Studio

Note:

This integration provides assistance for certain user-specific use cases, but it is not officially supported. Compatibility with all user requirements is not guaranteed.

Axe is a fast and lightweight accessibility testing tool for websites. The tool evaluates if your website follows the Web Content Accessibility Guidelines (WCAG) and other guidelines supported by axe-core library. To learn more about Axe, refer to the documentation on deque: Axe API Documentation.

With the axe-core library integrated in Katalon Studio, you can automate accessibility testing alongside your regular testing.

Add the axe-core library

You will use the axe-core library .jar file to import required packages in the code snippet.

To integrate the axe-core library with Katalon TestOps, do as follow:

1. Download the axe-core library.
   You can download the axe-core library .jar file from the Maven Repository website: AXe Selenium Integration.

   Note:

   • We recommend downloading the 4.4.1 version for this guide. The latest version of Axe lacks a few required packages for the code snippet.
2. Go to Project > Settings > Library Management, then Add the axe-core library .jar file to the External Libraries.
   

Run your accessibility test

After you added the axe-core library .jar file, you can run your accessibility test with a custom keyword.

Follow these steps:

1. Create a custom keyword. You can follow this guide here: Introduction to custom keywords in Katalon Studio.
2. Add the script below to your keyword file.

   import com.kms.katalon.core.util.KeywordUtil
   import java.text.SimpleDateFormat
   
   import com.kms.katalon.core.configuration.RunConfiguration 
   import com.kms.katalon.core.webui.driver.DriverFactory
   
   import com.deque.html.axecore.selenium.AxeBuilder;
   import com.deque.html.axecore.selenium.AxeReporter;
   import com.deque.html.axecore.selenium.ResultType;
   import com.deque.html.axecore.results.Results;
   import com.deque.html.axecore.results.Rule;
   import static com.deque.html.axecore.selenium.AxeReporter.getReadableAxeResults;
   
   
   	@Keyword
   	def checkAccessibility() {
   		Results results = new AxeBuilder().analyze(DriverFactory.getWebDriver())
   		List<Rule> violations = results.getViolations()
   		if(violations.size() == 0){
   			KeywordUtil.logInfo("No Violation Found")
   		}
   		String AxeReportPath = RunConfiguration.getReportFolder()+ File.separator
   		String timeStamp = new SimpleDateFormat("yyyy_MM_dd_HH_mm_ss").format(new java.util.Date())
   		String AxeViolationReportPath=AxeReportPath + "AccessibilityViolations_" + timeStamp
   		AxeReporter.writeResultsToJsonFile(AxeViolationReportPath,results)
   		KeywordUtil.logInfo("Violation Report Path"+ AxeViolationReportPath)
   
   		if(getReadableAxeResults(ResultType.Violations.getKey(),DriverFactory.getWebDriver(),violations) ){
   			AxeReporter.writeResultsToTextFile(AxeViolationReportPath,
   					AxeReporter.getAxeResultString())
   		}
   	}
   }

3. Add the keyword as a test step in your test case. You should add the keyword after you have navigated to your desired page on the website.

   

   For a more detailed instruction, see this guide here: Use custom keywords in the manual view and script view.
4. Add your test case to a test suite.
5. Execute your test.
   You can execute on any browser that Katalon Studio supports (Chrome, Mozilla Firefox, Edge Chromium, Safari). You can use this keyword for different pages on the website depends on your navigation flow.

After you execute the test, the accessibility test reports are generated in TXT and JSON format in Katalon Studio report folder.

View the accessibility test reports

You can view the accessibility test reports in:

• Katalon Studio: Right click on the Reports folder > Open Containing Folder to find your desired report.
• Katalon TestOps: If you have TestOps integrated in Katalon Studio, the reports are automatically uploaded to TestOps.

  Go to Reports > Test Runs. In the Test Runs list, click on the test run ID you want to view the uploaded files, then switch to the Files tab. Here you can download the TXT or JSON report file.

• The report in TXT file contains the violation information of the accessibility test.

• The report in JSON file contains the components information of the accessibility test.
  • passes (array): These results indicate what elements passed in the rules.
  • violations (array): These results indicate what elements failed in the rules.
  • incomplete (array): This contains results that were aborted and require further testing. This can happen due to technical restrictions to what the rule can test, or a JavaScript error occurred.
  • inapplicable (array): This indicates rules that did not run because there was no matching content found on the page.