Defects Report automations (Legacy)

Attention

This document covers TestOps Legacy version only.

What is a Defects Report automation?

Setting up a Defect report automation is the option to automate the linking of Jira defects to TestOps. It's useful for situations when there are multiple test results with the same issue that can be linked to a particular defect.

This makes the process of tracing defects easier. After executing a test run in TestOps, link your test results to an issue in Jira manually by using Report Defect. See: Link test runs to Jira defects in Katalon TestOps.

Then, create automation rules as shown below.

Create automation rules

• You have configured the Jira integration for Katalon TestOps. To learn how to set it up, see: Enable TestOps - Jira integration for test management.

Test results with similar failure conditions can be linked to a defect automatically by creating an automation through Report Defect. To do so, follow these steps:

1. Log in to Katalon TestOps and go to your project.
2. Go to Reports > Test Runs > Test Runs.
3. Select a failed test run by clicking on its ID. Then click on the Test Results tab.
   
   1. The Test Results page appears as below.
      
4. Click on the Report Defect button at the rightmost column. It has the bug icon.
5. Link or create your issue. To learn how, see: Link test runs to Jira defects in Katalon TestOps.
6. Click on Create Automation Rule to automatically link this defect to future test results.
   
7. Input the parameters for your automation rule. You can define the patterns using either exact text or regular expression syntax. Learn more about POSIX regular expressions here.
   Attention:

   • Only upcoming test results will be automatically linked to this defect when an error message or stack trace matches your pattern.

   • Failed tests with matching error or stack trace values will not automatically link to resolved defects. You need to manually link them.

   
   1. To view the full error message or stack trace, click the Expand button at the upper right corner.
   2. A pop-up displays more details about the error message.
8. You can also click View matched results to see if there are other test results made within the last 30 days that equal your defined pattern. This is disabled by default and will only be enabled once the valid values are entered for both patterns.
   
   1. A pop-up displays a list of matching results.
9. Click Save.
   1. You can also re-edit your automation by clicking on the Edit Automation Rule button.

A pop-up notification will confirm that you have successfully saved the settings for an automation rule. It applies once you have created and linked your test results to a defect.

To view all your automation rules or delete them, see: View and delete automation rules.

You can automatically get notified in Slack when this action is performed by setting up an integration. See: Defects Report Slack notifications.

View and delete automation rules

• You have a valid automation rule. To create one, see: Create automation rules.

Automation rules linked to a particular defect can be viewed or deleted. Here's how to do it:

1. Log in to Katalon TestOps and go to your project.
2. Click on Reports > Defects.
   1. In the displayed Defects page, apply any search filters you may need in order to view all defects.
3. Scroll down to your defects list and click the arrow > icon to open a side panel. This action displays two tabs:
   

   • Overview: Shows all the test cases linked to the defect.

   • Automation Rule: Shows all the Automation Rules you've made linked to the defect.

4. Click Automation Rule to view your automation rules list.

Delete automation rules

5. Click on the Trash icon to the right of the Automation Rule you'd like to delete.
   
6. A pop-up confirmation will ask you to confirm the deletion. Click Confirm.
   

You have successfully viewed or deleted your Automation Rule.

Troubleshoot automation rules

The Save button will remain greyed out if there are unacceptable parameters within your automation rule.

There are unacceptable parameters within your automation rule. All form fields must pass validation before you can click Save. Refer to the steps below for the following guidance in entering value for your automation rule.

1. If your Rule Name is blank or duplicated, enter a unique name with a maximum length of 255 characters.
2. If your Error Message or Stack Trace patterns are blank, enter a definition for your patterns using either the exact text or regular expression syntax. At least one of them must have a value with a minimum of 2 characters and a maximum of 2048.
3. If your Regex pattern for Error Message or Stack Trace is invalid, double-check your entry and refer to this document to learn more about it: Learn more about POSIX regular expressions here.