Skip to main content

Defects Report automations

What is a the Defects Report automation?

Setting up a Defect report automation is the option to automate the linking of Jira defects to TestOps. It's useful for situations when there are multiple test results with the same issue that can be linked to a particular defect.

This makes the process of tracing defects easier. After executing a test run in TestOps, link your test results to an issue in Jira manually by using Report Defect. See: Link test runs to Jira defects in Katalon TestOps.

Then, create automation rules as shown below.

Create automation rules

Test results with similar failure conditions can be linked to a defect automatically by creating an automation through Report Defect. To do so, follow these steps:
  1. Log in to Katalon TestOps and go to your project.
  2. Go to Reports > Test Runs > Test Runs.
  3. Select a failed test run by clicking on its ID. Then click on the Test Results tab.
    A failed test run report in the test run page.
    1. The Test Results page appears as below.
      A list of a test run's test results.
  4. Click on the Report Defect button at the rightmost column. It has the bug icon.The report defect bug icon in Katalon TestOps
  5. Link or create your issue. To learn how, see: Link test runs to Jira defects in Katalon TestOps.
  6. Click on Create Automation Rule to automatically link this defect to future test results.
    The Report Defect dialog in Katalon TestOps
  7. Input the parameters for your automation rule. You can define the patterns using either exact text or regular expression syntax. Learn more about POSIX regular expressions here.
    • Only upcoming test results will be automatically linked to this defect when an error message or stack trace matches your pattern.

    • Failed tests with matching error or stack trace values will not automatically link to resolved defects. You need to manually link them.

    The Automation Rule settings in the Report Defect dialog
    1. To view the full error message or stack trace, click the Expand button at the upper right corner. The expand button for error message and stack trace.
    2. A pop-up displays more details about the error message.Error message details when creating automation rules in Katalon TestOps.
  8. You can also click View matched results to see if there are other test results made within the last 30 days that equal your defined pattern. This is disabled by default and will only be enabled once the valid values are entered for both patterns.
    View matched results when creating an automation rule in Katalon TestOps.
    1. A pop-up displays a list of matching results. List of matched results when an automation rule is created in Katalon TestOps.
  9. Click Save.
    1. You can also re-edit your automation by clicking on the Edit Automation Rule button. The Edit Automation Rule button in Report Defect.
A pop-up notification will confirm that you have successfully saved the settings for an automation rule. It applies once you have created and linked your test results to a defect.

To view all your automation rules or delete them, see: View and delete automation rules.

You can automatically get notified in Slack when this action is performed by setting up an integration. See: Defects Report Slack notifications.

View and delete automation rules

Automation rules linked to a particular defect can be viewed or deleted. Here's how to do it:
  1. Log in to Katalon TestOps and go to your project.
  2. Click on Reports > Defects.
    1. In the displayed Defects page, apply any search filters you may need in order to view all defects.The Defects page within the Reports section of Katalon TestOps
  3. Scroll down to your defects list and click the arrow > icon to open a side panel. This action displays two tabs:
    The side panel within the Defects page.
    • Overview: Shows all the test cases linked to the defect.

    • Automation Rule: Shows all the Automation Rules you've made linked to the defect.

  4. Click Automation Rule to view your automation rules list.
Delete automation rules
  1. Click on the Trash icon to the right of the Automation Rule you'd like to delete.
    The trash icon for deleting an automation rule in Katalon TestOps.
  2. A pop-up confirmation will ask you to confirm the deletion. Click Confirm.
    The delete automation rule confirmation box in Katalon TestOps.
You have successfully viewed or deleted your Automation Rule.

Notification confirming the deletion of an Automation Rule in Katalon TestOps.

Troubleshoot automation rules

The Save button will remain greyed out if there are unacceptable parameters within your automation rule.

There are unacceptable parameters within your automation rule. All form fields must pass validation before you can click Save. Refer to the steps below for the following guidance in entering value for your automation rule.

  1. If your Rule Name is blank or duplicated, enter a unique name with a maximum length of 255 characters.
  2. If your Error Message or Stack Trace patterns are blank, enter a definition for your patterns using either the exact text or regular expression syntax. At least one of them must have a value with a minimum of 2 characters and a maximum of 2048.
  3. If your Regex pattern for Error Message or Stack Trace is invalid, double-check your entry and refer to this document to learn more about it: Learn more about POSIX regular expressions here.