Configure TestRail integration in Katalon Studio

The TestRail Integration plugin establishes the connection between Katalon Studio and TestRail to deliver the following advanced capabilities:

• In TestRail, you can view test results of test suites executed in Katalon Studio.
• In Katalon Studio, you can query test cases associated with test runs of TestRail in the dynamic test suite.

Notes

• From version 8.5.5, your TestRail password is encrypted. If you open your project in Katalon Studio version before 8.5.5, you might need to re-en ter your TestRail password.
• The TestRail Integration plugin only supports integrating Katalon Studio with TestRail Cloud.

This tutorial shows you how to configure TestRail integration.

Requirements

• An active Katalon Studio Enterprise license.
• The TestRail Integration plugin installed. You can find the latest version of the TestRail plugin (version 1.1.4) here: TestRail Integration plugin.

Enable TestRail integration in Katalon Studio

To enable the integration of Katalon Studio with TestRail, you need to configure both your TestRail site and Katalon Studio, do as follows:

1. Enable the TestRail API.

   Log in to your account, go to Administration > Site Settings > API, and check the Enable API option. Then click Save Settings.

   

2. Enable the TestRail Integration plugin.

   1. In Katalon Studio, go to the main menu, select Project > Settings > Plugins > TestRail and check the Enable TestRail option.

   2. Enter the credentials required for Authentication:

      • URL: Your TestRail instance. For example, https://<example>.testrail.io.
      • Username: Your TestRail username.
      • Password: Your TestRail password.
      • Project: your TestRail project ID (an integer). To get the project ID, open your TestRail project in the browser and view the ID at the end of the URL.

      For example, for the project with the URL https://company.testrail.io/index.php?/projects/overview/1, the project ID here is 1.

   3. Click Test Connection to verify your TestRail account.

      

   4. Click Appy and Close.

You have successfully enabled TestRail integration in Katalon Studio.

Map test cases between Katalon Studio and TestRail

To map a test case between Katalon Studio and TestRail, you need to get the TestRail test case ID. Do as follows:

1. To retrieve the TestRail test case ID, open your project in TestRail, then go to the Test Cases tab. Here you can see the list of test cases and their IDs.

2. In Katalon Studio, open the test case you want to map, switch to the Integration tab, and specify the respective test case ID in TestRail (only the integer part).

3. Save the Katalon Studio test cases.

You have successfully map test cases between Katalon Studio and TestRail.

For more information, see Upload test execution results from Katalon Studio to TestRail.

Map TestRail custom required field to Katalon Studio

Requirements

• Make sure you are using Katalon Studio Enterprise version 10.2.3 and later.
• Install the latest TestRail Integration plugin (version 1.1.4). Download it here: TestRail Integration plugin.
• Ensure that you have enabled the API option on the TestRail server. See Step 1 of Enable TestRail integration in Katalon Studio.

If you configure a required custom field in TestRail and use Katalon Studio to upload test results, you may encounter issues when executing a test suite and pushing results to TestRail. To prevent this, make sure the custom field is correctly mapped in Katalon Studio Project Settings under the TestRail configuration section.

1. In TestRail, create your custom field. For more information about creating custom fields in TestRail, see the following software documentation: Configuring custom fields.

2. Verify if the custom field is showing in Customizations > Result Fields list under Custom type.

   In the example below, you can see the custom required field Test Execution Unit.

   

3. In Katalon Studio, go to Project > Settings > Plugins > TestRail.

4. Use the Custom fields mapping section to manually enter the custom TestRail field in the first column to your Katalon predefined variable in the second column. Use Add, Delete and Clear buttons to edit or update your mapping variables

   

   Refer to the following predefined variables when mapping to TestRail fields:

Name of field	Value
testSuite.description	Test suite description.
hostName	Host name.
os	Operating system.
browser	Browser's name and version.
deviceId	ID of the executed device.
deviceName	Name of the executed device.
suiteName	Name of the test suite.
executionProfile	Execution profile used during the test.
startTime	Start time of the test execution.
duration	Duration of the test execution.
totalPassed	Total passed test cases.
totalFailed	Total failed test cases.
totalError	Total error test cases.
totalIncomplete	Total incomplete test cases.
totalSkipped	Total skipped test cases.

5. Click Apply or Apply and Close to save your changes.

You have successfully mapped your TestRail custom required field to Katalon Studio.

Click Restore to Defaults to reset your TestRail configuration to default settings.

Query test cases linked to TestRail in a dynamic test suite

• You have enabled the TestRail integration with Katalon Studio. See: Configure TestRail integration in Katalon Studio.

When the TestRail Integration plugin is enabled, the Query Provider in the dynamic test suite is updated with the TestRail query syntax standard. This allows you to query test cases associated with the TestRail test runs in the dynamic test suite.

To learn more about query syntax in the dynamic test suite, you can refer to this guide: Dynamic Test Suite.

Follow these steps:

1. Open a dynamic test suite in Katalon Studio.
2. In the Query text box, enter the ID of a TestRail test run, then click on the Preview button.

   We want to query the test cases associated with the test run R8, we enter the test run ID R8 in the Query text box.

   

The test cases associated with the TestRail test run are displayed. You can now execute the dynamic test suite linked to TestRail and verify the results in TestRail.