Schedule Test Runs in TestOps

Requirements

• You have created a Local Test Environment with Agent or you have already created a different Test Environment (Docker, Kubernetes, etc.).

• You have created a Git script repository and linked it to your project on TestOps. For more information, see Upload test scripts from the Git repository to Katalon TestOps.

Schedule Test Runs

Follow these steps:

1. Go to your Project and select the Test Execution tab.

   The Test Run Calendar page appears as below.

   
   
   

2. Click on the Schedule Test Run button.

   The Schedule Test Run dialog appears as below.

   
   
   
   
   

3. Fill in the information.

   • In the General Information section, create a name for the schedule.

   • In the What to run section, fill in the following information:

     • Script Repository: select your repository in the dropdown list.

       Note:

       • Katalon TestOps currently supports test scripts from Katalon Studio and Git Repositories (Azure Repos, Bitbucket, GitHub and GitLab). See: Upload Test Scripts from a Git Repository.

     • Type: choose between Test Suite, Test Suite Collection, Katalon Command and Generic Command to run your tests.

       Note:

       • You choose Test Suite if you want to run test suites created in TestOps or from Katalon Studio (KS).

         • Make sure you download a new local agent to execute test suites in the local environment. See: Create a Local Test Environment with an Agent.

         • Make sure Kobiton integration is disabled because you can only execute test suites on desktop applications (browsers). You can toggle Kobiton off in advanced settings. See: Advanced settings.

         • Once you have chosen Test Suite, the Run with section appears. The default browser is Chrome. You can select a different browser in the Run with section.

       • You choose Test Suite Collection if you want to select the Test Suite Collections directly from KS. Katalon TestOps will automatically fetch your Test Suite Collections.

       • The Visual Testing option is enabled.

         • You can specify an existing baseline collection in the Visual Testing section to associate it with the scheduled test runs.

         • If you do not specify a baseline collection, TestOps automatically generates a new baseline collection when the test run schedule is executed.

         • For more information on visual testing, see: TestOps Visual Testing and Use TestOps Visual Testing.

       • You choose Generic Command if you want to execute tests with other frameworks outside KS (e.g., Pytest).
       • You choose Katalon Command if you want to execute tests with KS. The Katalon commands can be generated from KS (in the Command Generator dialog). See: Command Syntax.

   • In the Where to run section, fill in the following information:

     • Test Environment Type: choose among Local Test Environment, TestCloud Test Environment, Kubernetes Test Environment or CircleCI Test Environment.

       Note:

       • If you are using Katalon Runtime Engine (KRE), check the Katalon Runtime Engine Version box and select the version you are using from the dropdown list.

       • If you choose Test Cloud Environment to run test suites, the browser options in the TestCloud section correspond to the browser you have selected in the Run with section (when selecting Test Suite as Type in the What to run section). For example, if the selected browser is Chrome, the available options in the TestCloud section are: Window - Chrome and Linux - Chrome. See also: Integrate TestCloud with TestOps.

     • Test Environments: select the Agents you want to use for test executions.

   • In the When to run section, you have the following options:

     • If you want to run tests periodically, select Trigger Automated Test, turn the Repeat toggle on, set the time period and the interval you want to run the tests (e.g., run tests every 2 days from 09/21/2021 10:49 to 10/20/2021 11:00), then click Schedule. You have created a trigger to run tests automatically.

       
       
       

     • If you want to run tests immediately, select Trigger Automated Test, turn the Repeat toggle off, then click Run. You have run the tests manually.

       
       
       

     • If you want to save what you have configured so far without running tests, select Save Configurations, then click Save. You then can come back another time to run tests manually. See: Execute Test Runs manually.

       
       
       

Advanced settings

You can optimize your configurations in the Advanced settings section of the Schedule Test Run dialog.

When you open the Schedule Test Run dialog, at the bottom left corner, click Advanced settings.

1. Open the Schedule Test Run dialog.

2. At the bottom of the dialog, click Advanced settings.

   The dialog now appears as below.

   
   
   

   • Execution Mode: choose between Sequential (run one test after another) or Parallel (run tests at the same time). See: Run multiple Test Suites in parallel with Agents.

     Note:

     • You can only run test suites in a sequential mode. If you choose Test Suite in the What to run section, there is no Parallel option.

   • Timeout in Minutes: define the time after which test execution is cancelled.

   • Kobiton integration: switch the Kobiton toggle on to enable the integration, then enter your Kobiton Device ID to run tests on that device. See: How to configure Kobiton integration with Katalon TestOps.

   • Release Version: select the release version you want to link your test runs to. See: Manage Test Runs by Release.

The dialog displays with the following settings:

• Visual Testing: select a Baseline Collection ID or generate a new Baseline Collection to compare with the captured images.

• Katalon Runtime Engine Version: choose a Katalon Runtime Engine version to run your test.

• Execution Mode: choose between Sequential (run one test after another) or Parallel (run tests at the same time). See: Run multiple Test Suites in parallel with Agents.

  Note:

  • You can only run test suites in a sequential mode. If you choose Test Suite, there is no Parallel option.
• Integration:
  • For Xray Test Plan and Xray Push Mechanism: see Link a Katalon test run with an Xray test plan.
  • For Kobiton: switch the Kobiton toggle on to enable the integration, then enter your Kobiton Device ID to run tests on that device. See: How to configure Kobiton integration with Katalon TestOps.

• Custom Field & Tag: assign custom fields and tags to your test run schedule. See: Custom Fields and Tags.

• Release Version: select the release version you want to link your test runs to. See: Manage Test Runs by Release.

• Timeout in Minutes: define the time after which test execution is canceled.

After creating the schedule, Katalon TestOps automatically directs you to the Test Run List page.

The Test Run List page shows you a collection of Test Runs with the same configurations (Test Environment, Script Repository).

View Test Run Types in Katalon Studio

Note:

• The Test Run List in TestOps is equivalent to the Test Run Types in Studio.

Important:

• Katalon Studio version 7.9.1 onwards.
• Katalon Studio Integration.

Follow these steps:

1. Open Katalon Studio and your Project.

2. In the Tests Explorer sidebar, go to TestOps > Test Run Types.

   You then can view a summary of Test Run Types that have been configured in TestOps.

   
   
   

3. Click on any of the Test Run names (e.g., a Verify Match exact name(git)). This opens the Test Runs Summary page in Katalon TestOps.

   
   
   

See also: Execute Test Runs manually.