Skip to main content

Azure DevOps Extension

Katalon TestOps Continous Integration (CI) is an easier way to execute Katalon Studio tests remotely or schedule remote Katalon Studio execution. See Test Planning Overview for detailed information.

This step-by-step guide shows you how to install the Katalon for Azure DevOps extension and run Katalon projects with Azure DevOps for Web UI testing.

Important:
  • An active Katalon Runtime Engine license.

Install the Azure DevOps Extension

To run Katalon projects with Azure DevOps, you first need to install our Katalon for Azure DevOps extension from the Visual Studio Market Place. This extension helps download, deploy, and execute Katalon Studio on Azure DevOps automatically.

To download and install the Azure DevOps extension, perform the following steps:

  1. Navigate to Visual Studio.

    Click on Get it Free.

    add task Get it Free

  2. From the Select an Azure DevOps Organization dropdown, select your organization. Then, click Install.

    add task Select an Azure DevOps Organization

Note:

If you're using Azure DevOps Server, clickDownload and follow instructions as given in the image below.

add task Install

Sample Projects

For Linux

The Azure DevOps extension was tested on Ubuntu 16.0.4 and Ubuntu 18.04. For continuous integration using this version, we recommend our sample scripts on Github: Azure DevOps sample script.

For Windows

To run UI tests on Azure Pipelines, you might need to adjust the screen resolution (See Microsoft documentation on Setting screen resolution).

You can also install the Screen Resolution Utility extension from the Visual Studio Marketplace.

See our Github examples on how to run a Katalon Studio test:

Note:

The difference between the two examples given below, of "For Windows" and "For Windows resolution", is that VM resolution is 1024 x 768 by default, for which you can run the first script under the "For Windows"example.

If you need to run your test on a larger resolution, you need to use the second script, that is the example "For Windows resolution".

Configure the "Execute Katalon Studio Tests" task

Once you have installed the extension, you need to configure the Execute Katalon Studio Tests task to complete the integration.

  1. Open your Katalon project in Azure DevOps. For more information on how to create a new project in Azure DevOps, see Microsoft Documentation.
    Note:
    • If you don't have a project in Azure DevOps, download or clone our demo project from our GitHub repository.

  2. In your opened project, go to Repos > Files and click Set up build.

    add task Set up Build

  3. In the Configure your pipeline page that displays, select Starter pipeline.

    add task Starter pipeline

  4. Click Show assistant. The Task section appears.

    add Task

    add Task

  5. In the Task section, search for Execute Katalon Studio Tests. You can do this by typing Execute Katalon Studio Tests, in the Search box or Task category.

    add task Execute Katalon Studio Tests

  6. Fill in the various fields under Execute Katalon Studio Tests such as:

    • Download Katalon Studio version: The version of Katalon Studio that you want to run your test with.

    • Use pre-installed Katalon Studio: Fill this field only when Katalon Studio cannot be downloaded automatically (often due to network conditions).

      For Example:

      • Linux: /var/lib/azureDevops KatalonStudioLinux_64-5.10.1

      • Windows: /var/lib/azureDevops KatalonStudioWindows_64-5.10.1

    • Command Arguments: Enter the command arguments directly in the text area. You can generate the arguments from Katalon Studio by using the command builder. To learn more, see Command Syntax.

      For Example:

      -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'

      enter the arguments

    • X11 DISPLAY (For Linux): Leave the X11 DISPLAY field blank.

    • Xvfb-run (For Linux): Configure Xvfb-run following Ubuntu Manuals on Xenial xvfb-run. The function still works if you only change the resolution to 1024x768x24 and leave other options as-is.

  7. Click Add to add these inputs to the pipeline script. The updated pipeline script looks like this:

    updated pipeline script

  8. After everything is set up, click Run. If the script is properly configured, it is successfully executed via Azure DevOps.

    Azure DevOps extension result 1

    Azure DevOps extension result 2

Checking for errors in the pipeline script

Execution of a test script is successful if all the steps in the script are accurate and devoid of any information that does not conform to the requirements of the application that is running the script. Thus, for a proper output to be obtained after running a test script, you have to ensure that all steps in the script are in correct order and devoid of any errors.

However, if the script fails to execute and you get an error message, perform the following steps:

  1. Click Edit pipeline to edit the script.

    Azure DevOps Edit pipeline

  2. The Azure DevOps YAML script displays accordingly. Make the relevant update to the script and click Save.

    Azure DevOps YAML script

  3. Click Run to rerun the script once again.

For any troubleshooting issues, go to troubleshooting in azure devops pipelines.