This post was written by Daniel Gront from Performance Protocols team
The Application Automation Tools plugin for the Jenkins continuous integration server provides a mechanism for executing LoadRunner Controller scenarios as part of a build script. This plugin allows you to trigger a performance test as a build step and present the results in the Jenkins user interface.
You can only integrate scenarios which have service level agreements (SLAs). This allows you to quickly determine whether the test passed or failed and if performance was affected.
- Install Jenkins (we support only the five latest LTS versions, see here for list).
- Install the Jenkins Micro Focus Application Automation Tools plugin
There are two types of jobs that you can create: FreeStyle or Pipeline. FreeStyle is more UI friendly and Pipeline is in a form of code which is more flexible.
Set up a job
- Go to the Jenkins Server home page.
- Click the New Itemlink or select an existing job.
- Enter an item name (for a new job).
- Select Freestyle Projectand click OK.
- In the Project Configuration section, select Restrict where this project can be run, and select the appropriate node. For details, see Create an execution node.
- Scroll down to the Build
- Expand the Add build stepdrop-down and select Execute Micro Focus tests from the file system.
- Click the LoadRunner-Specific Settingsbutton and specify the following settings:
- Controller Polling Interval.The interval in seconds by which to poll the Controller for the scenario status. The default is 30 seconds.
- Scenario Execution Timeout. The total time in seconds to allow for the execution of all listed scenarios. After this time, the Controller process will be terminated.
- Errors to Ignore.You can specify which errors to ignore during the run. For example Error: CPU usage for this load generator has exceeded 80%. Enter each error string on a separate line.
- Analysis Template. Apply a template for the build (path to a .tem file). Leave blank in order to use the default template.
- Display Controller. Display the controller while the scenario is running.
- In the Testsbox, enter a test with its full absolute path, or a folder or MTB containing one or more tests or LoadRunner scenarios. To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.
- Indicate a timeout in seconds after which the job will fail in the Timeoutbox (optional).
- Click Applyto save your changes and continue with more build steps. Click Save when you are finished adding build steps.
- Select Pipelineand click OK.
- To restrict where this project can run, first see Create an execution node and then Controlling your build environment.
- Scroll down to the Pipeline
- Click Pipeline Syntax.
- In Sample Step, select loadRunnerTest: Run LoadRunner performance scenario tests
- In the Testsbox, enter a test with its full absolute path, or a folder or MTB containing one or more tests or LoadRunner scenarios. To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network
- In Report archive mode follow Set up the Post Build Actions
- Follow FreeStyle Job section, steps 5 to 8.
- Click Generate Pipeline Script.
- Copy the output below the Generate Pipeline Script.
- Go back to the Pipeline
- Paste the copied code into the Script block (read here for how to write pipeline).
- Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.
Set up the Post Build Actions
- In the Post-build Actionssection, expand the Add post-build action drop-down and select Publish test result.
- Select an archiving option for the test run results:
- Archive test reports for failed tests: Only save test result reports for failed tests.
- Always archive test reports:Always save test result reports.
- Always archive and publish test reports:Always save and publish test result reports.
- Do not archive test reports:Never save the test results.
Run the job
Run or schedule the job as you would with any standard Jenkins job.
Review the results in the Run Results Viewer
- Extract the files from the archive file. When you when you extract the zip file contents, it creates the following folders and files:
- LRR folder- The Controller raw result folder, viewable in LoadRunner Analysis.
- LRA folder- The Analysis session folder, viewable in LoadRunner Analysis.
- HTML folder- HTML reports in two subfolders, IE and Netscape, in their respective formats.
- xml- An XML report showing the status of the SLA in the test.
- Open the Run Results Viewer and select File> Open.
- In the Open dialog box, select` Results XML fileand navigate to the xml file in the folder that you extracted.
- Click Openand view the results.
Note: The pass and fail criteria for LoadRunner scenario tests are based on the LoadRunner Service Level Agreement (SLA). If you do not define an SLA, the scenario will fail. For details, see the LoadRunner Help Center.
Review the results in Jenkins
You can also view the results from the Jenkins interface, in the following modes:
- Per job/project
- Per build/run
Per Job / Project (cross build/run results)
Click Project performance report in the left side Job menu. This lets you review the various results from the job, per scenario. You compare the job results using an SLA.
- Click Performance reportto examine the performance summary report with its different results.
- Click Transaction summaryto open a detailed view of the transactions statistics taken directly from LoadRunner Analysis report.
- Click Rich Reportto show a high-level report of the build run. (This requires you to enable Rich Reports in Analysis and generate a PDF. For details, see the Edit Template dialog box.
- Click Simple JUnit style SLA result
- Click on Test Result.
- Click on All tests to see the overview.
- Select a scenario by clicking on its name.
- Click on Historyin the left panel to compare the results with those from previous runs.
Note: These reports might require you to perform a Security header override. For details, see the Content Security Policy Header.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.