Create Jenkins Job for Robot Framework

Once you have created your first tests in Robot Framework, next step is to include those tests in your Continuous Integration (CI) System. Here I will show the different steps to do so in Jenkins.

Let’s assume you have

jenkins_begin

First we create a new job to launch our Robot tests:

create_job

Once the job is created, we configure it.

  1. set up Source Code Management for the source code of the tests:jenkins_svn
  2. set up a first “Build Trigger” on the success of the job that builds the SUT:trigger_build
  3. set up a second “Build Trigger” on changes in the Source Code of the tests:trigger_scm
    This way your tests will be launched either if there is a new build of the SUT or if your tests have changed. Second trigger is relevant because some modifications in your tests may have broken some of them, and you don’t want to wait for the next build of the SUT to find it out. In other words, when a test fails in Jenkins, it is good to know if this is a consequence of a change in the SUT of a change in the tests (if both changed, analysis will be tricker).
  4. get the artefact from the project that build your SUT so that your SUT is available from your Jenkins’ workspace where the Robot tests will be run. To do so you can either use the Jenkins Copy Artifact Plugin or write a piece of Batch/Shell script.screenshot-copy-artefacts
  5. The comes the step in which Robot tests are going to be launched. For this you create a “Execute Shell” Build step that contains, at least:
    pybot path/to/my_tests/

    and all the –variable, –include, –exclude etc. that you use to customise you run.
    One noteworthy command line option in the context of a CI server is –NoStatusRC, which force Robot’s return code to zero even when there is test that fails. This way the status of the Jenkins build can be driven by Robot Jenkins Plugin like you will see in final step.build_robot

  6. Finally, to have a more granular settings of the results of the tests, and keep a copy of the report/log of the test executions located in the Jenkins Server, you can use Robot Framework Plugin. Once the plugin is installed, it will be available in the list of “Post Builds Actions”. A simple configuration would be like this:pluginand after a couple of builds, the project page would look like that:plugin_project_page

Once this basic setup is working, you will find out many options in Jenkins and Robot Framework to get more value out of it. To give just one example, once the test portfolio becomes large and/or long, you might find out that this is not efficient to launch the full regression suite at once when there is a change in the SUT or the tests’ code. A good strategy is to have 2 Jenkins job. The first one (“smoke tests”) is running only a portion of the whole suite that runs quickly (say 5/10 minutes for example):

pybot path/to/my_tests/ --include smoke --exclude not_ready

and the second job (“full tests”) launches all the tests:

pybot path/to/my_tests/ --exclude not_ready

but is launched only when smoke tests are run successfully: chain_builds

so if your SUT or your tests have some essential feature (covered in the smoke test) broken, you will save your machine a “full test”, and, more important, the team have a quicker feedback on the quality of the SUT build.

 

18 thoughts on “Create Jenkins Job for Robot Framework

  1. I have tried this localy where my pybot was installed. I was wondering where do I install it if jenkins is on a remote server and my tests are placed in git. Do I ssh on jenkins server and install Python – robot framework?

    • If Jenkins is on remote server then you will have to install Python and Robot Framework on this remote server (connect via SSH is one good way to do it). But more often, you would have different machines/VM, one for Jenkins Server and other to execute the jobs (slaves). In that case you would have to install Python/Robot on your slaves.

  2. I have installed both robotframework and jenkins. my test runs fine when i run it my local machine. but when I run my test using jenkins some test fails

  3. I am having my tests stored in .txt format , I am not able to Run it with the use of Following Batch commad

    pybot –timestampoutputs -l NetBankLogs.html -r NetbankReport.html -d Path to test\Payment_List.txt

  4. Hi I have a question regarding setting up my jobs on my Local Jenkins.

    Do I have to setup my workspace for it to work?

    I always setup the batch windows command to change the directory to where I place my Robot Framework scripts. I notice that when I do this and build the job it always fails because the filemask output xml cannot be found in the directory. Am not sure what I’m doing wrong.

    hudson.AbortException: No files found in path C:\Program Files (x86)\Jenkins\jobs\Checkout Test Case\workspace with configured filemask: output.xml

    Is there something else I should configure for my tests to run smoothly?

    Thanks!

    • Hi there.

      Thanks. Should I put an absolute directory for this?

      I placed a directory but it still failed. Also for some reason the windows batch command doesn’t seem to recognize pybot even when I installed the Robot Framework plugin.

      C:\Users\MC\automated-tests\Robot Framework\testsuites>pybot –VARIABLE ENV:browserstack –VARIABLE COUNTRY:$Country Checkout.txt –outputdir Result/Test/$Country
      ‘pybot’ is not recognized as an internal or external command,
      operable program or batch file

      and then when creating the output it still has issues with the filemask

      C:\Users\MC\automated-tests\Robot Framework\testsuites>exit 9009
      Build step ‘Execute Windows batch command’ marked build as failure
      Robot results publisher started…
      -Parsing output xml:
      Failed!
      hudson.AbortException: No files found in path C:\Users\toledo-ni\Documents\Codebase\TIGER B2C\automated-tests\Robot Framework\Test\Result with configured filemask: Checkout

  5. Hi all,

    Firstly thank you for this post Laurent. I have a question about Robot Result Plugin. I’ve integrated it in my Jenkins automation job (Jenkins works on AWS). When I tried to open report.html from my local pc I get that error everytime. Do you have any iade about that?

    Best regards.

    Error
    Opening Robot Framework report failed

    Verify that you have JavaScript enabled in your browser.
    Make sure you are using a modern enough browser. Firefox 3.5, IE 8, or equivalent is required, newer browsers are recommended.
    Check are there messages in your browser’s JavaScript error log. Please report the problem if you suspect you have encountered a bug

    • sorry for the (very) late response.
      Where you able to fix your problem.

      > “When I tried to open report.html from my local pc”
      are you trying to open the report from the Jenkins job page?
      Do you have the Jenkins-Robot plugin installed?

  6. Hello,

    Thanks for your all efforts, here I am stuck with my Jenkins + Robot Framwork configuration. I have integrate my Automation script with Jenkins but it is not up and running also I am not getting any error code , After running I am just getting below message.

    Started by user Test Admin
    Building remotely on Slave in workspace D:\JENKINS\jenkins\workspace\
    No emails were triggered.
    [Fact24] $ C:\Windows\system32\cmd.exe -xe C:\Users\XXName~1.TUP\AppData\Local\Temp\hudson7217699004107166534.sh
    Microsoft Windows [Version 10.0.14393]
    (c) 2016 Microsoft Corporation. All rights reserved.

    D:\JENKINS\jenkins\workspace\Robot results publisher started…
    -Parsing output xml:
    Done!
    -Copying log files to build dir:
    Done!
    -Assigning results to build:
    Done!
    -Checking thresholds:
    Done!
    Done publishing Robot results.
    Build step ‘Publish Robot Framework test results’ changed build result to UNSTABLE
    Email was triggered for: Always
    Sending email for trigger: Always
    An attempt to send an e-mail to empty list of recipients, ignored.
    Finished: UNSTABLE

    • “unstable” is what you get when you have some test failures. Are the test executed? Are some of the test failing?

  7. I want to use Jenkins just to initiate my Robotframework tests. I am not using any version control, what I have is a folder structure which contains my Robot scripts.
    I want to trigger these tests nightly at a fixed time using jenkins and generate/view reports. Can you please advise?

    • then you don’t need to have the Version Control phase, and you can just launch a script in your Jenkins Job that launches “pybot” on your Jenkins slave.

Leave a Reply

Your email address will not be published. Required fields are marked *