Gitlab Pipelines

Now that we have successfully automated a couple of Use Cases, how about creating an automated process that can handle re-running our tests whenever changes to the source code happens?

For the remainder of this presentation I will cover how we can leverage Gitlab’s Pipelines to perform the following actions:

  • Rebuild this presentation
  • Execute our automated tests via pytest against
    • Firefox
    • Google Chrome
  • Lastly, assuming our tests pass, re-run them against a matrix of web browsers and operating systems on SauceLabs

All of these actions will be performed by our Pipeline automatically and without requiring any infrastructure from you, every time code changes get merged into this repository.

Pipelines

“A pipeline is a group of jobs that get executed in stages(batches). All of the jobs in a stage are executed in parallel (if there are enough concurrent Runners), and if they all succeed, the pipeline moves on to the next stage. If one of the jobs fails, the next stage is not (usually) executed. You can access the pipelines page in your project’s Pipelines tab. [1]”

Defining Pipelines

a pipeline on Gitlab is defined by adding a .gitlab-ci.yml file to your repository. The pipeline for this repository can be seen bellow:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
image: python:3.7

stages:
  - Generate Docs
  - Test Pytest
  - Test SauceLabs

Build Document:
  stage: Generate Docs
  before_script:
    - make install
  script:
    - make html
    - doc8 --ignore D001

UI Pytest Chrome:
  stage: Test Pytest
  before_script:
    - make install
  script:
    - pytest -vvv --driver Remote --capability browserName chrome --host selenium --port 4444 tests/pytest/test_SeleniumPytest.py
  services:
    - name: selenium/standalone-chrome
      alias: selenium

UI Pytest Firefox:
  stage: Test Pytest
  before_script:
    - make install
  script:
    - pytest -vvv --driver Remote --capability browserName firefox --host selenium --port 4444 tests/pytest/test_SeleniumPytest.py -k test_page_title
  services:
    - name: selenium/standalone-firefox
      alias: selenium

UI Pytest SauceLabs Matrix:
  stage: Test SauceLabs
  before_script:
    - make install
  script:
    - pytest -vvv --driver SauceLabs tests/saucelabs/test_SeleniumSauce.py -k test_page_title

There are a total of 4 unique jobs:

  • Build Document: executes make html and builds the document for this presentation
  • UI Pytest Chrome: Runs automated tests against a Dockerized instance of Google Chrome
  • UI Pytest Firefox: Runs a subset of automated tests against a Dockerized instance of Firefox
  • UI Pytest SauceLabs Matrix: Runs a subset of automated tests against a matrix of web browsers and operating systems on SauceLabs

Furthermore, these jobs are executed by stages, each stage being a requirement for the next one, as shown below:

  • Generate Docs
  • Test Pytest
  • Test SauceLabs
_images/gitlab-pipelines.png

Note

In order to run our automated tests against SauceLabs, we need to define to environmental variables that will grant the automation access to SauceLabs environment:

  • SAUCELABS_API_KEY
  • SAUCELABS_USERNAME
_images/saucelabs-variables.png

Since we don’t want to keep this information accessible to the WWW, Gitlab offers a convenient Variables page that let’s you store per-project or per-group variables which can then be accessed by your jobs at runtime. Furthermore, you can mark them as proctected so that they won’t be displayed anywhere on the UI or logs!

Though I won’t go into the steps required to create jobs or the syntax, this document is a very good start. The .gitlab-ci.yml file provided here should also provide you with the ‘seed’ for your onw file.

[1]Pipelines https://docs.gitlab.com/ee/ci/pipelines.html