<!--
.. title: Run pytest Tests Easily Locally and in Pipeline
.. slug: run-pytest-tests-easily-locally-and-in-pipeline
.. date: 2025-06-02 03:24:53 UTC
.. updated: 2025-06-02 03:24:53 UTC
.. tags: ansible, pytest, python
.. category:
.. link:
.. description:
.. type: text
-->

At my day job we write Ansible custom modules and filter plugins. Since these
are pure Python pieces of code, we also write unit tests for them. We love
pytest and all our tests are written in it.

The challenge we faced was how to easily run these tests locally on developer
machines and in CI/CD pipeline. The solution was easy enough once we figured
out how to meld Python with make.

<!-- TEASER_END: Read more -->

[TOC]

## Directory Layout

A typical layout for an Ansible repository would look something like,

```
$ tree -a my-ansible-repo
my-ansible-repo
├── filter_plugins
│   ├── __init__.py
│   └── mycustomfilters.py
├── library
│   ├── __init__.py
│   └── mycustommodule.py
├── plays
│   └── myplay.yml
├── roles
│   └── tasks
│       └── main.yml
└── tests
    ├── Makefile
    ├── filter_plugins
    │   ├── __init__.py
    │   └── test_mycustom.py
    └── library
        ├── __init__.py
        └── test_mycustom.py

9 directories, 11 files
```

## Makefile

The file my-ansible-repo/tests/Makefile, to easily run tests, would look like
below.

Since we use GitLab at work, that's what this file is geared towards. Notice
the use of CI_PROJECT_DIR environment variable. In GitHub Actions, the
equivalent is GITHUB_WORKSPACE. In Jenkins, the equivalent is WORKSPACE. Other
than this difference, this example should work for your environment just as
well.

Another caveat is that we use GNU make and the make magic we use is specific
to it.

```
ifeq ($(CI_PROJECT_DIR),)
	MAKEFILE_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
	MAKEFILE_DIR := $(dir $(MAKEFILE_PATH))
	ROOT_DIR := $(shell dirname $(MAKEFILE_DIR))
else
	ROOT_DIR := $(CI_PROJECT_DIR)
endif

ifeq ($(PYTHONPATH),)
	LIBRARY_PYTHONPATH=$(ROOT_DIR)/library
	FILTER_PLUGINS_PYTHONPATH=$(ROOT_DIR)/filter_plugins
else
	LIBRARY_PYTHONPATH=$(PYTHONPATH):$(ROOT_DIR)/library
	FILTER_PLUGINS_PYTHONPATH=$(PYTHONPATH):$(ROOT_DIR)/filter_plugins
endif

.PHONY: test
test: library-tests filter-plugins-tests

.PHONY: library-tests
library-tests:
	PYTHONPATH=$(LIBRARY_PYTHONPATH) pytest -vvv -rP $(ROOT_DIR)/tests/library/

.PHONY: filter-plugins-tests
filter-plugins-tests:
	PYTHONPATH=$(FILTER_PLUGINS_PYTHONPATH) pytest -vvv -rP $(ROOT_DIR)/tests/filter_plugins/
```

The way we use this Makefile is,

```
$ cd my-ansible-repo/tests

$ make test
```

## Tests

The tests would look something like,

```
$ cd my-ansible-repo/tests

$ head -n 3 library/test_mycustom.py
#!/usr/bin/env python3

import mycustommodule
```

```
$ cd my-ansible-repo/tests

$ head -n 3 filter_plugins/test_mycustom.py
#!/usr/bin/env python3

import mycustomfilters
```

As you can see, we import our relevant custom code assuming it is already in
[sys.path](https://docs.python.org/3/library/sys_path_init.html). How we
manipulate sys.path without adding it in our code is the real technique we want
to demonstrate here.

## PYTHONPATH

Ansible uses conventions for its
[directory layout](https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#directory-layout)
and can load and use Python code easily. On the other hand, since Python
doesn't know where our code (i.e. custom modules and filter plugins) lives, we
need to tell it. For this reason we use the environment variable
[PYTHONPATH](https://docs.python.org/3/library/sys_path_init.html).

We add the path of the custom modules (library) to PYTHONPATH and run tests
only for the modules in one make target. In another make target we run tests
only for the filter plugins and thus PYTHONPATH is set to that directory.
Although we could combine them in a single make target, we prefer to use
separate ones in case we only want to run one set of tests. We do create
another, default target that combines all these tests. This default target is
usually what is run in pipeline.

We use ROOT_DIR to get the absolute path of the repository on the file system.
This is added to PYTHONPATH to find our custom code.

## ROOT_DIR

We set the ROOT_DIR make variable differently depending on where make is being
run. On local developer machines we use GNU make magic. In pipelines we set the
value to the root directory of the platform running the pipeline, e.g. GitLab,
GitHub, Jenkins, etc.

### Local

We use make magic to determine the path of the my-ansible-repo directory. On a
developer machine it could be anywhere that the developer prefers. For example,
I like to store my code in ~/repos and my colleagues may like to store it in
~/code. No matter where we store it, once we use make magic, PYTHONPATH gets
the correct directory path. What's this magic that we speak of?

#### MAKEFILE_LIST

[MAKEFILE_LIST](https://www.gnu.org/software/make/manual/html_node/Special-Variables.html)
is a special variable that,

> Contains the name of each makefile that is parsed by make, in the order in
> which it was parsed. The name is appended just before make begins to parse
> the makefile.

The list at this point contains one item: Makefile.

#### lastword

From the above list we can extract the name of the current Makefile, which
unsurprisingly is Makefile in our case. But rather than hard coding it, which
we most certainly could in our case without any issues, we use programmatic
discovery.

The last item of the list is what we need to extract. For this we use the text
function
[lastword](https://www.gnu.org/software/make/manual/html_node/Text-Functions.html#index-lastword).

The value at this point is: Makefile.

#### abspath

We get the absolute path of the current Makefile by using the GNU make file
name function
[abspath](https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html#index-abspath-1).

The value on my machine at this point is ~/repos/my-ansible-repo/Makefile. On
my colleague's machine it is ~/code/my-ansible-repo/Makefile.

#### dir

The above path includes the file name. To remove the file name and only keep
the directory, we use the
[dir](https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html#index-dir)
file name function.

The value at this point on my machine is ~/code/my-ansible-repo/. Let's keep
using my convention and not bring in my colleague's from here on.

#### dirname

Finally, we use the command
[dirname](https://www.man7.org/linux/man-pages/man1/dirname.1.html)
to get the path of, not including the directory, my-ansible-repo. On my machine
the value is ~/repos.

### Pipeline

Depending on which platform we use to develop and run pipelines, the ROOT_DIR
will be different and is set to the following environment variable.

| Platform | Special Environment Variable |
| -------- | ---------------------------- |
| GitLab   | CI_PROJECT_DIR               |
| GitHub   | GITHUB_WORKSPACE             |
| Jenkins  | WORKSPACE                    |

## pytest

With both PYTHONPATH and ROOT_DIR set properly, pytest is able to load the
Python code from our custom Ansible pieces as well as our tests respectively.

We use the `-vvv` flag to increase verbosity. It lists all tests as it runs
them rather than the default of displaying dots to indicate a test.

We use the `-rP` flag to show the captured stdout as well, which can be very
useful in many cases.

For more details, see
[command line flags reference](https://docs.pytest.org/en/8.3.x/reference/reference.html#command-line-flags).

## Beyond Ansible

This technique can be re-used in non-Ansible cases. Let's say you have a Python
code base that is laid out in unconventional ways. You can still load certain
directories into the sys.path and run tests against them without needing to
modify your test code to search in different environments.