Run test cases directly in the host machine

The CanITrust Backend is designed to run all Selenium tests in a Docker-container environment where all requisites for a successful test run is prepared. However, somebody might find it more convenient to run them in their own host machine, especially during development. They can see in "live-mode" how the browser is spawned and the test_app is loaded. This article shows you how it can be done.

Steps to run Selenium tests directly in the host machine

Step 0: run the test_app

The test_app needs to be started before the driver runs. In the CIT backend folder run the following command: docker-compose -f docker-compose.local.yml up -d. This command starts the test_app and the dns_server.

> docker-compose -f docker-compose.local.yml up -d
Starting test_app ... 
Starting test_app ... done
Starting dns_server ... 
Starting dns_server ... done

Step 1: modify the code

By default, the driver checks for the presence of the test_app and the dns_server before it executes the tests. This check is done by a "ping" to the corresponding hostnames. However, such hostnames are only available in a docker-compose context. The check is no longer needed when we start the driver manually, as the test_app and dns_server have already been started in step 0. To disable the test, comment-out some lines of code in the driver/helper.py file, as following:

logger.debug('Wait for dns_server and test_app containers up...')
# start_containers_time = time.time()
# while not check_connect('dns_server', 53) or not check_connect('test_app', 80):
#     time.sleep(3)
#     if (time.time() - start_containers_time) > 60:
#         raise Exception('Waiting for dns_server and test_app containers up timeout > 60 secs')
# # Update DNS server
# os.system('echo "nameserver {}" > /etc/resolv.conf'.format(socket.gethostbyname('dns_server')))
logger.debug('Dns_server and test_app containers already up...')

If not being disabled, the check_connect commands in the code snippet above will never succeeds as the dns_server and test_app hostname won't be resolvable in your host machine. The script would be terminated when the timeout exception is thrown after ~60s.

Step 3: prepare the python environment

Some dependencies need to be installed for the python scripts to run properly. This can be done by running the following command:

pip install -r driver/requirements.txt

(Dependencies are listed in the driver/requirements.txt file).

Step 4: prepare the webdriver

When running in "Docker container" mode, the Backend's Selenium tests use a web-driver (Geckodriver), which is available in the driver Docker container. In order to run Selenium tests in your host machine, we will need to set up a web-driver in the host machine as well. The steps to setup a web-driver are different for different operation systems but the final goal is to have a geckodriver available in your system path. The binary of geckodriver can be downloaded from here. Apart from geckodriver, other web-drivers can also be used (e.g. chromedriver). In that case, the web-driver profile in driver/testcases/testCase.py needs to be updated accordingly.

Step 5: run the test!

After the above steps are taken, the Selenium tests are ready to be run directly in your host machine, for example:

python3 driver/driver_cli.py runlocal -t 38

By default, the browser (Firefox) is run in headless mode. If you want to see the browser in action, just find the following line in the driver/testcases/testCase.py file and comment it out:

options.add_argument("--headless")

Debugging

It is possible to debug the test script with debugpy. You can enable this module by running the test with the following command (instead of the command shown in step 5.):

python3 -m debugpy --listen 127.0.0.1:5678 --wait-for-client driver/driver_cli.py runlocal -t 38

Debugpy then will listen on port 5678. You will need to connect a debugpy client via this port to the debugpy module for the test to start running.

Troubleshooting

A dependency is not available

In step 5., if running the test throws the error: No module named <module-name>, we need to verify if step 3. has been done correctly. Sometimes, due to python-version conflict, using the pip install command does not install the dependencies in the correct python environment. It is, therefore, recommended to try installing dependencies again using this command:

python3 -m pip install -r driver/requirements.txt --user

Untrusted self-signed SSL certificate

When testing the test_app using https, the self-signed SSL certificate used in the test_app will not be trusted by your host machine by default. You can decide whether to "trust" the root CA (available in test_app/src/certs-local) or run "https" testcase in the Docker-container environment, where this CA should be trusted by default.

Connection failure

If the web-driver is spawned successfully but it exits immediately with an exception similar to:

Reached error page: about:neterror?e=connectionFailure&u=https%3A//cors-cache-misc.test-canitrust.com/website-jwt-co-xhr.html&c=UTF-8&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20cors-cache-misc.test-canitrust.com.

This indicates that the browser couldn't reach the test_app. Please check the following points:

• the test_app has been started correctly (step 0)
• the Selenium test script uses the correct port mapping of the test_app. The default mapping configuration in the docker-compose.local.yml file maps port 80 of the test_app to port 8080 on your host machine. If you want to access the test_app via your 127.0.0.1 IP, it is necessary to use port 8080. Therefore, the test script must open http://test-canitrust.com:8080 instead of http://test-canitrust.com (port 80).