Data Driven Testing

Data Driven Testing allows to dynamically generate tests using a data source. The data sources can be Python objects or data files.

Tests are turned into data driven tests by using decorators.

You can find more information in the Data Driven Testing documentation.

Pinging a List of Hosts

You can easily create a test prototype that can ping a host that is supplied as a parameter, and then turn this test into multiple tests trying to ping multiple hosts.

The ping demo is located in data_driven_testing/test_ping_hosts.py.

#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#

import htf
import os
import platform


"""
This test case shows how to ping many hosts using data driven testing.
"""


def ping(hostname: str) -> None:
    if platform.platform().startswith("Linux"):
        response = os.system("ping -c 1 " + hostname)
    else:
        response = os.system("ping -n 1 " + hostname)

    htf.assert_equal(response, 0, "the host {} could not be pinged".format(hostname))


@htf.test
def ping_single_host() -> None:
    ping("localhost")


@htf.data(["8.8.8.8", "heise.de", "hilster.io"])
def test_ping_hosts(hostname: str) -> None:
    ping(hostname)


@htf.yaml_data("hosts.yml")
def test_ping_hosts_file(hostname: str) -> None:
    ping(hostname)


if __name__ == "__main__":
    htf.main()

To run the demo, run

cd data_driven_testing
htf -o test_ping_hosts.py

To run the demo with asyncio, run

cd data_driven_testing_async
htf -o test_ping_hosts.py

External Test Data

It is also possible to put test data into external files that could be generated elsewhere.

The demo includes examples for YAML- and CSV-data.

This test data demo is located in data_driven_testing/test_data.py.

#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#

import htf


"""
This test case shows how to run data driven tests with external test data.
"""


@htf.yaml_data("test_data.yml")
def test_data_yaml(hostname: str, port: int) -> None:
    """
    The dictionaries in the YAML data source must match the parameters.
    A test is dynamically generatged for each item in the test data list.
    """
    print("test", hostname, port)


@htf.csv_data("test_data.csv")
def test_data_csv(hostname: str, port: int) -> None:
    """
    The headers must match the parameters.
    A test is dynamically generated for each line in the csv data.
    """
    print("test", hostname, port)


@htf.data(range(100))
def test_success(number: int) -> None:
    pass


if __name__ == "__main__":
    htf.main()

To run the demo, run

cd data_driven_testing
htf -o test_data.py

To run the demo with asyncio, run

cd data_driven_testing_async
htf -o test_data.py

Combinatorial Tests

Let’s assume you have a list of hosts and a list of ports of services you’d like to test.

You can dynamically create tests with multiple data files using the combinatorial decorators.

The demo is located in data_driven_testing/test_combinatorial.py.

#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#

import htf


@htf.product(htf.YAMLFileIterator("hosts.yml"), htf.YAMLFileIterator("ports.yml"))
def test_hostname_and_service(hostname: str, port: int) -> None:
    print("test", hostname, port)


if __name__ == "__main__":
    htf.main()

To run the demo, run

cd data_driven_testing
htf -o test_combinatorial.py

To run the demo with asyncio, run

cd data_driven_testing_async
htf -o test_combinatorial.py