Core Functionalities
This chapter demonstrates the core of the HILSTER Testing Framework.
Foreword
All test reports are best viewed with a modern browser e.g. Google Chrome or Mozilla Firefox. You should expect some restrictions using Microsoft Internet Explorer.
Test Results
The Test Results demo is located in core/test_results.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
import htf
import htf.assertions as assertions
def test_assert() -> None:
"""
This test shows the easiest way to do assertions.
"""
assert True
def test_success() -> None:
"""
This test succeeds.
"""
assertions.assert_true(True)
def test_failure() -> None:
"""
This test fails.
"""
assertions.assert_true(False)
def test_error() -> None:
"""
This test raises an exception.
"""
raise Exception("Error!")
@htf.skip_if(True, "this test is skipped")
def test_skip_if() -> None:
"""
This test is skipped.
"""
pass
if __name__ == "__main__":
htf.main()
To execute the test, run
htf -o core/test_results.py
To run the demo with asyncio, run
htf -o core_async/test_results.py
from the command line or run it from PyCharm. If you want to change the number of individual tests, you can do so by modifying the variable values at the top of the script.
This will run all tests in the core/test_results.py script and create an HTML test report named
testreport.html, which should be opened by default (because of the -o parameter).
Documentation for the HTML test report can be found here.
Docstrings
Tests can be documented using Python’s docstrings. htf supports docstrings in reStructured text format. This way the user can add some markup.
More information about reStructured text markup can be found here.
Docstrings are converted into html for the HTML test report, too.
The demo is located in core/test_docstrings.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
import htf
def test_docstrings() -> None:
"""
Section
=======
Docstrings support sections.
Bullet lists
============
- This is item 1
- This is item 2
Tables
======
Grid table:
+------------+------------+-----------+
| Header 1 | Header 2 | Header 3 |
+============+============+===========+
| body row 1 | column 2 | column 3 |
+------------+------------+-----------+
| body row 2 | Cells may span columns.|
+------------+------------+-----------+
| body row 3 | Cells may | - Cells |
+------------+ span rows. | - contain |
| body row 4 | | - blocks. |
+------------+------------+-----------+
Simple table:
===== ===== ======
Inputs Output
------------ ------
A B A or B
===== ===== ======
False False False
True False True
False True True
True True True
===== ===== ======
A transition marker is a horizontal line
of 4 or more repeated punctuation
characters.
------------
A transition should not begin or end a
section or document, nor should two
transitions be immediately adjacent."""
if __name__ == "__main__":
htf.main()
To execute the docstrings demo from the command line, run
htf -o core/test_docstrings.py
To run the demo with asyncio, run
htf -o core_async/test_docstrings.py
Steps
Tests can be split up into smaller chunks with a mechanism called steps. The test steps show up in the test reports and help to give tests a better structure and make debugging easier.
The steps demo is located in core/test_steps.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
import htf
def test_steps(step: htf.fixtures.step) -> None:
"""
This test shows how to use steps to structure your tests.
Args:
step (htf.fixtures.step): the step fixture
"""
with step("Initialization"):
print("init.. ")
with step("Body"):
print("body.. ")
with step("Nested step"):
print("This is printed in a nested step")
with step("Cleanup"):
print("cleanup.. ")
if __name__ == "__main__":
htf.main()
To execute the steps demo from the command line, run
htf -o core/test_steps.py
To run the demo with asyncio, run
htf -o core_async/test_steps.py
Attachments
Tests support file and url attachments. This is a great way to attach test artifact to test reports.
The demo is located in core/test_attachments.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
from typing import Tuple, Any
import htf
import numpy as np
import matplotlib.pyplot as plt # type: ignore
from mpl_toolkits.mplot3d import Axes3D # type: ignore
def test_file_attachment(step: htf.fixtures.step, attachments: htf.fixtures.attachments) -> None:
"""
This test plots a graph and attaches it to the test report.
It also shows the usage of tags.
Args:
step (htf.fixtures.step): the step fixture
attachments (htf.fixtures.attachments): the attachments fixture
"""
with step("Create plot"):
def lorenz(
xyz: Any, *, s: int = 10, r: int = 28, b: float = 2.667,
) -> Any:
x, y, z = xyz
x_dot = s * (y - x)
y_dot = r * x - y - x * z
z_dot = x * y - b * z
return np.array([x_dot, y_dot, z_dot])
dt = 0.01
num_steps = 10000
xyzs = np.empty((num_steps + 1, 3))
xyzs[0] = (0., 1., 1.05)
for i in range(num_steps):
xyzs[i + 1] = xyzs[i] + lorenz(xyzs[i]) * dt
# Plot
ax = plt.figure().add_subplot(projection='3d')
ax.plot(*xyzs.T, lw=0.5)
ax.set_xlabel("X Axis")
ax.set_ylabel("Y Axis")
ax.set_zlabel("Z Axis")
ax.set_title("Lorenz Attractor")
# plt.show()
plt.savefig("figure.png")
with step("Attach plot"):
attachments.attach_file("figure.png", "Lorenz Attractor")
def test_url_attachment(attachments: htf.fixtures.attachments) -> None:
"""
This test attaches a URL to a test results.
Args:
attachments (htf.fixtures.attachments): the attachments fixture
"""
attachments.attach_url("https://hilster.io", "HILSTER")
if __name__ == "__main__":
htf.main()
To execute the attachments demo from the command line, run
htf -o core/test_attachments.py
To run the demo with asyncio, run
htf -o core_async/test_attachments.py
Metadata
Tests can be expanded using metadata. htf supports metadata for test cases and for the test run.
More information about metadata can be found in Metadata for htf and
Metadata for htf.main() and in the Keywords section in the docs.
The metadata demo is located in core/test_metadata.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
import htf
Author = htf.Author("$Author: Chuck Norris $")
Source = htf.Source("$Source: /path/to/test_script.py $")
ReviewDate = htf.MetaData("01.01.1970")
def test_with_metadata(metadata: htf.fixtures.metadata) -> None:
"""
This test shows how to use metadata.
Args:
metadata (htf.fixtures.metadata): the metadata fixture
"""
metadata.set("key", "value")
metadata["key2"] = "value2"
if __name__ == "__main__":
# you can also add test run metadata
metadata = dict(date="today", time="now", foo="bar")
htf.main(title="Metadata", metadata=metadata)
To execute the test with metadata from the command line, run
htf -o core/test_metadata.py -mdate=today,time=now,foo=bar
To run the demo with asyncio, run
htf -o core_async/test_metadata.py -mdate=today,time=now,foo=bar
All metadata is included in the test reports.
Fixtures
Fixtures are resources needed by the tests. They can supplement for example communication features, data sources or settings.
- Scope:
Fixtures do not need to be attached to individual tests. Their lifetime can range from an individual test to an entire test session.
- Usage:
Fixtures have a name and tests with a parameter of that name will be started with the fixtures. If a fixture depends on another fixture, you can pass the needed fixture as a parameter.
- Implementation:
Fixtures implemented as Python Generators yielding relevant objects. The generator is decorated with
@htf.fixture(scope).
You can find more information in the Fixtures documentation.
The fixture demo is located in core/test_fixtures.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
from typing import Generator, Any, List
import htf
@htf.fixture("session")
def data_storage() -> Generator[List[Any], None, None]:
data: List[Any] = []
yield data
@htf.fixture("test")
def logger() -> Generator[None, None, None]:
print("Test started.")
yield None
print("Test stopped.")
@htf.test
@htf.data(range(10))
def new_style_test(i: int, data_storage: Any, logger: None) -> None:
data_storage.append(i)
print(data_storage)
assert True
if __name__ == "__main__":
htf.main()
To execute the fixture demo from the command line, run
htf -o core/test_fixtures.py
To run the demo with asyncio, run
htf -o core_async/test_fixtures.py
Threads
A common way to provide tests with the means to communicate to a device via a serial port or a network socket is by running the networking code concurrently to the test in a thread. htf offers a wrapper around threads that ties the life time of the thread to the life time of a test.
The threads demo is located in core/test_threads.py.
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
import htf
import time
def test_background(threads: htf.fixtures.threads) -> None:
"""
This test shows how to use threads.
Args:
threads (htf.fixtures.threads): the threads fixture
"""
def bg() -> None:
print("Running in background!")
threads.run_background(bg)
time.sleep(1.0)
def test_periodic(threads: htf.fixtures.threads) -> None:
"""
This test shows how to use threads to run a function periodically.
Args:
threads (htf.fixtures.threads): the threads fixture
"""
def p() -> None:
print("Periodically called!")
threads.run_periodic(p, period=1.0)
time.sleep(3.0)
if __name__ == "__main__":
htf.main()
To execute the test from command line, run
htf -o core/test_threads.py
Run All Tests
To execute all tests from core, run
htf -o core
This will also create a test report named testreport.html which contains the results of
all tests in the core demo.