Matrix Requirements

Matrix Requirements is an Application Lifecycle Management System and Quality Management System made to design medical products.

QABench offers an integration (called htf-matrixreq) to enable specification of tests within Matrix Requirements, as well as sending test results back.

Installation

htf-matrixreq is included when htf[full] is installed on your system. It can also be installed explicitly with htf[matrixreq].

See installation instructions for more details.

General Usage

In general, Matrix Requirements allows you to create and manage requirements. In addition, it allows you to specify tests that cover these requirements, and therefore link to them.

For tests, multiple test executions can be created and managed in folders.

htf-matrixreq is run with at least one of these folder ids, recursively collects the test execution items, finds the corresponding test specifications, runs the test and sends back the results for the implemented tests to the corresponding test execution items automatically.

htf-matrixreq needs at least one test execution folder in Matrix Requirements. To create that please follow the instructions in the Matrix Requirements Documentation.

Tests that will be run need to implement at least on test specification. See Linking Implemented Tests for more details.

To get the id of the test execution folder, simply click on it in the browser and get the id from the url or the title.

_images/matrixreq-test-execution-folder-id.png

In the example, the test execution folder id is F-XTC-7.

Linking Implemented Tests

You can link the implemented Matrix Requirements test specifications in to tests in htf using the decorator :func:htf.implements for script based tests or using the Implements: keyword for BDD tests.

The following example shows a test with linked test specifications in Python.

@htf.implements("TC-1", "https://<user>.matrixreq.com/<project>/TC-2")
def test_testcase_1_and_2(step):
    with step("First test step", expected_result="This step succeeds"):
        assert True

    with step("Second test step", expected_result="This step succeeds, too"):
        assert True

The following example shows a corresponding test in BDD.

Feature: Feature with implemented tests

    Implements: TC-1, https://<user>.matrixreq.com/<project>/TC-2

    Scenario: Scenario with implemented tests
        Implements: TC-3
        # ...

API Token

htf-matrixreq uses the Rest API of Matrix Requirements and thus needs an API token to function. To create an API token, login into Matrix Requirements and navigate to https://<user>.matrixreq.com/adminConfig/Token.

Create an API token and store it.

Usage from Command Line

htf-matrixreq supports the same command line arguments as htf does and adds a few other.

For example to run all tests in the folder tests for the test execution folder ids F-XTC-1 and F-XTC-7, run:

htf-matrixreq --items F-XTC-1,F-XTC-7 tests/

A full example in which all options are set via command line would be:

htf-matrixreq \
    --host https://<user>.matrixreq.com \
    --project <project> --items F-XTC-1,F-XTC-7 \
    --token <token> \
    --set Tester=tester \
    --set Version=1.2.3 \
    tests/

This example also sets the extra fields Version and Tester` within the Matrix Requirements test execution item, and gets all other options from command line.

Usage in Python

htf.matrixreq.main supports the same parameters as htf.main does and adds a few other.

from htf.matrixreq import main

if __name__ == "__main__":
    main(
        host="https://<user>.matrixreq.com",
        project="<project>",
        token="<token>",
        items=["F-XTC-1", "XTC-7"],
        extra_fields={
            "Tester": "tester",
            "Version": "1.2.3",
        },
        tests=["tests/"],
    )

htf.matrixreq.run is an alias for htf.matrixreq.main.

Python Functions

htf.matrixreq.main(host: str | None = None, project: str | None = None, items: List[str] | None = None, token: str | None = None, extra_fields: Dict[str, str] | None = None, insecure: bool = False, date_format: str = 'yyyy/MM/dd', date_time_format: str = 'yyyy/MM/dd HH:mm:ss', *htf_args: Any, **htf_kwargs: Any) → int

The htf.matrix.main() and htf-matrixreq command line utility is used to run tests and send the results back to Matrix ALM/QMS.

Parameters:

project – the name of the project
items – the items to search for test execution item
token – the REST API token
extra_fields – extra fields to set for updated test execution items
insecure – if set to True SSL/TLS certificates are not checked
(default (date_time_format) – “yyyy/MM/dd”): the date format for the project
(default – “yyyy/MM/dd HH:mm:ss”): the date time format for the project
*htf_args – htf positional arguments
**htf_kwargs – htf keyword arguments

Returns:

the return code of htf.main() plus the number of errors and warnings produced by this tool.

Return type:

int

htf.matrixreq.run(host: str | None = None, project: str | None = None, items: List[str] | None = None, token: str | None = None, extra_fields: Dict[str, str] | None = None, insecure: bool = False, date_format: str = 'yyyy/MM/dd', date_time_format: str = 'yyyy/MM/dd HH:mm:ss', *htf_args: Any, **htf_kwargs: Any) → int

The htf.matrix.main() and htf-matrixreq command line utility is used to run tests and send the results back to Matrix ALM/QMS.

Parameters:

project – the name of the project
items – the items to search for test execution item
token – the REST API token
extra_fields – extra fields to set for updated test execution items
insecure – if set to True SSL/TLS certificates are not checked
(default (date_time_format) – “yyyy/MM/dd”): the date format for the project
(default – “yyyy/MM/dd HH:mm:ss”): the date time format for the project
*htf_args – htf positional arguments
**htf_kwargs – htf keyword arguments

Returns:

the return code of htf.main() plus the number of errors and warnings produced by this tool.

Return type:

int

Options

To use htf-matrixreq, you need to specify the hostname, the project name, the API token and, if needed, the insecure flag.

These options can either be set via command line options, by storing them in htf.settings, and by setting them via environment variables.

Environment variables are overwritten by htf.settings, whereas command line options overwrite everything else.

Command Line Options

To specify the hostname via command line, use --host <host>.

To specify the project name via command line, use --project <project>.

To specify the API token via command line, use --token <API Token>.

The full list of command line options is:

usage: htf-matrixreq [-h] [--host HOST] [--project PROJECT] --items ITEMS [--token TOKEN] [--insecure]
                     [--set field1=value1,field2=value2,...,fieldN=valueN] [--date-format DATE_FORMAT]
                     [--date-time-format DATE_TIME_FORMAT]

htf-matrixreq - HILSTER Testing Framework integration for Matrix Requiements ALM/QMS System (HILSTER, https://hilster.io)

options:
  -h, --help            show this help message and exit
  --host HOST           The host of the Matrix Requirements project (required)
  --project PROJECT     The name of the Matrix Requirements project (required)
  --items ITEMS         The items to recursively search for tests in. Can be a comma-separated list of items, e.g.
                        --item 'F-XTC-1,F-XTC-2'. (required, can be repeated)
  --token TOKEN         The REST API token to use (required)
  --insecure            Disable SSL/TLS certificate check (can be useful behind some proxy servers)
  --set field1=value1,field2=value2,...,fieldN=valueN
                        Set extra fields on every item, e.g. Tester, Version, etc. Can be repeated.
  --date-format DATE_FORMAT
                        The date format for the project.
  --date-time-format DATE_TIME_FORMAT
                        The date time format for the project.

Settings

You can specify hostname, project name, API token and the insecure flag in htf.settings, too.

To do that add a section in the settings called matrixreq.

YAML Example:

matrixreq:
    host: https://<user>.matrixreq.com
    project: <project>
    token: <token>
    date_format: "yyyy/MM/dd"
    date_time_format: "yyyy/MM/dd HH:mm:ss"
    extra_fields:
        Tester: <tester>
        Version: <version>
    insecure: false  # true|false

JSON Example:

"matrixreq": {
    "host": "https://<user>.matrixreq.com",
    "project": "<project>",
    "token": "<token>",
    "date_format": "yyyy/MM/dd",
    "date_time_format": "yyyy/MM/dd HH:mm:ss",
    "extra_fields" {
        "Tester": "<tester>",
        "Version": "<version>"
    },
    "insecure": false
}

You can leave out any value if it is not needed, or if it is specified elsewhere.

Environment Variables

The following tables shows the supported environment variables and their meaning.

Environment Variable	Meaning
MATRIXREQ_HOST	Hostname, e.g. https://<user>.matrixreq.com
MATRIXREQ_INSECURE	Disable SSL/TLS certificate verification. Any value is supported.
MATRIXREQ_PROJECT	Project Name
MATRIXREQ_TOKEN	API Token, e.g. <token>
MATRIXREQ_DATE_FORMAT	The date format. Default: “yyyy/MM/dd”
MATRIXREQ_DATE_TIME_FORMAT	The date time format. Default: “yyyy/MM/dd HH:mm:ss”

Setting Extra Fields

You can set extra fields within Matrix Requirements test execution items such as Tester or Version when sending test results back to test execution items.

To do this in the command line, add --set name=value pairs. For example in order to set the Version to 1.2.3, use --item Version=1.2.3.

You can also set these extra values using htf.settings.

Date and Time Formats

htf-matrixreq needs to set the test execution timestamps in the specified format set within the Matrix Requirements project.

To view and change the formats, login into Matrix Requirements and navigate to https://<user>.matrixreq.com/adminConfig/settingsdatetime.

_images/matrixreq-date-and-time-settings.png

If the formats differ the test execution timestamps will not be set.

Proxy

You can specify proxies to use by setting the environment variable https_proxy to https://<username>:<password>@<host>:<port>.

On Linux enter:

export https_proxy=https://<username>:<password>@<host>:<port>

and on Windows enter:

set https_proxy=https://<username>:<password>@<host>:<port>

SSL/TLS Verification

In some circumstances it might be necessary to disable SSL/TLS verification. To do this simply add the option --insecure to htf-matrixreq or set the environment variable MATRIXREQ_INSECURE to any value.

On Linux enter:

export MATRIXREQ_INSECURE=true

and on Windows enter:

set MATRIXREQ_INSECURE=true

Debugging

To enable debugging of HTTPS communication set the environment variable MATRIXREQ_DEBUG to any value and htf-matrixreq.

On Linux enter:

export MATRIXREQ_DEBUG=true

and on Windows enter:

set MATRIXREQ_DEBUG=true

Changelog

htf-matrixreq-1.0.2

add support for Python 3.13

htf-matrixreq-1.0.1

fix setting of actual results

htf-matrixreq-1.0.0

initial integration