<!-- HILSTER htf 4.3.1 -->
**htf 4.3.1** · [HTML](../cli.html) · [Complete PDF manual](../_downloads/htf-4.3.1.pdf)

<a id="htf-cli-label"></a>

# htf — Command line utility

The HILSTER Testing Framework comes with a command line utility called `htf`.

`htf` can be used to run tests, open docs and view the included
open-source licenses.

## Running tests

To run all tests within the current folder simply run

```bash
htf
```

By default `htf` creates an HTML-testreport called `testreport.html` in the current folder.

<a id="htf-specifying-tests"></a>

## Specifying tests

`htf` may be followed by `run` and `<tests>`. You can also omit `run` and only supply test-specifiers, eg.

```bash
htf run <tests>
```

respectively

```bash
htf <tests>
```

`<tests>` is a list of test-specifiers. They are separated by whitespaces.

A test-specifier may be a python module or a feature file, a globbing-expression for python modules
(e.g. `test_*.py`), a globbing expression for feature files (e.g. `features/*.feature`), a folder or an
[import string](https://docs.python.org/3.5/reference/simple_stmts.html#import). Test-specifiers can be mixed.

If a test-specifier is a python module the file is imported and all testable
objects are collected into a test suite that is used for the test run.

```bash
htf test_1.py
```

If a test-specifier ist a globbing-expression the expression is evaluated and all matching files are imported.

```bash
htf test_\*.py
```

Note: the `\*` lets not `Bash` (or any other shell) evaluate it and passes it to `htf`.

If a test-specifier is a folder the folder is searched recursively and all python modules found within
(see [File pattern]()) are imported and all testable objects are collected into a test suite. Subsequently the test suite
is executed.

Consider the folder `tests`, which contains python modules with unit tests.
To execute all test cases in the folder `tests`, simply run

```bash
htf tests
```

In case `tests` is an importable package or module, located in the python search path, the names might collide.
In this case, simply use

```bash
htf tests/
```

If a test-specifier is an import string, this may be a package, a module, a class (inheriting from [`htf.TestCase`](writing_tests.md#htf.TestCase))
or a method.

If a test-specifier is a package or a module, it is imported and containing tests are executed.

```bash
htf tests  # Example for a python package containing tests
htf tests.test_example  # Example for a python module containing tests
```

If a class is used it should be a testcase inheriting from [`htf.TestCase`](writing_tests.md#htf.TestCase).
This testcase is used in a test suite and all tests are run.

```bash
htf tests.test_example.ExampleTestCase # Example for a class containing tests
```

If a method is used it should be a method within a test case.

```bash
htf tests.test_example.ExampleTestCase.test_one # Example for a test method
```

## Running multiple tests

To run multiple tests within a test run simply add more test-specifiers
at the end.

```bash
htf test1 test2 test3
```

Different test-specifiers can also be mixed

```bash
htf tests/ test1.Test1TestCase test2.py
```

## Test-title

To specify the test’s title add the `-T` or `--title` option followed by a string
that will be used as the title. If the title contains whitespaces, make sure to put quotes around it.

```bash
htf --title="This is my first test" tests/
```

## HTML-report

By default `htf` generates an HTML-report called `testreport.html`.
To specify an HTML-report add `-H` or `--html-report`.

```bash
htf --html-report="tests.html" tests/
```

This option may be repeated to generate multiple HTML-reports.

```bash
htf --html-report="tests1.html" --html-report="tests2.html" tests/
```

To open all HTML-reports use `-o` or `--open-report`.

```bash
htf --html-report="tests.html" -o tests/
```

## JUnit-XML-report

`htf` is also able to generate JUnit-XML-compatible XML-reports. The reports can be evaluated by different tools, eg.
Jenkins.

To specify an XML-report add `-x` or `--junit-xml-report`.

```bash
htf --junit-xml-report=results.xml tests/
```

This option may also be repeated.

## JSON-report

`htf` is also able to generate JSON-reports.

To specify a JSON-report add `-j` or `--json-report`.

```bash
htf --json-report=results.json tests/
```

This option may also be repeated.

## YAML-report

`htf` is also able to generate YAML-reports.

To specify a YAML-report add `-Y` or `--yaml-report`.

```bash
htf --yaml-report=results.yaml tests/
```

This option may also be repeated.

## Report Server

To send test results to a test report server (hre) use `-r` or `--report-server`.

```bash
htf -r https://reports.host/project/token
htf --report-server=https://reports.host/project/token
```

To disable SSL-verfification you can set the environment variable `HTF_INSECURE` to any value.

```bash
HTF_INSECURE=1 htf -r https://reports.host/project/token  # Linux
set HTF_INSECURE=1  # Windows
htf -r https://reports.host/project/token  # Windows
```

<a id="htf-draft-reports"></a>

## Draft Reports

To create draft reports after each test use `-1` or `--create-drafts`.

```bash
htf -1
```

or

```bash
htf --create-drafts
```

## Minimized Report Size

To minimize the report size use the `-M` or `--minimize-report-size`
switch. When used for example captured stdout and stderr is only kept for failed tests.
By default the report size is not minimized.

```bash
htf -M tests/
```

<a id="htf-shuffle"></a>

## Shuffle Tests

To shuffle tests before running add `-S` or ``--shuffle`.

```bash
htf --shuffle tests/
```

To set the shuffle seed use `--shuffle-seed=<int>|last`. If `'last'` is used the seed is loaded from
`'shuffle_seed.txt'`. If the file cannot be found one seed is created.

After a seed is created it is stored to be used next time. Using the seed the shuffle result can be reproduced.

```bash
htf --shuffle --shuffle-seed=12345 tests/
htf --shuffle --shuffle-seed=last tests/
```

## Filename-templates

Filenames for reports can include prepopulated templates.

Available templates:

> - `{{title}}` - the title as a filename
> - `{{date}}` - the current date `"%Y-%m-%d_%H-%M-%S"`
> - `{{hostname}}`, `{{host}}` or `{{node}}` - the hostname
> - `{{htf_version}}` or `{{version}}` - the htf version, eg. `htf_1.0.0`
> - `{{python_version}}` - the python version, eg. `Python_3.6.1`

To generate an HTML-report containing the current
node and the current date use

```bash
htf -H test_{{host}}_{{date}}.html tests/
```

## Fail fast

If `-f` or `--fail-fast` is used the test run ends after the first
failing test.

## File pattern

If a test-specifier is a folder the default pattern used for file discovery is `test*.py`.
The pattern can be changed using the `-p` or `--pattern` options.

```bash
htf -p \*.py tests/
```

Wildcards (\*) are supported. Depending on your shell, escaping must be applied.

<a id="htf-tagging"></a>

## Tagging

To select tests using a logical expression use the `-t` or `--tags` option followed by an expression.

```bash
htf --tags='foo|bar'
```

To list all available tags use the `tags` commands.

```bash
htf tags /path/to/test
```

To use tags for fixtures use `-F``or ``--fixture-tags` in addition.

```bash
htf -F hardware
htf --fixture-tags=hardware
```

<a id="htf-metadata"></a>

## Metadata

To add metadata about the test run use the `-m` or `--metadata` option followed by
a comma-separated list of key-value pairs.

These options can also be repeated.

This feature can for example be used to add information about a tested firmware version, etc.

```bash
htf -m key1=value1,key2=value2 --metadata key3=value3  # etc.
```

The metadata will occur in the test reports.

<a id="htf-settings-overwriting"></a>

## Overwriting Settings

You can overwrite settings and even hierarchical settings from command line using
`-s <name>=<value>` or `--setting <name>=<value>`.

Hierarchies in settings can be realized with a `.` e.g. `-s top.center.bottom=1`.

For example the following call

```bash
htf -s a=1,b=2 -s top.sub=3
```

leads to settings looking like

```python
{
    'a': '1',
    'b': '2',
    'top': {
        'sub': '3'
    }
}
```

Please note that all supplied settings are strings and you have to care about the casting when these will be used.

This option support to do manual experiments, etc.

<a id="htf-parameters"></a>

## Parameters

You can add parameters to test runs from command line to control the behavior of tests and fixtures.
Add `-P <name>=<value>` or `--parameter <name>=<value>` to the call.

The parameters are available via the [`htf.fixtures.Parameters`](fixtures.md#htf.fixtures.Parameters) fixture.

```bash
htf -P name1=value1,name2=value2 --parameter=name3=value3
```

The parameters will also appear in the test result metadata.

<a id="bash-completion"></a>

## Bash-completion

`htf` supports bash-completion which provides
automatic command completion for the `bash` shell.

Bash-completion completes the first positional commands (`run`,
`docs` and `licenses`, etc.) but also expands test-specifiers
(eg. file names, packages, modules, classes and methods) and
other parameters like `--html-report`, `--metadata`, etc.

Using the bash-completion makes it easy to specify tests (without typing-errors).

Let’s assume you a have an importable python package called `tests`.
Two test-modules called `test_1.py` and `test_2.py` are located within
`tests`.

Both contain a test-case called `TestCase1` and `TestCase2`, respectively.

To easily run `tests.test_1.TestCase1`, type

```bash
htf te<TAB>
    tests    tests.
htf tests.<TAB>
htf tests.test_<TAB>
    tests.test_1    tests.test_1.    tests.test_2    tests.test_2.
htf tests.test_1.<TAB>
htf tests.test_1.TestCa<TAB>
htf tests.test_1.TestCase<TAB>
    tests.test_1.TestCase    tests.test_1.TestCase.
htf tests.test_1.TestCase<ENTER>
# ...
```

## Enable Bash-completion

To enable bash-completion type

```bash
eval $(htf complete)
```

To persistently enable bash completion, add the command to your `~.bashrc`.
alternatively put it into a new file called `/etc/bash_completion.d/htf`, eg.

```bash
echo "eval $(htf complete)" >> ~/.bashrc
. ~/.bashrc
```

or

```bash
echo "eval $(htf complete)" > /etc/bash_completion.d/htf
. /etc/bash_completion.d/htf
```

To check if bash-completion is enabled type

```bash
complete -p | grep htf
```

## Open docs

To open the current docs in your webbrowser, type

```bash
htf docs
```

## View licenses

To view the included and used open-source licenses, type

```bash
htf licenses
```

or

```bash
htf licenses --compact
```

to print the compact form.

To output the contents in Markdown format, type

```bash
htf licenses --markdown
```

or

```bash
htf licenses --markdown --compact
```

to print the compact form.

## Show version

To show the `htf` version, type

```bash
htf version
```
