Behavior Driven Development
Behavior Driven Development (BDD) is an agile software development technique. BDD enables non-technical participants to participate in writing tests.
The HILSTER Testing Framework enables users to write features in a gherkin-like language with add-ons that can be run as tests with seamless integration into the existing ecosystem.
Tests are written in form of Features, Rules, Scenarios and Steps, which are either Given, When, or Then steps. It is also possible to create Data Driven Feature Tests.
Thus every step is implemented in Python based on HILSTER Testing Framework.
HILSTER Testing Framework also offers requirements coverage in combination with the Dashboard.
For more information read the documentation for Behavior Driven Development.
This demo includes end-to-end tests for a web application calculator that runs in a browser.
Setup Webdriver
To be able to run the demo you need a recent webdriver for selenium for your browser. Please check the selenium-python documentation and selenium documentation for more information.
The webdriver shall be placed into behavior_driven_development folder or put into the PATH environment
variable.
Behavior Driven Development based End-to-End Test
The software under test is a calculator running in a browser that can be used interactively.
There are 10 different requirements specified for the calculator.
ID |
Description |
|---|---|
REQ-1 |
The calculator shall have buttons for the digits 0 to 9 |
REQ-2 |
The calculator shall have a clear button |
REQ-3 |
The calculator shall have an equals button |
REQ-4 |
The calculator shall support the mathematical operation + |
REQ-5 |
The calculator shall support the mathematical operation - |
REQ-6 |
The calculator shall support the mathematical operation * |
REQ-7 |
The calculator shall support the mathematical operation / |
REQ-8 |
The calculator shall support integer inputs |
REQ-9 |
The calculator shall support float inputs |
REQ-10 |
The calculator shall have a decimal point button |
The demo consists of multiple tests that are written using features and Python glue code.
The following code blocks show the features covering all requirements.
Addition Feature
The Addition Feature covers requirement REQ-4 and checks wheather the calculator supports the addition operation.
Is uses a scenario outline.
Feature: Addition
This feature describes the addition.
Requirements: REQ-4
Scenario Outline: Addition
Given the calculator is cleared
When <a> is added to <b>
And the equals button is pressed
Then the result is <result>
Examples: Addition
| a | b |
| 1 | 1 |
| 2 | 12 |
| 123 | 456 |
Subtraction Feature
The Subtraction Feature covers requirement REQ-5 and checks wheather the calculator supports the subtraction operation.
Is uses a scenario outline.
Feature: Subtraction
This feature describes the subtraction.
Requirements: REQ-5
Scenario Outline: Subtraction
Given the calculator is cleared
When <a> is subtracted by <b>
And the equals button is pressed
Then the result is <result>
Examples: Subtraction
| a | b |
| 11 | 1 |
| 23 | 5 |
| 1337 | 23 |
Multiplication Feature
The Multiplication Feature covers requirement REQ-6 and checks wheather the calculator supports the multiplication
operation.
Is uses a scenario outline.
Feature: Multiplication
This feature describes the multiplication.
Requirements: REQ-6
Scenario Outline: Multiplication
Given the calculator is cleared
When <a> is multiplied by <b>
And the equals button is pressed
Then the result is <result>
Examples: Multiplication
| a | b |
| 11 | 1 |
| 23 | 5 |
| 1337 | 23 |
Division Feature
The Division Feature covers requirement REQ-7 and checks wheather the calculator supports the division operation.
Is uses a scenario outline.
Feature: Division
This feature describes the division.
Requirements: REQ-7
Scenario Outline: Division
Given the calculator is cleared
When <a> is divided by <b>
And the equals button is pressed
Then the result is <result>
Examples: Division
| a | b |
| 10 | 2 |
| 12 | 3 |
| 900 | 30 |
Decimal Point Feature
The Decimal Point Feature covert requirements REQ-8, REQ-9 and REQ-10 and checks wheather the calculator
supports integer and float inputs.
Feature: Decimal Point
This feature describes the decimal point.
Requirements: REQ-8, REQ-9, REQ-10
Scenario: Decimal Point
Given the calculator is cleared
When 10+2.5 is entered
And the equals button is pressed
Then the result is 12.5
External Data Feature
The External Data Feature covers requirements REQ-4, REQ-5, REQ-6 and REQ-7 to check all mathematical
operations using an external data source input-data.csv.
Feature: External Data
This feature describes the external data scenario outlines.
Requirements: REQ-4, REQ-5, REQ-6, REQ-7
Scenario Outline: External Data
Given the calculator is cleared
When <a><operation><b> is entered
And the equals button is pressed
Then the result is <result>
Examples: Foo
CSV: ../input-data.csv
The content of input-data.csv looks like the following:
a,operation,b,result
1,+,2,3
3,-,1,2
10,*,100,1000
100,/,10,10
Python Generator Data Feature
The Python Generator Data Feature covers requirements REQ-4, REQ-5, REQ-6 and REQ-7 to check all
mathematical operations using a Python based data generator.
Feature: Python Generator Data
This feature describes the external data scenario outlines.
Requirements: REQ-4, REQ-5, REQ-6, REQ-7
Scenario Outline: Python Generator Data
Given the calculator is cleared
When <a><operation><b> is entered
And the equals button is pressed
Then the result is <result>
Examples:
Generator: Calculations-Generator
Fixture
Communicating with the software under test is implemented using a fixture in calculator.py:
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
from typing import Generator, Union, Dict, Optional
import htf
import os
import time
from selenium import webdriver # type: ignore
from selenium.webdriver.common.by import By # type: ignore
from selenium.webdriver.remote.webelement import WebElement
@htf.fixture("session")
def driver() -> Generator[webdriver.Chrome, None, None]: # type: ignore
"""
Webdriver fixture
"""
os.environ["PATH"] = os.environ["PATH"] + ":."
driver = webdriver.Chrome()
yield driver
driver.close()
class Calculator:
"""
Digital twin for web application calculator.
"""
def __init__(self, driver: webdriver.Chrome, settings: htf.Settings) -> None: # type: ignore
self.driver = driver
self.settings = settings
self.operations: Dict[str, WebElement] = {}
self.numbers_elements: Dict[int, WebElement] = {}
self.decimal_point: Optional[WebElement] = None
self.clear: Optional[WebElement] = None
self.delay = settings.delay
# open webapp
self.driver.get(f"file://{os.path.join(os.path.dirname(__file__), settings.webapp)}")
# update elements
self.get_elements()
def get_elements(self) -> None:
"""
Get all elements.
"""
self.clear = self.driver.find_element(By.CSS_SELECTOR, "#clear") # Clear
self.result = self.driver.find_element(By.CSS_SELECTOR, "#viewer") # Results viewer
self.equals = self.driver.find_element(By.CSS_SELECTOR, "#equals") # Equal button
numbers = self.driver.find_elements(By.CSS_SELECTOR, ".num") # List of numbers
for number in numbers:
try:
self.numbers_elements[int(number.get_attribute("data-num"))] = number
except ValueError:
assert number.get_attribute("data-num") == "."
self.decimal_point = number
operations = self.driver.find_elements(By.CSS_SELECTOR, ".ops") # List of operators
for operation in operations:
self.operations[operation.get_attribute("data-ops")] = operation
def input(self, element: str) -> None:
"""
Input on an element in the web application.
Args:
element: the element to be clicked
"""
assert len(element) == 1, "element must contain 1 element"
if element in ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]:
# enter a number
self.numbers_elements[int(element)].click()
elif element in ["."]:
# enter decimal point
self.decimal_point.click() # type: ignore
elif element in ["+", "-", "*", "/"]:
# enter operation
mapping = {"+": "plus", "-": "minus", "*": "times", "/": "divided by"}
self.operations[mapping[element]].click()
elif element in ["="]:
# equals
self.equals.click()
elif element in ["C"]:
# clear
self.clear.click() # type: ignore
if self.delay:
time.sleep(self.delay)
def input_expression(self, expression: str) -> None:
"""
Input an expression
"""
if expression:
for element in expression:
self.input(element)
def calculate_result(self) -> None:
"""
Enter = to calculate the result.
"""
self.input("=")
def get_result(self) -> Union[float, str]:
"""
Enter = and return the result
"""
# get result
try:
return float(self.result.text)
except ValueError:
return self.result.text
def evaluate(self, expression: str) -> Union[float, str]:
"""
Evaluate expression and return the result.
Args:
expression: the expression to be evaluated
Returns:
float: the result from the web application
"""
self.input_expression(expression + "=")
return self.get_result()
def add(self, a: float, b: float) -> Union[float, str]:
"""
Evaluate a+b
"""
return self.evaluate(f"{a}+{b}")
def subtract(self, a: float, b: float) -> Union[float, str]:
"""
Evaluate a-b
"""
return self.evaluate(f"{a}-{b}")
def multiply(self, a: float, b: float) -> Union[float, str]:
"""
Evaluate a*b
"""
return self.evaluate(f"{a}*{b}")
def divide(self, a: float, b: float) -> Union[float, str]:
"""
Evaluate a/b
"""
return self.evaluate(f"{a}/{b}")
@htf.fixture("test")
def calculator(driver: webdriver.Chrome, settings: htf.Settings) -> Generator[Calculator, None, None]:
calculator = Calculator(driver, settings)
calculator.clear.click() # type: ignore
yield calculator
Steps
In order to be able to run features as tests, glue code is required.
The Python glue code is implemented in steps.py:
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
from typing import Dict, Any, Union, Generator
import htf
import htf.assertions as assertions
from behavior_driven_development.calculator import Calculator
@htf.given("the calculator is cleared")
@htf.when("the calculator is cleared")
def clear_calculator(calculator: Calculator) -> None:
calculator.clear.click() # type: ignore
@htf.when("{expression} is entered")
def enter_expression(expression: str, calculator: Calculator) -> None:
calculator.input_expression(expression)
@htf.when("{a} is added to {b:g}")
def add(a: float, b: float, calculator: Calculator) -> None:
calculator.input_expression(f"{a}+{b}")
@htf.when("{a} is subtracted by {b}")
def subtract(a: float, b: float, calculator: Calculator) -> None:
calculator.input_expression(f"{a}-{b}")
@htf.when("{a} is multiplied by {b}")
def multiply(a: float, b: float, calculator: Calculator) -> None:
calculator.input_expression(f"{a}*{b}")
@htf.when("{a} is divided by {b}")
def divide(a: float, b: float, calculator: Calculator) -> None:
calculator.input_expression(f"{a}/{b}")
@htf.when("the equals button is pressed")
def equals_button_pressed(calculator: Calculator) -> None:
calculator.calculate_result()
@htf.then("the result is {expected_result:g}")
def check_result(expected_result: float, calculator: Calculator) -> None:
assertions.assert_equal(expected_result, calculator.get_result())
def _clean_up_data(data: Dict[str, str]) -> Dict[str, float]:
a: Any = data["a"]
b: Any = data["b"]
try:
if "." in a:
a = float(a)
else:
a = int(a)
except TypeError:
pass
try:
if "." in b:
b = float(b)
else:
b = int(b)
except TypeError:
pass
return_data: Dict[str, float] = {}
return_data["a"] = float(a)
return_data["b"] = float(b)
return return_data
@htf.transformation("Addition")
def expected_addition_result(data: Dict[str, str]) -> Dict[str, float]:
clean_data = _clean_up_data(data)
clean_data["result"] = clean_data["a"] + clean_data["b"]
return clean_data
@htf.transformation("Subtraction")
def expected_subtraction_result(data: Dict[str, str]) -> Dict[str, float]:
clean_data = _clean_up_data(data)
clean_data["result"] = clean_data["a"] - clean_data["b"]
return clean_data
@htf.transformation("Multiplication")
def expected_multiplication_result(data: Dict[str, str]) -> Dict[str, float]:
clean_data = _clean_up_data(data)
clean_data["result"] = clean_data["a"] * clean_data["b"]
return clean_data
@htf.transformation("Division")
def expected_division_result(data: Dict[str, Union[float, str]]) -> Dict[str, Union[float, str]]:
data["result"] = float(data["a"]) / float(data["b"])
return data
@htf.data_generator("Calculations-Generator")
def generate_calculations() -> Generator[Dict[str, Union[float, str]], None, None]:
yield dict(a=1, operation="+", b=2, result=3)
yield dict(a=3, operation="-", b=1, result=2)
yield dict(a=10, operation="*", b=100, result=1000)
yield dict(a=100, operation="/", b=10, result=10)
Run Tests
To run the demo, run
cd behavior_driven_development
htf -o calculator.py steps.py features/*.feature
To run the demo with asyncio, run
cd behavior_driven_development_async
htf -o calculator.py steps.py features/*.feature
Alternative Implementation
There is an alternative implementation for the same tests without using Behavior Driven Development but plain Python code:
The tests are implemented in test_web_application.py:
#
# Copyright (c) 2023, HILSTER - https://hilster.io
# All rights reserved.
#
# example how to test a web application calculator
import htf
from behavior_driven_development.calculator import Calculator
@htf.requirements("REQ-1")
def test_digit_button(calculator: Calculator) -> None:
"""
This test asserts that all digits input buttons exists.
"""
htf.assert_almost_equal(calculator.evaluate("1234567890"), 1234567890) # type: ignore
@htf.requirements("REQ-2")
def test_clear_button(calculator: Calculator) -> None:
"""
This test asserts that the clear button clears the result to zero
"""
htf.assert_almost_equal(float(calculator.evaluate("23")), 23) # type: ignore
calculator.clear.click() # type: ignore
htf.assert_almost_equal(float(calculator.get_result()), 0) # type: ignore
@htf.requirements("REQ-3")
def test_equals_button(calculator: Calculator) -> None:
"""
This test asserts that the equals button evaluates the result of the input.
"""
htf.assert_almost_equal(float(calculator.evaluate("1+2")), 3)
@htf.requirements("REQ-4")
@htf.data([[1, 1], [2, 12], [123, 456]], unpack=True)
def test_addition(a: int, b: int, calculator: Calculator) -> None:
expected_result = a + b
htf.assert_almost_equal(float(calculator.add(a, b)), expected_result)
@htf.requirements("REQ-5")
@htf.data([[11, 1], [23, 5], [1337, 23]], unpack=True)
def test_subtraction(a: int, b: int, calculator: Calculator) -> None:
expected_result = a - b
htf.assert_almost_equal(float(calculator.subtract(a, b)), expected_result)
@htf.requirements("REQ-6")
@htf.data([[11, 1], [23, 5], [1337, 23]], unpack=True)
def test_multiplication(a: int, b: int, calculator: Calculator) -> None:
expected_result = a * b
htf.assert_almost_equal(float(calculator.multiply(a, b)), expected_result)
@htf.requirements("REQ-7")
@htf.data([[10, 2], [12, 3], [900, 30]], unpack=True)
def test_division(a: int, b: int, calculator: Calculator) -> None:
expected_result = a / b
htf.assert_almost_equal(float(calculator.divide(a, b)), expected_result)
@htf.requirements("REQ-4", "REQ-5", "REQ-6", "REQ-7")
@htf.csv_data("input-data.csv")
def test_external_data(a: int, operation: str, b: int, result: int) -> None:
print(a, operation, b, result)
@htf.requirements("REQ-8", "REQ-9", "REQ-10")
def test_integer_and_float_inputs(calculator: Calculator) -> None:
"""
This test asserts that integer as well as float inputs are supported.
"""
htf.assert_almost_equal(float(calculator.evaluate("10+2.5")), 12.5)
if __name__ == "__main__":
htf.main(open_report=True)
To run the demo, run
cd behavior_driven_development
htf -o calculator.py test_web_application.py
To run the demo with asyncio, run
cd behavior_driven_development_async
htf -o calculator.py test_web_application.py