Behave with Allure and Selenium/Playwright: Cucumber in the Python world

Welcome to my course of Cucumber framework for beginners. In this post we will go through setting up the project in Python. The project will be based on Selenium and the results will be reported by Allure reporting framework.

If you need some more general information about Cucumber and Gherkin please read the introduction.

Journey through Cucumber land: introduction to the course of Cucumber

Results

IDE

Python is a high level interpreted programming language developed by Guido van Rossum since 1991.

There are two IDE which I can recommend for editing Python code: JetBrains PyCharm CE and Visual Studio Code (or shortly “vscode”) which is multipurpose code editor with variety of plugins for various languages.

WHAT DO WE WANT TO TEST?

Our test page is pretty simple (the same for the whole course). It contains several typical elements of UI forms:

• text field (input),
• dropdown,
• checkbox,
• table with dynamic content,
• text element with dynamic content.

The URL address of this page is: https://danieldelimata.github.io/sample-page/.

SETTING UP THE SKELETON OF THE PROJECT

Python projects do not have so strictly defined structure as Java or C#, so the software for Python projects do not provide any template mechanism. However, testing software as Behave requires some directories to work correctly. We will now go through the setup process.

python3 -m venv venv
source venv/bin/activate
pip3 install allure-behave
pip3 install behave
pip3 install selenium
pip3 install assertpy
playwright install
deactivate

Lines 1, 2 and the last one are related to venv — lightweight virtual environment. It creates its own copy of Python binaries in the venv directory and can have its own independent set of installed Python packages in its site directories. It helps to perform all operations in isolation.

You can try, how it works, with the following example

which python
which pip
python3 -m venv my_venv
source my_venv/bin/activate
which python
which pip
deactivate

Pay attention to the Python which is run by CI/CD software. The CI/CD executor machine can have several Pythons with own packages (with no packages you need). Using venv you can refer to the Python in which you can have control on installed packages.

Having installed necessary packages in virtual environment we can go through the rest of process.

source venv/bin/activate
venv/bin/behave
deactivate

Behave will answer that it have not found steps directory in features directory. This means that we have to create the following structure of directories.

Project Folder
    ├── features
    │   ├── steps
    ├── venv

Then, we have to place files into appropriate directories. First, of course, we will place feature files.

Here is f01.feature:

Feature: F01 Searching - Clearing of searching criteria

  User story:
  * As a user of Customers page
  * I want to be able to clear searching criteria
  * in order to quickly type new criteria

  Acceptance criteria:
  * After clearing, search criteria and summary should be as in the very beginning.

  Scenario Outline: Clearing of searching criteria
    Given the user is on the page
    When the user enters the value "<search>" in the text-input
    And the user selects value "<column>" in the drop-down
    And the user sets case sensitivity switch to "<case>"
    And the user clears filters
    Then the user should see that search criteria are cleared
    And the user should see that the search result summary is as in the very beginning

    Examples:
      | Search  | Column | Case  |
      | Alice   | Name   | True  |
      | alice   | Name   | False |

At this moment we do not have the glue code yet. We can however start execution just to get some snippets for it. They are produced in logs.

Again we execute:

source venv/bin/activate
venv/bin/behave
deactivate

Running behave will create snippets for steps. They should be placed in steps directory in a file e.g. stepDefinitions.py.

Such snippets of methods may look as follows.

@given(u'the user is on the page')
def step_impl(context):
    raise NotImplementedError(u'STEP: Given the user is on the page')

We should however use more meaningful names than step_impl e.g. the_user_is_on_the_page.

Our automation needs running of Selenium. Let us prepare the file environment.py in the features directory and place there the code responsible for Selenium WebDriver. In the same file we place the code responsible for making screenshots when failure occurs.

environment.py:

import os
from xml.etree.cElementTree import Element, SubElement, ElementTree

import allure
from allure_commons.types import AttachmentType
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

from features.lib.pages.customers_page import CustomersPage


def before_scenario(context, scenario):
    options = Options()
    options.headless = True
    options.add_argument('--headless')
    context.base_url = 'http://'
    context.driver = webdriver.Chrome(options=options)
    context.customers_page = CustomersPage(context)


def after_scenario(context, scenario):
    if scenario.status == "failed":
        allure.attach(context.driver.get_screenshot_as_png(), name="Screenshot", attachment_type=AttachmentType.PNG)
    context.driver.quit()


def before_all(context):
    allure_results_dir = os.path.join("../../allure_results")
    os.makedirs(allure_results_dir, exist_ok=True)
    environment = Element("environment")
    for key, value in os.environ.items():
        param = SubElement(environment, "parameter")
        SubElement(param, "key").text = key
        SubElement(param, "value").text = value
    ElementTree(environment).write(os.path.join(allure_results_dir, "environment.xml"))

To complete our setup we will need one more thing. We want to implement Page Object Pattern. To do this we create two more directories and files basePage.py and customersPage.py. Our directory structure will look as follows.

Project Folder
    ├── features
    │   ├── lib
    │   │   ├── pages
    │   │   │   ├── basePage.py
    │   │   │   ├── customersPage.py
    │   │   │   ├── ...
    │   ├── steps
    │   │   ├── stepDefinitions.py
    │   ├── f01.feature
    │   ├── ...
    │   ├── environment.py
    ├── venv

base_page.py:

from selenium import webdriver
from selenium.webdriver.common.by import By


class BasePage(object):
    """
    Base class that all page models can inherit from
    """

    def __init__(self, context):
        self.context = context

    def find_element(self, locator):
        return self.context.driver.find_element(*locator)

customers_page.py:

from selenium.webdriver.common.by import By
from selenium.webdriver.support import ui
from selenium.webdriver.support.select import Select

from .base_page import BasePage


class CustomersPageLocators:
    CLEAR_BUTTON = (By.ID, "clear-button")
    SEARCH_INPUT = (By.ID, "search-input")
    DROP_DOWN = (By.ID, "search-column")
    MATCH_CASE = (By.ID, "match-case")
    SUMMARY = (By.ID, "table-resume")
    SEARCH_TERM = (By.ID, "search-slogan")
    TABLE = (By.XPATH, "//table")


class CustomersPage(BasePage):
    def __init__(self, context):
        BasePage.__init__(self, context)
        self.wait = ui.WebDriverWait(self.context.driver, 10)

    def clear_button_click(self):
        """
        Click on Clear Filters Button.

        :return: the Button element
        """
        clear_button = self.wait.until(
            lambda d: d.find_element(*CustomersPageLocators.CLEAR_BUTTON)
        )
        clear_button.click()
        return clear_button

    def set_search_input(self, search_input):
        """
        Set value to searchInput field.

        :param search_input: input which should be typed into the field
        """
        self.find_element(CustomersPageLocators.SEARCH_INPUT).send_keys(search_input)

    def set_search_column_drop_down_list_field(self, value):
        """
        Set value to Search Column Drop Down List field.

        :param value: String which should match with one of values visible on the dropdown
        """
        Select(self.find_element(CustomersPageLocators.DROP_DOWN)).select_by_visible_text(value)

    def set_match_case_checkbox_field(self, value):
        """
        Set Match Case Checkbox field to required value.

        :param value: boolean value of the checkbox status true - checked, false - unchecked
        """
        case_checkbox = self.find_element(CustomersPageLocators.MATCH_CASE)
        checkbox_is_selected = case_checkbox.is_selected()
        if str(checkbox_is_selected) != value:
            case_checkbox.click()

    def get_summary_text(self):
        return self.find_element(CustomersPageLocators.SUMMARY).get_attribute("innerText")

    def get_search_term_text(self):
        return self.find_element(CustomersPageLocators.SEARCH_TERM).get_attribute("innerText")

    def get_search_input_text(self):
        return self.find_element(CustomersPageLocators.SEARCH_INPUT).get_attribute("value")

    def get_search_results_table_text(self):
        return self.find_element(CustomersPageLocators.TABLE).text

    def open(self):
        application_url \
            = self.context.config.userdata.get("applicationUrl",
                                          'https://danieldelimata.github.io/sample-page/')

        return self.context.driver.get(application_url)

The above code is related to UI only. It should not contain methods related to tests. These should be placed in glue-code in stepDefinitions.py. This file can look as follows

from assertpy import assert_that
from behave import given, when, then

from features.lib.pages.customers_page import CustomersPage


@given(u'The user is on the page')
def the_user_is_on_the_page(context):
    customers_page = CustomersPage(context)
    customers_page.open()
    context.search_summary_at_very_beginning = customers_page.get_summary_text()


@when(u'the user enters the value "{value}" in the text-input')
def the_user_enters_the_value_in_the_text_input(context, value):
    customers_page = CustomersPage(context)
    customers_page.set_search_input(value)


@when(u'the user selects value "{value}" in the drop-down')
def the_user_selects_value_in_the_drop_down(context, value):
    customers_page = CustomersPage(context)
    customers_page.set_search_column_drop_down_list_field(value)


@when(u'the user sets case sensitivity switch to "{value}"')
def the_user_sets_case_sensitivity_switch_to(context, value):
    customers_page = CustomersPage(context)
    customers_page.set_match_case_checkbox_field(value)


@then(u'the user should see the following result summary "{value}"')
def the_user_should_see_the_following_result_summary(context, value):
    customers_page = CustomersPage(context)
    summary_text = customers_page.get_summary_text()
    assert_that(summary_text).is_equal_to(value)


@then(u'the user should see that the search term is "{value}"')
def the_user_should_see_that_the_search_term_is(context, value):
    customers_page = CustomersPage(context)
    search_term_text = customers_page.get_search_term_text()
    assert_that(search_term_text).starts_with(value)


@when(u'the user clears filters')
def the_user_clears_filters(context):
    customers_page = CustomersPage(context)
    customers_page.clear_button_click()


@then(u'the user should see that search criteria are cleared')
def the_user_should_see_that_search_criteria_are_cleared(context):
    customers_page = CustomersPage(context)
    assert_that(customers_page.get_search_input_text()).is_empty()


@then(u'the user should see that the search result summary is as in the very beginning')
def the_user_should_see_that_the_search_result_summary_is_as_in_the_very_beginning(context):
    customers_page = CustomersPage(context)
    summary_text = customers_page.get_summary_text()
    assert_that(summary_text).is_equal_to(context.search_summary_at_very_beginning)


@then(u'the user should see that the search results are as follows: "{value}"')
def the_user_should_see_that_the_search_results_are_as_follows(context, value):
    customers_page = CustomersPage(context)
    result = customers_page.get_search_results_table_text()
    assert_that(" ".join(result.split())).is_equal_to(value)

Everything together we can run with behave command. Adding -f allure_behave.formatter:AllureFormatter you can produce results in Allure framework format.

source venv/bin/activate
venv/bin/behave -f allure_behave.formatter:AllureFormatter -o ./allure_results
deactivate

Our results are stored in the directory allure_results. To visualize them we can execute:

allure serve allure_results

The results should open in the default web browser.

PLAYWRIGHT

Steps in case of Playwright looks almost identical.

python3 -m venv venv
source venv/bin/activate
pip3 install allure-behave
pip3 install behave
pip3 install playwright
pip3 install assertpy
playwright install
deactivate

The command playwright install causes installation of browsers.

Obviously also some files will look differently. If we properly prepared our step definitions then we will not need to change it, because all details related to browser and its interactions with the tested page should be encapsulated in page objects methods. In our example it is so.

The following files should be changed: environment.py, base_page.py, customers_page.py.

environment.py:

import os
from xml.etree.cElementTree import Element, SubElement, ElementTree

import allure
from allure_commons.types import AttachmentType
from behave.api.async_step import use_or_create_async_context
from behave.runner import Context
from playwright.async_api import async_playwright

from features.lib.pages.customers_page import CustomersPage


def before_scenario(context: Context, scenario):
    use_or_create_async_context(context)
    loop = context.async_context.loop
    context.playwright = loop.run_until_complete(async_playwright().start())
    context.browser = loop.run_until_complete(context.playwright.chromium.launch(headless=True))
    context.page = loop.run_until_complete(context.browser.new_page())
    context.customers_page = CustomersPage(context)


def after_scenario(context: Context, scenario):
    loop = context.async_context.loop

    if scenario.status == "failed":
        allure.attach(loop.run_until_complete(context.page.screenshot()), name="Screenshot",
                      attachment_type=AttachmentType.PNG)
    loop.run_until_complete(context.page.close())


def before_all(context):
    allure_results_dir = os.path.join("../../allure_results")
    os.makedirs(allure_results_dir, exist_ok=True)
    environment = Element("environment")
    for key, value in os.environ.items():
        param = SubElement(environment, "parameter")
        SubElement(param, "key").text = key
        SubElement(param, "value").text = value
    ElementTree(environment).write(os.path.join(allure_results_dir, "environment.xml"))

base_page.py:

class BasePage(object):
    """
    Base class that all page models can inherit from
    """

    def __init__(self, context):
        self.context = context

    def find_element(self, locator):
        return self.context.page.locator(locator)

customers_page.py:

from .base_page import BasePage


class CustomersPageLocators:
    CLEAR_BUTTON = "#clear-button"
    SEARCH_INPUT = "#search-input"
    DROP_DOWN = "#search-column"
    MATCH_CASE = "#match-case"
    SUMMARY = "#table-resume"
    SEARCH_TERM = "#search-slogan"
    TABLE = "//table"


class CustomersPage(BasePage):
    def __init__(self, context):
        BasePage.__init__(self, context)

    def clear_button_click(self):
        """
        Click on Clear Filters Button.

        :return: the Button element
        """
        loop = self.context.async_context.loop
        clear_button = self.find_element(CustomersPageLocators.CLEAR_BUTTON)
        loop.run_until_complete(clear_button.click())
        return clear_button

    def set_search_input(self, search_input):
        """
        Set value to searchInput field.

        :param search_input: input which should be typed into the field
        """
        loop = self.context.async_context.loop
        loop.run_until_complete(self.context.page.type(CustomersPageLocators.SEARCH_INPUT, search_input))

    def set_search_column_drop_down_list_field(self, value):
        """
        Set value to Search Column Drop Down List field.

        :param value: String which should match with one of values visible on the dropdown
        """
        loop = self.context.async_context.loop
        loop.run_until_complete(self.context.page.select_option(CustomersPageLocators.DROP_DOWN, value))

    def set_match_case_checkbox_field(self, value):
        """
        Set Match Case Checkbox field to required value.

        :param value: boolean value of the checkbox status true - checked, false - unchecked
        """
        loop = self.context.async_context.loop
        case_checkbox = self.find_element(CustomersPageLocators.MATCH_CASE)
        checkbox_is_checked = loop.run_until_complete(case_checkbox.is_checked())
        if str(checkbox_is_checked) != value:
            loop.run_until_complete(case_checkbox.click())

    def get_summary_text(self):
        loop = self.context.async_context.loop
        return loop.run_until_complete(self.find_element(CustomersPageLocators.SUMMARY).inner_text())

    def get_search_term_text(self):
        loop = self.context.async_context.loop
        return loop.run_until_complete(self.find_element(CustomersPageLocators.SEARCH_TERM).inner_text())

    def get_search_input_text(self):
        loop = self.context.async_context.loop
        return loop.run_until_complete(self.find_element(CustomersPageLocators.SEARCH_INPUT).input_value())

    def get_search_results_table_text(self):
        loop = self.context.async_context.loop
        return loop.run_until_complete(self.find_element(CustomersPageLocators.TABLE).text_content())

    def open(self):
        application_url = self.context.config.userdata.get("applicationUrl",
                                                      "https://danieldelimata.github.io/sample-page/")
        loop = self.context.async_context.loop
        loop.run_until_complete(self.context.page.goto(application_url))

Repositories

The whole projects you can find on GitHub:

https://github.com/DanielDelimata/sample-python-behave-selenium

GitHub - DanielDelimata/sample-python-behave-playwright

The story was originally created by me, but it may contain parts that were created with AI assistance. My original text has been corrected and partially rephrased by Chat Generative Pre-trained Transformer to improve the language.