Journey through Cucumber land: introduction to the course of Cucumber
Welcome to my series of blog posts on the Cucumber framework! As a software tester, I’ve had the opportunity to work with a variety of testing frameworks over the years, but there’s one that always seems to spark a lot of debate: Cucumber. Some testers swear by it, while others swear at it.
From this post you will learn what is Cucumber, why people are using it, how do tests look like in Cucumber and what are the most important parts of every testing project based on Cucumber. The more detailed information will be covered by subsequent posts devoted to specific programming languages.
In these blog posts, I’ll be sharing my own experiences working with Cucumber and discussing the pros and cons of using this framework. We’ll take a deep dive into its features, explore some examples of how it’s being used, and talk about some common issues you may encounter when working with it. Whether you’re a seasoned tester or new to the field, these posts will give you a comprehensive understanding of what Cucumber is, how it works, and whether it’s the right choice for your testing needs.
WHAT IS CUCUMBER?
Cucumber is a testing tool that uses semi-natural language Gherkin for describing tests. It allows writing tests that anybody can understand, regardless of their technical knowledge, especially people with a domain knowledge.
The Cucumber origins were in the Ruby language ecosystem, but nowadays, it is rather a general class of tools. Sometimes to distinguish versions of Cucumber they are suffixed e.g. Cucumber-ruby, Cucumber-JVM and so on.
Cucumber is commonly used for high level tests, especially for acceptance level tests. It binds the description in natural language with the underlying code. In this way it is easy to achieve traceability, which means that we can easily say which tests cover a given requirement, and the opposite, which requirement is connected with given testing code.
WHY TO USE CUCUMBER?
At the very beginning, it is good to ask the fundamental question: “why?”. Why do we need it? Why is it used in testing? There are opinions that this framework is useless, or even that it is so bad that it is the reason for failures.
I will try to answer these opinions. They are sometimes partially reasonable. When a talented and experienced person, an author of books about testing, an award winner, is trying to introduce this framework into the project, and it appears to be a failure, then it may mean something. Right? Well, partially, yes. It means that this tool is not as miraculous as one could think.
To look closer at the problem, we have to think about what promise Cucumber makes and if this promise is fulfilled, and if its cost is worth it. These three questions are crucial.
In my opinion, Cucumber (or other BDD tool) makes a promise of making the communication between the business side and people implementing tests easier. If we think about it, we see that:
In my opinion Cucumber (or other BDD tool) makes promise of making easier the communication between business side and people implementing tests. If we think about it, we see that:
- Such communication is not problematic everywhere, so it does not always need to be made easier.
- Such communication is not equally needed everywhere, so again it does not always need to be made easier.
It means that there are projects where Cucumber (or more generally BDD) is more useful and there are projects where it is really unnecessary language layer for tests. First advice: When you do not need it, do not use it!
If you need it, then you need to communicate. Communication is not something that can be introduced just by one tool. Communication is sometimes difficult and demanding. It is about people, not tools. Tools can be helpful to some extent, but they will not solve problems which are in people’s mentality.
When there is a need for communication and someone comes with a wonderful tool, it seems to everyone that the tool will solve the problem. I can answer: No, it will not.
I have seen an interesting project where an advanced BDD framework was implemented using the Screenplay Pattern, and when I started asking, it turned out that I was the first person to show feature files to the product owner. Can you be surprised that in such situations there could appear problems?
And for that, the tester’s will is not enough, the product owner’s will is not enough, the stakeholders’ will is not enough. It takes the common will and engagement of everyone, and that sometimes can be difficult.
Second advice: If you need it, then you have to really communicate with people. The tool is not a replacement of communication.
HOW TO USE CUCUMBER IN TESTING?
HOW DOES GHERKIN CODE LOOK LIKE?
Gherkin is a language used to describe tests in a semi-natural way, which allows anyone to understand them, even those without technical knowledge. It is commonly used to write high-level tests, especially acceptance level tests. Gherkin is used as the language for describing tests in Cucumber, a widely used testing tool. Although Cucumber was originally developed in the Ruby language ecosystem, it has since become a general class of tools and can now be found in various versions, such as Cucumber-JVM, SpecFlow and more.
The Gherkin language has a simple syntax and a limited number of keywords: Feature, Scenario, Scenario Outline, Given, When, Then, And, But, *, Background, Examples, “””, |, @, and #. Gherkin is line-oriented, which means that a single statement must be on one line only.
In Gherkin, a feature corresponds to one file and is identified by the feature extension. One feature file can contain several scenarios, which are identified by the Scenario or Scenario Outline keywords. A scenario defines a complete test and consists of steps, which are lines that start with a keyword, followed by a natural language description. Steps are reusable, meaning they can appear in multiple scenarios and files, with the underlying code defined only once.
Cucumber uses files named feature files and their extension is .feature
Let us list all Gherkin keywords with synonyms:
Feature
This keyword begins the file. It is some kind of the title of the whole document. The text after this word is not connected to the code. It is good idea to properly describe the part of software which is covered by the file. Please remember that one feature file can contain many scenarios. Next lines after this keyword can contain the general description of feature, and they are not connected with the executable code. In ISTQB terms feature can be considered as synonym of “test condition”.
Scenario
This keyword begins scenario and follows the title of scenario. Next lines should begin with *, Given, When, Then, And, But because they define test steps.
Given, When, Then
Three basic keywords beginning steps description. Given defines steps of the prerequisites type (what should happen before the proper test),When defines test action and Then specifies the expected result.
*, And, But
Synonyms of the above basic keywords. Most general * allows to describe the whole scenario without distinguishing between types of steps. And and But allows avoiding of repeating of the same keyword at beginning of multiple lines.
Scenario outline
This keyword is similar to Scenario but it starts its parametrized version. After Scenario outline there must be placed Examples keyword and a table with parameters values. After Scenario they cannot be placed. Scenario is executed only once. Scenario outline is executed as many times as the number of rows describing parameters' values.
Examples
This keyword starts the part with a table with parameters. The cells in such a table are separated with |.
Background
Sometimes we want to start multiple scenarios and scenarios outline with several exactly the same steps (usually Given type). To avoid repeating them we can place them in Background section.
It is considered a best practice to quote the requirements as a comment before the scenarios in Gherkin. The example above demonstrates how Gherkin can be used to describe a scenario where a user clears search criteria on a customer page, with specific acceptance criteria and examples provided.
Feature: F01 Searching — Clearing of searching criteria
User story:
* As a user of Customers page
* I want to be able to clear searching criteria
* in order to quickly type new criteria
Acceptance criteria:
* After clearing, search criteria and summary should be as in the very beginning.
Scenario Outline: Clearing of searching criteria
Given the user is on the page
When the user enters the value "<search>" in the text-input
And the user selects value "<column>" in the drop-down
And the user sets case sensitivity switch to "<case>"
And the user clears filters
Then the user should see that search criteria are cleared
And the user should see that the search result summary is as in the very beginning
Examples:
| Search | Column | Case |
| Alice | Name | True |
| alice | Name | False |ASSUMPTIONS FOR PROJECTS
In this part I am going to start a series of post about samples which allow quickly start working with Cucumber in various programming languages and frameworks.
Obviously these projects will be most beneficial for beginners or intermediate users (e.g. who want to switch to another language) in test automation. Advanced users all information collected here already know very well.
My assumptions:
- Each project should implement BDD (pure Cucumber, Serenity BDD).
- Each project should implement UI automation (Selenium, Puppeteer, Playwright).
- Each project should implement object approach to UI (Page Object Pattern, Screenplay Pattern).
- Each project should include user-friendly reporting with UI screenshots on failure like e.g. Allure.
- Each project should be started from CI like e.g. Jenkins.
CHOOSING THE LANGUAGE
Before we dive in details let us general look on the technologies and think about the choice of the programming languages for our project.
The execution time of automated tests can vary depending on several factors such as the specific test framework, hardware, and the complexity of the tests themselves. However, generally speaking, it is considered that compiled languages tend to have faster execution times compared to interpreted languages. However, in case of testing it turns out that web automation is much faster when using JavaScript.
It’s worth noting that the execution time of automated tests is not the only factor to consider when choosing a language for a project. Other factors such as developer expertise, the size of the community, and the availability of libraries and frameworks should also be taken into account.
The popularity of programming languages in automated testing can vary depending on the source and the criteria used for ranking. However, based on recent surveys and research, the top 5 popular programming languages for automated testing are:
- JavaScript: JavaScript is a popular choice for automated testing due to its flexibility, and the availability of a wide range of test frameworks and libraries such as Jest, Mocha, and Jasmine.
- Python: Python is another popular choice for automated testing due to its simplicity, readability, and the availability of a wide range of test frameworks such as pytest and unittest.
- Java: Java is a popular choice for automated testing due to its wide adoption in the industry and the availability of a wide range of test frameworks and tools such as JUnit, TestNG, and Selenium.
- C#: C# is a popular choice for automated testing in the .NET ecosystem, and it is widely used in combination with the Selenium WebDriver library.
- Ruby: Ruby is a popular choice for automated testing due to its simplicity, readability, and the availability of a wide range of test frameworks such as rspec and minitest.
Honorable mention: Go, F#.
List of projects, I am going to present:
Java
- Java + Selenium + JUnit5 + Page Object Pattern + Allure
- Java + Selenium + TestNG + Page Object Pattern + Allure
- Java + Playwright + JUnit4 + Page Object Pattern + Allure
- Java + Playwright + JUnit5 + Page Object Pattern + Allure
- Java + Selenium + Serenity BDD + Screenplay Pattern + Allure
- Java + Selenium + Serenity BDD + Page Object Pattern + Allure
C#
- C# + Selenium + SpecFlow + NUnit + Page Object Pattern + Allure
- C# + Selenium + SpecFlow + MSTest + Page Object Pattern + Allure
- C# + Selenium + SpecFlow + XUnit + Page Object Pattern + Allure
Python
- Python + Selenium + Behave + Cucumber + Page Object Pattern + Allure
JavaScript based languages
- Typescript + Selenium + CucumberJS + Page Object Pattern + Allure
- Typescript + Playwright + CucumberJS + Page Object Pattern + Allure
Other JVM based languages
- Kotlin + Selenium + Serenity BDD + Screenplay Pattern + Allure
- Scala + Selenium + JUnit4 + Page Object Pattern + Allure
Of course this list could be much longer. There are other programming languages which are supported by Selenium, Cucumber and Allure.
Survey of Selenium users in 2020 showed that the most popular languages for Selenium based testing are:
- Java (50%)
- Python (18.06%),
- JavaScript (13.89%),
- C# (12.5%),
- Ruby (2.78%),
- Groovy (1.39%),
- PHP (1.39%).
WHAT DOES SUCH A PROJECT CONSIST OF?
There are three basic elements of Cucumber testing automation project for web based application.
- Runner — the code responsible for running tests, collecting results, taking and saving screenshots etc.
- Page object representation — the code defining elements of pages
- Step definition (so-called glue) — the code assigned to specific steps of feature files
The general schema is simple. Runner executes tests. For each of subsequent lines of feature files, the corresponding code from the glue code is executed. The glue code operates on elements defined in the page object representation.
In next parts of this course we will see how similar are these parts between programming languages.
The story was originally created by me, but it may contain parts that were created with AI assistance. My original text has been corrected and partially rephrased by Chat Generative Pre-trained Transformer to improve the language.
