It’s crucial to know when to use Cucumber and BDD to test APIs, and when NOT to.
It’s a controversial topic and many have strong views on this, so in this article we’re going to take a good look – using real world examples – of when to use BDD, Cucumber, feature files and Gherkin to define our automated API tests, and when to simply use traditional test case code.
For those of you unfamiliar with this approach, Cucumber uses Gherkin in a Given – When – Then syntax to define our tests using natural language, for example:
Scenario: Check if user details can be successfully retrieved
Given a userid "joebloggs47"
When the users information is retrieved
Then the response code is "OK"
And the response user forename is "Joe"
And the response user surname is "bloggs"
Each of these steps has a corresponding code method which is executed when the test is run. Typically the Given steps have code that set up prerequisites and test data, the When calls the API, and the Then statements verify that the correct data has been returned.
The main advantage of Cucumber is that the Gherkin steps can be easily understood by non-technical stakeholders such as Business Analysts, Product Owners and Test Managers so they can quickly grasp the coverage of the automated tests.
The disadvantage of Cucumber is that it adds extra layers of complexity and maintenance to your test automation framework, making them more difficult and time consuming to create and maintain.
So are the benefits of Cucumber worth the extra burden? Well, that depends!
In my opinion the use of Cucumber and Gherkin for testing APIs is worthwhile when they express the core business rules of an organisation, but not the basic CRUD (create, read, update, delete) functionality an application needs to function.
For example the scenario above checks that a user’s details can be retrieved from the users API, and in the same feature file you might expect to also see scenarios to check that the API correctly creates, updates and deletes a user. But does the Product Owner really need to see this expressed in Gherkin so that they know the application can read and write data to the database? Surely that is such a basic function of the system that it’s not really necessary to have this written in Gherkin.
Plus, as more and more software development take a microservices approach to systems architecture the number of such basic APIs expands greatly, and I believe using Cucumber to test them becomes a large and unnecessary burden which provides little value to the business or technical teams.
However there are also situations where I would recommend using Cucumber to test APIs. As I mentioned above, this are where they define and test the business rules or logic of an organisation. Let’s look at an example from real life – a project I have actually worked on.
The application is a Mortgage Origination website, where a user applies for a mortgage online by entering (via a bank employee) details of the property they wish to purchase, how much they want to borrow, their personal details including salary / income, and so on.
At the end of the process they submit their mortgage application and will get back either an ‘Accept’, a ‘Decline’ or a ‘Refer’ (a ‘maybe’, with further discussion needed). The decision-making process is driven by a Decision Engine API which takes the information about each application and makes a decision on whether a loan should be approved.
This algorithm is obviously a core business process, and creating an automated test suite to rigorously and repeatedly check that the API is providing the expected results would be high on my list of priorities. Expressing these tests and rules as Gherkin makes perfect sense too because the Business Analysts, Product Owners, Underwriters and other stakeholders in the organisation need to be able to read them to make sure they understand how the Decision Engine is being tested, what is the coverage, and add or modify rules as required.
Let’s take a look at a couple of examples:
Scenario: Decline if applicant age at end of loan term is greater than 70 years old
Given a standard residential mortgage application
But the mortgage term is "20" years
And the applicant age is "51" years old
When the mortgage application is submitted
Then the response status is "OK"
And the decision is "Decline"
And the decision reasons include code "D435" "Applicant exceeds maximum age at end of term"
Scenario: Decline if buy-to-let loan amount exceeds maximum threshold of 90% of property value
Given a standard buy-to-let mortgage application
but the property value is "100,000" dollars
And the loan amount is "90,001" dollars
When the mortgage application is submitted
Then the response status is "OK"
And the decision is "Decline"
And the decision reasons include code "A203" "BTL loan-to-value exceeds 90%"
I hope you can see the value to the business of having tests that are understandable to non-technical stakeholders, and also that simple CRUD tests don’t need to be expressed using Cucumber and Gherkin because that provides little value to the business and technical teams. If you disagree (or agree!), please leave a comment below.