Skip to main content

Behat

https://github.com/Behat/Behat

https://docs.behat.org/en/latest/user_guide.html

A php framework for auto-testing your business expectations.

Vortex comes with pre-configured Behat profiles for Drupal projects.

Usage​

Since Behat requires a running Drupal instance, it can only be run within container environment.

ahoy test-bdd

Running all tests in the file​

ahoy test-bdd tests/behat/features/my.feature

Running tagged tests with @group_name tag

ahoy test-bdd --tags=group_name

Configuration​

See configuration reference.

All global configuration takes place in the behat.yml file.

By default, Behat will run all tests defined in tests/behat/features directory.

Adding or removing test targets:

default:
  suites:
    default:
      paths:
        - '%paths.base%/web/modules/custom/mymodule/tests/behat/features'

Contexts and extensions​

The configuration uses the following contexts and extensions:

  • Drupal Extension: The Drupal Extension is an integration layer between Behat, Mink Extension, and Drupal. It provides step definitions for common testing scenarios specific to Drupal sites.
  • Behat Screenshot Extension: Behat extension and step definitions to create HTML and image screenshots on demand or when tests fail.
  • Behat Progress Fail Output Extension: Behat output formatter to show progress as TAP and fails inline.
  • Behat Steps: Collection of Behat steps for Drupal

Reports and screenshots​

Behat is configured to generate test run reports in JUnit format stored in .logs/test_results/behat.

Screenshots for failed tests or purposely made screenshots are stored in .logs/screenshots directory.

Both test results and screenshots are stored as artifacts when run in continuous integration pipelines.

Profiles​

Behat runs with default profile defined in the configuration file: this runs all tests not tagged with @skipped tag.

There are also p1 and p2 profiles defined that run tests not tagged with @skipped and tagged with @p1 and @p2 tags respectively. These profiles are used in continuous integration pipeline to run tests in parallel if parallel runners number is greater than 1. Behat will run tests using @pX tags on tests, where X is the runner index starting from 0.

In the example below, if there is one runner, Behat will run all scenarios, but if there are 2 runners - Behat will run only scenarios with @p0 tag on the first runner and scenarios with @p1 tag on the second runner.

@homepage @smoke
Feature: Homepage

  Ensure that homepage is displayed as expected.

  @api @p0
  Scenario: Anonymous user visits homepage
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

  @api @javascript @p1
  Scenario: Anonymous user visits homepage
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

The profile can be overridden using $VORTEX_CI_BEHAT_PROFILE environment variable set in the continuous integration pipeline configuration.

Writing tests​

When writing tests, it is recommended to use the following structure:

@tag_describing_feature_group
Feature: Short feature description

  As a site owner
  I want to make sure that my feature does what it is supposed to do
  So that I can be sure that my site is working as expected

  @api
  Scenario: Anonymous user uses a feature
    Given I go to the homepage
    And I should be in the "<front>" path
    Then I save screenshot

  @api @javascript
  Scenario: Anonymous user uses a feature with AJAX
    Given I go to the homepage
    And I click ".button" element
    Then I save screenshot

Use @api tag for tests that do not require JavaScript and @api @javascript for tests that require JavaScript.

tip

JavaScript tests are slow. Try using as few JavaScript tests as possible to keep the test run time low.

FeatureContext.php​

The FeatureContext.php file is a Behat custom context file that is loaded by default. This is where custom step definitions and hooks should be placed.

It is recommended to use traits to organize step definitions and hooks and include those traits in the FeatureContext.php file. There are already several traits included from the Behat Steps package.

There are also example tests in the tests/behat/features directory that can be used as a starting point for writing tests. These tests define @smoke tests for the homepage and login page as well as tests for contributed modules.

Connecting to the browser​

For @javascript tests, Behat uses Selenium to connect to the browser. The browser runs within a container and can be observed from the host machine by using a browser thanks to the VNC server running in the container and noVNC.

  1. Get the link for Selenium VNC URL on host:
ahoy info
  1. Click on the link next to Selenium VNC URL on host to open the browser.
tip

When running tests, you can add And I wait for 60 seconds step to suspend the test execution for a minute and interact with the headless browser in a container through a browser on your host machine.

behat-novnc.gif

Ignoring fail in continuous integration pipeline​

This tool runs in continuous integration pipeline by default and fails the build if there are any violations.

Set VORTEX_CI_BEHAT_IGNORE_FAILURE environment variable to 1 to ignore failures. The tool will still run and report violations, if any.

note

Due to the size of your website and the available continuous integration runner resources, Behat tests may occasionally fail.

To prevent the pipeline from failing as a result, any failed Behat test will be retried automatically.