Getting started with both Functional Javascript and Nightwatch Testing in Drupal 8.6 or later

In a previous article, I looked into how to get started with Running Tests using Drupal Contributions on top of a local Lando container. I looked a little at the history of Drupal testing and in particular how to get started running PHP Unit tests. 

Types of Drupal Tests

If we look at core Drupal Views module under tests/src we can see the following in the core/tests folder 

Image

Lets look at the types of testing in Drupal and see how they are handled:

PHPUnit

also,

So an example of running the Javascript Functional tests would be

./vendor/bin/phpunit -v -c ./core/phpunit.xml 
./core/modules/system/tests/src/FunctionalJavascript/FrameworkTest.php

Note how the Functional Javascript test should be contained in there own folder within the tests folder.

Lets look at this

<?php
namespace Drupal\Tests\system\FunctionalJavascript\System;
use Drupal\Core\Datetime\Entity\DateFormat;
use Drupal\FunctionalJavascriptTests\WebDriverTestBase;
/**
 * Tests that date formats UI with JavaScript enabled.
 *
 * @group system
 */
class DateFormatTest extends WebDriverTestBase {
  /**
   * {@inheritdoc}
   */
  protected static $modules = ['block'];
  /**
   * {@inheritdoc}
   */
  protected $defaultTheme = 'stark';
  /**
   * {@inheritdoc}
   */
  protected function setUp(): void {
    parent::setUp();
    // Create admin user and log in admin user.
    $this->drupalLogin($this->drupalCreateUser([
      'administer site configuration',
    ]));
    $this->drupalPlaceBlock('local_actions_block');
  }
  /**
   * Tests XSS via date format configuration.
   */
  public function testDateFormatXss() {
    $page = $this->getSession()->getPage();
    $assert = $this->assertSession();
    $date_format = DateFormat::create([
      'id' => 'xss_short',
      'label' => 'XSS format',
      'pattern' => '\<\s\c\r\i\p\t\>\a\l\e\r\t\(\"\X\S\S\")\;\<\/\s\c\r\i\p\t\>',
    ]);
    $date_format->save();
    $this->drupalGet('admin/config/regional/date-time');
    $assert->assertEscaped('<script>alert("XSS");</script>', 'The date format was properly escaped');
    $this->drupalGet('admin/config/regional/date-time/formats/manage/xss_short');
    $assert->assertEscaped('<script>alert("XSS");</script>', 'The date format was properly escaped');
    // Add a new date format with HTML in it.
    $this->drupalGet('admin/config/regional/date-time/formats/add');
    $date_format = '& \<\e\m\>Y\<\/\e\m\>';
    $page->fillField('date_format_pattern', $date_format);
    $assert->waitForText('Displayed as');
    $assert->assertEscaped('<em>' . date("Y") . '</em>');
    $page->fillField('label', 'date_html_pattern');
    // Wait for the machine name ID to be completed.
    $assert->waitForLink('Edit');
    $page->pressButton('Add format');
    $assert->pageTextContains('Custom date format added.');
    $assert->assertEscaped('<em>' . date("Y") . '</em>');
  }
}

The important thing to note here is that Drupal Functional Javascript tests extends WebDriverTestBase. Also running the tests will require the following:

Requirements:

• Base install of Drupal 9 or higher.
• Google Chrome or Chromium.
• chromedriver (tested with 2.45 - you will typically need the version that matches the version of your installed Chrome browser).
• PHP 7.1 or higher.

Here's a more detailed explanation around the history of this.

Drupal 8.3 saw the ability to write tests that interacted with JavaScript when PhantomJS was added to Drupal’s testing capabilities. If you are not familiar with PhantomJS it is a headless web browser that was built on QtWebKit. And then in early 2018, a headless version of Chrome became available, essentially pushing PhantomJS into deprecated status. With Drupal 8.5 we now have the ability to execute WebDriver tests that run on headless Chrome. Honestly, this is a great improvement. The WebDriver API allows for using any client which respects its API (ie: FireFox) and not one-off integrations as was required for PhantomJS.

https://mglaman.dev/blog/running-drupals-functionaljavascript-tests-ddev

Also note:

When ChromeDriver is not running, it will say tests passed when in fact they did not run.

So Matt explains in his blog post how to get Chromium up and running when using a ddev installed docker image. Here is a slightly updated version that replaces the deprecated $DDEV_URL variable

version: '3.6'
services:
  chromedriver:
    container_name: ddev-${DDEV_SITENAME}-chromedriver
    image: drupalci/chromedriver:production
    shm_size: '1gb'
    ulimits:
      core:
        soft: -1
        hard: -1
    ports:
      - "4444:4444"
      - "9515:9515"
    entrypoint:
      - chromedriver
      - "--log-path=/tmp/chromedriver.log"
      - "--verbose"
      - "--allowed-ips="
      - "--allowed-origins=*"
    labels:
    # These labels ensure this service is discoverable by ddev
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT
      com.ddev.app-url: ${DDEV_PRIMARY_URL}
      
  # This links the Chromedriver service to the web service defined
  # in the main docker-compose.yml, allowing applications running
  # in the web service to access the driver at `chromedriver`.
  web:
    links:
      - chromedriver:$DDEV_HOSTNAME

One benefit of running Functional JavaScript tests in php unit is that php can be used to set up and tear down your environment. This is handled differently with Nightwatch where tests are run on Node JS. I understand there is still some dependency on PHP in these scripts, which is one reason why you need a running instance in order to run the tests. Each environment to be tested against can be configured to use a standard ODBC compatible database such as MySQL or even SQLite.

Nightwatch

Nightwatch testing was supported by Drupal back in 2018.

Drupal 8.6 sees the addition of Node.js based functional browser testing with Nightwatch.js. Nightwatch uses the W3C WebDriver API to perform commands and assertions on browser DOM elements in real-time.

https://www.lullabot.com/articles/nightwatch-in-drupal-core

It is possible to use Nightwatch for unit testing JavaScript. In order to unit test your JavaScript it is important that the JavaScript is written in a way that is testable. Writing unit-testable JavaScript is not specific to Drupal and you should look around the Internet for examples of how to do it.

https://www.drupal.org/docs/develop/automated-testing/types-of-tests

This shows the how Drupal core tests are organised by folder

Image

In drupal/core in package.json we also have the following:

that can called after installing any node dependencies using yarn. Node > 18  will be required.

    "test:nightwatch": "cross-env BABEL_ENV=development node -r dotenv-safe/config -r @babel/register ./node_modules/.bin/nightwatch --config ./tests/Drupal/Nightwatch/nightwatch.conf.js",

A gotcha here, is that there is another js dependency  that will need to be installed in order for node to connect with to connect to your locally installed browser. This is done for good reason, as your version on Chromium needs to match your installed browser. 

Update Feb 2024 - Actually you don't need to install chromedriver locally with yarn. Just make sure chromedriver is available and running on port 9515 (see docker ddev docker-compose file above).

Getting Started

Hopefully that's enough background to get you started. Let's attempt to get Nightwatch started using the Drupal Quickstart method that simplifies our setup so we can see how it runs. The steps are as follows:

First of all go and download the test repo that Matt has put together called Drupal Testing WorkShop from Github. 

To run locally we have the following dependencies:

1. php - preferably 8.1 or higher at the time of writing (HomeBrew) brew install php@8.2
2. SQLite - as a mac user I can use HomeBrew brew install sqlite
3. ChromeDriver - again we can use HomeBrew brew install chromedriver
4. Composer - composer.org/Homebrew brew install composer

Note that if running on a Mac, you may need to check your Privacy and Security settings as ChromeDriver is probably not installed from the Mac Store or an identified developer.

Once you cloned or downloaded the repo, we are going to run similar to how Drupal Quickstart runs. 

1. Install any dependencies e.g composer install
2. Launch the using the built in php web server using the same .ht.router.php that is also used by Drupal Quickstart e.g. 
   1. cd web
   2. php -S 127.0.0.1:8080 .ht.router.php

Note the use of an ip address here to make sure we do not use sockets to connect to mysql.

Next we want to copy the web/core/phpunit.xml.dist to web/core/phpunit.xml and configure the following:

SIMPLETEST_BASE_URL=http://127.0.0.1:8080
SIMPLETEST_DB=sqlite://localhost/sites/default/files/db.sqlite
BROWSERTEST_OUTPUT_DIRECTORY=../private/browser_output
MINK_DRIVER_ARGS_WEBDRIVER: '["chrome", {"browserName":"chrome","chromeOptions":{"args":["--disable-gpu", "--no-sandbox"]}}, "http://127.0.0.1:9515"]'

You will also need to create a private directory in you project root e.g. private/browser_output

Make sure that chromedriver is running on the host  e.g. chromedriver

So to run a test like so:

cd web
php ../vendor/bin/phpunit -c core/phpunit.xml core/modules/action

At the time of writing, this test contains a single Function Javascript test. If ChromeDriver is not available this will be skipped.

Ok, so what about Nightwatch?

First of we will need to copy the `web/core/.env.example` to `web/core/.env` e.g.

cd web/core
cp .env.example .env

Then set the following

DRUPAL_TEST_BASE_URL=http://127.0.0.1:8080
DRUPAL_TEST_CHROMEDRIVER_AUTOSTART=false

Make sure ChromeDriver is running and finally:

cd web/core
yarn test:nightwatch --tag nightwatch_example

Here we can see how a custom or contrib module can structure there Nightwatch tests

Image

and this is actually what a Nightwatch test looks like. Note how this test is tagged with nightwatch_example

module.exports = {
  // Tags are how tests are grouped and make it easier to run sets of tests.
  '@tags': ['nightwatch_example', 'workshop'],
  // There isn't a global runner that installs Drupal, like the PHPUnit tests.
  // Each test must implement `before` and install Drupal itself. When calling
  // the `drupalInstall` command, you will want to provide a setup file. This
  // setup file is used to install modules and setup configuration for the test.
  // This setup file is equivelant to setUp in a PHPUnit test.
  before: function (browser) {
    browser.drupalInstall({
      setupFile: 'modules/custom/nightwatch_example/tests/src/Nightwatch/TestSiteInstallTestScript.php',
    });
  },
  // Just as we don't have automated setup, each test must uninstall Drupal.
  after: function (browser) {
    browser
      .drupalUninstall();
  },
  // Now, we may define different tests!
  // There are different commands available, and they are in the
  // tests/Drupal/Nightwatch/Commands directory.
  'Visit a test page': (browser) => {
    browser
      .drupalRelativeURL('/');
    // We execute JavaScript in the browser and assert the result.
    browser.execute(function () {
      // Return the value of our key in Drupal settings.
      return drupalSettings.nightwatchTest;
    }, [], function (result) {
      // Assert the returned value!
      browser.assert.deepStrictEqual(result.value, {
        echo: 'Hello World!',
      });
    })
    .end();
  },
};

Summary

Knowing how testing in Drupal has evolved over time in Drupal hopefully helps us to understand the different types of tests and test frameworks that are available. In this article I have looked at Drupal's Functional Javascript Tests that have both both PHP, Node JS and ChomeDriver as dependencies as well as Drupal's Functional Nightwatch Tests that also has the same dependencies. However, we also looked at how Nightwatch can offer some JS Unit testing which is something that PHP Unit cannot do. 

Furthermore, I have looked at how you setup and run tests in Drupal using these different Testing Frameworks when using Drupal.