Automate your Drupal accessibility testing with aXe and NightwatchJS

I’m sure we all agree that the accessibility of ALL websites is important. Testing for accessibility (a11y) shouldn’t be limited to Government services. It shouldn’t be something we need to convince non-government clients to set aside extra budget for. It certainly shouldn’t be left as a pre-launch checklist item that only gets the proper attention if the allocated budget and timeframe hasn’t been swallowed up by some other feature.

Testing each new component or feature against an a11y checker, as it’s being developed, takes a small amount of time. Especially when compared to the budget required to check and correct an entire website before launch -- for the very first time. Remembering to run such tests after a components initial development is one thing. Remembering to re-check later down the line when a few changes and possible regressions have gone through is another. Our brains can only do so much, so why not let the nice, clever computer help out?

NightwatchJS

NightwatchJS is going to be included in Drupal 8.6.x, with some great Drupal specific commands to make functional javascript testing in Drupal super easy. It's early days so the documentation is still being formed.  But we don't have to wait for 8.6.x to start using Nightwatch, especially when we can test interactions against out living Styleguide rather than booting up Drupal.

So lets add it to our build tools;

$ npm install nightwatch

and create a basic nightwatch.json file;

{
  "src_folders": [
    "app/themes/my_theme/src/",
    "app/modules/custom/"
  ],
  "output_folder": "build/logs/nightwatch",
  "test_settings": {
    "default": {
      "filter": "**/tests/*.js",
      "launch_url": "http://127.0.0.1",
      "selenium_host": "127.0.0.1",
      "selenium_port": "4444",
      "screenshots": {
        "enabled": true,
        "on_failure": true,
        "on_error": true,
        "path": "build/logs/nightwatch"
      },
      "desiredCapabilities": {
        "browserName": "chrome"
      }
    }
  }
}

We're pointing to our theme and custom modules as the source of our JS tests as we like to keep the tests close to the original JS. Our test settings are largely based on the Docker setup described below, with the addition of the 'filter' setting which searches the source for .js files inside a tests directory.

A test could be as simple as checking for an attribute, like the following example;

/**
 * @file responsiveTableTest.js.
 */

module.exports = {
  'Responsive tables setup': (browser) => {
    browser
      .url(`${browser.launch_url}/styleguide/item-6-10.html?hideAll`)
      .pause(1000);
    browser.expect.element('td').to.have.attribute('data-label');
    browser.end();
  },
};

Which launches the Styleguides table component, waits a beat for the JS to initiate then checks that our td elements have the data-label that our JS added. Or is could be much more complex.

aXe: the Accessibility engine

aXe is a really nice tool for doing basic accessibility checks, and the Nightwatch Axe-Core node module integrates aXe with Nightwatch so we can include accessibility testing within our functional JS tests without needing to write out the rules ourself. Even if you don't write any component specific tests with your Nightwatch setup, including this one accessibility test will give you basic coverage.

$ npm install nightwatch-axe-core

Then we edit our nightwatch.json file to include the custom_commands_path;

{
  "src_folders": ["app/themes/previousnext_d8_theme/src/"],
  "output_folder": "build/logs/nightwatch",
  "custom_commands_path": ["./node_modules/nightwatch-axe-core/commands"],
 
  "test_settings": {
     ...
  }
}

Create a global aXe configuration file, called axe.conf.js in the project root;

module.exports = {
  context: '.kss-modifier__example',
  options: {
    runOnly: {
      type: 'tag',
      values: ['wcag2a', 'wcag2aa'],
    },
    verbose: true,
    timeout: 2000,
  }
};

Here we're configuring aXe core to check for wcag2a and wcag2aa, for anything inside the .kss-modifier__example selector of our Styleguide.

If we want to exclude a selector, instead of the '.kss-modifier__example' selector, we pass an include/exclude object { include: [['.kss-modifier__example']], exclude: [['.advertising']] }.

Then write a test to do the accessibility check;

/**
 * @file Run Axe tests with Nightwatch.
 */

module.exports = {
  'Accessibility test': (browser) => {
    browser
      .url(`${browser.launch_url}/styleguide/section-6.html`)
      .pause(1000)
      .axe()
      .end();
  },
};

Running this test with .axe() will check all of our components and tell us if it's found any accessibility issues. It'll also fail a build, so when hooked up with something like CircleCI, we know our Pull Requests will fail.

If you only add one test, add one like this. Hopefully once you get started writing Nightwatch tests you'll see how easy it is and eventually add more :)

You can include the accessibility test within another functional test too, for example a modal component. You'll want to test it opens and closes ok, but once it's open it might have some accessibility issues that the overall check couldn't test for. So we want to re-run the accessibility command once it's open;

/**
 * @file dialogTest.js
 */

const trigger = '#example-dialog-toggle';
const dialog = '.js-dialog';

module.exports = {
  'Dialog opens': (browser) => {
    browser
      .url(`${browser.launch_url}/styleguide/item-6-18.html`)
      .pause(1000);
    browser.click(trigger);
    browser.expect.element(dialog).to.be.visible.after(1000);
    browser.assert.attributeEquals(dialog, 'aria-hidden', 'false');
    browser.axe();
    browser.end();
  },
};

Docker

As mentioned above this all needs a little docker & selenium setup too. Selenium has docs for adding an image to Docker, but the setup basically looks like this;

@file docker-compose.yml

services:
  app:
    [general docker image stuff...]

  selenium:
    image: selenium/standalone-chrome
    network_mode: service:app
    volumes:
      - /dev/shm:/dev/shm

Then depending on what other CI tools you're using you may need some extra config. For instance, to get this running on CircleCI, we need to tell it about the Selenium image too;

@file .circleci/config.yml

jobs:
  test:
    docker:
     [other docker images...]
     - image: selenium/standalone-chrome

If you're not using docker or any CI tools and just want to test this stuff locally, there's a node module for adding the selenium-webdriver but I haven't tested it out with Nightwatch.

Don’t forget the manual checks!

There’s a lot more to accessibility testing than just these kinds of automated tests. A layer of manual testing will always be required to ensure a website is truly accessible. But automating the grunt work of running a checklist against a page is one very nice step towards an accessible internet.

UPDATED: 12/02/2021 with new nightwatch-axe-core npm package, rather than the initial nightwatch-accessibility one. This simplifies the setup somewhat and resolves some performance issues.