Automated accessibility testing in Emulsify

At Four Kitchens, we take accessibility very seriously. It is why we choose tools like Drupal that take it seriously as well. We are always looking for ways to improve our efforts on this front, which is why on a recent project for SDSU Extension, we implemented automated testing in our prototyping tool and Drupal 8 theme, Emulsify. This means that every custom component we now build in Emulsify gets automatically tested and any errors/warnings are flagged to the developer immediately. Awesome! Let’s dig into some of the details.

SDSU Extension

Education, among some other fields, has strict requirements around accessibility. We work hard with these clients to ensure their project meets the expected standard, but we also expect a certain baseline of accessibility on any project we deliver regardless of whether accessibility is a stated goal. For the SDSU Extension project, we needed to meet and/or exceed WCAG 2.0AA standards. Drupal core and many of the contributed modules put such a heavy focus on meeting that standard that an enormous amount of foundational work is done for us. However, when it comes to us designing and coding custom components, it is up to us to also deliver up to that standard. This is where automated testing comes in.

Automated testing using Pa11y

Even with the best intentions, vendors and clients often find themselves putting off best practices (accessibility, performance, documentation) in favor of deadlines. This is why automated testing is so important. With automated tests, accessibility issues are surfaced immediately during the build process and ideally then fixed within the scope of that same deliverable. This ensures the component is not only accessible upon delivery but also avoids the stress of last-minute fixes in the final phase of the project.

There are many tools out there for running automated tests, but one of the most popular is pa11y. Having had good experiences with their testing suite on a recent React project for PRI.org, we decided to use it in Emulsify as well. The results were fantastic! We were able to set a baseline of WCAG 2.0 AA but offer that and many more options in configuration to make it easy to change that for a given project. See below for more technical details.

Technical details

Unlike much of Emulsify Gulp, instead of writing a Gulp command to accomplish a task, we instead created a function to simply run a pa11y test (code). This has been added to our CSS and Pattern Lab Gulp (watch) tasks, so basically it is run whenever a component stylesheet or Twig file is saved. The test is run not just against the component code but the rendered Pattern Lab url that is generated from that component. This means that it tests the final code but also catches visual errors such as color contrast. Besides errors, we also show notices and warnings by default, exposing the handy recommendations that pa11y gives for things that automated tests can’t verify.  However, these settings and many more (including the standard itself) are available to be changed via the configuration file (see here for instructions on creating a local config in Emulsify).

Final note on accessibility testing and the future

Automated testing is a great tool in verifying the accessibility of a project but is only one tool and it does not guarantee a fully accessible product on its own. That said, we are so excited to have this in place for all our Drupal projects moving forward (and to offer it back to the community) as an important step towards building more accessible and responsible products for our clients. Also, we hope to add even more automated tests into our toolchain soon. Here’s to building a more accessible web!