Introducing Accessibility scanning using Selenium WebDriver

Ever wanted to do accessibility scanning in your WebDriver tests in Java? Introducing webdriver-accessibility, a Java library that helps you run accessibility audits using Selenium WebDriver and Google chrome accessibility developer tools. It uses Google chrome accessibility developer tools to run accessibility audits and then parses the report into a logical Java object.  webdriver-accessibility marks elements which violate accessibility rules on your webpage and takes a screenshots for better reporting. The output result of this tool contains accessibility errors, warnings for you to assert in your tests along with screenshot pointing to problematic elements on your web page.

How to use this library? It’s very simple. Launch your site using WebDriver, and pass along WebDriver object to the tool.

AccessibilityScanner scanner = new AccessibilityScanner(driver);
Map<String, Object> audit_report = scanner.runAccessibilityAudit();

And you could assert like below in your test,

if (audit_report.containsKey("error")) {
 List<Result> errors = (List<Result>) audit_report.get("error");
 assertThat("No accessibility errors expected", errors.size(),equalTo(0));
}

Output Map<String,Object> has following keys,

 /** @type {List<Result>} */
error, //contains all errors
/** @type {List<Result>} */
warning, //contains all warnings
/** @type {String} */
plain_report, //contains plain report
/** @type {byte[]} */
screenshot, //contains screenshot for reporting

Result object is made of following,

 /** @type {String} */
  rule, //contains specific rule information
  /** @type {List<String>} */
  elements, //contains all element locators with errors/warnings
  /** @type {String} */
  information_link, //link to [GoogleChrome accessibility-developer-tools audit rules][3] wiki for more details

If you want specific details of the error, you could scan through the List of errors like below.

List<Result> errors = (List<Result>) audit_report.get("error"); 
for (Result error : errors) {
 log.info(error.getRule());//e.g. AX_TEXT_01
 log.info(error.getUrl());//e.g. Url explaining the error
 for (String element : error.getElements()) //violated elements
  log.info(element);//e.g. #myForm > P > INPUT
}

Similarly you could scan all warnings for details. Just get the “warning” key from the audit_report map. Rest remains the same.

List<Result> warnings = (List<Result>) audit_report.get("warning");

Pros:

  • Light weight library
  • Easily extend your Selenium WebDriver suite to run accessibility audits
  • Use any version of Selenium WebDriver
  • No hard assertions, they are totally up to the user

Cons:

  • Selenium WebDriver uses browsers native APIs to simulate user interactions with the web app. webdriver-accessibility injects Java scripts to run accessibility audits and marking elements on the webpage. Injecting external Java script libraries pollutes global scope of the page under test. Therefore keep your functional and accessibility tests separate.

Here is a sample cucumber report of the accessibility audit that I ran on my sample site. In my sample site, I deliberately introduced some accessibility errors and warnings to demo how the tool captures them and highlights corresponding elements in the screenshot. In the screenshot, you can notice that input text boxes violated missing label rule and are therefore marked with red border. There are in fact 24 violations of missing label rule, in my example below however GoogleChrome accessibility-developer-tools at most provides 5 errors/warnings of each type. Therefore only 5 input boxes with this violation are marked with red border. Also the small pizza image violated missing ALT attribute rule as a warning and therefore marked as yellow.

report

I have hosted the project on github here. I am not an accessibility expert and open for suggestions to make this tool better. If you want to contribute, fork away. Feedback appreciated!

Advertisements