Pros and Cons of 7 Top Web Accessibility Testing Tools

How to ensure detect web accessibility errors and ensure compliance with current Web Content Accessibility Guidelines (WCAG 2.2)?

Many automatic testing and scanning tools are available to help fast-track the process and uncover common, but frequently overlooked errors. The challenge: these tools vary widely in terms of types of errors they detect, usability, and the degree to which they can help ensure ADA Accessibility compliance. 

The following is an evaluation of seven of the most popular tools or extensions currently available, and what we consider to be the key advantages and drawbacks to each. 

Keep in mind that there are inherent limitations to all automated tools. While automated web accessibility testing tools provide a step in the right direction and an essential starting point, in order to accurately and thoroughly audit a website for accessibility, manual testing and intervention is also required.

1. BrowserStack

Designed as a tool for testing and monitoring web accessibility, BrowserStack features fast scanning capabilities, real screen readers, and automatic daily monitoring. Recent milestones include: detection of more than 8 million accessibility issues, more than 25,000 web pages.  

Pros

1. The interface clearly shows which manual test reports have been conducted and when, along with a number of other high-level issues. 
2. On a report, the summary of the issues, and ability to nail down relevant details is l very clean and user friendly.
3. The screen reader option is a huge plus that not many other services, if any, would have. This feature is among one of the most significant advantages. 
4. The manual "assisted test," which allows for the viewing of keyboard tabs in order, and highlighted, is a strong advantage.

Cons

1. The tool is not designed to run through and entire site, and check all links/pages. 
2. It’s not clear that "website scanner" provides the ability to generate a report from a site. It’s an important capability that should appear high up on the list. 
3. The "how to fix '' is lacking in details, such as 4.5:1 ratios for color contrast issues, color recommendations, etc.
4. When looking at errors on the page and selecting the item, it would be helpful if the tool highlighted the actual item that is in question.
    

2. DubBot

DubBot provides automated solutions designed to simplify website management, with tools that help organizations to detect vulnerabilities that include web accessibility, web governance, SEO, detection of misspellings, broken links, and accessibility task management.   

Pros

• The DubBot dashboard is amazingly useful. The ability to not only see accessibility issues, but also broken links, governance red flags, and spelling mistakes is helpful at getting a high-level site summary. 
• DubBot provides the ability to filter by accessibility error types, view all errors on a single page, site or group, as well as create tasks, mark errors as resolved, or ignore errors on a site or group basis. 
• DubBot's support team extremely responsive, offering efficient access to subject matter experts who provide accurate and authoritative responses to questions and concerns. 
• DubBot also allows users to configure reporting, that enables up-to-date information on any website's current accessibility state. Reporting includes an error summary and can be sent daily or weekly, (depending on configuration options). This is a very helpful tool during agile development.
• DubBot runs on axe, which is helpful in leveraging automated accessibility testing due to an enterprise ready dashboard that has many configuration options. 

Cons

• Unlike a few of the other tools, DubBot does not have a Chrome plugin, which means that a website's needs to be compatible with the DubBot crawler tool. 
• Integrations with JIRA or other reporting/ticketing software would be helpful. 

3. WAVE

WAVE is a web accessibility evaluation tool developed by WebAIM for evaluating the accessibility of web contents. The tool is available for WAVE Firefox and Chrome extension or by entering the web page address (URL). 

Pros

• The WAVE Chrome and Firefox extensions allow you to review the accessibility of your site directly within the Chrome and Firefox browsers. 
• No information is sent to the WAVE server. 
• WAVE will show what errors are indicated and make some recommendations on how to correct these issues. 

Cons

• WAVE provides an overlay of icons to indicate any potential accessibility issues however it gets confused by absolute positioning and doesn’t show any other information about the element. 
• The overlay overlay of icons becomes further problematic when you are managing an element that is not visible on the page. It will take a bit of work with dev tools to determine which element is causing the issue. 

4. axe

axe is developed by Deque Systems for Chrome. axe analyzes a page for accessibility defects in just a few moments from a Chrome browser. The final output will show a list of accessibility issues with a link that provides the code that caused the issues and recommendations for fixing the issue. 

Pros

• axe rates the impact of accessibility issues differently than WAVE. It does not use the “alerts” category, rather all errors are given a severity level. Using this system prompts more errors to be fixed than if some errors are labeled as alerts implying they are not important. 
• axe will list elements that are not necessarily accessibility errors but warrant review based on the context of the element.

Cons

• axe will point out some color contrast issues that still need to be tested manually. 
• While not overtly a drawback with this tool, it does require a bit more knowledge by the user to understand the process for testing. 

5. Siteimprove

Siteimprove is an extremely popular tool especially within the areas of government and higher education. Siteimprove is available as a Chrome extension. It provides visual feedback concerning web content and highlights noncompliance issues visually on the page. 

Pros

• Siteimprove analysis is done entirely within the Chrome browser, allowing secure evaluation of password-protected or non-public pages, multi-step forms, and dynamic content.
• Siteimprove identifies accessibility noncompliance issues within the framework of WCAG, and points out instances where more than one guideline is being being violated. 

Cons

• The format in which the errors are listed can be confusing for those not familiar with WCAG standards for accessibility. 
• Several areas indicate manual testing is needed. As with axe, while not overtly a drawback for this tool, it does require a bit more knowledge by the user to understand the process for testing. 

6. Lighthouse

Lighthouse is developed by Google. If you are running an up-to-date version of Chrome, you most likely already have Lighthouse since it is built into devtools. Open devtools and go to the ‘Audits’ tab, then the ‘Perform an Audit’ button and you will be given a list of the audits Lighthouse can perform. 

Pros

• Built into Chrome devtools. 
• Can analyze more than just accessibility. Also analyzes factors such as SEO, performance and best practices. 

Cons

• Not as robust as some of the other tools listed. Lighthouse does use axe-core for accessibility audit, but it does not run the full set of about 70 tests that the axe extension does. 
• It is possible to get a 100% accessibility rating on Lighthouse and still have a site which is largely not accessible. 

7. Tenon.io

Tenon.io differs from the other tools listed here since it is a web service and not an extension. Use of Tenon.io simply requires a link or paste in the markup of the UI and it will generate a report. There are also many ways to integrate Tenon, but this is a paid service. 

Pros

• Does not require anything to be downloaded or installed on your computer. 

Cons

• Since Tenon works by passing the URL to a browser, your site/UI need to be publicly available. 
• Can be slower than the other tools covered here. 

Automated tools can serve to significantly streamline the evaluation of a website for ADA accessibility and detect important instances of noncompliance. Manual follow-up is always recommended, however, for an accurate analysis and assurance of WCAG compliance.

As one, small example of the limitation of automated accessibility tools: automated tools check to ensure the presence of alt text, but if the alt tag is inaccurate such as alt="cow" for a picture of a chicken, the automated tool will not detect it—although some will point an issue that needs to be verified.  

Download Promet's Accessibility Checklist for Content Editors.

Looking for help in determining the automated tool that’s appropriate for your needs, or consultation concerning the process of ensuring WCAG 2.2 compliance? We can help! That's what we do! Let's talk!