Skip to Main Content
android and woman facing each other to reflect human vs automated wcag testing

Human vs Automated WCAG Website Auditing for ADA Compliance

David Gibson

In today’s world, websites are essential for everyday life, however most websites are still not inaccessible to many seniors and people with disabilities. As such, they also violate the Americans With Disabilities Act.

The extent of these barriers are revealed in the latest report from the WebAIM team at Utah State (makers of the popular WAVE web page accessibility tester). The latest WebAIM Million report revealed that 96.3% of the top million homepages failed ADA compliance testing. And these didn't just have one or two missing barriers. Each had an average of 50.0 violations. 

So. Assume your website is not ADA compliant.

Why Is Website Accessibility Important?

Legal - Almost all websites are covered by either the ADA or Section 508 of the Rehabilitation Act. And in addition to legitimate plaintiffs and advocacy groups actively defending web accessibility rights, there is also a throng of serial plaintiffs and attorneys seeking easy settlements.  

DEI - Diversity, Equity and INCLUSION, includes the rights of people with disabilities to digital inclusion. Nothing has highlighted the value of digital access and the disparity of access better than the pandemic, when the world turned inward to the Web for nearly everything. 

Opportunity - Not only are there 61M people with a disability in the US, but also consider the 71M boomers who carry $548B in discretionary spending and who also share similar challenges with vision, hearing, cognition and fine motor skills.

For companies that want to make sure they’re not blocking millions of seniors and people with disabilities, and wish to avoid lawsuits, the process begins with an audit of the website to identify the gaps. But there are many levels and approaches. Many assume that in today's world, automated testing tools using AI would be effective. However, that is not the case here.

Automated Testing vs. Manual Testing for WCAG Website Audits

When it comes to auditing a website for Section 508 or ADA compliance, there are different approaches that can be taken - some good and some bad. While automated testing may seem like a convenient solution, it can only catch low-hanging fruit and is limited in its ability to detect many issues. To ensure that a website or app is truly WCAG compliant and accessible, manual human testing is essential.

Automated website testing using software is limited in its ability to accurately detect WCAG compliance issues. The Web Content Accessibility Guideline is nuanced and interpretive, and software is only good for binary testing. Even the best automated testing tools cannot detect more than about 30% of WCAG issues. Therefore, human testing is required to address the remaining balance.

Some examples of the limitations of AI and automated testing tools include:

  • Low contrast text that may be difficult to read for users with visual impairments
  • Lack of alternative text for non-text content, such as images, videos, or charts, which can make the content inaccessible for screen reader users
  • Inconsistent headings or labeling that may cause confusion for users who rely on the page structure to navigate
  • Keyboard accessibility issues, such as the inability to navigate certain elements or perform certain actions using only the keyboard
  • Video content without captions or transcripts, which can make the content inaccessible for users who are deaf or hard of hearing.

Best Practices for WCAG Website Auditing Requires Human Testers

While automated tools can catch some issues, manual testing is essential to ensure full compliance with the WCAG standard. A complete WCAG website audit should include human testing of "unique pages" that include examples for all templates and page level components. The first step is to identify the unique pages and elements that require testing, and then an accessibility consultant and development team can work together to review the code and test use-cases using actual assistive technologies, such as screen readers and keyboard-only navigation. The findings should be presented in an audit report that documents all WCAG violations, including guidance for remediation.

Use Testers Who Also Know The Code 

To effectively audit a website for WCAG compliance, it's essential for the auditor to have a strong understanding of the front-side code, including HTML, CSS, and JavaScript. This knowledge will allow the auditor to not only identify each issue but also provide guidance on how to fix it. Having both a web development and accessibility consulting teams under one roof, we’re fairly unique and especially qualified to provide the best guidance for rapid remediation.

The Cost of Manual WCAG Audits

A full audit that includes both automated and manual testing, as well as assistive technology testing, can be costly. The way our company prices such an audit is by the number of unique pages that require manual testing, which typically ranges from $10,000 to $30,000.

How to Reduce the Cost of Human WCAG Auditing

Yes, manual testing is not cheap. Smaller companies often cannot afford a full manual audit. As pragmatists, our approach is to first understand what you can afford, and then pair down the scope of the project to meet that number. The number and complexity of “unique pages” that require human auditing is our cost factor, so to reduce the cost, we simply reduce the number of manual test pages. Remember we will still run an automated scan of the entire site to catch the low-hanging fruit. And even cutting down the manual testing to the header and footer plus a few pages will reveal a great deal of key issues that will make a huge difference for real users.

Another option is to span over two budget cycles by taking a phased approach. Begin with an automated audit, remediate the issues found, and then return to complete the manual audit in the next budget cycle. Or start with an audit that includes automated testing plus a minimum number of keypages for human testing. Then return later to audit the rest.

Warning: Avoid Overlay Providers

Naturally, there is great demand for a quick and inexpensive solution. And even if one doesn't exist, that won't stop some from claiming such. Avoid what are known as "overlay solutions, toolbars, or widgets". These providers claim to make your website 100% ADA compliant overnight using their favorite buzzword: artificial intelligence. They use javascript to inject corrective code over the existing code - hence "overlay". However, since even the best automated tools can only detect 50% of WCAG issues, their claims are flatly false. Not only do they fail to provide adequate accessibility access or tools for people with disabilities, they also fail to provide legal protection - regardless of their claims of guarantees.

In practice websites that use overlays are getting specifically targeted. Since overlays cannot correct the underlying code, and provide only a leaky bandaid, trolling lawyers know they are not compliant and are now targeting these sites. More details on why overlay solutions fail and increase legal risk with links to supporting articles.

Final Thoughts

Automated testing tools are good for conducting a quick and minimum level of accessibility testing. However, they are limited in their ability to identify all accessibility issues on a website. This is why human testing becomes essential. Experienced and qualified human testers bring a level expertise that automated tools cannot match. They can identify accessibility issues that may not be caught by automated tools and provide a more comprehensive evaluation of a website's accessibility. Human testers can also provide insights into the user experience of people with disabilities, which is crucial for ensuring that a website is truly accessible. While automated testing tools can provide a starting point for accessibility testing, human testing is necessary for a thorough and accurate evaluation of a website's accessibility and WCAG compliance to ensure that all visitors can access your content while ensuring you avoid these lawsuits.