Accessibility Testing: Why Manual Testing Is Required

Accessibility Testing: Why Manual Testing Is Required

Last Edited August 19, 2019 by Garenne Bigby in Accessibility Testing

The process of testing your website to ensure it is fully accessible to all users is a lot of work. In fact, you might be tempted to use a few automated accessibility testing tools and call it a day. While there are quite a few fantastic automated accessibility testing tools that can catch issues, they won’t catch absolutely everything you need to find. They do an excellent job at catching issues that are programmatic, but they’re not an end-all-be-all solution. Your users are ultimately human beings, which means they’ll experience any given website differently than a computer will. Therefore, part of your routine when it comes to accessibility testing needs to involve real people who know what they’re looking for.

This is referred to as manual accessibility testing — an essential part of your routine wherein real people test to see if your website is usable for visitors with various disabilities. Typically, any type of accessibility testing will look for compatibility with various assistive technologies. Although automated accessibility testing is crucial, it’ll only catch some of the errors. A human element, on the other hand, will be able to find issues that would otherwise result in barriers for individuals with disabilities trying to use your website.

You might think your website is ready to go after running a few automated accessibility testing tools. But don’t stop there. Far too many business owners think that’s enough when it’s simply not. You need to confirm your website is accessible with the help of real people who know what they’re doing. Otherwise, you risk your website being unusable to a sizeable portion of the population.

When Should You Perform Manual Accessibility Testing?

As mentioned, you can’t rely on automated accessibility testing as a complete, turnkey solution. Unfortunately, computers aren’t able to understand context the way we, as humans, are. Manual accessibility testing is the right approach after you’ve run a few automated accessibility testing tools. This involves combing through by hand to catch issues that may cause barriers for specific users, in particular, those with disabilities. Ultimately, this keeps your website accessible, so you’re not losing business because of a lack of inclusivity.

Let’s take a look at two main areas where manual accessibility testing is absolutely necessary:


1. Compatibility with a Screen Reader:

Before you begin testing, it’s important to understand how screen readers work. They’re a commonly used assistive tool designed for those with low vision. Essentially, screen readers access or enter the DOM (Document Object Model) and use browser Application Programming Interfaces (APIs) to gather information. Typically, the screen reader will know what to say and when — announcing, in advance, what the user needs to know. If you’re relying on automated accessibility testing to ensure your website is compatible with a screen reader, you’re going to miss many issues.

When testing with a screen reader, start by having the screen reader go through the page sequentially and testing for the following:

  • Reading is smooth and understandable: The screen reader shouldn’t have trouble or get stuck at any point while reading the content.

  • Hidden content is not read to the user: If you have a hidden search form or off-canvas navigation, make sure the screen reader doesn’t read that to the user.

  • Content is organized and makes sense: This can potentially be a problem with things like dates, phone numbers, and abbreviations, so make sure it flows properly.

Remember to test with a few different screen readers. Voiceover, for example, is bundled with Macs. NVDA, on the other hand, is a free download for Windows. These two screen readers are the most popular screen readers available. JAWS is another widely-used commercial screen reader. Typically, screen readers rely on the title of the web page, then each text element in the order it appears throughout the source code. They don’t necessarily hear the source code, but all text content found in the page markup.

That means you need to pay special attention to this text, and more specifically, the order of this text. Make sure everything is set up properly for the screen reader, in order to make your content make the most possible sense for the user. Focus on headings, landmarks, searches within pages, and lastly, lists of links. These are the most common methods the screen reader will use to find content quickly.

Similar to any form of accessibility testing, start early, and do it as often as you can. Don’t worry too much about basic pages built with semantic HTML. These typically work well with screen readers, so don’t place any urgency on them. The latest version of the guidelines, WCAG 2.1, offers a few tips in regards to screen reader use:

  • Any non-textual content should have a “text alternative” to help users with screen readers understand the information. For instance, descriptive embedded captions can be added to images so screen readers can inform users about the content.

  • Words, phrases, or texts that are written in a foreign language should be programmatically determined, in order to ensure the screen reader pronounces these parts properly.


2.  Navigation with a Keyboard

Many users navigate websites using the keyboard rather than the mouse. Sometimes, expert users prefer the keyboard as the commands allow for greater efficiency. Sometimes, users with disabilities need to use the keyboard as they’re not able to perform the fine motor movements necessary to use the mouse. Regardless of the reason, your website needs to be compatible with keyboard-only navigation. This means a user should be able to do the following with a keyboard:

  • Access various menus throughout the site

  • Move from section to section on a specific page

  • Use top-of-page links to skip to important content

  • Highlight form fields and links as necessary

When you’re testing to ensure navigation is possible with a keyboard/without a mouse, make sure you’re verifying that:

  • All interactive elements are accessible: The user should be able to access all interactive elements, including drop-down menus, buttons, dialog boxes, forms, and other widgets.

  • A skip navigation link is available: Many websites allow sighted keyboard users to navigate the page via a tab order that’s coded to reflect the page’s layout. If there are many tabs, this is cumbersome. Mouse users can disregard the main navigation and interact with other links. A skip navigation link helps keyboard-only users do the same.

It’s essential to ensure you’re meeting the needs of all individuals who depend on the keyboard instead of the mouse to navigate through your website.

 

Debunking Common Myths About Accessibility

When we talk about testing for accessibility via automated or manual testing, it’s important to understand why you’re testing for accessibility in the first place. Technology is completely evolving the way we live and work. But we need to make sure everyone is able to access it, especially in the digital age when technology provides new opportunities to those who consume it. For those with disabilities, technology can empower them to gather more information and access resources they otherwise wouldn’t have.

When they’re able to access the web, they’re able to experience inclusivity with the rest of society, which is incredibly important for their mental health and well-being. This is a huge part of why it’s important to perform accessibility testing. Before we discuss more information on accessibility testing, let’s look at a few common myths about accessibility that need to be addressed, in order to ensure you’re performing your manual testing properly. 

  • MYTH 1: All Screen Reader Users Are Blind: This simply isn’t true. Many individuals who use screen readers simply have low vision and want to supplement what they see on the screen. In addition, some individuals who use screen readers are colorblind and need assistance understanding information that is shown in a color format they can’t see. Moreover, some individuals who use screen readers simply need help with reading comprehension.

  • MYTH 2: Web Accessibility Only Caters to Individuals who have Permanent Disabilities: Web accessibility is about more than ensuring accessibility for people with permanent disabilities. It’s about those who are experiencing a conditional disability brought about by their environment or circumstances. Let’s say they’re in a noisy airport or beside a window with a strong glare on their screen. There are many reasons to ensure accessibility for all people.

So what should you look for in a manual accessibility tester? How do you know if you’ve found someone who is capable of finding any issues and/or problems that need to be fixed? First and foremost, remember to look for more than one person. You’ll want at least two sets of eyes on your website. Similar to any type of editing, the biggest benefit of manual accessibility testing is getting a new perspective on the material.

If you can have your website tested by users who have disabilities, this is incredibly beneficial as they have firsthand experience with accessibility barriers. Sometimes, businesses may use their existing staff to perform manual accessibility testing. While this is a cost-efficient method, it’s better to have an outside perspective. This is preferable as they likely haven’t spent many hours looking at the website, so they’ll catch things your staff wouldn’t be able to notice.

Regardless, you’re getting human observation as opposed to computerized scanning, which is invaluable when it comes to accessibility. Sure, automated accessibility testing tools are advanced nowadays, but an algorithm still isn’t able to detect a lot of the issues impacting the overall user experience. When you’re looking for people to help you perform manual accessibility testing, look for those with the following:

  • Experience with Assistive Technologies: This is a huge plus for manual accessibility testing. If you can find a few people with experience with various assistive technologies, including screen readers, screen enlargement applications, voice recognition programs, and more, you’re able to ensure your website is truly up-to-par and meeting the needs of all users.

  • Familiarity with Accessibility Standards: Your ultimate goal is maintaining compliance with WCAG 2.1 Level AA accessibility standards. This means anyone you have working on your website or testing your website should be familiar with those standards. You don’t want to hire a manual accessibility tester who doesn’t know what they’re looking for and what’s acceptable.

At the end of the day, it’s crucial to ensure you’re using a combination of automated accessibility testing and manual accessibility testing. There are areas of your website that need to be checked via machine and human. There is no way around it. Most website owners perform automated accessibility testing first, in order to get a full overview of any potential issues across their website. Next, they perform extensive manual accessibility testing to find anything that may have been missed.

In addition, make sure you’re performing accessibility testing on a regular basis. This should be an ongoing project wherein you treat it as part of your website maintenance. Try to run both automated and manual accessibility testing at least twice per year to ensure you’re not missing anything.

Manual Accessibility Testing with DYNO Mapper

An accessible website is important to ensure you’re able to meet the needs of all individuals, and therefore, succeed as a business. And beyond that, it’s required by law. There are so many tools available, including but not limited to DYNO Mapper — a web accessibility tool that enables you to scan your website for Section 508/WCAG compliance while identifying any errors that don’t comply with ADA website standards. Naturally, web accessibility isn’t something you can achieve overnight. It takes time and effort, and of course, lots of testing.

When it comes to performing manual testing with DYNO Mapper, it’s helpful to have knowledge about various aspects of how a website works, such as CSS, HTML, and JavaScript. But fortunately, DYNO Mapper makes it easy to perform manual testing with minimal experience. DYNO Mapper allows you to test against the latest WCAG 2.1 from the World Wide Web Consortium (WC3). In fact, DYNO Mapper:

  • Flags all criteria that need manual testing in WCAG 2.1

  • Enables easy archival of issues after you’ve checked the problem manually

You’re able to manually archive problems that have been manually tested or remediated from your site, whether it’s in a test or live environment, by means of the Archive Problem feature. This can be used to remove any false positives that you feel aren’t correct. You can also keep that problem from popping up in future reports using the Always Archive Problem feature. If you want to keep that problem from popping up in future reports throughout your entire website, you can use the Always Archive Problem - All Pages feature.


Laws

As mentioned above, an accessible website is required by law. In fact, there are quite a few laws that protect those with disabilities — requiring those who create products and/or services make sure they’re accessible to all individuals, including people with disabilities. Here are some applicable laws to keep in mind:

  • Americans with Disabilities Act: The Americans with Disabilities Act, established in 1990, is a law under the civil rights section that prevents unfair treatment against people with disabilities throughout all areas of life — from jobs to transportation to schools. The law requires all schools, public buildings, employers, and other organizations to ensure assistive technology is accessible to those who need it. The law keeps people with disabilities from being denied an education, job or services due to their disability.

  • Web Content Accessibility Guidelines (WCAG): These guidelines offer ways to ensure your website is accessible for those with disabilities. The most up-to-date guidelines, WCAG 2.1, set the standard website owners need to follow. The principals create an acronym, known as POUR, which stands for: perceivable, operable, understandable, and robust. These principals should be kept in mind to ensure those with disabilities can leverage your website.

  • The Rehabilitation Act, Section 504, and Section 508: This law requires all public spaces, such as schools, buildings, and other locations, be made accessible to people with disabilities. Section 504 requires appropriate accommodations to be made for those with disabilities. 508 requires all technology offered to the public be made accessible as well. The Rehabilitation Act was established back in 1937 — long before the Americans with Disabilities Act.

These laws and guidelines are important for anyone who owns or operates a website. It’s necessary to ensure you’re not leaving a significant portion of the population out. Whether you’re building an entirely new website or you’re updating a website you currently have, it’s vital to know the reasons why you need to ensure accessibility.


Diversity

DYNO Mapper is a sophisticated website accessibility testing tool that enables you to test against a range of local and international guidelines, such as Section 508, Stanca Act, BITV 1.0 (Level 2), WCAG 1.0 (A, AA, AAA), WCAG 2.0 (A, AA, AAA), and WCAG 2.1 (A, AA, AAA). You’re able to leverage the visualize feature to view tests live in a browser — seeing all known, likely, and potential issues indicated with icons over the live website image.

It makes the process of website accessibility testing incredibly simple as you’re able to test projects as many times as you’d like with no limits on the number of tests per domain within any given month. You’re able to set up notifications of accessibility problems, in order to have them sent to your email.

As a member of the World Wide Consortium (W3C), the leading international standards organization for the web, as well as the International Association of Accessibility Standards (IAAP), a non-profit membership-based organization focused on raising awareness and knowledge of standards, DYNO Mapper is a trusted source for helping website owners achieve website accessibility.

The ability to manually archive issues is invaluable as it ensures no matter how many individuals are manually testing your website, you won’t have to worry about the same issues being brought to your attention. Instead, they’ll be archived so they won’t pop up in future reports. The entire experience is simplified to prevent any confusion that may drag out the testing process.


Conclusion

Manual accessibility testing is an essential part of ensuring your website is accessible to all individuals, including people with disabilities, as it involves human judgment — something that cannot be replaced with technology. Automated accessibility testing should still play a role in your overall website maintenance efforts, but don’t forego manual accessibility testing for it. There are many different avenues you can look at when it comes to manual accessibility testing, but keep in mind, DYNO Mapper’s tool makes it easier as it flags all criteria that need manual testing. Many reputable consulting firms offer the ability to hire users with disabilities to manually check pages and test functions on the website — allowing you to get a comprehensive review of what works and what doesn’t work.

Garenne Bigby
Author: Garenne BigbyWebsite: http://garennebigby.com
Founder of DYNO Mapper and Advisory Committee Representative at the W3C.

Back

Create Visual Sitemaps

Create, edit, customize, and share visual sitemaps integrated with Google Analytics for easy discovery, planning, and collaboration.

Popular Tags

Search Engine Optimization SEO Create Sitemaps Accessibility Testing Sitemaps UX User Experience Sitemap Generator Content Audit Website Content Audit
Create Interactive Visual Sitemaps

Discovery has never been easier.

Sign up today!