Manual accessibility evaluation

This manual procedure is designed to complement automated tools for web site accessibility evaluation, emphasizing areas that can only be evaluated realistically by human judgement. On screen or in print, it looks like a lot of work, but in fact each step can be done in a few minutes or even seconds. For many pages, the entire list (excluding Part 3) should take less than a half-hour to complete. There is an accompanying report form that can be used after becoming familiar with the descriptions and rationale given here. (www.tomjewett.com/accessibility/eval-report.html)

Although this procedure can help you to identify most major barriers to accessibility, it cannot by itself certify a site as being accessible. A thorough evaluation and retrofit program must also include authoritative sources such as those listed in my Section 508 quick reference page. (www.tomjewett.com/accessibility/section508.html) There are many additional checklists and references that may be helpful; the most comprehensive and current of these that I've seen is Web Accessibility by Jim Thatcher. More information is available at jimthatcher.com. (www.jimthatcher.com/)

Most sites that you will be evaluating consist of a number of individual pages that belong to the same organization or sub-unit—for example, a university academic department or staff agency—and are maintained by a single individual or development group. Normally, you will only need to look at a very few pages (one to three) from any one site, which should include:

• the home or entry page for the site;
• one page containing typical information that is posted on the site;
• possibly one page that represents many other similar pages, such as a faculty listing;
• any page with specialized content such as a calendar, request for information, or multimedia presentation.

Tools needed

Depending on your system configuration, a system administrator may have to install the software listed here.

• Firefox browser, latest version, downloaded from Mozilla.com. (www.mozilla.com/en-US/firefox/)
• Firefox web development toolbar (just called "the toolbar" below). In Firefox, open the add-ons page, (addons.mozilla.org/firefox/60/) and click "Install now" (it installs automatically). Then close and reopen Firefox to finish the installation. (Note: there are similar toolbars for Internet Explorer 6 and 7, but the Firefox version seems to be the most complete and easiest to use.)
• Fangs text-reader simulation for Firefox. Install like the developer toolbar above, from Sourceforge. (sourceforge.net/projects/fangs)
• Colour Contrast Analyser version 1.1 (yes, it's UK spelling, and yes, the version has to be 1.1 not 1.0). Go to the Vision Australia download page, (www.visionaustralia.org.au/info.aspx?page=628#download) and click the first link under "Download." Save the .zip file and extract it to a convenient place. You may want to make a shortcut to the .exe file on your desktop.

Part 1: finding barriers

This section is designed to require no knowledge of the inner workings of the web or of web coding languages. It is frankly meant to be done by a fully-sighted reader, who may also be learning about the barriers that disabled readers face every day.

Visual check

• Method: open your page in the Firefox browser; look at the page as displayed.
• Why this is important: most visitors to the page are fully sighted; some pages might be difficult for them to understand or navigate, too.
• What to look for:
  • text should be legible in size and font selection;
  • there should be sufficient visual contrast between text and background colors;
  • page elements should be positioned in a logical, organized, easy-to-find layout;
  • nothing should be moving or blinking at an unhealthy rate. (See W3C quidelines. Best practices: nothing should be moving or blinking at all, unless the motion is used to convey essential content.)
  • Important: if the page really requires frames, image maps, timed response, or active content such as video, audio, applets, or scripts, it should be flagged immediately for a thorough, specialized evaluation of those components.

Text size

• Method: use the browser's View -> Text Size menu item (or Ctrl+) two or three times to increase the size of the text, then drag the right side of the browser window (using the mouse) to make it narrower.
• Why this is important: Visitors with mildly reduced vision may be assisted by a simple increase in font size. However, pages that are incorrectly structured might force them to scroll horizontally to read the text, and some pages might not even respond to the browser's change in font size.
• What to look for:
  • all text on the page should respond to the change in size—exceptions might be large decorative logos or banners that contain text within an image;
  • text should "wrap" within columns as the browser width is changed, and should not be hidden past the right side of the page;
  • columns, page elements, and text lines should not overlap each other as these changes are made.
• When finished, put the text size and window width back to normal.

Tab order

• Method: use the keyboard Tab key to move through the sequence of links and form elements as they are ordered on the page.
• Why this is important: visitors who are unable to use the mouse will rely on the tab key for navigating between links and form elements.
• What to look for:
  • logical progression from element to element and between groups of elements such as links;
  • tab order between form elements that makes sense in the way that visitors would normally fill in the information.
  • Note: navigation around groups of links will be covered under "linearization" below.

Document outline

• Method: select the toolbar Information -> View Document Outline option. (The outline will open in a new tab).
• Why this is important: all visitors are helped by logically-organized page content that uses section headings and sub-headings in their proper order; visitors using assistive technology can use the headings for easy navigation between sections of the page.
• What to look for:
  • headings that are properly nested by level and make sense in the context of the document. (h1 should normally be the title of the page; the next level or organization should be h2, and so on. Heading levels should not be skipped, and headings should not be used for visual styling.)
• When finished, close the document outline tab.

Links

• Method: select the toolbar Information -> Display Link Details option.
• Why this is important: page elements that function as links (including text, images, and buttons) should be recognizable and usable by sighted viewers and assistive technology.
• What to look for:
  • all links, including text and images that are links, should look "clickable";
  • minimum use of images that contain only text as links;
  • no links should contain "href='javascript:...'". (Javascript-only links don't work well—or at all—with some assistive technologies, and in any event do not represent a valid URI as required by the href attribute).
• When finished, toggle the Display Link Details option back off.

Alt attributes

• Method: select the toolbar Images -> Display Alt Attributes option.
• Why this is important: assistive technologies such as screen readers can't read images; neither can search engines. Some sighted visitors might have images disabled for various reasons, or images might fail to load over a network. If alt-text is not given for images used as links, screen readers will read the whole URL instead, which is horribly confusing for listeners!
• What to look for:
  • The key requirement in all standards is that equivalent text be provided.
  • For images that contain only visual text, the alt-text should be the the same as the image text; however, images of text should be used very rarely.
  • For images or other non-text elements that contain significant or complex content, a domain expert should assess "equivalency," and additional content such as a longdesc attribute or an explanatory paragraph may be needed.
  • Note: although it won't show in this test, for best practices images should have title= as well as alt= attributes, since some browsers use the title for mouseover hinting.
  • For images that are only decorative, the alt attribute should be empty (alt=""), and the title attribute is irrelevant.
• When finished, toggle the Display Alt Attributes option back off.

Tables

• Method: select the toolbar Information -> Display Table Information option.
• Why this is important: visitors using assistive technologies need to associate the data in each table cell with its column and row headings, in order to know what the data means.
• What to look for:
  • column and row headings should be identified and given a scope= attribute (column or row);
  • data cells in complex tables (those with multi-column or multi-row headings) should associate their headers with the data cells (id= in headers and headers= in data cells);
  • tables that contain data should show a summary attribute if its meaning is not clear from other surrounding information;
  • Tables that do not contain data, but are used for layout only, should not have any of these attributes (summary or headers).
• When finished, toggle the Display Table Information option back off.

Style sheets

• Method: select the toolbar CSS -> Disable Styles -> All Styles option.
• Why this is important: many low vision visitors will disable the page author's styles, and may substitute a custom style sheet of their own.
• What to look for:
  • virtually all color, font information, and decoration should be rendered in the browser's default style;
  • headings, paragraphs, and lists should be obvious and sensible;
  • most text other than logos and banner headings should be rendered in text rather than images;
  • if text content appears that was invisible before this test, the CSS code must be checked to insure that this element is not also hidden from users with low vision style sheets or screen readers.
• When finished, leave styles disabled. They will be turned back on automatically in the next step.

Images

• Method: select the toolbar Images -> Disable Images -> All Images option.
• Why this is important: for the same reasons as the alt attributes test above.
• What to look for:
  • portions of the text content should not disappear entirely;
  • alternate text should make sense in the context of the page;
  • there should still be sufficient contrast between text and background colors. (If there are problems with this, it's probably a sign of hard-coded styles in HTML elements.)
• When finished, leave images disabled.

Linearization

• Method: again select the toolbar CSS -> Disable Styles -> All Styles option, then select Miscellaneous -> Linearize Page.
• Why this is important: this test is reasonably close to the way a visitor would experience the page using assistive technology or an adaptive style sheet.
• What to look for:
  • the reading order of headings and page elements should make sense (note: the order of items within data tables is especially critical);
  • each group of navigation links should have a way to skip reading and go directly to content;
  • multiple groups of links (for example, on a home page) should have easy navigation from one to the other.
• When finished, toggle the Disable Images option (from the Images check above) back on; this will also restore the original styles and non-linearized layout.

Part 2: technical evaluation

This section requires some knowledge of web coding languages. It might be done by web developers after others have looked at part 1.

Valid HTML

• Method: select the toolbar Tools -> Validate HTML option. (The validation results will open in a new tab.)
• Why this is important: contemporary, maintainable, accessible web design is based on complete separation of content (HTML/XHTML) from presentation (CSS) with both conforming to W3C standards; valid code contributes to accurate presentation by both standard browsers and assistive technology.
• What to look for: valid HTML, at best according to the XHTML 1.0 Strict or Transitional standard, or according to HTML 4.01 Strict or Transitional for older pages. (Note: this, and all automated tests, can only determine whether the code correctly follows syntactic rules; it won't tell you if the way the code is used makes any sense for the viewer.)
• When finished with this and each of the next three steps, you may want to close the new tab or just leave it there and return to the tab that contains the page you are evaluating.

Valid style sheets

• Method: select the toolbar Tools -> Validate CSS option. (The validation results will open in a new tab.)
• Why this is important: for the same reasons as HTML above.
• What to look for: valid CSS. (Also see note under HTML above.)

Speed report

• Method: select the toolbar Tools -> View Speed Report option. (The report will open in a new tab.)
• Why this is important: some visitors, with or without disabilities, may be accessing the page with a slow internet connection, although even those with faster connections will benefit from the smallest possible page load.
• What to look for: this is a very quick way to identify pages that are full of images substituting for text links, or images that have not been optimized for use on the web.

Color contrast

• Method: open the Colour Contrast Analyser program (see Tools needed, above). For each differently-colored portion of the page, click the Foreground eyedropper button, then move the mouse over the page and click again on a pixel that best represents the foreground color. Repeat this for the Background color.
• Why this is important: visitors both with full vision and with various forms of color blindness require sufficient contrast to read text easily.
• What to look for: a green checkmark in the Result-luminosity panel. Also click the checkbox labeled "Show contrast result for colour blindness," and look again for green checkmarks.

Color simulations

• Method: on the Colour Contrast Analyser menu, select Simulations -> Select window (List). In the Select window list of the popup dialog, click the page that you are evaluating, select the Greyscale Simulation radio button, then click the Preview button. Repeat for other listed simulations.
• Why this is important: this check provides a look at the full page, including images, as seen by visitors with various visual disabilities.
• What to look for: visual distinctions that that are made by color in the normal page should still be distinguishable even in grey scale; the page should be intelligible in each of the other simulations.
• When finished, close the Window list dialog and the Colour Contrast Analyser program.

Source code

• Method: select the toolbar View Source -> View Source option. (The code listing will open in a new window).
• Why this is important: some problems can be identified and evaluated most accurately with this check—especially details that may have been flagged by automated evaluation tools.
• What to look for:
  • Styles (whether in CSS or embedded in HTML elements) should not be used to simulate or substitute for headings or other semantic markup. (For best coding practice, styles should not be embedded in HTML elements at all.)
  • The code should be free of extraneous elements used for layout or decoration, for example invisible images or non-breaking space characters used for spacing, line break elements used to simulate lists, or horizontal rule elements used instead of CSS border styles.
  • Each form input should have an associated <label> element or other appropriate grouping element such as fieldset.
  • Data tables should be coded properly with (at least) <th> elements with scope= (col or row) attributes. Data tables with multiple heading levels should also have id= attributes for headers, and headers= attributes for data cells. For best practices, captions, summaries, table head and body elements should be used as needed in context. Tables used for layout only should have none of these elements.
  • Also check specific items identified by automated evaluation tools, remembering that in some cases, the flagged items are not problems at all!
• This check, of course, looks only at the code that was received by the browser. In most cases, the actual source—a template or script— will also have to be checked to determine how the browser code was generated.
• When finished, close the source listing window.

Part 3: user testing

This section ideally requires the assistance of users with actual disabilities. Alternatively, it requires some inventiveness and role-playing by non-disabled testers. Although only two checks are suggested, if more people are available to use the site under different conditions, the results will certainly be more robust.

Much of the adaptive technology in actual use today is very specialized and very expensive. You may want to enlist the support of a Disabled Student Services office or similar organization in order to do this section effectively.

These actual-use checks are likely to reinforce the observations that you made in parts 1 and 2 above, but they are also likely to identify problems that would be missed by any manual or automatic evaluation. If you have already identified any truly major barriers to accessibility, it might be best to delay this part until the most important fixes have been made.

Low vision

• Best method: a person with low vision should use the site (not just the specific pages that you tested above), with the normal adaptations that he or she uses for all web browsing. As many functions of the site as possible should be exercised this way, including navigation and search for specific information, form fill-in, and multimedia, applets, or scripts if present.
• Alternate method: Download the file lowvis.css to any convenient folder on your computer. (www.tomjewett.com/accessibility/lowvis.css)
  • In FireFox, from the Web Developer toolbar, select CSS -> Add User Style Sheet, browse to wherever you stored it and click the Open button. When you are finished, just un-check the "Add User Style Sheet" on the toolbar CSS menu.
  • In Microsoft Internet Explorer, select Tools -> Internet Options... -> Accessibility... and check the "Format documents using my style sheets" box. Using the Browse button, open the lowvis.css style sheet from wherever you stored it, and click OK to both the Accessibility and the Internet Options panels. When you are finished, simply un-check the "...my style sheets" box on the Accessibility panel.
• Note: this low-vision style sheet was developed by accessibility expert Dr. Wayne Dick for his personal use. (He also uses a voice reader along with it.) It represents just one possible adaptation for low vision; other users will require very different styles or techniques.

Blindness

• Best method: a blind person should use the site with his or her normal adaptive technology, as described under low vision above.
• Alternate method:
  • If you have a licensed, fully-functional copy of IBM Home Page Reader, it is probably the easiest to use, although it only operates with Microsoft Internet Explorer 6. The demo version is limited to specified sites, so is useless for this test.
  • The most popular full-screen reader, Jaws®, is available in a demo version that runs for 40 minutes at a time before requiring the computer to be re-booted. (www.freedomscientific.com/fs_downloads/jaws.asp)
  • For FireFox, you can download Charles Chen's free page reader, FireVox. (firevox.clcworld.net/downloads.html)
  • With any of these tools, open your site, put a piece of cardboard over the screen, keep your hand off the mouse, and do the best you can to struggle through the site.
• Note: no fully-sighted tester can use this technology as effectively as a truly blind user can. About the best you can do is to study the help system (visually) and try to learn at least the major navigation keys before you try the actual site.