Notes on some tools for checking and improving Web page accessibility (Bobby and A-Prompt)

There are several programs and online services that can be used to check Web pages for accessibility. This document presents some notes on the nature of accessibility checking, then remarks on two relatively well-known tools, Bobby and A-Prompt. In particular, some essential problems with them are described, in addition to the positive side. This document is intended for Web authors who have become aware of the importantance and impact of accessibility; for information on such matters, consult e.g. the Web Accessibility Initiative (WAI) material.

For a large list of other tools, see the document Evaluation, Repair, and Transformation Tools for Web Content Accessibility. However, the tools typically have limitations, such as checking some aspects of accessibility only, and they are often experimental or otherwise unsupported.

Jim Thatcher has written a detailed review of Bobby as well as other reviews of evaluation and repair tools (but not A-Prompt).

Bobby as described here does not exist any more, but the criticism presented here mostly applies to its successors, too. Links to Bobby now point to the WebXACT checker by Watchfire, which is even more confusing, since it performs many kinds of checks. The Accessibility section in its report corresponds largely to what Bobby did.

Accessibility checking - can programs do it?

The tools discussed here are mainly for checking compliance with "WAI recommendations", or Web Content Accessibility Guidelines 1.0 (WCAG 1.0) to be more specific. They may provide an option for checking against "Section 508" instead, but this does not make a big difference for our discussion.

The guidelines can be divided into four different categories as regards to automatic checkability:

1. Completely checkable. Example: "Avoid deprecated features of W3C technologies". For syntactic features that are clearly indicated as deprecated in W3C specifications, such as <font> markup, it can be automatically checked whether a document complies with the guideline or not. (Some questions might arise, however, about the exact meaning of the word "avoid". If it means less than "don't", e.g. if it means "don't use with a very good reason", then this guideline falls into the next category.)
2. Partly checkable: automatic analysis may yield a definitive result about not complying with the guidelines, but it cannot tell for sure that the document actually complies. Example: "Provide a text equivalent for every non-text element". It can be automatically checked whether, for example, an img element has an alt attribute. If it does not, we can definitively report a violation of the guidelines. But we cannot programmatically detect whether the content of the attribute is really a text equivalent for the image. It is misleading to say that a checker has verified that document complies with the guideline "Provide a text equivalent for every non-text element", since, for images for example, the checker has only verified the presence an attribute that should contain the text equivalent.
3. Checkable for alerts. Example: "Provide a means to skip over multi-line ASCII art." Since ASCII art (or, more generally, ASCII graphics) is generally inserted into HTML using <pre> markup, we might issue an alert about a <pre> element that is not immediately preceded by a link that refers to a location after the element. But maybe the <pre> does not contain ASCII graphics at all, or maybe the document has ASCII graphics embedded in another way. A program cannot say "yes" or "no" for sure whether a document complies with the guideline, but it could point at a construct that is probably a problem in this respect.
4. Not checkable. Example: "Mark up quotations." There's hardly any way to detect automatically whether, for example, a document contains a piece of quoted text that should have been marked up using <blockquote>. Similarly, if there is <blockquote> markup, how could a program decide whether it actually contains a quotation or is used only for the purpose of indenting text?

A big problem with many checkers is that they try to check for alertable constructs when they really can't, i.e. the relevant guideline is in the fourth category (not really checkable using automated tools). If you start flagging each <blockquote> as being potentially not a block quotation at all, you will be right quite often, but where will it end? Should all heading elements be flagged, since people have abused them to get some assumed display style for something that isn't a heading at all? If you alert all images since they might violate the guideline "avoid causing the screen to flicker" or if you alert all constructs related to colors since the author might "rely on color alone", shouldn't you also flag any text anywhere as potentially violating the guideline "Use the clearest and simplest language appropriate for a site's content"? (In fact, this guideline is probably violated far more often the guideline against flickering.)

As a criterion for genuinely alertable constructs, we can ask: If a checker informs about a possible violation of the guidelines, will it keep doing that no matter what the author does with the document? Naturally, we're assuming that the author is not willing to remove some content entirely or to do something absurd; he doesn't want not to stop using block quotations or to stop using the adequate markup for them just because a checker nags about any <blockquote>.

Within WAI, there has been an attempt at specifying principles for accessibility checkers. The working draft Techniques For Accessibility Evaluation And Repair Tools (2002-04-26) is, however, rather sketchy, and largely just a collection of suggested heuristics and specialized methods.

Notes on Bobby

Bobby software currently exists as a free version on the Web, to be used via a Web browser, and as a program that can be downloaded, for a fee, onto the user's computer and used locally. The comments below are based on using the free Web version. In that version, you use a simple form where you specify the Web address (URL) of a page to be checked.

Bobby is probably the most widely known of Web accessibility tools at present, and possibly the most commonly used. However, it has severe limitations and problems. "Bobby approval" should not be regarded as a criterion for accessibility; rather, Bobby reports help to notice some problems.

Does Bobby try too hard?

Bobby has been heavily criticized for issuing misleading messages. Perhaps the strongest view has been presented in Earth calling Su, an interview containing detailed criticism on Bobby. It seems to me that the heart of the matter is Bobby's wrong approach to the "checkability problem" that I described above. It simply tries too much and ends up with issuing messages that confuse the novice and irritate the expert.

For example, Bobby always says, even in "Bobby AAA Approved" messages: "If you can't make a page accessible, construct an alternate accessible version." (At least I think it always says that. The structure of Bobby's messages actually suggests that it would be "triggered" by some features on a page, but Bobby does not tell what those features might be.) How does that apply? Besides, it's basically wrong advice anyway: if you can't make a page accessible, you should try smarter.

Whenever a page uses a style sheet, Bobby says: "If style sheets are ignored or unsupported, are pages still readable and usable?" This might be seen as an alert based on the content of the document being checked, so it in some sense falls into category 3 above. But it's fairly redundant in practice. It would be much better if Bobby's approval reports just contained a general disclaimer that refers to a document that explains Bobby's limitations, including those WCAG 1.0 (or Section 508) guidelines against which Bobby does not really check the document at all.

Other problems with Bobby

Let us consider the example of links with the same link text. Consider, for simplicity, a page containing short headlines of news, each followed by a link named "More", pointing to a separate page that contains a more extensive article. There are good reasons to try to avoid such an approach, but it is less obvious what to do instead. Anyway, such constructs do appear on Web pages; what is the situation as judged by WCAG 1.0, and what does Bobby say about it?

Bobby reports a violation of the rules for link texts. It says: "Do not use the same link phrase more than once when the links point to different URLs." And partly on this ground (plus partly on the undeniable ground of a violation of a Priority 2 WAI requirement), Bobby reports that http://www.w3.org/ (sic; as accessed 2002-08-07) does not pass "AA" test. Note the wording: "This page does not yet meet the requirements for Bobby AA Approved status. To be Bobby AA Approved, a page must pass all of the Priority 1 and 2 accessibility checkpoints established in W3C Web Content Accessibility Guidelines 1.0." So it does not really claim that Bobby "AA" means W3C WAI "AA" - though this surely is the common impression.

What Bobby refers to in this context is Checkpoint 13.1: "Clearly identify the target of each link." But the explanations in that checkpoint do not present an explicit requirement about different link texts. The associated HTML techniques document however says: "If more than one link on a page shares the same link text, all those links should point to the same resource." So is this a WAI requirement, or is it not? Is it "should" or "must"? If it's just "should", why does Bobby report it as a violation and not an item to be checked by the user? If it's "must", why does the recommendation proceed by telling what to do when the requirement is not obeyed?

The HTML technique document suggests that for links with identical texts but different URLs, the title attribute be used to distinguish between links. Bobby quotes this (without specifying the exact source), and the W3C main page uses the technique - but this apparently does not prevent Bobby from reporting an error. The W3C main page has several links with the same text "News archive" but with different href values (though differing in the fragment id only, so this might be seen as a moot point); they have different title attributes, and Bobby still complains.

It is debatable whether the title attributes help much. I would say that from the accessibility point of view, as well as by HTML specifications more or less directly, the rule is: Never rely on title attributes. Use them to give additional information, which might help some users as "advisory titles" (as the HTML spec puts it), but don't count on it. In fact, the apparent idea, and the common implementation on browsers that support title, is that the advisory title is presented to the user upon request. For example, when you view a page on IE, you have no direct cue of what parts of the page have advisory titles, still less what the titles are; you can just move the cursor around and see some "tooltips". This might not be the optimal implementation, but it's a possible one, and a common one.

To summarize, it is understandable that Bobby alerts about links with identical texts and different destinations. This probably falls into my 3rd category, alertable. But it is questionable that Bobby reports it as a violation of the guidelines, and it is rather odd that Bobby recommends an option which, if taken, does not affect Bobby's complaints the least.

Notes on A-Prompt

The A-Prompt program can be downloaded for free, for use on Windows 95/98/NT/2000. When started, it asks you to select the file(s) on your hard disk that you wish to have checked. You can then get reports on the accessibility, and the program helps you fix them.

A-Prompt is a very useful program in many ways, but the user needs to understand the basics of accessibility and to learn some specific features and peculiarities in the program. The documentation is well-written and includes a help system that explains the checks and repairs in detail. Unfortunately it seems that there is no active maintenance and development for A-Prompt.

The document Overview of A-Prompt illustrates some of the basic features and the ease of use of the program. But it also illustrates some problems with it. There's a screen shot of a message from A-Prompt about a missing alt text, and it says:

Alternate("alt") text should be 10 words or less and include all text shown in the image.

Certain kind of images requite special "alt" text:

1. Spaceholder images should have " " as "alt" text.
2. Images serving as buttons should have "alt" text that emphasizes the action performed by the button.
3. Images used as bullets should have "*" as "alt" text.

Thus, the instructions are subjective and partly rather debatable. There are often good reasons to use alt texts that are longer than ten words. For practical reasons, mainly bugs and deficiencies in current graphic browsers, an alt attribute value works better if it is a one-liner, but for accessibility, it should simply be as long as needed.

While an asterisk would do as a bullet, it is by no means the only alternative, and there's no reason to say that it should be used. It us surely no accessibility requirement that a particular character be used.

What the A-Prompt message does not say is what alternate text is really for. Experience has shown that the word "alternate", however informative per se, is not sufficient for conveying the message that the alternate text should be written so that it works as a replacement for the image, in situations where the image is not displayed. It is a very common misconception that it should be a description; and A-Prompt's example reinforces this, since the suggested "alt" text is "rex the calico cat", which can hardly carry out the same function as the image (whatever that function is). It is true that alt attribute values sometimes need to be descriptions only, in lack of anything better, but then the common and useful convention is to enclose them into brackets. See my notes on alt attributes.

Other problems with A-Prompt include the following:

• Tables. A-Prompt asks you to specify whether your table is really tabular or for layout. This is a bit tedious, and there doesn't seem to be any way to tell in the document itself about this, so if you repeatedly work with a page using A-Prompt, you will always be prompted for the same information.
• Images: A-Prompt warns "this image may flicker" about every image. Every time. For some odd reason, it does not warn, for example, that this image may lack sufficient contrast, or may rely on color only, for example. Thus, the warnings about flickering are rather misleading.
• Repetitiveness is a problem with other features too. When you tell A-Prompt about something in the dialogue, you will need to repeat that whenever you open the page for checking on A-Prompt. The program misleads you by showing checkboxes that you can check to tell that you have manually checked something; it won't remember in later runs what you've said. (A checker could add pseudocomments or other invisible information into the document about such things; but A-Prompt doesn't.)
• Language codes. When you've omitted lang attribute from the <html> element, A-Prompt asks you to specify one, and it has a menu of languages to select from. But what it actually inserts is a three-letter codes, like eng for English. However, RFC 3066 mandates the use of two-letter codes (like en) for those languages that have one. Besides, three-letter codes have even narrower support than two-letter codes. Moreover, A-Prompt fails to process e.g. <html lang="en-US"> correctly. It claims that the document has failed to specify a language. That is, it not only fails to recognise this language code, it pretends it isn't there at all.

Due to the way A-Prompt works, it is best to reserve enough time for checking a document with it, so that you can fix it up to the accessibility level you have chosen as the goal. It might be a good idea start with priority level 1, though, even if your goal is higher, since otherwise the amount of messages can be confusing.

A word on accessibility stamps

Many authors decorate their pages with icons that indicate compliance with this or that W3C recommendation or some test that is said to verify such compliance. Would you use an icon that indicates that your document has been spelling checked? Why? It is sometimes said that icons saying "Valid HTML!" or "W3C WAI-AAA WCAG 1.0" are useful for promoting accessibility. Well, maybe some visitors who are themselves Web authors get the idea. What about all the rest?

Actually, as mentioned above, W3C's own main page carries the WAI-AA conformance icon, yet objectively fails to comply with one of the criteria, namely 12.4 Associate labels explicitly with their controls. Or maybe we should take the corresponding statement in the HTML techniques document as permitting lack of label markup, when it says, rather oddly: "A label is implicitly associated with its form control either through markup or positioning on the page. The following example shows how a label and form control may be implicitly associated with markup."

What about your documents? Let's suppose that you have any quotations within text. If you use quotation marks around them, you don't comply with WCAG 1.0, guideline 3.7, which tells you to use markup for quotations, with the note on technique that in HTML you should use <q> markup for "inline" quotations. And the definition of the <q> element tells you not to use quotation marks along with it. I do not advice you to use <q> markup, since it actually reduces accessibility, since so many browsers ignore the markup, so that the quotation is not indicated as quotation at all. But if you don't use it, as most authors don't, and you have any inline quotation, then you don't comply with WCAG 1.0 requirements. So why would you claim that you do?