Accessibility Supported?

I recently reviewed a couple of PDF forms for compliance with WCAG 2 and the inconsistent way the test screen readers handled these forms threw up several interesting issues or questions.

Sorry I am unable to show the forms or describe them in detail due to a NDA; however in general they both contain a lot of radio buttons and checkboxes as well as text input fields. Many of the radio buttons or checkboxes cause new sections of the form with additional questions to be presented when the button/box is selected. I am sure you know what I mean.

At the outset, I would like to make it clear that although this test involved PDF forms, my comments are not intended as a reflection on the general use or accessibility of PDF and could equally apply to any other web technology.

WCAG 2.0 has been developed with the idea that it is technology neutral so that it can apply to any current or future web technology. This is underpinned by the concept of “accessibility-supported” and the associated requirement that “only accessibility supported ways of using technologies can be relied upon to satisfy the Success Criteria” of WCAG 2 Guidelines.

I tested the forms with a friend who is a highly experienced screen reader user. We used JAWS 9.0 and Window Eyes 7.01 to see how well we could use the forms when they were presented using Adobe Reader 9.0 and Acrobat 8.1.2.

In effect, we were testing to see if the PDF forms complied with WCAG 2 if we assume Adobe Reader 9.0 and Acrobat 8.1.2 are accessibility supported technologies. (From past experience, we knew that it was possible to make PDF forms which can be successfully accessed and used with the technology combinations that we were using.)

During the testing we observed a number of interesting things, including:

  • When using Adobe Reader 9.0, sometimes a particular button or checkbox would be reported and sometimes the same button/checkbox would for all intent be basically ignored. Sometimes forcing the page to refresh/reload would cause an unidentified button/checkbox to be identified, but not always.
  • With Reader, when the buttons/checkboxes were identified, both screen readers always reported the state of the radio buttons or checkboxes as being unselected, even when they were selected.
  • With Acrobat 8.1.2, the screen readers were much more reliable in identifying the checkboxes and radio buttons as well as their state.
  • When using either Acrobat or Reader with both screen readers, sometimes the selection of a checkbox or radio button would cause the focus to move to the top of the page as if the page had refreshed.

When we later rechecked the forms using Adobe Reader 9.1, we found that Reader was able to reliably identify and report the status of checkboxes and buttons with both screen readers. However a number of other problems remained including the occasional and unexplained shifting of focus to the top of the page.

It seems to me that when it comes to using WCAG 2 in the wild as an accessibility compliance determination tool our experiences raised some interesting questions:

Consistency:

WCAG 2 contains “testable” Success Criteria, which are normative, and there appears to be a general belief that when evaluating a site or application, all testers will obtain the same or very similar results for the different Criteria. While there are a number of Conformance Requirements, none appear to relate directly to questions of consistency or reliability.

For example, let’s considered Success Criteria 1.3.1 “Info and Relationships: Information, structure, and relationships conveyed through presentation can be programmatically determined or are available in text.”

  • If 100% of the time I am able to associate a label with a checkbox for example, then the answer is simple, similarly if I am never able to do so. But what about situations like the one just described, where sometimes this happens and sometimes it doesn’t?
  • How many times should someone test in order to make the determination that something does or does not comply?
  • And then, there is the problem of ensuring the testable results are repeatable. If something does not perform consistently, I might test it and experience no problems but the next person might not be so lucky.

Accessibility Supported Technology versions:

The W3C provides information for “Understand Accessibility Support” and the level of support needed.

There is much to applaud in the accessibility supported approach and I understand the WCAG Working Groups’ desire for flexibility but I feel the reluctance to provide guidance in this area is a bit of a cop-out: “The Working Group, therefore, limited itself to defining what constituted support and defers the judgment of how much, how many, or which AT must support a technology to the community and to entities closer to each situation that set requirements for an organization, purchase, community, etc.

  • Who is going to determine what versions of a web technology are accessibility supported?
  • What do we do about “version-shopping” by organizations in an effort to get a compliance tick of approval?
  • What happens when there are competing determinations about which technologies and technology versions should be deemed supported? This could be within a jurisdiction and across jurisdictions.

Assistive Technologies:

As with Accessibility Supported web technologies, WCAG and the supporting documents do not indicate which assistive technologies or which versions of those technologies need to be able to access the material. Rather, the Working Group provides some general advice while recognising, “Individual authors will not usually be able to do all of the testing necessary to determine which ways of using which Web technologies are actually supported by which versions of assistive technologies and user agents. Authors may therefore rely on publicly documented compilations that document which assistive technologies support which ways of using which Web technologies.

  • Who will prepare these lists? And, how many are available today?
  • How will authors, clients and regulators know which lists to trust? The Working Group suggests that only documents that provide accurate information will achieve market recognition, in other words, we leave it up to the market. Not a great time to sell the notion that the “market” is a great source of regulatory advice.

NOTES:

BTW I used the editable “WCAG 2 Checklist“, which I prepared a short while ago, to record the results and it seemed to work pretty well.

Although we were testing for compliance with WCAG 2 it should be noted that the Australian Human Rights Commission still references WCAG 1.0 as the bench mark for web accessibility in Australia. I expect we will move to WCAG 2 in the near future.

I wish to fully acknowledge W3C ownership of WCAG 2 and all the supporting documents and express my respect and gratitude for the work of WCAG Working Group.

3 Comments

  1. Hi Roger, i realise you told me off lits that the PDFs being tested were not able to be shared, but it would be really useful to have a PDF to test that exhibited the results you found.

  2. The crux of the problem is that W3C cannot set a benchmark. Guidance on particular technologies, or particular versions of technologies, will quickly become outdated. WCAG cannot deal with such specifics.

    As for who should prepare these lists, I believe that the accessibility community can and should take an active role. People like you will no doubt contribute. This blog post is a good start.

    If people of influence actively contribute to the debate, we can prepare these lists as a community. We need a body of knowledge that we can reference as current best practice.

  3. I agree we need to get lists of recognised accessibility supported technology uses as soon as possible. I am particularly interested in the question of jurisdiction. The web, as we all know is world wide, so what happens if one college, company, state or country declares particular uses of a technology to be accessibility supported while another entity takes a different view and decides all uses of that technology can not be considered accessibility supported? With the specifics of how to make something accessible in the advisory techniques, rather than the normative document, I am concerned we might see an increase in confusion and cross-jurisdictional disputes. The WAI discussion about the accessibility supported questions I raised is at http://lists.w3.org/Archives/Public/w3c-wai-ig/2009AprJun/subject.html

Leave a Reply