Paul R. Bohman

George Mason University

Note: This is a working paper, subject to future revisions.
Last edited: July 20, 2007.
Comments welcome.

Abstract: Though suppositions and recommendations abound, evidence-based research focusing on how to make web content accessible to people with disabilities is scarce. Only seven studies could be found that 1) directly observed people with cognitive disabilities, and 2) addressed cognitive disability access from a web content design perspective. Most of these studies lack the rigor and/or focus to provide any firm evidence or rationale for guidelines, notwithstanding the many recommendations promoted by these studies. Certain broad observations surfaced in multiple studies, such as user difficulties typing text, understanding context, and navigating within web sites. This literature review summarizes and critiques the seven relevant studies, then discusses potential reasons for the dearth of research-based studies about cognitive disability access to the internet.

Research-based evidence about web development techniques that improve cognitive accessibility access to the web is scarce (Bohman, 2004; Boscarol, 2006; Carey, Friedman, & Bryen, 2005; Keates et al., 2007; Robertson & Hix, 2002; Small et al., 2005). An ambitious call for "an international research and development agenda" designed to move "toward an information society for all" (Stephanidis et al., 1998) inspired a modest amount of research, but physical and sensory disabilities (blindness, in particular) proved more popular than cognitive disabilities among researchers. Among those who researched cognitive disabilities, several investigated software interface design (e.g. Dickinson, Newell, Smith, & Hill, 2005), browser enhancements (e.g. Davies, Stock, & Wehmeyer, 2001), assistive technologies such as text readers (e.g. Gregor & Newell, 2000), or web services (e.g. Iaccarino, Malandrino, & Scarano, 2006; Kurniawan & Zaphiris, 2005),  Only a handful investigated techniques directly relevant to web developers, such as techniques for designing, organizing, displaying, or conveying content on web sites. Other writers have found that most research regarding computers and individuals with cognitive disabilities focuses on either assistive technologies or how to use technology in special education settings (Robertson & Hix, 2002). Outside of the research literature, written material available on the subject tends toward advocacy and position papers that call for the "inclusion of cognitive disabilities in the web accessibility movement" (Seeman, 2002), appealing more to (arguably important) egalitarian notions of social justice than to scientific notions of methodically-accumulated evidence. Some authors (e.g. Caldwell, 2006; Kolatch, 2000; Shneiderman, 2000) provide broad overviews of seemingly self-evident assertions that, in fact, derive from untested assumptions. Many of these assumptions draw upon analogous—but only indirectly-related—bodies of research, lending them a borrowed appearance of authenticity. As the web accessibility field matures and grows in sophistication, and as world governments continue to mandate accessibility (Thatcher et al.), borrowed appearances of authenticity are increasingly inadequate. If the field is to move beyond borrowed appearances, it must first examine the inventory of research—however small. It must distinguish between real and perceived needs, and between effective and ineffective design techniques. Judgments and opinions are allowable and important, but not as substitutes for research-based evidence. This literature review presents a critical overview of research related to web design techniques that benefit users with cognitive disabilities. The intent is to define a foundation upon which to construct a solid body of future research.

Criteria for inclusion in this literature review

The central question of this literature review is "what can web developers do to make web content more accessible to people with cognitive disabilities?" Only studies that seek to answer some aspect of this question are included here. Within this broad framework, the key criterion for inclusion is the direct observation of people with cognitive disabilities. This direct observation can take the form of experimental research, focus groups, case studies, usability testing, or any other format. This criterion excludes expert opinion, advocacy, theory, and studies that seek answers indirectly, for example by researching web developers, those who work with people with cognitive disabilities, or any other population that does not personally experience a cognitive disability of some kind. Some populations, such as ageing adults, experience a higher incidence of cognitive disabilities than the population at large. While research of these populations is important, studies of these populations were not included unless the researchers focused specifically on cognitive disabilities in their subjects.

Considering the paucity of research in the field—and in the interest of inclusiveness—this literature review employs an expansive interpretation of the term "cognitive disability." Severe disabilities that compromise a person’s ability to live independently, as is often the case in individuals with dementia, constitute one end of the scale. The other end of the scale includes mild intellectual or cognitive conditions, such as dyslexia, learning difficulties, and attention deficit disorders, even if these conditions are not always formally classified as "disabilities." Both developmental and acquired disabilities are relevant. Psychological and emotional conditions, though often cognitively disabling, are not represented in this literature review.

 Because the focus is on web content, studies related to browser enhancements, software interfaces, assistive technologies, and any other area not considered web content were excluded from this review. This is not to say that such studies are irrelevant. In fact, one can argue that web sites contain both content and an interface. Similarly, the line between web browsers, assistive technologies, and web content is sometimes blurry, as they can have overlapping roles. Even so, the intended focus is on web content—or web "pages"—as seen in the main viewing area of the browser. Also excluded from this review were studies which provide insight into the nature of cognitive disabilities (unless there was a web content component), or which discuss access issues for non-web technologies such as personal digital assistants, cell phones and other telecommunication devices, operating systems, and so on.

The search terms used for this review included web (and the related word internet), accessibility (and the related words access, usability, and usable), cognitive (and related words intellectual, developmental, mental, retardation, retarded, learning, dyslexia, dyslexic, aphasia, aphasic, autism, autistic, Down Syndrome, attention deficit disorder, attention deficit hyperactivity disorder, ADD, and ADHD), and disabilities (and related words disability, disabled, impairment, impaired, handicap, and handicapped)[1]. An attentive reader will notice that while most of the search terms related to the word cognitive are generic terms not specific to any particular clinical diagnosis, a few terms, such as Down Syndrome, dyslexia, and ADHD are in fact clinical diagnoses. It would be fair to ask why some clinical terms were included and not others. To some extent, this decision was arbitrary. It would have been impractical to include every possible type of cognitive disability by name in this review. At the same time, it would have been irresponsible to exclude some of the more common names, knowing that studies about a particular disability may be more likely to use the specific name than to use only the generic categories. A future review of literature with a more expansive list of clinical terms may reveal data not presented in this review. 

Searches were conducted in English in the following search engines and databases (listed in alphabetical order): Academic Search Premiere, ACM, Computer and Information System Abstracts, Digital Dissertations, ERIC, Google, Google Scholar, IEEE Xplore, INSPEC, JSTOR, LexisNexis, PsychInfo, Questia, and Springer LINK. As a gateway to multiple databases, Google Scholar proved especially useful. Within the specific databases, ACM publications contained the richest collection of web accessibility-related resources, followed by Springer LINK. From an initial list of 181 publications, a total of seven research studies met the criteria of this review. Each of these studies is discussed below, with the most recent publications first.

A Review of Research-Based Studies

Study 1: "Use of Electronic Technologies by People With Intellectual Disabilities" (Carey, Friedman,& Bryen, 2005)

Purpose and Background

The researchers of this study surveyed 83 "adults with intellectual disabilities" in order "to examine the use of and preferences related to electronic organizers." The primary research question seems to be what factors either encourage or discourage the use of electronic technologies among people with cognitive disabilities. This research question is only tangentially related to the topic of this literature review, but some of the results are pertinent.

Subjects

The subjects included 83 "adults with intellectual disabilities." Of these 83 individuals, 20 of them (15 female, 5 male) used the internet. According to Table 2 in the article, among these 20 there were a range of self-reported difficulties, including copying information correctly (7), staying organized (10), saying things clearly (5), and reading (confusingly, the authors report 27 respondents on this measure, though only 20 individuals fit this category). Other difficulties mentioned in the article but not explicitly linked to the subjects that used the internet included spelling, reading the news, using a calendar, seeing "ordinary news print," learning new information, using a clock, holding things with the fingers, pushing buttons with the fingers, and hearing "normal conversation."

Method

Subjects answered a survey about their use of electronic technologies, which included questions about which technologies they used, why they did or did not use them, and what difficulties they experienced using them. Subjects were not observed interacting with the technologies.

Results

According to this study, the most common uses of the internet among those surveyed included "searching for social event information (65%), playing Internet games (60%), and reading the news (55%)." When asked what would make the internet easier to use, respondents mentioned "making it easier to enter commands and addresses (4 participants); improving interaction with the user, including word prediction (i.e., software that predicts what word one is typing based on the initial letters, thereby reducing the typing involved), voice input and voice output (3); and speeding up connections (2)."

Critical Analysis

At first glance, the information gathered from this survey reveals only superficial data of limited use to web developers due to the vagueness of the data reported. This study is not rigorous enough to provide sure answers about anything. But with a little creative interpretation, the data are worth considering, if only to suggest future research directions. Of the recommended changes that would make the internet easier to use, the first three could almost be summarized as restatements of the same principle: facilitate error-free user input. Some respondents apparently found it difficult to input information in the address bar, and in forms (presumably this is at least part of what is meant by the term "commands"). The recommendation of "improving interaction with the user" is too broad to meaningful (it is as unspecific as saying the page must be made "accessible"). The suggested methods—word prediction and voice input are more concrete. Unfortunately, web developers will find these methods easy to ignore. Word prediction and voice input usually are assumed to be the realm of browsers, assistive technologies, and third-party software add-ons. It would be inefficient and unrealistic to expect every web developer to incorporate word-prediction or voice recognition technology in every web site. The most important message from this group of responses may be the larger principle already mentioned: facilitating error-free user input, presumably by reducing the need to type responses, marking the choices as easy as possible, and by making the physical action of making the choices as physically undemanding as possible. The last suggestion mentioned in the study, of "speeding up connections" sounds completely out of the realm of control for web developers, but perhaps it is not, at least not entirely. At the risk of reaching beyond the available data, a broad principle worth investigating would be the importance of web content responsiveness. Web content that loads slowly, or which does not offer feedback quickly at important moments may reduce the quality of the user experience for users with cognitive disabilities, especially for those with attention or memory deficits.

Study 2: "Web Accessibility for People with Cognitive Disabilities" (Small, Schallau, Brown, & Appleyard, 2005)

Purpose and Background

This study aims to "examine how individuals with mild to moderate DCD [developmental cognitive disabilities] navigate W3C accessibility compliant web sites," presumably to see how useful the W3C guidelines are in providing an accessible experience for users with cognitive disabilities. The authors do not state what level of compliance the two web sites claimed, nor do they describe any of their accessibility features. The study focuses on navigation issues within the web site, using the following four determinants of successful navigation, "1) situational awareness – a person’s knowledge of his or her surroundings, 2) spatial awareness – a person’s awareness of how content is located in relationship to navigational devices, 3) task-set switching – a person’s ability to quickly move from one task to another, 4) anticipated system response – a person’s perception of how the system should appropriately respond to a user’s action."

Subjects

Participants were 27 adults (13-14 split on gender) diagnosed with "mild to highly functioning moderate" DCD. The authors describe the abilities of the subjects as follows:

Based on self-reported data gathered in the focus groups, 78% of the participants were occasional to daily users of computers… On the preliminary tasks all but one of the participants could identify the mouse and most could center the mouse (77%) on a designated target. However, only 52% of participants could activate the mouse on a designated target with out prompting or help. When requested to type in URLs only one third (34%) of the participants could do so without prompting and the remaining participants required either extensive step by step prompting or the researcher to execute this step. Overall, only 19% of the participants could successfully navigate either site without extensive guidance.

Methods

Researchers studied video and audio recordings of subjects using web sites under controlled circumstances in the presence of an investigator who gave participants specific tasks to perform, and asked predetermined questions throughout the process.

Results

The authors noticed difficulty recognizing links (especially when links were not underlined text and not perceived as "clickable"), activating links, recognizing when they had arrived at the right page, typing (including lack of familiarity with the "Backspace" key), scrolling (52% did not scroll without prompting), returning to the home page of a site (some clicked on the browser’s home button; others closed the browser), and reading instructions. Not all participants understood that they were in a web environment. Direct intervention by the investigator was necessary across many of the tasks for many of the users. The authors conclude that many factors limited the accessibility of the tested web sites to their subjects. The implied conclusion is that the W3C accessibility guidelines are insufficient to ensure cognitive disability access.

Critical Analysis

This study destroys any hope that casual observance of web accessibility guidelines is sufficient to provide an accessible experience for users with cognitive disabilities. Some of the subjects experienced challenges at every stage and in every aspect of the study. There were individual differences, but none of the tasks or aspects of the sites’ designs qualified "accessible" to all of the users. The implications are sobering. Is web accessibility for people with cognitive disabilities even possible? If the W3C guidelines are insufficient, can any guidelines or principles be sufficient? Rather than accept a fatalist interpretation of these results, it might be better to think in terms of facilitating the user experience along an accessibility spectrum. For some users, full accessibility comparable to the experience of a user without disabilities is impossible, but increased accessibility along the spectrum is probably still achievable.

Study 3: Improving website accessibility for people with early-stage dementia: A preliminary investigation" (Freeman et al., 2005)

Purpose and Background

According to the authors, "the aim of this study was to produce and evaluate a website with the specific needs of those with early-stage (mild to moderate) dementia in mind." The authors identify four areas of interest: "ability to process complex information, memory functioning, attention, and the effects of overall cognitive load." They created two web sites, one designed with the anticipated needs of dementia patients in mind. The other site was used as a control. The authors provide a general description of the design principles behind their new design, but do not give examples of how these principles were implemented. The principles recommended include 1) using memory retrieval cues (such as pictures or icons combined with verbal cues) wherever appropriate, 2) keeping pages as structurally similar as possible, 3) avoiding overly complex language, 4) avoiding abstract or metaphorical language, 5) avoiding forcing users to make concurrent choices, 6) reducing the number of items on the page competing for attention (reduce content to only the necessary items), 7) and reducing the number of choices on a page.

Subjects

The participants were 5 males between 57-72 years old with early stage dementia, none of whom had used the internet before. Two of them had used computers before.

Methods

The investigators observed subjects as they navigated through 2 web sites. In addition, researchers administered a verbal questionnaire asking users to rate the site in terms of "page layout, ease of moving around the page, ease of moving between pages, feeling oriented, use of colour, size of text, ease of viewing, usefulness, and interest."

Results

Overall satisfaction was similar with both sites, and on both sites users were not always aware of the need to scroll to reveal more information. There were some significant differences favoring the new site (designed for accessibility) such as the decreased likelihood of users to feel lost or confused. Users were better able to distinguish links from non- on the old site links (bullets were sometimes perceived as links on the new site).

Critical Analysis

Although the authors provide a description of general principles of accessibility for users with dementia, readers are left to wonder how many of these principles the authors actually integrated into the design, on which pages, to what degree, and using what specific techniques. The differences between the old and new sites are not documented, so that any useful comparison for the reader is difficult. The fact that none of the subjects had used the internet before is a confounding factor that throws into question the whole study. How many of the difficulties were the result of inexperience, and how many were the result of early-stage dementia? How generalizable are the results to experienced internet users with dementia? In spite of these weaknesses, notable tentative conclusions coincide with other studies mentioned in this review: the problems recognizing the difference between links and non-links, and problems recognizing the need to scroll down, suggesting that most or all of the content should be visible in the first screen. This last observation is problematic, in that the size of the visible area of a web page depends on the screen resolution, size of the browser window (e.g. whether maximized or not, the presence or absence of browser add-ons or toolbars that decrease the viewing area, etc.), the enlargement of the font or viewing area (the proportion of visible content in the browser window usually decreases as the enlargement increases), and so on. Web developers cannot control these external circumstances. Even so, amount of visible content per page is within the control of the developers, even if hardware limitations and user preferences are not.

Studies 4 and 5: "Internet enabling design for people with cognitive limitations" (Bjorn Harrysson, 2003) & "How People with Developmental Disabilities Navigate the Internet" (B. Harrysson, Svensk, & Johansson, 2004)

Note: Although these are two separate publications, they refer to the same research, and have been grouped together here to economize space in the discussion.

Purpose and Background

"The aim of the project," according to the text, "was to gain knowledge and understanding of how people with cognitive limitations manage to use the Internet and in so doing, establish a basis for developing a method that would increase their accessibility to the Internet" (Bjorn Harrysson, 2003). These studies actually represent three separate stages of research. The purpose of the first stage was to describe "how you can build cognitive support into the design of a web page" and generate guidelines based on user testing. The purpose of the second stage was to discover how users with cognitive disabilities interacted with the Internet Explorer browser. The purpose of the third stage was to test an experimental method of accessing the web by using a pen input device that could automatically enter in web addresses and other text into the web site, bypassing the need for the user to input this information by hand. This method was invented as a result of observed difficulties of users with cognitive disabilities typing text.

Subjects

Five people participated in the first stage. The paper does not describe the characteristics of these subjects. Seven different people participated in the second and third stages. These subjects (5 female, 2 male) were age 15-25, with the exception of one subject who was 44. Four had mild developmental disabilities. Three had moderate developmental disabilities. All had computer experience, but little or no internet experience.

Method

In all three investigations, the authors emphasize the collaborative nature of the process, involving the subjects as much as possible. Subjects reported feeling privileged at this level of involvement, and for taking upon themselves the role of "test pilots," as they were called. For the first stage, the author used an eye-tracking device to record the eye movements of the subjects while accessing web content. The author chose this method due to the communicative difficulties experienced by some of the participants. In the second stage, the researchers videotaped subjects as they used web sites chosen by the researchers to coincide with the personal interests of the subjects. The researchers studied how participants used 1) forward, back, up, down, open, and close, 2) links, 3) favorites, 4) keywords for searching and search results, 5) URL addresses. The methods for observing subjects in the third stage are not described in detail, but the text describes the experimental solution inspired by the previous two investigations. This solution (dubbed by the author as ACCeL, for Assistive Computer Control easy to Learn) involves using a handheld pen-like device called C-Pen 10 to scan in web addresses and other text so that users do not have to type the text. The authors experimented with using pictures instead of text, allowing users to go to a web site by pointing the device at a picture. The assumption was that pictures would be easier to remember than text, thus making navigation to favorite sites an easier process.

Results

As in some previous studies mentioned in this review, text input proved problematic for the participants, and users felt overwhelmed when presented with many choices. Unlike previously reviewed studies, the authors of this study emphasized the importance of graphics and photos as comprehension aids. The first stage yielded 9 guidelines, though there is almost no discussion of the observations which led to these guidelines: 1) "design self-instructional web sites that allow the user to be interactive" [the author does not explain what it means to "allow the user to be interactive"], 2) allow user personalization and adaptation 3) "do not present more than 10-15 items on the same screen" [the author does not define the term "items," nor does the author explain the data supporting this specific numerical range], 4) "facilitate the users’ ability to get back to the starting point," 5) use graphic illustrations in combination with illustrative short words, 6) "use symbolic illustrations to represent categories, and photos to represent personal and local information," 7) place important information in the middle of the screen, and make it large, 8) "design clear-cut links with blank space around them," 9) "use general tools that are familiar to the user" [the author does not define the term "tools"].

The second stage yielded a list of common difficulties among participants: 1) "entering query text in search engine search boxes and browser web page address bars," 2) "making a selection from a large quantity of text," 3) "[understanding] the text due to deficiencies in reading ability."

The articles do not discuss the results of the third stage so much as they describe the promising potential of the ACCeL system (using the C-Pen 10) as an assistive technology for people with cognitive disabilities.

Critical Analysis

The list of guidelines may prove useful. Unfortunately, the link between the data gathered and the resulting guidelines is tenuous, at least as it is presented in these articles. The guidelines are themselves unclear and ambiguous, reducing their usefulness. The inexperience of the subjects in using the internet is a potential weakness of this study, though the authors note that participants "spontaneously used the tools in the Microsoft Internet Explorer Web Browser." They "used the close function in the upper right-hand corner" and "navigated forward and back without difficulty" and even had no difficulty using the scroll feature, in contrast to other studies. The ability of the participants to learn quickly to interact with a web browser may owe to their relative youth and previous experience with computer technologies.

Study 6: "Designing accessible Web-based instruction for all learners: Perspectives of students with disabilities and Web-based instructional personnel in higher education" (Roh, 2004)

Purpose and Background

This is a doctoral dissertation designed to 1) "discover how students with disabilities and WBI [Web Based Instruction] personnel in postsecondary educational institutions perceive web accessibility…" 2) "reveal the web accessibility issues students with disabilities and WBI personnel have experienced when either taking or designing WBI courses" and 3) "suggest some feasible solutions…"

Subjects

Subjects with various types of disabilities participated in this study. Of relevance here is a case study of one college student, named Cathy, with learning disabilities, ADD, dyslexia, and dysgraphia (difficulty writing) who uses the Kurzweil program to read text to her, and takes advantage of audio tapes of books from the Blind and Dyslexic Association.

Methods

Data were collected through a combination of 1) individual and focus group interviews, 2) videotaped observations, and 3) document analysis (accessibility evaluations of web content within an online course).

Results

Difficulties experienced by Cathy included 1) losing her place when entering large amounts of text in small text boxes which require scrolling, 2) technical problems, some of which may be interpreted as lack of technical expertise or computer savviness on her part, 3) navigating complex site structures, 4) distinguishing clickable from non-clickable images (due in part to inconsistent functionality on web sites), 5) remembering the instructions from the instructor regarding which image links led to additional course content, 6) reading text when contrast between text and background was low, 7) accessing graphics without alt text with her Kurzweil reader,and 8) accessing content on specific browsers and computers which were not always available to her.

Based upon these difficulties, Cathy and the researcher decided on the following list of recommendations: 1) provide lecture notes ahead of time 2) ensure that web-based content works in multiple browsers and operating systems, 3) provide a succinct summary at the end of each lesson, 4) provide clear and accurate instructions, 5) keep layout simple and clear, 6) minimize the number of windows to be opened, 7) minimize file sizes, 8) provide content in multiple file formats, 9) provide appropriate page, headline, and sub-headline titles, 10) avoid graphical text as much as possible, 11) ensure sufficient contrast between text and background color, 12) do not provide unnecessary information.

Critical Analysis

The author provides a thorough list of challenges experienced by Cathy. Some of the issues raised by Cathy, such as the need for cross-browser and cross-platform compatibility are not disability-specific. Reading through Cathy’s comments in the case study reveals her lack of patience or expertise with technology, undoubtedly due at least in part to her disabilities. Even so, the nature of Cathy’s comments—and the way they are meticulously documented—invite readers to wonder 1) whether Cathy’s frustrations might be more accurately ascribed to her temperament or other psychological or emotional characteristics, as opposed to her disabilities per se, and 2) whether Cathy was exaggerating or sensationalizing her claims to some degree, when given the opportunity to freely express her frustrations to a sympathetic listener. These are subjective questions of interpretation, unanswerable at their core, and perhaps even irrelevant if the correlation between her psychological traits and her disabilities proves to be inseparable. The question is raised, though, to underscore the potential complexity of the nature of cognitive disabilities themselves. In fairness to Cathy and the researcher, asking whether Cathy’s experience was psychological or exaggerated lies at the heart of her frustration. Her instructors did not always believe that Cathy needed accommodations, thus straining the instructor-student relationship and forcing Cathy to defend herself against their incredulity. The continuous scrutiny would test the patience and civility of anyone. Credit the researcher with giving voice to Cathy’s dissatisfaction, even if the emotions do not always translate directly into actionable web design techniques.

Cathy’s disabilities are minor in comparison to most of the other disabilities included in this literature review. With accommodations, she could function at a college level. Even so, issues of being able to distinguish the clickability of objects on a page, difficulty with text input, and the need for simple, clear layout are common elements to other studies in this literature review.

The list of recommendations provided by Cathy and the author is informative, though the association between the eight difficulties and twelve recommendations is not always clear. Which recommendations are meant to solve which difficulties? Two of the challenges mentioned (losing one’s place when entering large amounts of text in small text boxes, and distinguishing between clickable and non-clickable items), are not addressed at all in the list of recommendations.

Study 7: The effects of implementing Web accessibility standards on the success of secondary adolescents with learning disabilities (Opitz, 2003)

Purpose and Background

According to the author, "the goal of the study was to determine the effects of following or not following [Section 508 accessibility guidelines] in the creation of instructional, web-based learning modules for adolescents with and without disabilities."

Subjects

This study involved 60 secondary school students, ages 14-18. Thirty students with learning disabilities were compared to 30 students without learning disabilities.

Methods

The researcher created two similar web sites, one in accordance with accessibility guidelines, the other with violations of these guidelines. A user study was conducted to measure accuracy of responses to questions about the web site and length of time required for responses. Students also completed an attitude survey. To distinguish the sites, the "inaccessible" site 1) omitted alternative navigation (a "breadcrumb trail"), 2) utilized frames, 3) incorporated scrolling and blinking text, 4) omitted bulleted lists and other textual organizers, 5) utilized animated graphics, 5) lacked alternative text and captions for images or photographs.

Results

Accuracy was better for the "accessible" web site than for the "inaccessible" site for all users. Response time was not significantly different. Students without learning disabilities were more accurate and quicker than students with learning disabilities. Attitudes about both sites were equally positive.

Critical Analysis

This study is significant in that it is the earliest study that meets the criteria of this literature review. The inclusion of full screenshots of the two different web sites allows for an easy visual comparison between the two sites mentioned in the study. One of the few obvious differences between the sites is the use of bold text to highlight keywords. This feature alone may account for the difference in accuracy scores between the sites, and would be worth investigating further.

One problem with this research design is that the factors used to distinguish the "accessible" site from the "inaccessible" site are probably too subtle. The number of links was the same, the amount of textual content was virtually the same, and illustrative images are used in approximately the same locations and layout. None of the children used screen readers, so the addition of alt text likely had little or no impact. One could argue that the consistent appearance between the two sites is necessary to avoid confounding variables, but in this case, the differences seem more guideline-driven and less informed by user needs. In this sense, this study is related to Study 2 (Small et al., 2005), discussed previously. Both seek to verify whether the W3C guidelines improve accessibility for the research subjects. Small et al. conclude that the guidelines are insufficient. Opitz concludes that they can make a difference, without asking directly whether the difference is sufficient.

Other Studies

At least one other study (Lepistö, 2004) would have met the criteria for this literature review if an English version were available, but it is available only in Finnish, a language not accessible to this author. Other studies in other languages may prove useful, but will remain for other researchers to discover.

What do we Know, Based on the Available Research?

The studies themselves are uneven in quality, applicability, and scope. One of them obtained only minimal web accessibility data, incidental to a broader research question (Carey, Friedman, & Bryen, 2005).  Two of them (Freeman et al., 2005; Small, Schallau, Brown, & Appleyard, 2005) describe themselves as preliminary or exploratory studies, for which no follow-up data pertinent to this literature review have been published in the sources searched for this review.

Only two of the studies (Freeman et al., 2005; Opitz, 2003) investigate the effects of web development techniques designed to improve accessibility. The Freeman et al. (2005) study provides insufficient information about the differences between the "more accessible" and "less accessible" sites to be able to reproduce the effects of the study. The Opitz dissertation (2003) provides sufficient information in this regard, but both studies fail to isolate the effects of any particular design feature on the more accessible sites, making interpretation difficult. The observed positive effects of the more accessible sites could be due to any number of a large range of factors, or a combination of several of them.

The other five studies report on general difficulties experienced when using web sites not designed with accessibility in mind. Studying this list of difficulties can lead to design decisions that attempt to avoid the problems mentioned. Though valuable, this information does not provide a concrete set of proven techniques to emulate. It provides simply a list of things to avoid doing.

The researcher’s conclusions conform to the data only in a loose sense. The tendency among all of these studies is to generate a list of guidelines or recommendations, whether or not any specific data provide sufficient evidence to warrant the recommendations. Attempts to match observations to recommendations for this literature review were not particularly useful, due to the vagueness or absence of such associations within the research studies themselves.

Of the seven studies, only one researcher, Harrysson (2004, 2003) is represented by more than one publication, and the two articles published by this researcher discuss the same study. No studies meeting the criteria of this literature review were published during 2006 or 2007 (to date). This indicates that interest—or at least the commitment to research—is low even among researchers who have investigated the topic. A few of the researchers reviewed here have published more extensively on other aspects of cognitive disability research, none of them has studied web content accessibility long enough or in-depth enough to establish any kind of reputation among their peers with regard to the specific criteria of this review. Under these circumstances, conclusions are few and tentative.

Why the Scarcity of Empirical Research?

Seven studies over a period of six years do not constitute a meaningful body of literature. The combined total of subjects in all of these studies is 95, counting only those with cognitive disabilities studied in a web context (20 in the first study, 27 in the second, 5 in the third, 12 in the fourth and fifth, 1 in the sixth, and 30 in the seventh). There are other writings about cognitive disability access to the web, but only the seven cited here meet the criteria of this literature review.

Why such a small number of studies? Do researchers not find the topic interesting, compelling, or important? Is there a bias against people with cognitive disabilities among researchers? Do researchers share the skepticism of some who "question whether there is actually any point in trying to design websites for people with dementia [or other cognitive disabilities]" (Freeman et al., 2005)? Knowing that many people with cognitive disabilities require assistance to use the internet, do they assume that people with cognitive disabilities have less of a need to access the rich sources of information available on the internet? Do they assume that people with cognitive disabilities will always be unable to access mainstream web sites effectively, in essence abandoning the research before it has begun? This literature raises these types of questions.

Other technologies

Researchers may believe that the more promising lines of research address access to other forms of technology, such as cell phones (Brown, Harniss, & Johnson, 2006; Carey, Friedman, & Bryen, 2005),  PDAs, televisions, DVD players, cameras, home appliances, and other personal consumer devices (Brown, Harniss, & Johnson, 2006; Hanson, 2004; Keating & Ames, 2004; Lee & Phillips, 2002; Pollack, 2005; Salem Darrow, 1995),. These devices may be seen as more personally relevant to the day-to-day living for this population. There may be truth in this notion, but people with cognitive disabilities have demonstrated enthusiasm for internet use (Lepistö, 2004; Lepistö & Ovaska, 2004), especially those whose disabilities are mild.

Other approaches

Some researchers may feel that web content may not be the best place to address the needs of people with cognitive disabilities. They may believe that there is more power in developing better assistive technologies, web services and transcoders, or browsers. Undoubtedly these avenues of inquiry are valuable. It seems unlikely, though, that these solutions could ever be complete. Just as it is impossible for automated software tools to consistently generate contextually-meaningful alternative text for images, it is unlikely that tools can adequately compensate for web content that lacks qualities that may be necessary to people with cognitive disabilities, such as clarity, simplicity, organization, and so on.

Aesthetic opposition

Researchers may anticipate—or fear—a high level of resistance to guidelines that impose major restrictions on the "look and feel" of web designs. Whereas making web content accessible to people with blindness rarely requires visible alterations to a web site’s design, accommodations for people with cognitive disabilities may require radical changes, especially for users with moderate or severe cognitive disabilities. Too much resistance from web developers could create a backlash. In an attempt to bypass this resistance, researchers may divert their attention to less controversial lines of research, such as assistive technologies, browser adaptations and other special technologies that attempt to compensate for the accessibility flaws of existing content. One of the downsides to this approach is that it seems to absolve web developers of the need to add accessibility features to their content. Web developers may appreciate this level of freedom, but complete freedom can be counterproductive, severely limiting the compensatory effect of assistive technologies.

Denial

One troubling consequence of this uncertainty is a form of denial that some kinds of cognitive disabilities are real, or that they should be accommodated. When a student at Capella university filed a lawsuit claiming that the online courses were inaccessible to him due to his cognitive disabilities, he faced strong and swift opposition, anger, and disbelief, not only from the university, but among advocates of web accessibility as well (Carnavale, 2005; WebAIM, 2005). The merits and/or weaknesses of his case notwithstanding, the intensity of the resistance to his basic premise—that a student with cognitive disabilities is entitled to accommodations—is disheartening. Cathy, the student with learning disabilities in Study 6, recounted her own difficulties in convincing instructors that she should receive the accommodations she requested. Is it possible that researchers are equally reluctant to recognize cognitive disabilities as a legitimate condition?

Inadequate advocacy

Perhaps advocacy efforts on behalf of people with cognitive disabilities have fallen short. Self-advocacy groups of people with cognitive disabilities generally are less visible, less vocal, and less politically active than self-advocacy groups for people who experience some other types of disabilities such as blindness or deafness. Making the needs of people with cognitive disabilities more visible—as this literature review attempts to do—may increase interest among researchers.

Methodological concerns

Cognition is a complex, ill-defined domain in which answers often raise as many questions as they resolve. Categorizing, quantifying, and interpreting thought patterns can prove to be elusive goals. Ambiguity is the rule, rather than the exception. On a pragmatic level, researching people with cognitive disabilities is fraught with methodological and ethical dilemmas. On the most basic level, people with cognitive disabilities can be difficult to identify and recruit for research. Documenting and categorizing a person’s cognitive disability can be a challenge. Even when clinical diagnoses are available, to what extent is a clinical diagnosis meaningful in the context of web accessibility? Are assessments of functional disabilities (Bohman, 2006) useful? To what extent can cognitive disabilities be studied as separate phenomena, considering that many people with cognitive disabilities experience significant concurrent perceptual and/or motor disabilities (Oross & Woods, 2003)? How can researchers accurately represent the needs and views of a population that may lack many of the communication skills necessary to articulate their own needs and views? How reliable is the "evidence" in such research? How much of the data is merely the interpretation of the researchers? Can people with more severe cognitive disabilities truly give informed consent as research subjects? These are legitimate concerns. Nevertheless, they are not insurmountable. A few studies (e.g. Barry & Pitt, 2006; Williams, 2006; Williams & Nicholas, 2006) have addressed some of these concerns, and have provided some practical solutions. More discussion and research into the methodological and ethical concerns would benefit the field.

Usability vs. accessibility

The definitional conflict between accessibility and usability is a frequent topic of debate among web accessibility professionals (Petrie & Kheir, 2007). This is especially true where cognitive disabilities are concerned. If it is true that "you get a pretty good approximation [of what it is like to access the web with a cognitive disability]...if you say...all the things that everybody has trouble with about the web, people with cognitive disabilities have trouble with, only more so" (NCDAE, 2007), does cognitive disability access deserve its own line of research, or is it merely a subset of usability? While interesting as a theoretical debate, it seems premature to draw conclusions or to discount an entire field of research that has barely just made the attempt to become its own field of research. The data are too few to know whether the concerns differ only in terms of degree. Future research may expose conditions unique to users with cognitive disabilities. At the same time, understanding accessibility as an integral part of the larger context of usability, as some have urged (Hofstader, 2004; Law, Jacko, Peterson, & Tobias, 2005), may allow for a more inclusive treatment of the topic within web usability literature.

Analogous Research

Drawing upon analogous research is useful in any field of intellectual inquiry. In the field of cognitive disability access to the web, it is not just useful but necessary, considering that the data are so few. In a less restrictive literature review, it would be appropriate to analyze analogous research in an attempt to draw inferences applicable to web design. A future literature review focusing on analogous research data could certainly benefit web accessibility professionals Here, however, the exclusion of data from analogous sources of research serves to highlight the scarcity of directly-applicable data in the field itself.

Conclusion

The almost complete absence of empirical data has not completely discouraged accessibility "experts" from sharing their prescriptive advice (Friedman & Nelson, 2005) about how to make web content accessible to people with cognitive disabilities. This advice may stand the test of time, or it eventually may find itself in plentiful company among the many well-intentioned but incorrect opinions of other so-called experts of other topics throughout the ages. Expert opinions must eventually face the scrutiny of methodical investigators. Web design techniques proposed by developers (e.g. Hudson, Weakley, & Firminger, 2005) must be tested, and the results of these tests must be published. Researchers must learn from analogous research without falling into the trap of thinking that analogies can substitute for the real thing. As this author has said before, "we still know too little, and we do even less" (Bohman, 2004) about web development techniques for cognitive disability access.

References

Barry, M., & Pitt, I. (2006). Interaction Design: a Multidimensional Approach for Learners with Autism. Paper presented at IDC 2006. Retrieved November 6, 2006 from http://portal.acm.org/citation.cfm?id=1139073.1139086.

Bohman, P. R. (2004). Cognitive Disabilities, Part 1: We Still Know Too Little and We Do Even Less. Retrieved April 1, 2005, from http://www.webaim.org/articles/cognitive/cognitive_too_little/.

Bohman, P. R. (2006). Functional Cognitive Disabilities. Retrieved May 8, 2007 from http://paulbohman.com/accessibility/cognitive/functional.

Boscarol, M. (2006). Working with Others: Accessibility and User Research [Electronic Version]. A List Apart. Retrieved November 27, 2006 from http://www.alistapart.com/articles/workingwithothers.

Brown, P., Harniss, M., & Johnson, K. (2006). Cognitive Support Technologies: Implications for Design and User Interface. Paper presented at CSUN 2006. Retrieved November 25, 2006 from http://www.csun.edu/cod/conf/2006/proceedings/2853.htm.

Caldwell, R. (2006). Web-Accessibility, E-Learning, and Academic Libraries. International Journal of Public Information Systems, 2006(1), 1-9. Retrieved July 20, 2007 from http://www.ijpis.net/index.html?http%3A//www.ijpis.net/issues/no1_2006/no1_2006_p1.htm.

Carey, A. C., Friedman, M. G., & Bryen, D. N. (2005). Use of Electronic Technologies by People With Intellectual Disabilities. Mental Retardation, 43(5), 322-333.

Carnavale, D. (2005, August 12, 2005). Lawsuit Charges Online University Does Not Accommodate Learning-Disabled Students. Chronicle of Higher Education, The. Retrieved July 20, 2007 from http://chronicle.com/free/v51/i49/49a03301.htm.

Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2001). Enhancing Independent Internet Access for Individuals with Mental Retardation through Use of a Specialized Web Browser: A Pilot Study. Education and Training in Mental Retardation and Developmental Disabilities, 36(1), 107-113. Retrieved July 20, 2007 from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ626351&site=ehost-live.

Dickinson, A., Newell, A. F., Smith, M., & Hill, R. (2005). Introducing the internet to the over-60s: developing an email system for older novice computer users. Interacting with Computers, 17(6), 1-22.

Freeman, E., Clare, L., Savitch, N., Royan, L., Literhland, R., & Lindsay, M. (2005). Improving website accessibility for people with early-stage dementia: A preliminary investigation. Aging and Mental Health, 9(5), 442-448. Retrieved July 20, 2007 from http://taylorandfrancis.metapress.com/link.asp?id=vhx314710k76g51u.

Friedman, M. G., & Nelson, B. D. (2005). Web Accessibility Design Recommendations for People with Cognitive Disabilities: Institute on Disabilities, Temple University. Unpublished manuscript.

Gregor, P., & Newell, A. F. (2000). An empirical investigation of ways in which some of the problems encountered by some dyslexics may be alleviated using computer techniques Paper presented at the fourth international ACM conference on Assistive technologies Arlington, VA, USA. Retrieved July 20, 2007 from http://doi.acm.org/10.1145/354324.354347.

Hanson, V. L. (2004). The User Experience: Designs and Adaptations. Paper presented at the 2004 International Cross-Disciplinary Workshop on Web Accessibility (W4A), New York, New York, USA. Retrieved July 20, 2007 from http://portal.acm.org/citation.cfm?id=990659.

Harrysson, B. (2003). Internet enabling design for people with cognitive limitations. Lund University, Sweden.

Harrysson, B., Svensk, A., & Johansson, G. I. (2004). How People with Developmental Disabilities Navigate the Internet. British Journal of Special Education, 31(3), 138-142. Retrieved July 20, 2007 from http://search.epnet.com/login.aspx?direct=true&db=aph&an=14353363.

Hofstader, C. (2004). Internet Accessibility: Beyond Disability. Computer, 37(9), 103-105. Retrieved July 20, 2007 from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1333019.

Hudson, R., Weakley, R., & Firminger, P. (2005). Developing sites for users with Cognitive disabilities and learning difficulties. Retrieved November 22, 2006, from http://juicystudio.com/article/cognitive-impairment.php

Iaccarino, G., Malandrino, D., & Scarano, V. (2006). Personalizable edge services for web accessibility. Paper presented at the W4A 2006. Retrieved July 20, 2007 from http://www.w4a.info/2006/prog/4-iaccarino.pdf.

Keates, S., Adams, R., Bodine, C., Czaja, S. J., Gordon, W., Gregor, P., et al. (2007). Cognitive and learning difficulties and how they affect access to IT systems. Universal Access in the Information Society, 5(4), 329-339. Retrieved July 20, 2007 from http://www.springerlink.com/content/y4h866486t6x8667/fulltext.html.

Keating, T. J., & Ames, J. M. (2004). Cognitive Accessibility for Individuals with Disabilities: Picture Planner (tm) Graphic Activity Planner Software. Paper presented at the CSUN 2004, Los Angeles, California.

Kolatch, E. (2000). Designing for Users With Cognitive Disabilities. from http://www.otal.umd.edu/UUGuide/erica/

Kurniawan, S., & Zaphiris, P. (2005). Designing for individuals with memory and cognitive disabilities: Research-derived web design guidelines for older people. Paper presented at CSUN 2004. Retrieved July 20, 2007 from http://www.csun.edu/cod/conf/2004/proceedings/133.htm.

Law, C., Jacko, J. A., Peterson, B., & Tobias, J. (2005). Universal designs versus assistive technologies: research agendas and practical applications. Paper presented at 7th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2005). Retrieved July 20, 2007 from http://www.hsi.gatech.edu/cise/udip/contents/panel1-law.pdf.

Lee, C. M., & Phillips, C. P. (2002). Cognitive Disabilities and Assistive Technology: A Consumer's Perspective. Paper presented at the CSUN 2004, Los Angeles, California. Retrieved July 20, 2007 from http://www.csun.edu/cod/conf/2002/proceedings/105.htm.

Lepistö, A. (2004). "Siellä on ne meidän ystävät ja niitten kanssa kirjotan." Verkkopohjaisen oppimisympäristön käytettävyys kehitysvammaisten opiskelijoiden yhteydenpitovälineenä ("That’s where our friends are and with them I write." The usability of an Internet based learning environment as a means of communication by students with cognitive disabilities). University of Tampere, Tampere, Finland.

Lepistö, A., & Ovaska, S. (2004). Usability evaluation involving participants with cognitive disabilities Paper presented at the third Nordic conference on Human-computer interaction, Tempere, Finland. Retrieved July 20, 2007 from http://doi.acm.org/10.1145/1028014.1028061.

NCDAE. (2007). Cognitive Disabilities and the Web: What We Think We Know. Retrieved July 20, 2007 from http://ncdae.org/webcasts/cognitive.cfm.

Opitz, C. M. (2003). The effects of implementing Web accessibility standards on the success of secondary adolescents with learning disabilities. Arizona State University. Retrieved July 20, 2007 from http://worldcat.org/oclc/55135520.

Oross, S. O. I., & Woods, C. B. (2003). Explorting visual perception abilities in individuals with intellectual disabilities: assessment and implications. In S. J. Soraci & K. Murata-Soraci (Eds.), Visual information processing (pp. 35-79). Westport, CA: Praeger.

Petrie, H., & Kheir, O. (2007). The relationship between accessibility and usability of websites. Paper presented at the SIGCHI conference on Human factors in computing systems. Retrieved July 20, 2007 from http://portal.acm.org/citation.cfm?id=1240624.1240688.

Pollack, M. E. (2005). Intelligent Technology for an Aging Population:  The Use of AI to Assist Elders with Cognitive Impairment. AI Magazine, 26(2), 9-24. Retrieved July 20, 2007 from http://www.eecs.umich.edu/~pollackm/Pollack-web_files/distrib/agingtex-preprint.pdf.

Robertson, G. L., & Hix, G. (2002). Making the Computer Accessible to Mentally Retarded Adults. Communications of the ACM, 45(4), 171-183. Retrieved July 20, 2007 from http://doi.acm.org/10.1145/505248.506006.

Roh, S.-Z. (2004). Designing accessible Web-based instruction for all learners: Perspectives of students with disabilities and Web-based instructional personnel in higher education. Unpublished Doctoral dissertation, Indiana University. Retrieved July 20, 2007 from http://mutex.gmu.edu:2068/pqdweb?did=828407271&sid=3&Fmt=2&clientId=31810&RQT=309&VName=PQD.

Salem Darrow, M. (1995). Virtual Reality's Increasing Potential for Meeting Needs of Persons with Disabilities: What About Cognitive Impairments? Paper presented at the CSUN 1995, Los Angeles, California. Retrieved July 20, 2007 from http://www.csun.edu/cod/conf/1995/proceedings/0023.htm.

Shneiderman, B. (2000). Universal usability. Communications of the ACM, 43(5), 84-91. Retrieved July 20, 2007 from http://portal.acm.org/citation.cfm?id=332833.332843.

Small, J., Schallau, P., Brown, K., Ettinger, D., Blanchard, S., Krahn, G., et al. (2005). Web Accessibility for People with Cognitive Disabilities. Paper presented at the 28th Annual RESNA Conference. Retrieved July 20, 2007 from http://www.resna.org/ProfResources/Publications/Proceedings/2005/Research/CAC/Small.php.

Stephanidis, C., Salvendy, G., Akoumianakis, D., Bevan, N., Brewer, J., Emiliani, P. L., et al. (1998). Toward an Information Society for All: An International Research and Development Agenda. International Journal of Human-Computer Interaction, 10(2), 107-135. Retrieved July 20, 2007 from http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=7388646&site=ehost-live.

Thatcher, J., Burks, M. R., Heilmann, C., Kirkpatrick, A., Lauke, P. H., Lawson, B., et al. (2006). Web Accessibility: Web Standards and Regulatory Compliance: Friends of ED.

WebAIM. (2005). Capella. Retrieved May 25, 2007, from http://www.webaim.org/discussion/mail_thread.php?thread=2390

Williams, P. (2006). Developing methods to evaluate web usability with people with learning difficulties. British Journal of Special Education, 33(4), 173. Retrieved July 20, 2007 from http://www.blackwell-synergy.com/doi/abs/10.1111/j.1467-8578.2006.00436.x.

Williams, P., & Nicholas, D. (2006). Testing the usability of information technology applications with learners with special educational needs (SEN). Journal of Research in Special Educational Need, 6(31). Retrieved July 20, 2007 from http://www.blackwell-synergy.com/doi/full/10.1111/j.1471-3802.2006.00057.x.

[1] The inclusion of terms that some consider less desirable—such as handicapped—was an attempt to include as much literature as possible in the review, regardless of the appropriateness of the terminology used.