Accessibility Evaluation on Malaysian E-Government Websites

Introduction

The Multimedia Super Corridor (MSC) was successfully created in 1996 (MSC 2008). In order to achieve the objectives of the MSC, the Government of Malaysia has identified seven flagship applications for implementation and one of these applications is Electronic Government (MSC 2009). The vision of Electronic Government (EG) is one where Government, business and citizens work together for the benefit of the nation.

The prominent feature of the EG delivery is using internet technologies through web-based interfaces that enable people to access resources and services from anywhere and anytime.

These websites would also become an important tool for the government to market their institution to prospective customers as well as providing on-line government information and services. Thus, as to ensure the success of these government websites, they must be fully accessible to people of all level, including those with disabilities.

In Malaysia, the government has drafted act for disable people (Persons With Disabilities Act, 2002) to provide the basis for the equalization of opportunities for persons with disabilities (Omar 2002). The objective of this act is to ensure that persons with disabilities in Malaysia  have the same rights to equality before the law as the rest of the community in the country.

In addition, it also seeks to promote recognition and acceptance within the community that citizens with disabilities should have equal rights and opportunities just like their normal counterparts.

Against this background, this study was undertaken with twofold objective i.e. (i) to identify the accessibility of e-government websites based on the W3C accessibility guidelines (ii) to evaluate the practice of e-government websites development and maintenance on accessibility.

Literature Review

Web accessibility can be defined as the degree to which a site is accessible to the largest possible range of people. The more people are able to access a website, the more accessible is the site.

At its core, Web accessibility emphasizes making website accessible to persons with disabilities and involves removing potential barriers to access caused by inconsiderate website designs (Zeng 2004). The World Wide Web Consortium (W3C) is an international organization dedicated to the standardization of the World Wide Web (WC3).

In 1996, W3C established the Web Accessibility Initiative (WAI) campaigning for a more accessible Web for persons with disabilities. For the consortium, Web accessibility was defined as “access to the Web by everyone, regardless of disability” (Zeng 2004).

The WAI approach to Web accessibility revolves around three interrelated fronts: (i) the content accessibility of websites for persons with disabilities to perceive, understand, and use; (ii) making Web browsers and media players usable for persons with disabilities by making them operable through assistive technologies and (iii) Web authoring tools and technologies to support production of accessible Web content and sites, so that persons with disabilities can use them effectively.

An accessible web site is very similar to an accessible building. An accessible building offers curb cuts, ramps, and elevators to allow a person with disabilities to enter and navigate through the building with ease. Hence, an accessible web site offers similar functionality.

While the call for developing accessible websites has been intensified, many empirical studies on this topic unveiled most websites still fail to address the issue of accessibility.

According to Dorothy (2003), failing to develop fully accessible websites has caused the people with disabilities not being able to take the advantage of the opportunities afforded by the web.  A study by Lazar et al (2004) discovered that large percentage (70—98%, depending on the category of site) of websites is not accessible.

According to Gutierrez & Windsor (2005), despite the fact that as more and more web pages are designed using popular multimedia elements such as animation, graphics and video, they are however not accessible to those using screen readers and other disability-related accessibility technology (Gutierrez et al 2005). Their study discovered that the most home pages of the Fortune 500 companies’ websites did not comply with two key laws pertaining to disability-access i.e. the Americans with Disabilities Act (ADA) and Section 508 of the Rehabilitation Act of 1973.

Likewise, a study on e-tourism websites in four countries in Africa also revealed almost similar situation (Maswera, 2005). With regards to e-government websites’ accessibility, Kurniawan & Zaphiris’s (2001) study found that about only 52% of governmental health-related websites complied with WCAG.

As to ensure the accessibility of websites, many governments have also made web accessibility a requirement for government information on the web.

The United States, England, Canada, Portugal, and Australia require some types of government information to be accessible (Lazar et al 2004). In the context of Malaysia, the responsible agency that is assigned with similar task is Malaysia Administration Modernization and Management Planning Unit (MAMPU).

To ensure that all government websites are fully accessible, the agency has developed a guideline for Websites development in synchronize to the standard guidelines set by W3C (MAMPU, 2000).

Research Methods

Nine e-government websites were selected for the evaluations. These websites were chosen as they were highly accessed by the Malaysian citizens.

An automatic testing tool known as Bobby was chosen to evaluate the accessibility of the websites as it was widely used (Zacharia & Zaphiris 2001). Bobby is an online free portal developed in 1996 by the Centre for Applied Special Technology or CAST  (Watchfire 2004).

Figure 1 shows a screen capture of the Bobby software used in the study. Its purpose is to help web designers identify source code violations and repair their web pages or websites so as to ensure that they are accessible to the individuals with disabilities.

The Bobby program tests Web pages for conformance with Web Content Accessibility Guidelines 1.0 (WCAG 1.0) and generates analysis report based on priority levels. Table 1 depicts the general guidelines of WCAG 1.0 (W3C, 1999a).

Several studies had also adopted Bobby on evaluating websites accessibility (Zacharia & Zaphiris, 2001; Yuquan, 2006).

In order to use Bobby program for evaluating the accessibility of the websites, the address of the websites will have to be entered. The Bobby program will then scan the webpage of the given address and produces a report to identify accessibility barriers that do not conform to the checkpoints established in WCAG 1.0.

Table 1: General guidelines of WCAG 1.0

The checkpoints are categorized into 3 levels of importance or priority levels namely, Priority 1, Priority 2 and Priority 3 (W3C, 1999b). Priority 1 is considered the most important, because this level covers errors that seriously affect a site’s accessibility by people with disabilities.

Priority 2 and 3 levels should be considered after the Priority 1 level errors have been corrected. Because Priority 1 represents the minimum standard of acceptability, Priority 1 checkpoints were used in this study. Table 2 presents the detail description of these three checkpoints. Complete accessibility is achieved by receiving 0 errors on all 3 Priority Levels.

Websites that have reached this status (Bobby AAA Approved status) are eligible to post the Bobby logo on their website to display their commitment to web excellence and the needs of the disabled computer user.

In evaluating the accessibility of these websites, this study evaluated the entry point or the home pages only.

The justification of doing so was because the main page of a Websites serves as an index or table of contents to other documents stored at the site and also the first destination that the user will arrive at. According to Childres (2007), the home page is also the user’s first impression of the site; a good impression of a website welcomes the user and invites her to further explore the site’s contents.

All the evaluations were conducted in either Mozilla Firefox 2.0 or Internet Explorer 6.0 (IE) running under Windows XP Professional with Service Pack 2.

Both Firefox and IE were used because when doing pilot test using some e-government Web sites, it was found that for some Web sites when Bobby check was not successfully under Firefox it was successful under IE, and vice versa.

Each homepage of the Web sites was checked using Bobby Online Free Portal under Firefox first. If the check was unsuccessful, it was then re-done under IE. Sometimes, this procedure had to be repeated for several times before a homepage can be successfully checked. Figure 2 illustrates the flowchart of the evaluation procedure

Fig 2: Evaluation procedure

In addition to the accessibility testing, follow-up interviews were also conducted with webmasters and web developers of these websites as to learn about their perception and knowledge on the topic as well as their practices of web accessibility.

Findings

The results of the Bobby analysis indicated that there was no single e-government websites that passed the W3C Priority 1 accessibility checkpoints which measures errors on five respects namely (i) provides alternative text for all images (ii) provides alternative text for all image-type buttons in forms (iii) provides alternative text for all images map hot-spot (iv) provides alternative content for each OBJECT and (v) gives each frame a title.

Table 3 presents the summary of the analysis. Based on the results of the analysis, there were altogether 406 instances of errors found on the nine websites.

A total of 377 instances found on 8 websites were associated with errors for not providing alternative text for images. 10 instances from 3 websites were linked with errors for not providing alternative test. 14 instances from 2 websites were related to errors for not providing alternative text for images map hot spot. 3 instances from 2 websites were associated with errors for not providing alternative content. 2 instances from 2 websites were related with errors for not providing titles to frames.

Table 3: Summary of analysis using Bobby

These five types of errors violated with two WCAG checkpoints. Table 4 shows Priority 1 error with WCAG checkpoints. 

First checkpoint (checkpoint 1.1) is failing to provide a text equivalent for every non-text element was the worst violated. Eight of the nine websites have this problem with 404 instances of errors. Second checkpoint (checkpoint 12.1) was violated is failing to give each frame a title; two websites involved two instances of errors.

Table 4: Priority 1 error with WCAG checkpoints

In order to understand the practice of web development and maintenance of developers of these websites, follow-up interviews were conducted. A total of 17 respondents participated in the interview. Specifically, the questions put forwards were as follows:

Q1.    Are you familiar with the W3C Web Content Accessibility Guidelines?
Q2.    Do you refer or consult the W3C manual prior to the development of the websites?
Q3.    Have you ever attended any training in developing web content for accessibility?
Q4.    Are you aware that there are software tools that can check your website to see if it is accessible, and provide useful feedback?
Q5.    Have you ever used web accessibility tool, e.g., Bobby?
Q6.    Have you ever tested your website using a screen reader?

Table 5 displays the frequencies of the responses for question 1 to question 6, and Figure 3 displays the data graphically.

Table 5: Results of the interview

Fig. 3. Statistics of question 1- 6

For question 1, ‘‘Are you familiar with the WC3 Web Content Accessibility Guidelines?’’, six respondents (35.3%) indicated that they were familiar with the guidelines, ten respondents (58.8%) indicated that they were not familiar with the guidelines, and one respondent (5.9%) was not sure.

For question 2, ‘‘Do you refer or consult the WC3 manual prior to the development of the websites?’’, five respondents (29.4%) indicated that they had referred or consulted the WC3 manual prior to the development of the websites, 11 respondents (64.7%) indicated that they had not refer or consult the WC3 manual prior to the development of the websites, and one respondent (5.9%) was not sure.

For question 3, ‘‘Have you ever attended any training in developing web content for accessibility?’’, six respondents (35.3%) indicated that they had previously attended a training and 11 respondents (64.7%) indicated that they had not attended any training.

For question 4, ’’Are you aware that there are software tools that can check your web site to see if it is accessible, and provide useful feedback?’’, seven respondents (41.2%) were familiar with the availability of software tools, nine respondents (52.9%) were not familiar with software tools, and one respondent (5.9%) were not sure.

For question 5, ‘‘Have you ever used web accessibility tool, e.g. Bobby?’’, five respondents (29.4%) indicated that they had used web accessibility tools, 10 (58.8%) respondents indicated that they had not, and two respondents (11.8%) were not sure.

For question 6, ‘‘Have you ever tested your web site using a screen reader?’’, three respondents (17.6%) indicated that they had tested web sites using screen readers and, 14 respondents (82.4%) indicated that they had not tested sites using screen readers.

Results of the interviews indicated that based on the 102 questions put forwards to the 17 respondents, 61 or 59.8 % was a ‘NO’ answers while 35 or 34.4 % was a ‘YES’ answer. The remaining 6 or 0.06% was ‘NOT SURE’ respond. This result could be an explanation as to why all the websites being assessed did not pass even the W3C Priority 1 accessibility checkpoints.

Discussion

This study had revealed that Malaysia e-government Websites have significant Web accessibility problems. Based on the results, most of them failed one or more WCAG checkpoints. Failing to provide a text equivalent for every non-text element is the most serious problem identified in this study: All e-government websites have this problems.

The power of text equivalents lies in their capacity to be accessible to people from various disability groups using a variety of technologies (W3C 1999a).When a text equivalent is presented to the user, it fulfils essentially the same function as the original content upon presentation to the user. The simple and common method to provide the text equivalent is to use the ALT attributes for every non-text element in the web site.

Failing to give each frame a title is found to be the second serious problem in this study. Frame title gives the disabled user information about the contents of the frame, if they are not able to access multiple frames simultaneously. With this information, they can pick which particular frame to view. However, because frames can cause problems with some screen readers, it may be difficult or impossible to reach some of the frames on a page.

A non-frames version of the page thus should also be provided if frames are used (Frontend.com, 2005). Moreover, splitting a Web page into frames is very confusing even for users with perfect eyesight because frames break the unified model of the Web and introduce a new way of looking at data that has not been well integrated into the other aspects of the Web  (Nielsen 2000).

This could cause a series of potential usability issues, particularly when frames are not skilfully implemented.

Non-compliance with the WCAG is in fact a form of discrimination against individuals with disabilities. Therefore it is necessary for Malaysian government to pay attention to the accessibility of their Websites. 

Upon further investigation on webmasters knowledge and practices pertaining to accessibility issues, results of the interview unveiled that most webmasters did not fully adhere to the standard of WCAG. More interestingly, some of the respondents indicated of not being aware of the existence of WCAG. Evidently, this lacking could be a one of the contributing factors to the inaccessibility of these websites to the disabled people.

Conclusion

This research study clearly demonstrates the necessity for government of Malaysia to pay attention to the accessibility of their websites and have a plan ready for it. Some suggestions are proposed below based on the research findings.

I) Follow WCAG and proprietary accessibility guidelines

Since the WCAG from W3C are well established, as well as widely accepted and used around the world, Malaysia e-government websites should follow WCAG. In addition, e-government websites should also follow the proprietary guidelines established by accessibility initiatives from technology vendors.

For example, they could follow the ‘‘IBM Guidelines for Writing Accessible Applications Using 100% Pure Java’’ which is the joint effort by IBM and Sun for any Java components used on websites. Following the above-mentioned guidelines will ensure any web content is as accessible as possible (Yuquan 2005).

II) Use the ALT attribute to provide text equivalent

This study found that failing to provide a text equivalent for every non-text element (e.g., AREAs, images, image-type buttons, APPLET, etc.) is the most serious problem on the Malaysia e-government homepages. The provision of text equivalents for non-text elements is one of the most fundamental elements for accessibility.

Text can be readily output to speech synthesizers and braille displays, and can be presented visually (in a variety of sizes) on computer displays and paper. When a text equivalent is presented to the user, it fulfils essentially the same function (to the extent possible) as the original content.

The simple and common method to provide the text equivalent is to use the ALT attributes (W3C 1999). This costs nothing but need some time to put this elements into web pages. By doing this it can benefit everybody of all level, including those with disabilities.

III)Title each frame/try to avoid using frames

Failing to title each frame is another accessibility problem on the e-government homepages: all the homepages using frames have this problem. Screen reader users cannot quickly scan the contents of multiple frames. A frame title gives the screen reader users information about the contents of the frame. With this information they can pick which particular frame to view.

Hence, it is important to add a ‘‘title’’ attribute to each frame to describe the purpose and content of the frame. A non-frames version of the page should also be provided if frames are used  (Frontend.com 2005).

Moreover, splitting a web page into frames is very confusing even for users with perfect eyesight because frames break the unified model of the Web and introduce a new way of looking at data that has not been well integrated into the other aspects of the Web  (Nielsen, 2000). This could cause a series of potential usability issues, particularly when frames are not skilfully implemented. Hence, the use of frames should be generally avoided.

IV) Make it a routine

Web accessibility is a continual process and one that should be evaluated often. Each time a web page is updated or created, its accessibility should be checked immediately.

Otherwise, over time, Malaysia e-government websites will become inaccessible or more inaccessible and all the efforts and resources invested earlier on web accessibility will be cast to the winds. Hence, making it a routine to check a website’s accessibility is important for all Malaysia e-government websites.

V) Establish formal Web accessibility laws and guidelines

Experience from developed countries shows that the first important thing should be done is to establish formal Web accessibility laws and guidelines.

Although Malaysia enacted several legislations and governmental regulations concerning with the employment, transportation and public facilities of disabled people, Malaysia inevitably needs laws regarding Web accessibility to ensure the equal rights of people with disabilities in the new information era.

Only formal legislations enable people with disabilities to exercise their rights and help/force public agencies to establish related policies to ensure public information and services made available through the Internet accessible to people with disabilities.

VI) Training to webmasters

Further training of these webmasters on skills needed to developed websites that are highly accessible and comply with the standard is deemed very necessary and in the development of any government websites, suggestion and opinions from the disabled community should also be considered.

VII) Involve user in testing

Automatic testing tools alone cannot verify effective compliance with the W3C guidelines and human judgments are always necessary.

Hence, Web accessibility testing on e-government Web site should combine automated software tools with human judgment. Involving disabled people in the design and testing of Websites is very likely to improve usability for all because many of the characteristics which impede disabled users also make the site confusing to users in general.

Therefore, it is highly recommended to design Malaysia e-government Web sites by consulting the disabled community and also to test Web sites with assistive technologies used by disabled people.

While this study had successfully achieved its objectives, it bears several limitations. First, it employed the use of automatic checking instead of formal accessibility evaluation involving user testing. Software program such as Bobby should not be viewed as a comprehensive method for determining accessibility.

A comprehensive approach to accessibility validation would incorporate both automatic and manual checks  (Rowan et al 2000).

The second limitation of the study was on the data collection procedure i.e. data was collected on one snapshot i.e. on point in time. The third limitation of the study was that the assessments were only done on the home pages only.

The future research should examine Malaysia e-government website in more depth and expand to include other web pages rather than home pages only.

The evaluation using automatic testing tools alone cannot verify effective compliance with the WC3 guidelines and should involve disabled people in the testing of Websites.

References

Childres, J.D. (2007). Improving the Accessibility of Websitess for Visually Impaired Individuals A Master’s paper for the M.S. in I.S. degree. Chapel Hill: University of North Carolina.

Dorothy, A. A. (2003). Establishing standards for usable and accessible user services web sites, , ,2003. Proceedings of the 31st annual ACM SIGUCCS conference on User services (pp. 63 – 64). San Antonio, TX, USA: ACM Press.

Frontend.com. (2005). Accessibility and usability for e-government. [Online], [Retrieved January 20, 2009],
 http://www.frontend.com/accessibility/accessibility-usability-for-e-government.html

Gutierrez, C.F. & Windsor, J.C. (2005). An evaluation of Fortune 500 company home pages for disability-access. International Journal of Electronic Business , 137-153.

Kurniawan, S.H. & Zaphiris, P. (2001). Usability and accessibility comparison of governmental, organizational, educational and commercial aging/health-related websites. Proceedings of the 9th International Conference on Human Computer Interaction (pp. 34-36). New Orleans, LA: Abridged.

Lazar, J., Sponaugle, A.D. & Greenidge, K.D. (2004). Improving web accessibility: a study of webmaster perceptions. Computers in Human Behavior, 20 (2),  269-288.

MAMPU. (2000). Garis Panduan Malaysian Civil Service Link (MCSL) dan Laman Web Agensi Kerajaan. Jabatan Perdana Menteri.

Maswera, T. D. (2005). Analysis of usability and accessibility errors of e-commerce websites of tourist organizations in four African countries.  Information and Communication Technologies in Tourism , 531-542.

MSC. (2008). MSC Malaysia, [Online], MSC Malaysia, [Retrieved Disember 20, 2008],
 http://www.mscmalaysia.my/article

MSC. (2009). MSC Malaysia, [Online], MSC Malaysia, [Retrieved January 10, 2009],
http://www.mscmalaysia.my/topic/12071142100873

Nielsen, J. (2000). Designing Web Usability. New Rider Publishing.

Omar, H. (2002). Disabilities ACT. [Online],  Penang Down Syndrome Association [Retrieved January 10, 2009],
 http://www.dsapenang.org/Disability Act original.htm

Rowan, M., Gregor, P., Sloan, D., & Booth. (2000). Evaluating Web resources for disability access. Paper presented at the Fourth International ACM Conference on Assistive Technologies. Arlington, VA: ACM Press.

W3C. World Wide Consortium.,[Online], [Retrieved December 17, 2008], http://www.w3.org/

W3C. (1999a). Web content accessibility guidelines 1.0.,[Online], [Retrieved December 18, 2008],
 http://www.w3.org/TR/WAI-WEBCONTENT

W3C. (1999b). Checklist of Checkpoints for Web Content Accessibility Guidelines,[Online], [Retrieved December 20, 2008],
http://www.w3.org/TR/WCAG10/full-checklist.html

Watchfire. (2004). Bobby Online Service. [Online], [Retrieved January 10, 2006],  http://bobby.watchfire.com/bobby

Yuquan Shi. (2006). The accessibility of Chinese local government websites: An exploratory study. Science Direct .

Yuquan Shi. (2005). The accessibility of Queensland visitor information centres’ websites”, 2005. Science Direct .

Zacharia, G. & Zaphiris, P. (2001). Website content accessibility of 30,000 Cypriot websites. Proceedings of the 8th Panhellenic Conference on Informatics, 2001. Nicosia, Cyprus, .

Zeng. (2004). Evaluation and Enhancement of Web Content Accessibility for Persons with Disabilities. Ph.D. Thesis. University of Pittsburgh .