Electronic Journal of Academic and Special Librarianship v.10 no.2 (Summer 2009)

E-State Practices in Turkey: Evaluation of Governmental Web Sites

Feryal Turan, Assistant Professor
Department of Sociology, Ankara University, Turkey

Özlem Bayram, Associate Professor
Department of Information and Records Management, Ankara University, Turkey

Introduction

The development of information technologies (IT) supports significant transformations in the conventional working mechanisms of the Turkish State and affiliated public institutions.

E-State is defined by World Bank as the reformulation of the relations between citizens, businesses and State authorities through use of information technologies (such as Wide Area Networks, Internet, and mobile communication) by public administrative units.

Whereas important steps have been taken in e-State practices within the framework of the infrastructure of the institutional and national information networks, Turkey is still at the starting point with regards to meeting the information and communication technology-based needs of its administrative units. Assessment, therefore, is crucial. This study reveals the results of the evaluation of five well-known public institutions affiliated to the Turkish State.

Studies toward Assessment of Web Sites

In recent years, studies carried out on the evaluation of Web sites gradually have been increasing. One part of these studies is related to the assessment of the design process, while the other part is related to the assessment of content. In general, two methods were applied to the evaluation of Web sites in terms of quality.  The first method utilizes information technology-based tools.  It is directed at determining user satisfaction by analyzing user activities on the Web site. For analysis, Bauer and Scharl (2000), who used this technique, evaluated Web sites according to the criteria of content, user interaction and intra-Web circulation. In evaluation of educational sites, Hwang, Huang and Tseng (2004) benefited from an assessment group which was formed beforehand. They developed computer-assisted software, EWSE (Educational Web Site Evaluator), and used the technique of group decision on this program.

In evaluation of the sites in terms of Web design, the “protocol analysis” method of Benbunan-Fich (2001), which is performed on computer records, can be mentioned. This method assists in testing, through a user perspective, factors used in the process of Web design. Use of non-complex operations on Web sites may give better results than the ones of this technique.

Regarding the evaluation of Web sites in terms of content, generally, a conventional survey is used and evaluators are required to assess Web sites according to certain criteria (Stufflebeam 2001; Marsico, 2004). The majority of studies on the subject have been conducted for commercial (Benbunan-Fich 2001; Gonzalez 2004) and educational Web sites (Susser 2005; Hwang 2004).

Among such studies, Gonzalez and Palacios (2004) carried out the largest-scale application. In this study, the above-mentioned researchers evaluated the Web sites of 200 Spanish companies in terms of basic criteria such as functionality, usability, efficiency and reliability.  As a result of the evaluation, shortcomings of the Web sites were revealed. Official Web sites in the U.S. were assessed in a similar study (Eschenfelder 2004).

Almost all of the above-mentioned studies envisage working on excessively detailed variables for the efficiency of an evaluation with conventional dimension. This situation implies that if the study is conducted with a wide user mass, it will be carried out under difficult conditions. The criteria of content, access, interaction, accuracy, authority and currency defined by Smith (1997) have been used in literature ubiquitously by librarians and users of Internet information sources. In subsequent years, Olsina et al. (1999) also worked on evaluation criteria but these were considered by Bauer and Scharl (2000) as too detailed rendering application impossible.

Susser (2004), deviating from other studies, conducted an assessment study on two different student groups receiving education in Web design and tourism. This study tried to assess particularly the Web design assessment skills of the students taking Web design class, rather than the Web sites themselves. As a result of the study, Susser concluded that the students having taken classes on Web site design are more conscious of and sensitive to evaluation of the content of Web sites when compared to tourism students.

Having reviewed studies in the literature on Web site assessment, concerning application, we can accept the following criteria, which were mentioned in almost each case, as decisive factors:

• Content
• Visual design
• Access, usability and interaction
• Navigability
• Accessibility

Method

This study assesses the Web sites of five of Turkey’s ministries according to the selection criteria. For the purpose of attaining this objective, students taking Web design class from the Department of Management of Information and Document of the Faculty of Letters were required to evaluate the Web sites of five ministries. The 16 students, who took the class, filled 4-page-questionnaire on May 16, 2005 (See appendix).

The pilot study of this research was conducted in March 2005 and consequent to the research, necessary corrections were made in the questionnaire. The Web sites of five ministries were selected for this study: The Ministry of Culture, the Ministry of National Education, the Ministry of Labor, the Ministry of Health, and the Ministry of Interior. The questionnaire prepared for the assessment of the Web sites of the ministries consists of 11 main parts that include sub-evaluation criteria. Multiple choices of “yes”, “no” and “partly” were used in the evaluation of each question.

Findings and Discussion

1. Definition of Web Site

In this context, evaluators stated that the Web site of the Ministry of Labor is better in terms of defining objective/mission and that the Web site of the Ministry of Health is the most inadequate among those evaluated.

While the majority (11) stated that the source, type and target population of the information on the Web site of the Ministry of Culture had been clarified, less number of people pointed out to the Web site of the Ministry of National Education. Regarding this issue, the most sufficient Web site was determined as that of the Ministry of Culture. In general, it was expressed that all Web sites are relatively in conformity with the criteria on designation of the section where new information is added. At this point, if the fact that recent laws, announcements and amendments are often indicated on the Web sites of ministries is taken into account, that why more and more people consider these Web sites in a positive way in this respect becomes clear.

2. Content/scope

The majority (12) approved the Web site of the Ministry of Labor with regard to content and consistency with the Statements on the promotion of the Web site; on the other hand, the Ministries of National Education (7) and Interior (7) were selected by less people. One of the most important points about Web sites is that they provide information required by users. Concerning this issue, the Web site of the Ministry of Culture was regarded as the most competent by the evaluators (11). The evaluators believe that the State’s Web sites generally provide beneficial and necessary information. However, views about the balance and adequacy of relevant information vary. The majority shares the same view regarding use of technical language (political, contemptuous, and accusing expressions were avoided) and non-inclusion of prejudices in the content (distinctions based on race, class, sex and culture were not given place) of evaluated Web sites.

3. Currency

The evaluators Stated, with a close majority, that the Web sites are updated. But they are of the view that Web sites do not provide sufficient information on the time of updating.

4. Metadata

Whereas the majority of the evaluators (14) found the Web site of the Ministry of Culture adequate with regard to the specification of title, sub-titles, authors, sections, site definition and key words, less people (8) designated the Ministry of Health. The numbers of people, who think positively in the evaluation on the specification of title of each page, are very close. In the evaluation, terms (terminology) and titles given in sub-pages on the Web sites of the Ministries of Labor (15) and Culture (15) were considered compatible with the entire Web site.

5. Services

The majority of the evaluators (12) confirmed that services of the Ministries of Interior and Health are open to everybody on the Internet environment. The minority (2) regarded the Ministry of Culture in conformity with this criterion.

6. Privacy

In relation to the determination of attitudes of Web site users, in the evaluation on not requesting specific personal information or on the protection of the privacy of requested information, the majority (5) shared positive equal or similar view for all institutions.

7. External Recognition

In the evaluation towards recognition of Web sites on Internet, the majority (13) stated that the Web site of the Ministries of Interior and Culture give the number of visitors. The Ministry of Labor was also found in conformity with the same criteria by a close number of the majority (12). Only one person selected the Ministry of National Education for this view. In the context of the same evaluation, whereas the majority (13) indicated that there is no external link on the Web site of the Ministry of National Education, only the minority (1-2 people) gave a positive reply to this information for the Websites of other institutions.

8. User Friendliness

Firstly, currency of intra-site links/information was dealt with in evaluations on user friendliness of the Web sites. Whereas the majority (15) confirmed that the Web sites of the Ministries of Labor and National Education were updated, the Web site of the Ministries of Interior and Culture were chosen by the least number of people (8) regarding this criteria. Within the framework of the same evaluation, that there are shortcuts for those who use the site frequently is confirmed for almost all sites (7), primarily the Ministry of Labor (9). While the majority (12) deemed the Web sites of the Ministries of Interior and Culture in conformity with the criteria of giving information on the content regarding intra-site links, a close majority (9) affirmed the appropriateness of the Ministry of Labor.

9. Feedback

In the evaluation verifying the existence of an address that users can report their opinions on the concerned Web site in order to assess to what extent Web sites take into consideration the feedbacks of users, the majority (13) marked the Ministry of Labor whereas the minority (5) marked the Ministry of National Education.

10. Access

The first condition, upon which Web sites are evaluated, with respect to access, is speed.  Regarding speed, the Ministry of National Education ranks it first by majority (12). Nevertheless, all other institutions were also evaluated as speedy by a close majority (11). For the second choice on the inclusion of the title of the institution on site address, the majority of the evaluators (16) marked the Ministry of Health, whereas a close number of people gave a positive response for other institutions too.

11. Design

In the evaluation on the design of the Web sites, the majority (10) reported that the sites of the Ministries of Interior and Culture were in conformity with the subject in terms of format and graphic design. In this question, the least number of people (6) regarded the Ministry of Labor as compatible.

In the second choice on design, the majority of the evaluators (9) regarded the Web pages of the Ministry of National Education as clear and having adequate content, a less number of people (5) deemed the Ministry of Labor in conformity with the said criteria. On the other hand, with regard to informing on use of HTML in Web designs, again the majority (16) chose the Ministry of National Education; however, the other institutions were also deemed in accordance with this criterion by a close number of the majority. The final evaluation within the framework of design is related with the information confirming that only text or graphic, in other words alternative access facilities are offered in display of the Website. In this evaluation, while the majority (8) marked the Ministry of Health, the minority (2) chose the Ministries of Interior and Culture as affirming such criteria.

12. Navigability

The first choice in the context of the evaluation towards navigability consists of the confirmation of whether the Website has been organized in a way as to meet the expectations of users. The majority (9) marked the Ministries of Labor and National Education in this assessment; the least number of people (2) found the Ministry of Health appropriate. For the second criteria upon which appropriateness on such guidance as site map is assessed, the majority (12) chose the Ministries of Labor and National Education, whereas a close number of the majority (10) evaluated also other institutions positively. As the third evaluation choice towards intra-site links, the majority (14) shared the view that it is possible to access the Ministries of National Education, Interior, Culture and Health (13) from any search engine, the least number of people (8) pointed to the Ministry of Labor as in line with this criterion.

Conclusion

The first issue in evaluation of the Web sites discussed in the context of research consists of the promotion of the Web sites. It is clear that the information on the objective, subject, source of information, history and way of use of Web site is crucial for users. It is obvious that even if a Web site - the way of use of which is not clear - includes very important information, in case users cannot reach to such information easily, problems will emerge. The evaluation revealed that in general all of the Web-sites are relatively in accordance with the criteria.

A great similarity was found between the Web sites with respect to content, and satisfaction of the criteria of objective, mission and function.

There is a consensus among the evaluators on the currency of the Web sites and they are of the view that institutions frequently renew the relevant information on the electronic environment.

It was seen that there are differences between the Web sites on criteria of use of metadata elements covering the specification of the title, sub-title, authors, sections, site definition and key words of the Web site.

Despite the fact that services are arranged in a way as to meet the needs of everyone on Internet environment, there is no consistency between the sites on the criteria of accessibility by everyone.

All Web sites show consistency in evaluations over the inclusion of Statements on not requesting specific personal information or on the protection of the privacy of requested information. Therefore, it can be stated that Web sites are reliable with regard to personal information.

On the one hand those Web sites provide access to other sites in connection with the issue, for which they offer service, is important for individuals in need of service or information on the matter. On the other hand, it is a feature that would enable the acknowledgment of that institution on Internet. In this context, indication of the number of visitors on all Web sites other than that of the Ministry of National Education is insufficient.

In order to determine the user friendliness of the Web sites, the currency of site links and accessibility of information that concern every citizen are regarded as significant criteria. Within this framework, while the Ministry of Labor ranks first among all sites with regard to currency, concerning the other criteria, the Web sites of all institutions reveal similar results. In this context, it can be concluded that in general, sites are user friendly regarding access.

The indication of an address to which users can report their views about the site has importance for assessing to what extent feedbacks of users are taken into consideration. It can be stated that according to the related evaluation, among all institutions, the Ministry of Labor is the one which attaches value to feedbacks of users.

There is no problem in access rate of all Web sites under the scope of this evaluation. Full titles of all institutions are included on their site addresses.

When Web site designs are evaluated, whereas all Web sites are in conformity with standards regarding technical infrastructure, concerning content designs the Web site of the Ministry of Labor ranks low. However, with regard to the evaluation of such guidance as site map, the most suitable sites were chosen as those of the Ministries of Labor and National Education.

As a result, whereas one of the sites is able to meet the criteria, another is not. Even if they satisfy the criteria on average, there is a general inconsistency among five of them because it is impossible to talk about a standard structure. Therefore, it is favorable that each site offer service in a single portal. 

Bibliography

Bauer, C.  and A. Scharl, (2000) “Quantitative evaluation of Web site content and Structure”, Internet Research, Vol 10 No 1, pp.31-41.

Benbunan – Fich, R. (2001) “Using protocol analysis to evaluate the usability of a commercial Web site”, Information and Management Vol 39  No 2, pp.151–163.

Choi, J. and D. Kim, (2004) “A comparative analysis of automated Web site evaluation tools”, Proceeding of the Eighth Pacific-Asia Conference on Information Systems PACIS 2004, July 8-11. Available (http:www.pacis-net.org/file/2004/S37-001.PDF)

Eschenfelder, K. R. (2004) “Behind the Web site: An inside look at the production of Web-based textual government information”, Government Information Quarterly, Vol 21, pp. 337-358.

Gonzalez, F.J. and T. M. Banegil Palacios (2004) “Quantitative evaluation of commercial Web sites: an empirical study of Spanish Firms”, International Journal of Information Management, Vol 24, No 4, pp. 313-328.

Hwang, G.J. , T.C.K. Huang and J.C.R. Tseng. (2004) “A group-decision approach for evaluating educational Web sites”, Computers and Education, Vol 42 No 1, pp.65 – 86.

Marsico, M. and S. Levialdi. (2004) “Evaluating Web sites: exploiting user’s Expectations”, International Journal of Human-Computer Studies, Vol 60, No 3, pp. 381-416.

Smith, A. G. (1997) “Testing the surf: criteria for evaluating internet information resources”, The Public-Access Computer Systems Review, Vol 8, No 3, pp. 1-13.

Stufflebeam, D. L.  (2001) “Evaluation checklists: practical tools for guiding and judging evaluations”, The American Journal of Evaluation, Vol 22, No 1, pp. 71-79.

Susser, B. and T. Ariga. (2004) “Teaching e-commerce Web page evaluation and design: a pilot study using tourism destination sites”, Computers and Education, Vol 47 No 4, pp. 399 – 413.

World Bank E-Government Website. Available (http://Web.worldbank.org/WEBSITE/EXTERNAL/TOPICS/EXTINFORMATIONANDCOMMUNICATIONANDTECHNOLOGIES/EXTEGOVERNMENT/0,menuPK:702592~pagePK:149018~piPK:149093~theSitePK:702586,00.html)

Appendix

I. Information Content Criteria

II-Ease of Use

Back to Contents