Electronic Journal of Academic and Special Librarianship

v.9 no.3 (Winter 2008)

Back to Contents

Distance Education Library Services Assessment

Judy Block, Distance Education Librarian
Eastern Michigan University, USA
judy.block@emich.edu

Abstract

The primary purpose of assessment in academia is to support high-quality learning. To accomplish this tracking, monitoring and documenting student activity is important to providing excellent library service. By gathering information on student learning, assessments can pinpoint areas for improvement. Ideally, assessment should be quantitative, qualitative, and ongoing and have the ability to address different skills. Assessment is most effective when based on faculty teaching and student learning. Outcomes assessments are the only credible assessment for the distant learner. They are necessary to improve student learning results and are a means of gathering information about student learning that is built into and a natural part of the teaching-learning process. In the book, Assessing Learners Online, the comment was made “It is important to emphasize that the usefulness of an assessment depends largely on how well the domain being measured is defined or how adequately the norm group with whom students are being compared is known or described” (Oosterhof, Conrad, & Ely, 2008, p. 43). Means and methods that are developed by an institution's faculty are valuable tools when based on their teaching approaches and students successes and learning goals. The ability to track, monitor, and document students’ activities is a necessary and productive activity for the library. It is imperative that it is ongoing and assesses different skills. It is a proven way to assess instruction and learning. Comeaux says, “In general, e-assessment methods are classified into two categories, namely quantitative and qualitative assessment tools” (Comeaux, 2005, p. 90). Assessment measures must be quantitative as well as qualitative. Quantitative methods are aimed at the amount and frequency of assessment.  Qualitative assessment is one that does not rely on numbers and statistics as a method of proving a theory or identifying facts. Qualitative assessment addresses methods that rely on descriptions rather than numbers. This paper summarizes the key roles assessment plays such as providing opportunities for student feedback and measuring the library’s impact. “As the world of information becomes more complex, information literacy has become an increasingly important part of the education process”(Smith, 2001, p. 13). The Distance Education Librarian at Eastern Michigan University of Ypsilanti, Michigan, has primarily relied on questionnaires as a tool to evaluate library services provided to the distance learning community. The content of the questionnaires have open-ended questions so that the students’ can express themselves adequately. This allows the Distance Education Librarian to measure the impact of the library services provided the distance learning community. Through assessment, educators are able to provide quality learning experiences to students and support successful educational endeavors.

Assessment Defined

Oosterhof states, “The primary use of an assessment is to evaluate learners” (Oosterhof, 2008, p. 70). Assessment is an integral part of instruction and learning and any effective assessment tool may be used. All good assessment begins with statements of what we want our students to know and be able to do.  This begins with the development of meaningful goals and standards. One assessment tool is called authentic assessment. Authentic assessment presents students with real-world challenges that require them to apply their relevant skills and knowledge and it places priority on evaluating students' abilities in 'real-world' contexts. In other words, students learn how to apply their skills to authentic tasks and projects and utilize learning activities that encourage students to use higher-order thinking skills. Authentic assessment is any type of assessment that requires students to demonstrate skills and competencies that realistically represent problems and situations likely to be encountered in daily life. Students are required to produce ideas, to integrate knowledge, and to complete tasks that have real-world applications.

Information literacy involves mastery over a complex set of concepts and skills and their interplay. While knowledge of library and computer skills is an integral part of information literacy, these skills merely serve as tools for realizing its higher goals that include the ability to identify, access, evaluate, and interpret relevant information for making informed decisions. It takes reinforcement, sustained effort, and time to develop higher order information literacy skills (Shikha, 2007, p. 127.)

The Distance Education Librarian utilizes authentic assessment when she has the students practice searching the databases while at the Regional Centers. Searching the databases has real-world applications since in today’s world the students will need high quality computer technology skills. Knowing how to obtain and evaluate information will be a lifelong skill they will need to master because of its impact on all areas of their lives. “Authentic assessment overall provides for continual feedback, allowing the learner to adjust and improve performance” (Janesick, 2001, p. 80). In authentic assessment, students use remembered information in order to produce an original product, participate in a performance, or complete a process. With assessment we can improve instruction, and with good instruction we can increase the quality of achievement for all students. The feedback and results enable the Distance Education Librarian to design the next steps for improving both teaching and learning.

Library Services and Assessment

A needs assessment is necessary to discover the needs of the distance learning population when attempting to obtain a specific goal. Effective assessment methods are needed to make sure the distance learning program stays on track because how well you serve the students depends on how accurately you assess your program’s success. Since many distance learning students are returning to school after a long absence, the academic environment is a brave new world and the librarian must be patient with them. They require access to the full range of library services so it is vital to the distant learners to be able to access library materials and services because they are located a long distance away from the library physically. The adoption of a set of principles of good practice, coupled with periodical review and assessment, can be invaluable to institutions that aim to demonstrate that distance learning programs are subject to the same level and scope of scrutiny as conventional campus-based programs (Dasher-Alston & Patton, 1998, p. 15).  All of the above needs must be met in a timely manner to assure student success. The Distance Education Librarian must work very closely with the other members of the team for the distant learner’s success. For example, the librarian must work closely with the instructors of the courses to identify what is expected of the student so the librarian can give proper research assistance for the successful completion of the course.  Eastern Michigan University’s (EMU’s) Distance Education Librarian provides a week or two of orientation to the library as chat on eCollege where the students email questions and via emails they are answered. In partnership with the Continuing Education Department at EMU, the Distance Education Librarian has formulated a distance learning policy for library services at EMU. The goals for EMU’s distance learning program are:

  1. To provide a quality distance learning program.
  2. To provide training to students on how to access information.
  3. To enhance students’ critical thinking and research skills.
  4. To enhance students’ self-reliance in using online databases and the Internet for research.
  5. To accommodate the informational needs of the distance learning community.
  6. To assist distance learners to become independent in library literacy skills.
  7. To provide an environment for distance learners to become challenged to be problem solvers and seek their own solutions.
  8. To augment computer technology skills to enhance the educational process.

The Distance Education Librarian has published performance standards to clearly explain expectations to students. A quantitative assessment is essential to define the monitoring of the objectives.

Distance education students need to partner with the library because librarians will be supporting the students’ research needs. The librarian is an integral part of the distance education team because the library is a storehouse of knowledge. Each student must be able to access that knowledge. The librarian helps the student navigate through the storehouse. Successful implementation of a distance education program means everyone on the team must have a learner-focus. Continuous monitoring of the program is necessary to keep the quality high. Careful analysis is needed to see what is working and what needs changing. The team is actively and cooperatively involved in creating, providing, and improving the instructional program. If the programs are managed effectively, they will respond not only to the academic and professional needs of the students, but also to the educational and financial needs of the institutions of higher education. The Distance Education Librarian has developed a self-paced library tutorial for the online student. It is entitled, “Distance Education Library Resources” and has been created for both platforms supported by Eastern Michigan University (EMU), WebCT and eCollege. The students enrolled in a course presented via WebCT have to sign up to receive the class. The students enrolled in an online class via eCollege have the course automatically delivered to them along with their original course. The course has information given to them in conversational style. After they have read the information in a section, they have review questions and an answer key provided. Some instructors find it useful to have their students view the tutorial for class credit.

Methodology

Another tool for assessment the Distance Education Librarian uses is a survey that is located on her Web Page at http://ce.emich.edu/delr. This instrument tells the librarian the distance learning students’ research needs and gives them a chance to make comments about what would assist them to be successful in their studies. It is a needs assessment for the distance learning program of Eastern Michigan University. This survey makes them feel they have been able to voice their opinion about the educational life.

“An online questionnaire can be a good tool to conduct a survey of your students. It can be valuable for gathering opinions and views from the students.” (Smith, 2001, p. 146)

The questionnaire given at EMU is designed to identify strengths and weaknesses of library services provided to the distant learner. The survey questions have broad themes. These questionnaires are distributed to distance learning students at the regional centers and are online at the Distance Education Librarian’s Home Page for the online students to participate in the assessment process.

A questionnaire is a written list of questions distributed to and completed and returned by one or more users. Although the term is often used interchangeably with the word survey, a questionnaire is in fact an instrument that can be used to conduct a survey. Keep in mind when using a questionnaire that you are putting the burden on your users to return it to you (Smith, 2001, 176).

Formative assessments ought to be understood as the most important assessment practice. To promote student learning it is important to introduce a formative type of assessment. Formative assessments are ongoing, repetitive measures designed to provide information to both the instructor and students concerning students' understanding of small segments of course material. The key to formative assessment is the role of feedback. “Formative evaluation of students’ achievement means we are judging the quality of a student’s achievement while the student is still in the process of learning”(Nitko, 2007, p.8). The purpose of this technique is to improve quality of student learning. Formative assessment data, however, can contribute to a comprehensive assessment plan by enabling the Distance Education Librarian to identify particular points in a program to assess learning and monitor the progress being made towards achieving learning outcomes. Assessing student work on a formative basis helps this librarian evaluate student progress and gaps in learning. “Formative assessments must precede the conclusion of instruction”(Oosterhof, 2008, p. 7). It is essential to continually check for understanding. This is done by asking the students a series of questions after the presentation is completed. Since many people must collaborate to produce and disseminate quality distance educational programming, the need to plan and coordinate staff activity is essential.  One of the support services that cannot be overlooked is the library. Library services are essential to support a successful distance education program. The librarian can shift the focus from explaining library resources to meeting the ongoing information needs of the students in a broad information environment. “A librarian-administrator is needed to plan, implement, coordinate, and evaluate library resources and services addressing the information and skills needs of the distance learning community”(Bryant, 2001, p. 59). Since the Fall of 2000, this Distance Education Librarian has relied on a questionnaire to assess the quality of the program. The web page, http://people.emich.edu/jblock is always under constant revision due to the requests obtained from the distant learners. One respondent answered that he/she goes to the frequently asked questions section of the Web Page before asking the Distance Education Librarian because most of the time the question has already been addressed.

The Distance Learning Librarian’s job at EMU is to provide library research support to students enrolled in at least one distance education class who may never have the opportunity to physically enter in EMU’s library. Students need convenient access to services that provide the guidance and personal support required to complete their programs successfully. The surveys are handed out personally to all students in the Information Literacy sessions given at the Regional Centers.

Findings

The 2007-2008 surveys handed out personally to all students in the Information Literacy sessions given at the Regional Centers provided rich data. The number of surveys given out was:

Number of Surveys distributed Regional Centers
20 Brighton
29 Detroit
24 Jackson
10 Livonia
19 Monroe
5 Traverse City

Question 1: Which regional center are you a member of. The number of responses was as follows:

Number of Students Regional Center
13 Brighton
10 Detroit
3 Flint
3 Grand Rapids
10 Jackson
8 Livonia
9 Monroe
5 Traverse City
12 Online

Question 2: Have you visited the Distance Education Librarian’s Home Page? Survey results are as follows:

Results from Regional Centers Results from Online
Yes-19 Yes-12
No-8 No-1

Question 3: After reading the Web Page, do you see any ways it could be improved?

Results from Regional Center Results from Online
No-17 No-18
Have not visited-4 Have not visited-1

Question 4: Is it more convenient to correspond with your Distance Education Librarian through e-mail and have your questions answered individually?

Results from Regional Centers Results from Online
E-mail-21 E-mail-12
No-1 No-0
Comment: FAQ section on home page answers all of my questions.

Question 5: Are the instructions for various databases helpful?

Results from Regional Centers Results from Online
Yes-21 Yes-12
No-0 No-0

Question 6: Do you find the “How to Write the Research Paper” tutorial helpful?

Results from Regional Centers Results from Online
Yes-16 Yes-15
No-3 No-0
Have Never Used-6 Have Never Used-0

Question 7: Do you find “Doing Library Research on the World Wide Web” helpful? (Regional Centers)

Results from Regional Centers Results from Online
Yes-16 Yes-19
No-1 No-0
Have Never Used-0 Have Never Used-2

Question 8: What additional resources can I provide to enhance your Distance Learning experience? (Regional Centers)

Regional Center Comments Online Student Comments
No additional resources No additional resources

Question 9: I am always interested in any comments you have about the Distance Learning Program. Please comment.

Regional Center Comments Online Student Comments
-“Librarian always responds by the next business day”-2 “I am impressed with how well communication is delivered to the Distance Learning students”
-“Great program”-3 -“Great program”-3
“Program really makes the extended campus population feel welcomed and part of Eastern Michigan University” “Very positive experience”
“Distance Education Librarian is a wonderful resource” “Distance Education Librarian is a wonderful resource”

Some questions on the survey for the online students have been added.

Question 7: What additional resources can we provide to enhance your On-line Learning experience?

Online Results
None-12

Question 10:  Do you feel additional computer training would benefit you in preparation for your on-line class?

Yes No
8 3
Online Student Comments
-“I took the demonstration course on eCollege and learned what I had to do. Everyone should be made to take the demonstration class before they start an on-line class. It really helps!”-2

Question 11: Do you have a personal computer at home?

Yes No
12 0

Question 12: Do you feel comfortable communicating with your professor on-line?

Yes No
12 0

Conclusion

The field of distance education is growing rapidly. Due to unprecedented demand for distance education it is vital that we in the library profession respond to this exciting challenge for a new learning environment. Outcomes assessments provide credible information and feedback. Assessment is necessary because the distance learning students are an invisible population and librarians need to be assured that their needs are being met.

 

Appendix 1. Survey for Distance Education Students

  1. Which Regional Center are you a member of?
  2.  

  3. Have you visited the Distance Education Librarian’s Home Page at:  http://people.emich.edu/jblock?
  4.  

  5.  After having read the Web Page, do you see any ways it could be improved?
  6.  

  7. Is it more convenient to correspond with your Distance Education Librarian through e-mail and have your questions answered individually?
  8.  

  9. Are the instructions for the various databases helpful? How could they be improved?
  10.  

  11. Do you find the “How to write the Research paper” tutorial (http://people.emich.edu/jblock/research) helpful?
  12.  

  13. Do you find the “ Doing Library Research on the World Wide Web” (http://people.emich.edu/jblock/library.html) helpful?
  14.  

  15. What additional resources can I provide to enhance your Distance Learning experience?
  16.  

  17. I am always interested in any comments you have about the Distance Learning Program. Please comment.

 

Appendix 2. Survey for On-Line Learning Students

  1. Which On-line class(es) are you a member of?
  2.  

  3. Have you visited the Distance Education Librarian's Home Page at http://people.emich.edu/jblock/?
  4.  

  5. After having read the Web Page, do you see any ways that the Library could improve on its service to the On-line Learner?
  6.  

  7. Is it more convenient to correspond with your Distance Education Librarian through e-mail and have your questions answered individually?
  8.  

  9. Do you find "Doing Library Research on the World Wide Web" (http://people.emich.edu/jblock/library.html) helpful?
  10.  

  11. Do you find the “How to write the Research paper” tutorial (http://people.emich.edu/jblock/research) helpful?
  12.  

  13. What do you like about the On-line Learning experience?
  14.  

  15. What additional resources can we provide to enhance your On-line Learning experience?
  16.  

  17. The Library is interested in any comments you have about its Distance Learning Program. Please comment.
  18.  

  19. Do you feel additional computer training would benefit you in preparation for your On-line class?
  20.  

  21. Do you have a personal computer at home?
  22.  

  23. Do you feel comfortable communicating with your professor on-line?

 

References

Bryant, Eric. "Bridging the Gap." Library Journal 126 (2001): 58-60.

Dasher-Alston, Robin M., and Gerald W. Patton. "Evaluation Criteria for Distance Learning." Planning for Higher Education 27 (1998): 11-17.

Janesick, Valerie J. The Assessment Debate: A Reference Handbook. Santa Barbara, CA: ABC-CLIO, 2001.

Nitko, Anthony J. Educational Assessment of Students. 5th ed. Upper Saddle River, NJ: Pearson/Merrill Prentice Hall, 2007.

Oosterhof, Albert, Donald P. Ely, and Rita-Marie Conrad. Assessing Learners Online. Upper Saddle River: Pearson/Merrill Prentice Hall, 2008.

Sharma, Shikha. "From Chaos to Clarity: Using the Research Portfolio to Teach and Assess Information Literacy Skills." The Journal of Academic Librarianship 33 (2007): 127-35.

Smith, Susan Sharpless. Web-Based Instruction : A Guide for Libraries. 2nd ed. Chicago: American Library Association, 2006.

Back to Contents