Electronic Journal of Academic and Special Librarianship

v.8 no.3 (Winter 2007)

Back to Contents

Developing a Web-Based Evaluation Tool for Purchasing Electronic Resources: A Librarian-Faculty-Student Partnership

Peter Linberger, Associate Professor of Bibliography
University of Akron, Ohio, USA

Lori Jean Fielding, Associate Professor of Bibliography
University of Akron, Ohio, USA

Frank J. Bove, Assistant Professor of Bibliography and Electronic Resources Librarian
University of Akron, Ohio, USA


“While the principles of collection development, which were developed in the world of print publications, do not change radically with new publishing technologies, methods of decision making and specific selection guidelines must be adjusted significantly to incorporate new publishing formats.”  (Demas, S. G.)1

The goals of acquiring and maintaining a good collection of resources that directly benefit teaching and research at a university are important ones.  It is nearly impossible for most library budgets to acquire every available electronic research database and the task of deciding which of these resources to purchase has been mostly left to librarians.  In order to have a tool that would aid librarians in determining which electronic resources to acquire, The University of Akron Libraries developed an “Electronic Resources Previews Project Page” and introduced it on the library’s Web site in February 2005.   The purpose of the Previews Project is to involve faculty, students, and librarians together in the evaluation and assessment of new electronic resources.

With the volume and variety of research information becoming available electronically, the librarians at The University of Akron needed a more efficient way of keeping track of what is available and what information these resources provide.  Previously, vendors and publishers would contact any number of librarians and offer a temporary trial of their product; after that the subject librarians would notify other colleagues and various teaching faculty with the logon and trial information.  These librarians and faculty were then asked to logon and try out the resource and report their opinion on its value.  The subject librarians would then forward the information to the Head of Collection Management who would consider these resources for possible purchase depending on the opinions and available funds.  This was typically done through a series of email messages and phone conversations, and there was no consistent procedure in place.  Trials on a number of electronic resource databases have also been available through the statewide consortium, OhioLINK.  A procedure was needed to have a consistent way of arranging for and handling trials of electronic resources, and more importantly, having all the information in one place where librarians, faculty and students could have easy access for previewing and evaluating.   Use of the Previews Project page by the Head of Collection Management would ensure an organized and systematic file of previewed electronic resources.

With the creation and hiring of a new library position in 2004 (Electronic Resources Librarian), the Associate Dean of University Libraries envisioned the design and implementation of a Web-based tool for evaluating new electronic resources.  After the initial plan was written the Associate Dean, the Head of Collection Management, and the Electronic Resources Librarian met to discuss and draw a workflow diagram for the creation, implementation and use of this new Web-based tool which became known as the “e-Resources Previews Project.” The goals and outcomes of this project are as follows:  1) develop a new and easy Web-based evaluation instrument for faculty, librarians, and students to assess individual electronic resources; 2) assess the potential use of a new resource by faculty and students prior to committing financial resources; 3) gather documentation from faculty and students to use as support for any additional funding requests; 4) develop methods for active collaboration between librarians and teaching faculty for using the resources in promoting student success via classroom assignments; 5) use the information gathered from librarians, faculty, and students to help with purchasing decisions;  and 6) acquire the best resources and promote their use on campus. 

Literature Review

The University of Akron’s Previews Project is unique in that it allows for evaluation and assessment of a product not only by librarians and faculty, but by students as well; it provides a means for the University Libraries to gather data for purchasing decisions and additional sources of funding; it facilitates preliminary training on a product during its testing phase; and finally the Previews Project allows for the development of collaboration between librarians and teaching faculty by creating ways that the resource may be used for class assignments.

Hunter notes that in the past academic libraries have moved from collection development to collection management, and that the current trend is to combine collection management with access management and emphasize “providing the right electronic resources to the user at the right time.”2  Level and Myers explore how the traditional, print environment of collection development is changing, and their article provides the details of developing Web accessible collection development tools that have streamlined materials selection.3  Davis’ column focuses on the evaluation, selection, and acquisition of electronic resources as the topic was discussed at the 2000 Charleston Conference, with specific emphasis on the evaluation and assessment of electronic resources after their purchase.4

Much of the literature makes mention of the challenges that the collection of electronic resources presents to libraries.  Indeed, the 2000 Digital Library Federation (DLF) survey, which is explored by Jewel, is oft cited in that it emphasizes that digital collection development is the single greatest challenge confronting academic and research libraries.5  Furthermore, as Allison, McNeil, and Swanson note, collection of electronic resources is often done in the face of increasing costs of resources and decreasing library budgets.6  These authors state that while libraries share the common goals of enhancing learning and ensuring access to high-quality and accurate information, that this goal has become difficult in the realm of electronic resources as libraries must now be able to determine the most cost-effective resources and ways to deliver information.

Now more than ever selecting and acquiring new resources involves a variety of library departments and personnel to coordinate the evaluation, selection, and procurement of resources.  Morris and Larson observe that libraries are responding to customer preferences for electronic resources through the acquisition and management of these products.7 Meanwhile, electronic resources have significantly different characteristics than print resources when it comes to technical services management, and her article provides the details of the evaluation and selection of electronic resources for collection development; as well as contract negotiation, ordering, cataloging, and access management of these materials.  Loghry and Shannon discuss how a variety of library staff members may be involved in the selecting and processing of electronic resources, they note that “subject specialists, reference librarians, acquisitions and cataloging personnel, systems staff, and even library administration may have a role in selection, acquisition, license negotiation, technical setup, and other pre-implementation steps.”8  They specifically detail the workflow issues of managing electronic resources, and detail the confusion and problems that are likely to ensue with having so many departments and personnel involved in the management of electronic products.  They provide the details of a two-part electronic products work form developed at their library that is designed to streamline the workflow and organize the acquisition of electronic materials.  Foudy and McManus provide the details of the utilization of a decision grid to evaluate their institutions subscriptions to electronic resources.9

White writes that “librarians in academic institutions strive to develop and maintain relationships with faculty and students in areas of their subject specialty.”10  He asserts that “librarians in academic settings rely on faculty input for building collections to meet current research needs, curricular content, and changing and emerging disciplines.”11  White states that the advent of electronic resources has changed the approaches to liaison activities and collection development.  His article describes a strategic partnership that was formed between the library and faculty to analyze, evaluate, select, and collaboratively procure funding for electronic resources.  In conclusion he notes that, as this process is designed to focus on acquiring electronic products that support research throughout the university, his library can use the information gathered by this partnership to “leverage support for desired products by demonstrating broad needs throughout the university.”12  And finally, Higa, Bunnett, Maina, et. al., discuss the challenge and the impact that the evolution from print to electronic resources has wrought at their library, and state that this transition has lead to an exhausting reorganization of library staffing and services.13

Development of the Web-Based Page

The Previews Project Web page was originally conceived as part of a larger endeavor to build an in-house, home-grown electronic resource management system (ERMS) based on the recommendations issued by the Digital Library Federation in the summer of 2004.14 To the best of the authors’ awareness, few proprietary ERMS existed and those that did at that time had neither a method nor a process for collecting and collating evaluations of an electronic resource.  And if they did have that functionality, it was not widely known.  Considering the grand scope of building an ERMS from scratch, The University of Akron Libraries decided to purchase a proprietary ERMS to maintain its electronic resources and began to build the Previews Project page as a stand alone evaluative tool for acquiring electronic resources.

The first step in the process entailed determining how The University of Akron Libraries currently evaluated electronic resources.  As mentioned earlier, the Head of Collection Management would field inquiries and requests from the subject librarians and faculty in the form of emails, phone calls, memos, etc.  When the Electronic Resources Librarian investigated how other academic institutions evaluated resources, he found similar methods.

The former evaluation process usually did not include the classic evaluation measures of currency, accuracy, scope and usability; nor did the method inquire whether the resource met the specific research needs of faculty or the needs of students to complete classroom assignments. Therefore, the new tool needed to collect data from faculty and students that would directly address those needs. A workflow was developed that outlined the basic process of requesting a resource for trial, examining said resource, collecting data from the reviewers and collating the data into a unified report.

Image 1
Previews Project Process Diagram

Open source tools were chosen to create the Previews Project page by incorporating MySQL as the backend database and PHP as the scripting language on a Linux machine running Apache Web Server (classic LAMP setup). A request form was designed to notify Collection Management and the Electronic Resource Librarian of the desire by an individual to preview a resource. The form asks for the requestor’s contact information, and information on the resource such as scope, description, URL and vendor/publisher.  This information is stored in the database to populate the access and evaluative portions of the Web site. The evaluation forms were written so that users would be able to examine the resource and to complete and address their specific category - librarian, faculty, or student. The evaluation form asks specific questions to which they may respond to on a continuum ranging from strongly agree to strongly disagree. Upon submittal of the form, these responses are assigned a numeric value and stored in the database.  The questions also allow for a free-text response from the reviewer:

When the trial period ends, the resource is no longer available for evaluation. The continuum and free-text data is designed to tabulate and display as a generated report available for viewing on the Web site. The numbers are averaged and the free-text responses are displayed by user groups. Anonymity is preserved as the individual’s contact information is suppressed from any display.

Initially, the Previews Project page was hosted on the The University of Akron Libraries’ Intranet; however, students and faculty had a difficult time accessing the Web server. Due to this concern and a subsequent hard-drive crash on the Intranet, the Previews Project page was moved to a more accessible Web server running MS IIS server software. The new server provided the opportunity to upgrade the scripts for a new non-LAMP configuration of IIS 5.1, PHP 5.1.1 and MySQL 5.0.16. The Previews Project page is always being modified and updated with emphasis being placed on the user front-end. Much of the administrative tasks have yet to be developed and are currently maintained by accessing the database through phpMyAdmin, a GUI that allows direct manipulation of the database. At this time, an easy-to-use administrative back-end is in development.

Use of the Previews Project Page

Once the Previews Project page was posted on the library’s Web site, flyers were created and distributed across campus.  The page was also advertised in the university’s e-newsletter which is distributed electronically campus wide.  Additionally, subject librarians informed teaching faculty of the Web page and its intent.

As mentioned earlier, students, faculty, and librarians are able to request that an electronic resource be considered for preview by simply filling out a short form.  Notices of new and current products are sent regularly to librarians from various publishers and vendors.  At The University of Akron, trials of possible OhioLINK- acquired resources are also previewed here and the Head of Collection Management arranges for those databases to be added to the Previews Project page.  The Head of Collection Management decides whether or not to proceed with any trial or preview.  If so, the Head of Collection Management notifies the Electronic Resources Librarian who then contacts the publisher/vendor and arranges a trial.  Occasionally, some publishers do not allow for a trial of their products, but this is a rarity.  Once the trial information is available, the Electronic Resources Librarian posts the information on the Previews Project page and informs all librarians of its availability.  The posting of the e-resource trial on the Previews Project page includes such information as the name and description of the resource, publisher, scope, research level, and occasionally cost.  Subject librarians in turn inform their faculty and ideally, faculty alert their students.  Most trials for research databases last for thirty days and are only accessible for the faculty, librarians, and students at the university.

The tabulated evaluation results for each trial are viewable to anyone who logs on and are available even after the trial period has ended.  Every resource that has been reviewed is listed on the Previews Project page, and also states the status for each: r = requested, a= active, u= under consideration, a= accepted, R= rejected, ! = expires this week. 

Decision Making

Having a preview arranged for and evaluated in one online location makes organizing and analyzing much easier for the decision makers and the input from faculty and students is invaluable.  If a preview of a particular electronic resource has little response and poor evaluations from campus users, then the decision not to purchase is an easy one.  Conversely, a resource that has a high number of reviewers and high scores, and positive comments, then that too will make for an easy purchasing decision.  The cost and availability of funds are also a crucial factor, and is often the deciding factor.  A high response from the campus community for a particular resource may also be used for securing additional funds.

At The University of Akron Libraries, we have had several databases that have been previewed twice and then purchased.  A request to preview a resource a second time was made because enhancements or changes had been made to the product since the last time it was previewed, or there was interest in a particular electronic resource from a new faculty member or librarian.  These products include ArtStor, Biography Resource Center, Factiva, Global Financial Database, 19th Century U.S. Newspapers, and Sabin Americana Digital Archive.

Since February 2005, the Previews Project page has managed the preview and evaluation of fifty-nine electronic resources.  Of those fifty-nine products, the library has purchased twenty-two. Some of the purchases, such as the Early English Books Online database, had an overwhelmingly positive response from faculty and students.

Other benefits of the Previews Project page include the ability to compare similar resources and products from different publishers.  An example of this is the evaluation of The U.S. Congressional Serial Set which is available from two different vendors.  The Previews Project page user comments and reviews helped us in determining which vendor to choose.  Another example is the Mental Measurements Yearbook, which is also available from two different vendors, and again the composite evaluations were essential in determining which version to purchase.  In addition to this, input from the Previews Project page has been beneficial when deciding whether to retain various print reference sources.


Regardless of the published format of research information, librarians still need to apply the same criteria for evaluation and selection.  However, technological advances make the task more challenging because of the vast amount of online information.  This challenge has been confronted by the librarians at The University of Akron by creating an online management tool for previewing and critiquing electronic resources, the e-Resources Previews Project Page.    This Web page has been available to the campus community since February 2005 and enables librarians, faculty, and students to request a trial of a particular research database and test drive the product in order to determine its usefulness.  To date, this collaboration among librarians, faculty, and students has resulted in the preview and evaluation of fifty-nine electronic resources, and this input from the campus community has been valuable for librarians in making the best possible purchasing decisions.

Additionally, the Previews Project has streamlined the process of selecting, evaluating, acquiring, and implementing electronic resources at The University of Akron.  Library departments now have an efficient and manageable work flow for the procurement of e-resources.  Duplication of work between subject librarians and those facilitating purchasing decisions has been minimized.  Furthermore, this easily accessible online tool has taken the place of other communiqués such as brochures, flyers, emails, phone messages, and the like.  And in times of financial constraint, presenting the data gathered from the Previews Project to the administration may become necessary in order to procure additional funding.   

Appendix A.
Image 2

Appendix B.
Image 3

Appendix C.
Image 4

Appendix D.
Image 5

Appendix E.
Image 6

Appendix F.
Image 7

Appendix G.
Image 8


1. Demas, S. G. 1994. Collection Development for the Electronic Library: A Conceptual and Organizational Model. Library Hi Tech 12, (3): 71-80.

2. Hunter, Karen. 2005. Access management: Challenging orthodoxies. Journal of Library Administration 42, (2): 57.

3. Level, Allison, and Sarah Myers. 2003. Creating internal web tools for collection development. Collection Building 22, (4): 162-166.

4. Davis, Susan. 2001. Serials spoken here: Reports of conferences, institutes, and seminars. Serials Review 27, (1): 84.

5. Jewell, Timothy D. Washington, D.C. Digital Library Federation, Council on Library and Information Resources. 2001. Selection and presentation of commercially available electronic resources: Issues and Practices.

6. Allison, DeeAnn, Beth McNeil and Signe Swanson. 2000. Database selection: One size does not fit all. College & Research Libraries 61, (1) (01): 56.

7. Morris, Kathleen, and Betsy Larson. 2006. Revolution or revelation? Acquisitions for the digital library. Acquisitions Librarian 18, (35): 97-105.

8. Loghry, Patricia A., and Amy W. Shannon. 2000. Managing selection and implementation of electronic products: One tiny step in organization, one giant step for the University of Nevada, Reno. Serials Review 26, (3): 34.

9. Foudy, Gerri, and Alesia McManus. 2005. Using a decision grid process to build consensus in electronic resources cancellation decisions. Journal of Academic Librarianship 31, (6) (11): 533-538.

10. White, Gary W. 2004. Collaborative collection building of electronic resources: A business faculty/librarian partnership. Collection Building 23, (4): 177.

11. Ibid., 177.

12. Ibid., 180.

13. Higa, Mod Lou, Brian Bunnett, Bill Maina, Jeff Perkins, Therona Ramos, Laurie Thompson, and Richard Wayne. 2005. Redesigning a library's organizational structure. College & Research Libraries 66, (1) (01): 41-58.

14. Jewell, Timothy D., Ivy Anderson, Adam Chandler, Sharon E. Farb, Kimberly Parker, Angela Riggio and Nathan D. M. Robertson. 2004. Electronic Resource Management: Report of the DLF ERM Initiative. Digital Library Federation. http://www.diglib.org/pubs/dlf102/ERMFINAL.pdf (accessed July 6, 2007).

Back to Contents