Electronic Journal of Academic and Special Librarianship

v.8 no.1 (Spring 2007)

Back to Contents

Looking for a Link: Comparing Faculty Citations Pre and Post Big Deals

Donald Taylor
Simon Fraser University Library, Burnaby, BC, Canada
dstaylor@sfu.ca

Abstract

Big Deals expand an institution's access to scholarly literature, with usage statistics showing that previously unavailable journals receive significant usage. To determine if faculty use these new e-journals in their research, the Simon Fraser University (SFU) Library analyzed SFU citation data to journals from selected Big Deals for two years prior to signing a major Big Deal (1993 and 1998) and for two consecutive years following the Big Deal (2004 and 2005). Pre Big Deal, the percentage of citations to journals that are part of Big Deals but were previously not subscribed to was an average of 2.6%. Post Big Deal this increased to an average of 6.1%.

Introduction

Big Deals are often promoted by librarians on the premise that they will provide researchers with greater access to scholarly information, and by their own admission Big Deals are well liked by faculty whose institutions have Big Deals.1 Like many libraries, the Simon Fraser University Library (SFU Library) has greatly expanded its patrons online access to the journal literature through consortial Big Deals.2 By virtue of its membership in the Canadian Research Knowledge Network (CRKN) and the Council of Prairie and Pacific Libraries (COPPUL) the SFU Library has greatly increased SFU’s access to the journal literature through a number of Big Deals with publishers such as Elsevier, Cambridge University Press, Springer, and Project Muse. While usage statistics show that for Big Deals there is significant usage of titles not previously available in libraries, these statistics do not tell us who is using the journals or why the journals are being used.3 Also, the number of Big Deals is increasing in Canada due to the generally favourable view Canadian academic libraries have of them.4 Consequently, it is important for Canadian libraries to study the underpinnings of arguments in favour of Big Deals. To determine if faculty researchers were utilizing previously unavailable journals to a greater degree in their research than they were prior to gaining access to Big Deals, the Simon Fraser University Library analyzed citation data from Web of Science for a random sample of SFU authors for the years 1993, 1998, 2004 and 2005.

Background

Simon Fraser University Library is a medium sized academic library with a substantial investment in e-journals and a stated policy in favor of migrating from print to online.  Over 50% of our e-journals come via Big Deals with publishers like Elsevier, Springer, Cambridge, and Wiley. Like many institutions Simon Fraser University for many years purchased all, or almost all, journals published by certain scholarly societies such as the American Chemical Society and the Institute of Physics, and then simply moved these into the online environment at the appropriate time. SFU does not consider these arrangements Big Deals, even if they are done through consortia, since they did not provide SFU with significantly more content than we previously received in paper from these organizations.

Our participation in CRKN (originally known as the Canadian National Site Licensing Project) and COPPUL allowed us to participate in a variety of Big Deals. CRKN licensed Big Deals to Springer, Academic Press (now part of Elsevier) and Elsevier, providing us with access to almost all journals published by these publishers. Through COPPUL we licensed Wiley, Kluwer (now rolled into the CRKN Springer deal), Cambridge University Press, and Project Muse. In 2006 more Big Deals were signed onto via COPPUL and more are anticipated for 2007. It should be noted that Canadian libraries view Big Deals in a positive light and are continuing to move forward with further purchases. This attitude is different than the attitude held towards Big Deals in some other countries, notably large US academic libraries

Anecdotal evidence from students and teaching faculty, as well as analysis of usage statistics, online reserves, and declining interlibrary loan borrowing for journal articles, indicates that titles in Big Deals that were not previously available to SFU are being used for teaching purposes and for student papers. For example, usage statistics show that 13 of the top 100 most used e-journals at SFU were not available prior to gaining access via Big Deals.

The SFU Library, as a medium sized academic library, is the type of institution that should benefit significantly from Big Deals as we are able to create a research collection of electronic journals well beyond what we, or most of our consortium partners, could do individually.5 To illustrate this fact, table 1 compares the SFU Library’s original print holdings of Big Deal publishers to the size of the individual Big Deals we signed with these publishers.

Table 1. SFU Library print holdings vs. Big Deal holdings
Big Deal package Print holdings from publisher prior to signing Big Deal Number of journals in Big Deal package
Academic Press 77 205
Cambridge University Press 76 183
Elsevier 310 1688
Kluwer 68 685
Project Muse 94 281
Springer 61 494
Wiley 88 417

Usage statistics show that the Big Deal journals are used extensively at SFU, but are faculty utilizing these new e-journals in their research? Did the purchase of e-journals not previously held in print affect citation patterns of SFU faculty? To find out, the SFU Library, inspired by Parker and Bauer’s pre and post Big Deal citation analysis at Yale, analyzed citation data from Web of Science for a random sample of SFU authors for the years 1993, 1998, 2004 and 2005.6

Methodology

To determine if SFU faculty are utilizing the new journals now available to them via Big Deals, the SFU Library decided to compare SFU citation data from two different years prior to signing a Big Deal and for two consecutive years following the Big Deal. The dates 1993 and 1998 were chosen as the pre Big Deal dates and 2004 and 2005 were chosen as the post Big Deal dates. Originally 2003 and 2004 were chosen, but it was determined that with the delay in publication times, the time between when research begins and when a manuscript is submitted for publication, and the fact that Elsevier’s Science Direct Big Deal and Wiley’s Big Deal were not available to SFU until January 2003 and February 2003 respectively, that the citation data from 2003 publications would not provide an accurate picture of SFU faculty use of Big Deal e-journals. Table 2 lists the Big Deals that SFU analyzed for this project.

Table 2. Big Deals analyzed
Package Date deal signed Consortium
Academic Press* 2001 CRKN
Cambridge University Press 2003 COPPUL
Elsevier 2003 CRKN
Kluwer** 2002 COPPUL
Project Muse 1998 COPPUL
Springer 2001 CRKN
Wiley InterScience 2003 COPPUL
*Now part of the CRKN Elsevier deal.
**With the merger of Springer and Kluwer, the Kluwer contract was subsumed into the CRKN Springer deal in 2005.

SFU’s subscription to Project Muse began in January 1998 so it fell within the last year of print only citation analysis. However, none of the new journals contained in Project Muse were cited in the sample of SFU authored articles from 1998, so the fact it was available to faculty in 1998 makes no difference to the methodology or findings of this study.

Also, in 2002, SFU absorbed the small, three year old Technical University of British Columbia (TechBC), so any publications written prior to the spring of 2002 by their faculty that later joined SFU would not be represented in this study.

SFU utilized a similar methodology to that used by Parker and Bauer in their citation comparison of usage before and after the arrival of electronic journals at Yale.7 The Web of Science database, incorporating the Science, Social Sciences, and Arts and Humanities Citation Indexes, was used as the source of citation data. We did not analyze all SFU citations, but instead chose a random sample. The random sample was determined by searching for articles with an address of “Simon Fraser” in conjunction with an author’s last name beginning with the letter E, M, N or Y. These letters were chosen by using a random number generator to pick four numbers between 1 and 26 inclusive – where the number represents the corresponding letter of the alphabet. The sample sizes were sufficient to provide a margin of error of 6% at a 90% confidence level or better.  The resulting records from Web of Science from each year were exported to an End Note database, deduped, and then exported to an Excel spreadsheet for analysis. Citations to non-journal article documents and preprints were removed. The citations referenced for each article from 1993 and 1998 were checked against the SFU Library’s catalog to see if the journal had been subscribed to in print when it was referenced and to determine if Simon Fraser University had access to it now as part of a Big Deal. The citations from 2004 and 2005 were checked to see if we had access to the journals in either online format as part of a Big Deal or in print. After this was done, the journals were sorted by publisher and the citations from each year were analyzed to see what percentage of citations and cited journals were available from each of the publisher packages analyzed, and what had been the increase or decrease in citations to journals from these publishers over the years.

Findings

Table 3 lists the total number of Simon Fraser University authored articles in Web of Science for each year analyzed and size of the corresponding sample.

Table 3. Total citations and sample sizes
Year Total no. of SFU publications in Web of Science Sample size
1993 433 150
1998 557 175
2004 624 271
2005 880 305

The number of SFU publications indexed in Web of Science more than doubled between 1993 and 2005. This can be attributed to an increase in new faculty hirings resulting in a situation where more faculty are publishing in order to obtain tenure or promotion, and also a general trend by faculty members at SFU to publish more often in journals than in other types of media.

Analysis of citations

In 1993 and 1998 (pre Big Deal years), the percentage of citations that are from journals previously not subscribed to by the SFU Library but are now part of Big Deals was rather low, an average of 2.6%. This percentage increased to an average of 6.1% for 2004 and 2005, the years after Big Deals were signed and thus when faculty could more easily access these journals as they were now online. Although these journals would always have been available to faculty via the Library’s Document Delivery Service, one could infer that faculty are more willing to reference an article the easier it is to access.

Table 4. 1993 Citations
Package Percentage of total citations Percentage of Citations from previously unavailable journals
Academic Press 2.9% 0.03%
Cambridge University Press 1.0% 0.03%
Elsevier 14.8% 1.40%
Kluwer 1.8% 0.07%
Project Muse 0.4% 0.00%
Springer 2.5% 0.14%
Wiley 1.7% 0.07%
TOTAL 25.1% 1.70%

 

Table 5. 1998 Citations
Package Percentage of total citations Percentage of Citations from previously unavailable journals
Academic Press 2.1% 0.03%
Cambridge University Press 0.85% 0.00%
Elsevier 16.6% 1.80%
Kluwer 2.6% 0.65%
Project Muse 0.2% 0.03%
Springer 2.7% 0.80%
Wiley 2.1% 0.17%
TOTAL 27.2% 3.50%

 

Table 6. 2004 Citations
Package Percentage of total citations Percentage of Citations from previously unavailable journals
Academic Press 1.9% 0.08%
Cambridge University Press 0.82% 0.05%
Elsevier 14.1% 1.90%
Kluwer 2.0% 0.44%
Project Muse 0.18% 0.05%
Springer 3.0% 0.33%
Wiley 2.3% 0.53%
TOTAL 24.2% 3.4%

 

Table 7. 2005 Citations
Package Percentage of total citations Percentage of Citations from previously unavailable journals
Academic Press 3.8% 0.42%
Cambridge University Press 0.49% 0.06%
Elsevier 15.5% 5.7%
Kluwer 2.1% 1.10%
Project Muse 0.15% 0.01%
Springer 3.5% 0.61%
Wiley 2.8% 0.82%
TOTAL 28.4% 8.8%

Using just the number of citations, without worrying about the number of unique journals cited, 2005 appears to be the year where faculty citation of Big Deal journals had the most impact with fully 8.8% of all citations coming from journals that were only available to SFU through a Big Deal. Also, 2005 saw 28.4% of all citations come from a journal available through one of the seven Big Deals analyzed, which is the highest for any year.

Analysis of cited journals

For the years analyzed, a steady increase in the number of journals cited that were previously not subscribed to by the SFU Library but are now part of Big Deals is readily apparent. In 1993 and 1998 an average of 5.2% of all journals cited were from the group of newly acquired journals, whereas in 2004 and 2005 – after the Big Deals were signed - the average increased to where 10% of all journals cited came from the group of newly available journals.

In 1993 4.22% and in 1998 6.22% of all journals cited were from the group of newly acquired journals. After signing onto Big Deals, these numbers rose to 7.48% in 2004 and to 12.53% in 2005. 2005 also saw 30.76% of all journals cited come from one of the seven Big Deals analyzed, which was the highest of the four years analyzed.

Table 8. 1993 Journal Citations   
Package Number of journals cited that were previously unavailable Percentage of journals cited that previously were unavailable Total number of journals cited from publisher* Percentage of total journals cited
Academic Press 0 0% 22 2.73%
Cambridge Univ 1 0.12% 14 1.74%
Elsevier 25 3.1% 116 14.39%
Kluwer 4 0.5 % 19 2.36%
Project Muse 0 0.0% 6 0.74%
Springer 2 0.25% 24 2.98%
Wiley 2 0.25% 19 2.36%
TOTAL 34 4.22% 22027.3%
*806 journals in total were cited in the sample

 

Table 9. 1998 Journal Citations   
Package Number of journals cited that were previously unavailable Percentage of journals cited that previously were unavailable Total number of journals cited from publisher* Percentage of total journals cited
Academic Press 1 0.09% 26 2.34%
Cambridge Univ 0 0.% 16 1.44%
Elsevier 29 2.61% 155 13.98%
Kluwer 20 1.8 % 40 3.61%
Project Muse 1 0.09% 33 2.98%
Springer 11 0.99% 29 2.61%
Wiley 7 0.63% 8 0.72%
TOTAL 69 6.22% 30727.3%
*1109 journals in total were cited in the sample

 

Table 10. 2004 Journal Citations   
Package Number of journals cited that were previously unavailable Percentage of journals cited that previously were unavailable Total number of journals cited from publisher* Percentage of total journals cited
Academic Press 4 0.22% 26 1.41%
Cambridge Univ 2 0.11% 18 0.98%
Elsevier 72 3.9% 266 14.42%
Kluwer 17 0.92 % 42 2.66%
Project Muse 2 0.11% 9 0.49%
Springer 18 0.98% 71 3.85%
Wiley 23 1.25% 63 3.41%
TOTAL 138 7.48% 50227.21%
*1845 journals in total were cited in the sample

 

Table 11. 2005 Journal Citations   
Package Number of journals cited that were previously unavailable Percentage of journals cited that previously were unavailable Total number of journals cited from publisher* Percentage of total journals cited
Academic Press 15 0.82% 72 3.94%
Cambridge Univ 3 0.16% 13 0.71%
Elsevier 113 6.19% 269 14.72%
Kluwer 34 1.86% 55 3.01%
Project Muse 1 0.05% 6 0.33%
Springer 28 1.53% 71 3.89%
Wiley 35 1.92% 76 4.16%
TOTAL 229 12.53% 56230.76%
*1827 journals in total were cited in the sample

As seen in figure 1, there is a continuous increase in the number of journals cited that are part of Big Deals and were not previously subscribed to by the SFU Library, with the largest jump occurring between 2004 and 2005.

Conclusions

The citation of journals found in the seven analyzed Big Deals increased once the SFU Library signed onto the Big Deals, even though articles from these journals were always accessible to faculty through interlibrary loan. The biggest increase occurs not between 1998 (last year analyzed pre Big Deals) and 2004 (first year analyzed post Big Deals) but between 2004 and 2005. Possible reasons for this increase in citation of Big Deal journals in 2005 are research lag and manuscript preparation; publishing lag; and the SFU Library’s introduction in late 2003 of better link resolving software making it easier to find articles. A journal article appearing in early 2004 had its genesis at least 12-16 months earlier, which would be before, or just after, SFU signed onto the last of the Big Deals analyzed for this study. This was also a time when Big Deals were relatively new at SFU and therefore the uptake of the new e-journals was not fully realized and thus reduced the number of citations to Big Deal journals in 2004 articles as compared to 2005 articles. Also, the introduction of new link resolving software by the Library may have partially contributed to the increase in citations from 2004 to 2005, as the software had a more accurate knowledgebase of holdings and resolved directly to the article level. The old software depended on the holdings in the Library’s catalogue and only resolved to the catalogue record for the electronic journal, not to the article. With the new software it is much easier to find and read the online journal article, and this ease of accessibility to articles may account for a portion of the increased citation rate to Big Deal journals.

Interestingly, very few previously unavailable journals in Project Muse were cited. Possible reasons include the preponderance of STM journals in Web of Science even though Arts and Humanities Citation Index is included, the relative lack of journal archives in Project Muse, and the different publishing patterns of humanities faculty. Perhaps this finding should not be too surprising, since an analysis of OhioLINK e-journal usage found that only 10% of all e-journals accessed were from the arts and humanities.8

Although this analysis is only a snapshot of activity, the increase in citations in 2004 and 2005 to articles from Big Deal journals previously unavailable to the SFU Library demonstrates that SFU researchers are making use of these previously unavailable journals in their research. These findings validate the SFU Library’s rationale for signing onto Big Deals. However, further monitoring of citation rates to previously unavailable journals in the coming years will be needed to ensure that the trends seen in this study hold over the years. Each institution is unique, but it is reasonable to think that similar results would be obtained for other medium sized institutions that significantly expanded their access to the journal literature through Big Deals.

Notes

1.  Kenneth Frazier, “What’s the Big Deal,” Serials Librarian 48, no. 1/2 (2005): 49-59.

2.  Mark Ware, “E-only Journals: Is it Time to Drop Print?” Learned Publishing 18, no. 3 (2005): 193-199.

3.  David Kohl, “Doing Well by Doing Good,” Journal of Academic Librarianship 29, no. 4 (2003): 205-206.

4.  Andrew Waller and Gwen Bird, “ ‘We Own It’: Dealing with ‘Perpetual Access’ in Big Deals,” Serials Librarian 50, no. 1/2 (2006): 179-196.

5.  Loretta Ebert, “What’s the Big Deal? ‘Take 2’ or, How to Make it Work for You,” Serials Librarian  48, no. 1/2 (2005): 61-68.

6.  Kimberly Parker and Kathleen Bauer, “E-Journals and Citation Patterns: Is it all Worth it?” Serials Librarian 44, no. 3-4 (2003): 209-213. 

7.  Ibid., 210-211.

8.  David Nicholas, Paul Hungtington, Bill Russell, Anthony Watkinson, Hamid R. Jamali, Carol Tenopir, “The Big Deal Ten Years on,” Learned Publishing 18, no. 4 (2005): 251-257.

Back to Contents