Information Data

Mapping the Intellectual Impact of Library and

Information Science Research Through Citations: A

Tale of Two Databases - Scopus and Web of Science

Lokman I. Meho

School of Library and Information Science, Indiana University Bloomington

Bloomington, IN 47405 ****@*******.***

Cassidy R. Sugimoto

School of Information and Library Science, University of North Carolina at Chapel

Hill Chapel Hill, NC 27514 ******@*****.***.***

This study uses citations to a group of 42 LIS faculty members to examine the

differences between Scopus and WoS in mapping the faculties intellectual impact,

focusing on the sources of their citations by universities, journals, and countries.

The study also examines the effects of adding Scopus data on results obtained

from WoS. Findings show that while Scopus does not significantly influence the

rankings of the top citing journals, institutions, or countries, the database is

necessary to use in addition to WoS in order to accurately map the intellectual

impact of the faculty members examined and identify more correctly the extent of

their research influence on the wider scientific community.

Researchers using citations to identify, map, or visualize the intellectual impact of authors,

projects, programs, journals, knowledge domains, and countries often rely on data

exclusively from ISI databases (B rner, Chen, & Boyack, 2003; Goldstone & Leydesdorff,

2006; McKechnie, Goodall, Lajoie-Paquette, & Julien, 2005). This reliance on ISI databases

is partially due to the fact that the databases (currently offered through what is known as

Web of Science-WoS) were for decades the only comprehensive citation data sources that

allowed large-scale analyses of and between authors, journals, disciplines, and countries

(Meho, 2007; Moed, 2005). The emergence of Elsevier s Scopus in 2004, however, has

raised many questions among information professionals and scholars regarding the

differences between the two databases and the effects of using Scopus in addition to WoS

in mapping or visualizing scientific networks (B rner, Sanyal, & Vespignani, 2007). A major

reason encouraging these questions is that Scopus indexes over 15,000 peer-reviewed titles

(including 700 conference proceedings) in comparison to approximately 8,700 journals and

only a few dozen conference proceedings by WoS. For details on Scopus and WoS, see Meho

and Yang (in press). 1

This study uses two groups of faculty members from ALA-accredited library and information

science (LIS) Master s programs to examine the differences between Scopus and WoS in

mapping the faculties intellectual impact, focusing on the sources of their citations by

universities, journals, and countries. The first group, whose results are reported here, is

composed of 42 faculty members from Indiana University Bloomington and the University of

North Carolina at Chapel Hill. The second group is composed of 80 randomly selected

full-time LIS faculty representing 10% of the entire population of full-time LIS faculty

members in Canada and the United States.2 Given that WoS has been and continues to be

the standard data source for a major portion of citation-based research, the study also

examines the effects of using Scopus as an additional citation data source in such research.

The findings of the study will be important to anyone trying to determine whether mapping

or visualizing the scientific impact of LIS research should be limited to WoS or go beyond it

to include Scopus. The findings of the study will also be important to determine the effects

of using multiple citation sources on identifying the breadth of the intellectual impact of LIS

faculty.

METHODS

The 42 faculty included in this study cover most if not all of the mainstream LIS research

areas as identified by the Association of Library and Information Science Education (ALISE,

2006). Moreover, several of these faculty are among the most published and belong to two

of the most cited ALA-accredited LIS programs (Adkins & Budd, 2006; Budd, 2000; Persson

& str m, 2005).

All Scopus and WoS citation searching was conducted during the first week of January

2007. We used the exact match search approach in Scopus to identify citations to each of

the citable items published or produced by the study sample. This method uses the title of

an item (e.g., journal article and conference paper) as a search statement (e.g., Invoked on

the Web ) and tries to locate an exact match in the cited References field of the indexed

records. In cases where the title was too short or ambiguous to refer to the item in question,

we used additional information as keywords (e.g., the first author s last name) to ensure

that we retrieved only relevant citations. In cases where the title was too long, we used the

first few words of the title because utilizing all the words in a long title increases the

possibility of missing some relevant citations due to typing or indexing errors. We also

utilized the Author Search option in Scopus, although it did not identify any citations that

were not found by the exact match search approach. In WoS, we used the Cited Author

search option to identify relevant citations. Citations to items in which the faculty members

were second, third, or nth authors, as well as citations to dissertations and other research

materials written by the faculty members, were included in the study.3

To ensure that citations were not overlooked because of searching or indexing errors, we

cross-examined all of the citations that were missed by one of the databases. For example,

if a citation was found in Scopus but not in WoS, we conducted bibliographic searches in

WoS to see if the item were in fact indexed in the database. When the bibliographic record

of any of these missed citations was found in Scopus or WoS, we examined: (1) why it was

not retrieved through the citation search methods described above; and (2) whether it

should be counted as a citation. Items that were overlooked due to searching errors-88 (or

1.8%) of Scopus citations and 46 (or 1.1%) of WoS-were counted as citations toward their

respective databases; most of the searching errors were due to making typographical errors

when entering a search query in Scopus and having missed selecting a relevant entry when

browsing the cited references field in WoS. Items that were missed due to errors beyond the

control of the authors were tallied but were not counted as citations. Results showed that, in

total, Scopus missed 430 (or the equivalent of 8.8% of its relevant citations) due to errors

beyond the control of the investigators. One hundred sixty nine of these citations were

missed due to errors in the cited title information, 151 due to incomplete list of cited

references in the database, and 110 due to lack of cited references information. In the case

of WoS, 48 citations were missed due to errors in the cited reference information in the

database, 29 due to incomplete list of cited references in the database, and 9 due to lack of

cited references information. In total, WoS missed 86 (or the equivalent of 2.0% of its

relevant citations) due to errors beyond the control of the investigators.

All citation data from Scopus and WoS were entered into an Access database and were

coded by citing author, source, document type, publication year, institutional affiliation of

the corresponding author, country of the corresponding author, and language of the citing

document. More than 1,000 of the retrieved citations did not have institutional affiliation or

country information; this information, however, was identified by searching the Web and/or

locating other articles by the same authors that included institutional and country

information. Overall, data collection, cross-examination of missed citations by each

database, identifying country and institutional affiliation information of citations that did not

have this information available, and standardizing names of journals and conference

proceedings across the two databases consumed over 400 hours of work, for an average of

approximately 10 hours per faculty.

FINDINGS AND DISCUSSION

Because Scopus and WoS provide different coverage period for citations (Scopus from 1996

on in comparison to 1955 on for WoS), all analysis in this study is limited to citations from

1996 on-that is, the coverage period common to both databases. After removing duplicate

records, results of citation searching show that Scopus includes 610 (or 14.3%) more

citations than WoS for the study sample (4,871 citations in comparison to 4,261,

respectively). Results also show that the percentage of unique citations in Scopus is 26.0%

in comparison to 15.4% in WoS (see Figure 1).

Figure 1. Distribution of unique and overlapping citations in Scopus and WoS (N=5,757)

Despite the considerable number of unique citations found in both databases (2,382 or

41.4% of all the 5,757 citations), results show that there are no significant differences

between the two databases in terms of the top 15 citing sources, universities, or countries

(see Tables 1-3). In every case, the top 15 in each category remains the same across both

databases. Moreover, there is no significant difference between the two databases in

ranking the top citing sources, universities, or countries. Scopus also makes no significant

impact on changing the rankings of the top 15 citing sources, universities, or countries when

its data are combined with those of WoS, despite the fact that Scopus increases the citation

counts in certain cases by more than one-third.

A significant finding, however, is that Scopus uncovers much broader scientific impact of the

study sample than WoS does, as evidenced by the total number of citing sources,

universities, and countries. For example, WoS shows that faculty members of the two

schools covered in this study have been cited by 3,144 authors from 61 different countries

in comparison to 3,700 authors from 67 countries in Scopus. WoS also shows that the 42

faculty members have been cited in 980 different journals and conference proceedings

whereas Scopus shows that they have been cited in 1,261 different sources. The numbers

for the institutional affiliation of citing authors are 1,181 and 1,416 for WoS and Scopus,

respectively. Scopus shows that 63% of the citations come from non-LIS sources whereas in

WoS the amount is 53%. Most importantly, however, is that Scopus increases the citation

count of the study sample by 1,496 (or 35.1%), the number of countries by 7 (or 11.5%), the

number of unique citing titles by 495 (or 50.5%), and the number of citing institutions by

373 (or 31.6%) (see Figure 2).

Figure 2. Effects of adding Scopus data to those of WoS (Y axis refers to count)

While Scopus does not significantly influence the rankings of the top citing journals,

institutions, or countries, the database is necessary to use in addition to WoS in order to

accurately map the intellectual impact of the study sample and identify more correctly the

extent of their research influence on the wider scientific community. Without the use of

Scopus, an incomplete picture will be drawn of the research impact of the scholars

examined.

CONCLUSIONS

This study examined the differences between Scopus and WoS and the necessity and value

of using both databases in mapping or visualizing scientific networks. Using a group of 42

faculty members from two LIS schools, the study finds that to accurately map the

intellectual impact of the study sample, one has to employ both databases because they

complement each other. The study informs both researchers and practitioners that while

Scopus does not provide citation coverage for the period before 1996, it indexes a very

comprehensive set of post-1995 journals and conference proceedings that can significantly

influence the shape of any scientific or intellectual map or network one tries to draw

exclusively based on WoS for scholars, projects, programs, journals, research domains,

universities, or countries.

Although the results of this study cannot be generalized beyond the study group, the study

findings should alert researchers about the implications of limiting their citation data

sources to one database. Future studies should be based on larger, randomly selected

samples in order to generalize the findings to the larger LIS scientific community. Future

studies should also examine samples or populations from other disciplines in order to better

assess the effects, value, and necessity of using multiple citations sources in developing

global maps of knowledge and intellectual impact.

Table 1. Top 15 Citing Universities

WoS Scopus Both

% Increase

University Count Rank Count Rank Count Rank

University of Maryland at College Park 65 1 66 1 77 1 18.2

University of Illinois at Urbana-Champaign 56 3 56 2 72 2 28.6

University of Wolverhampton 52 4 53 3 59 3 13.2

Royal School of Information and Library Science 57 2 42 8 58 4 2.4

University of Toronto 48 5 48 4T 57 5 18.8

University of Arizona 43 7T 48 4T 53 6T 20.8

University of Western Ontario 44 6 48 4T 53 6T 18.8

University of Texas at Austin 43 7T 41 9 49 8 14.6

Rutgers State University 35 13 39 11T 48 9T 33.3

University of Sheffield 42 9T 43 7 48 9T 14.0

University of California at Los Angeles 38 12 39 11T 47 11 23.1

Loughborough University 42 9T 40 10 46 12 10.0

University of Washington **-**-**-**-**-** 36.8

University of Tampere **-**-**-**-**-** 5.9

Nanyang Technological University **-**-**-**-**-** 21.9

Table 2. Top 15 Citing Countries

WoS Scopus Both

% Increase

Country Count Rank Count Rank Count Rank

United States 2002 1-232*-*-**** 1 31.1

England 514-*-***-*-*** 2 28.3

Canada 253-*-***-*-*** 3 24.9

Germany 133-*-***-*-*** 4 30.8

Australia 112-*-***-*-*** 5 34.2

Scotland 99 6 100 6 122 6 23.0

Netherlands 80 7 96 7 111 7 32.3

Finland 76 8.5 94 8 107 8 33.0

Spain 71 10 72 11 92 9 29.2

Denmark 76 8.5 70 12 89 10 18.6

Sweden **-**-**-*-**-** 44.3

Italy **-**-**-**-**-** 28.1

France **-**-**-**-**-** 42.6

Taiwan **-**-**-**-**-** 27.4

China **-**-**-**-**-** 38.7

Table 3. Top 15 Citing Sources

WoS Scopus Both %

Count Rank Count Rank Count Rank Increase

Source Title

Journal of the American Society for Information

353-*-***-*-*** 1 5.9

Science and Technology

Information Processing & Management 146-*-***-*-*** 2 2.1

Proceedings of the ASIST Annual Meeting 97 5 76 8 115 3 23.7

Journal of Documentation 112-*-**-*-*** 4T 0.0

Scientometrics 109-*-**-*-*** 4T 3.2

Library & Information Science Research 86 6 83 5 89 6 3.6

Annual Review of Information Science &

85 7 81 6 86 7 1.2

Technology

Journal of Information Science 77 8 80 7 81 8 5.0

Journal of the Medical Library Association 67 10 60 11 72 9 8.3

International Journal of Human-Computer Studies 69 9 67 9 71 10 3.0

Library Trends **-**-**-**-**-** 4.8

Interacting with Computers **-**-**-**-**-** 1.8

Journal of Computer-Mediated Communication **-**-**-**-**-** 50.9

Information Research-An International Electronic

**-**-**-**-**-** 32.7

Journal

Journal of Academic Librarianship **-**-**-**-**-** 4.7

NOTES

1 Coverage in WoS goes back to 1955 for Science Citation Index, 1956 for Social Sciences Citation

Index, and 1975 for Arts & Humanities Citation Index (for a total of over 36.8 million records as of

February 2007), whereas coverage in Scopus goes back to 1996 for citations and to 1966 for

bibliographic records and abstracts for a total of approximately 30 million records (as of February

Back

2007).

2 At the time of the conference, results of both groups of faculty members will be presented. The first

Back

group will also include citation data for LIS faculty of the University of Illinois-Urbana Champaign.

3 It is important to note that in studies such as this, it is essential that the investigators have access

to complete lists of publications of the authors being examined. Without this information, there would

be major problems with the data collected, especially when there are authors with common names

among the study population. In our case, all 42 faculty members constituting the study population

either had their complete publication information available online or they provided it to us on request.

The lists were also cross examined with all the LIS databases recommended by Meho and Spurgin

(2005) for identifying LIS indexed literature. The complete lists of publications were particularly useful

in the cases of those faculty members with relatively common names, such as B. Cronin, S. Haas, N.

Hara, S. Herring, J. Mostafa, D. Shaw, P. Solomon, and K. Yang. The availability of their publication lists

Back

helped avoid including non-relevant citations.

References

http://www.cas.usf.edu/lis/lis6260/lectures/infosci.htm

Adkins, D., & Budd, J. (2006). Scholarly productivity of U.S. LIS faculty. Library &

Information Science Research, 28(3), 374-389.

Association for Library and Information Science Education. (2006). LIS research areas

classification scheme. Retrieved March 15, 2006

http://www.alise.org/research_class_guide.html

B rner, K., Chen, C. M., & Boyack, K. W. (2003). Visualizing knowledge domains. Annual

Review of Information Science and Technology, 37, 179-255.

B rner, K., Sanyal, S., & Vespignani, A. (2007). Network science. Annual Review of

Information Science and Technology, 41, 537-607.

Budd, J. M. (2000). Scholarly productivity of U.S. LIS faculty: An update. The Library

Quarterly, 70(2), 230-245.

Goldstone, R. L., & Leydesdorff, L. (2006). The import and export of Cognitive Science.

Cognitive Science 30(6), 983-993.

McKechnie, L. E. F., Goodall, G. R., Lajoie-Paquette, D., & Julien, H. (2005). How human

information behaviour researchers use each other's work: a basic citation analysis study.

Information Research-an International Electronic Journal 10(2), 12.

Meho, L. I. (2007). The rise and rise of citation analysis. Physics World 20(1), 32-36.

Meho, L. I., & Spurgin, K. M. (2005). Ranking the research productivity of LIS faculty and

schools: An evaluation of data sources and research methods. Journal of the American

Society for Information Science and Technology 5 6(12): 1314-1331.

Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht, The

Netherlands: Springer.

Meho, L. I., & Yang, K. (in press). Impact of data sources on citation counts and rankings

of LIS faculty: Web of Science vs. Scopus and Google Scholar. Journal of the American

Society for Information Science and Technology.

Persson, O., & str m, F. (2005). Most cited universities and authors in Library &

Information Science 1990-2004. Bibliometric Notes, 7(2). Retrieved November 15, 2006

http://www.umu.se/inforsk/BibliometricNotes/BN2-2005/BN2-2005.htm

Contact this candidate