This paper studies the potential of using the Google Scholar search engine for estimating the publication activities of universities and considers a procedure for such estimation with the help of queries for the English names of universitiesyesBelgorod State Universit

Moskovkin, V. M.

DSpace at Belgorod State University

 ISSN 0147-6882, Scientific and Technical Information Processing, 2009, Vol. 36, No. 4, pp. 198–202. © Allerton Press, Inc., 2009.Original Russian Text © V. M. Moskovkin, 2009, published in Nauchno-Technicheskaya Informatsiya, Seriya 1, 2009, No. 7, pp. 12–16. 198 A cluster of publications connected with the use ofthe Google Scholar search engine for carrying out suchinvestigations has appeared now in the foreign scien-tific literature devoted to the scientometric methods ofinvestigations. It is noted that the databases of theUS Institute for Scientific Information (ISI CitationIndexes) were a unique overall source of citation datauntil recently.These databases have long been widely used abroadin scientific management, coping with some of theirimperfections as well as the absence of other imperfec-tions.Two alternatives to these databases, i.e., the com-mercial Scopus search engine developed by the verylarge Elsevier Publishing House for scientific periodi-cals and open-access Google Scholar search enginehave appeared comparatively recently [1].Works [2, 3] show that Google Scholar covers amuch greater quantity of documents compared to thedatabases of the US Institute for Scientific Information,thus making a great contribution to the movementtowards open access to the results of scientific investi-gations.Work [4] notes that the Google Scholar searchengine provides a new method for discovering poten-tially relevant articles on these themes at the expense ofidentifying the articles cited in other works. Therefore,an important property of this search engine is thatresearchers can use it to track the mutual relationshipsbetween authors citing articles on a similar subject andalso to determine the frequency with which otherauthors cite a specific article (with the “cited by”option). It also makes the conclusion that the GoogleScholar search engine provides a free alternative andsupplements other citation indexes.The databases of the US Institute for ScientificInformation index approximately one third of the totalquantity of reviewed scientific journals, which numberabout 25 000 at present. This being the case, GoogleScholar and Google Books index many more scientificdocuments, but still are not able to achieve completecoverage, since only 15% of the current annual scien-tific output is represented by OA publications [5].Our review of scientometric investigations hasshown that there are no works studying the publicationstructure and webometric estimates of university scien-tific outputs with the help of the Google Scholar searchengine.When studying this output, we paid attention to theimpossibility of qualitatively obtaining it by measuringthe responses to the URL addresses of university sites.The large quantity of irrelevant responses in the form ofdifferent administrative information (decisions of a sci-entific council, university administration, etc.) oftenappear for post-soviet universities. The situation arisesfor western universities when, for example, in the caseof approximately equal publication activities of scien-tists from American universities (the Harvard and Chi-cago Universities) and British universities (Cambridgeand Oxford Universities), the American scientists hadmore responses to queries for their URL-addresses 1 , byan order of magnitude, although queries for the names 1 Harvard.edu, site: uchicago.edu, site: ox.ac.uk, site: cam.ac.ukgive 1310000, 60 400, 8090, and 9330 documents, respectively;the measurements were carried out by us at the beginning ofJanuary 2009. The Potential of Using the Google Scholar Search Engine for Estimating the Publication Activities of Universities V. M. Moskovkin Received March 20, 2009 Abstract —This paper studies the potential of using the Google Scholar search engine for estimating the pub-lication activities of universities and considers a procedure for such estimation with the help of queries for theEnglish names of universities. The publication structures for 2008 have been built for ten selected universitiesof the world, including MSU. The publication activities of the universities under consideration in 2007, hasbeen compared based on the citation database of the US Institute for Scientific Information (Web of Knowle-dge) and Google Scholar search engine (GS-publications). Key   words :  Google Scholar search engine, university publication activity, open access, citation indexes, pub-lication structures. DOI:  10.3103/S0147688209040029 SCIENTIFIC AND TECHNICAL INFORMATION PROCESSING        Vol. 36        No. 4        2009 THE POTENTIAL OF USING THE GOOGLE SCHOLAR SEARCH ENGINE 199 of these universities gave advantages to Britain univer-sities 2 .In our opinion, this is caused by the quality of theorganization of information on a site. Thus, the multi-plicity of responses to the Harvard University site isconnected with the existence of a scientific internetmagazine (sciencemag.org), and when a query is madeto the Chicago University site, the first thousandresponses that are shown by the Google Scholar searchengine are for the articles of the excellently composedjournal collection of the university (“Chicago Jour-nals”) that is found on the uchicago.press platform.It is obvious that the sites of universities present byno means all publications of their scientists, and, as forpost-soviet universities, the practice of placing scien-tific articles on their sites is completely nonexistent.In connection with the above-stated facts, wedecided to test the generally recognized English namesof universities with the help of the Google Enginesearch engine, instead of testing their URL-sites withits use, as one Spanish cybermetric laboratory doeswhen calculating the webometric ranking of theworld’s universities (www.webometrics.info). Theexperiments with the leading universities of the worldshowed the high relevance of this search. In the firstplace, Google Scholar finds articles placed on the on-line platforms of the largest publishing houses, such asElsevier, Springer, Blackwell, Wiley, etc., i.e., “con-vertible” articles included in the databases of the USInstitute for Scientific Information. In addition, thissearch engine efficiently finds articles from online jour-nals and open-access university repositories.Let us also note that Google Scholar also includes asmall percentage of scientific monographs provided byGoogle Books in the results of its search.We have already been able to show that the rele-vance of an advanced search with an exact phraseincreases in the following order: in the absence ofrestrictions on the fields of science and time intervals –> with the assignment of fields of sciences –> with thesimultaneous assignment of fields of science and timeintervals of search.Besides the total quantity of articles in a givenknowledge field (7 fields) that are obtained in responseto a query for the English name of a specific university,Google Scholar gives the values of the total number ofcitations to each article with the opportunity of brows-ing through the names of scientific works citing thisarticle (with the help of the “by cited” option). Our con-tacts with the Google Scholar team showed that a pro-cedure permitting one to summarize the citations in theentire assembly of found articles was still nonexistent,but the Google Scholar team received the idea of devel-oping this procedure with interest. If it is realized, thiswill provide the opportunity to calculate the full-valuewebometric ranking of the scientific-publication activ- 2 Advanced search with an exact phrase. ities of the world’s universities. When this ranking iscalculated, the problem of identifying all the generallyaccepted names of universities arises. For example,generally accepted French and English names must beused for the universities of the French-speaking Cana-dian provinces; all the main foreign-language names ofthe European universities of the non-English-speakingcountries must be used for them, as well as theirEnglish names. The permanent process of renaminguniversities should be taken into consideration for post-soviet countries. When working with the GoogleScholar search engine, we noted the fluctuation ofresponses to queries for the names of universities,which is connected with the possible temporaryabsence of access, exclusion of duplications and irrele-vant responses. Therefore, in our opinion, it is expedi-ent to use smoothing procedures (to calculate the aver-age temporal trend) when calculating a resultant webo-metric index.We believe that with time, as the creation of open-access university repositories becomes more intensive,the probability of duplicated responses to queries forthe URL-addresses of university sites will increase,since previously unpublished articles will be placed(self-archived) in such repositories (mainly, in the formof an author’s PDF-files).It is difficult to say in advance how effectively theGoogle Scholar search engine will cope with theincreasing scale of article duplication.Nine foreign universities that occupied the highestpositions according to the quantity of published articles(included in the SSI and SSCI databases of the USInstitute for Scientific Information) in the 2008 Chineseand Taiwanese rankings of universities were chosen byus as experiments with the Google Scholar searchengine. The leading Russian post-soviet university, theMSU, was chosen for comparison. Tables 1 and 2 showthe publication structures for 2008 for these universitiesin the quantitative and percentage terms, as well as theenlarged publication structure. The caps of these tablesgive the main names of the universities, according towhich an advanced search with exact phrase was per-formed. The inverted names of these universities (forexample, the University of Chicago—the Chicago Uni-versity) were also taken into account in the queries ofthe Google Scholar, with the exception of the HopkinsUniversity, University of California, University ofTokyo, and Moscow State University. The greatestshares of responses for the inverted names wereobserved for the Chicago, Cambridge, and Oxford uni-versities. The responses to the inverted name of theUniversity of Tokyo often brought to other universities(the Tokyo University of Agriculture, Technology, orScience); therefore, they were not taken into account insummary estimates. The data of table 2 were calculatedbased on the percentage distribution of the data given intable 1. For example, the share of publications in thefield of life sciences for Harvard University amounted 200 SCIENTIFIC AND TECHNICAL INFORMATION PROCESSING        Vol. 36        No. 4        2009 MOSKOVKIN Table 1.   The publication structure for the selected largest universities in the world in 2008, obtained using the Google Scholar search engine on January 22, 2009Fieldsof sciencesStanfordUniversityHarvardUniversityColumbiaUniversityUniversityof California-BerkeleyJohnsHopkinsUniversityChicagoUniversityCambridgeUniversityOxfordUniversityUniversityof TokyoMoskowStateUniversity1. Biology,Life Science,and Environmental Science2346/8.83102/9.72977/12.21670/14.22240/10.14453/8.617920/13.115100/12.32460/19.3560/14.82. Business,Administration, Finance, andEconomics2686/10.15802/18.12210/9.01090/9.31470/6.76850/13.311759/8.610520/8.6376/2.957/1.53. Chemistry and Materials Science1118/4.2822/2.5935/3.8980/8.3584/2.6736/1.47340/5.38167/6.71720/13.5957/25.24. Enginering,Computer Science, and Mathematics4455/16.72161/6.71680/6.92370/20.12200/9.91914/3.720230/14.79703/7.91910/15.0583/15.45. Medicine,Pharmacology, and Veterinary Science5236/19.71832/5.74391/18.0698/5.95180/23.54870/9.49670/7.119970/16.32320/18.271/1.96. Physics,Astronomy, and Planetary Science2112/7.91591/5.01523/6.21950/16.61630/7.44017/7.819190/14.07290/6.03260/25.61380/36.47. Social Sciences, Arts, andHumanities8682/32.616813/52.310729/43.93010/25.68780/39.828830/55.850980/37.251570/42.2703/5.5180/4.8Total26635/10032123/10024445/10011768/10022084/10051670/100137089/100122320/10012749/1003788/100 Note:The numerator gives the number of publications, and the denominator gives %. SCIENTIFIC AND TECHNICAL INFORMATION PROCESSING        Vol. 36        No. 4        2009 THE POTENTIAL OF USING THE GOOGLE SCHOLAR SEARCH ENGINE 201 to 9.7 + 5.7 = 15.4%. Table 2 shows that the scientificschools of the socioeconomic and humanitarian fieldsare predominant at Harvard and the Chicago Univer-sity. The opposite picture is observed for the Universi-ties of California, Tokyo, and Moscow. Scientificschools in the field of life sciences are most heavily rep-resented at the University of Tokyo and Johns HopkinsUniversity. The post-soviet publication structure repre-sented by the scientific output of the MSU is character-ized by the clear predominance of “convertible” natu-ral-scientific and technical publications and, conse-quently, scientific schools of the natural scientific andtechnical directions (with the exception of medical andbiological scientific schools).Let us compare now the publication activities of theuniversities under consideration that was obtainedbased on the citation databases of the US Institute forScientific Information (ISI) and Google Scholar searchengine (GS). For this purpose, we addressed ourselvesto the Taiwanese Ranking of Scientific Papers forWorld Universities. This ranking contains the “CurrentArticles” index, which represents the annual quantity ofpublications obtained based on the SCI and SSCI data-bases (Thomson–Reuter). This index in the 2008 rank-ing of world universities was calculated for 2007. Itsmaximal value, taken as 100% was for Harvard Univer-sity and was equal to 11 221 articles  3 . Its absolute val-ues were recalculated by us for the remaining universi-ties based on the maximal value of this index. Thequantities of scientific articles that were obtained by thepreviously described method with the help of the Goo-gle Scholar search engine were also calculated for thesame year (GS-publications in table 3). The surplus ofthe webometric index of the university publication 3 The absolute value of the “Current articles” index was kindly fur-nished to us by Ru-rong Hsiao (the Chief of the PerformanceEvaluation Section HEEACT of Taiwan. activities over its traditional index was calculated aswell in table 3. As is clear, this ratio varies ratherstrongly. Meanwhile, it is logical to suppose that theratio of the total number of publications to “convert-ible” publications (Thomson–Reuter) is an approxi-mately constant value for different universities, i.e.,there must be a good linear correlation between theseindices. The absence of such a correlation between theindices of Thomson–Reuter and GS-publicationsspeaks only for the bad Web-presentation of publica-tions for universities for which the ratio “GS-publica-tions/Thomson–Reuter” is low.The index of Thomson–Reuter must be included inarticles from the A&HCI database for more correct cal-culations, as the Google Scholar search engine coverssuch articles.It is to be noted that the PUB index completely cor-responds to the Current articles index (the TaiwaneseRanking) in the Shanghai Ranking of World Universi-ties, but it cannot be directly used to calculate the abso-lute values of university publications included in theSCI and SSCI databases, since a coefficient of 2 wasused for socioeconomic articles.Consequently, we have shown the possibility ofquantitatively estimating the publication activities ofuniversities with the help of the Google Scholar searchengine, confirmed the results of foreign researchers onthe wider coverage of scientific publications by thissearch engine in comparison with the databases of theUS Institute for Scientific information, and built thepublication structures for ten selected examples of theleading world universities. The further development ofthis approach must follow the path of separating bookpublications (the mark “Book”) and citations (the mark“Citation”) in the responses of the Google Scholarsearch engine, despite the large percentage of theseresponses. However, this work, together with the calcu- Table 2.   The enlarged publication structure for the selected largest universities in the world in 2008, obtained using the GoogleScholar search engine on January 22, 2009Enlargedfields of sciencesStanfordUniversityHarvardUniversityColumbiaUniversityUniversityof Califor-nia-Berke-leyJohnsHopkinsUniversityChicagoUniversityCam-bridgeUniversityOxfordUniversityUniversityof TokyoMoskowStateUniversityNatural and techical sciences28.8 14.2 16.9 45.0 19.9 12.9 34.0 20.6 54.1 77.0Lifesciences28.5 15.4 30.2 20.1 33.6 18.0 20.2 28.6 37.5 16.7Socio-eco-nomic sci-ences and humani-ties. Art.42.7 70.4 52.9 34.9 46.5 69.1 45.8 50.8 8.4 6.3Total 100 100 100 100 100 100 100 100 100 100 202 SCIENTIFIC AND TECHNICAL INFORMATION PROCESSING        Vol. 36        No. 4        2009 MOSKOVKIN lation of the total number of citations in all the foundacademic documents (the “by cited” option) can bedone only in cooperation with the Google Scholarteam.REFERENCES 1. Bar-Ilan, J., Which h-index? – A Comparison of WoS,Scopus and Google Scholar,  /Scientometrics , 2008,Vol. 74, no 2, pp. 257–271.2. Kousha, K., Thelwall, M., Google Scholar Citations andGoogle Web/URL Citations: A Multi-Discipline Explor-atory Analysis,  Journal of the American Society forInformation Science and Technology , 2007,Vol. 58, no 7,pp. 1055–1065.3. Kousha, K., Thelwall, M., Sources of Google ScholarCitation outside the Science Citation Index: A compari-son between Four Science Disciplines,  Scientometrics ,2008, Vol. 74, no 2, pp. 273–294.4. Noruzi, A., Google Scholar: The New Generation ofCitation Indexes,  Libri , 2005, Vol.55, no 4, pp. 170–180.5. Brody, T., Carr, L., Gingras, Y., Hajjem, Ch., Harnad, S.,Alma Swan. Incentivizing the Open Access ResearchWeb Publication – Archiving, Data-Archiving and Sci-entometrics,  CTWatch Quarterly , 2007 (August). Table 3.   The publication activity of the largest universities in the world obtained based on the data of the Taiwanese Rankingof World Universities and Google Scholar search engine, 2007UniversitiesThe   quantity   of   articlesGS-publications/Thomson-ReuterThomson-Reuter GS-publications% the   absolute   valueHarvard University 100 11221 46768 4.2University of Tokyo 62.51 7014 15495 2.2Johns Hopkins University 52.98 5878 27124 4.6University of California-Berkeley 47.67 5349 14571 2.7Stanford University 47.87 5372 35320 6.6Columbia University 43.10 4836 32990 6.8Oxford Universit 39.60 4444 142344 32.0Cambridge University 39.35 4416 173060 39.2Chicago University 34.78 3903 73016 18.7Moskow State University 28.05 3148 5021 1.6

The potential of using the Google Scholar search engine for estimating the publication activities of universities

http://dspace.bsu.edu.ru/bitstream/123456789/4407/1/Moskovkin%20V_M_The%20potential.pdf

The potential of using the Google Scholar search engine for estimating the publication activities of universities

Abstract

Similar works

Full text

Available Versions

DSpace at Belgorod State University