Citation data of scientific publications are essential for different purposes, such as evaluating research and building digital library collections. In this paper, we analyze to which extent citation data of publications are openly available, using the intersection of the Crossref metadata and unpaywall snapshot as publication dataset and the COCI dataset as open citation data. We reveal that for 24.2% of the publications, the citation data is openly available, while for 16.6%, the citation data is closed. We find that the percentage of publications with open citation data has increased over the years. We observe that whether publications are published with open access has no influence on whether their citations are openly available. However, publications published in journals from the Directory of Open Access Journals (DOAJ) tend to have more citation data openly available than publications from other journals

Färber, M.

Nishioka, C.

Information Technology and Nanotechnology

KITopen

Evaluating the Availability of Open CitationDataChifumi Nishioka1[0000−0002−1853−3038] andMichael Fa¨rber2[0000−0001−5458−8645]1 Kyoto University Library, Kyoto 606-8501, Japannishioka.chifumi.2c@kyoto-u.ac.jp2 Institute AIFB, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germanymichael.faerber@kit.eduAbstract. Citation data of scientific publications are essential for dif-ferent purposes, such as evaluating research and building digital librarycollections. In this paper, we analyze to which extent citation data ofpublications are openly available, using the intersection of the Cross-ref metadata and unpaywall snapshot as publication dataset and theCOCI dataset as open citation data. We reveal that for 24.2% of thepublications, the citation data is openly available, while for 16.6%, thecitation data is closed. We find that the percentage of publications withopen citation data has increased over the years. We observe that whetherpublications are published with open access has no influence on whethertheir citations are openly available. However, publications published injournals from the Directory of Open Access Journals (DOAJ) tend tohave more citation data openly available than publications from otherjournals.Keywords: Open citation · Open access · Open data.1 IntroductionCitation data3 of scholarly publications has been used for various purposes, suchas evaluating research impact [12] and building digital library collections [11].A variety of citation databases have been used so far. However, the providersoften charge high subscription fees and the datasets are mainly oriented towardshuman readability instead of machine readability for allowing data reuse [9]. Toresolve these challenges, several organizations have started to make citation dataopenly available in a machine readable way. One of the most notable projectsfor open citation data is the Initiative for Open Citations4 (I4OC), which wasestablished in 2017 to promote the unrestricted availability of citation data [9].The I4OC has made citation data open by encouraging scholarly publishers topublish reference lists of publications which they already deposit to Crossref. It3 In this paper, a citation refers to the directional link from a citing bibliographicentity to a cited bibliographic entity.4 https://i4oc.org/, last accessed on 04/24/020182 C. Nishioka and M. Fa¨rberhas achieved initial success, with many major scholarly publishers opening theirreference lists in this way.Note that the meaning of “open” in the context of citation data is differentfrom the one in the context of publications (e. g., open access). In the contextof publications, “open” indicates that the publications are freely accessible andreusable without restrictions. For citation data, “open” in addition means thatthe data is structured (i .e., expressed in a machine-readable format) and separate(i .e., available without the need to access a source publication) [8].Several recent studies (e.g., [4,3]) compare scholarly datasets containing alsocitation data. However, they do not distinguish between open citation data andclosed citation data (i.e., citation data that are intentionally closed by pub-lishers). Heibi et al. [6] present statistics of open citations, focusing on typesand publishers. They investigate the state of open citations based on the COCI(OpenCitations Index of Crossref open DOI-to-DOI references) dataset [5] (seealso Section 2), focusing on each citation. In contrast, this paper looks into theextent to which publications make their citation data open. Specifically, we focuson outgoing citations of publications.In this paper, we investigate the following questions: (1) What percentage ofpublications have made citation data open, and how does the percentage varyaccording to the publication types and publication year? (2) Do open access(OA) publications make citation data more open compared to their toll-accesscounterparts?To answer those questions, we conduct an analysis on 100 million publica-tions that are contained in both the Crossref metadata dataset and the unpay-wall snapshot. As open citation data, we use the COCI dataset [5]. The citationdata of the COCI dataset are originally from publishers. Thus, they are of highquality. We label each publication as “open citation publication,” “closed ci-tation publication,” or “others” (i.e., without any reference or with referencesnot registered in Crossref) and calculate statistical key figures of various kindsconcerning open citation data. The paper contributes to understand the cur-rent state of open citation data and their challenges (e.g., for which kinds ofpublications citation data are not available).The paper is organized as follows: Section 2 describes the datasets used forour analysis. We report the result of our analysis in Section 3 before concludingthe paper in Section 4.2 DatasetFig. 1 summarizes how the publication dataset to be analyzed is generated andhow the state of open citation of each publication is identified.As metadata of publications, we use the crossref metadata dataset as ofSeptember 5, 2018.5 The dataset has been built by querying the Crossref API65 https://archive.org/download/crossref_doi_dump_201809, last accessed on04/23/20196 https://github.com/greenelab/Crossref, last accessed on 04/23/2019Evaluating the Availability of Open Citation Data 3Fig. 1. Pipeline of the dataset generation.and contains metadata of 99,874,789 publications that are assigned CrossrefDOIs. To analyze the influence of OA on the availability of citation data, we usea snapshot of unpaywall7 from September 24, 2018, together with the Crossrefmetadata dataset. Unpaywall is a browser extension, which finds legal OA ver-sions of scholarly publications. The snapshot of the database used by the browserextension is publicly available. It covers 99,940,229 publications with CrossrefDOIs and contains information regarding the state of OA for each publication. Inthis paper, we use 99,848,571 publications that are included in both the Crossrefmetadata dataset and unpaywall snapshot as basis of our analysis.We mark each publication as (1) “open citation publication”, (2) “closed ci-tation publication”, or (3) “others”. We use the OpenCitations COCI dataset [5]generated on November 12, 2018 [7] as open citation data. Note that the COCIdatasets are generated based on the citation data that are deposited to Crossrefby publishers. Thus, they are of high quality compared to other datasets.(1) If a publication is included in the COCI dataset as a citing publication, wemark it as open citation publication (i .e., publication that makes citationdata open). We identify 24,178,446 of 99,848,571 publications (24.22%) asopen citation publication.(2) Crossref also allows publishers to keep their citation data closed. Thus, thereare publications whose citation data are included in Crossref metadata butnot publicly available. Publications which are not included in the COCIdataset as citing publications, but have a Crossref’s field references-countgreater than zero, are marked as “closed citation publications”. 16,589,545publications (16.62%) are judged as closed citation publications.(3) We mark publications that are neither open citation publication nor closedcitation publication as “others.” 59,080,580 publications (59.15%) are classi-fied as such. ”others” include both publications without any reference, suchas editorial notes, and publications whose reference data are not registeredat Crossref. Although they have different meanings, both the COCI dataset7 https://unpaywall.org/products/snapshot, last accessed on 04/23/20194 C. Nishioka and M. Fa¨rberTable 1. State of open citation depending on publication types.all open cit. pub. (%) closed cit. pub. (%) others (%)journal-article 73,397,619 20,120,816 (27.41) 15,258,852 (20.79) 38,017,951 (51.80)book-chapter 11,726,076 1,048,255 (8.94) 898,513 (7.66) 9,779,308 (83.40)proceedings-article 5,398,085 2,798,106 (51.84) 342,295 (6.34) 2,257,684 (41.82)component 3,380,129 9 (0.00) 2 (0.00) 3,380,118 (100.00)dataset 1,703,078 67,250 (3.95) 2,996 (0.18) 1,632,832 (95.88)and Crossref metadata dataset do not have a way to express no-value infor-mation [1], which indicates that a publication has no reference.3 ResultIn this section, we first report the state of open citation in each type of pub-lication. Then, we present the result in each publication year. Thereafter, weexamine whether open access publications promote open citations.3.1 TypeCrossref DOIs have been assigned to various types of objects, such as journalarticles and book chapters. Table 1 shows the state of open citation for thefive object types appearing most frequently (given by the number of CrossrefDOIs). We observe that the percentage of open citation publications whose typeis “journal article” is 27.41%. This is almost the same value as the percentagefor all types (24.22% as described in Section 2). We observe a high percentageof open citation publications and a low percentage of closed citation publica-tions in proceedings articles (51.84% and 6.32%, respectively). Regarding othertypes, it is reasonable that publications whose type is “component” (e .g., figures,supplemental materials) have no reference.3.2 Publication YearFig. 2 shows the acceptance of open citations per year. The bars in Fig. 2 rep-resent the number of total publications, open citation publications, and closedcitation publications per publication year. The polygonal lines provide the per-centage of open citation publications and closed citation publications per pub-lication year. Please note that the number of publications in 2018 is low be-cause we use the dataset captured in September, 2018. We see that the totalnumber of publications as well as the number of open citation publications hasincreased over decades. In addition, the percentage of open citation publicationshas gradually increased. While the percentage has been around 18% before 1998,it reached 36% in 2011. Although older publications are read less [10], makingthe citation data of older publications available is important to trace the evolu-tion of scholarly knowledge. Regarding closed citation publications, we see thatthe percentage decreases slightly over the years.Evaluating the Availability of Open Citation Data 5Fig. 2. State of open citation along with publication year.Table 2. State of open citation depending on whether a publication is open access(OA).all open cit. pub. (%) closed cit. pub. (%) others (%)OA 24,961,752 7,077,997 (28.36) 2,391,627 (9.58) 15,492,128 (62.06)non-OA 74,886,819 17,100,449 (22.84) 14,197,918 (18.96) 43,588,452 (58.21)3.3 Open AccessThis section examines the influence of open access on the state of open citation.Open Access (OA) We first investigate whether there is a difference in thestate of open citation between open access (OA) publications and non-openaccess (non-OA) publications. Table 2 shows the number of total publications,open citation publications, and closed publications as well as their percentages.Referring to the state of OA, Table 2 indicates that 25.00% of the publicationsare OA in some way (e.g., gold OA, green OA), which is consistent with theresult given by Piwowar et al. [10]. OA publications are more likely to have theircitation information open than non-OA publications. However, the difference issmall. One possible reason is that if a publication is hybrid OA or green OA,publishers may not make citation data open.Open Access (OA) Journal We examine whether there is a difference inthe state of open citation between publications in an OA journal and those innon-OA journals. Table 3 presents the number of all publications, open citationpublications, and closed publications as well as their percentages. Unlike theresult in Table 2, publications in OA journals make citation data more open.Compared to publications in non-OA journals, publications in OA journals arecategorized as “others” more frequently. The reason might be that many smallpublishers publish OA journals but do not have enough resources to organizecitation data.6 C. Nishioka and M. Fa¨rberTable 3. State of open citation depending on whether a journal of a publication is anopen access (OA) journal.all open cit. pub. (%) closed cit. pub. (%) others (%)OA journal 5,670,103 1,613,260 (28.45) 280,547 (4.95) 3,776,296 (66.60)non-OA journal 94,178,468 22,565,186 (23.96) 16,308,998 (17.32) 55,304,284 (58.72)Table 4. State of open citation depending on whether a journal of a publication isincluded in DOAJ.all open cit. pub. (%) closed cit. pub. (%) others (%)in DOAJ 3,227,017 1,580,857 (48.99) 273,392 (8.47) 1,372,768 (42.54)not in DOAJ 96,621,554 22,597,589 (23.39) 16,316,153 (16.89) 57,707,812 (59.72)DOAJ DOAJ (Directory of Open Access Journals) is an online platform thathosts a curated list of OA journals. The project defines OA journals as scholarlyjournals which make all their content available for free. These journals meethigh quality standards, notably by exercising peer review or editorial qualitycontrol8. As of July 3, 2019, over 13,000 journals have been registered at DOAJ.We verify whether there is a difference in the state of open citation betweenpublications from a journal in DOAJ and those from a journal not in DOAJ.The results are shown in Table 4. The percentage of open citation publicationsfor publications in DOAJ is significantly higher than that for publications notin DOAJ. Since DOAJ lists journals that meet quality standards, citation dataof publications are properly organized by publishers and deposited to Crossref.However, a certain amount of citation data (8.47%) is closed.4 ConclusionIn this paper, we analyzed the current state of open citation based on the COCIdataset. We found out that for 24.22% of the publications with assigned CrossrefDOIs, the citations were open. As described by Di Iorio [2], such a value is notsufficient for evaluating researchers. 16.62% of the publications are identifiedas closed citation publication and 59.15% are “others”. “Others” include bothpublications without any reference and publications whose reference data arenot registered to Crossref. Although they have different meanings, the existingdatasets do not have a way to express no-value information. We found that thepercentage of open citation publications has gradually increased over the years.We did not observe a difference regarding the availability of open citation databetween OA publications and the non-OA publications. However, publicationspublished in a journal listed in the Directory of Open Access Journals (DOAJ)were more likely to open the citation data than those published in other journals.8 https://doaj.org/publishers, last accessed on 05/14/2019Evaluating the Availability of Open Citation Data 7References1. Darari, F., Prasojo, R.E., Nutt, W.: Expressing no-value information in RDF. In:Proceedings of the International Semantic Web Conference (ISWC) Posters andDemonstrations Track. CEUR Workshop Proceedings (2015)2. Di Iorio, A., Peroni, S., Poggi, F.: Open data to evaluate academic re-searchers: an experiment with the Italian Scientific Habilitation. arXiv preprintarXiv:1902.03287 (2019)3. Fa¨rber, M.: The Microsoft Academic Knowledge Graph: A Linked Data Sourcewith 8 Billion Triples of Scholarly Data. In: Proceedings of the 18th InternationalSemantic Web Conference. ISWC’19 (2019)4. Haunschild, R., Hug, S.E., Bra¨ndle, M.P., Bornmann, L.: The number of linkedreferences of publications in Microsoft Academic in comparison with the Web ofScience. Scientometrics 114(1), 367–370 (2018)5. Heibi, I., Peroni, S., Shotton, D.: Coci, the opencitations index of crossref opendoi-to-doi citations. arXiv preprint arXiv:1904.06052 (2019)6. Heibi, I., Peroni, S., Shotton, D.: Crowdsourcing open citations with croci-ananalysis of the current status of open citations, and a proposal. arXiv preprintarXiv:1902.02534 (2019)7. OpenCitations: COCI CSV dataset of all the citation data (Version 3) (2018),https://doi.org/10.6084/m9.figshare.6741422.v38. Peroni, S., Shotton, D.: Open Citation: Definition (2018).https://doi.org/10.6084/m9.figshare.6683855.v19. Peroni, S., Shotton, D., Vitali, F.: One year of the OpenCitations Corpus. In:International Semantic Web Conference. pp. 184–192. Springer (2017)10. Piwowar, H., Priem, J., Larivie`re, V., Alperin, J.P., Matthias, L., Norlander, B.,Farley, A., West, J., Haustein, S.: The state of OA: a large-scale analysis of theprevalence and impact of Open Access articles. PeerJ 6, e4375 (2018)11. Smith, L.C.: Citation analysis. Library Trends 30(1), 83–106 (1981)12. Todeschini, R., Baccini, A.: Handbook of bibliometric indicators: quantitative toolsfor studying and evaluating research. John Wiley & Sons (2016)

Evaluating the availability of open citation data

4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019), co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019). Paris, France, July 25, 2019.Citation data of scientific publications are essential for different purposes, such as evaluating research and building digital library collections. In this paper, we analyze to which extent citation data of publications are openly available, using the intersection of the Crossref metadata and unpaywall snapshot as publication dataset and the COCI dataset as open citation data. We reveal that for 24.2% of the publications, the citation data is openly available, while for 16.6%, the citation data is closed. We find that the percentage of publications with open citation data has increased over the years. We observe that whether publications are published with open access has no influence on whether their citations are openly available. However, publications published in journals from the Directory of Open Access Journals (DOAJ) tend to have more citation data openly available than publications from other journals

Nishioka, Chifumi

Färber, Michael

Kyoto University Research Information Repository

Title Evaluating the Availability of Open Citation DataAuthor(s)Nishioka, Chifumi; Färber, MichaelCitationBIRNDL 2019 : Bibliometric-enhanced Information Retrievaland Natural Language Processing for Digital Libraries (2019),2414: 123-129Issue Date2019URL http://hdl.handle.net/2433/244871RightCopyright © 2019 for the individual papers by the papers'authors. Copying permitted for private and academic purposes.This volume is published and copyrighted by its editors.Type Journal ArticleTextversionpublisherKyoto UniversityEvaluating the Availability of Open CitationDataChifumi Nishioka1[0000−0002−1853−3038] andMichael Fa¨rber2[0000−0001−5458−8645]1 Kyoto University Library, Kyoto 606-8501, Japannishioka.chifumi.2c@kyoto-u.ac.jp2 Institute AIFB, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germanymichael.faerber@kit.eduAbstract. Citation data of scientific publications are essential for dif-ferent purposes, such as evaluating research and building digital librarycollections. In this paper, we analyze to which extent citation data ofpublications are openly available, using the intersection of the Cross-ref metadata and unpaywall snapshot as publication dataset and theCOCI dataset as open citation data. We reveal that for 24.2% of thepublications, the citation data is openly available, while for 16.6%, thecitation data is closed. We find that the percentage of publications withopen citation data has increased over the years. We observe that whetherpublications are published with open access has no influence on whethertheir citations are openly available. However, publications published injournals from the Directory of Open Access Journals (DOAJ) tend tohave more citation data openly available than publications from otherjournals.Keywords: Open citation · Open access · Open data.1 IntroductionCitation data3 of scholarly publications has been used for various purposes, suchas evaluating research impact [12] and building digital library collections [11].A variety of citation databases have been used so far. However, the providersoften charge high subscription fees and the datasets are mainly oriented towardshuman readability instead of machine readability for allowing data reuse [9]. Toresolve these challenges, several organizations have started to make citation dataopenly available in a machine readable way. One of the most notable projectsfor open citation data is the Initiative for Open Citations4 (I4OC), which wasestablished in 2017 to promote the unrestricted availability of citation data [9].The I4OC has made citation data open by encouraging scholarly publishers topublish reference lists of publications which they already deposit to Crossref. It3 In this paper, a citation refers to the directional link from a citing bibliographicentity to a cited bibliographic entity.4 https://i4oc.org/, last accessed on 04/24/020182 C. Nishioka and M. Fa¨rberhas achieved initial success, with many major scholarly publishers opening theirreference lists in this way.Note that the meaning of “open” in the context of citation data is differentfrom the one in the context of publications (e. g., open access). In the contextof publications, “open” indicates that the publications are freely accessible andreusable without restrictions. For citation data, “open” in addition means thatthe data is structured (i .e., expressed in a machine-readable format) and separate(i .e., available without the need to access a source publication) [8].Several recent studies (e.g., [4,3]) compare scholarly datasets containing alsocitation data. However, they do not distinguish between open citation data andclosed citation data (i.e., citation data that are intentionally closed by pub-lishers). Heibi et al. [6] present statistics of open citations, focusing on typesand publishers. They investigate the state of open citations based on the COCI(OpenCitations Index of Crossref open DOI-to-DOI references) dataset [5] (seealso Section 2), focusing on each citation. In contrast, this paper looks into theextent to which publications make their citation data open. Specifically, we focuson outgoing citations of publications.In this paper, we investigate the following questions: (1) What percentage ofpublications have made citation data open, and how does the percentage varyaccording to the publication types and publication year? (2) Do open access(OA) publications make citation data more open compared to their toll-accesscounterparts?To answer those questions, we conduct an analysis on 100 million publica-tions that are contained in both the Crossref metadata dataset and the unpay-wall snapshot. As open citation data, we use the COCI dataset [5]. The citationdata of the COCI dataset are originally from publishers. Thus, they are of highquality. We label each publication as “open citation publication,” “closed ci-tation publication,” or “others” (i.e., without any reference or with referencesnot registered in Crossref) and calculate statistical key figures of various kindsconcerning open citation data. The paper contributes to understand the cur-rent state of open citation data and their challenges (e.g., for which kinds ofpublications citation data are not available).The paper is organized as follows: Section 2 describes the datasets used forour analysis. We report the result of our analysis in Section 3 before concludingthe paper in Section 4.2 DatasetFig. 1 summarizes how the publication dataset to be analyzed is generated andhow the state of open citation of each publication is identified.As metadata of publications, we use the crossref metadata dataset as ofSeptember 5, 2018.5 The dataset has been built by querying the Crossref API65 https://archive.org/download/crossref_doi_dump_201809, last accessed on04/23/20196 https://github.com/greenelab/Crossref, last accessed on 04/23/2019Evaluating the Availability of Open Citation Data 3Fig. 1. Pipeline of the dataset generation.and contains metadata of 99,874,789 publications that are assigned CrossrefDOIs. To analyze the influence of OA on the availability of citation data, we usea snapshot of unpaywall7 from September 24, 2018, together with the Crossrefmetadata dataset. Unpaywall is a browser extension, which finds legal OA ver-sions of scholarly publications. The snapshot of the database used by the browserextension is publicly available. It covers 99,940,229 publications with CrossrefDOIs and contains information regarding the state of OA for each publication. Inthis paper, we use 99,848,571 publications that are included in both the Crossrefmetadata dataset and unpaywall snapshot as basis of our analysis.We mark each publication as (1) “open citation publication”, (2) “closed ci-tation publication”, or (3) “others”. We use the OpenCitations COCI dataset [5]generated on November 12, 2018 [7] as open citation data. Note that the COCIdatasets are generated based on the citation data that are deposited to Crossrefby publishers. Thus, they are of high quality compared to other datasets.(1) If a publication is included in the COCI dataset as a citing publication, wemark it as open citation publication (i .e., publication that makes citationdata open). We identify 24,178,446 of 99,848,571 publications (24.22%) asopen citation publication.(2) Crossref also allows publishers to keep their citation data closed. Thus, thereare publications whose citation data are included in Crossref metadata butnot publicly available. Publications which are not included in the COCIdataset as citing publications, but have a Crossref’s field references-countgreater than zero, are marked as “closed citation publications”. 16,589,545publications (16.62%) are judged as closed citation publications.(3) We mark publications that are neither open citation publication nor closedcitation publication as “others.” 59,080,580 publications (59.15%) are classi-fied as such. ”others” include both publications without any reference, suchas editorial notes, and publications whose reference data are not registeredat Crossref. Although they have different meanings, both the COCI dataset7 https://unpaywall.org/products/snapshot, last accessed on 04/23/20194 C. Nishioka and M. Fa¨rberTable 1. State of open citation depending on publication types.all open cit. pub. (%) closed cit. pub. (%) others (%)journal-article 73,397,619 20,120,816 (27.41) 15,258,852 (20.79) 38,017,951 (51.80)book-chapter 11,726,076 1,048,255 (8.94) 898,513 (7.66) 9,779,308 (83.40)proceedings-article 5,398,085 2,798,106 (51.84) 342,295 (6.34) 2,257,684 (41.82)component 3,380,129 9 (0.00) 2 (0.00) 3,380,118 (100.00)dataset 1,703,078 67,250 (3.95) 2,996 (0.18) 1,632,832 (95.88)and Crossref metadata dataset do not have a way to express no-value infor-mation [1], which indicates that a publication has no reference.3 ResultIn this section, we first report the state of open citation in each type of pub-lication. Then, we present the result in each publication year. Thereafter, weexamine whether open access publications promote open citations.3.1 TypeCrossref DOIs have been assigned to various types of objects, such as journalarticles and book chapters. Table 1 shows the state of open citation for thefive object types appearing most frequently (given by the number of CrossrefDOIs). We observe that the percentage of open citation publications whose typeis “journal article” is 27.41%. This is almost the same value as the percentagefor all types (24.22% as described in Section 2). We observe a high percentageof open citation publications and a low percentage of closed citation publica-tions in proceedings articles (51.84% and 6.32%, respectively). Regarding othertypes, it is reasonable that publications whose type is “component” (e .g., figures,supplemental materials) have no reference.3.2 Publication YearFig. 2 shows the acceptance of open citations per year. The bars in Fig. 2 rep-resent the number of total publications, open citation publications, and closedcitation publications per publication year. The polygonal lines provide the per-centage of open citation publications and closed citation publications per pub-lication year. Please note that the number of publications in 2018 is low be-cause we use the dataset captured in September, 2018. We see that the totalnumber of publications as well as the number of open citation publications hasincreased over decades. In addition, the percentage of open citation publicationshas gradually increased. While the percentage has been around 18% before 1998,it reached 36% in 2011. Although older publications are read less [10], makingthe citation data of older publications available is important to trace the evolu-tion of scholarly knowledge. Regarding closed citation publications, we see thatthe percentage decreases slightly over the years.Evaluating the Availability of Open Citation Data 5Fig. 2. State of open citation along with publication year.Table 2. State of open citation depending on whether a publication is open access(OA).all open cit. pub. (%) closed cit. pub. (%) others (%)OA 24,961,752 7,077,997 (28.36) 2,391,627 (9.58) 15,492,128 (62.06)non-OA 74,886,819 17,100,449 (22.84) 14,197,918 (18.96) 43,588,452 (58.21)3.3 Open AccessThis section examines the influence of open access on the state of open citation.Open Access (OA) We first investigate whether there is a difference in thestate of open citation between open access (OA) publications and non-openaccess (non-OA) publications. Table 2 shows the number of total publications,open citation publications, and closed publications as well as their percentages.Referring to the state of OA, Table 2 indicates that 25.00% of the publicationsare OA in some way (e.g., gold OA, green OA), which is consistent with theresult given by Piwowar et al. [10]. OA publications are more likely to have theircitation information open than non-OA publications. However, the difference issmall. One possible reason is that if a publication is hybrid OA or green OA,publishers may not make citation data open.Open Access (OA) Journal We examine whether there is a difference inthe state of open citation between publications in an OA journal and those innon-OA journals. Table 3 presents the number of all publications, open citationpublications, and closed publications as well as their percentages. Unlike theresult in Table 2, publications in OA journals make citation data more open.Compared to publications in non-OA journals, publications in OA journals arecategorized as “others” more frequently. The reason might be that many smallpublishers publish OA journals but do not have enough resources to organizecitation data.6 C. Nishioka and M. Fa¨rberTable 3. State of open citation depending on whether a journal of a publication is anopen access (OA) journal.all open cit. pub. (%) closed cit. pub. (%) others (%)OA journal 5,670,103 1,613,260 (28.45) 280,547 (4.95) 3,776,296 (66.60)non-OA journal 94,178,468 22,565,186 (23.96) 16,308,998 (17.32) 55,304,284 (58.72)Table 4. State of open citation depending on whether a journal of a publication isincluded in DOAJ.all open cit. pub. (%) closed cit. pub. (%) others (%)in DOAJ 3,227,017 1,580,857 (48.99) 273,392 (8.47) 1,372,768 (42.54)not in DOAJ 96,621,554 22,597,589 (23.39) 16,316,153 (16.89) 57,707,812 (59.72)DOAJ DOAJ (Directory of Open Access Journals) is an online platform thathosts a curated list of OA journals. The project defines OA journals as scholarlyjournals which make all their content available for free. These journals meethigh quality standards, notably by exercising peer review or editorial qualitycontrol8. As of July 3, 2019, over 13,000 journals have been registered at DOAJ.We verify whether there is a difference in the state of open citation betweenpublications from a journal in DOAJ and those from a journal not in DOAJ.The results are shown in Table 4. The percentage of open citation publicationsfor publications in DOAJ is significantly higher than that for publications notin DOAJ. Since DOAJ lists journals that meet quality standards, citation dataof publications are properly organized by publishers and deposited to Crossref.However, a certain amount of citation data (8.47%) is closed.4 ConclusionIn this paper, we analyzed the current state of open citation based on the COCIdataset. We found out that for 24.22% of the publications with assigned CrossrefDOIs, the citations were open. As described by Di Iorio [2], such a value is notsufficient for evaluating researchers. 16.62% of the publications are identifiedas closed citation publication and 59.15% are “others”. “Others” include bothpublications without any reference and publications whose reference data arenot registered to Crossref. Although they have different meanings, the existingdatasets do not have a way to express no-value information. We found that thepercentage of open citation publications has gradually increased over the years.We did not observe a difference regarding the availability of open citation databetween OA publications and the non-OA publications. However, publicationspublished in a journal listed in the Directory of Open Access Journals (DOAJ)were more likely to open the citation data than those published in other journals.8 https://doaj.org/publishers, last accessed on 05/14/2019Evaluating the Availability of Open Citation Data 7References1. Darari, F., Prasojo, R.E., Nutt, W.: Expressing no-value information in RDF. In:Proceedings of the International Semantic Web Conference (ISWC) Posters andDemonstrations Track. CEUR Workshop Proceedings (2015)2. Di Iorio, A., Peroni, S., Poggi, F.: Open data to evaluate academic re-searchers: an experiment with the Italian Scientific Habilitation. arXiv preprintarXiv:1902.03287 (2019)3. Fa¨rber, M.: The Microsoft Academic Knowledge Graph: A Linked Data Sourcewith 8 Billion Triples of Scholarly Data. In: Proceedings of the 18th InternationalSemantic Web Conference. ISWC’19 (2019)4. Haunschild, R., Hug, S.E., Bra¨ndle, M.P., Bornmann, L.: The number of linkedreferences of publications in Microsoft Academic in comparison with the Web ofScience. Scientometrics 114(1), 367–370 (2018)5. Heibi, I., Peroni, S., Shotton, D.: Coci, the opencitations index of crossref opendoi-to-doi citations. arXiv preprint arXiv:1904.06052 (2019)6. Heibi, I., Peroni, S., Shotton, D.: Crowdsourcing open citations with croci-ananalysis of the current status of open citations, and a proposal. arXiv preprintarXiv:1902.02534 (2019)7. OpenCitations: COCI CSV dataset of all the citation data (Version 3) (2018),https://doi.org/10.6084/m9.figshare.6741422.v38. Peroni, S., Shotton, D.: Open Citation: Definition (2018).https://doi.org/10.6084/m9.figshare.6683855.v19. Peroni, S., Shotton, D., Vitali, F.: One year of the OpenCitations Corpus. In:International Semantic Web Conference. pp. 184–192. Springer (2017)10. Piwowar, H., Priem, J., Larivie`re, V., Alperin, J.P., Matthias, L., Norlander, B.,Farley, A., West, J., Haustein, S.: The state of OA: a large-scale analysis of theprevalence and impact of Open Access articles. PeerJ 6, e4375 (2018)11. Smith, L.C.: Citation analysis. Library Trends 30(1), 83–106 (1981)12. Todeschini, R., Baccini, A.: Handbook of bibliometric indicators: quantitative toolsfor studying and evaluating research. John Wiley & Sons (2016)

English

Evaluating the Availability of Open Citation Data

https://publikationen.bibliothek.kit.edu/1000098324/40468623

Evaluating the availability of open citation data

Abstract

Similar works

Full text

Available Versions

KITopen

Kyoto University Research Information Repository