The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals

In our chapter we address the statistical analysis of percentiles: How should the citation impact of institutions be compared? In educational and psychological testing, percentiles are already used widely as a standard to evaluate an individual's test scores - intelligence tests for example - by comparing them with the percentiles of a calibrated sample. Percentiles, or percentile rank classes, are also a very suitable method for bibliometrics to normalize citations of publications in terms of the subject category and the publication year and, unlike the mean-based indicators (the relative citation rates), percentiles are scarcely affected by skewed distributions of citations. The percentile of a certain publication provides information about the citation impact this publication has achieved in comparison to other similar publications in the same subject category and publication year. Analyses of percentiles, however, have not always been presented in the most effective and meaningful way. New APA guidelines (American Psychological Association, 2010) suggest a lesser emphasis on significance tests and a greater emphasis on the substantive and practical significance of findings. Drawing on work by Cumming (2012) we show how examinations of effect sizes (e.g. Cohen's d statistic) and confidence intervals can lead to a clear understanding of citation impact differences

Plans for phase coherent long baseline interferometry for geophysical applications using the Anik-B communications satellite

A pilot project to establish an operational phase stable very long baseline interferometer (VLBI) for geophysical studies is described. Methods for implementation as well as practical applications are presented

Global Maps of Science based on the new Web-of-Science Categories

In August 2011, Thomson Reuters launched version 5 of the Science and Social Science Citation Index in the Web of Science (WoS). Among other things, the 222 ISI Subject Categories (SCs) for these two databases in version 4 of WoS were renamed and extended to 225 WoS Categories (WCs). A new set of 151 Subject Categories (SCs) was added, but at a higher level of aggregation. Since we previously used the ISI SCs as the baseline for a global map in Pajek (Rafols et al., 2010) and brought this facility online (at http://www.leydesdorff.net/overlaytoolkit), we recalibrated this map for the new WC categories using the Journal Citation Reports 2010. In the new installation, the base maps can also be made using VOSviewer (Van Eck & Waltman, 2010).Comment: Scientometrics, in pres

A recursive field-normalized bibliometric performance indicator: An application to the field of library and information science

Two commonly used ideas in the development of citation-based research performance indicators are the idea of normalizing citation counts based on a field classification scheme and the idea of recursive citation weighing (like in PageRank-inspired indicators). We combine these two ideas in a single indicator, referred to as the recursive mean normalized citation score indicator, and we study the validity of this indicator. Our empirical analysis shows that the proposed indicator is highly sensitive to the field classification scheme that is used. The indicator also has a strong tendency to reinforce biases caused by the classification scheme. Based on these observations, we advise against the use of indicators in which the idea of normalization based on a field classification scheme and the idea of recursive citation weighing are combined

Universality of Performance Indicators based on Citation and Reference Counts

We find evidence for the universality of two relative bibliometric indicators of the quality of individual scientific publications taken from different data sets. One of these is a new index that considers both citation and reference counts. We demonstrate this universality for relatively well cited publications from a single institute, grouped by year of publication and by faculty or by department. We show similar behaviour in publications submitted to the arXiv e-print archive, grouped by year of submission and by sub-archive. We also find that for reasonably well cited papers this distribution is well fitted by a lognormal with a variance of around 1.3 which is consistent with the results of Radicchi, Fortunato, and Castellano (2008). Our work demonstrates that comparisons can be made between publications from different disciplines and publication dates, regardless of their citation count and without expensive access to the whole world-wide citation graph. Further, it shows that averages of the logarithm of such relative bibliometric indices deal with the issue of long tails and avoid the need for statistics based on lengthy ranking procedures.Comment: 15 pages, 14 figures, 11 pages of supplementary material. Submitted to Scientometric

Outlier Edge Detection Using Random Graph Generation Models and Applications

Outliers are samples that are generated by different mechanisms from other normal data samples. Graphs, in particular social network graphs, may contain nodes and edges that are made by scammers, malicious programs or mistakenly by normal users. Detecting outlier nodes and edges is important for data mining and graph analytics. However, previous research in the field has merely focused on detecting outlier nodes. In this article, we study the properties of edges and propose outlier edge detection algorithms using two random graph generation models. We found that the edge-ego-network, which can be defined as the induced graph that contains two end nodes of an edge, their neighboring nodes and the edges that link these nodes, contains critical information to detect outlier edges. We evaluated the proposed algorithms by injecting outlier edges into some real-world graph data. Experiment results show that the proposed algorithms can effectively detect outlier edges. In particular, the algorithm based on the Preferential Attachment Random Graph Generation model consistently gives good performance regardless of the test graph data. Further more, the proposed algorithms are not limited in the area of outlier edge detection. We demonstrate three different applications that benefit from the proposed algorithms: 1) a preprocessing tool that improves the performance of graph clustering algorithms; 2) an outlier node detection algorithm; and 3) a novel noisy data clustering algorithm. These applications show the great potential of the proposed outlier edge detection techniques.Comment: 14 pages, 5 figures, journal pape

Long-Term Flux Monitoring of LSI +61 303 at 2.25 and 8.3 GHz

LSI +61 303 is an exotic binary system consisting of a ~10 Msun B star and a compact object which is probably a neutron star. The system is associated with the interesting radio source GT0236+610 that exhibits bright radio outbursts with a period of 26.5 days. We report the results of continuous daily radio interferometric observations of GT0236+610 at 2.25 and 8.3 GHz from 1994 January to 1996 February. The observations cover 25 complete (and 3 partial) cycles with multiple observations each day. We detect substantial cycle-to-cycle variability of the radio emission characterized by a rapid onset of the radio flares followed by a more gradual decrease of the emission. We detect a systematic change of the radio spectral index alpha which typically becomes larger than zero at the onset of the radio outbursts. This behavior is suggestive of expansion of material initially optically thick to radio frequencies, indicating either that synchrotron or inverse Compton cooling are important or that the free-free optical depth to the source is rapidly changing. After two years of observations, we see only weak evidence for the proposed 4-year periodic modulation in the peak flux of the outbursts. We observe a secular trend in the outburst phases according the the best published ephemeris. This trend indicates either orbital period evolution, or a drift in outburst orbital phase in response to some other change in the system.Comment: 23 pages, LaTex, 7 figures, to appear in ApJ, v491, Dec 10th issue, for associated info and preprints see http://www.srl.caltech.edu/personnel/paulr/lsi.htm

Innovations in peer review in scholarly publishing : a meta-summary [version 1; under peer-review]

Background: There are currently numerous innovations in peer review and quality assurance in scholarly publishing. The Research on Research Institute conducted a programme of co-produced projects investigating these innovations. This literature review was part of one such project ‘Experiments in peer review’ which created an inventory and framework of peer review innovations. The aim of this literature review was to aid the development of the inventory by identifying innovations in peer review reported in the scholarly literature and by providing a summary of the different approaches. Methods: This meta-summary is based on data identified from Web of Science and Scopus limited from 2010 to 2021. A total of 247 papers were screened, with 6 review articles chosen for the focus of the literature review. Items were selected that described approaches to innovating peer review or illustrated examples. Results: The summary of innovations are drawn from 6 review articles. The innovations are divided into three high-level categories: approaches to peer review, reviewer focussed initiatives and technology to support peer review with sub-categories of results presented in tabular form and summarised. A summary of all innovations found is also presented. Conclusions: From a simple synthesis of the review authors’ conclusions, three key messages are presented: observations on current practice; authors’ views on the implications of innovations in peer review; and calls for action in peer review research and practice