Empirical Study of Data Sharing by Authors Publishing in PLoS Journals

Many journals now require authors share their data with other investigators, either by depositing the data in a public repository or making it freely available upon request. These policies are explicit, but remain largely untested. We sought to determine how well authors comply with such policies by requesting data from authors who had published in one of two journals with clear data sharing policies.We requested data from ten investigators who had published in either PLoS Medicine or PLoS Clinical Trials. All responses were carefully documented. In the event that we were refused data, we reminded authors of the journal's data sharing guidelines. If we did not receive a response to our initial request, a second request was made. Following the ten requests for raw data, three investigators did not respond, four authors responded and refused to share their data, two email addresses were no longer valid, and one author requested further details. A reminder of PLoS's explicit requirement that authors share data did not change the reply from the four authors who initially refused. Only one author sent an original data set.We received only one of ten raw data sets requested. This suggests that journal policies requiring data sharing do not lead to authors making their data sets available to independent investigators

Psychometric evaluation of the computerized battery for neuropsychological evaluation of children (BENCI) among school aged children in the context of HIV in an urban Kenyan setting

IntroductionCulturally validated neurocognitive measures for children in Low- and Middle-Income Countries are important in the timely and correct identifcation of neurocognitive impairments. Such measures can inform development of interventions for children exposed to additional vulnerabilities like HIV infection. The Battery for Neuropsychological Evaluation of Children (BENCI) is an openly available, computerized neuropsychological battery specifcally developed to evaluate neurocognitive impairment. This study adapted the BENCI and evaluated its reliability and validity in Kenya.Methodology The BENCI was adapted using translation and back-translation from Spanish to English. The psy‑chometric properties were evaluated in a case–control study of 328 children (aged 6 – 14 years) living with HIV and 260 children not living with HIV in Kenya. We assessed reliability, factor structure, and measurement invariance withrespect to HIV. Additionally, we examined convergent validity of the BENCI using tests from the Kilif Toolkit. ResultsInternal consistencies (0.49<α<0.97) and test–retest reliabilities (-.34 to .81) were sufcient-to-good for most of the subtests. Convergent validity was supported by signifcant correlations between the BENCI’s Verbal memoryand Kilif’s Verbal List Learning (r=.41), the BENCI’s Visual memory and Kilif’s Verbal List Learning (r=.32) and the BENCI’s Planning total time test and Kilif’s Tower Test (r= -.21) and the BENCI’s Abstract Reasoning test and Kilif’s Raven’sProgressive Matrix (r=.21). The BENCI subtests highlighted meaningful diferences between children living with HIV and those not living with HIV. After some minor adaptions, a confrmatory four-factor model consisting of fexibility,fuency, reasoning and working memory ftted well (χ2=135.57, DF=51, N=604, p<.001, RMSEA=.052, CFI=.944, TLI=.914) and was partially scalar invariant between HIV positive and negative groups.ConclusionThe English version of the BENCI formally translated for use in Kenya can be further adapted and inte‑grated in clinical and research settings as a valid and reliable cognitive test battery

Modeling Interactions Between Latent Variables in Research on Type D Personality: A Monte Carlo Simulation and Clinical Study of Depression and Anxiety

Item does not contain fulltextSeveral approaches exist to model interactions between latent variables. However, it is unclear how these perform when item scores are skewed and ordinal. Research on Type D personality serves as a good case study for that matter. In Study 1, we fitted a multivariate interaction model to predict depression and anxiety with Type D personality, operationalized as an interaction between its two subcomponents negative affectivity (NA) and social inhibition (SI). We constructed this interaction according to four approaches: (1) sum score product; (2) single product indicator; (3) matched product indicators; and (4) latent moderated structural equations (LMS). In Study 2, we compared these interaction models in a simulation study by assessing for each method the bias and precision of the estimated interaction effect under varying conditions. In Study 1, all methods showed a significant Type D effect on both depression and anxiety, although this effect diminished after including the NA and SI quadratic effects. Study 2 showed that the LMS approach performed best with respect to minimizing bias and maximizing power, even when item scores were ordinal and skewed. However, when latent traits were skewed LMS resulted in more false-positive conclusions, while the Matched PI approach adequately controlled the false-positive rate

The Dawn of Open Access to Phylogenetic Data

The scientific enterprise depends critically on the preservation of and open access to published data. This basic tenet applies acutely to phylogenies (estimates of evolutionary relationships among species). Increasingly, phylogenies are estimated from increasingly large, genome-scale datasets using increasingly complex statistical methods that require increasing levels of expertise and computational investment. Moreover, the resulting phylogenetic data provide an explicit historical perspective that critically informs research in a vast and growing number of scientific disciplines. One such use is the study of changes in rates of lineage diversification (speciation - extinction) through time. As part of a meta-analysis in this area, we sought to collect phylogenetic data (comprising nucleotide sequence alignment and tree files) from 217 studies published in 46 journals over a 13-year period. We document our attempts to procure those data (from online archives and by direct request to corresponding authors), and report results of analyses (using Bayesian logistic regression) to assess the impact of various factors on the success of our efforts. Overall, complete phylogenetic data for ~60% of these studies are effectively lost to science. Our study indicates that phylogenetic data are more likely to be deposited in online archives and/or shared upon request when: (1) the publishing journal has a strong data-sharing policy; (2) the publishing journal has a higher impact factor, and; (3) the data are requested from faculty rather than students. Although the situation appears dire, our analyses suggest that it is far from hopeless: recent initiatives by the scientific community -- including policy changes by journals and funding agencies -- are improving the state of affairs

Peer review quality and transparency of the peer-review process in open access and subscription journals

BACKGROUND:Recent controversies highlighting substandard peer review in Open Access (OA) and traditional (subscription) journals have increased the need for authors, funders, publishers, and institutions to assure quality of peer-review in academic journals. I propose that transparency of the peer-review process may be seen as an indicator of the quality of peer-review, and develop and validate a tool enabling different stakeholders to assess transparency of the peer-review process. METHODS AND FINDINGS:Based on editorial guidelines and best practices, I developed a 14-item tool to rate transparency of the peer-review process on the basis of journals' websites. In Study 1, a random sample of 231 authors of papers in 92 subscription journals in different fields rated transparency of the journals that published their work. Authors' ratings of the transparency were positively associated with quality of the peer-review process but unrelated to journal's impact factors. In Study 2, 20 experts on OA publishing assessed the transparency of established (non-OA) journals, OA journals categorized as being published by potential predatory publishers, and journals from the Directory of Open Access Journals (DOAJ). Results show high reliability across items (α = .91) and sufficient reliability across raters. Ratings differentiated the three types of journals well. In Study 3, academic librarians rated a random sample of 140 DOAJ journals and another 54 journals that had received a hoax paper written by Bohannon to test peer-review quality. Journals with higher transparency ratings were less likely to accept the flawed paper and showed higher impact as measured by the h5 index from Google Scholar. CONCLUSIONS:The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals

Challenges in Data Intensive Analysis at Scientific Experimental User Facilities

Today's scientific challenges such as routes to a sustainable energy future, materials by design or biological and chemical environmental remediation methods, are complex problems that require the integration of a wide range of complementary expertise to be addressed successfully. Experimental and computational science research methods can hereby offer fundamental insights for their solution. Experimental facilities in particular can contribute through a large variety of investigative methods, which can span length scales from millions of kilometers (radar) to the sub-nucleus (LHC). These methods are used to probe structure, properties, and function of objects from single elements to whole communities. Hereby direct imaging techniques are a powerful means to develop an atomistic understanding of scientific issues. For example, the identification ofmechanisms associated with chemical, material, and biological transformations requires the direct observation of the reactions to build up an understanding of the atom-by-atom structural and chemical changes. Computational science can aid the planning of such experiments, correlate results, explain or predict the phenomena as they would be observed and thus aid their interpretation. Furthermore computational science can be essential for the investigation of phenomena that are difficult to observe due to their scale, reaction time or extreme conditions. Combining experimental and computational techniques provides scientists with the ability to research structures and processes at various levels of theory, e.g. providing molecular 'movies' of complex reactions that show bond breaking and reforming in natural time scales, along with the intermediate states to understand the mechanisms that govern the chemical transformations. This chapter will discuss the critical data intensive analysis challenges faced by the experimental science community at large scale and laboratory based facilities. The chapter will highlight current solutions and lay out perspectives for the future, such as methods to achieve real time analysis capabilities and the challenges and opportunities of data integration across experimental scales, levels of theory, and varying techniques

Replication is more than hitting the lottery twice

The main goal of our target article was to provide concrete recommendations for improving the replicability of research findings. Most of the comments focus on this point. In addition, a few comments were concerned with the distinction between replicability and generalizability and the role of theory in replication. We address all comments within the conceptual structure of the target article, and hope to convince readers that replication in psychological science amounts to much more than hitting the lottery twice