Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed

Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial

Background: In this study, we aimed to evaluate the effects of tocilizumab in adult patients admitted to hospital with COVID-19 with both hypoxia and systemic inflammation. Methods: This randomised, controlled, open-label, platform trial (Randomised Evaluation of COVID-19 Therapy [RECOVERY]), is assessing several possible treatments in patients hospitalised with COVID-19 in the UK. Those trial participants with hypoxia (oxygen saturation <92% on air or requiring oxygen therapy) and evidence of systemic inflammation (C-reactive protein ≥75 mg/L) were eligible for random assignment in a 1:1 ratio to usual standard of care alone versus usual standard of care plus tocilizumab at a dose of 400 mg–800 mg (depending on weight) given intravenously. A second dose could be given 12–24 h later if the patient's condition had not improved. The primary outcome was 28-day mortality, assessed in the intention-to-treat population. The trial is registered with ISRCTN (50189673) and ClinicalTrials.gov (NCT04381936). Findings: Between April 23, 2020, and Jan 24, 2021, 4116 adults of 21 550 patients enrolled into the RECOVERY trial were included in the assessment of tocilizumab, including 3385 (82%) patients receiving systemic corticosteroids. Overall, 621 (31%) of the 2022 patients allocated tocilizumab and 729 (35%) of the 2094 patients allocated to usual care died within 28 days (rate ratio 0·85; 95% CI 0·76–0·94; p=0·0028). Consistent results were seen in all prespecified subgroups of patients, including those receiving systemic corticosteroids. Patients allocated to tocilizumab were more likely to be discharged from hospital within 28 days (57% vs 50%; rate ratio 1·22; 1·12–1·33; p<0·0001). Among those not receiving invasive mechanical ventilation at baseline, patients allocated tocilizumab were less likely to reach the composite endpoint of invasive mechanical ventilation or death (35% vs 42%; risk ratio 0·84; 95% CI 0·77–0·92; p<0·0001). Interpretation: In hospitalised COVID-19 patients with hypoxia and systemic inflammation, tocilizumab improved survival and other clinical outcomes. These benefits were seen regardless of the amount of respiratory support and were additional to the benefits of systemic corticosteroids. Funding: UK Research and Innovation (Medical Research Council) and National Institute of Health Research

Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised controlled, open-label, platform trial

Background: Many patients with COVID-19 have been treated with plasma containing anti-SARS-CoV-2 antibodies. We aimed to evaluate the safety and efficacy of convalescent plasma therapy in patients admitted to hospital with COVID-19. Methods: This randomised, controlled, open-label, platform trial (Randomised Evaluation of COVID-19 Therapy [RECOVERY]) is assessing several possible treatments in patients hospitalised with COVID-19 in the UK. The trial is underway at 177 NHS hospitals from across the UK. Eligible and consenting patients were randomly assigned (1:1) to receive either usual care alone (usual care group) or usual care plus high-titre convalescent plasma (convalescent plasma group). The primary outcome was 28-day mortality, analysed on an intention-to-treat basis. The trial is registered with ISRCTN, 50189673, and ClinicalTrials.gov, NCT04381936. Findings: Between May 28, 2020, and Jan 15, 2021, 11558 (71%) of 16287 patients enrolled in RECOVERY were eligible to receive convalescent plasma and were assigned to either the convalescent plasma group or the usual care group. There was no significant difference in 28-day mortality between the two groups: 1399 (24%) of 5795 patients in the convalescent plasma group and 1408 (24%) of 5763 patients in the usual care group died within 28 days (rate ratio 1·00, 95% CI 0·93–1·07; p=0·95). The 28-day mortality rate ratio was similar in all prespecified subgroups of patients, including in those patients without detectable SARS-CoV-2 antibodies at randomisation. Allocation to convalescent plasma had no significant effect on the proportion of patients discharged from hospital within 28 days (3832 [66%] patients in the convalescent plasma group vs 3822 [66%] patients in the usual care group; rate ratio 0·99, 95% CI 0·94–1·03; p=0·57). Among those not on invasive mechanical ventilation at randomisation, there was no significant difference in the proportion of patients meeting the composite endpoint of progression to invasive mechanical ventilation or death (1568 [29%] of 5493 patients in the convalescent plasma group vs 1568 [29%] of 5448 patients in the usual care group; rate ratio 0·99, 95% CI 0·93–1·05; p=0·79). Interpretation: In patients hospitalised with COVID-19, high-titre convalescent plasma did not improve survival or other prespecified clinical outcomes. Funding: UK Research and Innovation (Medical Research Council) and National Institute of Health Research

Statistical modeling of received signal strength for an FSO link over maritime environment

Free space optical communications (FSO) have the potential to substantially improve communications technology in terms of channel capacity and offer an alternative to their RF counterpart. Additional characteristics related to security, immunity, flexibility and low cost issues render FSO a reasonable candidate for military applications. FSO technology does not come without challenges. Its major issue is the local meteorological parameters that give rise to various atmospheric phenomena. The purpose of this work is to facilitate the performance prediction of an FSO communication link over a maritime environment by utilizing macroscopic meteorological parameters, i.e. air temperature, wind speed, relative humidity, air pressure, dew point, solar radiation and sea temperature, obtained from point measurements. The received signal strength indicator (RSSI) of the FSO receiver has been utilized as the performance metric of the channel and a closed form expression has been deduced. The model has then been validated against real meteorological data and the predicted RSSI values exhibited a reasonably strong correlation with the observed ones. Atmospheric turbulence has been taken into account using the Navy Surface Layer Model (NAVSLaM) to estimate the structure index parameter from the same meteorological data and thus allowed for a statistical correlation between the refractive index structure parameter and RSSI. © 2021 Elsevier B.V

Experimental performance analysis of an optical communication channel over maritime environment

Free space optical communications (FSO), which make use of the visible and infrared spectrum for data transmission, offer significant advantages such as a very high data rate, security and immunity, low cost of installation and ease of use without any license restrictions. However, a significant challenge for FSO systems is their inherent constraints due to environmental conditions and especially atmospheric turbulence. This paper focuses on the experimental performance analysis of a real FSO system in a maritime environment. We propose a new model which allows an FSO link performance estimation over sea and depends upon point measurements of environmental parameters. The Received Signal Strength Indicator (RSSI) has been measured and a second-order polynomial has been constructed using regression modeling to quantify its relation with macroscopic environmental parameters collected by a weather station. This model has then been validated against real meteorological data over different period of times and exhibited a reasonably strong correlation. Atmospheric turbulence has been determined using bulk estimates of the structure index parameter extracted from the same meteorological data, and thus allowed for a statistical correlation between turbulence and RSSI. In the second part of the paper, the probability distribution of the RSSI data has been investigated and the Kullback-Leibler (KL) divergence has been used to investigate the difference between probability distributions over the same variable. As an illustrative example of the process, the Weibull, Lognormal and Gamma distributions have been evaluated against the RSSI data probability distribution and the latter has proved to exhibit the best fit. © 2020 by the authors. Licensee MDPI, Basel, Switzerland

Using machine learning algorithms for accurate received optical power prediction of an fso link over a maritime environment

The performance prediction of an optical communications link over maritime environments has been extensively researched over the last two decades. The various atmospheric phenomena and turbulence effects have been thoroughly explored, and long-term measurements have allowed for the construction of simple empirical models. The aim of this work is to demonstrate the prediction accuracy of various machine learning (ML) algorithms for a free-space optical communication (FSO) link performance, with respect to real time, non-linear atmospheric conditions. A large data set of received signal strength indicators (RSSI) for a laser communications link has been collected and analyzed against seven local atmospheric parameters (i.e., wind speed, pressure, temperature, humidity, dew point, solar flux and air-sea temperature difference). The k-nearest-neighbors (KNN), tree-based methods-decision trees, random forest and gradient boosting-and artificial neural networks (ANN) have been employed and compared among each other using the root mean square error (RMSE) and the coefficient of determination (R2 ) of each model as the primary performance indices. The regression analysis revealed an excellent fit for all ML models, indicative of their ability to offer a significant improvement in FSO performance modeling as compared to traditional regression models. The best-performing R2 model found to be the ANN approach (0.94867), while random forests achieved the most optimal RMSE result (7.37). © 2021 by the authors. Licensee MDPI, Basel, Switzerland

Optical turbulence measurements and modeling over Monterey Bay

Free space optical (FSO) communication in a maritime environment involves unique challenges due to the existing environmental conditions. An important degradation factor is atmospheric turbulence that causes irradiance fluctuations (scintillation) at the detector. A significant amount of theoretical and experimental research has been conducted to quantify those effects. This paper presents the results of an experimental campaign that took place during September and October of 2020 over the Monterey Bay in California. The main goal of this campaign was to measure atmospheric turbulence over the water and compare the results with a theoretical model called the Navy Surface Layer Model (NAVSLaM), developed by the Meteorology Department at the Naval Postgraduate School (NPS), as well as conduct a regression analysis for turbulence predictive modeling based on environmental parameters. The results showed very good agreement between theory and experiment. © 2022 Elsevier B.V