Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed

Two-Dimensional Modelling of the Mixing Patterns in a Tropical Semiarid Reservoir

Tropical semi-arid regions suffer with recurrent droughts and uncertain water availability, but a few research studies have been conducted to further understand those complexities and their relationships with reservoir hydrodynamics. This study assessed the hydrodynamic processes of a multiple-use reservoir located in the Brazilian semiarid region. The aim was to apply the CE-QUAL-W2 model to understand the lake’s thermal structure and its variabilities in time and space by using the Richardson’s number (Ri) as a reference. Meteorological patterns were also investigated. Results show that: (1) no significant changes were found by analysing the spatial variabilities of stratification; (2) seasonal changes were relevant as more robust stratification stability was observed in the wet period when water availability may be impacted by poor water quality; (3) from meteorological evaluations, rainfall showed a strong coefficient of determination with Ri (r² of 0.77); and (4) a threshold value of 60 mm in monthly precipitation was found as an indication of a stable stratification in the water column. Wind speed and water level partly influenced Ri’s variabilities, while low impact was noted for air temperature and inflow. These results can promote an improvement in water-resources management by linking rainfall regime and reservoir hydrodynamics

Desempenho de dois equipamentos na colheita semimecanizada da cultura da mandioca Performance of two equipments in semi-mechanized harvesting of cassava

Estudou-se o desempenho operacional de dois equipamentos na colheita da mandioca. Um afofador do solo e um arrancador de mandioca foram avaliados. Analisaram-se a patinagem das rodas motoras, as perdas de produto, os danos mecânicos provocados às raízes e a capacidade de campo dos conjuntos trator-equipamento. Depois da passagem dos conjuntos, as raízes inteiras colhidas e que permaneceram no solo e com danos mecânicos, foram separadas e pesadas. O conjunto trator-arrancador apresentou melhor dinâmica de corte, penetração no solo e menor patinagem das rodas. Não foram observadas perdas de raízes para os conjuntos avaliados. Os equipamentos apresentaram semelhantes danos mecânicos às raízes. O conjunto trator-afofador apresentou capacidade de campo 25% superior ao conjunto trator-arrancador. Entretanto, como o conjunto trator-afofador não deixa as raízes expostas e não possui dispositivo de corte frontal da parte aérea das plantas, foi observado aumento de 32% no tempo necessário para as operações de corte e separação das raízes. O desempenho operacional do conjunto trator-arrancador foi superior ao conjunto trator-afofador.<br>The operational performance of two equipments for the cassava harvest was studied. A soil disaggregating implement and an up root implement were evaluated. The wheel slip, the losses, the mechanical damage caused in roots and the field capacity of the tractor-implement systems were analyzed. In each test, the whole roots and the roots with mechanical damages were separated and their mass were obtained. Tractor-up root system showed the best dynamic of cut and soil penetration, which was considered the smallest wheel slip. No root losses were observed for the two systems. Between implements there was not difference for mechanical damages caused in roots. The tractor-disaggregating implement system presented field capacity of 25% upper than tractor-up root system. However, as tractor-disaggregating implement system did not leave roots exposed and it did not have a frontal device of plant pruning, it was observed an increase of 32% on necessary time for cutting and roots separation. The tractor-up root system provided higher operational performance

Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed