Crowdsourcing hypothesis tests: Making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer fiveoriginal research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams renderedstatistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.</div

Biyografya:Selim Nüzhet Gerçek:İlk basın tarihçimiz

Taha Toros Arşivi, Dosya No: 22-Selim Nüzhet GerçekUnutma İstanbul projesi İstanbul Kalkınma Ajansı'nın 2016 yılı "Yenilikçi ve Yaratıcı İstanbul Mali Destek Programı" kapsamında desteklenmiştir. Proje No: TR10/16/YNY/010

Understanding and modeling willingness-to-pay for public policies to enhance road safety: a perspective from Pakistan

Evaluating road safety improvements becomes important because it can assist policymakers in allocating economic resources to improve safety and implementing effective policy interventions. As such, this study aims to estimate the value of road safety risk measures using a new modeling approach for willingness-to-pay (WTP). Specifically, this study integrates a machine learning technique (decision tree) with a correlated random parameters Tobit with heterogeneity-in-means model. The decision tree identifies a priori relationships for higher-order interactions, while the model captures unobserved heterogeneity and the correlation between random parameters. The proposed modeling framework examines the determinants of public WTP for improving road safety using a sample of car drivers from Peshawar, Pakistan. WTP for fatal and severe injury risk reductions is estimated and used to calculate the values of corresponding risk reductions, which can be used for monetizing the cost of road traffic crashes in the country. Modeling results reveal that most respondents are willing to contribute to road safety improvement policies. Further, the model also uncovers significant heterogeneity in WTP corresponding to the safer perception of the overall road infrastructure and perceived risk of accident involvement. Systematic preference heterogeneity is also found in the model by including higher-order interactions, providing additional insights into the complex relationship of WTP with its determinants. Further, the marginal effects of explanatory variables indicate different sensitivities toward WTP, which can help to quantify the impacts of these variables on both the probability and magnitude of WTP. Overall, the proposed modeling framework has a twofold contribution. First, the modeling framework provides valuable insights into the determinants of public WTP, mainly when the heterogeneous effects of variables are interactive. Second, its implementation and consequent findings shall help prioritize different road safety policies/projects by better understanding public sensitivity to WTP.</p

Mobile phone use on a typical day and frequency.

<p>Mobile phone use on a typical day and frequency.</p

Self-reported attitudes and task-management strategies for mobile phone distracted driving.

<p>Self-reported attitudes and task-management strategies for mobile phone distracted driving.</p

Perceived crash risk per mobile phone sub-task.

<p>Perceived crash risk per mobile phone sub-task.</p

Logistic regression analysis: Predicting handheld conversations and texting/browsing engagement.

<p>Logistic regression analysis: Predicting handheld conversations and texting/browsing engagement.</p

The risks of using ChatGPT to obtain common safety-related information and advice

ChatGPT is a highly advanced AI language model that has gained widespread popularity. It is trained to understand and generate human language and is used in various applications, including automated customer service, chatbots, and content generation. While it has the potential to offer many benefits, there are also concerns about its potential for misuse, particularly in relation to providing inappropriate or harmful safety-related information. To explore ChatGPT's (specifically version 3.5) capabilities in providing safety-related advice, a multidisciplinary consortium of experts was formed to analyse nine cases across different safety domains: using mobile phones while driving, supervising children around water, crowd management guidelines, precautions to prevent falls in older people, air pollution when exercising, intervening when a colleague is distressed, managing job demands to prevent burnout, protecting personal data in fitness apps, and fatigue when operating heavy machinery. The experts concluded that there is potential for significant risks when using ChatGPT as a source of information and advice for safety-related issues. ChatGPT provided incorrect or potentially harmful statements and emphasised individual responsibility, potentially leading to ecological fallacy. The study highlights the need for caution when using ChatGPT for safety-related information and expert verification, as well as the need for ethical considerations and safeguards to ensure users understand the limitations and receive appropriate advice, especially in low- and middle-income countries. The results of this investigation serve as a reminder that while AI technology continues to advance, caution must be exercised to ensure that its applications do not pose a threat to public safety.</p

Crowdsourcing hypothesis tests: making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams rendered statistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim