Quality science from quality measurement: The role of measurement type with respect to replication and effect size magnitude in psychological research

Copyright: © 2018 Kornbrot et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.The quality of psychological studies is currently a major concern. The Many Labs Project (MLP) and the Open-Science-Collaboration (OSC) have collected key data on replicability and statistical effect sizes. We build on this work by investigating the role played by three measurement types: ratings, proportions and unbounded (measures without conceptual upper limits, e.g. time). Both replicability and effect sizes are dependent on the amount of variability due to extraneous factors. We predicted that the role of such extraneous factors might depend on measurement type, and would be greatest for ratings, intermediate for proportions and least for unbounded. Our results support this conjecture. OSC replication rates for unbounded, 43% and proportion 40% combined are reliably higher than those for ratings at 20% (effect size, w = .20). MLP replication rates for the original studies are: pro- portion = .74, ratings = .40 (effect size w = .33). Original effect sizes (Cohen’s d) are highest for: unbounded OSC cognitive = 1.45, OSC social = .90); next for proportions (OSC cogni- tive = 1.01, OSC social = .84, MLP = .82); and lowest for ratings (OSC social = .64, MLP = .31). These findings are of key importance to scientific methodology and design, even if the reasons for their occurrence are still at the level of conjecture.Peer reviewe

Decision making under time pressure: An independent test of sequential sampling models

Choice probability and choice response time data from a risk-taking decision-making task were compared with predictions made by a sequential sampling model. The behavioral data, consistent with the model, showed that participants were less likely to take an action as risk levels increased, and that time pressure did not have a uniform effect on choice probability. Under time pressure, participants were more conservative at the lower risk levels but were more prone to take risks at the higher levels of risk. This crossover interaction reflected a reduction of the threshold within a single decision strategy rather than a switching of decision strategies. Response time data, as predicted by the model, showed that participants took more time to make decisions at the moderate risk levels and that time pressure reduced response time across all risk levels, but particularly at the those risk levels that took longer time with no pressure. Finally, response time data were used to rule out the hypothesis that time pressure effects could be explained by a fast-guess strategy

Length of stay as a performance indicator : robust statistical methodology

This is a pre-copy-editing, author produced PDF of an article accepted for publication in IMA Journal of Management Mathematics following peer review. The definitive publisher-authenticated version [Kulinskaya, E. , Kornbrot, D. and Gao, H. (2005) 'Length of stay as a performance indicator : robust statistical methodology'. IMA Journal of Management Mathematics 16 (4) pp.369-381] is available online at : http://imaman.oxfordjournals.org/archive/index.dtl . --Copyright Institute of Mathematics and its Applications-- --DOI : 10.1093/imaman/dpi015Length of stay (LOS) is an important performance indicator for costing and hospital management and a key measure of efficiency of NHS. However, LOS is difficult to analyse because its statistical distribution is non-normal and LOS data habitually have many outliers. Furthermore, the usefulness of LOS for improving NHS performance is undermined because no adjustments are made for some key factors. This paper addresses both these problems. Health episodes statistics data from the UK NHS for 1997/98, and 1998/99 are analysed to investigate the effects of five key variables: admission method, discharge destination, provider (hospital) type, speciality and NHS region. All are found to influence LOS. The effects of some factors are substantial, and were not previously known, and so are not included in planned future NHS performance measures, e.g. LOS is at least 25% longer for patients transferred from other hospitals rather than admitted as an emergency; and LOS for patients discharged to private institutions is more than twice that for patients discharged to NHS institutions or their own home. The problem of finding the most appropriate statistical analysis for data of the LOS type is addressed by comparing standard general linear model methods with an advanced robust method called truncated maximum likelihood (TML). The TML methods are shown to have several advantages over standard methods, in terms of model fit and accuracy of parameter estimation. Implications of these findings for future use of LOS are considered.Peer reviewe

Estimates of utility function parameters from signal-detection experiments

Original article can be found at : http://content.apa.org/ Copyright American Psychological Association [Full text of this article is not available in the UHRA]Peer reviewe

The use of log and power transformations in the analysis of length of stay data

Full text of this article is not available in the UHRA.Peer reviewe