Quality science from quality measurement: The role of measurement type with respect to replication and effect size magnitude in psychological research

Copyright: © 2018 Kornbrot et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.The quality of psychological studies is currently a major concern. The Many Labs Project (MLP) and the Open-Science-Collaboration (OSC) have collected key data on replicability and statistical effect sizes. We build on this work by investigating the role played by three measurement types: ratings, proportions and unbounded (measures without conceptual upper limits, e.g. time). Both replicability and effect sizes are dependent on the amount of variability due to extraneous factors. We predicted that the role of such extraneous factors might depend on measurement type, and would be greatest for ratings, intermediate for proportions and least for unbounded. Our results support this conjecture. OSC replication rates for unbounded, 43% and proportion 40% combined are reliably higher than those for ratings at 20% (effect size, w = .20). MLP replication rates for the original studies are: pro- portion = .74, ratings = .40 (effect size w = .33). Original effect sizes (Cohen’s d) are highest for: unbounded OSC cognitive = 1.45, OSC social = .90); next for proportions (OSC cogni- tive = 1.01, OSC social = .84, MLP = .82); and lowest for ratings (OSC social = .64, MLP = .31). These findings are of key importance to scientific methodology and design, even if the reasons for their occurrence are still at the level of conjecture.Peer reviewe

How was it for you? Psychophysics and the evaluation of student experience of e-learning

DOC as published in Proceedings. PPT as delivered at meeting [some additional information]Evaluating the student experience of Higher Education has become a matter of national importance in several countries. For example, in England & Wales the National Student Survey (NSS) is administered on line to all students in the final year of their undergraduate degree. The NSS uses 5-point Likert scales, giving extent of agreement or disagreement with positive statements. There are 22 questions, covering 6 aspects of student experience. This presentation considers how psychophysical methods based on signal detection theory or Luceâ s choice theory can be used to analyze such data. Such methods can determine how well the questions discriminate different aspects of experience, as well as how favourably the students experience these aspects of their education. Particular emphasis is given to exploring discipline differences together with the effects of recent technologies, such as managed learning environments and web 2.0 social software.Final Accepted Versio

Misleading ‘quality’ measures in Higher Education : problems from combining diverse indicators that include subjective ratings and academic performance and costs

Original article can be found at: http://www.radstats.org.uk/Quality indicators are often derived from weighted sums of diverse items, including ordinal Likert items. This procedure can be dangerously misleading because it takes no account of correlations among indicators. It also takes no account of whether indicators are input measures, e.g. prior achievement of incoming students, or outcome measures, e.g. proportion getting a good degrees or student satisfaction. UK Higher Education data for 04-05 were analyzed taking these issues into account. Multiple regression showed, unsurprisingly, that ‘bright’ students with high prior achievement did well on all outcome indicators. Getting a good degree was not influenced by any other measure. Completing a course was additionally positively associated with academic pay and spend on library and computing facilities. A good destination (not currently seeking work) was additionally positively associated with number of staff per student and vice-chancellor pay. Student satisfaction was additionally influenced, but negatively, with vice-chancellor pay. The implications for evaluating university quality are discussedPeer reviewedFinal Accepted Versio

The influence of maternity units' intrapartum intervention rates and midwives' risk perception for women suitable for midwifery led care

Objective: to test the hypothesis that midwives working in higher intervention units would have a higher perception of risk for the intrapartum care of women suitable for midwifery-led care than midwives working in lower intervention units. Methods: an initial retrospective analysis of the computerised records of 9887 healthy Caucasian women in spontaneous labour enabled the categorisation of 11 units as either ‘lower intrapartum intervention’ or ‘higher intrapartum intervention’ units. A survey of the midwives involved in intrapartum care in these 11 units, using standardised scenario questionnaires, was used to investigate midwives’ options for intrapartum interventions, their perceptions of intrapartum risk and the accuracy of these perceptions in the light of actual maternity outcomes. Findings: midwives working in maternity units that had a higher level of intervention generally perceived intrapartum risks to be higher than midwives working in lower intervention units. However, midwives generally underestimated the ability of women to progress normally and overestimated the advantages of technological interventions, in particular epidural analgesia. Conclusions: variations in intrapartum care cannot be solely explained by the characteristics of the women. The influence of the workplace culture plays a significant role in shaping midwives’ perceptions of risk, but it seems even more likely that the medicalisation of childbirth has had an influence on midwives’ appreciation of intrapartum risks. Intervention rates for low-risk births are often higher than recommended by research. The level of interventions varies across hospitals and higher rates are associated with higher perception of risk by midwives. Attention needs to be given to the influence the workplace plays in shaping midwives’ perception of risk; and to the effect of organisational culture on intervention rates.Peer reviewe

An intrapartum intervention scoring system for the comparison of maternity units' intrapartum care of nulliparous women suitable for midwifery led care

Original article can be found at : http://www.sciencedirect.com/science/ Copyright ElsevierObjective: to develop an intrapartum intervention scoring tool which could be used to define maternity units as either ‘lower intrapartum intervention’ or ‘higher intrapartum intervention’ units. This scoring tool was designed to form the basis of a comparison of the perception of risk by midwives working in either ‘lower intrapartum intervention’ or ‘higher intrapartum intervention’ units. Design: three aspects were included: (1) the systematic data reduction of the St. Mary's Maternity Information System database used by 11 maternity units to include Caucasian nulliparous women suitable for midwifery-led care; (2) the calculation and the ranking of frequency distributions for the following interventions/management: (a) the management of breech presentation and of one previous caesarean section, the choice of home birth; and (b) augmentation of labour, use of electronic fetal monitoring, use of epidural, method of delivery; (3) the sum of the individual intrapartum ranking marks made up the final intrapartum score for each unit. Results: intrapartum interventions varied considerably between units. The scoring system enabled units to be described as either ‘Lower intrapartum intervention’ or ‘Higher intrapartum intervention’ units. Conclusions: routinely collected computerised data can be used to identify the outcomes of intrapartum care. This study suggests that the analysis of computerised data could provide a suitable basis for the audit and the comparison of intrapartum interventions for the care of women suitable for midwifery-led care.Peer reviewe