The reproducibility project:a model of large-scale collaboration for empirical research on reproducibility

The goal of science is to accumulate knowledge that answers questions such as "How do things work?" and "Why do they work that way?" Scientists use a variety of methodologies to describe, predict, and explain natural phenomena. These methods are so diverse that it is difficult to define a unique scientific method, although all scientific methodologies share the assumption of repeatability (Hempel & Oppenheim, 1948; Kuhn, 1962; Popper, 1934; Salmon, 1989). In this chapter, we first briefly review why replications are highly valued but rarely published. Then we describe a collaborative effort — the Reproducibility Project — to estimate the rate and predictors of reproducibility in psychological science. Finally, we detail how we are conducting this project as a large-scale, distributed, open collaboration. A description of the procedures and challenges may assist and inspire other teams to conduct similar projects in other areas of science

What p-hacking really looks like : a comment on Masicampo & LaLande (2012)

no abstract

Diagnosing collaboration in practice-based learning: Equality and intra-individual variability of physical interactivity

Collaborative problem solving (CPS), as a teaching and learning approach, is considered to have the potential to improve some of the most important skills to prepare students for their future. CPS often differs in its nature, practice, and learning outcomes from other kinds of peer learning approaches, including peer tutoring and cooperation; and it is important to establish what identifies collaboration in problem-solving situations. The identification of indicators of collaboration is a challenging task. However, students physical interactivity can hold clues of such indicators. In this paper, we investigate two non-verbal indexes of student physical interactivity to interpret collaboration in practice-based learning environments: equality and intra-individual variability. Our data was generated from twelve groups of three Engineering students working on open-ended tasks using a learning analytics system. The results show that high collaboration groups have member students who present high and equal amounts of physical interactivity and low and equal amounts of intra-individual variability

Machine and human observable differences in groups’ collaborative problem-solving behaviours

This paper contributes to our understanding of how to design learning analytics to capture and analyse collaborative problem-solving (CPS) in practice-based learning activities. Most research in learning analytics focuses on student interaction in digital learning environments, yet still most learning and teaching in schools occurs in physical environments. Investigation of student interaction in physical environments can be used to generate observable differences among students, which can then be used in the design and implementation of Learning Analytics. Here, we present several original methods for identifying such differences in groups CPS behaviours. Our data set is based on human observation, hand position (fiducial marker) and heads direction (face recognition) data from eighteen students working in six groups of three. The results show that the high competent CPS groups spend an equal distribution of time on their problem-solving and collaboration stages. Whereas, the low competent CPS groups spend most of their time in identifying knowledge and skill deficiencies only. Moreover, as machine observable data shows, high competent CPS groups present symmetrical contributions to the physical tasks and present high synchrony and individual accountability values. The findings have significant implications on the design and implementation of future learning analytics systems

Internal and external cooling methods and their effect on body temperature, thermal perception and dexterity

© 2018 The Authors. Published by PLOS. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1371/journal.pone.0191416© 2018 Maley et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Objective The present study aimed to compare a range of cooling methods possibly utilised by occupational workers, focusing on their effect on body temperature, perception and manual dexterity. Methods Ten male participants completed eight trials involving 30 min of seated rest followed by 30 min of cooling or control of no cooling (CON) (34C, 58% relative humidity). The cooling methods utilised were: ice cooling vest (CV0), phase change cooling vest melting at 14C (CV14), evaporative cooling vest (CVEV), arm immersion in 10C water (AI), portable water-perfused suit (WPS), heliox inhalation (HE) and ice slushy ingestion (SL). Immediately before and after cooling, participants were assessed for fine (Purdue pegboard task) and gross (grip and pinch strength) manual dexterity. Rectal and skin temperature, as well as thermal sensation and comfort, were monitored throughout. Results Compared with CON, SL was the only method to reduce rectal temperature (P = 0.012). All externally applied cooling methods reduced skin temperature (P0.05). Conclusion The present study observed that ice ingestion or ice applied to the skin produced the greatest effect on rectal and skin temperature, respectively. AI should not be utilised if workers require subsequent fine manual dexterity. These results will help inform future studies investigating appropriate pre-cooling methods for the occupational worker.This project is financially supported by the US Government through the Technical Support Working Group within the Combating Terrorism Technical Support Office.Published versio

Enteric-coated sodium bicarbonate supplementation improves high-intensity cycling performance in trained cyclists

Purpose: Enteric-coated sodium bicarbonate (NaHCO3) can attenuate gastrointestinal (GI) symptoms following acute bicarbonate loading, although the subsequent effects on exercise performance have not been investigated. The purpose of this study was to examine the effects of enteric-coated NaHCO3 supplementation on high-intensity exercise performance and GI symptoms. Methods: Eleven trained male cyclists completed three 4 km time trials after consuming; a placebo or 0.3 g∙kg–1 body mass NaHCO3 in enteric-coated or gelatin capsules. Exercise trials were timed with individual peak blood bicarbonate ion concentration ([HCO3–]). Blood acid–base balance was measured pre-ingestion, pre-exercise, and post-exercise, whereas GI symptoms were recorded pre-ingestion and immediately pre-exercise. Results: Pre-exercise blood [HCO3−] and potential hydrogen (pH) were greater for both NaHCO3 conditions (P &lt; 0.0005) when compared to placebo. Performance time was faster with enteric-coated (− 8.5 ± 9.6 s, P = 0.044) and gelatin (− 9.6 ± 7.2 s, P = 0.004) NaHCO3 compared to placebo, with no significant difference between conditions (mean difference = 1.1 ± 5.3 s, P = 1.000). Physiological responses were similar between conditions, although blood lactate ion concentration was higher with gelatin NaHCO3 (2.4 ± 1.7 mmol∙L–1, P = 0.003) compared with placebo. Furthermore, fewer participants experienced GI symptoms with enteric-coated (n = 3) compared to gelatin (n = 7) NaHCO3. Discussion: Acute enteric-coated NaHCO3 consumption mitigates GI symptoms at the onset of exercise and improves subsequent 4 km cycling TT performance. Athletes who experience GI side-effects after acute bicarbonate loading may, therefore, benefit from enteric-coated NaHCO3 supplementation prior to exercise performance.</p

The effect of postexercise carbohydrate and protein ingestion on bone metabolism

Purpose To investigate the effect of feeding carbohydrate and protein (CHO+PRO), immediately or 2 h after an exhaustive run, on the bone turnover response in endurance runners. Methods 10 men (age 28±5 y, height 1.74±0.05 m, body mass 69.7±6.3 kg) performed treadmill running at 75%VO2max, until exhaustion, on three occasions. Blood was collected before and immediately, 1, 2, 3, 4 and 24 h post-exercise, for measurement of β-CTX, P1NP, PTH, PO4, ACa and Ca2+. This was a randomised, counterbalanced, placebo-controlled, single-blinded, cross-over study. The three trials were; i) placebo (PLA), PLA solution was ingested immediately and 2 h post-exercise, ii) immediate feeding (IF), CHO+PRO (1.5 g.kgBM-1 dextrose and 0.5 g.kgBM-1 whey) were ingested immediately post-exercise and PLA 2 h post-exercise, and iii) delayed feeding (DF), PLA was ingested immediately post-exercise and CHO+PRO solution 2 h post-exercise. Data were analysed using repeated measures ANOVA and post-hoc Tukey’s HSD. Results At 1 and 2 h post-exercise, β-CTX concentrations were lower in the IF trial than the DF and PLA trials (P≤0.001). At 3 h post-exercise, β-CTX concentrations were higher in the PLA trial than the IF (P≤0.001) and DF trials (P=0.026). At 4 h post-exercise, β-CTX concentrations were lower in the DF trial than the IF (P=0.003) and PLA trials (P≤0.001). At 4 h post-exercise, P1NP was higher in the IF trial than in DF (P=0.026) and PLA trials (P=0.001). At 3 h post-exercise, PTH was higher in the IF trial than the DF trial (P≤0.001). Conclusions Following exhaustive running, immediate ingestion of CHO+PRO may be beneficial, as it decreases bone resorption marker concentrations and increases bone formation marker concentrations; creating a more positive bone turnover balance

Contrasting vertical and horizontal representations of affect in emotional visual search

The final publication is available at Springer via http://dx.doi.org/ 10.3758/s13423-015-0884-6Independent lines of evidence suggest that the representation of emotional evaluation recruits both vertical and horizontal spatial mappings. These two spatial mappings differ in their experiential origins and their productivity, and available data suggest that they differ in their saliency. Yet, no study has so far compared their relative strength in an attentional orienting reaction time task that affords the simultaneous manifestation of both of them. Here we investigated this question using a visual search task with emotional faces. We presented angry and happy face targets and neutral distracter faces in top, bottom, left, and right locations on the computer screen. Conceptual congruency effects were observed along the vertical dimension supporting the ‘up=good’ metaphor, but not along the horizontal dimension. This asymmetrical processing pattern was observed when faces were presented in a cropped (Experiment 1) and whole (Experiment 2) format. These findings suggest that the ‘up=good’ metaphor is more salient and readily activated than the ‘right=good’ metaphor, and that the former outcompetes the latter when the task context affords the simultaneous activation of both mappings