Recipes for replication:Applying open science principles to research software development and data collection with cognitive tasks

During the past decade, science witnessed a replication crisis where many findings of published research could not be reproduced by other researchers. Attempts to address this "replication crisis" have identified several avenues for improvement, such as making science more open. This allows for more transparency in the scientific workflow, thus reducing a researcher's degrees of freedom and enabling researchers to check each other's work more extensively. Open science is a multifaceted concept, but in practice, there seems to be a strong focus on pre-registration of research designs, open data, and open access publications. However, between the research design and data/publication, there is the phase of data collection. So far, this phase has received relatively little attention even though it is an essential part of the scientific workflow. Hence, this dissertation focuses on open data collection in the behavioral domain, with an emphasis on cognitive tasks. In modern behavioral science, cognitive task procedures are often automated by software running on a computer. Hence, the focus is on research software development and data collection with cognitive tasks, which are evaluated from the perspective of five schools of thought on open science: the democratic, infrastructure, pragmatic, measurement, and public school. I discuss how applying open science principles to research software development and behavioral data collection with cognitive tasks may address the replication crisis and may improve the quality of science in general

Mobility, kinship, and marriage in Indo-European society

The purpose of this chapter is to explore one area in which comparative linguistic data can play a role in interpreting late Neolithic and early Bronze Age genetic and archaeological data from western Eurasia. The sharp rise in available samples of ancient DNA enables the establishment of kinship relations between individuals in prehistoric graveyards. It also makes it possible to establish where their ancestors came from. The analysis of strontium, oxygen, carbon, and lead isotopes in the tooth enamel of these same individuals provides information about movements during their lives. When these techniques are combined, we obtain a much better idea about who moved where and when in prehistory. Being able to establish the diet of prehistoric individuals and which diseases they may have suffered from allows archaeologists to set up hypotheses as to why some of the population movements that can be observed in the archaeological data might have taken place. Linguistics can offer a valuable contribution to the discussion of why people moved by shedding light on factors other than diet or disease; it may, for example, help to explain cases in which males appear to have migrated differently from females. In order to understand how, this chapter will take a closer look at the linguistic evidence for kinship relations and the role of gender and age in Indo-European society. At the end of the chapter, hypotheses about mobility, kinship, and marriage in early Indo-European society as based on the linguistic data will be compared to the findings of recent research into ancient DNA and isotope analysis.Descriptive and Comparative Linguistic

Can we measure individual differences in cognitive measures reliably via smartphones? A comparison of the flanker effect across device types and samples

Research deployed via the internet and administered via smartphones could have access to more diverse samples than lab-based research. Diverse samples could have relatively high variation in their traits and so yield relatively reliable measurements of individual differences in these traits. Several cognitive tasks that originated from the experimental research tradition have been reported to yield relatively low reliabilities (Hedge et al., 2018) in samples with restricted variance (students). This issue could potentially be addressed by smartphone-mediated administration in diverse samples. We formulate several criteria to determine whether a cognitive task is suitable for individual differences research on commodity smartphones: no very brief or precise stimulus timing, relative response times (RTs), a maximum of two response options, and a small number of graphical stimuli. The flanker task meets these criteria. We compared the reliability of individual differences in the flanker effect across samples and devices in a preregistered study. We found no evidence that a more diverse sample yields higher reliabilities. We also found no evidence that commodity smartphones yield lower reliabilities than commodity laptops. Hence, diverse samples might not improve reliability above student samples, but smartphones may well measure individual differences with cognitive tasks reliably. Exploratively, we examined different reliability coefficients, split-half reliabilities, and the development of reliability estimates as a function of task length.</p

Methods to split cognitive task data for estimating split-half reliability: A comprehensive review and systematic assessment

Estimating the reliability of cognitive task datasets is commonly done via split-half methods. We review four methods that differ in how the trials are split into parts: a first-second half split, an odd-even trial split, a permutated split, and a Monte Carlo-based split. Additionally, each splitting method could be combined with stratification by task design. These methods are reviewed in terms of the degree to which they are confounded with four effects that may occur in cognitive tasks: effects of time, task design, trial sampling, and non-linear scoring. Based on the theoretical review, we recommend Monte Carlo splitting (possibly in combination with stratification by task design) as being the most robust method with respect to the four confounds considered. Next, we estimated the reliabilities of the main outcome variables from four cognitive task datasets, each (typically) scored with a different non-linear algorithm, by systematically applying each splitting method. Differences between methods were interpreted in terms of confounding effects inflating or attenuating reliability estimates. For three task datasets, our findings were consistent with our model of confounding effects. Evidence for confounding effects was strong for time and task design and weak for non-linear scoring. When confounding effects occurred, they attenuated reliability estimates. For one task dataset, findings were inconsistent with our model but they may offer indicators for assessing whether a split-half reliability estimate is appropriate. Additionally, we make suggestions on further research of reliability estimation, supported by a compendium R package that implements each of the splitting methods reviewed here

An Interpretation of Late Quaternary Glacial Flow Indicators in the Baie des Chaleurs Region, Northern New Brunswick

A sequence of late Quaternary geologic events in northern New Brunswick is determined from striation analysis derived from published data, open file reports, and field research conducted by the authors since 1985. These data are integrated with clast provenance and clast fabric trend analysis, as well as information from other studies in the surrounding area. South of the Baie des Chaleurs, a complicated Late Wisconsinan glacial history is preserved in the form of erosive features including nailhead striae, miniature crag-and-tails, and various scars, striations, and fractures. The rarity of sedimentary deposits and datable materials precludes simple stratigraphie interpretation. Based on over 1,000 striation sites, we conclude four major phases of glacial flow affected the area during the Late Wisconsinan: 1) an early flow to the southeast which reflects local Appalachian ice; 2) a second phase of glacial flow to the east indicating a Laurentide ice influence in western New Brunswick; 3) a third phase of glacial flow to the north-northeast, which may represent ice response to drawdown in the Baie des Chaleurs; and 4) a final multidirectional flow indicating localized ice response during the last stages of Late Wisconsinan glaciation. The absence of Canadian Shield erratics in northern New Brunswick is explained in terms of ice streaming along the St. Lawrence channel beneath a southward-flowing Laurentide Ice Sheet. Basal ice debris (including Shield erratics) was apparently truncated and removed by the obliquely flowing ice stream, leaving relatively clean ice in the Ice Sheet as it entered Gaspésie and ultimately New Brunswick.On a déterminé la séquence des événements géologiques du Quaternaire supérieur survenus dans le nord du Nouveau-Brunswick à partir de l'analyse des stries tirée de diverses sources. Au sud de la baie des Chaleurs, l'évolution glaciaire complexe du Wisconsinien supérieur peut être retracée à partir des formes d'érosion glaciaire dont les clouures, les crag-and-tails miniatures, les cicatrices de toutes sortes, les stries et les fractures. La rareté des sédiments et des matériaux pouvant être datés excluent toute interprétation fondée sur la stratigraphie. En s'appuyant sur plus de 1000 sites de stries, on croit que la région a connu quatre épisodes glaciaires pendant le Wisconsinien supérieur: 1 ) le premier mouvement, de faible importance, s'est fait vers le SE en provenance des Appalaches; 2) le deuxième écoulement glaciaire vers PE démontre la force des glaces laurentidiennes au nord du Nouveau-Brunswick; 3) le troisième mouvement glaciaire vers le NNE pourrait être une réaction à l'affaissement qui s'est produit dans la baie des Chaleurs; 4) le dernier mouvement multidirectionnel s'est fait à partir de calottes localisées pendant les derniers stades de la glaciation wisconsinienne. On explique l'absence des blocs erratiques dans le nord du Nouveau-Brunswick par un écoulement des glaces le long du chenal laurentien interrompant l'Inlandsis laurentidien s'écoulant vers le sud. Les débris glaciaires de fond (y compris les blocs erratiques du Bouclier) ont apparemment été tronqués et enlevés par un courant glaciaire oblique, épurant ainsi la glace qui pénétrait en Gaspésie, puis au Nouveau-Brunswick.Een Interpretatie van Laat-glaciale Ijsbewegingsindicaties in de Chaleur Bay Regio, Noord New Brunswick. De Kwartair geologische kartering van noord New Brunswick begon met de studies van Robert Chalmers aan het einde van de 19e eeuw. Sindsdien zijn verschillende pogingen gedaan om de glaciale geschiedenis van dit gebied te ontravelen en uiteindelijk zijn Chalmers ideeen herontdekt en aangepast aan moderne theorien en modellen van glaciale processen en isostasie. De afwezigheid van stratigrafische secties en de aanwezigheid van vêle glaciale erosieve en morphologische vormen heeft geleid tot de ontwikkeling van een "erosie-stratigrafie ". Deze informatie wordt gesteund door till lithologische en ges-teente orientatie studies. Het resultaat van dit gedetailleerde karterings project is de her-kenning van 4 verschillende ijs-bewegingen. Datering van deze gebeurtenissen is een probleem vanwege een gebrek aan dateer-bare materialen. Een eerste zuidoostelijke ijsbeweging is geregistreerd in het oostelijke deel van het gebied en in de Chaleur Coastal Plain. De tweede en sterkste gebeurtenis geeft een oostwaartse ijsbeweging te zien, die waarschijnlijk gedurende het gehele Wisconsinan (Weichselien) en mogelijk het gehele Pleistoceen van belang kan zijn geweest. Een derde ijsbeweging is in noord-oostelijke richt-ing in de Baie des Chaleurs. In een latere fase, lokale ijsbewegingen vonden plaats in verschillende richtingen vanaf de hooglanden en mogelijk vanuit Baie des Chaleurs. Er is een poging gedaan deze gebeurtenissen in een régionale context te plaatsen, waarin de interactie van de Laurentide Ice Sheet met het Appalachian Ice Complex en het concept van een Laurentian ijsstroomkanaal centraal staan

Effectiveness of two web-based cognitive bias modification interventions targeting approach and attentional bias in gambling problems:Study protocol for a pilot randomised controlled trial

BACKGROUND: Disordered gamblers have phenotypical and pathological similarities to those with substance use disorders (SUD), including exaggerated automatic cognitive processing of motivationally salient gambling cues in the environment (i.e., attentional and approach bias). Cognitive bias modification (CBM) is a family of computerised interventions that have proved effective in successfully re-training these automatic cognitive biases in SUD. CBM interventions can, in principle, be administered online, thus showing potential of being a low-cost, low-threshold addition to conventional treatments. This paper presents the design of a pilot randomised controlled trial exploring the effectiveness of two web-based CBM interventions targeting attentional and approach bias towards gambling cues in a sample of Dutch and Belgian problematic and pathological gamblers. METHODS/DESIGN: Participants (N = 182) are community-recruited adults experiencing gambling problems, who have gambled at least twice in the past 6 months and are motivated to change their gambling behaviour. After a baseline assessment session, participants are randomly assigned to one of four experimental conditions (attentional or approach bias training, or the placebo version of the two trainings) and complete six sessions of training. At baseline and before each training session, participants receive automated personalised feedback on their gambling motives and reasons to quit or reduce gambling. The post-intervention, 1-month, and 3-month follow-up assessments will examine changes in gambling behaviour, with frequency and expenditure as primary outcomes, and depressive symptoms and gambling-related attentional and approach biases as secondary outcomes. Secondary analyses will explore possible moderators (interference control capacity and trait impulsivity) and mediators (change in cognitive bias) of training effects on the primary outcomes. DISCUSSION: This study is the first to explore the effectiveness of an online CBM intervention for gambling problems. The results of this study can be extremely valuable for developing e-health interventions for gambling problems and further understanding the role of motivational implicit cognitive processes underlying problematic gambling behaviour. TRIAL REGISTRATION: Netherlands Trial Register, NTR5096 . Registered on 11 March 2015