Developing a comprehensive framework for multimodal feature extraction

Feature extraction is a critical component of many applied data science workflows. In recent years, rapid advances in artificial intelligence and machine learning have led to an explosion of feature extraction tools and services that allow data scientists to cheaply and effectively annotate their data along a vast array of dimensions---ranging from detecting faces in images to analyzing the sentiment expressed in coherent text. Unfortunately, the proliferation of powerful feature extraction services has been mirrored by a corresponding expansion in the number of distinct interfaces to feature extraction services. In a world where nearly every new service has its own API, documentation, and/or client library, data scientists who need to combine diverse features obtained from multiple sources are often forced to write and maintain ever more elaborate feature extraction pipelines. To address this challenge, we introduce a new open-source framework for comprehensive multimodal feature extraction. Pliers is an open-source Python package that supports standardized annotation of diverse data types (video, images, audio, and text), and is expressly with both ease-of-use and extensibility in mind. Users can apply a wide range of pre-existing feature extraction tools to their data in just a few lines of Python code, and can also easily add their own custom extractors by writing modular classes. A graph-based API enables rapid development of complex feature extraction pipelines that output results in a single, standardized format. We describe the package's architecture, detail its major advantages over previous feature extraction toolboxes, and use a sample application to a large functional MRI dataset to illustrate how pliers can significantly reduce the time and effort required to construct sophisticated feature extraction workflows while increasing code clarity and maintainability

Contributions of episodic retrieval and mentalizing to autobiographical thought: Evidence from functional neuroimaging, resting-state connectivity, and fMRI meta-analyses

A growing number of studies suggest the brain's "default network" becomes engaged when individuals recall their personal past or simulate their future. Recent reports of heterogeneity within the network raise the possibility that these autobiographical processes comprised of multiple component processes, each supported by distinct functional-anatomic subsystems. We previously hypothesized that a medial temporal subsystem contributes to autobiographical memory and future thought by enabling individuals to retrieve prior information and bind this information into a mental scene. Conversely, a dorsal medial subsystem was proposed to support social-reflective aspects of autobiographical thought, allowing individuals to reflect on the mental states of one's self and others (i.e. "mentalizing"). To test these hypotheses, we first examined activity in the default network subsystems as participants performed two commonly employed tasks of episodic retrieval and mentalizing. In a subset of participants, relationships among task-evoked regions were examined at rest, in the absence of an overt task. Finally, large-scale fMRI meta-analyses were conducted to identify brain regions that most strongly predicted the presence of episodic retrieval and mentalizing, and these results were compared to meta-analyses of autobiographical tasks. Across studies, laboratory-based episodic retrieval tasks were preferentially linked to the medial temporal subsystem, while mentalizing tasks were preferentially linked to the dorsal medial subsystem. In turn, autobiographical tasks engaged aspects of both subsystems. These results suggest the default network is a heterogeneous brain system whose subsystems support distinct component processes of autobiographical thought

Strategic Insight and Age-Related Goal-Neglect Influence Risky Decision-Making

Maximizing long-run gains often requires taking on some degree of risk, yet decision-makers often exhibit risk aversion (RA), rejecting risky prospects even when these have higher expected value (EV) than safer alternatives. We investigated whether explicit strategy instruction and practice can decrease prepotent RA, and whether aging impacts the efficacy of such an intervention. Participants performed a paired lottery task with options varying in risk and magnitude, both before and after practice with a similar task that encouraged maximization of EV and instruction to use this strategy in risky decisions. In both younger and older adults (OAs), strategy training reduced RA. Although RA was age-equivalent at baseline, larger training effects were observed in younger adults (YAs). These effects were not explained by risk-related (i.e., affective) interference effects or computation ability, but were consistent with a progressive, age-related neglect of the strategy across trials. Our findings suggest that strategy training can diminish RA, but that training efficacy is reduced among OAs, potentially due to goal neglect. We discuss implications for neural mechanisms that may distinguish older and YAs’ risky decision-making

Regional specialization within the human striatum for diverse psychological functions

Decades of animal and human neuroimaging research have identified distinct, but overlapping, striatal zones, which are interconnected with separable corticostriatal circuits, and are crucial for the organization of functional systems. Despite continuous efforts to subdivide the human striatum based on anatomical and resting-state functional connectivity, characterizing the different psychological processes related to each zone remains a work in progress. Using an unbiased, data-driven approach, we analyzed large-scale coactivation data from 5,809 human imaging studies. We (i) identified five distinct striatal zones that exhibited discrete patterns of coactivation with cortical brain regions across distinct psychological processes and (ii) identified the different psychological processes associated with each zone. We found that the reported pattern of cortical activation reliably predicted which striatal zone was most strongly activated. Critically, activation in each functional zone could be associated with distinct psychological processes directly, rather than inferred indirectly from psychological functions attributed to associated cortices. Consistent with well-established findings, we found an association of the ventral striatum (VS) with reward processing. Confirming less well-established findings, the VS and adjacent anterior caudate were associated with evaluating the value of rewards and actions, respectively. Furthermore, our results confirmed a sometimes overlooked specialization of the posterior caudate nucleus for executive functions, often considered the exclusive domain of frontoparietal cortical circuits. Our findings provide a precise functional map of regional specialization within the human striatum, both in terms of the differential cortical regions and psychological functions associated with each striatal zone

Predicting future cognitive decline from non-brain and multimodal brain imaging data in healthy and pathological aging

Previous literature has focused on predicting a diagnostic label from structural brain imaging. Since subtle changes in the brain precede a cognitive decline in healthy and pathological aging, our study predicts future decline as a continuous trajectory instead. Here, we tested whether baseline multimodal neuroimaging data improve the prediction of future cognitive decline in healthy and pathological aging. Nonbrain data (demographics, clinical, and neuropsychological scores), structural MRI, and functional connectivity data from OASIS-3 (N = 662; age = 46–96 years) were entered into cross-validated multitarget random forest models to predict future cognitive decline (measured by CDR and MMSE), on average 5.8 years into the future. The analysis was preregistered, and all analysis code is publicly available. Combining non-brain with structural data improved the continuous prediction of future cognitive decline (best test-set performance: R2 = 0.42). Cognitive performance, daily functioning, and subcortical volume drove the performance of our model. Including functional connectivity did not improve predictive accuracy. In the future, the prognosis of age-related cognitive decline may enable earlier and more effective individualized cognitive, pharmacological, and behavioral interventions

How open science helps researchers succeed

Open access, open data, open source, and other open scholarship practices are growing in popularity and necessity. However, widespread adoption of these practices has not yet been achieved. One reason is that researchers are uncertain about how sharing their work will affect their careers. We review literature demonstrating that open research is associated with increases in citations, media attention, potential collaborators, job opportunities, and funding opportunities. These findings are evidence that open research practices bring significant benefits to researchers relative to more traditional closed practices

Putting the self in self-correction: findings from the loss-of-confidence project

Science is often perceived to be a self-correcting enterprise. In principle, the assessment of scientific claims is supposed to proceed in a cumulative fashion, with the reigning theories of the day progressively approximating truth more accurately over time. In practice, however, cumulative self-correction tends to proceed less efficiently than one might naively suppose. Far from evaluating new evidence dispassionately and infallibly, individual scientists often cling stubbornly to prior findings. Here we explore the dynamics of scientific self-correction at an individual rather than collective level. In 13 written statements, researchers from diverse branches of psychology share why and how they have lost confidence in one of their own published findings. We qualitatively characterize these disclosures and explore their implications. A cross-disciplinary survey suggests that such loss-of-confidence sentiments are surprisingly common among members of the broader scientific population yet rarely become part of the public record. We argue that removing barriers to self-correction at the individual level is imperative if the scientific community as a whole is to achieve the ideal of efficient self-correction

BOLD Correlates of Trial-by-Trial Reaction Time Variability in Gray and White Matter: A Multi-Study fMRI Analysis

Reaction time (RT) is one of the most widely used measures of performance in experimental psychology, yet relatively few fMRI studies have included trial-by-trial differences in RT as a predictor variable in their analyses. Using a multi-study approach, we investigated whether there are brain regions that show a general relationship between trial-by-trial RT variability and activation across a range of cognitive tasks.The relation between trial-by-trial differences in RT and brain activation was modeled in five different fMRI datasets spanning a range of experimental tasks and stimulus modalities. Three main findings were identified. First, in a widely distributed set of gray and white matter regions, activation was delayed on trials with long RTs relative to short RTs, suggesting delayed initiation of underlying physiological processes. Second, in lateral and medial frontal regions, activation showed a "time-on-task" effect, increasing linearly as a function of RT. Finally, RT variability reliably modulated the BOLD signal not only in gray matter but also in diffuse regions of white matter.The results highlight the importance of modeling trial-by-trial RT in fMRI analyses and raise the possibility that RT variability may provide a powerful probe for investigating the previously elusive white matter BOLD signal

A tale of two 'opens': intersections between Free and Open Source Software and Open Scholarship

There is no clear-cut boundary between Free and Open Source Software and Open Scholarship, and the histories, practices, and fundamental principles between the two remain complex. In this study, we critically appraise the intersections and differences between the two movements. Based on our thematic comparison here, we conclude several key things. First, there is substantial scope for new communities of practice to form within scholarly communities that place sharing and collaboration/open participation at their focus. Second, Both the principles and practices of FOSS can be more deeply ingrained within scholarship, asserting a balance between pragmatism and social ideology. Third, at the present, Open Scholarship risks being subverted and compromised by commercial players. Fourth, the shift and acceleration towards a system of Open Scholarship will be greatly enhanced by a concurrent shift in recognising a broader range of practices and outputs beyond traditional peer review and research articles. In order to achieve this, we propose the formulation of a new type of institutional mandate. We believe that there is substantial need for research funders to invest in sustainable open scholarly infrastructure, and the communities that support them, to avoid the capture and enclosure of key research services that would prevent optimal researcher behaviours. Such a shift could ultimately lead to a healthier scientific culture, and a system where competition is replaced by collaboration, resources (including time and people) are shared and acknowledged more efficiently, and the research becomes inherently more rigorous, verified, and reproducible