Quantification of inter-brain coupling: A review of current methods used in haemodynamic and electrophysiological hyperscanning studies

Hyperscanning is a form of neuroimaging experiment where the brains of two or more participants are imaged simultaneously whilst they interact. Within the domain of social neuroscience, hyperscanning is increasingly used to measure inter-brain coupling (IBC) and explore how brain responses change in tandem during social interaction. In addition to cognitive research, some have suggested that quantification of the interplay between interacting participants can be used as a biomarker for a variety of cognitive mechanisms aswell as to investigate mental health and developmental conditions including schizophrenia, social anxiety and autism. However, many different methods have been used to quantify brain coupling and this can lead to questions about comparability across studies and reduce research reproducibility. Here, we review methods for quantifying IBC, and suggest some ways moving forward. Following the PRISMA guidelines, we reviewed 215 hyperscanning studies, across four different brain imaging modalities: functional near-infrared spectroscopy (fNIRS), functional magnetic resonance (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG). Overall, the review identified a total of 27 different methods used to compute IBC. The most common hyperscanning modality is fNIRS, used by 119 studies, 89 of which adopted wavelet coherence. Based on the results of this literature survey, we first report summary statistics of the hyperscanning field, followed by a brief overview of each signal that is obtained from each neuroimaging modality used in hyperscanning. We then discuss the rationale, assumptions and suitability of each method to different modalities which can be used to investigate IBC. Finally, we discuss issues surrounding the interpretation of each method

Neural foundations of cooperative social interactions

The embodied-embedded-enactive-extended (4E) approach to study cognition suggests that interaction with the world is a crucial component of our cognitive processes. Most of our time, we interact with other people. Therefore, studying cognition without interaction is incomplete. Until recently, social neuroscience has only focused on studying isolated human and animal brains, leaving interaction unexplored. To fill this gap, we studied interacting participants, focusing on both intra- and inter-brain (hyperscanning) neural activity. In the first study, we invited dyads to perform a visual task in both a cooperative and a competitive context while we measured EEG. We found that mid-frontal activity around 200-300 ms after receiving monetary rewards was sensitive to social context and differed between cooperative and competitive situations. In the second study, we asked participants to coordinate their movements with each other and with a robotic partner. We found significantly stronger EEG amplitudes at frontocentral electrodes when people interacted with a robotic partner. Lastly, we performed a comprehensive literature review and the first meta-analysis in the emerging field of hyperscanning that validated it as a method to study social interaction. Taken together, our results showed that adding a second participant (human or AI/robotic) fostered our understanding of human cognition. We learned that the activity at frontocentral electrodes is sensitive to social context and type of partner (human or robotic). In both studies, the participants’ interaction was required to show these novel neural processes involved in action monitoring. Similarly, studying inter-brain neural activity allows for the exploration of new aspects of cognition. Many cognitive functions involved in successful social interactions are accompanied by neural synchrony between brains, suggesting the extended form of our cognition

Exploring neural markers of language processing using fNIRS in typically developed children and children with Developmental Language Disorder

Background: Developmental language disorder (DLD) is a life-long condition with no clear biological causes that affects approximately 8% of the population. The diagnosis currently relies on behavioural testing that is not reliably performed on children younger than school age. Consequently, the diagnosis and treatment of DLD is often delayed until after children enter formal education. Early work in the field suggests that neural markers of language processing could be used to develop an objective diagnostic tool that will allow for accurate and early identification of DLD in preschool years and thus access to early interventions. Here we propose the use of a novel non-invasive neuroimaging technique called functional near infrared spectroscopy (fNIRS) to identify neural markers of language processing in children with DLD. Additionally, we argue that to understand atypical language processing, it is imperative to also investigate typical cortical activations in response to language processing to establish the developmental trajectories of the language network. Parallel to these studies we also investigate patterns of neural synchrony during parent-child interactions. Speech and language development in children is thought to rely on successful parent-child interactions, however, little is known regarding the underlying neural mechanisms from which they arise. Methods: Cross-Sectional fNIRS Studies: A total of 36 participants aged 6–16-year-old (1 participant with DLD) were recruited in two cross-sectional fNIRS studies. Participants underwent a 10-minute resting state imaging session and completed a series of computer-administered language and cognitive tasks while their brain activity was recorded using fNIRS from the bilateral inferior frontal gyrus (IFG) and the bilateral auditory cortices. Hyperscanning fNIRS Study: 12 children aged between 3 and 5 years old and their mothers participated in this study. Neural synchrony in mother-child dyads was measured bilaterally over frontal and temporal areas using fNIRS whilst the dyads were asked to play together (interactive condition) and separately (independent condition). Communication patterns were captured via video recordings and conversational turns were coded. Survey Study: 43 parents of children with DLD and 44 clinicians with DLD expertise completed a qualitative online survey detailing their views, concerns and recommendations regarding the use of neuroimaging-based tool for the diagnosis and monitoring of DLD. Results: Cross sectional fNIRS studies: In typically developed children and adolescents, widespread connections between the language regions and the right IFG appear to continue decreasing as age increases. In contrast connections between temporal regions are well established by late childhood. Increased activity over right auditory regions is associated with decreased language skills. Whilst data from the DLD participant is described, further analysis was not possible due to the limited sample size (n=1). Hyperscanning fNIRS study: We successfully recorded inter-brain synchrony in bilateral prefrontal and temporal cortices in mother-child dyads while they engaged in cooperative and independent play. Compared to the independent condition, mother-child dyads showed increased neural synchrony in the interactive condition across the prefrontal cortex and temporo-parietal junction. There was no significant relationship found between neural synchrony and turn-taking, but neural synchrony was negatively correlated with the child’s levels of surgency. Survey study: Clinicians and parents perceived that a potential tool that could diagnose children with DLD earlier would positively impact the children as it would allow them to access interventions earlier. This study offered a unique account of the factors to be considered in the design and implementation of clinical measures for language disorders from the viewpoints of parents and language professionals. Conclusions: Overall, this research aimed to identify neural markers of language processing in children with DLD and typically developed children to help develop an objective early diagnostic tool. Ultimately, this research might help maximize the benefits of speech and language therapies to improve the quality of life for children with DLD. This can be very impactful translational research in language development given that currently no objective neural-based tools exist for DLD

Brain-to-brain communication during musical improvisation: a performance case study [version 4; peer review: 1 approved, 2 approved with reservations]

Understanding and predicting others' actions in ecological settings is an important research goal in social neuroscience. Here, we deployed a mobile brain-body imaging (MoBI) methodology to analyze inter-brain communication between professional musicians during a live jazz performance. Specifically, bispectral analysis was conducted to assess the synchronization of scalp electroencephalographic (EEG) signals from three expert musicians during a three-part 45 minute jazz performance, during which a new musician joined every five minutes. The bispectrum was estimated for all musician dyads, electrode combinations, and five frequency bands. The results showed higher bispectrum in the beta and gamma frequency bands (13-50 Hz) when more musicians performed together, and when they played a musical phrase synchronously. Positive bispectrum amplitude changes were found approximately three seconds prior to the identified synchronized performance events suggesting preparatory cortical activity predictive of concerted behavioral action. Moreover, a higher amount of synchronized EEG activity, across electrode regions, was observed as more musicians performed, with inter-brain synchronization between the temporal, parietal, and occipital regions the most frequent. Increased synchrony between the musicians' brain activity reflects shared multi-sensory processing and movement intention in a musical improvisation task

Exploring neural markers of language processing using fNIRS in typically developed children and children with Developmental Language Disorder

Background: Developmental language disorder (DLD) is a life-long condition with no clear biological causes that affects approximately 8% of the population. The diagnosis currently relies on behavioural testing that is not reliably performed on children younger than school age. Consequently, the diagnosis and treatment of DLD is often delayed until after children enter formal education. Early work in the field suggests that neural markers of language processing could be used to develop an objective diagnostic tool that will allow for accurate and early identification of DLD in preschool years and thus access to early interventions. Here we propose the use of a novel non-invasive neuroimaging technique called functional near infrared spectroscopy (fNIRS) to identify neural markers of language processing in children with DLD. Additionally, we argue that to understand atypical language processing, it is imperative to also investigate typical cortical activations in response to language processing to establish the developmental trajectories of the language network. Parallel to these studies we also investigate patterns of neural synchrony during parent-child interactions. Speech and language development in children is thought to rely on successful parent-child interactions, however, little is known regarding the underlying neural mechanisms from which they arise. Methods: Cross-Sectional fNIRS Studies: A total of 36 participants aged 6–16-year-old (1 participant with DLD) were recruited in two cross-sectional fNIRS studies. Participants underwent a 10-minute resting state imaging session and completed a series of computer-administered language and cognitive tasks while their brain activity was recorded using fNIRS from the bilateral inferior frontal gyrus (IFG) and the bilateral auditory cortices. Hyperscanning fNIRS Study: 12 children aged between 3 and 5 years old and their mothers participated in this study. Neural synchrony in mother-child dyads was measured bilaterally over frontal and temporal areas using fNIRS whilst the dyads were asked to play together (interactive condition) and separately (independent condition). Communication patterns were captured via video recordings and conversational turns were coded. Survey Study: 43 parents of children with DLD and 44 clinicians with DLD expertise completed a qualitative online survey detailing their views, concerns and recommendations regarding the use of neuroimaging-based tool for the diagnosis and monitoring of DLD. Results: Cross sectional fNIRS studies: In typically developed children and adolescents, widespread connections between the language regions and the right IFG appear to continue decreasing as age increases. In contrast connections between temporal regions are well established by late childhood. Increased activity over right auditory regions is associated with decreased language skills. Whilst data from the DLD participant is described, further analysis was not possible due to the limited sample size (n=1). Hyperscanning fNIRS study: We successfully recorded inter-brain synchrony in bilateral prefrontal and temporal cortices in mother-child dyads while they engaged in cooperative and independent play. Compared to the independent condition, mother-child dyads showed increased neural synchrony in the interactive condition across the prefrontal cortex and temporo-parietal junction. There was no significant relationship found between neural synchrony and turn-taking, but neural synchrony was negatively correlated with the child’s levels of surgency. Survey study: Clinicians and parents perceived that a potential tool that could diagnose children with DLD earlier would positively impact the children as it would allow them to access interventions earlier. This study offered a unique account of the factors to be considered in the design and implementation of clinical measures for language disorders from the viewpoints of parents and language professionals. Conclusions: Overall, this research aimed to identify neural markers of language processing in children with DLD and typically developed children to help develop an objective early diagnostic tool. Ultimately, this research might help maximize the benefits of speech and language therapies to improve the quality of life for children with DLD. This can be very impactful translational research in language development given that currently no objective neural-based tools exist for DLD

Interactive brains:How infant cognition interacts with the dynamic social world

Research taking a cognitive neuroscience approach has shed light on social cognition during infancy. These studies have provided invaluable knowledge about how infants process social information, but a number of concepts regarding infant social cognition are often discussed based on research utilising rigidly controlled experimental paradigms where the role of infants is typically passive as an observer of stimuli. Increasing evidence suggests differences between the social cognitive processes that occurs when we act as observers of others (a ‘third-person’ perspective) and the processes that emerge when we are actively engaging with other people in an interactional context (a ‘second-person’ perspective) (e.g., Redcay and Schilbach, 2019; Siposova & Carpenter, 2019). Accordingly, there has been a growing recognition that we need a ‘second-person’ perspective, as compared to conventional “third-person” approach. The aim of the current thesis is to explore the interplay between infant cognition and the social world surrounding them, by moving research settings to a more naturalistic and dynamic one where infants are positioned as part of interaction. Towards this goal, Study 1 (Chapter 2) reviewed the current progress of “second-person” neuroscience research to evaluate the validity and robustness of simultaneous dual brain scanning techniques, often referred to as hyperscanning. The review identified large heterogeneity in reported effect sizes between published studies, suggesting the need to improve comparability of research, such as establishing standardised methods or promoting open science practices including code and data sharing to achieve higher reproducibility. This thesis then turned to research using various techniques from a conventional screen-based paradigm to a more dynamic setting, with the aim of building a stable platform towards second-person cognitive neuroscience approaches that investigate infant cognition while the infant actively interacts with other people. Study 2 (Chapter 3) explored how infants encode information differently from two adults who give gaze cues to a target object with different levels of accuracy. Whilst the study utilised a conventional event-related potential paradigm using screen-based stimuli, this paradigm could be adapted to enable future studies to investigate how infants’ social cognitive ability to discriminate reliable and unreliable informants can inform their subsequent behaviour observed in a social interactional behavioural task. Study 3 (Chapter 4) moved towards the use of more dynamic video stimuli and explored the neural processing of unexpected events. The study identified challenges in using dynamic perceptual inputs as stimuli. Study 4 (Chapter 5) transitioned into more naturalistic social contexts and analysed infant cognition while 10-month-old infants were faced with an adult demonstrating novel object labels in a live interaction. The study not only showed the feasibility of second-person neuroscientific research with infant participants, but also advanced our knowledge about infant word learning a step further, and demonstrated the trajectory from the encoding of semantic word information to its consolidation as knowledge. Study 5 (Chapter 6) also utilised a naturalistic interactional setting where infants were able to actively engage in a social task with an experimenter in a live manner, and aimed to identify systematic differences in neural activity between 9-month-old infants who make perseverative errors originally reported by Piaget (1954) and those who do not. This study was, to our knowledge, the first of its kind to validate the feasibility of utilising neurophysiological measures in this traditional interactive behavioural paradigm, in such a way that it does not interfere with the standard procedure. This thesis produced a series of studies which jointly demonstrate the potential for conducting research in a more dynamic setting that investigates infant social cognition taking a ‘second-person’ cognitive neuroscience approach to advance our knowledge about the intricate interaction between infant cognition, behaviour and the environment. We conclude this thesis by addressing the challenges of such an approach, to which we also attempt to propose solutions, as well as discussing future directions for the field

A functional neuro-anatomical model of human attachment (NAMA): Insights from first- and second-person social neuroscience

Attachment theory, developed by Mary Ainsworth and John Bowlby about seventy years ago, has become one of the most influential and comprehensive contemporary psychology theories. It predicts that early social interactions with significant others shape the emergence of distinct self- and other-representations, the latter affecting how we initiate and maintain social relationships across the lifespan. A person's attachment history will therefore associate with inter-individual differences in emotional and cognitive mechanisms sustaining representations, modeling, and understanding of others on the biological and brain level. This review aims at summarizing the currently available social neuroscience data in healthy participants on how inter-individual differences in attachment associate with brain anatomy and activity across the lifespan, and to integrate these data into an extended and refined functional neuro-anatomical model of human attachment (NAMA). We first propose a new prototypical initial attachment pathway and its derivatives as a function of attachment security, avoidance, and anxiety. Based on these pathways, we suggest a neural attachment system composed of two emotional mentalization modules (aversion and approach) and two cognitive mentalization modules (emotion regulation and mental state representation) and provide evidence on their functionality depending on inter-individual differences in attachment. We subsequently expand this first-person social neuroscience account by also considering a second-person social neuroscience perspective comprising the concepts of bio-behavioral synchrony and particularly inter-brain coherence. We hope that such extended and refined NAMA can inform attachment theory and ultimately help devising new prevention and intervention strategies for individuals and families at risk for attachment-related psychopathology

Neural Synchrony In Successful Communication

Communicating our experiences to others relies on complex shared social, cultural, and psychological mechanisms. Research increasingly shows that shared neural mechanisms also play a role in the success of interpersonal communication. Synchronous activity in shared or complementary regions of the brain promotes emotional connections, cooperation, and memory between communicators. Regions of the brain involved in social and self-relevant information processes – (1) mentalizing, or thinking about the thoughts of others, and (2) self-relevance, or prospecting about the importance of information to the self – show synchrony in ways that correlate with communication outcomes. Synchrony can occur between two individuals, like speakers and their listeners, but it can also occur among a group of listeners, the audience. We use a form of neuroimaging called functional near-infrared spectroscopy to study neural activity as people tell and hear stories. First, we measure synchrony between storytellers and listeners. Chapter 2 shows that synchrony in mentalizing brain regions between a storyteller and her listeners predicts effective communication of emotional states. Next, we consider how synchrony across larger groups of audience members relates to successful communication. Chapter 3 demonstrates that an individual listener\u27s similarity to the average brain response in other audience members, in self-relevance processing regions, predicts the listener\u27s ability to authentically re-tell a story. Finally, extending this work, we also examine whether shared preferences predict neural synchrony in audience members. Chapter 4 integrates information about audience members’ individual preferences for content with audience-level neural synchrony. Within audiences of sports fans and theater lovers, self-reported content preferences predict behavioral liking for entertainment, but neural synchrony does not predict similar preferences in this case. Together these studies explore how synchrony between individuals predicts understanding and ability to transmit stories