Cognitive, neural, and social mechanisms of rhythmic interpersonal coordination

Humans possess the exceptional capacity to temporally coordinate their movements with one another with a high degree of accuracy, precision, and flexibility. Musical ensemble performance is a refined example of this, where a range of cognitive and sensory-motor processes work together to support rhythmic interpersonal coordination. However, the influence of social factors on the underlying cognitive-motor and neural mechanisms that facilitate rhythmic interpersonal coordination is yet to be established. This thesis draws on theoretical perspectives related to joint action, including co-representation, self-other integration and segregation, and theoretical models of sensorimotor synchronisation to consider this topic. Three experiments were conducted to investigate how social factors influence rhythmic interpersonal coordination. This broad empirical question was broken down by considering both extrinsic factors—such as the social context and perceived characteristics of an interaction partner (e.g. the degree of partner intentionality and responsiveness)—as well as intrinsic social factors, such as individual differences in attitudes and social preferences. This thesis concludes that extrinsic and intrinsic social factors affect rhythmic interpersonal coordination at multiple levels. A key aspect of this influence relates to how people regulate the integration and segregation of their representations of self and others. However, importantly, these effects are mediated by individual differences in intrinsic social factors such as personal preferences and biases. Top-down processes related to beliefs thus influence bottom-up sensorimotor processes during joint action, but the nature of this influence appears to be different for different people. This outcome highlights the necessity of taking individual differences into account, particularly when investigating the nuances of social processing during dynamic social interactions. Furthermore, the current findings suggest that beliefs about a partner during social interaction may be just as, or even more so, influential on performance than the actual characteristics of the partner. Recognising the potency of social beliefs has implications not only for research into basic psychological mechanisms underpinning rhythmic interpersonal coordination, but also for understanding the broader social dynamics of real-life situations involving cooperative joint action understanding the broader social dynamics of real-life situations involving cooperative joint action

Explorations in engagement for humans and robots

This paper explores the concept of engagement, the process by which individuals in an interaction start, maintain and end their perceived connection to one another. The paper reports on one aspect of engagement among human interactors--the effect of tracking faces during an interaction. It also describes the architecture of a robot that can participate in conversational, collaborative interactions with engagement gestures. Finally, the paper reports on findings of experiments with human participants who interacted with a robot when it either performed or did not perform engagement gestures. Results of the human-robot studies indicate that people become engaged with robots: they direct their attention to the robot more often in interactions where engagement gestures are present, and they find interactions more appropriate when engagement gestures are present than when they are not.Comment: 31 pages, 5 figures, 3 table

Temporal structure of neural oscillations underlying sensorimotor coordination: a theoretical approach with evolutionary robotics

The temporal structure of neural oscillations has become a widespread hypothetical \mechanism" to explain how neurodynamics give rise to neural functions. Despite the great number of empirical experiments in neuroscience and mathematical and computa- tional modelling investigating the temporal structure of the oscillations, there are still few systematic studies proposing dynamical explanations of how it operates within closed sensorimotor loops of agents performing minimally cognitive behaviours. In this thesis we explore this problem by developing and analysing theoretical models of evolutionary robotics controlled by oscillatory networks. The results obtained suggest that: i) the in- formational content in an oscillatory network about the sensorimotor dynamics is equally distributed throughout the entire range of phase relations; neither synchronous nor desyn- chronous oscillations carries a privileged status in terms of informational content in relation to an agent's sensorimotor activity; ii) although the phase relations of oscillations with a narrow frequency difference carry a relatively higher causal relevance than the rest of the phase relations to sensorimotor coordinations, overall there is no privileged functional causal contribution to either synchronous or desynchronous oscillations; and iii) oscilla- tory regimes underlying functional behaviours (e.g. phototaxis, categorical perception) are generated and sustained by the agent's sensorimotor loop dynamics, they depend not only on the dynamic structure of a sensory input but also on the coordinated coupling of the agent's motor-sensory dynamics. This thesis also contributes to the Coordination Dynam- ics framework (Kelso, 1995) by analysing the dynamics of the HKB (Haken-Kelso-Bunz) equation within a closed sensorimotor loop and by discussing the theoretical implications of such an analysis. Besides, it contributes to the ongoing philosophical debate about whether actions are either causally relevant or a constituent of cognitive functionalities by bringing this debate to the context of oscillatory neurodynamics and by illustrating the constitutive notion of actions to cognition

Self-organized criticality and stochastic resonance in the human brain

The human brain spontaneously generates neuronal network oscillations at around 10 and 20 Hz with a large variability in amplitude, duration, and recurrence. Despite more than 70 years of research, the complex dynamics and functional significance of these oscillations have remained poorly understood. This Thesis concerns the dynamic character and functional significance of noninvasively recorded 10- and 20-Hz oscillations in the human brain. The hypotheses, experimental paradigms, data analyses, and interpretations of the results are inspired by recent insights from physics - most notable the theory of self-organized criticality and the phenomenon of stochastic resonance whose applicability to large-scale neuronal networks is explained. We show that amplitude fluctuations of 10- and 20-Hz oscillations during wakeful rest are correlated over thousands of oscillation cycles and that the decay of temporal correlations exhibits power-law scaling behavior. However, when these ongoing oscillations are perturbed with sensory stimuli, the amplitude attenuates quickly, reliably, and transiently, and the long-range temporal dynamics is affected as evidenced by changes in scaling exponents compared to rest. In addition to the rich temporal dynamics in local areas of the cortex, ongoing oscillations tend to synchronize their phases and exhibit correlated amplitude fluctuations across the two hemispheres, as shown for oscillations in homologous areas of the sensorimotor cortices. Finally, it is revealed that intermediate amplitude levels of ongoing oscillations provide the optimal oscillatory state of the sensorimotor cortex for reliable and quick conscious detection of weak somatosensory stimuli. We propose that the long-range temporal correlations, the power-law scaling behavior, the high susceptibility to stimulus perturbations, and the large amplitude variability of ongoing oscillations may find a unifying explanation within the theory of self-organized criticality. This theory offers a general mechanism for the ubiquitous emergence of complex dynamics with power-law decay of spatiotemporal correlations in non-linear self-organizing stochastic systems consisting of many units. The optimal ability to detect consciously and respond behaviorally to weak somatosensory stimuli at intermediate levels of ongoing sensorimotor oscillations is attributed to stochastic resonance - the intuitively paradoxical phenomenon that the signal-to-noise ratio of detecting or transmitting a signal in a non-linear system can be enhanced by noise. Based on the above results, we conjecture that a mechanism of intrinsic stochastic resonance between self-organized critical and stimulus-induced activities may be a general organizing principle of great importance for central nervous system function and account for some of the variability in the way we perceive and react to the outside world.reviewe

Emergent coordination between humans and robots

Emergent coordination or movement synchronization is an often observed phenomenon in human behavior. Humans synchronize their gait when walking next to each other, they synchronize their postural sway when standing closely, and they also synchronize their movement behavior in many other situations of daily life. Why humans are doing this is an important question of ongoing research in many disciplines: apparently movement synchronization plays a role in children’s development and learning; it is related to our social and emotional behavior in interaction with others; it is an underlying principle in the organization of communication by means of language and gesture; and finally, models explaining movement synchronization between two individuals can also be extended to group behavior. Overall, one can say that movement synchronization is an important principle of human interaction behavior. Besides interacting with other humans, in recent years humans do more and more interact with technology. This was first expressed in the interaction with machines in industrial settings, was taken further to human-computer interaction and is now facing a new challenge: the interaction with active and autonomous machines, the interaction with robots. If the vision of today’s robot developers comes true, in the near future robots will be fully integrated not only in our workplace, but also in our private lives. They are supposed to support humans in activities of daily living and even care for them. These circumstances however require the development of interactional principles which the robot can apply to the direct interaction with humans. In this dissertation the problem of robots entering the human society will be outlined and the need for the exploration of human interaction principles that are transferable to human-robot interaction will be emphasized. Furthermore, an overview on human movement synchronization as a very important phenomenon in human interaction will be given, ranging from neural correlates to social behavior. The argument of this dissertation is that human movement synchronization is a simple but striking human interaction principle that can be applied in human-robot interaction to support human activity of daily living, demonstrated on the example of pick-and-place tasks. This argument is based on five publications. In the first publication, human movement synchronization is explored in goal-directed tasks which bare similar requirements as pick-and-place tasks in activities of daily living. In order to explore if a merely repetitive action of the robot is sufficient to encourage human movement synchronization, the second publication reports a human-robot interaction study in which a human interacts with a non-adaptive robot. Here however, movement synchronization between human and robot does not emerge, which underlines the need for adaptive mechanisms. Therefore, in the third publication, human adaptive behavior in goal-directed movement synchronization is explored. In order to make the findings from the previous studies applicable to human-robot interaction, in the fourth publication the development of an interaction model based on dynamical systems theory is outlined which is ready for implementation on a robotic platform. Following this, a brief overview on a first human-robot interaction study based on the developed interaction model is provided. The last publication describes an extension of the previous approach which also includes the human tendency to make use of events to adapt their movements to. Here, also a first human-robot interaction study is reported which confirms the applicability of the model. The dissertation concludes with a discussion on the presented findings in the light of human-robot interaction and psychological aspects of joint action research as well as the problem of mutual adaptation.Spontan auftretende Koordination oder Bewegungssynchronisierung ist ein häufig zu beobachtendes Phänomen im Verhalten von Menschen. Menschen synchronisieren ihre Schritte beim nebeneinander hergehen, sie synchronisieren die Schwingbewegung zum Ausgleich der Körperbalance wenn sie nahe beieinander stehen und sie synchronisieren ihr Bewegungsverhalten generell in vielen weiteren Handlungen des täglichen Lebens. Die Frage nach dem warum ist eine Frage mit der sich die Forschung in der Psychologie, Neuro- und Bewegungswissenschaft aber auch in der Sozialwissenschaft nach wie vor beschäftigt: offenbar spielt die Bewegungssynchronisierung eine Rolle in der kindlichen Entwicklung und beim Erlernen von Fähigkeiten und Verhaltensmustern; sie steht in direktem Bezug zu unserem sozialen Verhalten und unserer emotionalen Wahrnehmung in der Interaktion mit Anderen; sie ist ein grundlegendes Prinzip in der Organisation von Kommunikation durch Sprache oder Gesten; außerdem können Modelle, die Bewegungssynchronisierung zwischen zwei Individuen erklären, auch auf das Verhalten innerhalb von Gruppen ausgedehnt werden. Insgesamt kann man also sagen, dass Bewegungssynchronisierung ein wichtiges Prinzip im menschlichen Interaktionsverhalten darstellt. Neben der Interaktion mit anderen Menschen interagieren wir in den letzten Jahren auch zunehmend mit der uns umgebenden Technik. Hier fand zunächst die Interaktion mit Maschinen im industriellen Umfeld Beachtung, später die Mensch-Computer-Interaktion. Seit kurzem sind wir jedoch mit einer neuen Herausforderung konfrontiert: der Interaktion mit aktiven und autonomen Maschinen, Maschinen die sich bewegen und aktiv mit Menschen interagieren, mit Robotern. Sollte die Vision der heutigen Roboterentwickler Wirklichkeit werde, so werden Roboter in der nahen Zukunft nicht nur voll in unser Arbeitsumfeld integriert sein, sondern auch in unser privates Leben. Roboter sollen den Menschen in ihren täglichen Aktivitäten unterstützen und sich sogar um sie kümmern. Diese Umstände erfordern die Entwicklung von neuen Interaktionsprinzipien, welche Roboter in der direkten Koordination mit dem Menschen anwenden können. In dieser Dissertation wird zunächst das Problem umrissen, welches sich daraus ergibt, dass Roboter zunehmend Einzug in die menschliche Gesellschaft finden. Außerdem wird die Notwendigkeit der Untersuchung menschlicher Interaktionsprinzipien, die auf die Mensch-Roboter-Interaktion transferierbar sind, hervorgehoben. Die Argumentation der Dissertation ist, dass die menschliche Bewegungssynchronisierung ein einfaches aber bemerkenswertes menschliches Interaktionsprinzip ist, welches in der Mensch-Roboter-Interaktion angewendet werden kann um menschliche Aktivitäten des täglichen Lebens, z.B. Aufnahme-und-Ablege-Aufgaben (pick-and-place tasks), zu unterstützen. Diese Argumentation wird auf fünf Publikationen gestützt. In der ersten Publikation wird die menschliche Bewegungssynchronisierung in einer zielgerichteten Aufgabe untersucht, welche die gleichen Anforderungen erfüllt wie die Aufnahme- und Ablageaufgaben des täglichen Lebens. Um zu untersuchen ob eine rein repetitive Bewegung des Roboters ausreichend ist um den Menschen zur Etablierung von Bewegungssynchronisierung zu ermutigen, wird in der zweiten Publikation eine Mensch-Roboter-Interaktionsstudie vorgestellt in welcher ein Mensch mit einem nicht-adaptiven Roboter interagiert. In dieser Studie wird jedoch keine Bewegungssynchronisierung zwischen Mensch und Roboter etabliert, was die Notwendigkeit von adaptiven Mechanismen unterstreicht. Daher wird in der dritten Publikation menschliches Adaptationsverhalten in der Bewegungssynchronisierung in zielgerichteten Aufgaben untersucht. Um die so gefundenen Mechanismen für die Mensch-Roboter Interaktion nutzbar zu machen, wird in der vierten Publikation die Entwicklung eines Interaktionsmodells basierend auf Dynamischer Systemtheorie behandelt. Dieses Modell kann direkt in eine Roboterplattform implementiert werden. Anschließend wird kurz auf eine erste Studie zur Mensch- Roboter Interaktion basierend auf dem entwickelten Modell eingegangen. Die letzte Publikation beschreibt eine Weiterentwicklung des bisherigen Vorgehens welche der Tendenz im menschlichen Verhalten Rechnung trägt, die Bewegungen an Ereignissen auszurichten. Hier wird außerdem eine erste Mensch-Roboter- Interaktionsstudie vorgestellt, die die Anwendbarkeit des Modells bestätigt. Die Dissertation wird mit einer Diskussion der präsentierten Ergebnisse im Kontext der Mensch-Roboter-Interaktion und psychologischer Aspekte der Interaktionsforschung sowie der Problematik von beiderseitiger Adaptivität abgeschlossen

New concepts in tele-autonomous systems

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/76226/1/AIAA-1987-1686-200.pd

Neurophysiological correlates underlying social behavioural adjustment of conformity

[eng] Conformity is the act of changing one’s behaviour to adjust to other human beings. It is a crucial social adaptation that happens when people cooperate, where one sacrifices their own perception, expectations, or beliefs to reach convergence with another person. The aim of the present study was to increase the understanding of the neurophysiological underpinnings regarding the social behavioural adjustment of conformity. We start by introducing cooperation and how it is ingrained in human behaviour. Then we explore the different processes that the brain requires for the social behavioural adjustment of conformity. To engage in this social adaptation, a person needs a self-referenced learning mechanism based on a predictive model that helps them track the prediction errors from unexpected events. Also, the brain uses its monitoring and control systems to encode different value functions used in action selection. The use of different learning models in neuroscience, such as reinforcement learning (RL) algorithms, has been a success story identifying learning systems by means of the mapped activity of different regions in the brain. Importantly, experimental paradigms which has been used to study conformity have not been based in a social interaction setting and, hence, the results, cannot be used to explain an inherently social phenomenon. The main goal of the present thesis is to study the neurophysiological mechanisms underlying the social behavioural adjustment of conformity and its modulation with repeated interaction. To reach this goal, we have first designed a new experimental task where conformity appears spontaneously between two persons and in a reiterative way. This design exposes learning acquisition processes, which require iterative loops, as well as other cognitive control mechanisms such as feedback processing, value-based decision making and attention. The first study shows that people who previously cooperate increase their level of convergence and report a significantly more satisfying overall experience. In addition, participants learning on their counterparts’ behaviour can be explained using a RL algorithm as opposed to when they do not have previously cooperated. In the second study, we have studied the event-related potentials (ERP) and oscillatory power underlying conformity. ERP results show different levels of cognitive engagement that are associated to distinct levels of conformity. Also, time-frequency analysis shows evidence in theta, alpha and beta related to different functions such as cognitive control, attention and, also, reward processing, supporting the idea that convergence between dyads acts as a social reward. Finally, in the third study, we explored the intra- and inter- oscillatory connectivity between electrodes related to behavioural convergence. In intra-brain oscillatory connectivity coherence, we have found two different dynamics related to attention and executive functions in alpha. Also, we have found that the learning about peer’s behaviour as computed using a RL is mediated by theta oscillatory connectivity. Consequently, combined evidence from Study 2 and Study 3 suggests that both cognitive control and learning computations happening in the social behavioural adaptation of conformity are signalled in theta frequency band. The present work is one of the first studies describing, with credible evidence, that conformity, when this occurs willingly and spontaneously rather than induced, engages different brain activity underlying reward-guided learning, cognitive control, and attention.[spa] La conformidad es el acto de cambiar el comportamiento de uno a favor de ajustarnos a otros seres humanos. Se trata de una adaptación crucial que ocurre cuando la gente coopera, donde uno sacrifica su propia percepción, expectativas o creencias en aras de conseguir una convergencia con la otra persona. El objetivo del presente estudio ha sido tratar de aportar a la comprensión de las estructuras neurofisiológicas que soportan un ajuste social como el de la conformidad. En la primera parte de esta tesis comenzamos hablando de la cooperación y lo profundamente arraigada que está en nuestro comportamiento. Más tarde exploramos diferentes procesos que el cerebro requiere en el ajuste social de la conformidad. Así pues, para involucrarse en esta adaptación social, una persona requiere de un mecanismo de aprendizaje auto-referenciado basado en un modelo predictivo que le ayude a seguir el rastro de los errores de predicción que acompañan a los eventos inesperados. Además, el cerebro usa sus sistemas de control y predicción para codificar diferentes funciones de valor usadas en la selección de acción. El uso de diferentes modelos de aprendizaje en neurociencia, como los algoritmos de aprendizaje por refuerzo (RL), han sido una historia de éxito a la hora de identificar los sistemas de aprendizaje a través del mapeo de la actividad de diferentes regiones del cerebro. Es importante destacar que los paradigmas experimentales que se han usado para estudiar la conformidad no se han basado en entornos de interacción social y que, por lo tanto, sus resultados no pueden usarse para explicar un fenómeno inherentemente social. El objetivo principal de la presente tesis es el estudio de los mecanismos neurofisiológicos que fundamentan el comportamiento de ajuste social de la conformidad y su modulación con la interacción repetida. Para alcanzar este objetivo, primero hemos diseñado una nueva tarea experimental en la que la conformidad aparece de forma espontánea entre dos personas y, además, de forma reiterativa. Este diseño permite exponer tanto los procesos de adquisición del aprendizaje, que requieren de ciclos iterativos, así como otros mecanismos de control cognitivo tales como el procesamiento de la retroalimentación, las tomas de decisiones basadas en procesos valorativos y la atención. El primer estudio nos muestra que la gente que coopera previamente incrementa sus niveles de convergencia y reportan significativamente una experiencia generalmente más satisfactoria en el experimento. Adicionalmente, un modelo de RL nos explica que los participantes tratan de aprender del comportamiento de sus parejas en mayor medida si estos han cooperado previamente. En el segundo estudio, hemos estudiado los potenciales relacionados con eventos (ERP) y el poder de las oscilaciones que sustentan la conformidad. Los estudios de ERP muestran diferentes niveles de implicación cognitiva asociados con diferentes niveles de conformidad. Además, los análisis de tiempo-frecuencia muestran evidencia en theta, alfa y beta relacionados con diferentes funciones como el control cognitivo, la atención, y, también, el procesamiento de la recompensa, apoyando la idea de que la convergencia entre díadas actúa como una recompensa social. Finalmente, en el tercer estudio, exploramos la conectividad oscilatoria intra e inter entre electrodos que se pudieran relacionar con la conducta de convergencia. A propósito de la conectividad oscilatoria coherente intra, hemos hallado dos dinámicas relacionadas con la atención y las funciones ejecutivas en alfa. Asimismo, hemos encontrado que el aprendizaje de la conducta de la pareja computada a través de RL está mediada a través de la conectividad oscilatoria de theta. Consecuentemente, la evidencia combinada entre el estudio 2 y el estudio 3 sugiere que conjuntamente el control cognitivo y las computaciones de aprendizaje que ocurren en la conducta de adaptación social de la conformidad están relacionadas con la actividad de la banda de frecuencia theta. Este trabajo constituye uno de los primeros estudios que describen, con evidencia creíble, que la conformidad, cuando ocurre voluntaria y espontáneamente a diferencia cuando esta es inducida, involucra actividad del cerebro que se fundamenta en el aprendizaje guiado por reforzamiento, el control cognitivo y la atención