Emergence of Content-Agnostic Information Processing by a Robot Using Active Inference, Visual Attention, Working Memory, and Planning

Generalization by learning is an essential cognitive competency for humans. For example, we can manipulate even unfamiliar objects and can generate mental images before enacting a preplan. How is this possible? Our study investigated this problem by revisiting our previous study (Jung, Matsumoto, & Tani, 2019), which examined the problem of vision-based, goal-directed planning by robots performing a task of block stacking. By extending the previous study, our work introduces a large network comprising dynamically interacting submodules, including visual working memory (VWMs), a visual attention module, and an executive network. The executive network predicts motor signals, visual images, and various controls for attention, as well as masking of visual information. The most significant difference from the previous study is that our current model contains an additional VWM. The entire network is trained by using predictive coding and an optimal visuomotor plan to achieve a given goal state is inferred using active inference. Results indicate that our current model performs significantly better than that used in Jung et al. (2019), especially when manipulating blocks with unlearned colors and textures. Simulation results revealed that the observed generalization was achieved because content-agnostic information processing developed through synergistic interaction between the second VWM and other modules during the course of learning, in which memorizing image contents and transforming them are dissociated. This letter verifies this claim by conducting both qualitative and quantitative analysis of simulation results

Precision modulation in predictive coding hierarchies: theoretical, behavioural and neuroimaging investigations

Estimation of uncertainty is an important aspect of perception and a prerequisite for effective action. This thesis explores the implementation of uncertainty estimation as precision modulation within a predictive coding hierarchy, optimised within a neurbiologically-plausible message-passing scheme via the minimisation of free-energy. This thesis consists of six chapters. The first presents a new model of a classic visual illusion, the Cornsweet illusion, which demonstrates that the Cornsweet illusion is a natural consequence of Bayes-optimal perception under the free-energy principle, and demonstrates that increasing contrast can be modelled by increasing signal-to-noise ratio. The second chapter describes dynamic causal modelling of EEG data collected from participants viewing the Cornsweet illusion, demonstrating that a reduction in precision, or superficial pyramidal cell gain, in lower visual hierarchical levels, is sufficient to explain contrast-dependent changes in ERPs. The third describes a model of a simple attentional paradigm – the Posner paradigm – recasting attention as the optimal modulation of precision in sensory channels. The fourth describes an MEG study of the Posner paradigm, using Bayesian model selection to explore the role of changes in backwards and modulatory connections and changes in local superficial pyramidal cell gain in producing the electrophysiological and behavioural correlates of the Posner paradigm. The fifth chapter recasts the Posner paradigm in the motor domain to investigate the level (intrinsic vs. extrinsic) of precision modulation by motor cues. The sixth describes a new model of sensory attenuation based on using precision modulation to balance the imperatives to act and perceive. I hope to demonstrate that precision modulation within predictive coding hierarchies, under the free-energy principle, is a flexible and powerful way of describing and explaining both behavioural and neuroimaging data

Listening effort allocation, stimulus-driven, goal-driven, or both?

The research in audiology to date about how people listen has been focused too narrowly on the impact of the task demand (e.g., speech complexity) on the effort exerted for listening. Very few studies conducted on how intention associated factors affect listening effort regulation, and little is known about how to characterize the individual quality of effort expenditure in terms of efficiency. This study primarily aimed to fill the gap by testing a compensatory control model for effort regulation, specifically, to investigate how reward would modulate the effect of task demand on listening effort. The secondary aim was to propose a modified computational approach for effort efficiency calculation. The nonclinical sample was comprised of 40 college volunteer participants with normal hearing. All participants completed the Need for Cognition scale, a speech comprehension task which required a cost-benefit decision making, and a self-report strategy use survey. The pupil dilation was measured throughout the speech comprehension task as the indicator of listening effort. Results supported the model in that effort regulation during an intended activity is determined not only by stimulus-driven factors such as task demand, but also by goal-driven factors such as reward. Significant interaction effects emerged. Furthermore, the effort efficiency derived by using goal-oriented performance variables demonstrated the superiority of distinguishing individuals compared to the use of mere performance accuracy. This study contributes to the limited literature available on proactive listening effort regulation. Examining further how hearing, cognitions, and personality interact neurophysiologically and functionally in normal hearing and hearing impaired populations can help clinicians and researchers better understand the underlying mechanism of listening effort control, and facilitate implementing strategies to aid effective listening through audiologic interventions

TUTTI! - Music Composition as Dialogue

As an engineer, when I could not comprehend a physical phenomenon, I turned to mathematics. As a mathematician, when I could not link sciences to humanity, I turned to music. As a music composer, I no longer see things, I see others. The novel method of music composition presented herein is a first comprehensive framework, system and architectonic template relying on the ideologies of Mikhail Bakhtin's dialogism as well as on research in auditory perception and cognition to create music dialogue as a means of including and engaging participants in musical communication. Beyond immediate artistic intent, I strive to compose music that fosters inclusiveness and collaboration as a relational social gesture in hope that it might incite people and society to embrace their differences and collaborate with the 'others' around them. After probing aesthetics, communication studies and sociology, I argue that dialogism reveals itself well-suited to the aims of the current research. With dialogism as a guiding philosophy, the chapters then look at the relationship between music and language, perception as authorship, intertextuality, the interplay of imagination and understanding, means of arousal in music, mimesis, motion in music and rhythmic entrainment. Employing findings from Gestalt psychology, psychoacoustics, auditory scene analysis, cognition and psychology of expectation, the remaining chapters propose a cognitively informed polyphonic music composition method capable of reproducing the different constituents of dialogic communication by creating and organizing melodic, harmonic, rhythmic and structural elements. Music theory and principles of orchestration then move to music composition as examples demonstrate how dialogue scored between voice-parts provides opportunities for performers to interact with each other and, consequently, engage listeners experiencing the collaboration. As dialogue can be identified in various works, I postulate that the presented Dialogical Music Composition Method can also serve as a method of music analysis. This personal method of composition also supplies tools that other musicians can opt to employ when endeavouring to build balanced dialogue in music. If visibility is key to identity, then composing music that potentially enters into dialogue which each and every voice promotes 'humanity' through inclusivity, yielding a united Tutti

An investigation into the effects of social influence on moral behaviour using immersive virtual reality

Much of the research surrounding social influence investigates its effects in specifically non-moral situations while almost no research has looked at its effects during moral emergencies. At the same time, studies of moral psychology tend to focus on the intricacies of moral decision-making during the responses of individual participants. This thesis aims to bridge this gap between social influence and moral psychology by having participants respond to moral dilemmas while under the duress of social influence. In order to investigate the effects of social influence on moral behaviours, immersive virtual reality (IVR) was used, allowing participants to be placed in a life-like virtual simulation of events that they would normally only read about in a text-based vignette, probing their observed moral behaviours instead of just their abstract moral judgments. The benefits of using IVR include the ethical and controllable nature of questionnaires along with the verisimilitude of real-life. Another focus of this thesis is to compare moral judgments to moral behaviours. In two out of the three studies presented in this thesis, the virtual moral dilemma was replicated in a text-based questionnaire in order to compare the results from the two media. Moral judgments in response to text-based moral dilemma can miss out key contextual information such the motoric feedback of having to physically act out a movement. These factors can lead to a divergence between moral judgments and behaviours. The thesis starts with a literature review on IVR technology and moral decision-making and social influence research. After this, the three studies conducted as part of this thesis are described. The major findings from these studies include the demonstration of a preference to take action regardless of outcome only when in IVR and the inability for compliance attempts to influence specifically moral behaviour

Analyse électroencéphalographique de l’activité fréquentielle des aires sensorimotrices du néocortex associée à l’attente d’un retour visuel lors d’une tâche de pointage

Pour produire un mouvement, les aires motrices du cerveau génèrent ce qui est appelé une commande motrice, qui se rend jusqu’au système musculaire en passant par la moelle épinière. La théorie des modèles internes propose qu’une copie de cette commande motrice, la copie d’efférence, permette de prédire les conséquences sensorielles du mouvement. Une autre théorie, l’inférence active, propose que les aires motrices génèrent de telles prédictions, notamment pour l’envoi de la commande motrice qui serait plutôt une commande d’afférences sensorielles; le système réflexe au niveau de la moelle épinière se chargerait de contrôler les contractions musculaires. Cette théorie est supportée par des études qui observent une modulation de l’activité des aires motrices en fonction des afférences sensorielles, et notamment des conséquences sensorielles d’un mouvement. Ces modulations ont été observées en imagerie par résonnance magnétique fonctionnelle, en électroencéphalographie avec l’analyse de potentiels évoqués, et même comportementalement avec l’utilisation de la stimulation magnétique transcrânienne. Par contre elles n’ont pas encore été étudiées dans le domaine fréquentiel de l’électroencéphalographie. La bande de fréquence bêta (15-30 Hz) est impliquée dans la planification des mouvements au niveau des aires sensorimotrices, et la bande de fréquence thêta (3-7 Hz) aurait un rôle à jouer dans l’intégration sensorimotrice, dans l’aspect spatial de la planification d’un mouvement, et dans la prédiction d’évènements sensoriels. Nous avons testé l’hypothèse d’une modulation de ces bandes de fréquence au niveau des aires sensorimotrices, en fonction de l’anticipation d’un retour visuel associé à un mouvement de pointage. Les sujets réalisaient un mouvement de pointage vers une cible unique, alors que le retour visuel sous forme de curseur, si présent, pouvait avoir trois directions différentes contrôlées par rotation visuomotrice. Au début de chaque essai, les sujets étaient prévenus de la condition de curseur. Cela permettait d’étudier l’activité fréquentielle au niveau des aires sensorimotrices lors de la planification de mouvements similaires, alors que ces mouvements engendraient des retours visuels différents. Dans les aires sensorimotrices controlatérales, nous avons trouvé une plus grande synchronisation en thêta lorsqu’aucun retour visuel n’était présenté par rapport aux conditions où un curseur était présenté, mais aucune différence associée à la direction du curseur. Nous avons interprété ce résultat en termes d’implication de la bande de fréquence thêta dans un processus de prédiction des conséquences sensorielles des actions, ou dans un processus d’orientation de l’attention vers la modalité sensorielle d’intérêt. Nous n’avons pas trouvé de modulation en bêta dans les aires sensorimotrices controlatérales, mais nous avons trouvé une modulation dans les aires sensorimotrices ipsilatérales, avec une plus grande désynchronisation dans les conditions de rotation visuomotrice par rapport aux conditions où il n’y avait pas de rotation visuomotrice. Cela renforce une idée préétablie selon laquelle la puissance en bêta controlatéral reflète une activité préparatoire générale au mouvement, et suggère que du côté ipsilatéral cette bande de fréquence est impliquée dans des aspects plus cognitifs et/ou visuels du mouvement. Pour contrôler l’implication attentionnelle des sujets dans la tâche, nous avons étudié l’activité en alpha (8-14 Hz) pariéto-occipitale pendant la planification du mouvement. Une désynchronisation plus marquée du côté controlatéral à la direction attendue du curseur par rapport au côté ipsilatéral confirme que les sujets orientaient leur attention du côté où le retour visuel était attendu. Notre étude contribue à la compréhension des processus neurophysiologiques engagés par le cerveau pour la génération des mouvements. Elle a sa place dans une littérature qui considère que l’activité des aires sensorimotrices reflète un lien très étroit entre la perception et l’action

An adaptive environment for personal information management

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This dissertation reports the results of research into the provision of adaptive user interfaces to support individuals in the management of their personal information. Many individuals find that they have increased responsibility for managing aspects of their own lives, including the information associated with their jobs. In contrast with traditional approaches to information management, which are generally driven by organisational or business requirements, the requirements of personal information management systems tend to be less rigidly defined. This dissertation employs research from the areas of personal information management and adaptive user interfaces - systems which can monitor how they are used, and adapt on a personal level to their user - to address some of the particular requirements of personal information management systems. An adaptive user interface can be implemented using a variety of techniques, and this dissertation draws on research from the area of software agents to suggest that reactive software agents can be fruitfully applied to realise the required adaptivity. The reactive approach is then used in the specification and development of an adaptive interface which supports simple elements of personal information management tasks. The resulting application is evaluated by means of user trials and a usability inspection, and the theoretical architectures and techniques used in the specification and development of the software are critically appraised. The dissertation demonstrates an application of reactive software agents in adaptive systems design and shows how the behaviour of the system can be specified based on the analysis of some representative personal information management tasks.EPSRC (Award Reference Number 95700906

How Do I Love Thee? Adult Attachment and Reinforcement Sensitivity

This thesis aimed to examine the nature of the relations between individual differences in adult attachment patterns and the sensitivity of motivational systems – the Behavioural Approach System (BAS), the Fight-Flight-Freeze System (FFFS), and the Behavioural Inhibition System (BIS) – proposed by the revised reinforcement sensitivity theory (r-RST). In Study 1, psychology undergraduates (N=225) completed self-reported measures of adult attachment and reinforcement sensitivity. Both attachment dimensions were significantly related to BIS sensitivity, which suggests that sensitivity to motivational ambivalence is a central feature of attachment insecurity. In Study 2, psychology undergraduates (N=200) experienced virtual separation and reunion scenarios with a ‘virtual spouse,’ and subsequently completed adult attachment and reinforcement sensitivity questionnaires. Adult attachment, but not reinforcement sensitivity, were predictive of behavioural and emotional responses to separation and reunion. This suggests that adult attachment has unique predictive power to dyadic behaviour. Finally, Study 3 (N=63) examined the links between self-reported adult attachment and reinforcement sensitivity and neurobiological markers of approach and avoidance motivation (8 minutes of resting EEG). Neither adult attachment nor reinforcement sensitivity exhibited robust associations with the resting EEG indices. This may reflect the construct heterogeneity of the attachment dimensions and reinforcement sensitivity, such that they do not neatly map onto neural correlates of approach and avoidance. Together, the studies reported in this thesis suggest modest overlaps between individual differences in adult attachment and reinforcement sensitivities at the self-report level, but the two domains are largely independent in relation to attachment behaviour and neural correlates of approach-avoidance