Learning Task Priorities from Demonstrations

Bimanual operations in humanoids offer the possibility to carry out more than one manipulation task at the same time, which in turn introduces the problem of task prioritization. We address this problem from a learning from demonstration perspective, by extending the Task-Parameterized Gaussian Mixture Model (TP-GMM) to Jacobian and null space structures. The proposed approach is tested on bimanual skills but can be applied in any scenario where the prioritization between potentially conflicting tasks needs to be learned. We evaluate the proposed framework in: two different tasks with humanoids requiring the learning of priorities and a loco-manipulation scenario, showing that the approach can be exploited to learn the prioritization of multiple tasks in parallel.Comment: Accepted for publication at the IEEE Transactions on Robotic

Probabilistic prioritization of movement primitives

Movement prioritization is a common approach to combine controllers of different tasks for redundant robots, where each task is assigned a priority. The priorities of the tasks are often hand-tuned or the result of an optimization, but seldomly learned from data. This paper combines Bayesian task prioritization with probabilistic movement primitives to prioritize full motion sequences that are learned from demonstrations. Probabilistic movement primitives (ProMPs) can encode distributions of movements over full motion sequences and provide control laws to exactly follow these distributions. The probabilistic formulation allows for a natural application of Bayesian task prioritization. We extend the ProMP controllers with an additional feedback component that accounts inaccuracies in following the distribution and allows for a more robust prioritization of primitives. We demonstrate how the task priorities can be obtained from imitation learning and how different primitives can be combined to solve even unseen task-combinations. Due to the prioritization, our approach can efficiently learn a combination of tasks without requiring individual models per task combination. Further, our approach can adapt an existing primitive library by prioritizing additional controllers, for example, for implementing obstacle avoidance. Hence, the need of retraining the whole library is avoided in many cases. We evaluate our approach on reaching movements under constraints with redundant simulated planar robots and two physical robot platforms, the humanoid robot “iCub” and a KUKA LWR robot arm

Afektiivsete individuaalsete erinevuste psühholoogiliste mehhanismide uurimine EEG korrelaatidega

Väitekirja elektrooniline versioon ei sisalda publikatsiooneAfektiivsed individuaalsed erinevused on võrdlemisi stabiilsed kalduvused kogeda teatud afektiivseid seisundeid. Afektiivsete individuaalsete erinevuste psühholoogiliste mehhanismide kaardistamine võib anda väärtuslikku infot nende seostest vaimse tervisega. Käesolev väitekiri pakub varasemale kirjandusele tuginedes välja teoreetilise mudeli antud mehhanismide mõtestamiseks ja uurimiseks. Konstrukti-Protsessi-Konteksti (KPK) mudeli kohaselt avalduvad psühholoogilised mehhanismid uuritava individuaalse erinevuse konstrukti, seda iseloomustava afektiivse protsessi eripära ning viimase avaldumist mõjutavate kontekstuaalsete tegurite kokkupuutealal. Väitekirja empiirilised tööd rakendavad KPK mudelit erinevate afektiivsete individuaalsete erinevuste mehhanismide uurimiseks elektroentsefalograafia (EEG) abil. Töid ühendab keskendumine mehhanismidele, mis seisnevad lähenemis- ja vältimismotivatsiooni või motivatsioonilise tähelepanu eripärades. Uuringud I ja III täpsustavad antud protsesside hindamise metodoloogilisi aspekte. Uuringu II tulemused osutavad sellele, et kõrge Neurootilisusega (negatiivse afektiivsuse ja psüühikahäirete riskiga seotud isiksuseomadus) inimesi iseloomustab vältimismotivatsiooni kogemine vastusena pilkkontaktiga mudeldatud neutraalsele sotsiaalsele tähelepanule. Uuring IV demonstreerib, kuidas hõivatus kehakaalust ja välimusest (oluline söömishäirete riskitegur) on seotud ülemäärase tähelepanuga enda kehasuuruse muutustele ja puuduliku tähelepanuga kellegi teise kehasuuruse muutustele. Uuring V näitab, et teadvelolekut (arendatav kalduvus käesoleva kogemuse hinnanguvabaks teadvustamiseks, mis on seotud madalama emotsionaalse reaktiivsuse ja parema vaimse tervisega) iseloomustab kiirem negatiivse informatsiooni eelistöötluse vähenemine ehk tõhusam afektiivne adaptatsioon. Väitekirja uuringutest järeldub, et KPK mudeli rakendamine võib aidata tuvastada väga erinevate afektiivsete individuaalsete erinevuste võimalikke psühholoogilisi mehhanisme.Affective individual differences are relatively stable tendencies to experience certain patterns of affective states. Clarifying the psychological mechanisms of affective individual differences can offer valuable insights into mental health and illness. Building on previous work, the current dissertation proposed the Construct-Process-Context (CPC) framework as a conceptual tool for studying these mechanisms. The CPC framework suggests that affective individual differences involve biases in specific affective processes that arise in specific contexts. The empirical studies of the dissertation applied the CPC framework to electroencephalography (EEG) based investigation of the mechanisms of affective individual differences that involve either approach-avoidance motivation or motivated attention. Studies I and III clarified open methodological questions to inform the design and interpretation of individual difference studies. Study II demonstrated that high levels of Neuroticism, a personality dimension that is characterized by negative affectivity and elevated risk for psychopathology, may be related to the activation of avoidance tendencies in response to another person’s directed attention (i.e., eye contact) during a neutral social encounter. Study IV demonstrated that high preoccupation with body image, which is a risk factor for eating disorders, may involve attentional over-prioritization of own body size and under-prioritization of peer body size. Finally, Study V demonstrated that cultivated mindfulness, a trainable tendency to be aware of and nonjudgmental toward one’s experiences that has been related to reduced affective reactivity and better mental health, is characterized by improved affective adaptation as indicated by initial increase and subsequent decrease in the salience of negative stimuli. It was concluded that the CPC framework may be helpful for clarifying the psychological mechanisms of very different affective individual difference constructs

Theta synchronization over occipito‐temporal cortices during visual perception of body parts

Categorical clustering in the visual system is thought to have evolved as a function of intrinsic (intra-areal) and extrinsic (interareal) connectivity and experience. In the visual system, the extrastriate body area (EBA), an occipito-temporal region, responds to full body and body part images under the organizational principle of their functional/semantic meaning. Although frequency-specific modulations of neural activity associated with perceptive and cognitive functions are increasingly attracting the interest of neurophysiologists and cognitive neuroscientists, perceiving single body parts with different functional meaning and full body images induces time-frequency modulations over occipito-temporal electrodes are yet to be described. Here, we studied this issue by measuring EEG in participants who passively observed fingers, hands, arms and faceless full body images with four control plant stimuli, each bearing hierarchical analogy with the body stimuli. We confirmed that occipito-temporal electrodes (compatible with the location of EBA) show a larger event-related potential (ERP, N190) for body-related images. Furthermore, we identified a body part-specific (i.e. selective for hands and arms) theta event-related synchronization increase under the same electrodes. This frequency modulation associated with the perception of body effectors over occipito-temporal cortices is in line with recent findings of categorical organization of neural responses to human effectors in the visual system

Learning soft task priorities for control of redundant robots

Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is precoded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset

The upside-down self: One's own face recognition is affected by inversion

One's own face is recognized more efficiently than any other face, although the neural mechanisms underlying this phenomenon remain poorly understood. Considering the extensive visual experience that we have with our own face, some authors have proposed that self-face recognition involves a more analytical perceptual strategy (i.e., based on face features) than other familiar faces, which are commonly processed holistically (i.e., as a whole). However, this hypothesis has not yet been tested with brain activity data. In the present study, we employed an inversion paradigm combined with event-related potential (ERP) recordings to investigate whether the self-face is processed more analytically. Sixteen healthy participants were asked to identify their own face and a familiar face regardless of its orientation, which could either be upright or inverted. ERP analysis revealed an enhanced amplitude and a delayed latency for the N170 component when faces were presented in an inverted orientation. Critically, both the self and a familiar face were equally vulnerable to the inversion effect, suggesting that the self-face is not processed more analytically than a familiar face. In addition, we replicated the recent finding that the attention-related P200 component is a specific neural index of self-face recognition. Overall, our results suggest that the advantage for self-face processing might be better explained by the engagement of self-related attentional mechanisms than by the use of a more analytical visuoperceptual strategyThis work was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) (UAMA13-4E- 2192) and FEDER/Ministry of Science, Innovation and Universities (MCIU)—National Research Agency (AEI) (PGC2018-100682- B- I00), and the Community of Madrid (SAPIENTIA-CM H2019/HUM-570), under agreement with the Autonomous University of Madrid (2017-T2/ SOC-5569; SI1-PJI- 2019- 00011

Task Feasibility Maximization using Model-Free Policy Search and Model-Based Whole-Body Control

Producing feasible motions for highly redundant robots, such as humanoids, is a complicated and high-dimensional problem.Model-based whole-body control of such robots, can generate complex dynamic behaviors through the simultaneous execution of multiple tasks.Unfortunately, tasks are generally planned without close consideration for the underlying controller being used, or the other tasks being executed, and are often infeasible when executed on the robot. Consequently, there is no guarantee that the motion will be accomplished.In this work, we develop an optimization loop which automatically improves task feasibility using model-free policy search in conjunction with model-based whole-body control.This combination allows problems to be solved, which would be otherwise intractable using simply one or the other.Through experiments on both the simulated and real iCub humanoid robot, we show that by optimizing task feasibility, initially infeasible complex dynamic motions can be realized --- specifically, a sit-to-stand transition

The Impact of Motivation on Object-Based Visual Attention Indexed by Continuous Flash Suppression.

Motivationally-relevant stimuli summon our attention and benefit from enhanced processing, but the neural mechanisms underlying this prioritization are not well understood. Using an interocular suppression technique and functional neuroimaging, this work has the ultimate aim of understanding how motivation impacts visual perception. In Chapter 2a, we demonstrate that novel objects with a more rich reward history are prioritized in awareness more quickly than objects with a lean reward history. In Chapter 2b, we show that faces are prioritized in awareness following social rejection, and that the amount faces are prioritized correlates with individual differences in social motivation. Chapters 3 & 4 use a combination of functional neuroimaging and flash suppression to suppress fearful faces and houses from awareness. Using binocular rivalry and motion flash suppression in Chapter 3, we find that suppressed fearful faces activate the amygdala relative to suppressed houses, and the amygdala increases coherence with a network of regions involved in attention, including bilateral pulvinar, bilateral insula, left frontal eye fields, left inferior parietal cortex, and early visual cortex. Using the more robust technique, continuous flash suppression, in Chapter 4, we find no differentiation between stimuli based on mean amygdala responses. However, we show increased connectivity between the amygdala, the pulvinar, and inferior parietal cortex specific to fearful faces. Overall, these results indicate that motivationally-relevant stimuli activate the amygdala prior to awareness. Enhanced connectivity between the amygdala and regions involved in attention may underlie the enhanced processing seen for salient stimuli

Effects of aging on the relationship between cognitive demand and step variability during dual-task walking

A U-shaped relationship between cognitive demand and gait control may exist in dual-task situations, reflecting opposing effects of external focus of attention and attentional resource competition. The purpose of the study was twofold: to examine whether gait control, as evaluated from step-to-step variability, is related to cognitive task difficulty in a U-shaped manner and to determine whether age modifies this relationship. Young and older adults walked on a treadmill without attentional requirement and while performing a dichotic listening task under three attention conditions: non-forced (NF), forced-right (FR), and forced-left (FL). The conditions increased in their attentional demand and requirement for inhibitory control. Gait control was evaluated by the variability of step parameters related to balance control (step width) and rhythmic stepping pattern (step length and step time). A U-shaped relationship was found for step width variability in both young and older adults and for step time variability in older adults only. Cognitive performance during dual tasking was maintained in both young and older adults. The U-shaped relationship, which presumably results from a trade-off between an external focus of attention and competition for attentional resources, implies that higher-level cognitive processes are involved in walking in young and older adults. Specifically, while these processes are initially involved only in the control of (lateral) balance during gait, they become necessary for the control of (fore-aft) rhythmic stepping pattern in older adults, suggesting that attentional resources turn out to be needed in all facets of walking with aging. Finally, despite the cognitive resources required by walking, both young and older adults spontaneously adopted a “posture second” strategy, prioritizing the cognitive task over the gait task

Learning soft task priorities for safe control of humanoid robots with constrained stochastic optimization

Multi-task prioritized controllers are able to generate complex robot behaviors that concurrently satisfy several tasks and constraints. To perform, they often require a human expert to define the evolution of the task priorities in time. In a previous paper [1] we proposed a framework to automatically learn the task priorities thanks to a stochastic optimization algorithm (CMA-ES) maximizing the robot performance on a certain behavior. Here, we learn the task priorities that maximize the robot performance, ensuring that the optimized priorities lead to safe behaviors that never violate any of the robot and problem constraints. We compare three constrained variants of CMA-ES on several benchmarks, among which two are new robotics benchmarks of our design using the KUKA LWR. We retain (1+1)-CMA-ES with covariance constrained adaptation [2] as the best candidate to solve our problems, and we show its effectiveness on two whole-body experiments with the iCub humanoid robot