Robust subspace learning for static and dynamic affect and behaviour modelling

Machine analysis of human affect and behavior in naturalistic contexts has witnessed a growing attention in the last decade from various disciplines ranging from social and cognitive sciences to machine learning and computer vision. Endowing machines with the ability to seamlessly detect, analyze, model, predict as well as simulate and synthesize manifestations of internal emotional and behavioral states in real-world data is deemed essential for the deployment of next-generation, emotionally- and socially-competent human-centered interfaces. In this thesis, we are primarily motivated by the problem of modeling, recognizing and predicting spontaneous expressions of non-verbal human affect and behavior manifested through either low-level facial attributes in static images or high-level semantic events in image sequences. Both visual data and annotations of naturalistic affect and behavior naturally contain noisy measurements of unbounded magnitude at random locations, commonly referred to as ‘outliers’. We present here machine learning methods that are robust to such gross, sparse noise. First, we deal with static analysis of face images, viewing the latter as a superposition of mutually-incoherent, low-complexity components corresponding to facial attributes, such as facial identity, expressions and activation of atomic facial muscle actions. We develop a robust, discriminant dictionary learning framework to extract these components from grossly corrupted training data and combine it with sparse representation to recognize the associated attributes. We demonstrate that our framework can jointly address interrelated classification tasks such as face and facial expression recognition. Inspired by the well-documented importance of the temporal aspect in perceiving affect and behavior, we direct the bulk of our research efforts into continuous-time modeling of dimensional affect and social behavior. Having identified a gap in the literature which is the lack of data containing annotations of social attitudes in continuous time and scale, we first curate a new audio-visual database of multi-party conversations from political debates annotated frame-by-frame in terms of real-valued conflict intensity and use it to conduct the first study on continuous-time conflict intensity estimation. Our experimental findings corroborate previous evidence indicating the inability of existing classifiers in capturing the hidden temporal structures of affective and behavioral displays. We present here a novel dynamic behavior analysis framework which models temporal dynamics in an explicit way, based on the natural assumption that continuous- time annotations of smoothly-varying affect or behavior can be viewed as outputs of a low-complexity linear dynamical system when behavioral cues (features) act as system inputs. A novel robust structured rank minimization framework is proposed to estimate the system parameters in the presence of gross corruptions and partially missing data. Experiments on prediction of dimensional conflict and affect as well as multi-object tracking from detection validate the effectiveness of our predictive framework and demonstrate that for the first time that complex human behavior and affect can be learned and predicted based on small training sets of person(s)-specific observations.Open Acces

Spatio-temporal judgements of observed actions : Contrasts between first- and third-person perspective after motor priming.

When observing actions, motor simulation processes aid the prediction and understanding of future events. A central issue concerns whether such action simulation serves social functions of interpreting other people, where performance is predicted to be better when third-person perspective (3PP) actions are viewed; or whether it is most beneficial to guide self actions, whereby the first-person perspective (1PP) would be advantageous. We show that in a spatio-temporal judgement task there is an advantage for the prediction of 1PP. However, this is only detected after motor priming whereby participants perform the observed actions prior to making spatio-temporal judgements. The results, firstly, confirm that we draw on our motor experience for the accurate simulation and prediction of action. Secondly, the results suggest that such experience facilitates more accurate state estimation for actions perceived in the 1PP which map more closely onto visual input of self-generated action. More forward prediction error is retained for 3PP viewed actions, which may however have the benefit of compensating for the uncertainty involved in interacting with others

Predictive Processing and the Phenomenology of Time Consciousness: A Hierarchical Extension of Rick Grush’s Trajectory Estimation Model

This chapter explores to what extent some core ideas of predictive processing can be applied to the phenomenology of time consciousness. The focus is on the experienced continuity of consciously perceived, temporally extended phenomena (such as enduring processes and successions of events). The main claim is that the hierarchy of representations posited by hierarchical predictive processing models can contribute to a deepened understanding of the continuity of consciousness. Computationally, such models show that sequences of events can be represented as states of a hierarchy of dynamical systems. Phenomenologically, they suggest a more fine-grained analysis of the perceptual contents of the specious present, in terms of a hierarchy of temporal wholes. Visual perception of static scenes not only contains perceived objects and regions but also spatial gist; similarly, auditory perception of temporal sequences, such as melodies, involves not only perceiving individual notes but also slightly more abstract features (temporal gist), which have longer temporal durations (e.g., emotional character or rhythm). Further investigations into these elusive contents of conscious perception may be facilitated by findings regarding its neural underpinnings. Predictive processing models suggest that sensorimotor areas may influence these contents

Understanding Legal Understanding: The Legal Subject and the Problem of Legal Coherence

This is an essay about the law\u27s coherence. It is also an essay about the various forms of legal understanding, the contributions that we make to these forms of understanding, and the effects that legal understanding has upon us. There is an intimate relation between these matters. Coherence is more than a property of law; it is the result of a particular way of thinking about the law. The experience of coherence is an activity of understanding; it is something we do with and to the law, and through this activity, we ourselves are changed

Progress toward an understanding of cortical computation

The additional data, perspectives, questions, and criticisms contributed by the commentaries strengthen our view that local cortical processors coordinate their activity with the context in which it occurs using contextual fields and synchronized population codes. We therefore predict that whereas the specialization of function has been the keynote of this century the coordination of function will be the keynote of the next

USING EMOTIONS: BIOLOGICAL AND SOCIAL FACTORS INFLUENCING EMOTION UNDERSTANDING AND ANTISOCIALITY

People are guided by their emotions which in turn are a consequence of their understanding of others’ emotion expressions. Their skills to read and accurately identify others’ emotion expressions are a key ingredient for good emotion understanding. That is, accurate emotion identification can be considered as the first frontier of successful emotion understanding, and as the first step of a sequence which results in empathic responding. Impairment within this sequence might mean that the way people respond to their environment may not be appropriate or even cause harm to others. Children and adolescents with callous-unemotional traits have difficulties reading emotional cues correctly, specifically those cues which show others in distress. Such an impairment is thought to underlie a distinct pathway to severe and stable antisocial behaviour. Conventional methods of curbing the antisocial behaviour of children with high callous-unemotional traits such as punishment or time-out do not have the desired effect. Instead, this group of individuals seems to respond well to parental warmth and sensitive responding. Given that children start to learn early how to read and respond to emotions in an empathic manner through interactions, parents have a potential role by intervening early to foster good emotional and social skills even in children with high callous-unemotional traits. Study 1 tested whether adolescent boys with high callous-unemotional traits exhibit an impairment that is specific to distress cues such as fear, sadness or pain as difficulties to recognise such cues in others may impair typical inhibition to behave in an antisocial manner. In Study 2, it was expected that successful parental scaffolding is dependent on parent’s own emotion understanding skills, and therefore, study 2 investigated ways in which parents can scaffold emotion understanding in typically developing children, e.g. through talking about others’ emotion states and through engaging children in mutual eye gaze. Study 3 examined the impact that varying levels of child callous-unemotional traits have on parent-child interaction. Specifically, it was of interest whether children with high callous-unemotional traits are willing to engage with their parents on an emotional level permitting successful parental scaffolding. Parental understanding of emotions was tested in terms of promoting parental sensitive responsiveness. In sum, there are three main points the present thesis contributed: first, findings of Study 1 and 3 support a theory of emotion processing impairment that is not specific to fear or sadness, but describe a broader impairment of a failure to engage with the emotional environment and attend to salient emotional stimuli. Second, this thesis confirms the value of studying callous-unemotional traits in adolescents and young children as well as their parents. Third, findings of Studies 2 and 3 support the important role parents play in the lives of their children with callous-unemotional traits, specifically through their own emotion understanding

Methods and Apparatus for Autonomous Robotic Control

Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on "stovepiped," or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements

Weaving webs of insecurity: fear, weakness and power in the post-Soviet South Caucasus

This thesis' central aim is the application of a Wendtian-constructivist expansion of Regional Security Complex Theory (RSCT) on a specific case study: the South Caucasus. To that effect, three concepts of RSCT – amity/enmity, state incoherence, and great power penetration – are expanded and developed within the broader above-mentioned ontological-epistemological framework. Amity-enmity is elaborated into an integrated spectrum founded on varying ideational patterns of securitisation alongside objective characteristics, and encompassing conflict formations, security regimes and security communities. States are conceptualised as ideational-institutional-material "providers of security"; their incoherence is characterised over three tiers and two dimensions, leading to a distinction between vertical and horizontal inherent weakness, ostensible instability and failure. Great power penetration is dissected into its objective, subjective and intersubjective elements, resulting in a 1+3+1 typology of its recurring patterns: unipolar, multipolar-cooperative and multipolar-competitive, bounded by hegemony and disengagement. After the specification of a methodology incorporating both objective macro- and interpretive micro-perspectives, two working hypotheses are specified. Firstly, that state incoherence engenders high levels of regional enmity, and, secondly, that patterns of great power penetration primarily affect transitions of regional amity/enmity between conflict formations and security regimes. The framework is subsequently used to triangulate these hypotheses through an application of the theoretical framework on the post-Soviet Southern Caucasus. An initial macro-overview is subsequently provided of the Southern Caucasus as a regional security complex; the three expanded concepts are consequently investigated, in turn, from the discursive micro-perspective. The South Caucasus is categorised into a "revisionist conflict formation", the nature of its states' incoherence is characterised, and existing patterns of great power penetration are identified as competitive-multipolar. In the final chapter, the hypotheses are largely confirmed, and various scenarios as to the possible emergence of a regional security regime are investigated