On Riemannian tools for classification improvement in Brain-Computer Interfaces

A Brain Computer Interface (BCI) or Brain Machine Interface (BMI) is a device that allows the exchange of information between the brain of a person and a computer without the need of physical interaction. This technology promises to change the way in which we interact with machines, but it is not yet affordable, robust or quick enough to substitute other classic human to machine interfaces for the general public. This being said, the lack of need of interaction makes them a very promising solution that would provide people with severe motor disabilities with a new way of interacting with their surroundings, improving their quality of life. The most extended method of extracting information about brain activity and the one used for this project is the Electroencefalogram (EEG). This device consists of multiple electrodes mounted on a helmet-like structure that is placed on the user’s scalp. The electrodes detect the sum of action potentials from large populations of neurons on the brain’s cortex. The main advantages of this technique are the relative low cost of the device, portability, and the high temporal resolution and ease of use of a non invasive technique. This is not free of disadvantages, as the method suffers from a low signal to noise ratio, low robustness to interference, low spatial resolution and the effects of inter and intra session drift, that is, the movement of the electrodes during and between sessions produce variations on the acquisition of the signal. There are also multiple paradigms in the field of BCI, each one of them focusing on a different brain signal. This work is centered around the Motor Imagery Brain Computer Interface (MI-BCI), which differs from other BCIs in the fact that it directly decodes the intention of the user without the need of inducing a specific response in the brain by presenting an stimulus. This approach is considered to be more natural and can be more comfortable, but also requires a higher level of mental effort and proficiency from part of the user. The MI-BCI is based on a signal of unknown origin that is produced on the sensorymotor cortex, responsible for voluntary movements and touch among others, the Sensorimotor Rhythms (SMR). This signal is atenuated when the person performs or thinks about performing a movement, which is called an Event Related Desynchronization (ERD) and amplified when going back to the idling state, an Event Related Synchronization (ERS). As the brain is a distributed system, the origin of these events can be estimated and is related to the movement that the person imagined. In an implementation, these movements are limited to a discrete set of posibilities and each one is mapped to a computer instruction, allowing the unidirectional transfer of information between brain and machine. The classical machine learning approach to this problem has been to use very specific signal processing techniques to extract relevant features for this problem that can then be fed to a general classification algorithm. The main tecnique is known as Common Spatial Patterns (CSP) followed by classification with Linear Discriminant Analysis (LDA) or Support Vector Machine (SVM). This has some advantages such as a relative low requirement of training samples, but also lacks the capability of generalisation, and a system fine tuned for one user cannot be used for other users or even for another session from the same user reliably. In this work we study an alternative framework that uses the covariance matrices of the EEG signals as observations and exploits the Riemannian geometry of Symmetric Positive Definite (SPD) matrices to classify them in their natural space. This is not only a more general signal processing approach that has been used in other fields of research, but also opens the possibility of transfering some information between users and sessions, which may result in a more robust system or in a system that requires less data for training. This is crucial for the usability of MI-BCI because recording a training session before each use of the system is mentally exhausting and time consuming.Universidad de Sevilla. Máster Universitario en Ingeniería de Telecomunicació

STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits

We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of $2,177$ human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89% on E-Gait, which is 14 - 30% more accurate over prior methods

Computational Methods for Cognitive and Cooperative Robotics

In the last decades design methods in control engineering made substantial progress in the areas of robotics and computer animation. Nowadays these methods incorporate the newest developments in machine learning and artificial intelligence. But the problems of flexible and online-adaptive combinations of motor behaviors remain challenging for human-like animations and for humanoid robotics. In this context, biologically-motivated methods for the analysis and re-synthesis of human motor programs provide new insights in and models for the anticipatory motion synthesis. This thesis presents the author’s achievements in the areas of cognitive and developmental robotics, cooperative and humanoid robotics and intelligent and machine learning methods in computer graphics. The first part of the thesis in the chapter “Goal-directed Imitation for Robots” considers imitation learning in cognitive and developmental robotics. The work presented here details the author’s progress in the development of hierarchical motion recognition and planning inspired by recent discoveries of the functions of mirror-neuron cortical circuits in primates. The overall architecture is capable of ‘learning for imitation’ and ‘learning by imitation’. The complete system includes a low-level real-time capable path planning subsystem for obstacle avoidance during arm reaching. The learning-based path planning subsystem is universal for all types of anthropomorphic robot arms, and is capable of knowledge transfer at the level of individual motor acts. Next, the problems of learning and synthesis of motor synergies, the spatial and spatio-temporal combinations of motor features in sequential multi-action behavior, and the problems of task-related action transitions are considered in the second part of the thesis “Kinematic Motion Synthesis for Computer Graphics and Robotics”. In this part, a new approach of modeling complex full-body human actions by mixtures of time-shift invariant motor primitives in presented. The online-capable full-body motion generation architecture based on dynamic movement primitives driving the time-shift invariant motor synergies was implemented as an online-reactive adaptive motion synthesis for computer graphics and robotics applications. The last chapter of the thesis entitled “Contraction Theory and Self-organized Scenarios in Computer Graphics and Robotics” is dedicated to optimal control strategies in multi-agent scenarios of large crowds of agents expressing highly nonlinear behaviors. This last part presents new mathematical tools for stability analysis and synthesis of multi-agent cooperative scenarios.In den letzten Jahrzehnten hat die Forschung in den Bereichen der Steuerung und Regelung komplexer Systeme erhebliche Fortschritte gemacht, insbesondere in den Bereichen Robotik und Computeranimation. Die Entwicklung solcher Systeme verwendet heutzutage neueste Methoden und Entwicklungen im Bereich des maschinellen Lernens und der künstlichen Intelligenz. Die flexible und echtzeitfähige Kombination von motorischen Verhaltensweisen ist eine wesentliche Herausforderung für die Generierung menschenähnlicher Animationen und in der humanoiden Robotik. In diesem Zusammenhang liefern biologisch motivierte Methoden zur Analyse und Resynthese menschlicher motorischer Programme neue Erkenntnisse und Modelle für die antizipatorische Bewegungssynthese. Diese Dissertation präsentiert die Ergebnisse der Arbeiten des Autors im Gebiet der kognitiven und Entwicklungsrobotik, kooperativer und humanoider Robotersysteme sowie intelligenter und maschineller Lernmethoden in der Computergrafik. Der erste Teil der Dissertation im Kapitel “Zielgerichtete Nachahmung für Roboter” behandelt das Imitationslernen in der kognitiven und Entwicklungsrobotik. Die vorgestellten Arbeiten beschreiben neue Methoden für die hierarchische Bewegungserkennung und -planung, die durch Erkenntnisse zur Funktion der kortikalen Spiegelneuronen-Schaltkreise bei Primaten inspiriert wurden. Die entwickelte Architektur ist in der Lage, ‘durch Imitation zu lernen’ und ‘zu lernen zu imitieren’. Das komplette entwickelte System enthält ein echtzeitfähiges Pfadplanungssubsystem zur Hindernisvermeidung während der Durchführung von Armbewegungen. Das lernbasierte Pfadplanungssubsystem ist universell und für alle Arten von anthropomorphen Roboterarmen in der Lage, Wissen auf der Ebene einzelner motorischer Handlungen zu übertragen. Im zweiten Teil der Arbeit “Kinematische Bewegungssynthese für Computergrafik und Robotik” werden die Probleme des Lernens und der Synthese motorischer Synergien, d.h. von räumlichen und räumlich-zeitlichen Kombinationen motorischer Bewegungselemente bei Bewegungssequenzen und bei aufgabenbezogenen Handlungs übergängen behandelt. Es wird ein neuer Ansatz zur Modellierung komplexer menschlicher Ganzkörperaktionen durch Mischungen von zeitverschiebungsinvarianten Motorprimitiven vorgestellt. Zudem wurde ein online-fähiger Synthesealgorithmus für Ganzköperbewegungen entwickelt, der auf dynamischen Bewegungsprimitiven basiert, die wiederum auf der Basis der gelernten verschiebungsinvarianten Primitive konstruiert werden. Dieser Algorithmus wurde für verschiedene Probleme der Bewegungssynthese für die Computergrafik- und Roboteranwendungen implementiert. Das letzte Kapitel der Dissertation mit dem Titel “Kontraktionstheorie und selbstorganisierte Szenarien in der Computergrafik und Robotik” widmet sich optimalen Kontrollstrategien in Multi-Agenten-Szenarien, wobei die Agenten durch eine hochgradig nichtlineare Kinematik gekennzeichnet sind. Dieser letzte Teil präsentiert neue mathematische Werkzeuge für die Stabilitätsanalyse und Synthese von kooperativen Multi-Agenten-Szenarien

Reconstructions of science

'In vier Rekonstruktionen wird versucht, die natur- und sozialwissenschaftliche Diskussion der Basiskonzepte von Raum und Zeit zu vereinen. Dazu bedarf es einer neuen Diskursform, die bereits in Alfred North Whiteheads Naturphilosophie anklingt. Auf sozial-wissenschaftlicher Seite besinnen wir uns grundlegender Themen von Sozialphänomenologie, Strukturalismus und Interaktionismus. Fragestellungen von PrähistorikerInnen, ÄgyptologInnen und EthnomathematikerInnen werden wichtig, wo wir zeigen, daß unsere Konzepte von Raum und Zeit kulturelle Institutionen der Bedeutung sind, die ihrerseits Gesellschaft konstituieren und konstanter Rekonstruktion bedürfen. Die vierte Rekonstruktion greift die Frage der theoretischen Physik auf und stützt sich auf das integrative Instrument der Theorie der geometrischen Clifford Algebren. Wir leiten ab, daß und wie die inneren Symmetrien der Materie mit den äußeren Symmetrien der Raum-Zeit verbunden sind und daß die Metapher vom 'achtfachen Pfad', die Gell-Mann für einen Teil des Standardmodells verwendete, entgegen seiner Auffassung nicht als Witz zu verstehen ist. Der Faktor (D4)m in der Dirac-Gruppe jeder geometrischen Clifford Algebra C/p,q bildet eine Grundstruktur von Orientierung und Logik ab und korrespondiert daher mit einem Interface zwischen Geist und Materie.' (Autorenreferat)'In four reconstructions it is attempted to lead the natural and social science debate of the basis concepts of space and time in common. For this we need a new mode of science discourse which has already been initiated in Alfred North Whitehead's philosophy of nature. In social science we reconsider the basis themes of social phenomenology, structuralism and interactionism as far as those contribute to a space-time topic. Investigations of prehistorians, egyptologists and ethno-mathematicians are of importance where we demonstrate that our concepts of space and time represent cultural institutions of meaning which on their part constitute society and require that we constantly reconstruct them. The fourth reconstruction deals with the space-time of postmodern theoretical physics and is founded on the integrative instrument of the theory of geometric Clifford algebras. We show that and how the inner symmetrics of matter are connected with the outer symmetries of space-time and that Gell-Mann's metaphor of the 'eightfold path' that he used to denote part of the standard model of physics cannot be interpreted as quirk, in opposition to his own intention. The factor (D4)m in the Dirac group of any geometric Clifford Algebra C/p,q represents a ground template (or archetypal structure) for both orientation and logic and corresponds therefore with an interface between matter and mind.' (author's abstract)