Discovering Causal Relations and Equations from Data

Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.Comment: 137 page

Guidage non-intrusif d'un bras robotique à l'aide d'un bracelet myoélectrique à électrode sèche

Depuis plusieurs années la robotique est vue comme une solution clef pour améliorer la qualité de vie des personnes ayant subi une amputation. Pour créer de nouvelles prothèses intelligentes qui peuvent être facilement intégrées à la vie quotidienne et acceptée par ces personnes, celles-ci doivent être non-intrusives, fiables et peu coûteuses. L’électromyographie de surface fournit une interface intuitive et non intrusive basée sur l’activité musculaire de l’utilisateur permettant d’interagir avec des robots. Cependant, malgré des recherches approfondies dans le domaine de la classification des signaux sEMG, les classificateurs actuels manquent toujours de fiabilité, car ils ne sont pas robustes face au bruit à court terme (par exemple, petit déplacement des électrodes, fatigue musculaire) ou à long terme (par exemple, changement de la masse musculaire et des tissus adipeux) et requiert donc de recalibrer le classifieur de façon périodique. L’objectif de mon projet de recherche est de proposer une interface myoélectrique humain-robot basé sur des algorithmes d’apprentissage par transfert et d’adaptation de domaine afin d’augmenter la fiabilité du système à long-terme, tout en minimisant l’intrusivité (au niveau du temps de préparation) de ce genre de système. L’aspect non intrusif est obtenu en utilisant un bracelet à électrode sèche possédant dix canaux. Ce bracelet (3DC Armband) est de notre (Docteur Gabriel Gagnon-Turcotte, mes co-directeurs et moi-même) conception et a été réalisé durant mon doctorat. À l’heure d’écrire ces lignes, le 3DC Armband est le bracelet sans fil pour l’enregistrement de signaux sEMG le plus performant disponible. Contrairement aux dispositifs utilisant des électrodes à base de gel qui nécessitent un rasage de l’avant-bras, un nettoyage de la zone de placement et l’application d’un gel conducteur avant l’utilisation, le brassard du 3DC peut simplement être placé sur l’avant-bras sans aucune préparation. Cependant, cette facilité d’utilisation entraîne une diminution de la qualité de l’information du signal. Cette diminution provient du fait que les électrodes sèches obtiennent un signal plus bruité que celle à base de gel. En outre, des méthodes invasives peuvent réduire les déplacements d’électrodes lors de l’utilisation, contrairement au brassard. Pour remédier à cette dégradation de l’information, le projet de recherche s’appuiera sur l’apprentissage profond, et plus précisément sur les réseaux convolutionels. Le projet de recherche a été divisé en trois phases. La première porte sur la conception d’un classifieur permettant la reconnaissance de gestes de la main en temps réel. La deuxième porte sur l’implémentation d’un algorithme d’apprentissage par transfert afin de pouvoir profiter des données provenant d’autres personnes, permettant ainsi d’améliorer la classification des mouvements de la main pour un nouvel individu tout en diminuant le temps de préparation nécessaire pour utiliser le système. La troisième phase consiste en l’élaboration et l’implémentation des algorithmes d’adaptation de domaine et d’apprentissage faiblement supervisé afin de créer un classifieur qui soit robuste au changement à long terme.For several years, robotics has been seen as a key solution to improve the quality of life of people living with upper-limb disabilities. To create new, smart prostheses that can easily be integrated into everyday life, they must be non-intrusive, reliable and inexpensive. Surface electromyography provides an intuitive interface based on a user’s muscle activity to interact with robots. However, despite extensive research in the field of sEMG signal classification, current classifiers still lack reliability due to their lack of robustness to short-term (e.g. small electrode displacement, muscle fatigue) or long-term (e.g. change in muscle mass and adipose tissue) noise. In practice, this mean that to be useful, classifier needs to be periodically re-calibrated, a time consuming process. The goal of my research project is to proposes a human-robot myoelectric interface based on transfer learning and domain adaptation algorithms to increase the reliability of the system in the long term, while at the same time reducing the intrusiveness (in terms of hardware and preparation time) of this kind of systems. The non-intrusive aspect is achieved from a dry-electrode armband featuring ten channels. This armband, named the 3DC Armband is from our (Dr. Gabriel Gagnon-Turcotte, my co-directors and myself) conception and was realized during my doctorate. At the time of writing, the 3DC Armband offers the best performance for currently available dry-electrodes, surface electromyographic armbands. Unlike gel-based electrodes which require intrusive skin preparation (i.e. shaving, cleaning the skin and applying conductive gel), the 3DC Armband can simply be placed on the forearm without any preparation. However, this ease of use results in a decrease in the quality of information. This decrease is due to the fact that the signal recorded by dry electrodes is inherently noisier than gel-based ones. In addition, other systems use invasive methods (intramuscular electromyography) to capture a cleaner signal and reduce the source of noises (e.g. electrode shift). To remedy this degradation of information resulting from the non-intrusiveness of the armband, this research project will rely on deep learning, and more specifically on convolutional networks. The research project was divided into three phases. The first is the design of a classifier allowing the recognition of hand gestures in real-time. The second is the implementation of a transfer learning algorithm to take advantage of the data recorded across multiple users, thereby improving the system’s accuracy, while decreasing the time required to use the system. The third phase is the development and implementation of a domain adaptation and self-supervised learning to enhance the classifier’s robustness to long-term changes

自然に学ぶ知的アルゴリズムによる最適化及び予測問題に関する研究

富山大学・富理工博甲第147号・劉燕婷・2018/09/28富山大学201

Advances in Evolutionary Algorithms

With the recent trends towards massive data sets and significant computational power, combined with evolutionary algorithmic advances evolutionary computation is becoming much more relevant to practice. Aim of the book is to present recent improvements, innovative ideas and concepts in a part of a huge EA field

when channels cooperate or capacitance varies

Die elektrische Signalverarbeitung in Nervenzellen basiert auf deren erregbarer Zellmembran. Üblicherweise wird angenommen, dass die in der Membran eingebetteten leitfähigen Ionenkanäle nicht auf direkte Art gekoppelt sind und dass die Kapazität des von der Membran gebildeten Kondensators konstant ist. Allerdings scheinen diese Annahmen nicht für alle Nervenzellen zu gelten. Im Gegenteil, verschiedene Ionenkanäle “kooperieren” und auch die Vorstellung von einer konstanten spezifischen Membrankapazität wurde kürzlich in Frage gestellt. Die Auswirkungen dieser Abweichungen auf die elektrischen Eigenschaften von Nervenzellen ist das Thema der folgenden kumulativen Dissertationsschrift. Im ersten Projekt wird gezeigt, auf welche Weise stark kooperative spannungsabhängige Ionenkanäle eine Form von zellulärem Kurzzeitspeicher für elektrische Aktivität bilden könnten. Solche kooperativen Kanäle treten in der Membran häufig in kleinen räumlich getrennte Clustern auf. Basierend auf einem mathematischen Modell wird nachgewiesen, dass solche Kanalcluster als eine bistabile Leitfähigkeit agieren. Die dadurch entstehende große Speicherkapazität eines Ensembles dieser Kanalcluster könnte von Nervenzellen für stufenloses persistentes Feuern genutzt werden -- ein Feuerverhalten von Nutzen für das Kurzzeichgedächtnis. Im zweiten Projekt wird ein neues Dynamic Clamp Protokoll entwickelt, der Capacitance Clamp, das erlaubt, Änderungen der Membrankapazität in biologischen Nervenzellen zu emulieren. Eine solche experimentelle Möglichkeit, um systematisch die Rolle der Kapazität zu untersuchen, gab es bisher nicht. Nach einer Reihe von Tests in Simulationen und Experimenten wurde die Technik mit Körnerzellen des *Gyrus dentatus* genutzt, um den Einfluss von Kapazität auf deren Feuerverhalten zu studieren. Die Kombination beider Projekte zeigt die Relevanz dieser oft vernachlässigten Facetten von neuronalen Membranen für die Signalverarbeitung in Nervenzellen.Electrical signaling in neurons is shaped by their specialized excitable cell membranes. Commonly, it is assumed that the ion channels embedded in the membrane gate independently and that the electrical capacitance of neurons is constant. However, not all excitable membranes appear to adhere to these assumptions. On the contrary, ion channels are observed to gate cooperatively in several circumstances and also the notion of one fixed value for the specific membrane capacitance (per unit area) across neuronal membranes has been challenged recently. How these deviations from the original form of conductance-based neuron models affect their electrical properties has not been extensively explored and is the focus of this cumulative thesis. In the first project, strongly cooperative voltage-gated ion channels are proposed to provide a membrane potential-based mechanism for cellular short-term memory. Based on a mathematical model of cooperative gating, it is shown that coupled channels assembled into small clusters act as an ensemble of bistable conductances. The correspondingly large memory capacity of such an ensemble yields an alternative explanation for graded forms of cell-autonomous persistent firing – an observed firing mode implicated in working memory. In the second project, a novel dynamic clamp protocol -- the capacitance clamp -- is developed to artificially modify capacitance in biological neurons. Experimental means to systematically investigate capacitance, a basic parameter shared by all excitable cells, had previously been missing. The technique, thoroughly tested in simulations and experiments, is used to monitor how capacitance affects temporal integration and energetic costs of spiking in dentate gyrus granule cells. Combined, the projects identify computationally relevant consequences of these often neglected facets of neuronal membranes and extend the modeling and experimental techniques to further study them

Formal methods paradigms for estimation and machine learning in dynamical systems

Formal methods are widely used in engineering to determine whether a system exhibits a certain property (verification) or to design controllers that are guaranteed to drive the system to achieve a certain property (synthesis). Most existing techniques require a large amount of accurate information about the system in order to be successful. The methods presented in this work can operate with significantly less prior information. In the domain of formal synthesis for robotics, the assumptions of perfect sensing and perfect knowledge of system dynamics are unrealistic. To address this issue, we present control algorithms that use active estimation and reinforcement learning to mitigate the effects of uncertainty. In the domain of cyber-physical system analysis, we relax the assumption that the system model is known and identify system properties automatically from execution data. First, we address the problem of planning the path of a robot under temporal logic constraints (e.g. "avoid obstacles and periodically visit a recharging station") while simultaneously minimizing the uncertainty about the state of an unknown feature of the environment (e.g. locations of fires after a natural disaster). We present synthesis algorithms and evaluate them via simulation and experiments with aerial robots. Second, we develop a new specification language for tasks that require gathering information about and interacting with a partially observable environment, e.g. "Maintain localization error below a certain level while also avoiding obstacles.'' Third, we consider learning temporal logic properties of a dynamical system from a finite set of system outputs. For example, given maritime surveillance data we wish to find the specification that corresponds only to those vessels that are deemed law-abiding. Algorithms for performing off-line supervised and unsupervised learning and on-line supervised learning are presented. Finally, we consider the case in which we want to steer a system with unknown dynamics to satisfy a given temporal logic specification. We present a novel reinforcement learning paradigm to solve this problem. Our procedure gives "partial credit'' for executions that almost satisfy the specification, which can lead to faster convergence rates and produce better solutions when the specification is not satisfiable

Seventh Annual Workshop on Space Operations Applications and Research (SOAR 1993), volume 1

This document contains papers presented at the Space Operations, Applications and Research Symposium (SOAR) Symposium hosted by NASA/Johnson Space Center (JSC) on August 3-5, 1993, and held at JSC Gilruth Recreation Center. SOAR included NASA and USAF programmatic overview, plenary session, panel discussions, panel sessions, and exhibits. It invited technical papers in support of U.S. Army, U.S. Navy, Department of Energy, NASA, and USAF programs in the following areas: robotics and telepresence, automation and intelligent systems, human factors, life support, and space maintenance and servicing. SOAR was concerned with Government-sponsored research and development relevant to aerospace operations. More than 100 technical papers, 17 exhibits, a plenary session, several panel discussions, and several keynote speeches were included in SOAR '93