Causal Confusion in Imitation Learning

Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive "causal misidentification" phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions---either environment interaction or expert queries---to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations.Comment: Published at NeurIPS 2019 9 pages, plus references and appendice

An immune system based genetic algorithm using permutation-based dualism for dynamic traveling salesman problems

Copyright @ Springer-Verlag Berlin Heidelberg 2009.In recent years, optimization in dynamic environments has attracted a growing interest from the genetic algorithm community due to the importance and practicability in real world applications. This paper proposes a new genetic algorithm, based on the inspiration from biological immune systems, to address dynamic traveling salesman problems. Within the proposed algorithm, a permutation-based dualism is introduced in the course of clone process to promote the population diversity. In addition, a memory-based vaccination scheme is presented to further improve its tracking ability in dynamic environments. The experimental results show that the proposed diversification and memory enhancement methods can greatly improve the adaptability of genetic algorithms for dynamic traveling salesman problems.This work was supported by the Key Program of National Natural Science Foundation (NNSF) of China under Grant No. 70431003 and Grant No. 70671020, the Science Fund for Creative Research Group of NNSF of China under GrantNo. 60521003, the National Science and Technology Support Plan of China under Grant No. 2006BAH02A09 and the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant No. EP/E060722/1

A Study of Deep Reinforcement Learning in Autonomous Racing Using DeepRacer Car

Reinforcement learning is thought to be a promising branch of machine learning that has the potential to help us develop an Artificial General Intelligence (AGI) machine. Among the machine learning algorithms, primarily, supervised, semi supervised, unsupervised and reinforcement learning, reinforcement learning is different in a sense that it explores the environment without prior knowledge, and determines the optimal action. This study attempts to understand the concept behind reinforcement learning, the mathematics behind it and see it in action by deploying the trained model in Amazon\u27s DeepRacer car. DeepRacer, a 1/18th scaled autonomous car, is the agent which is trained to race autonomously on a track. Optimum race line coordinates were calculated which allowed the agent to follow the fastest possible route on a given track. The agent was then trained using proximal policy optimization (PPO). Performance metrics such as the average reward per episode and cumulative reward were examined to fine tune the model. To further understand the distribution of action spaces, log analyses tools provided by the amazon was used. Based on the log analysis data, any un-used action was removed for efficient training. The trained model was uploaded into the DeepRacer car to test it in a race track outside of simulation

Application of an AIS to the problem of through life health management of remotely piloted aircraft

The operation of RPAS includes a cognitive problem for the operators(Pilots, maintainers, ,managers, and the wider organization) to effectively maintain their situational awareness of the aircraft and predict its health state. This has a large impact on their ability to successfully identify faults and manage systems during operations. To overcome these system deficiencies an asset health management system that integrates more cognitive abilities to aid situational awareness could prove beneficial. This paper outlines an artificial immune system (AIS) approach that could meet these challenges and an experimental method within which to evaluate it

GNSS/LiDAR-Based Navigation of an Aerial Robot in Sparse Forests

Autonomous navigation of unmanned vehicles in forests is a challenging task. In such environments, due to the canopies of the trees, information from Global Navigation Satellite Systems (GNSS) can be degraded or even unavailable. Also, because of the large number of obstacles, a previous detailed map of the environment is not practical. In this paper, we solve the complete navigation problem of an aerial robot in a sparse forest, where there is enough space for the flight and the GNSS signals can be sporadically detected. For localization, we propose a state estimator that merges information from GNSS, Attitude and Heading Reference Systems (AHRS), and odometry based on Light Detection and Ranging (LiDAR) sensors. In our LiDAR-based odometry solution, the trunks of the trees are used in a feature-based scan matching algorithm to estimate the relative movement of the vehicle. Our method employs a robust adaptive fusion algorithm based on the unscented Kalman filter. For motion control, we adopt a strategy that integrates a vector field, used to impose the main direction of the movement for the robot, with an optimal probabilistic planner, which is responsible for obstacle avoidance. Experiments with a quadrotor equipped with a planar LiDAR in an actual forest environment is used to illustrate the effectiveness of our approach

Intranodal administration of mRNA encoding nucleoprotein provides cross-strain immunity against influenza in mice

Background: Current human influenza vaccines lack the adaptability to match the mutational rate of the virus and therefore require annual revisions. Because of extensive manufacturing times and the possibility that antigenic alterations occur during viral vaccine strain production, an inherent risk exists for antigenic mismatch between the new influenza vaccine and circulating viruses. Targeting more conserved antigens such as nucleoprotein (NP) could provide a more sustainable vaccination strategy by inducing long term and heterosubtypic protection against influenza. We previously demonstrated that intranodal mRNA injection can induce potent antigen-specific T-cell responses. In this study, we investigated whether intranodal administration of mRNA encoding NP can induce T-cell responses capable of protecting against a heterologous influenza virus challenge. Methods: BALB/c mice were immunized in the inguinal lymph nodes with different vaccination regimens of mRNA encoding NP. Immune responses were compared with NP DNA vaccination via IFN-gamma ELISPOT and in vivo cytotoxicity. For survival experiments, mice were prime-boost vaccinated with 17 mu g NP mRNA and infected with 1LD50 of H1N1 influenza virus 8weeks after boost. Weight was monitored and viral titers, cytokines and immune cell populations in the bronchoalveolar lavage, and IFN-gamma responses in the spleen were analyzed. Results: Our results demonstrate that NP mRNA induces superior systemic T-cell responses against NP compared to classical DNA vaccination. These responses were sustained for several weeks even at low vaccine doses. Upon challenge infection, vaccination with NP mRNA resulted in reduced lung viral titers and improved recovery from infection. Finally, we show that vaccination with NP mRNA affects the immune response in infected lungs by lowering immune cell infiltration while increasing the fraction of T cells, monocytes and MHC II+ alveolar macrophages within immune infiltrates. This change was associated with altered levels of both pro- and anti-inflammatory cytokines. Conclusions: These findings suggest that intranodal vaccination with NP mRNA induces cross-strain immunity against influenza, but also highlight a paradox of influenza immunity, whereby robust immune responses can provide protection, but can also transiently exacerbate symptoms during infection

Botnet lab creation with open source tools and usefulness of such a tool for researchers

Botnets are large scale networks, which can span across the internet and comprise of computers, which have been infected by malicious software and are centrally controlled from a remote location. Botnets pose a great security risk and their size has been rising drastically over the past few years. The use of botnets by the underground community as a medium for online crime, bundled with their use for profit has shined the spotlight on them. Numerous researchers have proposed and designed infrastructures and frameworks that identify newly formed botnets and their traffic patterns. In this research, the design of a unified modular open source laboratory is proposed, with the use of virtual machines and open source tools, which can be used in analyzing and dissecting newly found bots in the wild. Furthermore, the usefulness and flexibility of the open source laboratory is evaluated by infecting my test machines with the Zeus Bot

Behavioral Modeling Based on Probabilistic Finite Automata: An Empirical Study

Imagine an agent that performs tasks according to different strategies. The goal of Behavioral Recognition (BR) is to identify which of the available strategies is the one being used by the agent, by simply observing the agent?s actions and the environmental conditions during a certain period of time. The goal of Behavioral Cloning (BC) is more ambitious. In this last case, the learner must be able to build a model of the behavior of the agent. In both settings, the only assumption is that the learner has access to a training set that contains instances of observed behavioral traces for each available strategy. This paper studies a machine learning approach based on Probabilistic Finite Automata (PFAs), capable of achieving both the recognition and cloning tasks. We evaluate the performance of PFAs in the context of a simulated learning environment (in this case, a virtual Roomba vacuum cleaner robot), and compare it with a collection of other machine learning approaches.This work was partially supported by project PAC::LFO (MTM2014-55262-P) of Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia, Ministerio de Ciencia e Innovación (MICINN), Spain, and by the National Science Foundation (NSF) project SCH-1521943, USA