Crossmodal Attentive Skill Learner

This paper presents the Crossmodal Attentive Skill Learner (CASL), integrated with the recently-introduced Asynchronous Advantage Option-Critic (A2OC) architecture [Harb et al., 2017] to enable hierarchical reinforcement learning across multiple sensory inputs. We provide concrete examples where the approach not only improves performance in a single task, but accelerates transfer to new tasks. We demonstrate the attention mechanism anticipates and identifies useful latent features, while filtering irrelevant sensor modalities during execution. We modify the Arcade Learning Environment [Bellemare et al., 2013] to support audio queries, and conduct evaluations of crossmodal learning in the Atari 2600 game Amidar. Finally, building on the recent work of Babaeizadeh et al. [2017], we open-source a fast hybrid CPU-GPU implementation of CASL.Comment: International Conference on Autonomous Agents and Multiagent Systems (AAMAS) 2018, NIPS 2017 Deep Reinforcement Learning Symposiu

Real-time event-based unsupervised feature consolidation and tracking for space situational awareness

Earth orbit is a limited natural resource that hosts a vast range of vital space-based systems that support the international community's national, commercial and defence interests. This resource is rapidly becoming depleted with over-crowding in high demand orbital slots and a growing presence of space debris. We propose the Fast Iterative Extraction of Salient targets for Tracking Asynchronously (FIESTA) algorithm as a robust, real-time and reactive approach to optical Space Situational Awareness (SSA) using Event-Based Cameras (EBCs) to detect, localize, and track Resident Space Objects (RSOs) accurately and timely. We address the challenges of the asynchronous nature and high temporal resolution output of the EBC accurately, unsupervised and with few tune-able parameters using concepts established in the neuromorphic and conventional tracking literature. We show this algorithm is capable of highly accurate in-frame RSO velocity estimation and average sub-pixel localization in a simulated test environment to distinguish the capabilities of the EBC and optical setup from the proposed tracking system. This work is a fundamental step toward accurate end-to-end real-time optical event-based SSA, and developing the foundation for robust closed-form tracking evaluated using standardized tracking metrics

Estimation and stability of nonlinear control systems under intermittent information with applications to multi-agent robotics

This dissertation investigates the role of intermittent information in estimation and control problems and applies the obtained results to multi-agent tasks in robotics. First, we develop a stochastic hybrid model of mobile networks able to capture a large variety of heterogeneous multi-agent problems and phenomena. This model is applied to a case study where a heterogeneous mobile sensor network cooperatively detects and tracks mobile targets based on intermittent observations. When these observations form a satisfactory target trajectory, a mobile sensor is switched to the pursuit mode and deployed to capture the target. The cost of operating the sensors is determined from the geometric properties of the network, environment and probability of target detection. The above case study is motivated by the Marco Polo game played by children in swimming pools. Second, we develop adaptive sampling of targets positions in order to minimize energy consumption, while satisfying performance guarantees such as increased probability of detection over time, and no-escape conditions. A parsimonious predictor-corrector tracking filter, that uses geometrical properties of targets\u27 tracks to estimate their positions using imperfect and intermittent measurements, is presented. It is shown that this filter requires substantially less information and processing power than the Unscented Kalman Filter and Sampling Importance Resampling Particle Filter, while providing comparable estimation performance in the presence of intermittent information. Third, we investigate stability of nonlinear control systems under intermittent information. We replace the traditional periodic paradigm, where the up-to-date information is transmitted and control laws are executed in a periodic fashion, with the event-triggered paradigm. Building on the small gain theorem, we develop input-output triggered control algorithms yielding stable closed-loop systems. In other words, based on the currently available (but outdated) measurements of the outputs and external inputs of a plant, a mechanism triggering when to obtain new measurements and update the control inputs is provided. Depending on the noise environment, the developed algorithm yields stable, asymptotically stable, and Lp-stable (with bias) closed-loop systems. Control loops are modeled as interconnections of hybrid systems for which novel results on Lp-stability are presented. Prediction of a triggering event is achieved by employing Lp-gains over a finite horizon in the small gain theorem. By resorting to convex programming, a method to compute Lp-gains over a finite horizon is devised. Next, we investigate optimal intermittent feedback for nonlinear control systems. Using the currently available measurements from a plant, we develop a methodology that outputs when to update the control law with new measurements such that a given cost function is minimized. Our cost function captures trade-offs between the performance and energy consumption of the control system. The optimization problem is formulated as a Dynamic Programming problem, and Approximate Dynamic Programming is employed to solve it. Instead of advocating a particular approximation architecture for Approximate Dynamic Programming, we formulate properties that successful approximation architectures satisfy. In addition, we consider problems with partially observable states, and propose Particle Filtering to deal with partially observable states and intermittent feedback. Finally, we investigate a decentralized output synchronization problem of heterogeneous linear systems. We develop a self-triggered output broadcasting policy for the interconnected systems. Broadcasting time instants adapt to the current communication topology. For a fixed topology, our broadcasting policy yields global exponential output synchronization, and Lp-stable output synchronization in the presence of disturbances. Employing a converse Lyapunov theorem for impulsive systems, we provide an average dwell time condition that yields disturbance-to-state stable output synchronization in case of switching topology. Our approach is applicable to directed and unbalanced communication topologies.\u2

A two phase framework for visible light-based positioning in an indoor environment: performance, latency, and illumination

Recently with the advancement of solid state lighting and the application thereof to Visible Light Communications (VLC), the concept of Visible Light Positioning (VLP) has been targeted as a very attractive indoor positioning system (IPS) due to its ubiquity, directionality, spatial reuse, and relatively high modulation bandwidth. IPSs, in general, have 4 major components (1) a modulation, (2) a multiple access scheme, (3) a channel measurement, and (4) a positioning algorithm. A number of VLP approaches have been proposed in the literature and primarily focus on a fixed combination of these elements and moreover evaluate the quality of the contribution often by accuracy or precision alone. In this dissertation, we provide a novel two-phase indoor positioning algorithmic framework that is able to increase robustness when subject to insufficient anchor luminaries and also incorporate any combination of the four major IPS components. The first phase provides robust and timely albeit less accurate positioning proximity estimates without requiring more than a single luminary anchor using time division access to On Off Keying (OOK) modulated signals while the second phase provides a more accurate, conventional, positioning estimate approach using a novel geometric constrained triangulation algorithm based on angle of arrival (AoA) measurements. However, this approach is still an application of a specific combination of IPS components. To achieve a broader impact, the framework is employed on a collection of IPS component combinations ranging from (1) pulsed modulations to multicarrier modulations, (2) time, frequency, and code division multiple access, (3) received signal strength (RSS), time of flight (ToF), and AoA, as well as (4) trilateration and triangulation positioning algorithms. Results illustrate full room positioning coverage ranging with median accuracies ranging from 3.09 cm to 12.07 cm at 50% duty cycle illumination levels. The framework further allows for duty cycle variation to include dimming modulations and results range from 3.62 cm to 13.15 cm at 20% duty cycle while 2.06 cm to 8.44 cm at a 78% duty cycle. Testbed results reinforce this frameworks applicability. Lastly, a novel latency constrained optimization algorithm can be overlaid on the two phase framework to decide when to simply use the coarse estimate or when to expend more computational resources on a potentially more accurate fine estimate. The creation of the two phase framework enables robust, illumination, latency sensitive positioning with the ability to be applied within a vast array of system deployment constraints

Event Driven Tactile Sensors for Artificial Devices

Present-day robots are, to some extent, able to deal with high complexity and variability of the real-world environment. Their cognitive capabilities can be further enhanced, if they physically interact and explore the real-world objects. For this, the need for efficient tactile sensors is growing day after day in such a way are becoming more and more part of daily life devices especially in robotic applications for manipulation and safe interaction with the environment. In this thesis, we highlight the importance of touch sensing in humans and robots. Inspired by the biological systems, in the the first part, we merge between neuromorphic engineering and CMOS technology where the former is a eld of science that replicates what is biologically (neurons of the nervous system) inside humans into the circuit level. We explain the operation and then characterize different sensor circuits through simulation and experiment to propose finally new prototypes based on the achieved results. In the second part, we present a machine learning technique for detecting the direction and orientation of a sliding tip over a complete skin patch of the iCub robot. Through learning and online testing, the algorithm classies different trajectories across the skin patch. Through this part, we show the results of the considered algorithm with a future perspective to extend the work

Real-time vehicle speed estimation using Unmanned Aerial Vehicles for traffic surveillance

Drones are an emerging tool for traffic surveillance; however, they inherently lack the capability to solely obtain vehicle speed on the road. This Bachelor's thesis presents the design, implementation and study of a system to detect the position, velocity and type of vehicles using the video stream obtained from drones. The solution is created to be used with any kind of aerial vehicle but is tailored for the drones in the European project LABYRINTH, of which the thesis has been a part. The tool utilizes the video feed from a sole camera and the telemetry data from the drone to detect, track and project the objects present on the road from the image into reality. This allows for an estimation of their position and speed. The detection and tracking algorithm implemented is the Simple Online Real Time algorithm, which is often referred to as SORT. Once the position has been acquired, another stream is generated that displays the same video, but with the bounding boxes, velocity and confidence ratings of all identified vehicles, with an overall computing time lower than the frame rate. After implementation, the tool underwent testing in a simulated environment to determine its assets and shortcomings, and was used during the LABYRINTH traffic monitoring flight tests. The Bachelor's thesis achieves the aimed objectives with minimum resource utilization, using readily available logic and open-source software to strike an optimal balance between real-time functionality and precise detection of vehicle position.Outgoin