Active Sensing as Bayes-Optimal Sequential Decision Making

Sensory inference under conditions of uncertainty is a major problem in both machine learning and computational neuroscience. An important but poorly understood aspect of sensory processing is the role of active sensing. Here, we present a Bayes-optimal inference and control framework for active sensing, C-DAC (Context-Dependent Active Controller). Unlike previously proposed algorithms that optimize abstract statistical objectives such as information maximization (Infomax) [Butko & Movellan, 2010] or one-step look-ahead accuracy [Najemnik & Geisler, 2005], our active sensing model directly minimizes a combination of behavioral costs, such as temporal delay, response error, and effort. We simulate these algorithms on a simple visual search task to illustrate scenarios in which context-sensitivity is particularly beneficial and optimization with respect to generic statistical objectives particularly inadequate. Motivated by the geometric properties of the C-DAC policy, we present both parametric and non-parametric approximations, which retain context-sensitivity while significantly reducing computational complexity. These approximations enable us to investigate the more complex problem involving peripheral vision, and we notice that the difference between C-DAC and statistical policies becomes even more evident in this scenario.Comment: Scheduled to appear in UAI 201

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Learning to look in different environments: an active-vision model which learns and readapts visual routines

One of the main claims of the active vision framework is that finding data on the basis of task requirements is more efficient than reconstructing the whole scene by performing a complete visual scan. To be successful, this approach requires that agents learn visual routines to direct overt attention to locations with the information needed to accomplish the task. In ecological conditions, learning such visual routines is difficult due to the partial observability of the world, the changes in the environment, and the fact that learning signals might be indirect. This paper uses a reinforcement-learning actor-critic model to study how visual routines can be formed, and then adapted when the environment changes, in a system endowed with a controllable gaze and reaching capabilities. The tests of the model show that: (a) the autonomously-developed visual routines are strongly dependent on the task and the statistical properties of the environment; (b) when the statistics of the environment change, the performance of the system remains rather stable thanks to the re-use of previously discovered visual routines while the visual exploration policy remains for long time sub-optimal. We conclude that the model has a robust behaviour but the acquisition of an optimal visual exploration policy is particularly hard given its complex dependence on statistical properties of the environment, showing another of the difficulties that adaptive active vision agents must face

Multi-SLAM Systems for Fault-Tolerant Simultaneous Localization and Mapping

Mobile robots need accurate, high fidelity models of their operating environments in order to complete their tasks safely and efficiently. Generating these models is most often done via Simultaneous Localization and Mapping (SLAM), a paradigm where the robot alternatively estimates the most up-to-date model of the environment and its position relative to this model as it acquires new information from its sensors over time. Because robots operate in many different environments with different compute, memory, sensing, and form constraints, the nature and quality of information available to individual instances of different SLAM systems varies substantially. `One-size-fits-all\u27 solutions are thus exceedingly difficult to engineer, and highly specialized systems, which represent the state-of-the-art for most types of deployments, are not robust to operating conditions in which their assumptions are not met. This thesis seeks to investigate an alternative approach to these robustness and universality problems by incorporating existing SLAM solutions within a larger framework supported by planning and learning. The central idea is to combine learned models that estimate SLAM algorithm performance under a variety of sensory conditions, in this case neural networks, with planners designed for planning under uncertainty and partial observability, in this case partially observable Markov decision problems (POMDPs). Models of existing SLAM algorithms can be learned, and these models can then be used online to estimate the performance of a range of solutions to the SLAM problem at hand. The POMDP policy then selects the appropriate algorithm, given the estimated performance, cost of switching methods, and other information. This general approach may also be applicable to many other robotics problems that rely on data-fusion, such as grasp planning, motion planning, or object identification

Problems in Control, Estimation, and Learning in Complex Robotic Systems

In this dissertation, we consider a range of different problems in systems, control, and learning theory and practice. In Part I, we look at problems in control of complex networks. In Chapter 1, we consider the performance analysis of a class of linear noisy dynamical systems. In Chapter 2, we look at the optimal design problems for these networks. In Chapter 3, we consider dynamical networks where interactions between the networks occur randomly in time. And in the last chapter of this part, in Chapter 4, we look at dynamical networks wherein coupling between the subsystems (or agents) changes nonlinearly based on the difference between the state of the subsystems. In Part II, we consider estimation problems wherein we deal with a large body of variables (i.e., at large scale). This part starts with Chapter 5, in which we consider the problem of sampling from a dynamical network in space and time for initial state recovery. In Chapter 6, we consider a similar problem with the difference that the observations instead of point samples become continuous observations that happen in Lebesgue measurable observations. In Chapter 7, we consider an estimation problem in which the location of a robot during the navigation is estimated using the information of a large number of surrounding features and we would like to select the most informative features using an efficient algorithm. In Part III, we look at active perception problems, which are approached using reinforcement learning techniques. This part starts with Chapter 8, in which we tackle the problem of multi-agent reinforcement learning where the agents communicate and classify as a team. In Chapter 9, we consider a single agent version of the same problem, wherein a layered architecture replaces the architectures of the previous chapter. Then, we use reinforcement learning to design the meta-layer (to select goals), action-layer (to select local actions), and perception-layer (to conduct classification)

Estimation and Control of Robotic Radiation-Based Processes

This dissertation presents a closed-loop control and state estimation framework for a class of distributed-parameter processes employing a moving radiant actuator. These radiation-based processes have the potential to significantly reduce the energy consumption and environmental impact of traditional industrial processes. Successful implementation of these approaches in large-scale applications requires precise control systems. This dissertation provides a comprehensive framework for: 1) integration of trajectory generation and feedback control, 2) online distributed state and parameter estimation, and 3) optimal coordination of multiple manipulated variables, so as to achieve elaborate control of these radiation-based processes for improved process quality and energy efficiency. The developed framework addresses important issues for estimation and control of processes employing a moving radiant actuator from both practical and theoretical aspects. For practical systems, an integrated motion and process control approach is first developed to compensate for disturbances by adjusting either the radiant power of the actuator or the speed of the robot end effector based on available process measurements, such as temperature distribution. The control problem is then generalized by using a 1D scanning formulation that describes common characteristics of typical radiant source actuated processes. Based on this 1D scanning formulation, a distributed state and parameter estimation scheme that incorporates a dual extended Kalman filter (DEKF) approach is developed to provide real-time process estimation. In this estimation scheme, an activating policy accompanying the moving actuator is applied in order to reduce the computational cost and compensate for observability changes caused by the actuator\u27s movement. To achieve further improvements in process quality, a static optimization and a rule-based feedback control strategy are used to coordinate multiple manipulated variables in open-loop and closed-loop manners. Finally, a distributed model predictive control (MPC) framework is developed to integrate process optimization and closed-loop coordination of manipulated variables. Simulation studies conducted on a robotic ultraviolet (UV) paint curing process show that the developed estimation and control framework for radiant source actuated processes provide improved process quality and energy efficiency by adaptively compensating for disturbances and optimally coordinating multiple manipulated variables

Vision-based control of multi-agent systems

Scope and Methodology of Study: Creating systems with multiple autonomous vehicles places severe demands on the design of decision-making supervisors, cooperative control schemes, and communication strategies. In last years, several approaches have been developed in the literature. Most of them solve the vehicle coordination problem assuming some kind of communications between team members. However, communications make the group sensitive to failure and restrict the applicability of the controllers to teams of friendly robots. This dissertation deals with the problem of designing decentralized controllers that use just local sensor information to achieve some group goals.Findings and Conclusions: This dissertation presents a decentralized architecture for vision-based stabilization of unmanned vehicles moving in formation. The architecture consists of two main components: (i) a vision system, and (ii) vision-based control algorithms. The vision system is capable of recognizing and localizing robots. It is a model-based scheme composed of three main components: image acquisition and processing, robot identification, and pose estimation.Using vision information, we address the problem of stabilizing groups of mobile robots in leader- or two leader-follower formations. The strategies use relative pose between a robot and its designated leader or leaders to achieve formation objectives. Several leader-follower formation control algorithms, which ensure asymptotic coordinated motion, are described and compared. Lyapunov's stability theory-based analysis and numerical simulations in a realistic tridimensional environment show the stability properties of the control approaches

Robust state estimation for the control of flexible robotic manipulators

In this thesis, a novel robust estimation strategy for observing the system state variables of robotic manipulators with distributed flexibility is established. Motivation for the derived approach stems from the observation that lightweight, high speed, and large workspace robotic manipulators often suffer performance degradation because of inherent structural compliance. This flexibility often results in persistent residual vibration, which must be damped before useful work can resume. Inherent flexibility in robotic manipulators, then, increases cycle times and shortens the operational lives of the robots. Traditional compensation techniques, those which are commonly used for the control of rigid manipulators, can only approach a fraction of the open-loop system bandwidth without inducing significant excitation of the resonant dynamics. To improve the performance of these systems, the structural flexibility cannot simply be ignored, as it is when the links are significantly stiff and approximate rigid bodies. One thus needs a model to design a suitable compensator for the vibration, but any model developed to correct this problem will contain parametric error. And in the case of very lightly damped systems, like flexible robotic manipulators, this error can lead to instability of the control system for even small errors in system parameters. This work presents a systematic solution for the problem of robust state estimation for flexible manipulators in the presence of parametric modeling error. The solution includes: 1) a modeling strategy, 2) sensor selection and placement, and 3) a novel, multiple model estimator. Modeling of the FLASHMan flexible gantry manipulator is accomplished using a developed hybrid transfer matrix / assumed modes method (TMM/AMM) approach to determine an accurate low-order state space representation of the system dynamics. This model is utilized in a genetic algorithm optimization in determining the placement of MEMs accelerometers for robust estimation and observability of the system’s flexible state variables. The initial estimation method applied to the task of determining robust state estimates under conditions of parametric modeling error was of a sliding mode observer type. Evaluation of the method through analysis, simulations and experiments showed that the state estimates produced were inadequate. This led to the development of a novel, multiple model adaptive estimator. This estimator utilizes a bank of similarly designed sub-estimators and a selection algorithm to choose the true value from a given set of possible system parameter values as well as the correct state vector estimate. Simulation and experimental results are presented which demonstrate the applicability and effectiveness of the derived method for the task of state variable estimation for flexible robotic manipulators.Ph.D

A Survey on Aerial Swarm Robotics

The use of aerial swarms to solve real-world problems has been increasing steadily, accompanied by falling prices and improving performance of communication, sensing, and processing hardware. The commoditization of hardware has reduced unit costs, thereby lowering the barriers to entry to the field of aerial swarm robotics. A key enabling technology for swarms is the family of algorithms that allow the individual members of the swarm to communicate and allocate tasks amongst themselves, plan their trajectories, and coordinate their flight in such a way that the overall objectives of the swarm are achieved efficiently. These algorithms, often organized in a hierarchical fashion, endow the swarm with autonomy at every level, and the role of a human operator can be reduced, in principle, to interactions at a higher level without direct intervention. This technology depends on the clever and innovative application of theoretical tools from control and estimation. This paper reviews the state of the art of these theoretical tools, specifically focusing on how they have been developed for, and applied to, aerial swarms. Aerial swarms differ from swarms of ground-based vehicles in two respects: they operate in a three-dimensional space and the dynamics of individual vehicles adds an extra layer of complexity. We review dynamic modeling and conditions for stability and controllability that are essential in order to achieve cooperative flight and distributed sensing. The main sections of this paper focus on major results covering trajectory generation, task allocation, adversarial control, distributed sensing, monitoring, and mapping. Wherever possible, we indicate how the physics and subsystem technologies of aerial robots are brought to bear on these individual areas

Formation and reconfiguration control for multi-robotic systems

Master'sMASTER OF ENGINEERIN