Elevator‘s External Button Recognition and Detection for Vision-based System

Recently, autonomous transporter offers the assistance and delivery for user but they are only focusing on single floor environment. To widen up fields of robotic, they teach robot to use an elevator because elevator provides an essential means of faster movement across level. However, most of the mobile service robot failed to detect elevator’s position due to the complex background and reflection on the elevator door and button panel itself. This paper presents a new strategy for recognition method to detect elevator by detecting their external button efficiently. Sobel is use as edge detection operator to find the estimated absolute gradient magnitude at each point in an input grayscale image. Then, but we enhanced the technique by combining it with wiener filter to reduce the amount of noise present in a signal by comparing the signal with an estimation of the desired noiseless signal. This filter helps to eliminate the reflection image on elevator’s button panel before it can be converted to black and white image (binarization). The process followed by some morphological and structuring elements process. Tests have been done and the results shown that elevator’s external button can be recognized and detected by those entire framework

Attention and Anticipation in Fast Visual-Inertial Navigation

We study a Visual-Inertial Navigation (VIN) problem in which a robot needs to estimate its state using an on-board camera and an inertial sensor, without any prior knowledge of the external environment. We consider the case in which the robot can allocate limited resources to VIN, due to tight computational constraints. Therefore, we answer the following question: under limited resources, what are the most relevant visual cues to maximize the performance of visual-inertial navigation? Our approach has four key ingredients. First, it is task-driven, in that the selection of the visual cues is guided by a metric quantifying the VIN performance. Second, it exploits the notion of anticipation, since it uses a simplified model for forward-simulation of robot dynamics, predicting the utility of a set of visual cues over a future time horizon. Third, it is efficient and easy to implement, since it leads to a greedy algorithm for the selection of the most relevant visual cues. Fourth, it provides formal performance guarantees: we leverage submodularity to prove that the greedy selection cannot be far from the optimal (combinatorial) selection. Simulations and real experiments on agile drones show that our approach ensures state-of-the-art VIN performance while maintaining a lean processing time. In the easy scenarios, our approach outperforms appearance-based feature selection in terms of localization errors. In the most challenging scenarios, it enables accurate visual-inertial navigation while appearance-based feature selection fails to track robot's motion during aggressive maneuvers.Comment: 20 pages, 7 figures, 2 table

Self-Localization for Autonomous Driving Using Vector Maps and Multi-Modal Odometry

One of the fundamental requirements in automated driving is having accurate vehicle localization. It is because different modules such as motion planning and control require accurate location and heading of the ego-vehicle to navigate within the drivable region safely. Global Navigation Satellite Systems (GNSS) can provide the geolocation of the vehicle in different outdoor environments. However, they suffer from poor observability and even signal loss in GNSS-denied environments such as city canyons. Map-based self-localization systems are the other tools to estimate the pose of the vehicle in known environments. The main purpose of this research is to design a real-time self-localization system for autonomous driving. To provide short-term constraints over the self-localization system a multi-modal vehicle odometry algorithm is developed that fuses an Inertial Measurement Unit (IMU), a camera, a Lidar, and a GNSS through an Error-State Kalman Filter (ESKF). Additionally, a Machine-Learning (ML)-based odometry algorithm is developed to compensate for the self-localization unavailability through kernel-based regression models that fuse IMU, encoders, and a steering sensor along with recent historical measurement data. The simulation and experimental results demonstrate that the vehicle odometry can be estimated with good accuracy. Based on the main objective of the thesis, a novel computationally efficient self-localization algorithm is developed that uses geospatial information from High-Definition (HD) maps along with observation of nearby landmarks. This approach uses situation- and uncertainty-aware attention mechanisms to select “suitable” landmarks at any drivable location within the known environment based on their observability and level of uncertainty. By using landmarks that are invariant to seasonal changes and knowing “where to look” proactively, robustness and computational efficiency are improved. The developed localization system is implemented and experimentally evaluated on WATonoBus, the University of Waterloo's autonomous shuttle. The experimental results confirm excellent computational efficiency and good accuracy

Calage robuste et accéléré de nuages de points en environnements naturels via l'apprentissage automatique

En robotique mobile, un élément crucial dans la réalisation de la navigation autonome est la localisation du robot. En utilisant des scanners laser, ceci peut être réalisé en calant les nuages de points consécutifs. Pour ce faire, l’utilisation de points de repères appelés descripteurs sont généralement efficaces, car ils permettent d’établir des correspondances entre les nuages de points. Cependant, nous démontrons que dans certains environnements naturels, une proportion importante d’entre eux peut ne pas être fiable, dégradant ainsi les performances de l’alignement. Par conséquent, nous proposons de filtrer les descripteurs au préalable afin d’éliminer les nuisibles. Notre approche consiste à utiliser un algorithme d’apprentissage rapide, entraîné à la volée sous le paradigme positive and unlabeled learning sans aucune intervention humaine nécessaire. Les résultats obtenus montrent que notre approche permet de réduire significativement le nombre de descripteurs utilisés tout en augmentant la proportion de descripteurs fiables, accélérant et augmentant ainsi la robustesse de l’alignement.Localization of a mobile robot is crucial for autonomous navigation. Using laser scanners, this can be facilitated by the pairwise alignment of consecutive scans. For this purpose, landmarks called descriptors are generally effective as they facilitate point matching. However, we show that in some natural environments, many of them are likely to be unreliable. The presence of these unreliable descriptors adversely affects the performances of the alignment process. Therefore, we propose to filter unreliable descriptors as a prior step to alignment. Our approach uses a fast machine learning algorithm, trained on-the-fly under the positive and unlabeled learning paradigm without the need for human intervention. Our results show that the number of descriptors can be significantly reduced, while increasing the proportion of reliable ones, thus speeding up and improving the robustness of the scan alignment process

Problems in Control, Estimation, and Learning in Complex Robotic Systems

In this dissertation, we consider a range of different problems in systems, control, and learning theory and practice. In Part I, we look at problems in control of complex networks. In Chapter 1, we consider the performance analysis of a class of linear noisy dynamical systems. In Chapter 2, we look at the optimal design problems for these networks. In Chapter 3, we consider dynamical networks where interactions between the networks occur randomly in time. And in the last chapter of this part, in Chapter 4, we look at dynamical networks wherein coupling between the subsystems (or agents) changes nonlinearly based on the difference between the state of the subsystems. In Part II, we consider estimation problems wherein we deal with a large body of variables (i.e., at large scale). This part starts with Chapter 5, in which we consider the problem of sampling from a dynamical network in space and time for initial state recovery. In Chapter 6, we consider a similar problem with the difference that the observations instead of point samples become continuous observations that happen in Lebesgue measurable observations. In Chapter 7, we consider an estimation problem in which the location of a robot during the navigation is estimated using the information of a large number of surrounding features and we would like to select the most informative features using an efficient algorithm. In Part III, we look at active perception problems, which are approached using reinforcement learning techniques. This part starts with Chapter 8, in which we tackle the problem of multi-agent reinforcement learning where the agents communicate and classify as a team. In Chapter 9, we consider a single agent version of the same problem, wherein a layered architecture replaces the architectures of the previous chapter. Then, we use reinforcement learning to design the meta-layer (to select goals), action-layer (to select local actions), and perception-layer (to conduct classification)

The exploration of unknown environments by affective agents

Tese de doutoramento em Engenharia Informática apresentada à Fac. de Ciências e Tecnologia de CoimbraIn this thesis, we study the problem of the exploration of unknown environments populated with entities by affective autonomous agents. The goal of these agents is twofold: (i) the acquisition of maps of the environment – metric maps – to be stored in memory, where the cells occupied by the entities that populate that environment are represented; (ii) the construction of models of those entities. We examine this problem through simulations because of the various advantages this approach offers, mainly efficiency, more control, and easy focus of the research. Furthermore, the simulation approach can be used because the simplifications that we made do not influence the value of the results. With this end, we have developed a framework to build multi-agent systems comprising affective agents and then, based on this platform, we developed an application for the exploration of unknown environments. This application is a simulated multi-agent environment in which, in addition to inanimate agents (objects), there are agents interacting in a simple way, whose goal is to explore the environment. By relying on an affective component plus ideas from the Belief-Desire-Intention model, our approach to building artificial agents is that of assigning agents mentalistic qualities such as feelings, basic desires, memory/beliefs, desires/goals, and intentions. The inclusion of affect in the agent architecture is supported by the psychological and neuroscience research over the past decades which suggests that emotions and, in general, motivations play a critical role in decision-making, action, and reasoning, by influencing a variety of cognitive processes (e.g., attention, perception, planning, etc.). Reflecting the primacy of those mentalistic qualities, the architecture of an agent includes the following modules: sensors, memory/beliefs (for entities - which comprises both analogical and propositional knowledge representations -, plans, and maps of the environment), desires/goals, intentions, basic desires (basic motivations/motives), feelings, and reasoning. The key components that determine the exhibition of the exploratory behaviour in an agent are the kind of basic desires, feelings, goals and plans with which the agent is equipped. Based on solid, psychological experimental evidence, an agent is equipped in advance with the basic desires for minimal hunger, maximal information gain (maximal reduction of curiosity), and maximal surprise, as well as with the correspondent feelings of hunger, curiosity and surprise. Each one of those basic desires drives the agent to reduce or to maximize a particular feeling. The desire for minimal hunger, maximal information gain and maximal surprise directs the agent, respectively, to reduce the feeling of hunger, to reduce the feeling of curiosity (by maximizing information gain) and to maximize the feeling of surprise. The desire to reduce curiosity does not mean that the agent dislike curiosity. Instead, it means the agent desires selecting actions whose execution maximizes the reduction of curiosity, i.e., actions that are preceded by maximal levels of curiosity and followed by minimal levels of curiosity, which corresponds to maximize information gain. The intensity of these feelings is, therefore, important to compute the degree of satisfaction of the basic desires. For the basic desires of minimal hunger and maximal surprise it is given by the expected intensities of the feelings of hunger and surprise, respectively, after performing an action, while for the desire of maximal information gain it is given by the intensity of the feeling of curiosity before performing the action (this is the expected information gain). The memory of an agent is setup with goals and decision-theoretic, hierarchical task-network plans for visiting entities that populate the environment, regions of the environment, and for going to places where the agent can recharge its battery. New goals are generated for each unvisited entity of the environment, for each place in the frontier of the explored area, and for recharging battery, by adapting past goals and plans to the current world state computed based on sensorial information and on the generation of expectations and assumptions for the gaps in the environment information provided by the sensors. These new goals and respective plans are then ranked according to their Expected Utility which reflects the positive and negative relevance for the basic desires of their accomplishment. The first one, i.e., the one with highest Expected Utility is taken as an intention. Besides evaluating the computational model of surprise, we experimentally investigated through simulations the following issues: the role of the exploration strategy (role of surprise, curiosity, and hunger), environment complexity, and amplitude of the visual field on the performance of the exploration of environments populated with entities; the role of the size or, to some extent, of the diversity of the memory of entities, and environment complexity on map-building by exploitation. The main results show that: the computational model of surprise is a satisfactory model of human surprise; the exploration of unknown environments populated with entities can be robustly and efficiently performed by affective agents (the strategies that rely on hunger combined or not with curiosity or surprise outperform significantly the others, being strong contenders to the classical strategy based on entropy and cost)