Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Coordinated Unmanned Aerial Vehicles for Surveillance of Targets

PhDThis thesis investigates the coordination approaches of multiple mobile and autonomous robots, especially resource-limited small-scale UAVs, for the surveillance of pre-de ned ground targets in a given environment. A key research issue in surveillance task is the coordination among the robots to determine the target's time varying locations. The research focuses on two applications of surveillance: (i) cooperative search of stationary targets, and (ii) cooperative observation of moving targets. The objective in cooperative search is to minimize the time and errors in nding the locations of stationary targets. The objective of cooperative observation is to maximize the collective time and quality of observation of moving targets. The thesis presents a survey of the approaches in a larger domain of multi-robot systems for the surveillance of pre-de ned targets in a given environment. This survey identi es various factors and application scenarios that a ect the performance of multi-robot surveillance systems. The thesis proposes a distributed strategy for merging delayed and incomplete information, which is a result of sensing and communication limitations, collected by di erent UAVs. An analytic derivation of the number of required observations is provided to declare the absence or existence of a target in a region. This number of required observations is integrated into an iterative use of Travelling Salesman Problem (TSP) and Multiple Travelling Salesmen Problem (MTSP) for autonomous path planning of UAVs. Additionally, it performs an exploration of the algorithmic design space and analyzes the e ects of centralized and distributed coordination on the cooperative search of stationary targets in the presence of sensing and communication limitations. The thesis also proposes the application of UAVs for observing multiple moving targets with di erent resolutions. A key contribution is to use the quad-tree data-structure for modelling the environment and movement of UAVs. This modelling has helped in the dynamic sensor placement of UAVs to maximize the observation of the number of moving targets as well as the resolution of observation.European Regional Development Fund and the Carinthian Economic Promotion Fund (KWF) under grant 20214/21530/32602

Exploiting Heterogeneity in Networks of Aerial and Ground Robotic Agents

By taking advantage of complementary communication technologies, distinct sensing functionalities and varied motion dynamics present in a heterogeneous multi-robotic network, it is possible to accomplish a main mission objective by assigning specialized sub-tasks to specific members of a robotic team. An adequate selection of the team members and an effective coordination are some of the challenges to fully exploit the unique capabilities that these types of systems can offer. Motivated by real world applications, we focus on a multi-robotic network consisting off aerial and ground agents which has the potential to provide critical support to humans in complex settings. For instance, aerial robotic relays are capable of transporting small ground mobile sensors to expand the communication range and the situational awareness of first responders in hazardous environments. In the first part of this dissertation, we extend work on manipulation of cable-suspended loads using aerial robots by solving the problem of lifting the cable-suspended load from the ground before proceeding to transport it. Since the suspended load-quadrotor system experiences switching conditions during this critical maneuver, we define a hybrid system and show that it is differentially-flat. This property facilitates the design of a nonlinear controller which tracks a waypoint-based trajectory associated with the discrete states of the hybrid system. In addition, we address the case of unknown payload mass by combining a least-squares estimation method with the designed controller. Second, we focus on the coordination of a heterogeneous team formed by a group of ground mobile sensors and a flying communication router which is deployed to sense areas of interest in a cluttered environment. Using potential field methods, we propose a controller for the coordinated mobility of the team to guarantee inter-robot and obstacle collision avoidance as well as connectivity maintenance among the ground agents while the main goal of sensing is carried out. For the case of the aerial communications relays, we combine antenna diversity with reinforcement learning to dynamically re-locate these relays so that the received signal strength is maintained above a desired threshold. Motivated by the recent interest of combining radio frequency and optical wireless communications, we envision the implementation of an optical link between micro-scale aerial and ground robots. This type of link requires maintaining a sufficient relative transmitter-receiver position for reliable communications. In the third part of this thesis, we tackle this problem. Based on the link model, we define a connectivity cone where a minimum transmission rate is guaranteed. For example, the aerial robot has to track the ground vehicle to stay inside this cone. The control must be robust to noisy measurements. Thus, we use particle filters to obtain a better estimation of the receiver position and we design a control algorithm for the flying robot to enhance the transmission rate. Also, we consider the problem of pairing a ground sensor with an aerial vehicle, both equipped with a hybrid radio-frequency/optical wireless communication system. A challenge is positioning the flying robot within optical range when the sensor location is unknown. Thus, we take advantage of the hybrid communication scheme by developing a control strategy that uses the radio signal to guide the aerial platform to the ground sensor. Once the optical-based signal strength has achieved a certain threshold, the robot hovers within optical range. Finally, we investigate the problem of building an alliance of agents with different skills in order to satisfy the requirements imposed by a given task. We find this alliance, known also as a coalition, by using a bipartite graph in which edges represent the relation between agent capabilities and required resources for task execution. Using this graph, we build a coalition whose total capability resources can satisfy the task resource requirements. Also, we study the heterogeneity of the formed coalition to analyze how it is affected for instance by the amount of capability resources present in the agents

Task-adaptable, Pervasive Perception for Robots Performing Everyday Manipulation

Intelligent robotic agents that help us in our day-to-day chores have been an aspiration of robotics researchers for decades. More than fifty years since the creation of the first intelligent mobile robotic agent, robots are still struggling to perform seemingly simple tasks, such as setting or cleaning a table. One of the reasons for this is that the unstructured environments these robots are expected to work in impose demanding requirements on a robota s perception system. Depending on the manipulation task the robot is required to execute, different parts of the environment need to be examined, the objects in it found and functional parts of these identified. This is a challenging task, since the visual appearance of the objects and the variety of scenes they are found in are large. This thesis proposes to treat robotic visual perception for everyday manipulation tasks as an open question-asnswering problem. To this end RoboSherlock, a framework for creating task-adaptable, pervasive perception systems is presented. Using the framework, robot perception is addressed from a systema s perspective and contributions to the state-of-the-art are proposed that introduce several enhancements which scale robot perception toward the needs of human-level manipulation. The contributions of the thesis center around task-adaptability and pervasiveness of perception systems. A perception task-language and a language interpreter that generates task-relevant perception plans is proposed. The task-language and task-interpreter leverage the power of knowledge representation and knowledge-based reasoning in order to enhance the question-answering capabilities of the system. Pervasiveness, a seamless integration of past, present and future percepts, is achieved through three main contributions: a novel way for recording, replaying and inspecting perceptual episodic memories, a new perception component that enables pervasive operation and maintains an object belief state and a novel prospection component that enables robots to relive their past experiences and anticipate possible future scenarios. The contributions are validated through several real world robotic experiments that demonstrate how the proposed system enhances robot perception

Autonomous ground vehicles in urban last-mile delivery : an exploration of the implementation feasibility and consumer’s acceptance

E-Commerce has rapidly changed the urban last-mile delivery in recent years, and Courier-, Express- and Parcel (CEP) companies are challenged by the increasing demand. Service robotics with autonomous vehicles are subject to be the catalyst for transforming the industry. Considering the infancy and lack of research on the subject, the purpose of this study is to explore the concept of autonomous ground vehicles (AGVs) in urban last-mile delivery from two perspectives. First, data about the industry and insights from the technology provider summarize the status quo of recent developments and implementation barriers with the help of expert interviews. The findings show obstacles in the technological maturity and regulatory framework. Moreover, although only road-AGVs (rAGVs) will significantly change the industry, sidewalk-AGVs (sAGVs) act as a proof of concept as the implementation is more feasible. In addition, they create new premium services for the consumers. Second, an attempt to determine the consumer’s acceptance of sAGVs, using the combination of the technology acceptance model and the technology readiness index, is made with an online survey. The proposed research model is analysed by means of simple regression analysis, and all hypotheses are supported. The majority of the respondents have a positive attitude towards the concept of sAGVs for delivery and consider using it when the safety of their delivery goods is guaranteed. This dissertation enriches the literature on human-robot acceptance as well as the management of CEP-companies to increase the engagement in the implementation of sidewalk-AGVs to increase service innovation for consumers.O comércio electrónico mudou rapidamente a entrega urbana de bens ao consumidor, e as empresas de Correio Expresso Urgente são desafiadas pela procura crescente. Os serviços robóticos com veículos autónomos serão provavelmente o catalisador da transformação desta indústria. Considerando a falta e o estágio inicial de investigação, este estudo explora o conceito de veículos autónomos terrestres (AGVs) na entrega urbana de bens ao consumidor considerando duas perspetivas. Uma primeira será a de recolher dados sobre a indústria e insights de fornecedores da tecnologia, sumarizando os mais recentes desenvolvimentos e as barreiras à implementação, com a ajuda de entrevistas a especialistas. Os resultados revelam obstáculos na maturidade tecnológica e enquadramento regulamentar. Adicionalmente, embora apenas os AGVs rodoviários (rAGVs) virão a alterar significativamente a indústria, os AGVs de passeio (sAGVs) atuam como prova de conceito, dada a sua implementação viável. Em segundo lugar, a aceitação de sAGVs por parte do consumidor é determinada através da combinação de modelos de aceitação tecnológica e do índex de prontidão de tecnologia, via questionário online. O modelo de investigação proposto é testado por meio de análise de regressão simples, e todas as hipóteses são suportadas. A maioria dos participantes tem uma atitude positiva em relação aos sAGVs para entrega, e considera usá-los se a segurança dos seus bens for garantida. Esta dissertação enriquece a literatura sobre aceitação humana-robot, bem como a gestão de empresas de Correio Expresso Urgente, aumentando o envolvimento na implementação de sAGVs e fomentando a inovação em serviços para o consumidor

Statistical models and decision making for robotic scientific information gathering

Submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution September 2018.Mobile robots and autonomous sensors have seen increasing use in scientific applications, from planetary rovers surveying for signs of life on Mars, to environmental buoys measuring and logging oceanographic conditions in coastal regions. This thesis makes contributions in both planning algorithms and model design for autonomous scientific information gathering, demonstrating how theory from machine learning, decision theory, theory of optimal experimental design, and statistical inference can be used to develop online algorithms for robotic information gathering that are robust to modeling errors, account for spatiotemporal structure in scientific data, and have probabilistic performance guarantees. This thesis first introduces a novel sample selection algorithm for online, irrevocable sampling in data streams that have spatiotemporal structure, such as those that commonly arise in robotics and environmental monitoring. Given a limited sampling capacity, the proposed periodic secretary algorithm uses an information-theoretic reward function to select samples in real-time that maximally reduce posterior uncertainty in a given scientific model. Additionally, we provide a lower bound on the quality of samples selected by the periodic secretary algorithm by leveraging the submodularity of the information-theoretic reward function. Finally, we demonstrate the robustness of the proposed approach by employing the periodic secretary algorithm to select samples irrevocably from a seven-year oceanographic data stream collected at the Martha’s Vineyard Coastal Observatory off the coast of Cape Cod, USA. Secondly, we consider how scientific models can be specified in environments – such as the deep sea or deep space – where domain scientists may not have enough a priori knowledge to formulate a formal scientific model and hypothesis. These domains require scientific models that start with very little prior information and construct a model of the environment online as observations are gathered. We propose unsupervised machine learning as a technique for science model-learning in these environments. To this end, we introduce a hybrid Bayesian-deep learning model that learns a nonparametric topic model of a visual environment. We use this semantic visual model to identify observations that are poorly explained in the current model, and show experimentally that these highly perplexing observations often correspond to scientifically interesting phenomena. On a marine dataset collected by the SeaBED AUV on the Hannibal Sea Mount, images of high perplexity in the learned model corresponded, for example, to a scientifically novel crab congregation in the deep sea. The approaches presented in this thesis capture the depth and breadth of the problems facing the field of autonomous science. Developing robust autonomous systems that enhance our ability to perform exploratory science in environments such as the oceans, deep space, agricultural and disaster-relief zones will require insight and techniques from classical areas of robotics, such as motion and path planning, mapping, and localization, and from other domains, including machine learning, spatial statistics, optimization, and theory of experimental design. This thesis demonstrates how theory and practice from these diverse disciplines can be unified to address problems in autonomous scientific information gathering

Statistical models and decision making for robotic scientific information gathering

Thesis: S.M., Joint Program in Applied Ocean Physics and Engineering (Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science; and the Woods Hole Oceanographic Institution), 2018.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 97-107).Mobile robots and autonomous sensors have seen increasing use in scientific applications, from planetary rovers surveying for signs of life on Mars, to environmental buoys measuring and logging oceanographic conditions in coastal regions. This thesis makes contributions in both planning algorithms and model design for autonomous scientific information gathering, demonstrating how theory from machine learning, decision theory, theory of optimal experimental design, and statistical inference can be used to develop online algorithms for robotic information gathering that are robust to modeling errors, account for spatiotemporal structure in scientific data, and have probabilistic performance guarantees. This thesis first introduces a novel sample selection algorithm for online, irrevocable sampling in data streams that have spatiotemporal structure, such as those that commonly arise in robotics and environmental monitoring. Given a limited sampling capacity, the proposed periodic secretary algorithm uses an information-theoretic reward function to select samples in real-time that maximally reduce posterior uncertainty in a given scientific model. Additionally, we provide a lower bound on the quality of samples selected by the periodic secretary algorithm by leveraging the submodularity of the information-theoretic reward function. Finally, we demonstrate the robustness of the proposed approach by employing the periodic secretary algorithm to select samples irrevocably from a seven-year oceanographic data stream collected at the Martha's Vineyard Coastal Observatory off the coast of Cape Cod, USA. Secondly, we consider how scientific models can be specified in environments - such as the deep sea or deep space - where domain scientists may not have enough a priori knowledge to formulate a formal scientific model and hypothesis. These domains require scientific models that start with very little prior information and construct a model of the environment online as observations are gathered. We propose unsupervised machine learning as a technique for science model-learning in these environments. To this end, we introduce a hybrid Bayesian-deep learning model that learns a nonparametric topic model of a visual environment. We use this semantic visual model to identify observations that are poorly explained in the current model, and show experimentally that these highly perplexing observations often correspond to scientifically interesting phenomena. On a marine dataset collected by the SeaBED AUV on the Hannibal Sea Mount, images of high perplexity in the learned model corresponded, for example, to a scientifically novel crab congregation in the deep sea. The approaches presented in this thesis capture the depth and breadth of the problems facing the field of autonomous science. Developing robust autonomous systems that enhance our ability to perform exploratory science in environments such as the oceans, deep space, agricultural and disaster-relief zones will require insight and techniques from classical areas of robotics, such as motion and path planning, mapping, and localization, and from other domains, including machine learning, spatial statistics, optimization, and theory of experimental design. This thesis demonstrates how theory and practice from these diverse disciplines can be unified to address problems in autonomous scientific information gathering.by Genevieve Elaine Flaspohler.S.M