Exploration and Mapping of Spatio-Temporal Pedestrian Flow Patterns for Mobile Robots

Socially compliant robot navigation is one of the key aspects for long-term acceptance of mobile robots in human-populated environments. One of the current barriers for this acceptance is that many navigation methods are based only on reactive behaviours, which can lead to frequent re-plannings, causing an erratic or aggressive robot behaviour. Instead, giving the ability to model and predict in advance how the people are likely to behave, from a long-term perspective, is an important enabler for safe and efficient navigation. For example, a robot may use its knowledge of the expected human motion to go with the main direction of flow to minimise the possibility of collisions or trajectory re-plannings. In order to provide robots with knowledge of the expected activity patterns of people at different places and times,the first main contribution of this thesis is the introduction of a Spatio-Temporal Flow map (STeF-map). This is a time-dependent probabilistic map able to model and predict the flow patterns of people in the environment. The proposed representation models the likelihood of motion directions on a grid-based map by a set of harmonic functions, which efficiently capture long-term variations of crowd movements over time. The experimental evaluation shows that the proposed model enables a better human motion prediction than spatial-only approaches and an increased capacity for socially compliant robot navigation. Obtaining this knowledge from a mobile robot platform is, however, not a trivial task, as usually they can only observe a fraction of the environment at a time, while the activity patterns of people may also change at different times. Therefore, the second main contribution is the investigation of a new methodology for mobile robot exploration to maximise the knowledge of human activity patterns, by deciding where and when to collect observations based on an exploration policy driven by the entropy levels in a spatio-temporal map. The evaluation is performed by simulating mobile robot exploration using real sensory data from three long-term pedestrian datasets, and the results show that for certain scenarios, the proposed exploration system can learn STeF-maps more quickly and better predict the flow patterns than uninformed strategies

Advances in Robot Navigation

Robot navigation includes different interrelated activities such as perception - obtaining and interpreting sensory information; exploration - the strategy that guides the robot to select the next direction to go; mapping - the construction of a spatial representation by using the sensory information perceived; localization - the strategy to estimate the robot position within the spatial map; path planning - the strategy to find a path towards a goal location being optimal or not; and path execution, where motor actions are determined and adapted to environmental changes. This book integrates results from the research work of authors all over the world, addressing the abovementioned activities and analyzing the critical implications of dealing with dynamic environments. Different solutions providing adaptive navigation are taken from nature inspiration, and diverse applications are described in the context of an important field of study: social robotics

Human robot interaction in a crowded environment

Human Robot Interaction (HRI) is the primary means of establishing natural and affective communication between humans and robots. HRI enables robots to act in a way similar to humans in order to assist in activities that are considered to be laborious, unsafe, or repetitive. Vision based human robot interaction is a major component of HRI, with which visual information is used to interpret how human interaction takes place. Common tasks of HRI include finding pre-trained static or dynamic gestures in an image, which involves localising different key parts of the human body such as the face and hands. This information is subsequently used to extract different gestures. After the initial detection process, the robot is required to comprehend the underlying meaning of these gestures [3]. Thus far, most gesture recognition systems can only detect gestures and identify a person in relatively static environments. This is not realistic for practical applications as difficulties may arise from people‟s movements and changing illumination conditions. Another issue to consider is that of identifying the commanding person in a crowded scene, which is important for interpreting the navigation commands. To this end, it is necessary to associate the gesture to the correct person and automatic reasoning is required to extract the most probable location of the person who has initiated the gesture. In this thesis, we have proposed a practical framework for addressing the above issues. It attempts to achieve a coarse level understanding about a given environment before engaging in active communication. This includes recognizing human robot interaction, where a person has the intention to communicate with the robot. In this regard, it is necessary to differentiate if people present are engaged with each other or their surrounding environment. The basic task is to detect and reason about the environmental context and different interactions so as to respond accordingly. For example, if individuals are engaged in conversation, the robot should realize it is best not to disturb or, if an individual is receptive to the robot‟s interaction, it may approach the person. Finally, if the user is moving in the environment, it can analyse further to understand if any help can be offered in assisting this user. The method proposed in this thesis combines multiple visual cues in a Bayesian framework to identify people in a scene and determine potential intentions. For improving system performance, contextual feedback is used, which allows the Bayesian network to evolve and adjust itself according to the surrounding environment. The results achieved demonstrate the effectiveness of the technique in dealing with human-robot interaction in a relatively crowded environment [7]

Navigation, Path Planning, and Task Allocation Framework For Mobile Co-Robotic Service Applications in Indoor Building Environments

Recent advances in computing and robotics offer significant potential for improved autonomy in the operation and utilization of today’s buildings. Examples of such building environment functions that could be improved through automation include: a) building performance monitoring for real-time system control and long-term asset management; and b) assisted indoor navigation for improved accessibility and wayfinding. To enable such autonomy, algorithms related to task allocation, path planning, and navigation are required as fundamental technical capabilities. Existing algorithms in these domains have primarily been developed for outdoor environments. However, key technical challenges that prevent the adoption of such algorithms to indoor environments include: a) the inability of the widely adopted outdoor positioning method (Global Positioning System - GPS) to work indoors; and b) the incompleteness of graph networks formed based on indoor environments due to physical access constraints not encountered outdoors. The objective of this dissertation is to develop general and scalable task allocation, path planning, and navigation algorithms for indoor mobile co-robots that are immune to the aforementioned challenges. The primary contributions of this research are: a) route planning and task allocation algorithms for centrally-located mobile co-robots charged with spatiotemporal tasks in arbitrary built environments; b) path planning algorithms that take preferential and pragmatic constraints (e.g., wheelchair ramps) into consideration to determine optimal accessible paths in building environments; and c) navigation and drift correction algorithms for autonomous mobile robotic data collection in buildings. The developed methods and the resulting computational framework have been validated through several simulated experiments and physical deployments in real building environments. Specifically, a scenario analysis is conducted to compare the performance of existing outdoor methods with the developed approach for indoor multi-robotic task allocation and route planning. A simulated case study is performed along with a pilot experiment in an indoor built environment to test the efficiency of the path planning algorithm and the performance of the assisted navigation interface developed considering people with physical disabilities (i.e., wheelchair users) as building occupants and visitors. Furthermore, a case study is performed to demonstrate the informed retrofit decision-making process with the help of data collected by an intelligent multi-sensor fused robot that is subsequently used in an EnergyPlus simulation. The results demonstrate the feasibility of the proposed methods in a range of applications involving constraints on both the environment (e.g., path obstructions) and robot capabilities (e.g., maximum travel distance on a single charge). By focusing on the technical capabilities required for safe and efficient indoor robot operation, this dissertation contributes to the fundamental science that will make mobile co-robots ubiquitous in building environments in the near future.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/143969/1/baddu_1.pd

Mobile Robots

The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations

Workshop on Science and the Human Exploration of Mars

The exploration of Mars will be a multi-decadal activity. Currently, a scientific program is underway, sponsored by NASA's Office of Space Science in the United States, in collaboration with international partners France, Italy, and the European Space Agency. Plans exist for the continuation of this robotic program through the first automated return of Martian samples in 2014. Mars is also a prime long-term objective for human exploration, and within NASA, efforts are being made to provide the best integration of the robotic program and future human exploration missions. From the perspective of human exploration missions, it is important to understand the scientific objectives of human missions, in order to design the appropriate systems, tools, and operational capabilities to maximize science on those missions. In addition, data from the robotic missions can provide critical environmental data - surface morphology, materials composition, evaluations of potential toxicity of surface materials, radiation, electrical and other physical properties of the Martian environment, and assessments of the probability that humans would encounter Martian life forms. Understanding of the data needs can lead to the definition of experiments that can be done in the near-term that will make the design of human missions more effective. This workshop was convened to begin a dialog between the scientific community that is central to the robotic exploration mission program and a set of experts in systems and technologies that are critical to human exploration missions. The charge to the workshop was to develop an understanding of the types of scientific exploration that would be best suited to the human exploration missions and the capabilities and limitations of human explorers in undertaking science on those missions

An intelligent multi-floor mobile robot transportation system in life science laboratories

In this dissertation, a new intelligent multi-floor transportation system based on mobile robot is presented to connect the distributed laboratories in multi-floor environment. In the system, new indoor mapping and localization are presented, hybrid path planning is proposed, and an automated doors management system is presented. In addition, a hybrid strategy with innovative floor estimation to handle the elevator operations is implemented. Finally the presented system controls the working processes of the related sub-system. The experiments prove the efficiency of the presented system

A sensor-based personal navigation system and its application for incorporating humans into a human-robot team

In this thesis methods for the sensor-based localisation of human beings are studied. The thesis presents the theory, test results and a realisation of the methods, which is called PeNa. PeNa is further applied to incorporate a human into a human-robot team that performs a simulated search and rescue task. Human-robot teamwork provides the vision for this thesis. Furthermore, the PeLoTe project and its search and rescue task provided the primary motivation for the research. However, the major part of this work and contribution is on sensor-based personal navigation. The approaches studied for personal navigation systems are based on sensor-based dead reckoning, laser-based dead reckoning, and map-based localisation. Sensor-based dead reckoning is based on heading estimation using a compass and gyro and step length estimation. Two alternative step length estimation methods are presented, ultrasound-based and accelerometer-based. Two laser dead reckoning methods are presented; a pose correlation method and a combined angle histogram matcher with position correlation. Furthermore, there are three variations for map-based localisation based on the well-known Monte Carlo Localisation (MCL): topological MCL, scan-based MCL, and a combined MCL method. As a result of the research it can be stated that it is possible to build a personal navigation system that can localise a human being indoors using only self-contained sensors. The results also show that this can be achieved using various combinations of sensors and methods. Furthermore, the personal navigation system that was developed is used to incorporate a human being into a human-robot team performing a search and rescue task. The initial results show that the location information provides a basis for creating situational awareness for a spatially distributed team

Drama, a connectionist model for robot learning: experiments on grounding communication through imitation in autonomous robots

The present dissertation addresses problems related to robot learning from demonstra¬ tion. It presents the building of a connectionist architecture, which provides the robot with the necessary cognitive and behavioural mechanisms for learning a synthetic lan¬ guage taught by an external teacher agent. This thesis considers three main issues: 1) learning of spatio-temporal invariance in a dynamic noisy environment, 2) symbol grounding of a robot's actions and perceptions, 3) development of a common symbolic representation of the world by heterogeneous agents.We build our approach on the assumption that grounding of symbolic communication creates constraints not only on the cognitive capabilities of the agent but also and especially on its behavioural capacities. Behavioural skills, such as imitation, which allow the agent to co-ordinate its actionn to that of the teacher agent, are required aside to general cognitive abilities of associativity, in order to constrain the agent's attention to making relevant perceptions, onto which it grounds the teacher agent's symbolic expression. In addition, the agent should be provided with the cognitive capacity for extracting spatial and temporal invariance in the continuous flow of its perceptions. Based on this requirement, we develop a connectionist architecture for learning time series. The model is a Dynamical Recurrent Associative Memory Architecture, called DRAMA. It is a fully connected recurrent neural network using Hebbian update rules. Learning is dynamic and unsupervised. The performance of the architecture is analysed theoretically, through numerical simulations and through physical and simulated robotic experiments. Training of the network is computationally fast and inexpensive, which allows its implementation for real time computation and on-line learning in a inexpensive hardware system. Robotic experiments are carried out with different learning tasks involving recognition of spatial and temporal invariance, namely landmark recognition and prediction of perception-action sequence in maze travelling.The architecture is applied to experiments on robot learning by imitation. A learner robot is taught by a teacher agent, a human instructor and another robot, a vocabulary to describe its perceptions and actions. The experiments are based on an imitative strategy, whereby the learner robot reproduces the teacher's actions. While imitating the teacher's movements, the learner robot makes similar proprio and exteroceptions to those of the teacher. The learner robot grounds the teacher's words onto the set of common perceptions they share. We carry out experiments in simulated and physical environments, using different robotic set-ups, increasing gradually the complexity of the task. In a first set of experiments, we study transmission of a vocabulary to designate actions and perception of a robot. Further, we carry out simulation studies, in which we investigate transmission and use of the vocabulary among a group of robotic agents. In a third set of experiments, we investigate learning sequences of the robot's perceptions, while wandering in a physically constrained environment. Finally, we present the implementation of DRAMA in Robota, a doll-like robot, which can imitate the arms and head movements of a human instructor. Through this imitative game, Robota is taught to perform and label dance patterns. Further, Robota is taught a basic language, including a lexicon and syntactical rules for the combination of words of the lexicon, to describe its actions and perception of touch onto its body

Generating timed trajectories foran autonomous robot

Tese de Doutoramento Programa Doutoral em Engenharia Electrónica e ComputadoresThe inclusion of timed movements in control architectures for mobile navigation has received an increasing attention over the last years. Timed movements allow modulat- ing the behavior of the mobile robot according to the elapsed time, such that the robot reaches a goal location within a specified time constraint. If the robot takes longer than expected to reach the goal location, its linear velocity is increased for compen- sating the delay. Timed movements are also relevant when sequences of missions are considered. The robot should follow the predefined time schedule, so that the next mission is initiated without delay. The performance of the architecture that controls the robot can be validated through simulations and field experiments. However, ex- perimental tests do not cover all the possible solutions. These should be guided by a stability analysis, which might provide directions to improve the architecture design in cases of inadequate performance of the architecture. This thesis aims at developing a navigation architecture and its stability analysis based on the Contraction Theory. The architecture is based on nonlinear dynamical systems and must guide a mobile robot, such that it reaches a goal location within a time constraint while avoiding unexpected obstacles in a cluttered and dynamic real environment. The stability analysis based on the Contraction Theory might provide conditions to the dynamical systems parameters, such that the dynamical systems are designed as contracting, ensuring the global exponential stability of the architecture. Furthermore, Contraction Theory provides solutions to analyze the success of the mis- sion as a stability problem. This provides formal results that evaluate the performance of the architecture, allowing the comparison to other navigation architectures. To verify the ability of the architecture to guide the mobile robot, several experi- mental tests were conducted. The obtained results show that the proposed architecture is able to drive mobile robots with timed movements in indoor environments for large distances without human intervention. Furthermore, the results show that the Con- traction Theory is an important tool to design stable control architectures and to analyze the success of the robotic missions as a stability problem.A inclusão de movimentos temporizados em arquitecturas de controlo para navegação móvel tem aumentado ao longo dos últimos anos. Movimentos temporizados permitem modular o comportamento do robô de tal forma que ele chegue ao seu destino dentro de um tempo especificado. Se o robô se atrasar, a sua velocidade linear deve ser aumen- tada para compensar o atraso. Estes movimentos são também importantes quando se consideram sequências de missões. O robô deve seguir o escalonamento da sequência, de tal forma que a próxima missão seja iniciada sem atraso. O desempenho da arqui- tectura pode ser validado através de simulações e experiências reais. Contudo, testes experimentais não cobrem todas as possíveis soluções. Estes devem ser conduzidos por uma análise de estabilidade, que pode fornecer direcções para melhorar o desempenho da arquitectura. O objectivo desta tese é desenvolver uma arquitectura de navegação e analisar a sua estabilidade através da teoria da Contracção. A arquitectura é baseada em sistemas dinâmicos não lineares e deve controlar o robô móvel num ambiente real, desordenado e dinâmico, de tal modo que ele chegue à posição alvo dentro de uma restrição de tempo especificada. A análise de estabilidade baseada na teoria da Contracção pode fornecer condições aos parâmetros dos sistemas dinâmicos de modo a desenha-los como contracções, e assim garantir a estabilidade exponencial global da arquitectura. Esta teoria fornece ainda soluções interessantes para analisar o sucesso da missão como um problema de estabilidade. Isto providencia resultados formais que avaliam o desem- penho da arquitectura e permitem a comparação com outras arquitecturas. Para verificar a habilidade da arquitectura em controlar o robô móvel, foram con- duzidos vários testes experimentais. Os resultados obtidos mostram que a arquitectura proposta é capaz de controlar robôs móveis com movimentos temporizados em ambi- entes interiores durante grandes distâncias e sem intervenção humana. Além disso, os resultados mostram que a teoria da Contracção é uma ferramenta importante para desenhar arquitecturas de controlo estáveis e para analisar o sucesso das missões efec- tuadas pelo robô como um problema de estabilidade.Portuguese Science and Technology Foundation (FCT) SFRH/BD/68805/2010