Topological Mapping and Navigation in Real-World Environments

We introduce the Hierarchical Hybrid Spatial Semantic Hierarchy (H2SSH), a hybrid topological-metric map representation. The H2SSH provides a more scalable representation of both small and large structures in the world than existing topological map representations, providing natural descriptions of a hallway lined with offices as well as a cluster of buildings on a college campus. By considering the affordances in the environment, we identify a division of space into three distinct classes: path segments afford travel between places at their ends, decision points present a choice amongst incident path segments, and destinations typically exist at the start and end of routes. Constructing an H2SSH map of the environment requires understanding both its local and global structure. We present a place detection and classification algorithm to create a semantic map representation that parses the free space in the local environment into a set of discrete areas representing features like corridors, intersections, and offices. Using these areas, we introduce a new probabilistic topological simultaneous localization and mapping algorithm based on lazy evaluation to estimate a probability distribution over possible topological maps of the global environment. After construction, an H2SSH map provides the necessary representations for navigation through large-scale environments. The local semantic map provides a high-fidelity metric map suitable for motion planning in dynamic environments, while the global topological map is a graph-like map that allows for route planning using simple graph search algorithms. For navigation, we have integrated the H2SSH with Model Predictive Equilibrium Point Control (MPEPC) to provide safe and efficient motion planning for our robotic wheelchair, Vulcan. However, navigation in human environments entails more than safety and efficiency, as human behavior is further influenced by complex cultural and social norms. We show how social norms for moving along corridors and through intersections can be learned by observing how pedestrians around the robot behave. We then integrate these learned norms with MPEPC to create a socially-aware navigation algorithm, SA-MPEPC. Through real-world experiments, we show how SA-MPEPC improves not only Vulcan’s adherence to social norms, but the adherence of pedestrians interacting with Vulcan as well.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144014/1/collinej_1.pd

Adaptive shared-control of a robotic walker to improve human-robot cooperation in gait biomechanical rehabilitation

Dissertação de mestrado integrado em Engenharia Biomédica (especialização em Eletrónica Médica)Sessões de reabilitação de pacientes com deficiências na marcha é importante para que a qualidade de vida dos mesmos seja recuperada. Quando auxiliadas por andarilhos robóticos inteligentes as sessões têm mostrado melhorias significativas, face aos resultados obtidos por métodos clássicos. O andarilho WALKit é um dos dispositivos mencionados e permite ser conduzido por parte do paciente enquanto um especialista supervisiona todo o processo de forma a evitar colisões e quedas. Este processo de supervisão é moroso e requer constante presença de um especialista para cada paciente. Nesta dissertação é proposto um controlador autónomo e inteligente capaz de partilhar a condução do andarilho pelo paciente e pelo supervisor evitando colisões com obstáculos. Para remover a necessidade constante do médico supervisor, um módulo de condução autónoma foi desenvolvido. O modo autónomo proposto usa um sensor Light Detection and Ranging e o algoritmo de Simultaneous Localization and Mapping (Cartographer) para obter mapas e a localização do andarilho. Seguidamente, os planeadores global e local , A* e Dynamic Window Approach respetivamente, traçam caminhos válidos para o destino, interpretáveis pelo andarilho. Usando o modo autónomo como especialista e as intenções do paciente, o controlador partilhado usa o algoritmo Proximal Policy Optimization, aprendendo o comportamento pretendido através de um processo de tentiva e erro, maximizando a recompensa recebida através de uma função pré-estabelecida. Uma rede neuronal com camadas convolucionais e lineares é capaz de inferir o risco enfrentado pelo sistema paciente-WALKit e determinar se o modo autónomo deve assumir controlo de forma a neutralizar o risco mencionado. Globalmente foram detetados erros inferiores a 38 cm no sistema de mapeamento e localização. Quer nos cenários de testagem do controlador autónomo, quer nos do controlador partilhado, nenhuma colisão foi registada garantindo em todas as tentativas a chegada ao destino escolhido. O modo autónomo, apesar de evitar obstáculos, não foi capaz de alcançar certos destinos não contemplados em ambientes de reabilitação. O modo partilhado mostrou também certas transições bruscas entre modo autónomo e intenção que podem comprometer a segurança do paciente. É necessário, como trabalho futuro, estabelecer métricas de validação objetivas e testar o controlador com pacientes de forma a corretamente estimar o desempenho.Rehabilitation sessions of patients with gait disabilities is important to restore quality of life. When aided by intelligent robotic walkers the sessions have shown significant improvements when compared to the results obtained by classical methods. The WALKit walker is one of the devices mentioned and allows the patient to drive it while a medical expert supervises the entire process in order to avoid collisions and falls. This supervision process takes time and requires constant presence of a medical expert for each patient. This dissertation proposes an intelligent controller capable of sharing the walker’s drivability by the patient and the supervisor, avoiding collisions with obstacles. To remove the constant need of a supervisor, an autonomous driving module was developed. The proposed autonomous mode uses a Light Detection and Ranging sensor and the Simultaneous Localization and Mapping (Cartographer ) algorithm to obtain maps and the location of the walker. Then, the global and local planners, A * and Dynamic Window Approach respectively, draw valid paths to the destination, interpretable by the walker. Using the autonomous mode as a expert and the patient’s intentions, the SC uses the Proximal Policy Optimization algorithm, learning the intended behavior through a trial and error process, maximizing the reward received through a pre-established function. One neural network with convolutional and linear layers is able to infer the risk faced by the patient-WALKit system and determine whether the autonomous mode should take control in order to neutralize the mentioned risk. Globally, errors smaller than 38 cm were detected in the mapping and localization system. In the testing scenarios of the autonomous controller and in the SC no collisions were recorded guaranteeing the arrival at the chosen destination in all attempts. The autonomous mode, despite avoiding obstacles, was not able to reach certain destinations not covered in rehabilitation environments. The shared mode has also shown certain sudden transitions between autonomous mode and intention that could compromise patient safety. It is necessary, as future work, to establish objective validation metrics and testing the controller with patients is necessary in order to correctly estimate performance

Taming Numbers and Durations in the Model Checking Integrated Planning System

The Model Checking Integrated Planning System (MIPS) is a temporal least commitment heuristic search planner based on a flexible object-oriented workbench architecture. Its design clearly separates explicit and symbolic directed exploration algorithms from the set of on-line and off-line computed estimates and associated data structures. MIPS has shown distinguished performance in the last two international planning competitions. In the last event the description language was extended from pure propositional planning to include numerical state variables, action durations, and plan quality objective functions. Plans were no longer sequences of actions but time-stamped schedules. As a participant of the fully automated track of the competition, MIPS has proven to be a general system; in each track and every benchmark domain it efficiently computed plans of remarkable quality. This article introduces and analyzes the most important algorithmic novelties that were necessary to tackle the new layers of expressiveness in the benchmark problems and to achieve a high level of performance. The extensions include critical path analysis of sequentially generated plans to generate corresponding optimal parallel plans. The linear time algorithm to compute the parallel plan bypasses known NP hardness results for partial ordering by scheduling plans with respect to the set of actions and the imposed precedence relations. The efficiency of this algorithm also allows us to improve the exploration guidance: for each encountered planning state the corresponding approximate sequential plan is scheduled. One major strength of MIPS is its static analysis phase that grounds and simplifies parameterized predicates, functions and operators, that infers knowledge to minimize the state description length, and that detects domain object symmetries. The latter aspect is analyzed in detail. MIPS has been developed to serve as a complete and optimal state space planner, with admissible estimates, exploration engines and branching cuts. In the competition version, however, certain performance compromises had to be made, including floating point arithmetic, weighted heuristic search exploration according to an inadmissible estimate and parameterized optimization

Navegação multi-objetivo de um robô móvel usando aprendizagem por reforço hierárquica

Currently, there is a growing interest in the development of autonomous navigation technologies for applications in domestic, urban and industrial environments. Machine Learning tools such as neural networks, reinforcement learning and deep learning have been the main choice to solve many problems associated with autonomous mobile robot navigation. This dissertation mainly focus on solving the problem of mobile robot navigation in maze-like environments with multiple goals. The center point here is to apply a hierarchical structure of reinforcement learning algorithms (QLearning and R-Learning) to a robot in a continuous environment so that it can navigate in a maze. Both the state-space and the action-space are obtained by discretizing the data collected by the robot in order to prevent them from being too large. The implementation is done with a hierarchical approach, which is a structure that allows to split the complexity of the problem into many easier sub-problems, ending up with a set of lower-level tasks followed by a higher-level one. The robot performance is evaluated in two maze-like environments, showing that the hierarchical approach is a very feasible solution to reduce the complexity of the problem. Besides that, two more scenarios are presented: a multi-goal situation where the robot navigates across multiple goals relying on the topological representation of the environment and the experience memorized during learning and a dynamic behaviour situation where the robot must adapt its policies according to the changes that happen in the environment (such as blocked paths). In the end, both scenarios were successfully accomplished and it has been concluded that a hierarchical approach has many advantages when compared to a classic reinforcement learning approach.Atualmente, há um crescente interesse no desenvolvimento de tecnologias de navegação autónoma para aplicações em ambientes domésticos, urbanos e industriais. Ferramentas de Aprendizagem Automática, como redes neurais, aprendizagem por reforço e aprendizagem profunda têm sido a escolha principal para resolver muitos problemas associados à navegação autónoma de robôs móveis. Esta dissertação tem como foco principal a solução do problema de navegação de robôs móveis em ambientes tipo labirínto com múltiplos objetivos. O ponto central aqui é aplicar uma estrutura hierárquica de algoritmos de aprendizagem por reforço (Q-Learning e R-Learning) a um robô num ambiente contínuo para que ele possa navegar num labirinto. Tanto o espaço de estados quanto o espaço de ações são obtidos através da discretização dos dados recolhidos pelo robô para evitar que estes sejam demasiado extensos. A implementação é feita com uma abordagem hierárquica, que é uma estrutura que permite dividir a complexidade do problema em vários subproblemas mais fáceis, ficando com um conjunto de tarefas de baixo-nível seguido por um de alto-nível. O desempenho do robô é avaliado em dois ambientes tipo labirinto, mostrando que a abordagem hierárquica é uma solução bastante viável para reduzir a complexidade do problema. Além disso, dois cenários diferentes são apresentados: uma situação de multi-objetivo onde o robô navega por múltiplos objetivos usando a representação topológica do ambiente e a experiência memorizada durante a aprendizagem e uma situação de comportamento dinâmico onde o robô deve adaptar suas políticas de acordo com os mudanças que acontecem no ambiente (como caminhos bloqueados). No final, ambos os cenários foram realizados com sucesso e concluiu-se que uma abordagem hierárquica tem muitas vantagens quando comparada a uma abordagem de aprendizagem por reforço clássica.Mestrado em Engenharia de Computadores e Telemátic

An Extension of BIM Using AI: a Multi Working-Machines Pathfinding Solution

Multi working-machines pathfinding solution enables more mobile machines simultaneously to work inside of a working site so that the productivity can be expected to increase evolutionary. To date, the potential cooperation conflicts among construction machinery limit the amount of construction machinery investment in a concrete working site. To solve the cooperation problem, civil engineers optimize the working site from a logistic perspective while computer scientists improve pathfinding algorithms’ performance on the given benchmark maps. In the practical implementation of a construction site, it is sensible to solve the problem with a hybrid solution; therefore, in our study, we proposed an algorithm based on a cutting-edge multi-pathfinding algorithm to enable the massive number of machines cooperation and offer the advice to modify the unreasonable part of the working site in the meantime. Using the logistic information from BIM, such as unloading and loading point, we added a pathfinding solution for multi machines to improve the whole construction fleet’s productivity. In the previous study, the experiments were limited to no more than ten participants, and the computational time to gather the solution was not given; thus, we publish our pseudo-code, our tested map, and benchmark our results. Our algorithm’s most extensive feature is that it can quickly replan the path to overcome the emergency on a construction site

Improving Traffic Safety and Efficiency by Adaptive Signal Control Systems Based on Deep Reinforcement Learning

As one of the most important Active Traffic Management strategies, Adaptive Traffic Signal Control (ATSC) helps improve traffic operation of signalized arterials and urban roads by adjusting the signal timing to accommodate real-time traffic conditions. Recently, with the rapid development of artificial intelligence, many researchers have employed deep reinforcement learning (DRL) algorithms to develop ATSCs. However, most of them are not practice-ready. The reasons are two-fold: first, they are not developed based on real-world traffic dynamics and most of them require the complete information of the entire traffic system. Second, their impact on traffic safety is always a concern by researchers and practitioners but remains unclear. Aiming at making the DRL-based ATSC more implementable, existing traffic detection systems on arterials were reviewed and investigated to provide high-quality data feeds to ATSCs. Specifically, a machine-learning frameworks were developed to improve the quality of and pedestrian and bicyclist\u27s count data. Then, to evaluate the effectiveness of DRL-based ATSC on the real-world traffic dynamics, a decentralized network-level ATSC using multi-agent DRL was developed and evaluated in a simulated real-world network. The evaluation results confirmed that the proposed ATSC outperforms the actuated traffic signals in the field in terms of travel time reduction. To address the potential safety issue of DRL based ATSC, an ATSC algorithm optimizing simultaneously both traffic efficiency and safety was proposed based on multi-objective DRL. The developed ATSC was tested in a simulated real-world intersection and it successfully improved traffic safety without deteriorating efficiency. In conclusion, the proposed ATSCs are capable of effectively controlling real-world traffic and benefiting both traffic efficiency and safety

Application of Data Mining in Air Traffic Forecasting

The main goal of the study centers on developing a model for the purpose of air traffic forecasting by using off-the-shelf data mining and machine learning techniques. Although data driven modeling has been extensively applied in the aviation sector, little research has been done in the area of air traffic forecasting. This study is inspired by previous research focused on improving the Federal Aviation Administration (FAA) Terminal Area Forecasting (TAF) methodology, which historically assumed that the US air transportation system (ATS) network structure was static. Recent developments use data mining algorithms to predict the likelihood of previously un-connected airport-pairs being connected in the future, and the likelihood of connected airport-pairs becoming un-connected. Despite the innovation of this research, it does not focus on improving the FAA’s existing methodology for forecasting future air traffic levels on existing routes, which is based on relatively simple regression and growth models. We investigate different approaches for improving and developing new features within the existing data mining applications in air traffic forecasting. We focus particularly on predicting detailed traffic information for the US ATS. Initially, a 2-stage log-log model is applied to establish the significance of different inputs and to identify issues of endogeneity and multi-colinearity, while maintaining the simplicity of current models. Although the model shows high goodness of fit, it tested positive for both mentioned issues as well as presenting problems with causality. With the objective of solving these issues, a 3-stage model that is under development is introduced. This model employs logistic regression and discrete choice modelling. As part of future work, machine learning techniques such as clustering and neural networks will be applied to improve this model’s performance

Optimization and Technology-Based Strategies to Improve Public Transit Performance Accounting for Demand Distribution

Public transit is important to societies worldwide. The operation of public transit systems is generally associated with great benefits for the users, but there are also cases in which these systems demonstrate inefficient performance. Quantifying transit performance is an important area of research over the last decades. This dissertation presents models to improve transit system performance through optimization techniques and new technologies, recognizing the effects of non-uniform distribution of demand over space and time. The contributions span fixed route transit services and on-demand transit, as well as models for flexible transit operations that lie in between. Regarding fixed route systems, a methodology is proposed to estimate the number of passengers being left-behind subway train vehicles due to overcrowding. Methods to identify appropriate time periods and locations for studying this phenomenon are presented. The effects of overcrowding on passenger waiting times are also investigated. The challenging case of transit networks where passengers tap-in only upon entrance is analyzed, adding a new methodology to a very short list of similar studies and enhancing previous work in this field. For demand responsive systems, this dissertation focuses on optimizing the operation of paratransit services through coordination with alternative providers in order to decrease high operating costs of such a service. The analysis includes a heuristic-based method. The proposed model is more detailed than existing aggregated methods and is able to perform well in high demand levels, unlike existing exact approaches. This part of the dissertation also assists in making transportation network companies a complementary part of public transit, rather than a competitor. Finally, flexible transit systems are studied to identify the operational and demand related characteristics of a service area that could serve as indicators of such systems\u27 efficient performance. The focus here is on route deviation flexible services. Continuous approximation is used to model this flexible system. A new optimized hybrid transit system with elements of both fixed route and flexible services is proposed. Finally, it is highlighted that the current COVID-19 pandemic has proven the need for public transit systems that could be adjusted to accommodate changes in transit demand