LSTM Path-Maker : une nouvelle stratégie pour la patrouille multiagent basée sur l'architecture LSTM

National audienceAbstract For over a decade, the multi-agent patrol task has received a growing attention from the multi-agent community due to its wide range of potential applications. However, the existing patrolling-specific algorithms based on deep learning algorithms are still in preliminary stages. In this paper, we propose to integrate a recurrent neural network as part of * Paper presented at the 52nd Hawaii International Conference on System Sciences (HICSS52 2019), titre, résumé et mots-clés en français ajou-tés. a multi-agent patrolling strategy. Hence we proposed a formal model of an LSTM-based agent strategy named LSTM Path Maker. The LSTM network is trained over simulation traces of a coordinated strategy, then embedded on each agent of the new strategy to patrol efficiently without communicating. Finally this new LSTM-based strategy is evaluated in simulation and compared with two representative strategies : a coordinated one and a reactive one. Preliminary results indicate that the proposed strategy is better than the reactive.Depuis plus d'une décennie, la tâche de la patrouille mul-tiagent a attiré l'attention de la communauté multiagent de manière croissante, en raison de son grand nombre d'applications potentielles. Cependant, les algorithmes ba-sés sur des méthodes d'apprentissage profond pour traiter cette tâche sont à ce jour peu développés. Dans cet article, nous proposons d'intégrer un réseau de neurone récurrent à une stratégie de patrouille multiagent. Ce faisant, nous avons proposé un modèle formel de stratégie d'agent basée sur l'architecture LSTM, que nous avons nommé LSTM-Path-Maker. Le réseau LSTM est entraîné sur des traces de simulation d'une stratégie coordonnée et centralisée, puis embarqué dans chaque agent en vue de patrouiller effica-cement sans communication. Enfin, cette nouvelle stratégie basée sur l'architecture LSTM est évaluée en simulation et comparée d'une part à une stratégie coordonnée et d'autre part à une stratégie réactive. Les résultats préliminaires in-diquent que la stratégie proposée est meilleure que la stra-tégie réactive

의미론적 환경 이해 기반 인간 로봇 협업

학위논문(박사)--서울대학교 대학원 :공과대학 전기·정보공학부,2020. 2. 이범희.Human-robot cooperation is unavoidable in various applications ranging from manufacturing to field robotics owing to the advantages of adaptability and high flexibility. Especially, complex task planning in large, unconstructed, and uncertain environments can employ the complementary capabilities of human and diverse robots. For a team to be effectives, knowledge regarding team goals and current situation needs to be effectively shared as they affect decision making. In this respect, semantic scene understanding in natural language is one of the most fundamental components for information sharing between humans and heterogeneous robots, as robots can perceive the surrounding environment in a form that both humans and other robots can understand. Moreover, natural-language-based scene understanding can reduce network congestion and improve the reliability of acquired data. Especially, in field robotics, transmission of raw sensor data increases network bandwidth and decreases quality of service. We can resolve this problem by transmitting information in the form of natural language that has encoded semantic representations of environments. In this dissertation, I introduce a human and heterogeneous robot cooperation scheme based on semantic scene understanding. I generate sentences and scene graphs, which is a natural language grounded graph over the detected objects and their relationships, with the graph map generated using a robot mapping algorithm. Subsequently, a framework that can utilize the results for cooperative mission planning of humans and robots is proposed. Experiments were performed to verify the effectiveness of the proposed methods. This dissertation comprises two parts: graph-based scene understanding and scene understanding based on the cooperation between human and heterogeneous robots. For the former, I introduce a novel natural language processing method using a semantic graph map. Although semantic graph maps have been widely applied to study the perceptual aspects of the environment, such maps do not find extensive application in natural language processing tasks. Several studies have been conducted on the understanding of workspace images in the field of computer vision; in these studies, the sentences were automatically generated, and therefore, multiple scenes have not yet been utilized for sentence generation. A graph-based convolutional neural network, which comprises spectral graph convolution and graph coarsening, and a recurrent neural network are employed to generate sentences attention over graphs. The proposed method outperforms the conventional methods on a publicly available dataset for single scenes and can be utilized for sequential scenes. Recently, deep learning has demonstrated impressive developments in scene understanding using natural language. However, it has not been extensively applied to high-level processes such as causal reasoning, analogical reasoning, or planning. The symbolic approach that calculates the sequence of appropriate actions by combining the available skills of agents outperforms in reasoning and planning; however, it does not entirely consider semantic knowledge acquisition for human-robot information sharing. An architecture that combines deep learning techniques and symbolic planner for human and heterogeneous robots to achieve a shared goal based on semantic scene understanding is proposed for scene understanding based on human-robot cooperation. In this study, graph-based perception is used for scene understanding. A planning domain definition language (PDDL) planner and JENA-TDB are utilized for mission planning and data acquisition storage, respectively. The effectiveness of the proposed method is verified in two situations: a mission failure, in which the dynamic environment changes, and object detection in a large and unseen environment.인간과 이종 로봇 간의 협업은 높은 유연성과 적응력을 보일 수 있다는 점에서 제조업에서 필드 로보틱스까지 다양한 분야에서 필연적이다. 특히, 서로 다른 능력을 지닌 로봇들과 인간으로 구성된 하나의 팀은 넓고 정형화되지 않은 공간에서 서로의 능력을 보완하며 복잡한 임무 수행을 가능하게 한다는 점에서 큰 장점을 갖는다. 효율적인 한 팀이 되기 위해서는, 팀의 공통된 목표 및 각 팀원의 현재 상황에 관한 정보를 실시간으로 공유할 수 있어야 하며 함께 의사 결정을 할 수 있어야 한다. 이러한 관점에서, 자연어를 통한 의미론적 환경 이해는 인간과 서로 다른 로봇들이 모두 이해할 수 있는 형태로 환경을 인지한다는 점에서 가장 필수적인 요소이다. 또한, 우리는 자연어 기반 환경 이해를 통해 네트워크 혼잡을 피함으로써 획득한 정보의 신뢰성을 높일 수 있다. 특히, 대량의 센서 데이터 전송에 의해 네트워크 대역폭이 증가하고 통신 QoS (Quality of Service) 신뢰도가 감소하는 문제가 빈번히 발생하는 필드 로보틱스 영역에서는 의미론적 환경 정보인 자연어를 전송함으로써 통신 대역폭을 감소시키고 통신 QoS 신뢰도를 증가시킬 수 있다. 본 학위 논문에서는 환경의 의미론적 이해 기반 인간 로봇 협동 방법에 대해 소개한다. 먼저, 로봇의 지도 작성 알고리즘을 통해 획득한 그래프 지도를 이용하여 자연어 문장과 검출한 객체 및 각 객체 간의 관계를 자연어 단어로 표현하는 그래프를 생성한다. 그리고 자연어 처리 결과를 이용하여 인간과 다양한 로봇들이 함께 협업하여 임무를 수행할 수 있도록 하는 프레임워크를 제안한다. 본 학위 논문은 크게 그래프를 이용한 의미론적 환경 이해와 의미론적 환경 이해를 통한 인간과 이종 로봇 간의 협업 방법으로 구성된다. 먼저, 그래프를 이용한 의미론적 환경 이해 부분에서는 의미론적 그래프 지도를 이용한 새로운 자연어 처리 방법에 대해 소개한다. 의미론적 그래프 지도 작성 방법은 로봇의 환경 인지 측면에서 많이 연구되었지만 이를 이용한 자연어 처리 방법은 거의 연구되지 않았다. 반면 컴퓨터 비전 분야에서는 이미지를 이용한 환경 이해 연구가 많이 이루어졌지만, 연속적인 장면들은 다루는데는 한계점이 있다. 따라서 우리는 그래프 스펙트럼 이론에 기반한 그래프 컨볼루션과 그래프 축소 레이어로 구성된 그래프 컨볼루션 신경망 및 순환 신경망을 이용하여 그래프를 설명하는 문장을 생성한다. 제안한 방법은 기존의 방법들보다 한 장면에 대해 향상된 성능을 보였으며 연속된 장면들에 대해서도 성공적으로 자연어 문장을 생성한다. 최근 딥러닝은 자연어 기반 환경 인지에 있어 급속도로 큰 발전을 이루었다. 하지만 인과 추론, 유추적 추론, 임무 계획과 같은 높은 수준의 프로세스에는 적용이 힘들다. 반면 임무를 수행하는 데 있어 각 에이전트의 능력에 맞게 행위들의 순서를 계산해주는 상징적 접근법(symbolic approach)은 추론과 임무 계획에 있어 뛰어난 성능을 보이지만 인간과 로봇들 사이의 의미론적 정보 공유 방법에 대해서는 거의 다루지 않는다. 따라서, 인간과 이종 로봇 간의 협업 방법 부분에서는 딥러닝 기법들과 상징적 플래너(symbolic planner)를 연결하는 프레임워크를 제안하여 의미론적 이해를 통한 인간 및 이종 로봇 간의 협업을 가능하게 한다. 우리는 의미론적 주변 환경 이해를 위해 이전 부분에서 제안한 그래프 기반 자연어 문장 생성을 수행한다. PDDL 플래너와 JENA-TDB는 각각 임무 계획 및 정보 획득 저장소로 사용한다. 제안한 방법의 효용성은 시뮬레이션을 통해 두 가지 상황에 대해서 검증한다. 하나는 동적 환경에서 임무 실패 상황이며 다른 하나는 넓은 공간에서 객체를 찾는 상황이다.1 Introduction 1 1.1 Background and Motivation 1 1.2 Literature Review 5 1.2.1 Natural Language-Based Human-Robot Cooperation 5 1.2.2 Artificial Intelligence Planning 5 1.3 The Problem Statement 10 1.4 Contributions 11 1.5 Dissertation Outline 12 2 Natural Language-Based Scene Graph Generation 14 2.1 Introduction 14 2.2 Related Work 16 2.3 Scene Graph Generation 18 2.3.1 Graph Construction 19 2.3.2 Graph Inference 19 2.4 Experiments 22 2.5 Summary 25 3 Language Description with 3D Semantic Graph 26 3.1 Introduction 26 3.2 Related Work 26 3.3 Natural Language Description 29 3.3.1 Preprocess 29 3.3.2 Graph Feature Extraction 33 3.3.3 Natural Language Description with Graph Features 34 3.4 Experiments 35 3.5 Summary 42 4 Natural Question with Semantic Graph 43 4.1 Introduction 43 4.2 Related Work 45 4.3 Natural Question Generation 47 4.3.1 Preprocess 49 4.3.2 Graph Feature Extraction 50 4.3.3 Natural Question with Graph Features 51 4.4 Experiments 52 4.5 Summary 58 5 PDDL Planning with Natural Language 59 5.1 Introduction 59 5.2 Related Work 60 5.3 PDDL Planning with Incomplete World Knowledge 61 5.3.1 Natural Language Process for PDDL Planning 63 5.3.2 PDDL Planning System 64 5.4 Experiments 65 5.5 Summary 69 6 PDDL Planning with Natural Language-Based Scene Understanding 70 6.1 Introduction 70 6.2 Related Work 74 6.3 A Framework for Heterogeneous Multi-Agent Cooperation 77 6.3.1 Natural Language-Based Cognition 78 6.3.2 Knowledge Engine 80 6.3.3 PDDL Planning Agent 81 6.4 Experiments 82 6.4.1 Experiment Setting 82 6.4.2 Scenario 84 6.4.3 Results 87 6.5 Summary 91 7 Conclusion 92Docto

Learning spatiotemporal patterns for monitoring smart cities and infrastructure

Recent advances in the Internet of Things (IoT) have changed the way we interact with the world. The ability to monitor and manage objects in the physical world electronically makes it possible to bring data-driven decision making to new realms of city infrastructure and management. Large volumes of spatiotemporal data have been collected from pervasive sensors in both indoor and outdoor environments, and this data reveals dynamic patterns in cities, infrastructure, and public property. In light of the need for new approaches to analysing such data, in this thesis, we propose present relevant data mining techniques and machine learning approaches to extract knowledge from spatiotemporal data to solve real-world problems. Many challenges and problems are under-addressed in smart cities and infrastructure monitoring systems such as indoor person identification, evaluation of city regions segmentation with parking events, fine collection from cars in violations, parking occupancy prediction and airport aircraft path map reconstruction. All the above problems are associated with both spatial and temporal information and the accurate pattern recognition of these spatiotemporal data are essential for determining problem solutions. Therefore, how to incorporate spatiotemporal data mining techniques, artificial intelligence approaches and expert knowledge in each specific domain is a common challenge. In the indoor person identification area, identifying the person accessing a secured room without vision-based or device-based systems is very challenging. In particular, to distinguish time-series patterns on high-dimensional wireless signal channels caused by different activities and people, requires novel time-series data mining approaches. To solve this important problem, we established a device-free system and proposed a two-step solution to identify a person who has accessed a secure area such as an office. Establishing smart parking systems in cities is a key component of smart cities and infrastructure construction. Many sub-problems such as parking space arrangements, fine collection and parking occupancy prediction are urgent and important for city managers. Arranging parking spaces based on historical data can improve the utilisation rate of parking spaces. To arrange parking spaces based on collected spatiotemporal data requires reasonable region segmentation approaches. Moreover, evaluating parking space grouping results needs to consider the correlation between the spatial and temporal domains since these are heterogeneous. Therefore, we have designed a spatiotemporal data clustering evaluation approach, which exploits the correlation between the spatial domain and the temporal domain. It can evaluate the segmentation results of parking spaces in cities using historical data and similar clustering results that group data consisting of both spatial and temporal domains. For fine collection problem, using the sensor instrumentation installed in parking spaces to detect cars in violation and issue infringement notices in a short time-window to catch these cars in time is significantly difficult. This is because most cars in violation leave within a short period and multiple cars are in violation at the same time. Parking officers need to choose the best route to collect fines from these drivers in the shortest time. Therefore, we proposed a new optimisation problem called the Travelling Officer Problem and a general probability-based model. We succeeded in integrating temporal information and the traditional optimisation algorithm. This model can suggest to parking officers an optimised path that maximise the probability to catch the cars in violation in time. To solve this problem in real-time, we incorporated the model with deep learning methods. We proposed a theoretical approach to solving the traditional orienteering problem with deep learning networks. This approach could improve the efficiency of similar urban computing problems as well. For parking occupancy prediction, a key problem in parking space management is with providing a car parking availability prediction service that can inform car drivers of vacant parking lots before they start their journeys using prediction approaches. We proposed a deep learning-based model to solve this parking occupancy prediction problem using spatiotemporal data analysis techniques. This model can be generalised to other spatiotemporal data prediction problems also. In the airport aircraft management area, grouping similar spatiotemporal data is widely used in the real world. Determining key features and combining similar data are two key problems in this area. We presented a new framework to group similar spatiotemporal data and construct a road graph with GPS data. We evaluated our framework experimentally using a state-of-the-art test-bed technique and found that it could effectively and efficiently construct and update airport aircraft route map. In conclusion, the studies in this thesis aimed to discover intrinsic and dynamic patterns from spatiotemporal data and proposed corresponding solutions for real-world smart cities and infrastructures monitoring problems via spatiotemporal pattern analysis and machine learning approaches. We hope this research will inspire the research community to develop more robust and effective approaches to solve existing problems in this area in the future

Advances in Artificial Intelligence: Models, Optimization, and Machine Learning

The present book contains all the articles accepted and published in the Special Issue “Advances in Artificial Intelligence: Models, Optimization, and Machine Learning” of the MDPI Mathematics journal, which covers a wide range of topics connected to the theory and applications of artificial intelligence and its subfields. These topics include, among others, deep learning and classic machine learning algorithms, neural modelling, architectures and learning algorithms, biologically inspired optimization algorithms, algorithms for autonomous driving, probabilistic models and Bayesian reasoning, intelligent agents and multiagent systems. We hope that the scientific results presented in this book will serve as valuable sources of documentation and inspiration for anyone willing to pursue research in artificial intelligence, machine learning and their widespread applications

Data-centric methods for optimization and pattern discovery in networked systems

In this thesis, we examine two data-driven solutions to problems in operational networks. The first problem is concerned with assessing the resilience of the US air transportation network from an operational perspective. As a complex network comprising over 5,000 public airports and countless interfaces with other transportation systems, the impact of any disruption in the air network undoubtedly extends to other inter-connected economic and functional domains. Our solution to the resilience assessment problem is a tri-level optimization program that is able to simulate worst-case disruptions in the air network as well as propose the optimal ways to mitigate their effects. These mitigation steps take the form of investment recommendations for the air routes that are in most need of augmentation by other high-speed transportation modes. Our methodology and results for this application are explained in detail in Chapter 3. The second problem discussed in this thesis is centered on identifying design patterns in architecture graph representations of operational systems. Design patterns have been well documented and researched in software systems as a valuable design tool since the nineties. However, their usage has not been significantly expanded beyond software architectures, and their discovery methods have generally remained structured and supervised. We propose an end-to-end, unsupervised graph generation and pattern identification framework that is able to find unknown and potentially useful patterns in architecture graphs using machine learning. Our method is not limited to software systems, and is designed to be able to make possible pattern predictions even with a single architecture graph input. We detail our framework and experimental results in Chapter 4. Organizationally, Chapter 1 of the thesis starts with an introduction to network theory and graph representations, Chapter 2 provides background on network optimization and graph machine learning, and Chapter 5 concludes the thesis with our final thoughts and future research directions

Analyzing Granger causality in climate data with time series classification methods

Attribution studies in climate science aim for scientifically ascertaining the influence of climatic variations on natural or anthropogenic factors. Many of those studies adopt the concept of Granger causality to infer statistical cause-effect relationships, while utilizing traditional autoregressive models. In this article, we investigate the potential of state-of-the-art time series classification techniques to enhance causal inference in climate science. We conduct a comparative experimental study of different types of algorithms on a large test suite that comprises a unique collection of datasets from the area of climate-vegetation dynamics. The results indicate that specialized time series classification methods are able to improve existing inference procedures. Substantial differences are observed among the methods that were tested

Exploration and Mapping of Spatio-Temporal Pedestrian Flow Patterns for Mobile Robots

Socially compliant robot navigation is one of the key aspects for long-term acceptance of mobile robots in human-populated environments. One of the current barriers for this acceptance is that many navigation methods are based only on reactive behaviours, which can lead to frequent re-plannings, causing an erratic or aggressive robot behaviour. Instead, giving the ability to model and predict in advance how the people are likely to behave, from a long-term perspective, is an important enabler for safe and efficient navigation. For example, a robot may use its knowledge of the expected human motion to go with the main direction of flow to minimise the possibility of collisions or trajectory re-plannings. In order to provide robots with knowledge of the expected activity patterns of people at different places and times,the first main contribution of this thesis is the introduction of a Spatio-Temporal Flow map (STeF-map). This is a time-dependent probabilistic map able to model and predict the flow patterns of people in the environment. The proposed representation models the likelihood of motion directions on a grid-based map by a set of harmonic functions, which efficiently capture long-term variations of crowd movements over time. The experimental evaluation shows that the proposed model enables a better human motion prediction than spatial-only approaches and an increased capacity for socially compliant robot navigation. Obtaining this knowledge from a mobile robot platform is, however, not a trivial task, as usually they can only observe a fraction of the environment at a time, while the activity patterns of people may also change at different times. Therefore, the second main contribution is the investigation of a new methodology for mobile robot exploration to maximise the knowledge of human activity patterns, by deciding where and when to collect observations based on an exploration policy driven by the entropy levels in a spatio-temporal map. The evaluation is performed by simulating mobile robot exploration using real sensory data from three long-term pedestrian datasets, and the results show that for certain scenarios, the proposed exploration system can learn STeF-maps more quickly and better predict the flow patterns than uninformed strategies