A Dynamical System Approach for Resource-Constrained Mobile Robotics

The revolution of autonomous vehicles has led to the development of robots with abundant sensors, actuators with many degrees of freedom, high-performance computing capabilities, and high-speed communication devices. These robots use a large volume of information from sensors to solve diverse problems. However, this usually leads to a significant modeling burden as well as excessive cost and computational requirements. Furthermore, in some scenarios, sophisticated sensors may not work precisely, the real-time processing power of a robot may be inadequate, the communication among robots may be impeded by natural or adversarial conditions, or the actuation control in a robot may be insubstantial. In these cases, we have to rely on simple robots with limited sensing and actuation, minimal onboard processing, moderate communication, and insufficient memory capacity. This reality motivates us to model simple robots such as bouncing and underactuated robots making use of the dynamical system techniques. In this dissertation, we propose a four-pronged approach for solving tasks in resource-constrained scenarios: 1) Combinatorial filters for bouncing robot localization; 2) Bouncing robot navigation and coverage; 3) Stochastic multi-robot patrolling; and 4) Deployment and planning of underactuated aquatic robots. First, we present a global localization method for a bouncing robot equipped with only a clock and contact sensors. Space-efficient and finite automata-based combinatorial filters are synthesized to solve the localization task by determining the robot’s pose (position and orientation) in its environment. Second, we propose a solution for navigation and coverage tasks using single or multiple bouncing robots. The proposed solution finds a navigation plan for a single bouncing robot from the robot’s initial pose to its goal pose with limited sensing. Probabilistic paths from several policies of the robot are combined artfully so that the actual coverage distribution can become as close as possible to a target coverage distribution. A joint trajectory for multiple bouncing robots to visit all the locations of an environment is incrementally generated. Third, a scalable method is proposed to find stochastic strategies for multi-robot patrolling under an adversarial and communication-constrained environment. Then, we evaluate the vulnerability of our patrolling policies by finding the probability of capturing an adversary for a location in our proposed patrolling scenarios. Finally, a data-driven deployment and planning approach is presented for the underactuated aquatic robots called drifters that creates the generalized flow pattern of the water, develops a Markov-chain based motion model, and studies the long- term behavior of a marine environment from a flow point-of-view. In a broad summary, our dynamical system approach is a unique solution to typical robotic tasks and opens a new paradigm for the modeling of simple robotics system

Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach

Optimization of Military Convoy Routing

Motoriseeritud rännakukolonnide optimeerimine on matemaatilise optimeerimise probleem, milles püütakse leida optimaalset marsruutimislahendust ja vastavat ajakava samaaegsetelt liikuvatele rännakukolonnidele. Käesolevas töös luuakse valik erinevatel optimeerimistehnikatel põhinevaid meetodeid, mida testides püütakse leida parimat Eesti oludele vastavat rännakukolonnide marsruutimise optimeerimismeetodit. Häid tulemusi saavutati kasutades osalise täisarvulise planeerimise mudelit koos heuristiliste täiendustega, rakendades jaga-ja-piira tehnikal põhinevat täpset algoritmi, kui ka kasutades fikseeritud järjestusega marsruutimislahendust. Lisaks töötati bakalaureusetöö koostamise käigus välja optimeerimismeetodeid kasutav rakendus, mille abil on võimalik võrrelda erinevate meetodite käitumist ja omadusi, esitada arvutuste tulemusena leitud teekondi ja ajagraafikuid ning animeerida Eesti kaardil rännakukolonnide liikumist. Töö tulemusena võib väita, et matemaatilise optimeerimise meetodid on sobivad päriseluliste rännakukolonnide optimeerimisprobleemide kiireks ja kvaliteetseks lahendamiseks ja et neid meetodeid kasutades on võimalik parandada rännakukolonnide kavandamisel tehtavate planeerimisotsuste kvaliteeti.Convoy movement problem is a mathematical optimization problem which tries to find optimal routing and scheduling solution for concurrent military convoy movements. In this thesis several optimization methodologies are designed and tested to find best suited algorithm for solving practical convoy routing instances in Estonia. Encouraging results are obtained by using a mixed integer programming model together with simple heuristics, by creating an exact branch-and-bound methodology and by developing fixed-order based routing approach. Bachelor’s thesis also provides a complementary application to compare qualities of designed methods, to present calculated routes and schedules and to display convoy movement animations on the map of Estonia. Thesis illustrates that methods of mathematical optimization can be used to solve realworld instances of convoy movement problem fast and with quality results and hence improve decisionmaking in operational convoy planning practice

A modest approach to Markov automata

A duplicate of https://zenodo.org/record/5758839. Reason: The submitter forgot to indicate the DOI before publishing, so it got another one assigned automatically, which is unchangeable

OPERATIONAL PLANNING AND OPTIMIZATION OF SMALL DOMAIN SWARM DEFENSE STRATEGIES

This thesis considers the case of a drone defending a high-value target from a number of inbound attacking drones. The defending drone is equipped with short-range weapons and must destroy each of the attacking drones in the most efficient manner. This problem sits at the intersection of several open problems in applied mathematics, such as optimal motion planning in the presence of attrition, as well as solving a “traveling salesman problem” (TSP) with moving targets. The purpose of our research was to analyze this problem by decomposing it into the component problems and then presenting proof-of-concept solutions of each component. The primary results of this thesis include a modeling framework where optimization can be performed without requiring constraints; comparing the strengths of using different types of cost functions for optimization (e.g., minimizing the chance of high-value unit destruction versus a metric based on the path of the defender relative to attackers); and solving moving-target TSP in certain limits by mapping it onto standard TSP or using machine learning.Cruiser/ONRMajor, United States Marine CorpsApproved for public release. Distribution is unlimited