Towards a Reduced Dependency Framework for Autonomous Unified Inspect-Explore Missions

The task of establishing and maintaining situational awareness in an unknown environment is a critical step to fulfil in a mission related to the field of rescue robotics. Predominantly, the problem of visual inspection of urban structures is dealt with view-planning being addressed by map-based approaches. In this article, we propose a novel approach towards effective use of Micro Aerial Vehicles (MAVs) for obtaining a 3-D shape of an unknown structure of objects utilizing a map-independent planning framework. The problem is undertaken via a bifurcated approach to address the task of executing a closer inspection of detected structures with a wider exploration strategy to identify and locate nearby structures, while being equipped with limited sensing capability. The proposed framework is evaluated experimentally in a controlled indoor environment in presence of a mock-up environment validating the efficacy of the proposed inspect-explore policy

En bild, många insikter: ett synergistiskt tillvägagångssätt för att möjliggöra autonom visuell inspektion

Visual inspection in autonomous robotics is a task in which autonomous agents are required to gather visual information of objects of interest, in a manner that ensures safety, efficiency and comprehensive coverage. It is, therefore, crucial for identifying key landmarks, detecting cracks or defects, or reconstructing the observed object for detailed analysis. This thesis delves into the  challenges encountered by autonomous agents in executing such tasks and presents frameworks for scenarios ranging from operations by multiple spacecrafts in close proximity to celestial bodies in Deep Space to terrestrial deployments of Unmanned Aerial Vehicles (UAVs) for inspection of large-scale infrastructures. The research thus pursues two main directions: Firstly, a novel formation control strategy is developed to enable autonomous agents to perform proximity operations safely, efficiently, and accurately in order to map the surface of Small Celestial Bodies (SCBs). This investigation encompasses control and coordination strategies, leveraging a realistic astrodynamic model of the orbital environment to navigate safely around SCBs. Along this direction, the contributions focus on enabling a distributed autonomy framework in the form of a cooperative stereo configuration between two spacecraft, allowing acquisition of 3D topological information of the candidate SCB. The framework employs a Leader-Follower approach, treating the maintenance of the desired stereo-formation as a 6 Degree-of-Freedom (DoF) nonlinear model predictive control (NMPC) problem. The second research direction focuses on addressing the problem of enabling robotic inspection for terrestrial applications. With the growing demand for efficient and reliable inspection techniques to improve in-situ situational awareness, the research concentrates on addressing the problem of obtaining detailed visual scan of available structures without any a priori knowledge of either the environment nor the structures. Thus, the key contributions of the presented work reside in the implementation of a unified autonomy, with the unification drawing it's root from the merging of two distinct research perspectives: Inspection and Exploration planning. The contribution establishes a novel solution by introducing a map-independent approach with a synergistic formulation of a reactive profile-adaptive view-planner coupled with a hierarchical exploration strategy and an environment-invariant scene recognition module. By integrating exploration and inspection methodologies, this research seeks to enhance the capabilities of UAVs in navigating and inspecting unknown structures in unfamiliar environments.  Through theoretical developments, extensive simulations and experimental validations, this thesis contributes to the advancement of the state-of-the-art in visual inspection with autonomous robots. Moreover, the findings extend current capabilities of autonomous agents in the field of space exploration as well as in disaster response and complex infrastructure inspection

Cooperative Navigation in Space in-proximity of Small Bodies

Autonomous proximity operations are the future of Deep space robotic exploration for searchof life, mining for resources and to establish outposts. Part of that future depends on howwell the spacecraft is capable to navigate around the complex environment of the smallcelestial body. The shift from huge monolithic spacecraft to a lightweight distributed Spacesystems has opened up a new opportunity for early characterization and global mappingmissions around these bodies. This project aims to contribute to help solve a part of thedream, wherein multiple spacecrafts operate cooperatively in proximity of small celestialbodies. To that extent, a 6 DoF controlled software-in-loop simulation is performed withsimulated optical sensors and IMU on board the spacecraft for verification of the controlledcooperative operation of two spacecrafts in a Leader-Follower configuration

En bild, många insikter: ett synergistiskt tillvägagångssätt för att möjliggöra autonom visuell inspektion

Visual inspection in autonomous robotics is a task in which autonomous agents are required to gather visual information of objects of interest, in a manner that ensures safety, efficiency and comprehensive coverage. It is, therefore, crucial for identifying key landmarks, detecting cracks or defects, or reconstructing the observed object for detailed analysis. This thesis delves into the  challenges encountered by autonomous agents in executing such tasks and presents frameworks for scenarios ranging from operations by multiple spacecrafts in close proximity to celestial bodies in Deep Space to terrestrial deployments of Unmanned Aerial Vehicles (UAVs) for inspection of large-scale infrastructures. The research thus pursues two main directions: Firstly, a novel formation control strategy is developed to enable autonomous agents to perform proximity operations safely, efficiently, and accurately in order to map the surface of Small Celestial Bodies (SCBs). This investigation encompasses control and coordination strategies, leveraging a realistic astrodynamic model of the orbital environment to navigate safely around SCBs. Along this direction, the contributions focus on enabling a distributed autonomy framework in the form of a cooperative stereo configuration between two spacecraft, allowing acquisition of 3D topological information of the candidate SCB. The framework employs a Leader-Follower approach, treating the maintenance of the desired stereo-formation as a 6 Degree-of-Freedom (DoF) nonlinear model predictive control (NMPC) problem. The second research direction focuses on addressing the problem of enabling robotic inspection for terrestrial applications. With the growing demand for efficient and reliable inspection techniques to improve in-situ situational awareness, the research concentrates on addressing the problem of obtaining detailed visual scan of available structures without any a priori knowledge of either the environment nor the structures. Thus, the key contributions of the presented work reside in the implementation of a unified autonomy, with the unification drawing it's root from the merging of two distinct research perspectives: Inspection and Exploration planning. The contribution establishes a novel solution by introducing a map-independent approach with a synergistic formulation of a reactive profile-adaptive view-planner coupled with a hierarchical exploration strategy and an environment-invariant scene recognition module. By integrating exploration and inspection methodologies, this research seeks to enhance the capabilities of UAVs in navigating and inspecting unknown structures in unfamiliar environments.  Through theoretical developments, extensive simulations and experimental validations, this thesis contributes to the advancement of the state-of-the-art in visual inspection with autonomous robots. Moreover, the findings extend current capabilities of autonomous agents in the field of space exploration as well as in disaster response and complex infrastructure inspection

Cooperative Navigation in Space in-proximity of Small Bodies

Autonomous proximity operations are the future of Deep space robotic exploration for searchof life, mining for resources and to establish outposts. Part of that future depends on howwell the spacecraft is capable to navigate around the complex environment of the smallcelestial body. The shift from huge monolithic spacecraft to a lightweight distributed Spacesystems has opened up a new opportunity for early characterization and global mappingmissions around these bodies. This project aims to contribute to help solve a part of thedream, wherein multiple spacecrafts operate cooperatively in proximity of small celestialbodies. To that extent, a 6 DoF controlled software-in-loop simulation is performed withsimulated optical sensors and IMU on board the spacecraft for verification of the controlledcooperative operation of two spacecrafts in a Leader-Follower configuration

FLIE: First-Look Enabled Inspect-Explore Autonomy Toward Visual Inspection of Unknown Distributed and Discontinuous Structures

In this article, the problem of an online autonomous aerial inspection, specifically for discontinuous and distributed objects is presented. The proposed approach imposes view culling and photogrammetric constraints based on a geometrically modeled three-dimensional view pyramid, a view cone to filter surfaces by desired observation angle, a framework-integrated passive collision-avoidance scheme with the object under inspection, and a dynamically enveloping bounding-box region to map the visited surfaces. Furthermore, the proposed inspect-explore framework is validated for the case of an unknown environment with no prior knowledge of the object model under inspection. The overall inspection scheme is based on the novel First-Look approach, enabling the UAV to progressively adapt its inspection path to match the profile of the structure autonomously. The implemented exploration strategy imposes a tiered policy enabling the UAV to search, identify and navigate towards the structure for inspection. The presented work utilizes a unified architecture of the aforementioned inspect-explore framework to improve situational awareness in a previously unknown environment by enabling the UAV to explore its surrounding space and identify structures to execute closer inspection tasks. Extended simulations to evaluate the efficacy of the proposed inspect-explore framework are presented with multiple structure scenarios.Validerad;2023;Nivå 2;2023-04-13 (joosat);Licens fulltext: CC BY License</p

Nonlinear Model Predictive Control based Cooperative Stereo-Visual Coverage of an Asteroid

In this article, a 3D visual coverage problem of an asteroid by establishing a stereo-visual based sensor from monocular cameras is considered. The cameras are individually mounted on two small low-thrust spacecraft, which are maintained in a tight stereo formation using a nonlinear model predictive control scheme to maintain an overlapping field-of-view and a specific baseline distance. The proposed control algorithm adopts a leader-follower architecture to define the relative pose between the spacecraft. However, asteroids provide a challenging environment for such missions deriving from their slow rotation rate and irregular shape. As such, they generate a low-order but irregular gravitational field. Paired with the influence of the solar radiation pressure acting on the spacecraft, the dynamic environment near the asteroid is highly perturbed. In addition, gravitational torque generated by the rotating body is also accounted for as it has a coupling effect between the orbital and attitude dynamics of the spacecraft. As a result, this article considers a full-state nonlinear control approach to plan correction maneuvers to maintain the desired pose of the spacecraft. The efficacy of the proposed control scheme is demonstrated within a realistic simulation scenario where the results are visualized utilizing the GAZEBO simulation environment

Towards a Reduced Dependency Framework for Autonomous Unified Inspect-Explore Missions

The task of establishing and maintaining situational awareness in an unknown environment is a critical step to fulfil in a mission related to the field of rescue robotics. Predominantly, the problem of visual inspection of urban structures is dealt with view-planning being addressed by map-based approaches. In this article, we propose a novel approach towards effective use of Micro Aerial Vehicles (MAVs) for obtaining a 3-D shape of an unknown structure of objects utilizing a map-independent planning framework. The problem is undertaken via a bifurcated approach to address the task of executing a closer inspection of detected structures with a wider exploration strategy to identify and locate nearby structures, while being equipped with limited sensing capability. The proposed framework is evaluated experimentally in a controlled indoor environment in presence of a mock-up environment validating the efficacy of the proposed inspect-explore policy

Towards Visual Inspection of Distributed and Irregular Structures: A Unified Autonomy Approach

This paper highlights the significance of maintaining and enhancing situational awareness in Urban Search and Rescue (USAR) missions. It focuses specifically on investigating the capabilities of Unmanned Aerial Vehicles (UAV) equipped with limited sensing capabilities and onboard computational resources to perform visual inspections of apriori unknown fractured and collapsed structures in unfamiliar environments. The proposed approach, referred to as First Look Inspect-Explore (FLIE), employs a flexible bifurcated behavior tree that leverages real-time RGB image and depth cloud data. By employing a recursive and reactive synthesis of safe view pose within the inspection module, FLIE incorporates a novel active visual guidance scheme for identifying previously inspected surfaces. Furthermore, the integration of a tiered hierarchical exploration module with the visual guidance system enables the UAV to navigate towards new and unexplored structures without relying on a map. This decoupling reduces memory overhead and computational effort by eliminating the need to plan based on an incrementally built, error-prone global map. The proposed autonomy is extensively evaluated through simulation and experimental verification under various scenarios and compared against state-of-art approaches, demonstrating its performance and effectiveness.Validerad;2023;Nivå 2;2023-09-28 (hanlid)</p

Towards a Reduced Dependency Framework for Autonomous Unified Inspect-Explore Missions

The task of establishing and maintaining situational awareness in an unknown environment is a critical step to fulfil in a mission related to the field of rescue robotics. Predominantly, the problem of visual inspection of urban structures is dealt with view-planning being addressed by map-based approaches. In this article, we propose a novel approach towards effective use of Micro Aerial Vehicles (MAVs) for obtaining a 3-D shape of an unknown structure of objects utilizing a map-independent planning framework. The problem is undertaken via a bifurcated approach to address the task of executing a closer inspection of detected structures with a wider exploration strategy to identify and locate nearby structures, while being equipped with limited sensing capability. The proposed framework is evaluated experimentally in a controlled indoor environment in presence of a mock-up environment validating the efficacy of the proposed inspect-explore policy