confined spaces industrial inspection with micro aerial vehicles and laser range finder localization

This work addresses the problem of semi-automatic inspection and navigation in confined environments. A system that overcomes many challenges at the state of the art is presented. It comprises a mu..

Automated multi-rotor draft survey of large vessels

In maritime sector draft survey has a significant importance as it is used to determine many important factors used in maritime transportation. Draft is the vertical displacement from the bottom of the keel (the bottom-most element of a vessel) to the water line (the line of meeting point of hull and the water surface). It is used to measure the minimum water depth for safe navigation of vessel and to evaluate mass of cargo in the vessel by the change in displacement on the draft scale after loading of the cargo in the vessel. Draft measurement of a vessel has a vital role in maritime sector to ensure a safe equilibrium between maximum and minimum cargo that can be loaded in the vessel. Draft survey performed at the time of loading and unloading of cargo (Iron Ore) at the Narvik port to read out draft markings traditionally involved a round trip around the vessel in a small crew boat and it is a time consuming and challenging task specially in darkness (during night), shadows and when difficult to safely reach the crew boat close enough due to anchors and buoys. The goal of this study is to develop an autonomous multi-rotor system that can survey the large vessel to capture all the necessary draft measurements by reaching close enough even in challenging environments like nighttime and in presence of obstacles. This involves developing the solution for path planning to perform flight operation autonomously, developing guidance and control algorithm for the flight operation to enable the multi-rotor to follow the designated path and perform the inspection while avoiding all the hurdles using collision avoidance system. Along with developing the specifications for a multi-rotor that can perform the inspection and suggest necessary system components including multi-rotor itself and additional components such as sensors, lights and camera, and necessities for on-board data handling

Maritime Robotics and Autonomous Systems Operations: Exploring Pathways for Overcoming International Techno-Regulatory Data Barriers

The current regulatory landscape that applies to maritime service robotics, aptly termed as robotics and autonomous systems (RAS), is quite complex. When it comes to patents, there are multifarious considerations in relation to vessel survey, inspection, and maintenance processes under national and international law. Adherence is challenging, given that the traditional delivery methods are viewed as unsafe, strenuous, and laborious. Service robotics, namely micro aerial vehicles (MAVs) or drones, magnetic-wheeled crawlers (crawlers), and remotely operated vehicles (ROVs), function by relying on the architecture of the Internet of Robotic Things. The aforementioned are being introduced as time-saving apparatuses, accompanied by the promise to acquire concrete and sufficient data for the identification of vessel structural weaknesses with the highest level of accuracy to facilitate decision-making processes upon which temporary and permanent measures are contingent. Nonetheless, a noticeable critical issue associated with RAS effective deployment revolves around non-personal data governance, which comprises the main analytical focus of this research effort. The impetus behind this study stems from the need to enquire whether “data” provisions within the realm of international technological regulatory (techno-regulatory) framework is sufficient, well organized, and harmonized so that there are no current or future conflicts with promulgated theoretical dimensions of data that drive all subject matter-oriented actions. As is noted from the relevant expository research, the challenges are many. Engineering RAS to perfection is not the end-all and be-all. Collateral impediments must be avoided. A safety net needs to be devised to protect non-personal data. The results here indicate that established data decision dimensions call for data security and protection, as well as a consideration of ownership and liability details. An analysis of the state-of-the-art and the comparative results assert that the abovementioned remain neglected in the current international setting. The findings reveal specific data barriers within the existing international framework. The ways forward include strategic actions to remove data barriers towards overall efficacy of maritime RAS operations. The overall findings indicate that an effective transition to RAS operations requires optimizing the international regulatory framework for opening the pathways for effective RAS operations. Conclusions were drawn based on the premise that policy reform is inevitable in order to push the RAS agenda forward before the emanation of 6G and the era of the Internet of Everything, with harmonization and further standardization being very high priority issues

Vision-based Learning for Drones: A Survey

Drones as advanced cyber-physical systems are undergoing a transformative shift with the advent of vision-based learning, a field that is rapidly gaining prominence due to its profound impact on drone autonomy and functionality. Different from existing task-specific surveys, this review offers a comprehensive overview of vision-based learning in drones, emphasizing its pivotal role in enhancing their operational capabilities under various scenarios. We start by elucidating the fundamental principles of vision-based learning, highlighting how it significantly improves drones' visual perception and decision-making processes. We then categorize vision-based control methods into indirect, semi-direct, and end-to-end approaches from the perception-control perspective. We further explore various applications of vision-based drones with learning capabilities, ranging from single-agent systems to more complex multi-agent and heterogeneous system scenarios, and underscore the challenges and innovations characterizing each area. Finally, we explore open questions and potential solutions, paving the way for ongoing research and development in this dynamic and rapidly evolving field. With growing large language models (LLMs) and embodied intelligence, vision-based learning for drones provides a promising but challenging road towards artificial general intelligence (AGI) in 3D physical world

Underwater Drone Architecture for Marine Digital Twin: Lessons Learned from SUSHI DROP Project

The ability to observe the world has seen significant developments in the last few decades, alongside the techniques and methodologies to derive accurate digital replicas of observed environments. Underwater ecosystems present greater challenges and remain largely unexplored, but the need for reliable and up-to-date information motivated the birth of the Interreg Italy–Croatia SUSHI DROP Project (SUstainable fiSHeries wIth DROnes data Processing). The aim of the project is to map ecosystems for sustainable fishing and to achieve this goal a prototype of an Unmanned Underwater Vehicle (UUV), named Blucy, has been designed and developed. Blucy was deployed during project missions for surveying the benthic zone in deep waters of the Adriatic Sea with noninvasive techniques compared to the use of trawl nets. This article describes the strategies followed, the instruments applied and the challenges to be overcome to obtain an accurately georeferenced underwater survey with the goal of creating a marine digital twin

Evaluation of the utility and performance of an autonomous surface vehicle for mobile monitoring of waterborne biochemical agents

Real-time water quality monitoring is crucial due to land utilization increases which can negatively impact aquatic ecosystems from surface water runoff. Conventional monitoring methodologies are laborious, expensive, and spatio-temporally limited. Autonomous surface vehicles (ASVs), equipped with sensors/instrumentation, serve as mobile sampling stations that reduce labor and enhance data resolution. However, ASV autopilot navigational accuracy is affected by environmental forces (wind, current, and waves) that can alter trajectories of planned paths and negatively affect spatio-temporal resolution of water quality data. This study demonstrated a commercially available solar powered ASV equipped with a multi-sensor payload ability to operate autonomously to accurately and repeatedly maintain established A-B line transects under varying environmental conditions, where lateral deviation from a planned linear route was measured and expressed as cross-track error (XTE). This work provides a framework for development of spatial/temporal resolution limitations of ASVs for real-time monitoring campaigns and future development of in-situ sampling technologies

Machine learning techniques for robotic and autonomous inspection of mechanical systems and civil infrastructure

Machine learning and in particular deep learning techniques have demonstrated the most efficacy in training, learning, analyzing, and modelling large complex structured and unstructured datasets. These techniques have recently been commonly deployed in different industries to support robotic and autonomous system (RAS) requirements and applications ranging from planning and navigation to machine vision and robot manipulation in complex environments. This paper reviews the state-of-the-art with regard to RAS technologies (including unmanned marine robot systems, unmanned ground robot systems, climbing and crawler robots, unmanned aerial vehicles, and space robot systems) and their application for the inspection and monitoring of mechanical systems and civil infrastructure. We explore various types of data provided by such systems and the analytical techniques being adopted to process and analyze these data. This paper provides a brief overview of machine learning and deep learning techniques, and more importantly, a classification of the literature which have reported the deployment of such techniques for RAS-based inspection and monitoring of utility pipelines, wind turbines, aircrafts, power lines, pressure vessels, bridges, etc. Our research provides documented information on the use of advanced data-driven technologies in the analysis of critical assets and examines the main challenges to the applications of such technologies in the industry

Underwater Drone Architecture for Marine Digital Twin: Lessons Learned from SUSHI DROP Project

The ability to observe the world has seen significant developments in the last few decades, alongside the techniques and methodologies to derive accurate digital replicas of observed environments. Underwater ecosystems present greater challenges and remain largely unexplored, but the need for reliable and up-to-date information motivated the birth of the Interreg Italy-Croatia SUSHI DROP Project (SUstainable fiSHeries wIth DROnes data Processing). The aim of the project is to map ecosystems for sustainable fishing and to achieve this goal a prototype of an Unmanned Underwater Vehicle (UUV), named Blucy, has been designed and developed. Blucy was deployed during project missions for surveying the benthic zone in deep waters of the Adriatic Sea with non-invasive techniques compared to the use of trawl nets. This article describes the strategies followed, the instruments applied and the challenges to be overcome to obtain an accurately georeferenced underwater survey with the goal of creating a marine digital twin

Marine Vessel Inspection as a Novel Field for Service Robotics: A Contribution to Systems, Control Methods and Semantic Perception Algorithms.

This cumulative thesis introduces a novel field for service robotics: the inspection of marine vessels using mobile inspection robots. In this thesis, three scientific contributions are provided and experimentally verified in the field of marine inspection, but are not limited to this type of application. The inspection scenario is merely a golden thread to combine the cumulative scientific results presented in this thesis. The first contribution is an adaptive, proprioceptive control approach for hybrid leg-wheel robots, such as the robot ASGUARD described in this thesis. The robot is able to deal with rough terrain and stairs, due to the control concept introduced in this thesis. The proposed system is a suitable platform to move inside the cargo holds of bulk carriers and to deliver visual data from inside the hold. Additionally, the proposed system also has stair climbing abilities, allowing the system to move between different decks. The robot adapts its gait pattern dynamically based on proprioceptive data received from the joint motors and based on the pitch and tilt angle of the robot's body during locomotion. The second major contribution of the thesis is an independent ship inspection system, consisting of a magnetic wall climbing robot for bulkhead inspection, a particle filter based localization method, and a spatial content management system (SCMS) for spatial inspection data representation and organization. The system described in this work was evaluated in several laboratory experiments and field trials on two different marine vessels in close collaboration with ship surveyors. The third scientific contribution of the thesis is a novel approach to structural classification using semantic perception approaches. By these methods, a structured environment can be semantically annotated, based on the spatial relationships between spatial entities and spatial features. This method was verified in the domain of indoor perception (logistics and household environment), for soil sample classification, and for the classification of the structural parts of a marine vessel. The proposed method allows the description of the structural parts of a cargo hold in order to localize the inspection robot or any detected damage. The algorithms proposed in this thesis are based on unorganized 3D point clouds, generated by a LIDAR within a ship's cargo hold. Two different semantic perception methods are proposed in this thesis. One approach is based on probabilistic constraint networks; the second approach is based on Fuzzy Description Logic and spatial reasoning using a spatial ontology about the environment