Smartphone-based object recognition with embedded machine learning intelligence for unmanned aerial vehicles

Existing artificial intelligence solutions typically operate in powerful platforms with high computational resources availability. However, a growing number of emerging use cases such as those based on unmanned aerial systems (UAS) require new solutions with embedded artificial intelligence on a highly mobile platform. This paper proposes an innovative UAS that explores machine learning (ML) capabilities in a smartphone‐based mobile platform for object detection and recognition applications. A new system framework tailored to this challenging use case is designed with a customized workflow specified. Furthermore, the design of the embedded ML leverages TensorFlow, a cutting‐edge open‐source ML framework. The prototype of the system integrates all the architectural components in a fully functional system, and it is suitable for real‐world operational environments such as seek and rescue use cases. Experimental results validate the design and prototyping of the system and demonstrate an overall improved performance compared with the state of the art in terms of a wide range of metrics

Using Unmanned Aerial Vehicles for Wireless Localization in Search and Rescue

This thesis presents how unmanned aerial vehicles (UAVs) can successfully assist in search and rescue (SAR) operations using wireless localization. The zone-grid to partition to capture/detect WiFi probe requests follows the concepts found in Search Theory Method. The UAV has attached a sensor, e.g., WiFi sniffer, to capture/detect the WiFi probes from victims or lost people’s smartphones. Applying the Random-Forest based machine learning algorithm, an estimation of the user\u27s location is determined with a 81.8% accuracy. UAV technology has shown limitations in the navigational performance and limited flight time. Procedures to optimize these limitations are presented. Additionally, how the UAV is maneuvered during flight is analyzed, considering different SAR flight patterns and Li-Po battery consumption rates of the UAV. Results show that controlling the UAV by remote-controll detected the most probes, but it is less power efficient compared to control it autonomously

Cyber-Physical Embedded Systems with Transient Supervisory Command and Control: A Framework for Validating Safety Response in Automated Collision Avoidance Systems

The ability to design and engineer complex and dynamical Cyber-Physical Systems (CPS) requires a systematic view that requires a definition of level of automation intent for the system. Since CPS covers a diverse range of systemized implementations of smart and intelligent technologies networked within a system of systems (SoS), the terms “smart” and “intelligent” is frequently used in describing systems that perform complex operations with a reduced need of a human-agent. The difference between this research and most papers in publication on CPS is that most other research focuses on the performance of the CPS rather than on the correctness of its design. However, by using both human and machine agency at different levels of automation, or autonomy, the levels of automation have profound implications and affects to the reliability and safety of the CPS. The human-agent and the machine-agent are in a tidal lock of decision-making using both feedforward and feedback information flows in similar processes, where a transient shift within the level of automation when the CPS is operating can have undesired consequences. As CPS systems become more common, and higher levels of autonomy are embedded within them, the relationship between human-agent and machine-agent also becomes more complex, and the testing methodologies for verification and validation of performance and correctness also become more complex and less clear. A framework then is developed to help the practitioner to understand the difficulties and pitfalls of CPS designs and provides guidance to test engineering design of soft computational systems using combinations of modeling, simulation, and prototyping

Detection of exposed steel rebars based on deep-learning techniques and unmanned aerial vehicles

In recent years deep-learning techniques have been developed and applied to inspect cracks in RC structures. The accuracy of these techniques leads to believe that they may also be applied to the identification of other pathologies. This article proposes a technique for automated detection of exposed steel rebars. The tools developed rely on convolutional neural networks (CNNs) based on transfer-learning using AlexNet. Experiments were conducted in large-scale structures to assess the efficiency of the method. To circumvent limitations on the proximity access to structures as large as the ones used in the experiments, as well as increase cost efficiency, the image capture was performed using an unmanned aerial system (UAS). The final goal of the proposed methodology is to generate orthomosaic maps of the pathologies or structure 3D models with superimposed pathologies. The results obtained are promising, confirming the high adaptability of CNN based methodologies for structural inspection.This work was financially supported by: Base Funding - UIDB/04708/2020 and Programmatic Funding - UIDP/04708/2020 of the CONSTRUCT - Instituto de I&D em Estruturas e Construções funded by national funds through the FCT/MCTES (PIDDAC). Additionally, the author Rafael Cabral acknowledges the support provided by the doctoral grant UI/BD/150970/2021 - Portuguese Science Foundation, FCT/MCTES.info:eu-repo/semantics/publishedVersio

A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment