Computer Vision Applications for Autonomous Aerial Vehicles

Undoubtedly, unmanned aerial vehicles (UAVs) have experienced a great leap forward over the last decade. It is not surprising anymore to see a UAV being used to accomplish a certain task, which was previously carried out by humans or a former technology. The proliferation of special vision sensors, such as depth cameras, lidar sensors and thermal cameras, and major breakthroughs in computer vision and machine learning fields accelerated the advance of UAV research and technology. However, due to certain unique challenges imposed by UAVs, such as limited payload capacity, unreliable communication link with the ground stations and data safety, UAVs are compelled to perform many tasks on their onboard embedded processing units, which makes it difficult to readily implement the most advanced algorithms on UAVs. This thesis focuses on computer vision and machine learning applications for UAVs equipped with onboard embedded platforms, and presents algorithms that utilize data from multiple modalities. The presented work covers a broad spectrum of algorithms and applications for UAVs, such as indoor UAV perception, 3D understanding with deep learning, UAV localization, and structural inspection with UAVs. Visual guidance and scene understanding without relying on pre-installed tags or markers is the desired approach for fully autonomous navigation of UAVs in conjunction with the global positioning systems (GPS), or especially when GPS information is either unavailable or unreliable. Thus, semantic and geometric understanding of the surroundings become vital to utilize vision as guidance in the autonomous navigation pipelines. In this context, first, robust altitude measurement, safe landing zone detection and doorway detection methods are presented for autonomous UAVs operating indoors. These approaches are implemented on Google Project Tango platform, which is an embedded platform equipped with various sensors including a depth camera. Next, a modified capsule network for 3D object classification is presented with weight optimization so that the network can be fit and run on memory-constrained platforms. Then, a semantic segmentation method for 3D point clouds is developed for a more general visual perception on a UAV equipped with a 3D vision sensor. Next, this thesis presents algorithms for structural health monitoring applications involving UAVs. First, a 3D point cloud-based, drift-free and lightweight localization method is presented for depth camera-equipped UAVs that perform bridge inspection, where GPS signal is unreliable. Next, a thermal leakage detection algorithm is presented for detecting thermal anomalies on building envelopes using aerial thermography from UAVs. Then, building on our thermal anomaly identification expertise gained on the previous task, a novel performance anomaly identification metric (AIM) is presented for more reliable performance evaluation of thermal anomaly identification methods

Survey on video anomaly detection in dynamic scenes with moving cameras

The increasing popularity of compact and inexpensive cameras, e.g.~dash cameras, body cameras, and cameras equipped on robots, has sparked a growing interest in detecting anomalies within dynamic scenes recorded by moving cameras. However, existing reviews primarily concentrate on Video Anomaly Detection (VAD) methods assuming static cameras. The VAD literature with moving cameras remains fragmented, lacking comprehensive reviews to date. To address this gap, we endeavor to present the first comprehensive survey on Moving Camera Video Anomaly Detection (MC-VAD). We delve into the research papers related to MC-VAD, critically assessing their limitations and highlighting associated challenges. Our exploration encompasses three application domains: security, urban transportation, and marine environments, which in turn cover six specific tasks. We compile an extensive list of 25 publicly-available datasets spanning four distinct environments: underwater, water surface, ground, and aerial. We summarize the types of anomalies these datasets correspond to or contain, and present five main categories of approaches for detecting such anomalies. Lastly, we identify future research directions and discuss novel contributions that could advance the field of MC-VAD. With this survey, we aim to offer a valuable reference for researchers and practitioners striving to develop and advance state-of-the-art MC-VAD methods.Comment: Under revie

Autonomous Systems, Robotics, and Computing Systems Capability Roadmap: NRC Dialogue

Contents include the following: Introduction. Process, Mission Drivers, Deliverables, and Interfaces. Autonomy. Crew-Centered and Remote Operations. Integrated Systems Health Management. Autonomous Vehicle Control. Autonomous Process Control. Robotics. Robotics for Solar System Exploration. Robotics for Lunar and Planetary Habitation. Robotics for In-Space Operations. Computing Systems. Conclusion

Exploring the use of unmanned aerial vehicles for automated oil spill identification

The potential use of unmanned aerial vehicles for the identification of oil spills poses a number of critical challenges related to the selection and justification of the hardware and software to be used. The purpose of this article is to analyse the technical solutions and characteristics required to enable the use of unmanned aerial vehicles in oil spill identification tasks. The author provides a comprehensive study of the issue presented, including a description of the necessary technical equipment and requirements in relation to the composition of the unmanned aerial vehicle

GPS Anomaly Detection And Machine Learning Models For Precise Unmanned Aerial Systems

The rapid development and deployment of 5G/6G networks have brought numerous benefits such as faster speeds, enhanced capacity, improved reliability, lower latency, greater network efficiency, and enablement of new applications. Emerging applications of 5G impacting billions of devices and embedded electronics also pose cyber security vulnerabilities. This thesis focuses on the development of Global Positioning Systems (GPS) Based Anomaly Detection and corresponding algorithms for Unmanned Aerial Systems (UAS). Chapter 1 provides an overview of the thesis background and its objectives. Chapter 2 presents an overview of the 5G architectures, their advantages, and potential cyber threat types. Chapter 3 addresses the issue of GPS dropouts by taking the use case of the Dallas-Fort Worth (DFW) airport. By analyzing data from surveillance drones in the (DFW) area, its message frequency, and statistics on time differences between GPS messages were examined. Chapter 4 focuses on modeling and detecting false data injection (FDI) on GPS. Specifically, three scenarios, including Gaussian noise injection, data duplication, data manipulation are modeled. Further, multiple detection schemes that are Clustering-based and reinforcement learning techniques are deployed and detection accuracy were investigated. Chapter 5 shows the results of Chapters 3 and 4. Overall, this research provides a categorization and possible outlier detection to minimize the GPS interference for UAS enhancing the security and reliability of UAS operations