Performance Analysis and Evaluation of Object Detection Algorithms for Drone Networks

openThe YOLOv5 and YOLOv8 object identification algorithms are optimised in this thesis for edge devices such as the Raspberry Pi 4, which are becoming more and more important in decentralised computing environments where real-time processing capabilities are essential. The main goals are to guarantee high accuracy and low latency in applications like active surveillance and driverless driving, while also improving the operational efficiency of these algorithms on resource-constrained edge devices. Reducing power consumption is essential to this optimisation since it increases the lifespan of devices in distant or mobile environments where energy supplies are limited. Adapting these algorithms to edge-specific frameworks such as OpenVINO and NCNN is the goal of the research, which tries to preserve object detection integrity without sacrificing speed or power economy. The thesis assesses the performance trade-offs associated with these modifications by methodical testing, offering insights into how speed, precision, and power economy are balanced. The results could establish new standards for the efficient deployment of intelligent systems in resource-constrained environments, making a substantial contribution to the domains of autonomous technologies and real-time data processing.The YOLOv5 and YOLOv8 object identification algorithms are optimised in this thesis for edge devices such as the Raspberry Pi 4, which are becoming more and more important in decentralised computing environments where real-time processing capabilities are essential. The main goals are to guarantee high accuracy and low latency in applications like active surveillance and driverless driving, while also improving the operational efficiency of these algorithms on resource-constrained edge devices. Reducing power consumption is essential to this optimisation since it increases the lifespan of devices in distant or mobile environments where energy supplies are limited. Adapting these algorithms to edge-specific frameworks such as OpenVINO and NCNN is the goal of the research, which tries to preserve object detection integrity without sacrificing speed or power economy. The thesis assesses the performance trade-offs associated with these modifications by methodical testing, offering insights into how speed, precision, and power economy are balanced. The results could establish new standards for the efficient deployment of intelligent systems in resource-constrained environments, making a substantial contribution to the domains of autonomous technologies and real-time data processing

Enabling Multi-LiDAR Sensing in GNSS-Denied Environments: SLAM Dataset, Benchmark, and UAV Tracking with LiDAR-as-a-camera

The rise of Light Detection and Ranging (LiDAR) sensors has profoundly impacted industries ranging from automotive to urban planning. As these sensors become increasingly affordable and compact, their applications are diversifying, driving precision, and innovation. This thesis delves into LiDAR's advancements in autonomous robotic systems, with a focus on its role in simultaneous localization and mapping (SLAM) methodologies and LiDAR as a camera-based tracking for Unmanned Aerial Vehicles (UAV). Our contributions span two primary domains: the Multi-Modal LiDAR SLAM Benchmark, and the LiDAR-as-a-camera UAV Tracking. In the former, we have expanded our previous multi-modal LiDAR dataset by adding more data sequences from various scenarios. In contrast to the previous dataset, we employ different ground truth-generating approaches. We propose a new multi-modal multi-lidar SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. Additionally, we also supplement our data with new open road sequences with GNSS-RTK. This enriched dataset, supported by high-resolution LiDAR, provides detailed insights through an evaluation of ten configurations, pairing diverse LiDAR sensors with state-of-the-art SLAM algorithms. In the latter contribution, we leverage a custom YOLOv5 model trained on panoramic low-resolution images from LiDAR reflectivity (LiDAR-as-a-camera) to detect UAVs, demonstrating the superiority of this approach over point cloud or image-only methods. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform. Overall, our research underscores the transformative potential of integrating advanced LiDAR sensors with autonomous robotics. By bridging the gaps between different technological approaches, we pave the way for more versatile and efficient applications in the future

Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons

With the advancement of maritime unmanned aerial vehicles (UAVs) and deep learning technologies, the application of UAV-based object detection has become increasingly significant in the fields of maritime industry and ocean engineering. Endowed with intelligent sensing capabilities, the maritime UAVs enable effective and efficient maritime surveillance. To further promote the development of maritime UAV-based object detection, this paper provides a comprehensive review of challenges, relative methods, and UAV aerial datasets. Specifically, in this work, we first briefly summarize four challenges for object detection on maritime UAVs, i.e., object feature diversity, device limitation, maritime environment variability, and dataset scarcity. We then focus on computational methods to improve maritime UAV-based object detection performance in terms of scale-aware, small object detection, view-aware, rotated object detection, lightweight methods, and others. Next, we review the UAV aerial image/video datasets and propose a maritime UAV aerial dataset named MS2ship for ship detection. Furthermore, we conduct a series of experiments to present the performance evaluation and robustness analysis of object detection methods on maritime datasets. Eventually, we give the discussion and outlook on future works for maritime UAV-based object detection. The MS2ship dataset is available at \href{https://github.com/zcj234/MS2ship}{https://github.com/zcj234/MS2ship}.Comment: 32 pages, 18 figure

A Comparative study for object detection using UAV: Histogram vs YOLO

This thesis paper presents a comparative study of two popular face detection algorithms, You Only Look Once (YOLO) and Histogram of Oriented Gradients (HOG), for their performance in detecting faces using drones. The study compares the performance of YOLO and HOG on various parameters such as accuracy, speed, and detection rate. We also explore the impact of drone altitude and camera resolution on the performance of these algorithms. The experiments were conducted using a DJI Tello drone and a dataset of images captured at different altitudes and camera resolutions. The findings of this study have important implications for the use of drones for face detection in various fields. YOLO is a promising algorithm for face detection using drones due to its high accuracy, speed, and real-time capabilities. Our study contributes to the growing body of research on the use of drones for face detection and provides valuable insights into the comparative performance of YOLO and HOG

Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?

Despite the commercial abundance of UAVs, aerial data acquisition remains challenging, and the existing Asia and North America-centric open-source UAV datasets are small-scale or low-resolution and lack diversity in scene contextuality. Additionally, the color content of the scenes, solar-zenith angle, and population density of different geographies influence the data diversity. These two factors conjointly render suboptimal aerial-visual perception of the deep neural network (DNN) models trained primarily on the ground-view data, including the open-world foundational models. To pave the way for a transformative era of aerial detection, we present Multiview Aerial Visual RECognition or MAVREC, a video dataset where we record synchronized scenes from different perspectives -- ground camera and drone-mounted camera. MAVREC consists of around 2.5 hours of industry-standard 2.7K resolution video sequences, more than 0.5 million frames, and 1.1 million annotated bounding boxes. This makes MAVREC the largest ground and aerial-view dataset, and the fourth largest among all drone-based datasets across all modalities and tasks. Through our extensive benchmarking on MAVREC, we recognize that augmenting object detectors with ground-view images from the corresponding geographical location is a superior pre-training strategy for aerial detection. Building on this strategy, we benchmark MAVREC with a curriculum-based semi-supervised object detection approach that leverages labeled (ground and aerial) and unlabeled (only aerial) images to enhance the aerial detection. We publicly release the MAVREC dataset: https://mavrec.github.io

Estimation of sorghum seedling number from drone image based on support vector machine and YOLO algorithms

Accurately counting the number of sorghum seedlings from images captured by unmanned aerial vehicles (UAV) is useful for identifying sorghum varieties with high seedling emergence rates in breeding programs. The traditional method is manual counting, which is time-consuming and laborious. Recently, UAV have been widely used for crop growth monitoring because of their low cost, and their ability to collect high-resolution images and other data non-destructively. However, estimating the number of sorghum seedlings is challenging because of the complexity of field environments. The aim of this study was to test three models for counting sorghum seedlings rapidly and automatically from red-green-blue (RGB) images captured at different flight altitudes by a UAV. The three models were a machine learning approach (Support Vector Machines, SVM) and two deep learning approaches (YOLOv5 and YOLOv8). The robustness of the models was verified using RGB images collected at different heights. The R2 values of the model outputs for images captured at heights of 15 m, 30 m, and 45 m were, respectively, (SVM: 0.67, 0.57, 0.51), (YOLOv5: 0.76, 0.57, 0.56), and (YOLOv8: 0.93, 0.90, 0.71). Therefore, the YOLOv8 model was most accurate in estimating the number of sorghum seedlings. The results indicate that UAV images combined with an appropriate model can be effective for large-scale counting of sorghum seedlings. This method will be a useful tool for sorghum phenotyping

Bangladeshi Native Vehicle Detection in Wild

The success of autonomous navigation relies on robust and precise vehicle recognition, hindered by the scarcity of region-specific vehicle detection datasets, impeding the development of context-aware systems. To advance terrestrial object detection research, this paper proposes a native vehicle detection dataset for the most commonly appeared vehicle classes in Bangladesh. 17 distinct vehicle classes have been taken into account, with fully annotated 81542 instances of 17326 images. Each image width is set to at least 1280px. The dataset's average vehicle bounding box-to-image ratio is 4.7036. This Bangladesh Native Vehicle Dataset (BNVD) has accounted for several geographical, illumination, variety of vehicle sizes, and orientations to be more robust on surprised scenarios. In the context of examining the BNVD dataset, this work provides a thorough assessment with four successive You Only Look Once (YOLO) models, namely YOLO v5, v6, v7, and v8. These dataset's effectiveness is methodically evaluated and contrasted with other vehicle datasets already in use. The BNVD dataset exhibits mean average precision(mAP) at 50% intersection over union (IoU) is 0.848 corresponding precision and recall values of 0.841 and 0.774. The research findings indicate a mAP of 0.643 at an IoU range of 0.5 to 0.95. The experiments show that the BNVD dataset serves as a reliable representation of vehicle distribution and presents considerable complexities.Comment: 13 pages, 8 figure

Agricultural Object Detection with You Look Only Once (YOLO) Algorithm: A Bibliometric and Systematic Literature Review

Vision is a major component in several digital technologies and tools used in agriculture. The object detector, You Look Only Once (YOLO), has gained popularity in agriculture in a relatively short span due to its state-of-the-art performance. YOLO offers real-time detection with good accuracy and is implemented in various agricultural tasks, including monitoring, surveillance, sensing, automation, and robotics. The research and application of YOLO in agriculture are accelerating rapidly but are fragmented and multidisciplinary. Moreover, the performance characteristics (i.e., accuracy, speed, computation) of the object detector influence the rate of technology implementation and adoption in agriculture. Thus, the study aims to collect extensive literature to document and critically evaluate the advances and application of YOLO for agricultural object recognition. First, we conducted a bibliometric review of 257 articles to understand the scholarly landscape of YOLO in agricultural domain. Secondly, we conducted a systematic review of 30 articles to identify current knowledge, gaps, and modifications in YOLO for specific agricultural tasks. The study critically assesses and summarizes the information on YOLO's end-to-end learning approach, including data acquisition, processing, network modification, integration, and deployment. We also discussed task-specific YOLO algorithm modification and integration to meet the agricultural object or environment-specific challenges. In general, YOLO-integrated digital tools and technologies show the potential for real-time, automated monitoring, surveillance, and object handling to reduce labor, production cost, and environmental impact while maximizing resource efficiency. The study provides detailed documentation and significantly advances the existing knowledge on applying YOLO in agriculture, which can greatly benefit the scientific community

Machine learning and Unmanned Aerial Systems for crop monitoring and agrochemicals distribution optimization in orchard and horticultural systems

The work aims at discovering the potential and the efficiency of Unmanned Aerial Systems (UAS) and Machine Learning (ML) in agriculture scenario, focusing on crop management and agrochemicals distribution optimization in orchard and horticultural cropping systems. The dissertation includes a general introduction, three experimental chapters and a general conclusion. Chapter 2 illustrates an operational approach to estimate individual and aggregate vineyards’ canopy volume using the manual Tree-Row-Volume (TRV) and the remotely sensed Canopy Height Model (CHM) techniques, processed with MATLAB scripts, and validated through ArcGIS tools. The results confirm how the extensive use of TRV is recommended when supported by remote sensing, to better qualify errors and heterogeneities in field estimates. Chapter 3 presents the development of a grape bunch detector based on a deep convolutional neural network trained to work directly on the field in an uncontrolled environment. The presented results are promising since most of the bunches were correctly detected with a 91% mean average precision, not only on the GrapeCS-ML database used to train the system, but also on an internal dataset, confirming the portability to different scenarios. Chapter 4 reports artichoke plant deep learning-based detection and georeferencing as the first step for an on-the-fly UAS spraying system and uses the gathered information to crop development monitoring in a multi-temporal approach. The Feature Pyramid Network, trained and compared with the YOLOv5 network, showed a high detection level with an average F1 score of around 90%, and satisfactory off-line performances on the Nvidia Jetson Nano board. The multi-temporal approach influenced detection performances, with an inverse response of precision and recall metrics. The growing index trend showed a distinct value in October, peaking at the beginning of December as expectedThe work aims at discovering the potential and the efficiency of Unmanned Aerial Systems (UAS) and Machine Learning (ML) in agriculture scenario, focusing on crop management and agrochemicals distribution optimization in orchard and horticultural cropping systems. The dissertation includes a general introduction, three experimental chapters and a general conclusion. Chapter 2 illustrates an operational approach to estimate individual and aggregate vineyards’ canopy volume using the manual Tree-Row-Volume (TRV) and the remotely sensed Canopy Height Model (CHM) techniques, processed with MATLAB scripts, and validated through ArcGIS tools. The results confirm how the extensive use of TRV is recommended when supported by remote sensing, to better qualify errors and heterogeneities in field estimates. Chapter 3 presents the development of a grape bunch detector based on a deep convolutional neural network trained to work directly on the field in an uncontrolled environment. The presented results are promising since most of the bunches were correctly detected with a 91% mean average precision, not only on the GrapeCS-ML database used to train the system, but also on an internal dataset, confirming the portability to different scenarios. Chapter 4 reports artichoke plant deep learning-based detection and georeferencing as the first step for an on-the-fly UAS spraying system and uses the gathered information to crop development monitoring in a multi-temporal approach. The Feature Pyramid Network, trained and compared with the YOLOv5 network, showed a high detection level with an average F1 score of around 90%, and satisfactory off-line performances on the Nvidia Jetson Nano board. The multi-temporal approach influenced detection performances, with an inverse response of precision and recall metrics. The growing index trend showed a distinct value in October, peaking at the beginning of December as expected

Human-assisted self-supervised labeling of large data sets

There is a severe demand for, and shortage of, large accurately labeled datasets to train supervised computational intelligence (CI) algorithms in domains like unmanned aerial systems (UAS) and autonomous vehicles. This has hindered our ability to develop and deploy various computer vision algorithms in/across environments and niche domains for tasks like detection, localization, and tracking. Herein, I propose a new human-in-the-loop (HITL) based growing neural gas (GNG) algorithm to minimize human intervention during labeling large UAS data collections over a shared geospatial area. Specifically, I address human driven events like new class identification and mistake correction. I also address algorithm-centric operations like new pattern discovery and self-supervised labeling. Pattern discovery and identification through self-supervised labeling is made possible through open set recognition (OSR). Herein, I propose a classifier with the ability to say "I don't know" to identify outliers in the data and bootstrap deep learning (DL) models, specifically convolutional neural networks (CNNs), with the ability to classify on N+1 classes. The effectiveness of the algorithms are demonstrated using simulated realistic ray-traced low altitude UAS data from the Unreal Engine. The results show that it is possible to increase speed and reduce mental fatigue over hand labeling large image datasets.Includes bibliographical references