Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network

The detection performance of small objects in remote sensing images is not satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) shows remarkable image enhancement performance, but reconstructed images miss high-frequency edge information. Therefore, object detection performance degrades for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we apply a new edge-enhanced super-resolution GAN (EESRGAN) to improve the image quality of remote sensing images and use different detector networks in an end-to-end manner where detector loss is backpropagated into the EESRGAN to improve the detection performance. We propose an architecture with three components: ESRGAN, Edge Enhancement Network (EEN), and Detection network. We use residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we use the faster region-based convolutional network (FRCNN) (two-stage detector) and single-shot multi-box detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) and a self-assembled (oil and gas storage tank) satellite dataset show superior performance of our method compared to the standalone state-of-the-art object detectors.Comment: This paper contains 27 pages and accepted for publication in MDPI remote sensing journal. GitHub Repository: https://github.com/Jakaria08/EESRGAN (Implementation

Object detection for single tree species identification with high resolution aerial images

Dissertation submitted in partial fulfilment of the requirements for the degree of Master of Science in Geospatial TechnologiesObject recognition is one of the computer vision tasks developing rapidly with the invention of Region-based Convolutional Neural Network (RCNN). This thesis contains a study conducted using RCNN base object detection technique to identify palm trees in three datasets having RGB images taken by Unnamed Aerial Vehicles (UAVs). The method was entirely implemented using TensorFlow object detection API to compare the performance of pre-trained faster RCNN object detection models. According to the results, best performance was recorded with the highest overall accuracy of 93.1 ± 4.5 % and the highest speed of 9m 57s from faster RCNN model which was having inceptionv2 as feature extractor. The poorest performance was recorded with the lowest overall accuracy of 65.2 ± 10.9% and the lowest speed of 5h 39m 15s from faster RCNN model which was having inception_resnetv2 as feature extractor

Enhanced faster region-based convolutional neural network for oil palm tree detection

Oil palm trees are important economic crops in Malaysia. One of the audit procedures is to count the number of oil palm trees for plantation management, which helps the manager predict the plantation yield and the amount of fertilizer and labor force needed. However, the current counting method for oil palm tree plantation is manually counting using GIS software, which is tedious and inefficient for large scale plantation. To overcome this problem, researchers proposed automatic counting methods based on machine learning and image processing. However, traditional machine learning and image processing methods used handcrafted feature extraction methods. It can only extract low-middle level features from the image and lack of generalization ability. It’s applicable only for one application and will need reprogramming for other applications. The widely used feature extraction methods are local binary patterns (LBP), scale-invariant feature transform (SIFT), and the histogram of oriented gradients (HOG), which usually achieve low accuracy because of their limited feature representation ability and without generalization capability. Hence, this research aims to close the research gaps by exploring the deep learning-based object detection algorithm and the classical convolutional neural network (CNN) to build an automatic deep learning-based oil palm tree detection and counting framework. This study proposed a new deep learning method based on Faster RCNN for oil palm tree detection and counting. To reduce the overfitting problem during the training, this study uses the image processing method to augment the training dataset by random flipping the image and to increase the data’s contrast and brightness. The transfer learning model of ResNet50 was used as the CNN backbone and the Faster RCNN network was retrained to get the weight for automatic oil palm tree counting. To improve the performance of Faster RCNN, feature concatation method was used to integrate the high-level and low-level feature from ResNet50. The proposed model validated the testing dataset of three palm tree regions with mature, young, and mixed mature and young palm trees. The detection results were compared with two machine learning methods of ANN, SVM, image processing-based TM method, and the original Faster RCNN model respectively. The proposed enhanced Faster RCNN model shows a promising result of oil palm tree detection and counting. It achieved an overall accuracy of 97% in the testing dataset, 97.2% in the mixed palm tree region, and 96.9% in the mature and young palm tree region, while the traditional ANN, SVM, and TM methods are less than 90%. The accuracy of comparison reveals that the proposed EFRCNN model outperforms the Faster RCNN and the traditional ANN, SVM, and TM methods. It has the potential to apply in counting a large area of oil palm tree plantation

Autonomous palm tree detection from remote sensing images-UAE dataset

Autonomous detection and counting of palm trees is a research field of interest to various countries around the world, including the UAE. Automating this task saves effort and resources by minimizing human intervention and reducing potential errors in counting. This paper introduces a new High Resolution (HR) remote sensing dataset for autonomous detection of palm trees in the UAE. The dataset is collected using Unmanned Aerial Vehicles (UAV), and it is labeled properly in PASCAL VOC and YOLO formats after preprocessing and visually inspecting its quality. A comparative evaluation between Faster-RCNN and YOLOv4 networks is then conducted to observe the usability of the dataset in addition to the strengths and weaknesses of each network. The dataset is publicly available at https://github.com/Nour093/Palm-Tree-Dataset

Computer vision for plant and animal inventory

The population, composition, and spatial distribution of the plants and animals in certain regions are always important data for natural resource management, conservation and farming. The traditional ways to acquire such data require human participation. The procedure of data processing by human is usually cumbersome, expensive and time-consuming. Hence the algorithms for automatic animal and plant inventory show their worth and become a hot topic. We propose a series of computer vision methods for automated plant and animal inventory, to recognize, localize, categorize, track and count different objects of interest, including vegetation, trees, fishes and livestock animals. We make use of different sensors, hardware platforms, neural network architectures and pipelines to deal with the varied properties and challenges of these objects. (1) For vegetation analysis, we propose a fast multistage method to estimate the coverage. The reference board is localized based on its edge and texture features. And then a K-means color model of the board is generated. Finally, the vegetation is segmented at pixel level using the color model. The proposed method is robust to lighting condition changes. (2) For tree counting in aerial images, we propose a novel method called density transformer, or DENT, to learn and predict the density of the trees at different positions. DENT uses an efficient multi-receptive field network to extract visual features from different positions. A transformer encoder is applied to filter and transfer useful contextual information across different spatial positions. DENT significantly outperformed the existing state-of-art CNN detectors and regressors on both the dataset built by ourselves and an existing cross-site dataset. (3) We propose a framework of fish classification system using boat cameras. The framework contains two branches. A branch extracts the contextual information from the whole image. The other branch localizes all the individual fish and normalizes their poses. The classification results from the two branches are weighted based on the clearness of the image and the familiarness of the context. Our system achieved the top 1 percent rank in the competition of The Nature Conservancy Fisheries Monitoring. (4) We also propose a video-based pig counting algorithm using an inspection robot. We adopt a novel bottom-up keypoint tracking method and a novel spatial-aware temporal response filtering method to count the pigs. The proposed approach outperformed the other methods and even human competitors in the experiments.Includes bibliographical references

Using Prior Knowledge for Verification and Elimination of Stationary and Variable Objects in Real-time Images

With the evolving technologies in the autonomous vehicle industry, now it has become possible for automobile passengers to sit relaxed instead of driving the car. Technologies like object detection, object identification, and image segmentation have enabled an autonomous car to identify and detect an object on the road in order to drive safely. While an autonomous car drives by itself on the road, the types of objects surrounding the car can be dynamic (e.g., cars and pedestrians), stationary (e.g., buildings and benches), and variable (e.g., trees) depending on if the location or shape of an object changes or not. Different from the existing image-based approaches to detect and recognize objects in the scene, in this research 3D virtual world is employed to verify and eliminate stationary and variable objects to allow the autonomous car to focus on dynamic objects that may cause danger to its driving. This methodology takes advantage of prior knowledge of stationary and variable objects presented in a virtual city and verifies their existence in a real-time scene by matching keypoints between the virtual and real objects. In case of a stationary or variable object that does not exist in the virtual world due to incomplete pre-existing information, this method uses machine learning for object detection. Verified objects are then removed from the real-time image with a combined algorithm using contour detection and class activation map (CAM), which helps to enhance the efficiency and accuracy when recognizing moving objects

Artificial Neural Networks in Agriculture

Modern agriculture needs to have high production efficiency combined with a high quality of obtained products. This applies to both crop and livestock production. To meet these requirements, advanced methods of data analysis are more and more frequently used, including those derived from artificial intelligence methods. Artificial neural networks (ANNs) are one of the most popular tools of this kind. They are widely used in solving various classification and prediction tasks, for some time also in the broadly defined field of agriculture. They can form part of precision farming and decision support systems. Artificial neural networks can replace the classical methods of modelling many issues, and are one of the main alternatives to classical mathematical models. The spectrum of applications of artificial neural networks is very wide. For a long time now, researchers from all over the world have been using these tools to support agricultural production, making it more efficient and providing the highest-quality products possible

A tree species classification model based on improved YOLOv7 for shelterbelts

Tree species classification within shelterbelts is crucial for shelterbelt management. The large-scale satellite-based and low-altitude drone-based approaches serve as powerful tools for forest monitoring, especially in tree species classification. However, these methods face challenges in distinguishing individual tree species within complex backgrounds. Additionally, the mixed growth of trees within protective forest suffers from similar crown size among different tree species. The complex background of the shelterbelts negatively impacts the accuracy of tree species classification. The You Only Look Once (YOLO) algorithm is widely used in the field of agriculture and forestry, ie., plant and fruit identification, pest and disease detection, and tree species classification in forestry. We proposed a YOLOv7-Kmeans++_CoordConv_CBAM (YOLOv7-KCC) model for tree species classification based on drone RGB remote sensing images. Firstly, we constructed a dataset for tree species in shelterbelts and adopted data augmentation methods to mitigate overfitting due to limited training data. Secondly, the K-means++ algorithm was employed to cluster anchor boxes in the dataset. Furthermore, to enhance the YOLOv7 backbone network’s Efficient Layer Aggregation Network (ELAN) module, we used Coordinate Convolution (CoordConv) replaced the ordinary 1×1 convolution. The Convolutional Block Attention Module (CBAM) was integrated into the Path Aggregation Network (PANet) structure to facilitate multiscale feature extraction and fusion, allowing the network to better capture and utilize crucial feature information. Experimental results showed that the YOLOv7-KCC model achieves a mean average [email protected] of 98.91%, outperforming the Faster RCNN-VGG16, Faster RCNN-Resnet50, SSD, YOLOv4, and YOLOv7 models by 5.71%, 11.75%, 5.97%, 7.86%, and 3.69%, respectively. The GFlops and Parameter values of the YOLOv7-KCC model stand at 105.07G and 143.7MB, representing an almost 5.6% increase in F1 metrics compared to YOLOv7. Therefore, the proposed YOLOv7-KCC model can effectively classify shelterbelt tree species, providing a scientific theoretical basis for shelterbelt management in Northwest China focusing on Xinjiang

Landscape Inventory and Harvest Strategies for Individual Western Juniper in Eastern Oregon

Western juniper is a native species in eastern Oregon that became invasive during the last century since its range increased fivefold from 1936 to 1988. Western juniper’s ability to absorb rainfall and groundwater has deleterious effects on stream flow and sensitive sage grouse habitat in eastern Oregon. New methods of western juniper remediation have been proposed, including harvesting and processing western juniper as merchantable timber. An essential part of these methods is the accurate survey of trees and a harvesting plan. The objective of my thesis was to estimate the existing juniper resource in Wheeler county, Oregon, and to develop a harvesting plan for the resource. To estimate the existing juniper resource, I implemented two methods of segmenting individual tree using their ground-projected crown and applied them to orthorectified multispectral aerial images. This work involved estimating the canopy height model from multispectral aerial imagery, which was performed with a generative adversarial network. The generated canopy height model was used with the most accurate tree crown detection method to create a map of juniper locations, with their corresponding height. Based on the knowledge of location and size of each juniper I developed a novel individual tree level harvest strategy. The strategy was evaluated using two heuristic techniques, simulated annealing and record to record, on two areas, each of approximately 1600 ha. The results indicate that landscape restoration by removing junipers depends on the amount and value of the junipers, and in many cases, financial incentives would likely be needed