Ship Detection Feature Analysis in Optical Satellite Imagery through Machine Learning Applications

Ship detection remains an important challenge within the government and the commercial industry. Current research has focused on deep learning and has found high success with large labeled datasets. However, deep learning becomes insufficient for limited datasets as well as when explainability is required. There exist scenarios in which explainability and human-in-the-loop processing are needed, such as in naval applications. In these scenarios, handcrafted features and traditional classification algorithms can be useful. This research aims at analyzing multiple textures and statistical features on a small optical satellite imagery dataset. The feature analysis consists of Haar-like features, Haralick features, Hu moments, Histogram of Oriented Gradients, grayscale intensity histograms, and Local Binary Patterns. Feature performance is measured using 8 different classification algorithms, including K-Nearest Neighbors, Logistic Regression, Gradient Boosting, Extreme Gradient Boosting, Support Vector Machine, Random Decision Forest, Extremely Randomized Trees, and Bagging. The features are analyzed individually and in different combinations. Individual feature analysis results found Haralick features achieved a precision of 92.2% and were computationally efficient. The best combination of features was Haralick features paired with Histogram of Oriented Gradients and grayscale intensity histograms. This combination achieved a precision score of 96.18% and an F1 score of 94.23%

Remote Sensing Image Scene Classification: Benchmark and State of the Art

Remote sensing image scene classification plays an important role in a wide range of applications and hence has been receiving remarkable attention. During the past years, significant efforts have been made to develop various datasets or present a variety of approaches for scene classification from remote sensing images. However, a systematic review of the literature concerning datasets and methods for scene classification is still lacking. In addition, almost all existing datasets have a number of limitations, including the small scale of scene classes and the image numbers, the lack of image variations and diversity, and the saturation of accuracy. These limitations severely limit the development of new approaches especially deep learning-based methods. This paper first provides a comprehensive review of the recent progress. Then, we propose a large-scale dataset, termed "NWPU-RESISC45", which is a publicly available benchmark for REmote Sensing Image Scene Classification (RESISC), created by Northwestern Polytechnical University (NWPU). This dataset contains 31,500 images, covering 45 scene classes with 700 images in each class. The proposed NWPU-RESISC45 (i) is large-scale on the scene classes and the total image number, (ii) holds big variations in translation, spatial resolution, viewpoint, object pose, illumination, background, and occlusion, and (iii) has high within-class diversity and between-class similarity. The creation of this dataset will enable the community to develop and evaluate various data-driven algorithms. Finally, several representative methods are evaluated using the proposed dataset and the results are reported as a useful baseline for future research.Comment: This manuscript is the accepted version for Proceedings of the IEE

RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images

Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. Despite some saliency models were proposed to solve the intrinsic problem of optical RSIs (such as complex background and scale-variant objects), the accuracy and completeness are still unsatisfactory. To this end, we propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs in this paper. The relational reasoning module that integrates the spatial and the channel dimensions is designed to infer the semantic relationship by utilizing high-level encoder features, thereby promoting the generation of more complete detection results. The parallel multi-scale attention module is proposed to effectively restore the detail information and address the scale variation of salient objects by using the low-level features refined by multi-scale attention. Extensive experiments on two datasets demonstrate that our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.Comment: 11 pages, 9 figures, Accepted by IEEE Transactions on Geoscience and Remote Sensing 2021, project: https://rmcong.github.io/proj_RRNet.htm

Object Detection in 20 Years: A Survey

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Its development in the past two decades can be regarded as an epitome of computer vision history. If we think of today's object detection as a technical aesthetics under the power of deep learning, then turning back the clock 20 years we would witness the wisdom of cold weapon era. This paper extensively reviews 400+ papers of object detection in the light of its technical evolution, spanning over a quarter-century's time (from the 1990s to 2019). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed up techniques, and the recent state of the art detection methods. This paper also reviews some important detection applications, such as pedestrian detection, face detection, text detection, etc, and makes an in-deep analysis of their challenges as well as technical improvements in recent years.Comment: This work has been submitted to the IEEE TPAMI for possible publicatio

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Ship recognition on the sea surface using aerial images taken by Uav : a deep learning approach

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesOceans are very important for mankind, because they are a very important source of food, they have a very large impact on the global environmental equilibrium, and it is over the oceans that most of the world commerce is done. Thus, maritime surveillance and monitoring, in particular identifying the ships used, is of great importance to oversee activities like fishing, marine transportation, navigation in general, illegal border encroachment, and search and rescue operations. In this thesis, we used images obtained with Unmanned Aerial Vehicles (UAVs) over the Atlantic Ocean to identify what type of ship (if any) is present in a given location. Images generated from UAV cameras suffer from camera motion, scale variability, variability in the sea surface and sun glares. Extracting information from these images is challenging and is mostly done by human operators, but advances in computer vision technology and development of deep learning techniques in recent years have made it possible to do so automatically. We used four of the state-of-art pretrained deep learning network models, namely VGG16, Xception, ResNet and InceptionResNet trained on ImageNet dataset, modified their original structure using transfer learning based fine tuning techniques and then trained them on our dataset to create new models. We managed to achieve very high accuracy (99.6 to 99.9% correct classifications) when classifying the ships that appear on the images of our dataset. With such a high success rate (albeit at the cost of high computing power), we can proceed to implement these algorithms on maritime patrol UAVs, and thus improve Maritime Situational Awareness

Detección de embarcaciones utilizando Deep Learning e imágenes satelitales ópticas

La detección de embarcaciones es un tema prioritario que ayuda a combatir la pesca ilegal, en búsqueda y rescate de navíos perdidos, entre otras actividades prioritarias en el mar Actualmente el uso técnicas de Aprendizaje Profundo en la detección de objetos está dando buenos resultados sobre imágenes satelitales. En la presente investigación se presenta un modelo que permite detectar embarcaciones dentro de las 100 millas del borde costero del Perú, utilizando técnicas de Aprendizaje Profundo e Imágenes Satelitales. Se realizó una comparación entre la última versión de You Only Look Once (YOLO) y You Only Look Twice (YOLT) para resolver el problema de detectar objetos pequeños (barcos) en el mar sobre imágenes satelitales ópticas debido a la gran diversidad de embarcaciones que existen en el Perú. Se trabajó con dos conjuntos de datos: High-Resolution Ship Collection (HRSC) y Mini Ship Data Set (MSDS), este último fue construido a partir de embarcaciones provenientes del borde costero del Perú. El ancho promedio de los objetos para HRSC y MSDS son 150 y 50 píxeles respectivamente. Los resultados mostraron que YOLT es bueno solo para objetos pequeños con 76,06% de Average Precision (AP), mientras que YOLO alcanzó 69,80 % en el conjunto de datos HRSC. Además, en el caso del conjunto de datos HRSC donde tienen objetos de diferentes tamaños, YOLT obtuvo un 40% de AP contra 75% de YOL