Integration of stacked-autoencoders and convolutional neural networks for hyperspectral image classification

Orientador: Prof. Dr. Jorge Antônio Silva CentenoTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências da Terra, Programa de Pós-Graduação em Ciências Geodésicas. Defesa : Curitiba, 24/05/2021Inclui referências: p. 97-103Resumo: Deep Learning ou aprendizado profundo abriu novas possibilidades para o pré-processamento, processamento e análise de dados hiperespectrais usando várias camadas de redes neurais e pode ser usado como ferramenta de extração de atributos. Nesta pesquisa, é desenvolvido um modelo híbrido baseado em pixels que integra Stacked-Autoencoders (SAE) y Redes Neurais Convolucionais (CNN) para classificar dados hiperespectrais. O núcleo do modelo integrado (SAE-1DCNN) é um Autoencoder que é aprimorado usando camadas convolucionais nas etapas de codificação (encoding) e decodificação (decoding). Isso permite melhorar a discriminação de dados no treinamento não supervisionado e reduzir o tempo no processamento, pois permite uma descrição dos atributos baseada na assinatura hiperespectral do pixel e aproveita a eficácia da arquitetura profunda com base nas camadas convolucionais e pooling. Como filtros unidimensionais foram aplicados no modelo integrado, o tempo de processamento é consideravelmente menor do que ao usar filtros 2D-CNN. Em uma primeira etapa, o modelo SAE-1DCNN é usado para extração de atributos e, em seguida, esses resultados são usados em uma etapa final para uma classificação supervisionada. Assim, na primeira etapa os parâmetros da rede são ajustados usando amostras de treinamento e após na segunda etapa uma abordagem fine-tuning composta de regressão logística com base na função de ativação softmax foi aplicada para classificação. Três aspectos são analisados nesta pesquisa: a capacidade do modelo de excluir bandas ruidosas, sua capacidade de redução da dimensionalidade e seu potencial para realizar a classificação da cobertura da terra usando dados hiperespectrais. Os experimentos foram realizados com diferentes conjuntos de dados hiperespectrais: Indian Pines, Universidade de Pavia e Salinas, amplamente utilizados pela comunidade científica, e uma imagem hiperespectral capturada na Fazenda Canguiri da Universidade Federal do Paraná (UFPR) no Paraná-Brasil. Para validar a metodologia proposta, os resultados obtidos foram comparados aos métodos tradicionais de aprendizado de máquina para verificar o potencial da integração de autoencoders (AE) e redes convolucionais. Os resultados obtidos mostraram similaridade com os métodos tradicionais em termos de acurácia da classificação hiperespectral, porém demandaram menos tempo de processamento, portanto, a metodologia proposta (SAE-1DCNN) é considerada promissora, sólida e pode ser uma alternativa para o pré-processamento de dados hiperespectrais e processamento.Abstract: Deep learning opened new possibilities for hyperspectral data processing and analysis using multiple neural nets layers and can be used as a feature extraction tool. In this research, a pixel-based hybrid model is developed that integrates Stacked-Autoencoders (SAE) and Convolutional Neural Network (CNN) for hyperspectral image classification. The core of the integrated model (SAE-1DCNN) is an autoencoder that is improved by using convolutional layers in the encoding and decoding steps. This allows improving data discrimination in unsupervised training and reducing the processing time because it allows a feature-based description of the pixel's hyperspectral signature and takes advantage of the effectiveness of deep architecture based on the convolutional and pooling layers. As one-dimensional filters are applied, the processing time is considerably lower than when using 2D-CNN filters. In a first step, the SAE-1DCNN model is used for feature extraction and then these results are used in a final supervised classification step. Thus, in the first stage, the parameters of the net are adjusted using training samples and then, in the second stage, a fine-tuning approach followed by logistic regression based on the softmax activation function was applied for classification. Three aspects are analyzed in detail: the capacity of the model to exclude noisy bands, its ability to dimensionality reduction, and its potential to perform land cover classification based on hyperspectral data. Experiments were performed using different hyperspectral data sets: Indian Pines University of Pavia and Salinas, widely used by the scientific community, and a hyperspectral image captured at the Canguiri Farm of the Federal University of Paraná (UFPR) in Paraná-Brazil. To validate the proposed methodology, the obtained results were compared to traditional machine learning methods to verify the potential of the integration of autoencoders (AE) and convolutional nets. These obtained results showed similarity with traditional methods in terms of hyperspectral classification accuracy, however, they demanded less time for processing, therefore, the proposed methodology (SAE-1DCNN) is considered promising, solid, and can be an alternative for hyperspectral data pre-processing and processing.Resumen: Deep Learning o aprendizaje profundo abrió nuevos desafíos para el preprocesamiento, procesamiento y análisis de datos hiperespectrales usando varias capas de redes neuronales y puede ser usado como herramienta de extracción de atributos. En esta investigación, se desarrolla un modelo híbrido basado en pixeles que integra Stacked-Autoencoders (SAE) y redes Neuronales Convolucionales (CNN) para clasificar datos hiperespectrales. Este enfoque uso un modelo basado en pixeles que integra Convolutional Neural Networks (CNN) y Stacked-Autoencoders (SAE). El núcleo del modelo integrado (SAE-1DCNN) es un Autoencoder (AE) mejorado que usa capas convolucionales en las etapas de codificación y decodificación. Esto permite mejorar la discriminación de datos a través de un entrenamiento supervisado y además reducir el tiempo en el procesamiento, pues permite una descripción de los atributos basad en la respuesta hiperespectral del pixel y aprovecha la efectividad de la arquitectura profunda en las capas convolucionales (convolutional) y de agrupamiento (pooling). En este modelo integrado se aplican filtros unidimensionales lo que permite que el tiempo en el procesamiento sea menor si se compara con los filtros bidimensionales 2D-CNN. En una primera etapa, el modelo SAE-1DCNN es usado para la extracción de atributos y en seguida, esos resultados son usados para la etapa final basada en la clasificación supervisada. De esta forma, en la primera etapa los parámetros de la red son ajustados usando las muestras de entrenamiento y después en la segunda etapa el enfoque conocido como fine-tuning fue aplicado para la clasificación de cobertura terrestre basado en regresión logística y la función de activación softmax. Tres aspectos son analizados en esta investigación, la capacidad del modelo para excluir bandas ruidosas, la capacidad para seleccionar las bandas redundantes y así reducir la dimensionalidad y el potencial para realizar la clasificación de la cobertura terrestre usando datos hiperespectrales. Los experimentos fueron realizados con diferentes conjuntos de datos hiperespectrales: Indian Pines, Universidad de Pavia y Salinas, ampliamente usados en trabajos científicos, y una imagen hiperespectral capturada en la Hacienda Canguiri de la Universidad Federal de Paraná (UFPR) en Paraná-Brasil. Para validar la metodología propuesta, los resultados obtenidos se compararon con métodos tradicionales de aprendizaje de máquina (machine learning) para verificar el potencial de la integración de Autoencoders (AE) y redes convolucionales. Los resultados obtenidos mostraron similitud con los métodos tradicionales en cuanto a la precisión de clasificación hiperespectral, sin embargo, exigieron menos tiempo de procesamiento, por lo que, la metodología propuesta (SAE-1DCNN) se considera prometedora, sólida y puede ser una alternativa para el pré-procesamiento y procesamiento de datos hiperespectrales

Klasifikasi Citra Hiperspektral Pada Kasus Tutupan Lahan Menggunakan Metode Convolutional Neural Network (CNN)

Informasi tutupan lahan dengan citra penginderaan jauh (inderaja) berbasis hiperspektral sangat efektif dalam pengelolaan peruntukan penggunaan lahan secara tepat. Selain dapat memberikan informasi keragaman spasial secara luas, cepat dan mudah, citra ini memiliki ratusan band spektral yang dapat memberikan struktur informasi permukaan bumi berdasarkan reflektansi gelombang elektromagnetik yang diterimanya. Metode One Dimensional Convolutional Neural Network (1D CNN) menunjukkan performa yang cukup baik pada klasifikasi tutupan lahan berbasis citra hiperspektral. Pada penelitian ini akan dilakukan analisis performa metode 1D CNN pada dataset Indian Pines 16 kelas, dimana sebelumnya metode 1D CNN diimplementasikan pada dataset Indian Pines 9 kelas. Hasil klasifikasi terbaik diperoleh pada percobaan 10000 epoch dari lima percobaan epoch yang berbeda dengan Overall Accuracy (OA) 88.97% dan Kappa 87.4%

Hyperspectral Image Classification -- Traditional to Deep Models: A Survey for Future Prospects

Hyperspectral Imaging (HSI) has been extensively utilized in many real-life applications because it benefits from the detailed spectral information contained in each pixel. Notably, the complex characteristics i.e., the nonlinear relation among the captured spectral information and the corresponding object of HSI data make accurate classification challenging for traditional methods. In the last few years, Deep Learning (DL) has been substantiated as a powerful feature extractor that effectively addresses the nonlinear problems that appeared in a number of computer vision tasks. This prompts the deployment of DL for HSI classification (HSIC) which revealed good performance. This survey enlists a systematic overview of DL for HSIC and compared state-of-the-art strategies of the said topic. Primarily, we will encapsulate the main challenges of traditional machine learning for HSIC and then we will acquaint the superiority of DL to address these problems. This survey breakdown the state-of-the-art DL frameworks into spectral-features, spatial-features, and together spatial-spectral features to systematically analyze the achievements (future research directions as well) of these frameworks for HSIC. Moreover, we will consider the fact that DL requires a large number of labeled training examples whereas acquiring such a number for HSIC is challenging in terms of time and cost. Therefore, this survey discusses some strategies to improve the generalization performance of DL strategies which can provide some future guidelines

Land Use and Land Cover Classification Using Deep Learning Techniques

abstract: Large datasets of sub-meter aerial imagery represented as orthophoto mosaics are widely available today, and these data sets may hold a great deal of untapped information. This imagery has a potential to locate several types of features; for example, forests, parking lots, airports, residential areas, or freeways in the imagery. However, the appearances of these things vary based on many things including the time that the image is captured, the sensor settings, processing done to rectify the image, and the geographical and cultural context of the region captured by the image. This thesis explores the use of deep convolutional neural networks to classify land use from very high spatial resolution (VHR), orthorectified, visible band multispectral imagery. Recent technological and commercial applications have driven the collection a massive amount of VHR images in the visible red, green, blue (RGB) spectral bands, this work explores the potential for deep learning algorithms to exploit this imagery for automatic land use/ land cover (LULC) classification. The benefits of automatic visible band VHR LULC classifications may include applications such as automatic change detection or mapping. Recent work has shown the potential of Deep Learning approaches for land use classification; however, this thesis improves on the state-of-the-art by applying additional dataset augmenting approaches that are well suited for geospatial data. Furthermore, the generalizability of the classifiers is tested by extensively evaluating the classifiers on unseen datasets and we present the accuracy levels of the classifier in order to show that the results actually generalize beyond the small benchmarks used in training. Deep networks have many parameters, and therefore they are often built with very large sets of labeled data. Suitably large datasets for LULC are not easy to come by, but techniques such as refinement learning allow networks trained for one task to be retrained to perform another recognition task. Contributions of this thesis include demonstrating that deep networks trained for image recognition in one task (ImageNet) can be efficiently transferred to remote sensing applications and perform as well or better than manually crafted classifiers without requiring massive training data sets. This is demonstrated on the UC Merced dataset, where 96% mean accuracy is achieved using a CNN (Convolutional Neural Network) and 5-fold cross validation. These results are further tested on unrelated VHR images at the same resolution as the training set.Dissertation/ThesisMasters Thesis Computer Science 201