Exploring the use of neural network-based band selection on hyperspectral imagery to identify informative wavelengths for improving classifier task performance

Hyperspectral imagery is a highly dimensional type of data resulting in high computational costs during analysis. Band selection aims to reduce the original hyperspectral image to a smaller subset that reduces these costs while preserving the maximum amount of spectral information within the data. This thesis explores various types of band selection techniques used in hyperspectral image processing. Modifying Neural network-based techniques and observing the effects on the band selection process due to the change in network architecture or objective are of particular focus in this thesis. Herein, a generalized neural network-based band selection technique is developed and compared to state-of-the-art algorithms that are applied to a unique dataset and the Pavia City Center dataset where the subsequent selected bands are fed into a classifier to gather comparison results

A new feature extraction approach based on non linear source separation

A new feature extraction approach is proposed in this paper to improve the classification performance in remotely sensed data. The proposed method is based on a primary sources subset (PSS) obtained by nonlinear transform that provides lower space for land pattern recognition. First, the underlying sources are approximated using multilayer neural networks. Given that, Bayesian inferences update unknown sources’ knowledge and model parameters with information’s data. Then, a source dimension minimizing technique is adopted to provide more efficient land cover description. The support vector machine (SVM) scheme is developed by using feature extraction. The experimental results on real multispectral imagery demonstrates that the proposed approach ensures efficient feature extraction by using several descriptors for texture identification and multiscale analysis. In a pixel based approach, the reduced PSS space improved the overall classification accuracy by 13% and reaches 82%. Using texture and multi resolution descriptors, the overall accuracy is 75.87% for the original observations, while using the reduced source space the overall accuracy reaches 81.67% when using jointly wavelet and Gabor transform and 86.67% when using Gabor transform. Thus, the source space enhanced the feature extraction process and allow more land use discrimination than the multispectral observations

Terrain classification using machine learning algorithms in a multi-temporal approach A QGIS plug-in implementation

Land cover and land use (LCLU) maps are essential for the successful administration of a nation’s topography, however, conventional on-site data gathering methods are costly and time-consuming. By contrast, remote sensing data can be used to generate up-to-date maps regularly with the help of machine learning algorithms, in turn, allowing for the assessment of a region’s dynamics throughout time. The present dissertation will focus on the implementation of an automated land use and land cover classifier based on remote sensing imagery provided by the mod ern sentinel-2 satellite constellation. The project, with Portugal at its focus, will expand on previous approaches by utilizing temporal data as an input variable in order to harvest the contextual information contained in the vegetation cycles. The pursued solution investigated the implementation of a 9-class classifier plug-in for an industry standard, open-source geographic information system. In the course of the testing procedure, various processing techniques and machine learning algorithms were evaluated in a multi-temporal approach. Resulting in a final overall accuracy of 65,9% across the targeted classes.Mapas de uso e ocupação do solo são cruciais para o entendimento e administração da topografia de uma nação, no entanto, os métodos convencionais de aquisição local de dados são caros e demorados. Contrariamente, dados provenientes de métodos de senso riamento remoto podem ser utilizados para gerar regularmente mapas atualizados com a ajuda de algoritmos de aprendizagem automática. Permitindo, por sua vez, a avaliação da dinâmica de uma região ao longo do tempo. Utilizando como base imagens de sensoriamento remoto fornecidas pela recente cons telação de satélites Sentinel-2, a presente dissertação concentra-se na implementação de um classificador de mapas de uso e ocupação do solo automatizado. O projeto, com foco em Portugal, irá procurar expandir abordagens anteriores através do aproveitamento de informação contextual contida nos ciclos vegetativos pela utilização de dados temporais adicionais. A solução adotada investigou a produção e implementação de um classificador geral de 9 classes num plug-in de um sistema de informação geográfico de código aberto. Durante o processo de teste, diversas técnicas de processamento e múltiplos algoritmos de aprendizagem automática foram avaliados numa abordagem multi-temporal, culminando num resultado final de precisão geral de 65,9% nas classes avaliadas

Mapping grass nutrient phosphorus (P) and sodium (NA) across different grass communities using Sentinel-2 data

A research report submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in partial fulfillment of the requirement for the degree of Master of Science (Environmental Sciences) at the School of Geography, Archaeology & Environmental Studies March 2017Accurate estimates and mapping of grass quality is important for effective rangeland management. The purpose of this research was to map different grass species as well as nutrient Phosphorus (P) and Sodium (Na) concentration across grass communities using Sentinel-2 imagery in Telperion game reserve. The main objectives of the study were to: map the most common grass communities at the Telperion game reserve using Sentinel-2 imagery using artificial neural network (ANN) classifier and to evaluate the use of Sentinel-2 (MSI) in quantifying grass phosphorus and sodium concentration across different grass communities. Grass phosphorus and sodium concentrations were estimated using Random Forest (RF) regression algorithm, normalized difference vegetation index (NDVI) and the simple ratios (SR) which were calculated from all two possible band combination of Sentinel-2 data. Results obtained demonstrated woody vegetation as the dominant vegetation and Aristida congesta as the most common grass species. The overall classification accuracy = 81%; kappa =0.78 and error rate=0.18 was achieved using the ANN classifier. Regression model for leaf phosphorus concentration prediction both NDVI and SR data sets yielded similar results (R2 =0.363; RMSE=0.017%) and (R2 =0.36 2; RMSE=0.0174%). Regression model for leaf sodium using NDVI and SR data sets yielded dissimilar results (R2 =0.23; RMSE=16.74 mg/kg) and (R2 =0.15; RMSE =34.08 mg/kg). The overall outcomes of this study demonstrate the capability of Sentinel 2 imagery in mapping vegetation quality (phosphorus and sodium) and quantity. The study recommends the mapping of grass communities and both phosphorus and sodium concentrations across different seasons to fully understand the distribution of different species across the game reserve as well as variations in foliar concentration of the elements. Such information will guide the reserve managers on resource use and conservation strategies to implement within the reserve. Furthermore, the information will enable conservation managers to understand wildlife distribution and feeding patterns. This will allow integration of effective conservation strategies into decisions on stocking capacity.MT 201

Deep learning for land cover and land use classification

Recent advances in sensor technologies have witnessed a vast amount of very fine spatial resolution (VFSR) remotely sensed imagery being collected on a daily basis. These VFSR images present fine spatial details that are spectrally and spatially complicated, thus posing huge challenges in automatic land cover (LC) and land use (LU) classification. Deep learning reignited the pursuit of artificial intelligence towards a general purpose machine to be able to perform any human-related tasks in an automated fashion. This is largely driven by the wave of excitement in deep machine learning to model the high-level abstractions through hierarchical feature representations without human-designed features or rules, which demonstrates great potential in identifying and characterising LC and LU patterns from VFSR imagery. In this thesis, a set of novel deep learning methods are developed for LC and LU image classification based on the deep convolutional neural networks (CNN) as an example. Several difficulties, however, are encountered when trying to apply the standard pixel-wise CNN for LC and LU classification using VFSR images, including geometric distortions, boundary uncertainties and huge computational redundancy. These technical challenges for LC classification were solved either using rule-based decision fusion or through uncertainty modelling using rough set theory. For land use, an object-based CNN method was proposed, in which each segmented object (a group of homogeneous pixels) was sampled and predicted by CNN with both within-object and between-object information. LU was, thus, classified with high accuracy and efficiency. Both LC and LU formulate a hierarchical ontology at the same geographical space, and such representations are modelled by their joint distribution, in which LC and LU are classified simultaneously through iteration. These developed deep learning techniques achieved by far the highest classification accuracy for both LC and LU, up to around 90% accuracy, about 5% higher than the existing deep learning methods, and 10% greater than traditional pixel-based and object-based approaches. This research made a significant contribution in LC and LU classification through deep learning based innovations, and has great potential utility in a wide range of geospatial applications

Remote sensing of endangered tree species in the fragmented Dukuduku Indigenous Forest of KwaZulu-Natal, South Africa.

Doctor of Philosophy in Environmental Sciences. University of KwaZulu-Natal, Pietermaritzburg, 2016.Abstract available in PDF file

A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification

The contextual-based convolutional neural network (CNN) with deep architecture and pixel-based multilayer perceptron (MLP) with shallow structure are well-recognized neural network algorithms, representing the state-of-the-art deep learning method and the classical non-parametric machine learning approach, respectively. The two algorithms, which have very different behaviours, were integrated in a concise and effective way using a rule-based decision fusion approach for the classification of very fine spatial resolution (VFSR) remotely sensed imagery. The decision fusion rules, designed primarily based on the classification confidence of the CNN, reflect the generally complementary patterns of the individual classifiers. In consequence, the proposed ensemble classifier MLP-CNN harvests the complementary results acquired from the CNN based on deep spatial feature representation and from the MLP based on spectral discrimination. Meanwhile, limitations of the CNN due to the adoption of convolutional filters such as the uncertainty in object boundary partition and loss of useful fine spatial resolution detail were compensated. The effectiveness of the ensemble MLP-CNN classifier was tested in both urban and rural areas using aerial photography together with an additional satellite sensor dataset. The MLP-CNN classifier achieved promising performance, consistently outperforming the pixel-based MLP, spectral and textural-based MLP, and the contextual-based CNN in terms of classification accuracy. This research paves the way to effectively address the complicated problem of VFSR image classification

Textile Fingerprinting for Dismount Analysis in the Visible, Near, and Shortwave Infrared Domain

The ability to accurately and quickly locate an individual, or a dismount, is useful in a variety of situations and environments. A dismount\u27s characteristics such as their gender, height, weight, build, and ethnicity could be used as discriminating factors. Hyperspectral imaging (HSI) is widely used in efforts to identify materials based on their spectral signatures. More specifically, HSI has been used for skin and clothing classification and detection. The ability to detect textiles (clothing) provides a discriminating factor that can aid in a more comprehensive detection of dismounts. This thesis demonstrates the application of several feature selection methods (i.e., support vector machines with recursive feature reduction, fast correlation based filter) in highly dimensional data collected from a spectroradiometer. The classification of the data is accomplished with the selected features and artificial neural networks. A model for uniquely identifying (fingerprinting) textiles are designed, where color and composition are determined in order to fingerprint a specific textile. An artificial neural network is created based on the knowledge of the textile\u27s color and composition, providing a uniquely identifying fingerprinting of a textile. Results show 100% accuracy for color and composition classification, and 98% accuracy for the overall textile fingerprinting process

Integration of stacked-autoencoders and convolutional neural networks for hyperspectral image classification

Orientador: Prof. Dr. Jorge Antônio Silva CentenoTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências da Terra, Programa de Pós-Graduação em Ciências Geodésicas. Defesa : Curitiba, 24/05/2021Inclui referências: p. 97-103Resumo: Deep Learning ou aprendizado profundo abriu novas possibilidades para o pré-processamento, processamento e análise de dados hiperespectrais usando várias camadas de redes neurais e pode ser usado como ferramenta de extração de atributos. Nesta pesquisa, é desenvolvido um modelo híbrido baseado em pixels que integra Stacked-Autoencoders (SAE) y Redes Neurais Convolucionais (CNN) para classificar dados hiperespectrais. O núcleo do modelo integrado (SAE-1DCNN) é um Autoencoder que é aprimorado usando camadas convolucionais nas etapas de codificação (encoding) e decodificação (decoding). Isso permite melhorar a discriminação de dados no treinamento não supervisionado e reduzir o tempo no processamento, pois permite uma descrição dos atributos baseada na assinatura hiperespectral do pixel e aproveita a eficácia da arquitetura profunda com base nas camadas convolucionais e pooling. Como filtros unidimensionais foram aplicados no modelo integrado, o tempo de processamento é consideravelmente menor do que ao usar filtros 2D-CNN. Em uma primeira etapa, o modelo SAE-1DCNN é usado para extração de atributos e, em seguida, esses resultados são usados em uma etapa final para uma classificação supervisionada. Assim, na primeira etapa os parâmetros da rede são ajustados usando amostras de treinamento e após na segunda etapa uma abordagem fine-tuning composta de regressão logística com base na função de ativação softmax foi aplicada para classificação. Três aspectos são analisados nesta pesquisa: a capacidade do modelo de excluir bandas ruidosas, sua capacidade de redução da dimensionalidade e seu potencial para realizar a classificação da cobertura da terra usando dados hiperespectrais. Os experimentos foram realizados com diferentes conjuntos de dados hiperespectrais: Indian Pines, Universidade de Pavia e Salinas, amplamente utilizados pela comunidade científica, e uma imagem hiperespectral capturada na Fazenda Canguiri da Universidade Federal do Paraná (UFPR) no Paraná-Brasil. Para validar a metodologia proposta, os resultados obtidos foram comparados aos métodos tradicionais de aprendizado de máquina para verificar o potencial da integração de autoencoders (AE) e redes convolucionais. Os resultados obtidos mostraram similaridade com os métodos tradicionais em termos de acurácia da classificação hiperespectral, porém demandaram menos tempo de processamento, portanto, a metodologia proposta (SAE-1DCNN) é considerada promissora, sólida e pode ser uma alternativa para o pré-processamento de dados hiperespectrais e processamento.Abstract: Deep learning opened new possibilities for hyperspectral data processing and analysis using multiple neural nets layers and can be used as a feature extraction tool. In this research, a pixel-based hybrid model is developed that integrates Stacked-Autoencoders (SAE) and Convolutional Neural Network (CNN) for hyperspectral image classification. The core of the integrated model (SAE-1DCNN) is an autoencoder that is improved by using convolutional layers in the encoding and decoding steps. This allows improving data discrimination in unsupervised training and reducing the processing time because it allows a feature-based description of the pixel's hyperspectral signature and takes advantage of the effectiveness of deep architecture based on the convolutional and pooling layers. As one-dimensional filters are applied, the processing time is considerably lower than when using 2D-CNN filters. In a first step, the SAE-1DCNN model is used for feature extraction and then these results are used in a final supervised classification step. Thus, in the first stage, the parameters of the net are adjusted using training samples and then, in the second stage, a fine-tuning approach followed by logistic regression based on the softmax activation function was applied for classification. Three aspects are analyzed in detail: the capacity of the model to exclude noisy bands, its ability to dimensionality reduction, and its potential to perform land cover classification based on hyperspectral data. Experiments were performed using different hyperspectral data sets: Indian Pines University of Pavia and Salinas, widely used by the scientific community, and a hyperspectral image captured at the Canguiri Farm of the Federal University of Paraná (UFPR) in Paraná-Brazil. To validate the proposed methodology, the obtained results were compared to traditional machine learning methods to verify the potential of the integration of autoencoders (AE) and convolutional nets. These obtained results showed similarity with traditional methods in terms of hyperspectral classification accuracy, however, they demanded less time for processing, therefore, the proposed methodology (SAE-1DCNN) is considered promising, solid, and can be an alternative for hyperspectral data pre-processing and processing.Resumen: Deep Learning o aprendizaje profundo abrió nuevos desafíos para el preprocesamiento, procesamiento y análisis de datos hiperespectrales usando varias capas de redes neuronales y puede ser usado como herramienta de extracción de atributos. En esta investigación, se desarrolla un modelo híbrido basado en pixeles que integra Stacked-Autoencoders (SAE) y redes Neuronales Convolucionales (CNN) para clasificar datos hiperespectrales. Este enfoque uso un modelo basado en pixeles que integra Convolutional Neural Networks (CNN) y Stacked-Autoencoders (SAE). El núcleo del modelo integrado (SAE-1DCNN) es un Autoencoder (AE) mejorado que usa capas convolucionales en las etapas de codificación y decodificación. Esto permite mejorar la discriminación de datos a través de un entrenamiento supervisado y además reducir el tiempo en el procesamiento, pues permite una descripción de los atributos basad en la respuesta hiperespectral del pixel y aprovecha la efectividad de la arquitectura profunda en las capas convolucionales (convolutional) y de agrupamiento (pooling). En este modelo integrado se aplican filtros unidimensionales lo que permite que el tiempo en el procesamiento sea menor si se compara con los filtros bidimensionales 2D-CNN. En una primera etapa, el modelo SAE-1DCNN es usado para la extracción de atributos y en seguida, esos resultados son usados para la etapa final basada en la clasificación supervisada. De esta forma, en la primera etapa los parámetros de la red son ajustados usando las muestras de entrenamiento y después en la segunda etapa el enfoque conocido como fine-tuning fue aplicado para la clasificación de cobertura terrestre basado en regresión logística y la función de activación softmax. Tres aspectos son analizados en esta investigación, la capacidad del modelo para excluir bandas ruidosas, la capacidad para seleccionar las bandas redundantes y así reducir la dimensionalidad y el potencial para realizar la clasificación de la cobertura terrestre usando datos hiperespectrales. Los experimentos fueron realizados con diferentes conjuntos de datos hiperespectrales: Indian Pines, Universidad de Pavia y Salinas, ampliamente usados en trabajos científicos, y una imagen hiperespectral capturada en la Hacienda Canguiri de la Universidad Federal de Paraná (UFPR) en Paraná-Brasil. Para validar la metodología propuesta, los resultados obtenidos se compararon con métodos tradicionales de aprendizaje de máquina (machine learning) para verificar el potencial de la integración de Autoencoders (AE) y redes convolucionales. Los resultados obtenidos mostraron similitud con los métodos tradicionales en cuanto a la precisión de clasificación hiperespectral, sin embargo, exigieron menos tiempo de procesamiento, por lo que, la metodología propuesta (SAE-1DCNN) se considera prometedora, sólida y puede ser una alternativa para el pré-procesamiento y procesamiento de datos hiperespectrales