DALF: An AI Enabled Adversarial Framework for Classification of Hyperspectral Images

Hyperspectral image classification is very complex and challenging process. However, with deep neural networks like Convolutional Neural Networks (CNN) with explicit dimensionality reduction, the capability of classifier is greatly increased. However, there is still problem with sufficient training samples. In this paper, we overcome this problem by proposing an Artificial Intelligence (AI) based framework named Deep Adversarial Learning Framework (DALF) that exploits deep autoencoder for dimensionality reduction, Generative Adversarial Network (GAN) for generating new Hyperspectral Imaging (HSI) samples that are to be verified by a discriminator in a non-cooperative game setting besides using aclassifier. Convolutional Neural Network (CNN) is used for both generator and discriminator while classifier role is played by Support Vector Machine (SVM) and Neural Network (NN). An algorithm named Generative Model based Hybrid Approach for HSI Classification (GMHA-HSIC) which drives the functionality of the proposed framework is proposed. The success of DALF in accurate classification is largely dependent on the synthesis and labelling of spectra on regular basis. The synthetic samples made with an iterative process and being verified by discriminator result in useful spectra. By training GAN with associated deep learning models, the framework leverages classification performance. Our experimental results revealed that the proposed framework has potential to improve the state of the art besides having an effective data augmentation strategy

State-of-the-art and gaps for deep learning on limited training data in remote sensing

Deep learning usually requires big data, with respect to both volume and variety. However, most remote sensing applications only have limited training data, of which a small subset is labeled. Herein, we review three state-of-the-art approaches in deep learning to combat this challenge. The first topic is transfer learning, in which some aspects of one domain, e.g., features, are transferred to another domain. The next is unsupervised learning, e.g., autoencoders, which operate on unlabeled data. The last is generative adversarial networks, which can generate realistic looking data that can fool the likes of both a deep learning network and human. The aim of this article is to raise awareness of this dilemma, to direct the reader to existing work and to highlight current gaps that need solving.Comment: arXiv admin note: text overlap with arXiv:1709.0030

Formation of an informative index for recognizing specified objects in hyperspectral data

Работа посвящена разработке подхода, позволяющему по малому числу наблюдений создавать правила различения заданных объектов гиперспектральных данных. Разработка подобного подхода способствовала бы развитию методов и алгоритмов для оперативного анализа гиперспектральных данных, применимых как для предварительной обработки, так и для выполнения разметки гиперспектральных данных. Для реализации подхода предлагается применять технологию, заключающуюся в совместном использовании общих правил вычисления индексов и критериев информативности. В рамках данной работы при реализации предлагаемой технологии индекс задается нормализованной разностной формулой, а информативность оценивается на основе значения критерия разделимости дискриминантного анализа. В результате проведённых исследований, было показано, что с использованием алгоритма, реализующего технологию, была решена задача различения областей гиперспектральных данных с разной растительностью. Сформированный алгоритмом индекс оказался близким по значениям к NDVI. Применяемая технология является генерализацией подхода к формированию правил анализа гиперспектральных данных по малому числу признаков и может быть использована для формирования индексов, информативных в различных задачах.Результаты исследования были получены при поддержке государственного задания Минобрнауки России Самарскому университету в рамках работ НИЛ-602 "Фотоника для умного дома и умного города" тема 19в-Р001-602 43/21Б (экспериментальная часть), в рамках проекта № 0777-2020-0017 (программная реализация и разработка технологии), при частичной финансовой поддержке РФФИ в рамках научного проекта № 20-51-05008 (теоретическая часть)

A Social Network Image Classification Algorithm Based on Multimodal Deep Learning

The complex data structure and massive image data of social networks pose a huge challenge to the mining of associations between social information. For accurate classification of social network images, this paper proposes a social network image classification algorithm based on multimodal deep learning. Firstly, a social network association clustering model (SNACM) was established, and used to calculate trust and similarity, which represent the degree of similarity between users. Based on artificial ant colony algorithm, the SNACM was subject to weighted stacking, and the social network image association network was constructed. After that, the social network images of three modes, i.e. RGB (red-green-blue) image, grayscale image, and depth image, were fused. Finally, a three-dimensional neural network (3D NN) was constructed to extract the features of the multimodal social network image. The proposed algorithm was proved valid and accurate through experiments. The research results provide a reference for applying multimodal deep learning to classify the images in other fields

Generative adversarial deep learning in images using Nash equilibrium game theory

A generative adversarial learning (GAL) algorithm is presented to overcome the manipulations that take place in adversarial data and to result in a secured convolutional neural network (CNN). The main objective of the generative algorithm is to make some changes to initial data with positive and negative class labels in testing, hence the CNN results in misclassified data. An adversarial algorithm is used to manipulate the input data that represents the boundaries of learner’s decision-making process. The algorithm generates adversarial modifications to the test dataset using a multiplayer stochastic game approach, without learning how to manipulate the data during training. Then the manipulated data is passed through a CNN for evaluation. The multi-player game consists of an interaction between adversaries which generates manipulations and retrains the model by the learner. The Nash equilibrium game theory (NEGT) is applied to Canadian Institute for Advance Research (CIFAR) dataset. This was done to produce a secure CNN output that is more robust to adversarial data manipulations. The experimental results show that proposed NEGT-GAL achieved a grater mean value of 7.92 and takes less wall clock time of 25,243 sec. Therefore, the proposed NEGT-GAL outperforms the compared existing methods and achieves greater performance

Hybrid deep neural network for Bangla automated image descriptor

Automated image to text generation is a computationally challenging computer vision task which requires sufficient comprehension of both syntactic and semantic meaning of an image to generate a meaningful description. Until recent times, it has been studied to a limited scope due to the lack of visual-descriptor dataset and functional models to capture intrinsic complexities involving features of an image. In this study, a novel dataset was constructed by generating Bangla textual descriptor from visual input, called Bangla Natural Language Image to Text (BNLIT), incorporating 100 classes with annotation. A deep neural network-based image captioning model was proposed to generate image description. The model employs Convolutional Neural Network (CNN) to classify the whole dataset, while Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) capture the sequential semantic representation of text-based sentences and generate pertinent description based on the modular complexities of an image. When tested on the new dataset, the model accomplishes significant enhancement of centrality execution for image semantic recovery assignment. For the experiment of that task, we implemented a hybrid image captioning model, which achieved a remarkable result for a new self-made dataset, and that task was new for the Bangladesh perspective. In brief, the model provided benchmark precision in the characteristic Bangla syntax reconstruction and comprehensive numerical analysis of the model execution results on the dataset

Virtual Hyperspectral Images Using Symmetric Autoencoders

Spectral data acquired through remote sensing are invaluable for environmental and resource studies. However, these datasets are often marred by nuisance phenomena such as atmospheric interference and other complexities, which pose significant challenges for accurate analysis. We show that an autoencoder architecture, called symmetric autoencoder (SymAE), which leverages symmetry under reordering of the pixels, can learn to disentangle the influence of these nuisance from surface reflectance features on a pixel-by-pixel basis. The disentanglement provides an alternative to atmospheric correction, without relying on radiative transfer modelling, through a purely data-driven process. More importantly, SymAE can generate virtual hyperspectral images by manipulating the nuisance effects of each pixel. We demonstrate using AVIRIS instrument data that these virtual images are valuable for subsequent image analysis tasks. We also show SymAE's ability to extract intra-class invariant features, which is very useful in clustering and classification tasks, delivering state-of-the-art classification performance for a purely spectral method