State-of-the-art and gaps for deep learning on limited training data in remote sensing

Deep learning usually requires big data, with respect to both volume and variety. However, most remote sensing applications only have limited training data, of which a small subset is labeled. Herein, we review three state-of-the-art approaches in deep learning to combat this challenge. The first topic is transfer learning, in which some aspects of one domain, e.g., features, are transferred to another domain. The next is unsupervised learning, e.g., autoencoders, which operate on unlabeled data. The last is generative adversarial networks, which can generate realistic looking data that can fool the likes of both a deep learning network and human. The aim of this article is to raise awareness of this dilemma, to direct the reader to existing work and to highlight current gaps that need solving.Comment: arXiv admin note: text overlap with arXiv:1709.0030

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.Comment: IEEE Transactions on Fuzzy System

Introducing Fuzzy Layers for Deep Learning

Many state-of-the-art technologies developed in recent years have been influenced by machine learning to some extent. Most popular at the time of this writing are artificial intelligence methodologies that fall under the umbrella of deep learning. Deep learning has been shown across many applications to be extremely powerful and capable of handling problems that possess great complexity and difficulty. In this work, we introduce a new layer to deep learning: the fuzzy layer. Traditionally, the network architecture of neural networks is composed of an input layer, some combination of hidden layers, and an output layer. We propose the introduction of fuzzy layers into the deep learning architecture to exploit the powerful aggregation properties expressed through fuzzy methodologies, such as the Choquet and Sugueno fuzzy integrals. To date, fuzzy approaches taken to deep learning have been through the application of various fusion strategies at the decision level to aggregate outputs from state-of-the-art pre-trained models, e.g., AlexNet, VGG16, GoogLeNet, Inception-v3, ResNet-18, etc. While these strategies have been shown to improve accuracy performance for image classification tasks, none have explored the use of fuzzified intermediate, or hidden, layers. Herein, we present a new deep learning strategy that incorporates fuzzy strategies into the deep learning architecture focused on the application of semantic segmentation using per-pixel classification. Experiments are conducted on a benchmark data set as well as a data set collected via an unmanned aerial system at a U.S. Army test site for the task of automatic road segmentation, and preliminary results are promising.Comment: 6 pages, 4 figures, published in 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE

A Transformer-Based Network for Dynamic Hand Gesture Recognition

Transformer-based neural networks represent a successful self-attention mechanism that achieves state-of-the-art results in language understanding and sequence modeling. However, their application to visual data and, in particular, to the dynamic hand gesture recognition task has not yet been deeply investigated. In this paper, we propose a transformer-based architecture for the dynamic hand gesture recognition task. We show that the employment of a single active depth sensor, specifically the usage of depth maps and the surface normals estimated from them, achieves state-of-the-art results, overcoming all the methods available in the literature on two automotive datasets, namely NVidia Dynamic Hand Gesture and Briareo. Moreover, we test the method with other data types available with common RGB-D devices, such as infrared and color data. We also assess the performance in terms of inference time and number of parameters, showing that the proposed framework is suitable for an online in-car infotainment system

Deep learning-based insights on T:R ratio behaviour during prolonged screening for S-ICD eligibility.

Peer reviewedPublisher PD

A hybrid neural network for continuous and non-invasive estimation of blood pressure from raw electrocardiogram and photoplethysmogram waveforms

Background and objectives: Continuous and non-invasive blood pressure monitoring would revolutionize healthcare. Currently, blood pressure (BP) can only be accurately monitored using obtrusive cuff-based devices or invasive intra-arterial monitoring. In this work, we propose a novel hybrid neural network for the accurate estimation of blood pressure (BP) using only non-invasive electrocardiogram (ECG) and photoplethysmogram (PPG) waveforms as inputs. Methods: This work proposes a hybrid neural network combines the feature detection abilities of temporal convolutional layers with the strong performance on sequential data offered by long short-term memory layers. Raw electrocardiogram and photoplethysmogram waveforms are concatenated and used as network inputs. The network was developed using the TensorFlow framework. Our scheme is analysed and compared to the literature in terms of well known standards set by the British Hypertension Society (BHS) and the Association for the Advancement of Medical Instrumentation (AAMI). Results: Our scheme achieves extremely low mean absolute errors (MAEs) of 4.41 mmHg for SBP, 2.91 mmHg for DBP, and 2.77 mmHg for MAP. A strong level of agreement between our scheme and the gold-standard intra-arterial monitoring is shown through Bland Altman and regression plots. Additionally, the standard for BP devices established by AAMI is met by our scheme. We also achieve a grade of 'A' based on the criteria outlined by the BHS protocol for BP devices. Conclusions: Our CNN-LSTM network outperforms current state-of-the-art schemes for non-invasive BP measurement from PPG and ECG waveforms. These results provide an effective machine learning approach that could readily be implemented into non-invasive wearable devices for use in continuous clinical and at-home monitoring