Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing

Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these hyperspectral (HS) products mainly by means of seasoned experts. However, with the ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges on reducing the burden of manual labor and improving efficiency. For this reason, it is, therefore, urgent to develop more intelligent and automatic approaches for various HS RS applications. Machine learning (ML) tools with convex optimization have successfully undertaken the tasks of numerous artificial intelligence (AI)-related applications. However, their ability in handling complex practical problems remains limited, particularly for HS data, due to the effects of various spectral variabilities in the process of HS imaging and the complexity and redundancy of higher dimensional HS signals. Compared to the convex models, non-convex modeling, which is capable of characterizing more complex real scenes and providing the model interpretability technically and theoretically, has been proven to be a feasible solution to reduce the gap between challenging HS vision tasks and currently advanced intelligent data processing models

Towards Generalized Frameworks for Object Recognition

Over the past few years, deep convolutional neural network (DCNN) based approaches have been immensely successful in tackling a diverse range of object recognition problems. Popular DCNN architectures like deep residual networks (ResNets) are highly generic, not just for classification, but also for high level tasks like detection/tracking which rely on classification DCNNs as their backbone. The generality of DCNNs however doesn't extend to image-to-image(Im2Im) regression tasks (eg: super-resolution, denoising, rgb-to-depth, relighting, etc). For such tasks, DCNNs are often highly task-specific and require specific ancillary post-processing methods. The major issue plaguing the design of generic architectures for such tasks is the tradeoff between context/locality given a fixed computation/memory budget. We first present a generic DCNN architecture for Im2Im regression that can be trained end-to-end without any further machinery. Our proposed architecture, the Recursively Branched Deconvolutional Network (RBDN), which features a cheap early multi-context image representation, an efficient recursive branching scheme with extensive parameter sharing and learnable upsampling. We provide qualitative/quantitative results on 3 diverse tasks: relighting, denoising and colorization and show that our proposed RBDN architecture obtains comparable results to the state-of-the-art on each of these tasks when used off-the-shelf without any post processing or task-specific architectural modifications. Second, we focus on gradient flow and optimization in ResNets. In particular, we theoretically analyze why pre-activation(v2) ResNets outperform the original ResNets(v1) on CIFAR datasets but not on ImageNet. Our analysis reveals that although v1-ResNets lack ensembling properties, they can have a higher effective depth in comparison to v2-ResNes. Subsequently, we show that downsampling projections (while only few in number) have a significantly detrimental effect on performance. We show that by simply replacing downsampling-projections with identity-like dense-reshape shortcuts, the classification results of standard residual architectures like ResNets, ResNeXts and SE-Nets improve by up to 1.2% on ImageNet, without any increase in computational complexity (FLOPs). Finally, we present a robust non-parametric probabilistic ensemble method for multi-classification, which outperforms the state-of-the-art ensemble methods on several machine learning and computer vision datasets for object recognition with statistically significant improvements. The approach is particularly geared towards multi-classification problems with very low training data and/or a fairly high proportion of outliers, for which training end-to-end DCNNs is not very beneficial

The Role of Riemannian Manifolds in Computer Vision: From Coding to Deep Metric Learning

A diverse number of tasks in computer vision and machine learning enjoy from representations of data that are compact yet discriminative, informative and robust to critical measurements. Two notable representations are offered by Region Covariance Descriptors (RCovD) and linear subspaces which are naturally analyzed through the manifold of Symmetric Positive Definite (SPD) matrices and the Grassmann manifold, respectively, two widely used types of Riemannian manifolds in computer vision. As our first objective, we examine image and video-based recognition applications where the local descriptors have the aforementioned Riemannian structures, namely the SPD or linear subspace structure. Initially, we provide a solution to compute Riemannian version of the conventional Vector of Locally aggregated Descriptors (VLAD), using geodesic distance of the underlying manifold as the nearness measure. Next, by having a closer look at the resulting codes, we formulate a new concept which we name Local Difference Vectors (LDV). LDVs enable us to elegantly expand our Riemannian coding techniques to any arbitrary metric as well as provide intrinsic solutions to Riemannian sparse coding and its variants when local structured descriptors are considered. We then turn our attention to two special types of covariance descriptors namely infinite-dimensional RCovDs and rank-deficient covariance matrices for which the underlying Riemannian structure, i.e. the manifold of SPD matrices is out of reach to great extent. %Generally speaking, infinite-dimensional RCovDs offer better discriminatory power over their low-dimensional counterparts. To overcome this difficulty, we propose to approximate the infinite-dimensional RCovDs by making use of two feature mappings, namely random Fourier features and the Nystrom method. As for the rank-deficient covariance matrices, unlike most existing approaches that employ inference tools by predefined regularizers, we derive positive definite kernels that can be decomposed into the kernels on the cone of SPD matrices and kernels on the Grassmann manifolds and show their effectiveness for image set classification task. Furthermore, inspired by attractive properties of Riemannian optimization techniques, we extend the recently introduced Keep It Simple and Straightforward MEtric learning (KISSME) method to the scenarios where input data is non-linearly distributed. To this end, we make use of the infinite dimensional covariance matrices and propose techniques towards projecting on the positive cone in a Reproducing Kernel Hilbert Space (RKHS). We also address the sensitivity issue of the KISSME to the input dimensionality. The KISSME algorithm is greatly dependent on Principal Component Analysis (PCA) as a preprocessing step which can lead to difficulties, especially when the dimensionality is not meticulously set. To address this issue, based on the KISSME algorithm, we develop a Riemannian framework to jointly learn a mapping performing dimensionality reduction and a metric in the induced space. Lastly, in line with the recent trend in metric learning, we devise end-to-end learning of a generic deep network for metric learning using our derivation

Integrative Assessment and Modelling of the Non Timber Forest Products Potential in Nuba Mountains of Sudan by Field Methods, Remote Sensing and GIS

Pressure imposed at any one place or point in time results in a complexity of spatial and temporal interactions within topographical ecosystems. It can be propagated through the system and may have implications for future ecosystem functions over a wide array of various spatial and temporal scales. Under conditions of wars and other socio-economic conflicts, these processes are most forceful in developing countries amidst declining economic growth, lack of awareness, deterioration of ecosystem services, loss of existing traditional knowledge bases and weak governance structures. Forests are an essential part of ecosystem services, not only as a resource but as a contributor to biological systems as well. They represent one of the most important sectors in the context of Environmental Change (EC), both from the point of mitigation as well as adaptation. While forests are projected to be adversely impacted under EC, they are also providing opportunities to mitigate these changes. Yet this is one of the least understood sectors, especially at the regional level - many of its fundamental metrics such as mitigation potential, vulnerability and the likely impacts of EC are still not well understood until now. Thus, there is a need for research and field investigations into the synergy of mitigation and adaptation so that the cost of addressing EC impacts can be reduced and the co-benefits can be increased. The aim of this study is to focus particularly on forest-based ecosystem services and to use forests as a strategy for inducing environmental change within the Nuba Mountains in Sudan, specifically for systems in poor condition under EC, and furthermore to explore forests as an entry point for investigating the relationship between urban and rural development and ecosystem services. In addition, the aim is also to raise understanding of the relations between patterns of local-level economic and demographic changes, the nature and value of local ecosystem services, and the role of such services in increasingly interlinked urban and rural livelihood systems. The methodology applied in the current research is three-pronged: a formal literature review, a socio–economic survey (based on semi-structured interviews of household heads via Rapid Rural Appraisal (RRA), with a focus on group discussions, informal meetings, free listening and key informant techniques), and multitemporal optical satellite data analysis (i.e. Landsat and RapidEye). Landsat imagery was utilized to gather the spatial characteristics of the region and to study the Land Use/Land Cover (LU/LC) changes during the period from 1984 to 2014. Meanwhile, RapidEye imagery was used to generate the tree species distribution map. Qualitative and quantitative techniques were applied to analyze socio-economic data. Moreover, Food Consumption Score (FCS) was used to gauge both diversity and frequency of food consumption in surveyed areas. Geographic object-based image analysis (i.e. K-Nearest Neighbour classifier and knowledge-based classifiers) based on a developed model of integrated features (such as vegetation indices, DEM, thematic layers and meteorological information) was applied. Post Classification Analysis (PCA) as well as Post Change Detection (PCD) techniques were used. Hotspot analysis was conducted to detect the areas affected by deforestation. Furthermore, Ordinary Least Squares regression (OLS), Autocorrelation (Moran's) analysis, and Geographically Weighted Regression analyses (GWR) were applied to address the interaction of the different socioeconomic/ecological factors on Non Timber Forest Products (NTFPs) collection and to simulate the dependency scenarios of NTFPs along with their impact on poverty alleviation. Additionally, simulation was performed to estimate the future forest density and predict the dependency on forest services. An increasing impact of intensive interactions between the rural and urban areas has long been acknowledged. However, recent changes in the global political economy and environmental systems, as well as local dynamics of the study area driven by war, drought and deforestation, have led to an increasing rapidity and depth in rural transformation, as well as to a significant impact on urban areas. Like most environmental problems, the effects of these drivers are complex and are stressed diversely across different geographic regions by the socio-political processes that underlie recent economic and cultural globalization. These interactions and processes have increasingly brought rapid changes in land cover, social, institutional and livelihood transformation across broad areas of South Kordofan. Moreover, the study unveils new dynamics such as high rates of migration and mobility by the indigenous population and the increasing domination of market-centric livelihoods in many villages that were once dominated by rural agricultural and natural resourcesbased socio-economic systems. Furthermore, the research highlights the significant roles of NTFPs and trees in contributing to Nuba Mountains’ economic development, food security and environmental health, indicating which requirements need to be addressed in order to improve these potentials. The study proves that drawing on a wide range of these products for livelihood strengthens rural people’s ability to deal with and adapt to both EC and extreme events. Moreover, the results underline the importance of participatory approaches of rural women and their impact on NTFPs management with recommendations of more emphasis on potential roles and the ability of women to participate in public fora. Furthermore, the study shows that the use of high-resolution satellite imagery, integrated with model-based terrestrial information, provides a precise knowledge about the magnitude and distribution of LU/LC patterns. These methods can make an important contribution towards a better understanding of EC dynamics over time. The study reveals that more information exchange is needed to inform actors and decision makers regarding specific experiences, capacity gaps and knowledge to address EC. Subsequently, new policies and strategies are required to much more specifically focus on how to deal with consequences of longer-term EC rather than with the impacts of sudden natural disasters