General Adaptive Neighborhood Image Processing for Biomedical Applications

In biomedical imaging, the image processing techniques using spatially invariant transformations, with fixed operational windows, give efficient and compact computing structures, with the conventional separation between data and operations. Nevertheless, these operators have several strong drawbacks, such as removing significant details, changing some meaningful parts of large objects, and creating artificial patterns. This kind of approaches is generally not sufficiently relevant for helping the biomedical professionals to perform accurate diagnosis and therapy by using image processing techniques. Alternative approaches addressing context-dependent processing have been proposed with the introduction of spatially-adaptive operators (Bouannaya and Schonfeld, 2008; Ciuc et al., 2000; Gordon and Rangayyan, 1984;Maragos and Vachier, 2009; Roerdink, 2009; Salembier, 1992), where the adaptive concept results from the spatial adjustment of the sliding operational window. A spatially-adaptive image processing approach implies that operators will no longer be spatially invariant, but must vary over the whole image with adaptive windows, taking locally into account the image context by involving the geometrical, morphological or radiometric aspects. Nevertheless, most of the adaptive approaches require a priori or extrinsic informations on the image for efficient processing and analysis. An original approach, called General Adaptive Neighborhood Image Processing (GANIP), has been introduced and applied in the past few years by Debayle & Pinoli (2006a;b); Pinoli and Debayle (2007). This approach allows the building of multiscale and spatially adaptive image processing transforms using context-dependent intrinsic operational windows. With the help of a specified analyzing criterion (such as luminance, contrast, ...) and of the General Linear Image Processing (GLIP) (Oppenheim, 1967; Pinoli, 1997a), such transforms perform a more significant spatial and radiometric analysis. Indeed, they take intrinsically into account the local radiometric, morphological or geometrical characteristics of an image, and are consistent with the physical (transmitted or reflected light or electromagnetic radiation) and/or physiological (human visual perception) settings underlying the image formation processes. The proposed GAN-based transforms are very useful and outperforms several classical or modern techniques (Gonzalez and Woods, 2008) - such as linear spatial transforms, frequency noise filtering, anisotropic diffusion, thresholding, region-based transforms - used for image filtering and segmentation (Debayle and Pinoli, 2006b; 2009a; Pinoli and Debayle, 2007). This book chapter aims to first expose the fundamentals of the GANIP approach (Section 2) by introducing the GLIP frameworks, the General Adaptive Neighborhood (GAN) sets and two kinds of GAN-based image transforms: the GAN morphological filters and the GAN Choquet filters. Thereafter in Section 3, several GANIP processes are illustrated in the fields of image restoration, image enhancement and image segmentation on practical biomedical application examples. Finally, Section 4 gives some conclusions and prospects of the proposed GANIP approach

Efficient Data Driven Multi Source Fusion

Data/information fusion is an integral component of many existing and emerging applications; e.g., remote sensing, smart cars, Internet of Things (IoT), and Big Data, to name a few. While fusion aims to achieve better results than what any one individual input can provide, often the challenge is to determine the underlying mathematics for aggregation suitable for an application. In this dissertation, I focus on the following three aspects of aggregation: (i) efficient data-driven learning and optimization, (ii) extensions and new aggregation methods, and (iii) feature and decision level fusion for machine learning with applications to signal and image processing. The Choquet integral (ChI), a powerful nonlinear aggregation operator, is a parametric way (with respect to the fuzzy measure (FM)) to generate a wealth of aggregation operators. The FM has 2N variables and N(2N − 1) constraints for N inputs. As a result, learning the ChI parameters from data quickly becomes impractical for most applications. Herein, I propose a scalable learning procedure (which is linear with respect to training sample size) for the ChI that identifies and optimizes only data-supported variables. As such, the computational complexity of the learning algorithm is proportional to the complexity of the solver used. This method also includes an imputation framework to obtain scalar values for data-unsupported (aka missing) variables and a compression algorithm (lossy or losselss) of the learned variables. I also propose a genetic algorithm (GA) to optimize the ChI for non-convex, multi-modal, and/or analytical objective functions. This algorithm introduces two operators that automatically preserve the constraints; therefore there is no need to explicitly enforce the constraints as is required by traditional GA algorithms. In addition, this algorithm provides an efficient representation of the search space with the minimal set of vertices. Furthermore, I study different strategies for extending the fuzzy integral for missing data and I propose a GOAL programming framework to aggregate inputs from heterogeneous sources for the ChI learning. Last, my work in remote sensing involves visual clustering based band group selection and Lp-norm multiple kernel learning based feature level fusion in hyperspectral image processing to enhance pixel level classification

Multiple Instance Choquet Integral for multiresolution sensor fusion

Imagine you are traveling to Columbia, MO for the first time. On your flight to Columbia, the woman sitting next to you recommended a bakery by a large park with a big yellow umbrella outside. After you land, you need directions to the hotel from the airport. Suppose you are driving a rental car, you will need to park your car at a parking lot or a parking structure. After a good night's sleep in the hotel, you may decide to go for a run in the morning on the closest trail and stop by that recommended bakery under a big yellow umbrella. It would be helpful in the course of completing all these tasks to accurately distinguish the proper car route and walking trail, find a parking lot, and pinpoint the yellow umbrella. Satellite imagery and other geo-tagged data such as Open Street Maps provide effective information for this goal. Open Street Maps can provide road information and suggest bakery within a five-mile radius. The yellow umbrella is a distinctive color and, perhaps, is made of a distinctive material that can be identified from a hyperspectral camera. Open Street Maps polygons are tagged with information such as "parking lot" and "sidewalk." All these information can and should be fused to help identify and offer better guidance on the tasks you are completing. Supervised learning methods generally require precise labels for each training data point. It is hard (and probably at an extra cost) to manually go through and label each pixel in the training imagery. GPS coordinates cannot always be fully trusted as a GPS device may only be accurate to the level of several pixels. In many cases, it is practically infeasible to obtain accurate pixel-level training labels to perform fusion for all the imagery and maps available. Besides, the training data may come in a variety of data types, such as imagery or as a 3D point cloud. The imagery may have different resolutions, scales and, even, coordinate systems. Previous fusion methods are generally only limited to data mapped to the same pixel grid, with accurate labels. Furthermore, most fusion methods are restricted to only two sources, even if certain methods, such as pan-sharpening, can deal with different geo-spatial types or data of different resolution. It is, therefore, necessary and important, to come up with a way to perform fusion on multiple sources of imagery and map data, possibly with different resolutions and of different geo-spatial types with consideration of uncertain labels. I propose a Multiple Instance Choquet Integral framework for multi-resolution multisensor fusion with uncertain training labels. The Multiple Instance Choquet Integral (MICI) framework addresses uncertain training labels and performs both classification and regression. Three classifier fusion models, i.e. the noisy-or, min-max, and generalized-mean models, are derived under MICI. The Multi-Resolution Multiple Instance Choquet Integral (MR-MICI) framework is built upon the MICI framework and further addresses multiresolution in the fusion sources in addition to the uncertainty in training labels. For both MICI and MR-MICI, a monotonic normalized fuzzy measure is learned to be used with the Choquet integral to perform two-class classifier fusion given bag-level training labels. An optimization scheme based on the evolutionary algorithm is used to optimize the models proposed. For regression problems where the desired prediction is real-valued, the primary instance assumption is adopted. The algorithms are applied to target detection, regression and scene understanding applications. Experiments are conducted on the fusion of remote sensing data (hyperspectral and LiDAR) over the campus of University of Southern Mississippi - Gulfpark. Clothpanel sub-pixel and super-pixel targets were placed on campus with varying levels of occlusion and the proposed algorithms can successfully detect the targets in the scene. A semi-supervised approach is developed to automatically generate training labels based on data from Google Maps, Google Earth and Open Street Map. Based on such training labels with uncertainty, the proposed algorithms can also identify materials on campus for scene understanding, such as road, buildings, sidewalks, etc. In addition, the algorithms are used for weed detection and real-valued crop yield prediction experiments based on remote sensing data that can provide information for agricultural applications.Includes biblographical reference

Hierarchical information fusion for decision making in craniofacial superimposition

Craniofacial superimposition is one of the most important skeleton-based identification methods. The process studies the possible correspondence between a found skull and a candidate (missing person) through the superimposition of the former over a variable number of images of the face of the latter. Within craniofacial superimposition we identified three different stages, namely: (1) image acquisition-processing and landmark location; (2) skull-face overlay; and (3) decision making. While we have already proposed and validated an automatic skull-face overlay technique in previous works, the final identification stage, decision making, is still performed manually by the expert. This consists of the determination of the degree of support for the assertion that the skull and the ante-mortem image belong to the same person. This decision is made through the analysis of several criteria assessing the skull-face anatomical correspondence based on the resulting skull-face overlay. In this contribution, we present a hierarchical framework for information fusion to support the anthropologist expert in the decision making stage. The main goal is the automation of this stage based on the use of several skull-face anatomical criteria combined at different levels by means of fuzzy aggregation functions. We have implemented two different experiments for our framework. The first aims to obtain the most suitable aggregation functions for the system and the second validates the proposed framework as an identification system. We tested the framework with a dataset of 33 positive and 411 negative identification instances. The present proposal is the first automatic craniofacial superimposition decision support system evaluated in an objective and statistically meaningful way. © 2017 Elsevier B.V

Densification of spatially-sparse legacy soil data at a national scale: a digital mapping approach

Digital soil mapping (DSM) is a viable approach to providing spatial soil information but its adoption at the national scale, especially in sub-Saharan Africa, is limited by low spread of data. Therefore, the focus of this thesis is on optimizing DSM techniques for densification of sparse legacy soil data using Nigeria as a case study. First, the robustness of Random Forest model (RFM) was tested in predicting soil particle-size fractions as a compositional data using additive log-ratio technique. Results indicated good prediction accuracy with RFM while soils are largely coarse-textured especially in the northern region. Second, soil organic carbon (SOC) and bulk density (BD) were predicted from which SOC density and stock were calculated. These were overlaid with land use/land cover (LULC), agro-ecological zone (AEZ) and soil maps to quantify the carbon sequestration of soils and their variation across different AEZs. Results showed that 6.5 Pg C with an average of 71.60 Mg C ha–1 abound in the top 1 m soil depth. Furthermore, to improve the performance of BD and effective cation exchange capacity (ECEC) pedotransfer functions (PTFs), the inclusion of environmental data was explored using multiple linear regression (MLR) and RFM. Results showed an increase in performance of PTFs with the use of soil and environmental data. Finally, the application of Choquet fuzzy integral (CI) technique in irrigation suitability assessment was assessed. This was achieved through multi-criteria analysis of soil, climatic, landscape and socio-economic indices. Results showed that CI is a better aggregation operator compared to weighted mean technique. A total of 3.34 x 106 ha is suitable for surface irrigation in Nigeria while major limitations are due to topographic and soil attributes. Research findings will provide quantitative basis for framing appropriate policies on sustainable food production and environmental management, especially in resource-poor countries of the world

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin