Urban air pollution modelling with machine learning using fixed and mobile sensors

Detailed air quality (AQ) information is crucial for sustainable urban management, and many regions in the world have built static AQ monitoring networks to provide AQ information. However, they can only monitor the region-level AQ conditions or sparse point-based air pollutant measurements, but cannot capture the urban dynamics with high-resolution spatio-temporal variations over the region. Without pollution details, citizens will not be able to make fully informed decisions when choosing their everyday outdoor routes or activities, and policy-makers can only make macroscopic regulating decisions on controlling pollution triggering factors and emission sources. An increasing research effort has been paid on mobile and ubiquitous sampling campaigns as they are deemed the more economically and operationally feasible methods to collect urban AQ data with high spatio-temporal resolution. The current research proposes a Machine Learning based AQ Inference (Deep AQ) framework from data-driven perspective, consisting of data pre-processing, feature extraction and transformation, and pixelwise (grid-level) AQ inference. The Deep AQ framework is adaptable to integrate AQ measurements from the fixed monitoring sites (temporally dense but spatially sparse), and mobile low-cost sensors (temporally sparse but spatially dense). While instantaneous pollutant concentration varies in the micro-environment, this research samples representative values in each grid-cell-unit and achieves AQ inference at 1 km \times 1 km pixelwise scale. This research explores the predictive power of the Deep AQ framework based on samples from only 40 fixed monitoring sites in Chengdu, China (4,900 {\mathrm{km}}^\mathrm{2}, 26 April - 12 June 2019) and collaborative sampling from 28 fixed monitoring sites and 15 low-cost sensors equipped with taxis deployed in Beijing, China (3,025 {\mathrm{km}}^\mathrm{2}, 19 June - 16 July 2018). The proposed Deep AQ framework is capable of producing high-resolution (1 km \times 1 km, hourly) pixelwise AQ inference based on multi-source AQ samples (fixed or mobile) and urban features (land use, population, traffic, and meteorological information, etc.). This research has achieved high-resolution (1 km \times 1 km, hourly) AQ inference (Chengdu: less than 1% spatio-temporal coverage; Beijing: less than 5% spatio-temporal coverage) with reasonable and satisfactory accuracy by the proposed methods in urban cases (Chengdu: SMAPE \mathrm{<} 20%; Beijing: SMAPE \mathrm{<} 15%). Detailed outcomes and main conclusions are provided in this thesis on the aspects of fixed and mobile sensing, spatio-temporal coverage and density, and the relative importance of urban features. Outcomes from this research facilitate to provide a scientific and detailed health impact assessment framework for exposure analysis and inform policy-makers with data driven evidence for sustainable urban management.Open Acces

A Tutorial on Speckle Reduction in Synthetic Aperture Radar Images

Speckle is a granular disturbance, usually modeled as a multiplicative noise, that affects synthetic aperture radar (SAR) images, as well as all coherent images. Over the last three decades, several methods have been proposed for the reduction of speckle, or despeckling, in SAR images. Goal of this paper is making a comprehensive review of despeckling methods since their birth, over thirty years ago, highlighting trends and changing approaches over years. The concept of fully developed speckle is explained. Drawbacks of homomorphic filtering are pointed out. Assets of multiresolution despeckling, as opposite to spatial-domain despeckling, are highlighted. Also advantages of undecimated, or stationary, wavelet transforms over decimated ones are discussed. Bayesian estimators and probability density function (pdf) models in both spatial and multiresolution domains are reviewed. Scale-space varying pdf models, as opposite to scale varying models, are promoted. Promising methods following non-Bayesian approaches, like nonlocal (NL) filtering and total variation (TV) regularization, are reviewed and compared to spatial- and wavelet-domain Bayesian filters. Both established and new trends for assessment of despeckling are presented. A few experiments on simulated data and real COSMO-SkyMed SAR images highlight, on one side the costperformance tradeoff of the different methods, on the other side the effectiveness of solutions purposely designed for SAR heterogeneity and not fully developed speckle. Eventually, upcoming methods based on new concepts of signal processing, like compressive sensing, are foreseen as a new generation of despeckling, after spatial-domain and multiresolution-domain method

Code-Aligned Autoencoders for Unsupervised Change Detection in Multimodal Remote Sensing Images

Image translation with convolutional autoencoders has recently been used as an approach to multimodal change detection (CD) in bitemporal satellite images. A main challenge is the alignment of the code spaces by reducing the contribution of change pixels to the learning of the translation function. Many existing approaches train the networks by exploiting supervised information of the change areas, which, however, is not always available. We propose to extract relational pixel information captured by domain-specific affinity matrices at the input and use this to enforce alignment of the code spaces and reduce the impact of change pixels on the learning objective. A change prior is derived in an unsupervised fashion from pixel pair affinities that are comparable across domains. To achieve code space alignment, we enforce pixels with similar affinity relations in the input domains to be correlated also in code space. We demonstrate the utility of this procedure in combination with cycle consistency. The proposed approach is compared with the state-of-the-art machine learning and deep learning algorithms. Experiments conducted on four real and representative datasets show the effectiveness of our methodology

Human-Centered Content-Based Image Retrieval

Retrieval of images that lack a (suitable) annotations cannot be achieved through (traditional) Information Retrieval (IR) techniques. Access through such collections can be achieved through the application of computer vision techniques on the IR problem, which is baptized Content-Based Image Retrieval (CBIR). In contrast with most purely technological approaches, the thesis Human-Centered Content-Based Image Retrieval approaches the problem from a human/user centered perspective. Psychophysical experiments were conducted in which people were asked to categorize colors. The data gathered from these experiments was fed to a Fast Exact Euclidean Distance (FEED) transform (Schouten & Van den Broek, 2004), which enabled the segmentation of color space based on human perception (Van den Broek et al., 2008). This unique color space segementation was exploited for texture analysis and image segmentation, and subsequently for full-featured CBIR. In addition, a unique CBIR-benchmark was developed (Van den Broek et al., 2004, 2005). This benchmark was used to explore what and how several parameters (e.g., color and distance measures) of the CBIR process influence retrieval results. In contrast with other research, users judgements were assigned as metric. The online IR and CBIR system Multimedia for Art Retrieval (M4ART) (URL: http://www.m4art.org) has been (partly) founded on the techniques discussed in this thesis. References: - Broek, E.L. van den, Kisters, P.M.F., and Vuurpijl, L.G. (2004). The utilization of human color categorization for content-based image retrieval. Proceedings of SPIE (Human Vision and Electronic Imaging), 5292, 351-362. [see also Chapter 7] - Broek, E.L. van den, Kisters, P.M.F., and Vuurpijl, L.G. (2005). Content-Based Image Retrieval Benchmarking: Utilizing Color Categories and Color Distributions. Journal of Imaging Science and Technology, 49(3), 293-301. [see also Chapter 8] - Broek, E.L. van den, Schouten, Th.E., and Kisters, P.M.F. (2008). Modeling Human Color Categorization. Pattern Recognition Letters, 29(8), 1136-1144. [see also Chapter 5] - Schouten, Th.E. and Broek, E.L. van den (2004). Fast Exact Euclidean Distance (FEED) transformation. In J. Kittler, M. Petrou, and M. Nixon (Eds.), Proceedings of the 17th IEEE International Conference on Pattern Recognition (ICPR 2004), Vol 3, p. 594-597. August 23-26, Cambridge - United Kingdom. [see also Appendix C

Coupled modelling of land surface microwave interactions using ENVISAT ASAR data

In the last decades microwave remote sensing has proven its capability to provide valuable information about the land surface. New sensor generations as e.g. ENVISAT ASAR are capable to provide frequent imagery with an high information content. To make use of these multiple imaging capabilities, sophisticated parameter inversion and assimilation strategies have to be applied. A profound understanding of the microwave interactions at the land surface is therefore essential. The objective of the presented work is the analysis and quantitative description of the backscattering processes of vegetated areas by means of microwave backscattering models. The effect of changing imaging geometries is investigated and models for the description of bare soil and vegetation backscattering are developed. Spatially distributed model parameterisation is realized by synergistic coupling of the microwave scattering models with a physically based land surface process model. This enables the simulation of realistic SAR images, based on bioand geophysical parameters. The adequate preprocessing of the datasets is crucial for quantitative image analysis. A stringent preprocessing and sophisticated terrain geocoding and correction procedure is therefore suggested. It corrects the geometric and radiometric distortions of the image products and is taken as the basis for further analysis steps. A problem in recently available microwave backscattering models is the inadequate parameterisation of the surface roughness. It is shown, that the use of classical roughness descriptors, as the rms height and autocorrelation length, will lead to ambiguous model parameterisations. A new two parameter bare soil backscattering model is therefore recommended to overcome this drawback. It is derived from theoretical electromagnetic model simulations. The new bare soil surface scattering model allows for the accurate description of the bare soil backscattering coefficients. A new surface roughness parameter is introduced in this context, capable to describe the surface roughness components, affecting the backscattering coefficient. It is shown, that this parameter can be directly related to the intrinsic fractal properties of the surface. Spatially distributed information about the surface roughness is needed to derive land surface parameters from SAR imagery. An algorithm for the derivation of the new surface roughness parameter is therefore suggested. It is shown, that it can be derived directly from multitemporal SAR imagery. Starting from that point, the bare soil backscattering model is used to assess the vegetation influence on the signal. By comparison of the residuals between measured backscattering coefficients and those predicted by the bare soil backscattering model, the vegetation influence on the signal can be quantified. Significant difference between cereals (wheat and triticale) and maize is observed in this context. It is shown, that the vegetation influence on the signal can be directly derived from alternating polarisation data for cereal fields. It is dependant on plant biophysical variables as vegetation biomass and water content. The backscattering behaviour of a maize stand is significantly different from that of other cereals, due to its completely different density and shape of the plants. A dihedral corner reflection between the soil and the stalk is identified as the major source of backscattering from the vegetation. A semiempirical maize backscattering model is suggested to quantify the influences of the canopy over the vegetation period. Thus, the different scattering contributions of the soil and vegetation components are successfully separated. The combination of the bare soil and vegetation backscattering models allows for the accurate prediction of the backscattering coefficient for a wide range of surface conditions and variable incidence angles. To enable the spatially distributed simulation of the SAR backscattering coefficient, an interface to a process oriented land surface model is established, which provides the necessary input variables for the backscattering model. Using this synergistic, coupled modelling approach, a realistic simulation of SAR images becomes possible based on land surface model output variables. It is shown, that this coupled modelling approach leads to promising and accurate estimates of the backscattering coefficients. The remaining residuals between simulated and measured backscatter values are analysed to identify the sources of uncertainty in the model. A detailed field based analysis of the simulation results revealed that imprecise soil moisture predictions by the land surface model are a major source of uncertainty, which can be related to imprecise soil texture distribution and soil hydrological properties. The sensitivity of the backscattering coefficient to the soil moisture content of the upper soil layer can be used to generate soil moisture maps from SAR imagery. An algorithm for the inversion of soil moisture from the upper soil layer is suggested and validated. It makes use of initial soil moisture values, provided by the land surface process model. Soil moisture values are inverted by means of the coupled land surface backscattering model. The retrieved soil moisture results have an RMSE of 3.5 Vol %, which is comparable to the measurement accuracy of the reference field data. The developed models allow for the accurate prediction of the SAR backscattering coefficient. The various soil and vegetation scattering contributions can be separated. The direct interface to a physically based land surface process model allows for the spatially distributed modelling of the backscattering coefficient and the direct assimilation of remote sensing data into a land surface process model. The developed models allow for the derivation of static and dynamic landsurface parameters, as e.g. surface roughness, soil texture, soil moisture and biomass from remote sensing data and their assimilation in process models. They are therefore reliable tools, which can be used for sophisticated practice oriented problem solutions in manifold manner in the earth and environmental sciences

Technological innovations in magnetic resonance for early detection of cardiovascular diseases

Most recent technical innovations in cardiovascular MR imaging (CMRI) are presented in this review. They include hardware and software developments, and novelties in parametric mapping. All these recent improvements lead to high spatial and temporal resolution and quantitative information on the heart structure and function. They make it achievable ambitious goals in the field of mapletic resonance, such as the early detection of cardiovascular pathologies. In this review article, we present recent innovations in CMRI, emphasizing the progresses performed and the solutions proposed to some yet opened technical problems

Supervised learning and inference of semantic information from road scene images

Premio Extraordinario de Doctorado de la UAH en el año académico 2013-2014Nowadays, vision sensors are employed in automotive industry to integrate advanced functionalities that assist humans while driving. However, autonomous vehicles is a hot field of research both in academic and industrial sectors and entails a step beyond ADAS. Particularly, several challenges arise from autonomous navigation in urban scenarios due to their naturalistic complexity in terms of structure and dynamic participants (e.g. pedestrians, vehicles, vegetation, etc.). Hence, providing image understanding capabilities to autonomous robotics platforms is an essential target because cameras can capture the 3D scene as perceived by a human. In fact, given this need for 3D scene understanding, there is an increasing interest on joint objects and scene labeling in the form of geometry and semantic inference of the relevant entities contained in urban environments. In this regard, this Thesis tackles two challenges: 1) the prediction of road intersections geometry and, 2) the detection and orientation estimation of cars, pedestrians and cyclists. Different features extracted from stereo images of the KITTI public urban dataset are employed. This Thesis proposes a supervised learning of discriminative models that rely on strong machine learning techniques for data mining visual features. For the first task, we use 2D occupancy grid maps that are built from the stereo sequences captured by a moving vehicle in a mid-sized city. Based on these bird?s eye view images, we propose a smart parameterization of the layout of straight roads and 4 intersecting roads. The dependencies between the proposed discrete random variables that define the layouts are represented with Probabilistic Graphical Models. Then, the problem is formulated as a structured prediction, in which we employ Conditional Random Fields (CRF) for learning and convex Belief Propagation (dcBP) and Branch and Bound (BB) for inference. For the validation of the proposed methodology, a set of tests are carried out, which are based on real images and synthetic images with varying levels of random noise. In relation to the object detection and orientation estimation challenge in road scenes, this Thesis goal is to compete in the international challenge known as KITTI evaluation benchmark, which encourages researchers to push forward the current state of the art on visual recognition methods, particularized for 3D urban scene understanding. This Thesis proposes to modify the successful part-based object detector known as DPM in order to learn richer models from 2.5D data (color and disparity). Therefore, we revisit the DPM framework, which is based on HOG features and mixture models trained with a latent SVM formulation. Next, this Thesis performs a set of modifications on top of DPM: I) An extension to the DPM training pipeline that accounts for 3D-aware features. II) A detailed analysis of the supervised parameter learning. III) Two additional approaches: "feature whitening" and "stereo consistency check". Additionally, a) we analyze the KITTI dataset and several subtleties regarding to the evaluation protocol; b) a large set of cross-validated experiments show the performance of our contributions and, c) finally, our best performing approach is publicly ranked on the KITTI website, being the first one that reports results with stereo data, yielding an increased object detection precision (3%-6%) for the class 'car' and ranking first for the class cyclist

Supervised learning and inference of semantic information from road scene images

Premio Extraordinario de Doctorado de la UAH en el año académico 2013-2014Nowadays, vision sensors are employed in automotive industry to integrate advanced functionalities that assist humans while driving. However, autonomous vehicles is a hot field of research both in academic and industrial sectors and entails a step beyond ADAS. Particularly, several challenges arise from autonomous navigation in urban scenarios due to their naturalistic complexity in terms of structure and dynamic participants (e.g. pedestrians, vehicles, vegetation, etc.). Hence, providing image understanding capabilities to autonomous robotics platforms is an essential target because cameras can capture the 3D scene as perceived by a human. In fact, given this need for 3D scene understanding, there is an increasing interest on joint objects and scene labeling in the form of geometry and semantic inference of the relevant entities contained in urban environments. In this regard, this Thesis tackles two challenges: 1) the prediction of road intersections geometry and, 2) the detection and orientation estimation of cars, pedestrians and cyclists. Different features extracted from stereo images of the KITTI public urban dataset are employed. This Thesis proposes a supervised learning of discriminative models that rely on strong machine learning techniques for data mining visual features. For the first task, we use 2D occupancy grid maps that are built from the stereo sequences captured by a moving vehicle in a mid-sized city. Based on these bird?s eye view images, we propose a smart parameterization of the layout of straight roads and 4 intersecting roads. The dependencies between the proposed discrete random variables that define the layouts are represented with Probabilistic Graphical Models. Then, the problem is formulated as a structured prediction, in which we employ Conditional Random Fields (CRF) for learning and convex Belief Propagation (dcBP) and Branch and Bound (BB) for inference. For the validation of the proposed methodology, a set of tests are carried out, which are based on real images and synthetic images with varying levels of random noise. In relation to the object detection and orientation estimation challenge in road scenes, this Thesis goal is to compete in the international challenge known as KITTI evaluation benchmark, which encourages researchers to push forward the current state of the art on visual recognition methods, particularized for 3D urban scene understanding. This Thesis proposes to modify the successful part-based object detector known as DPM in order to learn richer models from 2.5D data (color and disparity). Therefore, we revisit the DPM framework, which is based on HOG features and mixture models trained with a latent SVM formulation. Next, this Thesis performs a set of modifications on top of DPM: I) An extension to the DPM training pipeline that accounts for 3D-aware features. II) A detailed analysis of the supervised parameter learning. III) Two additional approaches: "feature whitening" and "stereo consistency check". Additionally, a) we analyze the KITTI dataset and several subtleties regarding to the evaluation protocol; b) a large set of cross-validated experiments show the performance of our contributions and, c) finally, our best performing approach is publicly ranked on the KITTI website, being the first one that reports results with stereo data, yielding an increased object detection precision (3%-6%) for the class 'car' and ranking first for the class cyclist

Background Subtraction in Video Surveillance

The aim of thesis is the real-time detection of moving and unconstrained surveillance environments monitored with static cameras. This is achieved based on the results provided by background subtraction. For this task, Gaussian Mixture Models (GMMs) and Kernel density estimation (KDE) are used. A thorough review of state-of-the-art formulations for the use of GMMs and KDE in the task of background subtraction reveals some further development opportunities, which are tackled in a novel GMM-based approach incorporating a variance controlling scheme. The proposed approach method is for parametric and non-parametric and gives us the better method for background subtraction, with more accuracy and easier parametrization of the models, for different environments. It also converges to more accurate models of the scenes. The detection of moving objects is achieved by using the results of background subtraction. For the detection of new static objects, two background models, learning at different rates, are used. This allows for a multi-class pixel classification, which follows the temporality of the changes detected by means of background subtraction. In a first approach, the subtraction of background models is done for parametric model and their results are shown. The second approach is for non-parametric models, where background subtraction is done using KDE non-parametric model. Furthermore, we have done some video engineering, where the background subtraction algorithm was employed so that, the background from one video and the foreground from another video are merged to form a new video. By doing this way, we can also do more complex video engineering with multiple videos. Finally, the results provided by region analysis can be used to improve the quality of the background models, therefore, considerably improving the detection results

Quantitative Estimation of Surface Soil Moisture in Agricultural Landscapes using Spaceborne Synthetic Aperture Radar Imaging at Different Frequencies and Polarizations

Soil moisture and its distribution in space and time plays an important role in the surface energy balance at the soil-atmosphere interface. It is a key variable influencing the partitioning of solar energy into latent and sensible heat flux as well as the partitioning of precipitation into runoff and percolation. Due to their large spatial variability, estimation of spatial patterns of soil moisture from field measurements is difficult and not feasible for large scale analyses. In the past decades, Synthetic Aperture Radar (SAR) remote sensing has proven its potential to quantitatively estimate near surface soil moisture at high spatial resolutions. Since the knowledge of the basic SAR concepts is important to understand the impact of different natural terrain features on the quantitative estimation of soil moisture and other surface parameters, the fundamental principles of synthetic aperture radar imaging are discussed. Also the two spaceborne SAR missions whose data was used in this study, the ENVISAT of the European Space Agency (ESA) and the ALOS of the Japanese Aerospace Exploration Agency (JAXA), are introduced. Subsequently, the two essential surface properties in the field of radar remote sensing, surface soil moisture and surface roughness are defined, and the established methods of their measurement are described. The in situ data used in this study, as well as the research area, the River Rur catchment, with the individual test sites where the data was collected between 2007 and 2010, are specified. On this basis, the important scattering theories in radar polarimetry are discussed and their application is demonstrated using novel polarimetric ALOS/PALSAR data. A critical review of different classical approaches to invert soil moisture from SAR imaging is provided. Five prevalent models have been chosen with the aim to provide an overview of the evolution of ideas and techniques in the field of soil moisture estimation from active microwave data. As the core of this work, a new semi-empirical model for the inversion of surface soil moisture from dual polarimetric L-band SAR data is introduced. This novel approach utilizes advanced polarimetric decomposition techniques to correct for the disturbing effects from surface roughness and vegetation on the soil moisture retrieval without the use of a priori knowledge. The land use specific algorithms for bare soil, grassland, sugar beet, and winter wheat allow quantitative estimations with accuracies in the order of 4 Vol.-%. Application of remotely sensed soil moisture patterns is demonstrated on the basis of mesoscale SAR data by investigating the variability of soil moisture patterns at different spatial scales ranging from field scale to catchment scale. The results show that the variability of surface soil moisture decreases with increasing wetness states at all scales. Finally, the conclusions from this dissertational research are summarized and future perspectives on how to extend the proposed model by means of improved ground based measurements and upcoming advances in sensor technology are discussed. The results obtained in this thesis lead to the conclusion that state-of-the-art spaceborne dual polarimetric L-band SAR systems are not only suitable to accurately retrieve surface soil moisture contents of bare as well as of vegetated agricultural fields and grassland, but for the first time also allow investigating within-field spatial heterogeneities from space