3-D Center-Weighted Vector Directional Filters for Noisy Color Sequences

This paper focuses on a noise filtering in color image sequences, where a new class of center-weighted vector directional filters is provided. According to high dimensionality of color image sequences, where besides the spatial frequencies in the frames it is necessary to consider the temporal correlation of an image sequence and the correlation between color channels too, the processing of color image sequences represents very important and interesting problem. Clearly, the color image sequences represent three-dimensional (3-D) vector-valued image signals and thus, the 3-D vector filters provide optimal approach, only. Novelty of this paper lies in the impulse noise suppression by a new class of center-weighted vector directional filters, where the influence of the filter parameter to filter performance is analyzed. The interesting behavior of a new filter class is illustrated by a number of experimental results and comparisons with the well-known filtering algorithms for color image sequences

Reduction of smoothing levels in LUM FTC filter structure

In this paper, we analyze the possibility of the reduction of smoothing levels in 3-D adaptive lower-upper-middle (LUM) smoother based on the fixed threshold control (FTC). Besides the excellent noise attenuation capability with the simultaneous signal-detail preservation, recently introduced LUM FTC filter with a window size N is characterized by a relatively complex structure, where an estimate is formed according to (N +1)/2 decision rules. This fact can constrain its possible hardware filter implementation in real motion video applications. In order to simplify the filter complexity, however, to retain the excellent filter performance simultaneously, we propose two approaches such as linear reduction of smoothing levels and optimal reduction based on genetic algorithm.Dans cet article, nous analysons la possibilité de réduire le nombre de niveaux de lissage d'un filtre LUM (lower-upper-middle) adaptatif 3-D basé sur un contrôle par seuils fixes (FTC = fixed threshold control). Outre son excellente capacité d'atténuation du bruit tout en assurant la conservation des détails, le filtre LUM FTC avec une fenêtre de taille N, est caractérisé par une structure relativement complexe, où l'estimation de la valeur de sortie est faite en fonction de (N + 1)/2 règles de décision. Ceci peut entraver l'implémentation matérielle de tels filtres dans des applications vidéo temps réel. Afin de simplifier la complexité du filtre tout en gardant ses excellentes performances, nous proposons deux approches qui sont la réduction linéaire du nombre de niveaux de lissage et la réduction optimale basée sur un algorithme génétique

Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization

Fuzzy techniques for noise removal in image sequences and interval-valued fuzzy mathematical morphology

Image sequences play an important role in today's world. They provide us a lot of information. Videos are for example used for traffic observations, surveillance systems, autonomous navigation and so on. Due to bad acquisition, transmission or recording, the sequences are however usually corrupted by noise, which hampers the functioning of many image processing techniques. A preprocessing module to filter the images often becomes necessary. After an introduction to fuzzy set theory and image processing, in the first main part of the thesis, several fuzzy logic based video filters are proposed: one filter for grayscale video sequences corrupted by additive Gaussian noise and two color extensions of it and two grayscale filters and one color filter for sequences affected by the random valued impulse noise type. In the second main part of the thesis, interval-valued fuzzy mathematical morphology is studied. Mathematical morphology is a theory intended for the analysis of spatial structures that has found application in e.g. edge detection, object recognition, pattern recognition, image segmentation, image magnification… In the thesis, an overview is given of the evolution from binary mathematical morphology over the different grayscale morphology theories to interval-valued fuzzy mathematical morphology and the interval-valued image model. Additionally, the basic properties of the interval-valued fuzzy morphological operators are investigated. Next, also the decomposition of the interval-valued fuzzy morphological operators is investigated. We investigate the relationship between the cut of the result of such operator applied on an interval-valued image and structuring element and the result of the corresponding binary operator applied on the cut of the image and structuring element. These results are first of all interesting because they provide a link between interval-valued fuzzy mathematical morphology and binary mathematical morphology, but such conversion into binary operators also reduces the computation. Finally, also the reverse problem is tackled, i.e., the construction of interval-valued morphological operators from the binary ones. Using the results from a more general study in which the construction of an interval-valued fuzzy set from a nested family of crisp sets is constructed, increasing binary operators (e.g. the binary dilation) are extended to interval-valued fuzzy operators

Depth Enhancement and Surface Reconstruction with RGB/D Sequence

Surface reconstruction and 3D modeling is a challenging task, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. It is fundamental to many applications such as robot navigation, animation and scene understanding, industrial control and medical diagnosis. In this dissertation, I take advantage of the consumer depth sensors for surface reconstruction. Considering its limited performance on capturing detailed surface geometry, a depth enhancement approach is proposed in the first place to recovery small and rich geometric details with captured depth and color sequence. In addition to enhancing its spatial resolution, I present a hybrid camera to improve the temporal resolution of consumer depth sensor and propose an optimization framework to capture high speed motion and generate high speed depth streams. Given the partial scans from the depth sensor, we also develop a novel fusion approach to build up complete and watertight human models with a template guided registration method. Finally, the problem of surface reconstruction for non-Lambertian objects, on which the current depth sensor fails, is addressed by exploiting multi-view images captured with a hand-held color camera and we propose a visual hull based approach to recovery the 3D model

Efficient 3D Segmentation, Registration and Mapping for Mobile Robots

Sometimes simple is better! For certain situations and tasks, simple but robust methods can achieve the same or better results in the same or less time than related sophisticated approaches. In the context of robots operating in real-world environments, key challenges are perceiving objects of interest and obstacles as well as building maps of the environment and localizing therein. The goal of this thesis is to carefully analyze such problem formulations, to deduce valid assumptions and simplifications, and to develop simple solutions that are both robust and fast. All approaches make use of sensors capturing 3D information, such as consumer RGBD cameras. Comparative evaluations show the performance of the developed approaches. For identifying objects and regions of interest in manipulation tasks, a real-time object segmentation pipeline is proposed. It exploits several common assumptions of manipulation tasks such as objects being on horizontal support surfaces (and well separated). It achieves real-time performance by using particularly efficient approximations in the individual processing steps, subsampling the input data where possible, and processing only relevant subsets of the data. The resulting pipeline segments 3D input data with up to 30Hz. In order to obtain complete segmentations of the 3D input data, a second pipeline is proposed that approximates the sampled surface, smooths the underlying data, and segments the smoothed surface into coherent regions belonging to the same geometric primitive. It uses different primitive models and can reliably segment input data into planes, cylinders and spheres. A thorough comparative evaluation shows state-of-the-art performance while computing such segmentations in near real-time. The second part of the thesis addresses the registration of 3D input data, i.e., consistently aligning input captured from different view poses. Several methods are presented for different types of input data. For the particular application of mapping with micro aerial vehicles where the 3D input data is particularly sparse, a pipeline is proposed that uses the same approximate surface reconstruction to exploit the measurement topology and a surface-to-surface registration algorithm that robustly aligns the data. Optimization of the resulting graph of determined view poses then yields globally consistent 3D maps. For sequences of RGBD data this pipeline is extended to include additional subsampling steps and an initial alignment of the data in local windows in the pose graph. In both cases, comparative evaluations show a robust and fast alignment of the input data

Hierarchical representations for spatio-temporal visual attention: modeling and understanding

Mención Internacional en el título de doctorDentro del marco de la Inteligencia Artificial, la Visión Artificial es una disciplina científica que tiene como objetivo simular automaticamente las funciones del sistema visual humano, tratando de resolver tareas como la localización y el reconocimiento de objetos, la detección de eventos o el seguimiento de objetos....Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Luis Salgado Álvarez de Sotomayor.- Secretario: Ascensión Gallardo Antolín.- Vocal: Jenny Benois Pinea

Optimal Mass Transport with Lagrangian Workflow Reveals Advective and Diffusion Driven Solute Transport in the Glymphatic System

The glymphatic system (GS) hypothesis states that advective driven cerebrospinal fluid (CSF) influx from the perivascular spaces into the interstitial fluid space rapidly transport solutes and clear waste from brain. However, the presence of advection in neuropil is contested and solutes are claimed to be transported by diffusion only. To address this controversy, we implemented a regularized version of the optimal mass transport (rOMT) problem, wherein the advection/diffusion equation is the only a priori assumption required. rOMT analysis with a Lagrangian perspective of GS transport revealed that solute speed was faster in CSF compared to grey and white matter. Further, rOMT analysis also demonstrated 2-fold differences in regional solute speed within the brain. Collectively, these results imply that advective transport dominates in CSF while diffusion and advection both contribute to GS transport in parenchyma. In a rat model of cerebral small vessel disease (cSVD), solute transport in the perivascular spaces (PVS) and PVS-to-tissue transfer was slower compared to normal rats. Thus, the analytical framework of rOMT provides novel insights in the local dynamics of GS transport that may have implications for neurodegenerative diseases. Future studies should apply the rOMT analysis approach to confirm GS transport reductions in humans with cSVD