From uncertainty to adaptivity : multiscale edge detection and image segmentation

This thesis presents the research on two different tasks in computer vision: edge detection and image segmentation (including texture segmentation and motion field segmentation). The central issue of this thesis is the uncertainty of the joint space-frequency image analysis, which motivates the design of the adaptive multiscale/multiresolution schemes for edge detection and image segmentation. Edge detectors capture most of the local features in an image, including the object boundaries and the details of surface textures. Apart from these edge features, the region properties of surface textures and motion fields are also important for segmenting an image into disjoint regions. The major theoretical achievements of this thesis are twofold. First, a scale parameter for the local processing of an image (e.g. edge detection) is proposed. The corresponding edge behaviour in the scale space, referred to as Bounded Diffusion, is the basis of a multiscale edge detector where the scale is adjusted adaptively according to the local noise level. Second, an adaptive multiresolution clustering scheme is proposed for texture segmentation (referred to as Texture Focusing) and motion field segmentation. In this scheme, the central regions of homogeneous textures (motion fields) are analysed using coarse resolutions so as to achieve a better estimation of the textural content (optical flow), and the border region of a texture (motion field) is analysed using fine resolutions so as to achieve a better estimation of the boundary between textures (moving objects). Both of the above two achievements are the logical consequences of the uncertainty principle. Four algorithms, including a roof edge detector, a multiscale step edge detector, a texture segmentation scheme and a motion field segmentation scheme are proposed to address various aspects of edge detection and image segmentation. These algorithms have been implemented and extensively evaluated

Scale-based surface understanding using diffusion smoothing

The research discussed in this thesis is concerned with surface understanding from the viewpoint of recognition-oriented, scale-related processing based on surface curvatures and diffusion smoothing. Four problems below high level visual processing are investigated: 1) 3-dimensional data smoothing using a diffusion process; 2) Behaviour of shape features across multiple scales, 3) Surface segmentation over multiple scales; and 4) Symbolic description of surface features at multiple scales. In this thesis, the noisy data smoothing problem is treated mathematically as a boundary value problem of the diffusion equation instead of the well-known Gaussian convolution, In such a way, it provides a theoretical basis to uniformly interpret the interrelationships amongst diffusion smoothing, Gaussian smoothing, repeated averaging and spline smoothing. It also leads to solving the problem with a numerical scheme of unconditional stability, which efficiently reduces the computational complexity and preserves the signs of curvatures along the surface boundaries. Surface shapes are classified into eight types using the combinations of the signs of the Gaussian curvature K and mean curvature H, both of which change at different scale levels. Behaviour of surface shape features over multiple scale levels is discussed in terms of the stability of large shape features, the creation, remaining and fading of small shape features, the interaction between large and small features and the structure of behaviour of the nested shape features in the KH sign image. It provides a guidance for tracking the movement of shape features from fine to large scales and for setting up a surface shape description accordingly. A smoothed surface is partitioned into a set of regions based on curvature sign homogeneity. Surface segmentation is posed as a problem of approximating a surface up to the degree of Gaussian and mean curvature signs using the depth data alone How to obtain feasible solutions of this under-determined problem is discussed, which includes the surface curvature sign preservation, the reason that a sculptured surface can be segmented with the KH sign image alone and the selection of basis functions of surface fitting for obtaining the KH sign image or for region growing. A symbolic description of the segmented surface is set up at each scale level. It is composed of a dual graph and a geometrical property list for the segmented surface. The graph describes the adjacency and connectivity among different patches as the topological-invariant properties that allow some object's flexibility, whilst the geometrical property list is added to the graph as constraints that reduce uncertainty. With this organisation, a tower-like surface representation is obtained by tracking the movement of significant features of the segmented surface through different scale levels, from which a stable description can be extracted for inexact matching during object recognition

A study of optimization problems involving stochastic systems with jumps

The optimization problems involving stochastic systems are often encountered in financial systems, networks design and routing, supply-chain management, actuarial science, telecommunications systems, statistical pattern recognition analysis associated with electronic commerce and medical diagnosis.This thesis aims to develop computational methods for solving three optimization problems, where their dynamical systems are described by three different classes of stochastic systems with jumps.In Chapter 1, a brief review on optimization problems involving stochastic systems with jumps is given. It is then followed by the introduction of three optimization problems, where their dynamical systems are described by three different classes of stochastic systems with jumps. These three stochastic optimization problems will be studied in detail in Chapters 2, 3 and 4, respectively. The literature reviews on optimization problems involving these three stochastic systems with jumps are presented in the last three sections of each of Chapters 2, 3 and 4, respectively.In Chapter 2, an optimization problem involving nonparametric regression with jump points is considered. A two-stage method is proposed for nonparametric regression with jump points. In the first stage, we identify the rough locations of all the possible jump points of the unknown regression function. In the second stage, we map the yet to be decided jump points into pre-assigned fixed points. In this way, the time domain is divided into several sections. Then the spline function is used to approximate each section of the unknown regression function. These approximation problems are formulated and subsequently solved as optimization problems. The inverse time scaling transformation is then carried out, giving rise to an approximation to the nonparametric regression with jump points. For illustration, several examples are solved by using this method. The result obtained are highly satisfactory.In Chapter 3, the optimization problem involving nonparametric regression with jump curves is studied. A two-stage method is presented to construct an approximating surface with jump location curve from a set of observed data which are corrupted with noise. In the first stage, we detect an estimate of the jump location curve in a surface. In the second stage, we shift the jump location curve into a row pixels or column pixels. The shifted region is then divided into two disjoint subregions by the jump location row pixels. These subregions are expanded to two overlapping expanded subregions, each of which includes the jump location row pixels. We calculate artificial values at these newly added pixels by using the observed data and then approximate the surface on each expanded subregions in which the artificial values at the pixels in the jump location row pixels for each expanded subregion. The curve with minimal distance between the two surfaces is chosen as the curve dividing the region. Subsequently, two nonoverlapping tensor product cubic spline surfaces are obtained. Then, by carrying out the inverse space scaling transformation, the two fitted smooth surfaces in the original space are obtained. For illustration, a numerical example is solved using the method proposed.In Chapter 4, a class of stochastic optimal parameter selection problems described by linear Ito stochastic differential equations with state jumps subject to probabilistic constraints on the state is considered, where the times at which the jumps occurred as well as their heights are decision variables. We show that this constrained stochastic impulsive optimal parameter selection problem is equivalent to a deterministic impulsive optimal parameter selection problem subject to continuous state inequality constraints, where the times at which the jumps occurred as well as their heights remain as decision variables. Then we show that this constrained deterministic impulsive optimal parameter selection problem can be transformed into an equivalent constrained deterministic impulsive optimal parameter selection problem with fixed jump times. We approximate the continuous state inequality constraints by a sequence of canonical inequality constraints. This leads to a sequence of approximate deterministic impulsive optimal parameter selection problems subject to canonical inequality constraints. For each of these approximate problems, we derive the gradient formulas of the cost function and the constraint functions. On this basis, an efficient computational method is developed. For illustration, a numerical example is solved.Finally, Chapter 5 contains some concluding remarks and suggestions for future studies

Feature Extraction for image super-resolution using finite rate of innovation principles

To understand a real-world scene from several multiview pictures, it is necessary to find the disparities existing between each pair of images so that they are correctly related to one another. This process, called image registration, requires the extraction of some specific information about the scene. This is achieved by taking features out of the acquired images. Thus, the quality of the registration depends largely on the accuracy of the extracted features. Feature extraction can be formulated as a sampling problem for which perfect re- construction of the desired features is wanted. The recent sampling theory for signals with finite rate of innovation (FRI) and the B-spline theory offer an appropriate new frame- work for the extraction of features in real images. This thesis first focuses on extending the sampling theory for FRI signals to a multichannel case and then presents exact sampling results for two different types of image features used for registration: moments and edges. In the first part, it is shown that the geometric moments of an observed scene can be retrieved exactly from sampled images and used as global features for registration. The second part describes how edges can also be retrieved perfectly from sampled images for registration purposes. The proposed feature extraction schemes therefore allow in theory the exact registration of images. Indeed, various simulations show that the proposed extraction/registration methods overcome traditional ones, especially at low-resolution. These characteristics make such feature extraction techniques very appropriate for applications like image super-resolution for which a very precise registration is needed. The quality of the super-resolved images obtained using the proposed feature extraction meth- ods is improved by comparison with other approaches. Finally, the notion of polyphase components is used to adapt the image acquisition model to the characteristics of real digital cameras in order to run super-resolution experiments on real images

Symmetry in Structural Health Monitoring

In this Special Issue on symmetry, we mainly discuss the application of symmetry in various structural health monitoring. For example, considering the health monitoring of a known structure, by obtaining the static or dynamic response of the structure, using different signal processing methods, including some advanced filtering methods, to remove the influence of environmental noise, and extract structural feature parameters to determine the safety of the structure. These damage diagnosis methods can also be effectively applied to various types of infrastructure and mechanical equipment. For this reason, the vibration control of various structures and the knowledge of random structure dynamics should be considered, which will promote the rapid development of the structural health monitoring. Among them, signal extraction and evaluation methods are also worthy of study. The improvement of signal acquisition instruments and acquisition methods improves the accuracy of data. A good evaluation method will help to correctly understand the performance with different types of infrastructure and mechanical equipment

Information Extraction and Modeling from Remote Sensing Images: Application to the Enhancement of Digital Elevation Models

To deal with high complexity data such as remote sensing images presenting metric resolution over large areas, an innovative, fast and robust image processing system is presented. The modeling of increasing level of information is used to extract, represent and link image features to semantic content. The potential of the proposed techniques is demonstrated with an application to enhance and regularize digital elevation models based on information collected from RS images

Filtering of image sequences: on line edge detection and motion reconstruction

L'argomento della Tesi riguarda líelaborazione di sequenze di immagini, relative ad una scena in cui uno o pi˘ oggetti (possibilmente deformabili) si muovono e acquisite da un opportuno strumento di misura. A causa del processo di misura, le immagini sono corrotte da un livello di degradazione. Si riporta la formalizzazione matematica dellíinsieme delle immagini considerate, dellíinsieme dei moti ammissibili e della degradazione introdotta dallo strumento di misura. Ogni immagine della sequenza acquisita ha una relazione con tutte le altre, stabilita dalla legge del moto della scena. Líidea proposta in questa Tesi Ë quella di sfruttare questa relazione tra le diverse immagini della sequenza per ricostruire grandezze di interesse che caratterizzano la scena. Nel caso in cui si conosce il moto, líinteresse Ë quello di ricostruire i contorni dellíimmagine iniziale (che poi possono essere propagati attraverso la stessa legge del moto, in modo da ricostruire i contorni della generica immagine appartenente alla sequenza in esame), stimando líampiezza e del salto del livello di grigio e la relativa localizzazione. Nel caso duale si suppone invece di conoscere la disposizione dei contorni nellíimmagine iniziale e di avere un modello stocastico che descriva il moto; líobiettivo Ë quindi stimare i parametri che caratterizzano tale modello. Infine, si presentano i risultati dellíapplicazione delle due metodologie succitate a dati reali ottenuti in ambito biomedicale da uno strumento denominato pupillometro. Tali risultati sono di elevato interesse nellíottica di utilizzare il suddetto strumento a fini diagnostici

A Knowledge Integration Framework for 3D Shape Reconstruction

The modern emergence of automation in many industries has given impetus to extensive research into mobile robotics. Novel perception technologies now enable cars to drive autonomously, tractors to till a field automatically and underwater robots to construct pipelines. An essential requirement to facilitate both perception and autonomous navigation is the analysis of the 3D environment using sensors like laser scanners or stereo cameras. 3D sensors generate a very large number of 3D data points in sampling object shapes within an environment, but crucially do not provide any intrinsic information about the environment in which the robots operate with. This means unstructured 3D samples must be processed by application-specific models to enable a robot, for instance, to detect and identify objects and infer the scene geometry for path-planning more efficiently than by using raw 3D data. This thesis specifically focuses on the fundamental task of 3D shape reconstruction and modelling by presenting a new knowledge integration framework for unstructured 3D samples. The novelty lies in the representation of surfaces by algebraic functions with limited support, which enables the extraction of smooth consistent shapes from noisy samples with a heterogeneous density. Moreover, many surfaces in urban environments can reasonably be assumed to be planar, and the framework exploits this knowledge to enable effective noise suppression without loss of detail. This is achieved by using a convex optimization technique which has linear computational complexity. Thus is much more efficient than existing solutions. The new framework has been validated by critical experimental analysis and evaluation and has been shown to increase the accuracy of the reconstructed shape significantly compared to state-of-the-art methods. Applying this new knowledge integration framework means that less accurate, low-cost 3D sensors can be employed without sacrificing the high demands that 3D perception must achieve. This links well into the area of robotic inspection, as for example regarding small drones that use inaccurate and lightweight image sensors

Street Surfaces and Boundaries from Depth Image Sequences Using Probabilistic Models

This thesis presents an approach for the detection and reconstruction of street surfaces and boundaries from depth image sequences. Active driver assistance systems which monitor and interpret the environment based on vehicle mounted sensors to support the driver embody a current research focus of the automotive industry. An essential task of these systems is the modeling of the vehicle's static environment. This comprises the determination of the vertical slope and curvature characteristics of the street surface as well as the robust detection of obstacles and, thus, the free drivable space (alias free-space). In this regard, obstacles of low height, e.g. curbs, are of special interest since they often embody the first geometric delimiter of the free-space. The usage of depth images acquired from stereo camera systems becomes more important in this context due to the high data rate and affordable price of the sensor. However, recent approaches for object detection are often limited to the detection of objects which are distinctive in height, such as cars and guardrails, or explicitly address the detection of particular object classes. These approaches are usually based on extremely restrictive assumptions, such as planar street surfaces, in order to deal with the high measurement noise. The main contribution of this thesis is the development, analysis and evaluation of an approach which detects the free-space in the immediate maneuvering area in front of the vehicle and explicitly models the free-space boundary by means of a spline curve. The approach considers in particular obstacles of low height (higher than 10 cm) without limitation on particular object classes. Furthermore, the approach has the ability to cope with various slope and curvature characteristics of the observed street surface and is able to reconstruct this surface by means of a flexible spline model. In order to allow for robust results despite the flexibility of the model and the high measurement noise, the approach employs probabilistic models for the preprocessing of the depth map data as well as for the detection of the drivable free-space. An elevation model is computed from the depth map considering the paths of the optical rays and the uncertainty of the depth measurements. Based on this elevation model, an iterative two step approach is performed which determines the drivable free-space by means of a Markov Random Field and estimates the spline parameters of the free-space boundary curve and the street surface. Outliers in the elevation data are explicitly modeled. The performance of the overall approach and the influence of key components are systematically evaluated within experiments on synthetic and real world test scenarios. The results demonstrate the ability of the approach to accurately model the boundary of the drivable free-space as well as the street surface even in complex scenarios with multiple obstacles or strong curvature of the street surface. The experiments further reveal the limitations of the approach, which are discussed in detail.Schätzung von Straßenoberflächen und -begrenzungen aus Sequenzen von Tiefenkarten unter Verwendung probabilistischer Modelle Diese Arbeit präsentiert ein Verfahren zur Detektion und Rekonstruktion von Straßenoberflächen und -begrenzungen auf der Basis von Tiefenkarten. Aktive Fahrerassistenzsysteme, welche mit der im Fahrzeug verbauten Sensorik die Umgebung erfassen, interpretieren und den Fahrer unterstützen, sind ein aktueller Forschungsschwerpunkt der Fahrzeugindustrie. Eine wesentliche Aufgabe dieser Systeme ist die Modellierung der statischen Fahrzeugumgebung. Dies beinhaltet die Bestimmung der vertikalen Neigungs- und Krümmungseigenschaften der Fahrbahn, sowie die robuste Detektion von Hindernissen und somit des befahrbaren Freiraumes. Hindernisse von geringer Höhe, wie z.B. Bordsteine, sind in diesem Zusammenhang von besonderem Interesse, da sie häufig die erste geometrische Begrenzung des Fahrbahnbereiches darstellen. In diesem Kontext gewinnt die Verwendung von Tiefenkarten aus Stereo-Kamera-Systemen wegen der hohen Datenrate und relativ geringen Kosten des Sensors zunehmend an Bedeutung. Aufgrund des starken Messrauschens beschränken sich herkömmliche Verfahren zur Hinderniserkennung jedoch meist auf erhabene Objekte wie Fahrzeuge oder Leitplanken, oder aber adressieren einzelne Objektklassen wie Bordsteine explizit. Dazu werden häufig extrem restriktive Annahmen verwendet wie z.B. planare Straßenoberflächen. Der Hauptbeitrag dieser Arbeit besteht in der Entwicklung, Analyse und Evaluation eines Verfahrens, welches den befahrbaren Freiraum im Nahbereich des Fahrzeugs detektiert und dessen Begrenzung mit Hilfe einer Spline-Kurve explizit modelliert. Das Verfahren berücksichtigt insbesondere Hindernisse geringer Höhe (größer als 10 cm) ohne Beschränkung auf bestimmte Objektklassen. Weiterhin ist das Verfahren in der Lage, mit verschiedenartigen Neigungs- und Krümmungseigenschaften der vor dem Fahrzeug liegenden Fahrbahnoberfläche umzugehen und diese durch Verwendung eines flexiblen Spline-Modells zu rekonstruieren. Um trotz der hohen Flexibilität des Modells und des hohen Messrauschens robuste Ergebnisse zu erzielen, verwendet das Verfahren probabilistische Modelle zur Vorverarbeitung der Eingabedaten und zur Detektion des befahrbaren Freiraumes. Aus den Tiefenkarten wird unter Berücksichtigung der Strahlengänge und Unsicherheiten der Tiefenmessungen ein Höhenmodell berechnet. In einem iterativen Zwei-Schritt-Verfahren werden anhand dieses Höhenmodells der befahrbare Freiraum mit Hilfe eines Markov-Zufallsfeldes bestimmt sowie die Parameter der begrenzenden Spline-Kurve und Straßenoberfläche geschätzt. Ausreißer in den Höhendaten werden dabei explizit modelliert. Die Leistungsfähigkeit des Gesamtverfahrens sowie der Einfluss zentraler Komponenten, wird im Rahmen von Experimenten auf synthetischen und realen Testszenen systematisch analysiert. Die Ergebnisse demonstrieren die Fähigkeit des Verfahrens, die Begrenzung des befahrbaren Freiraumes sowie die Fahrbahnoberfläche selbst in komplexen Szenarien mit multiplen Hindernissen oder starker Fahrbahnkrümmung akkurat zu modellieren. Weiterhin werden die Grenzen des Verfahrens aufgezeigt und detailliert untersucht

Quadtree Structured Approximation Algorithms

The success of many image restoration algorithms is often due to their ability to sparsely describe the original signal. Many sparse promoting transforms exist, including wavelets, the so called ‘lets’ family of transforms and more recent non-local learned transforms. The first part of this thesis reviews sparse approximation theory, particularly in relation to 2-D piecewise polynomial signals. We also show the connection between this theory and current state of the art algorithms that cover the following image restoration and enhancement applications: denoising, deconvolution, interpolation and multi-view super resolution. In [63], Shukla et al. proposed a compression algorithm, based on a sparse quadtree decomposition model, which could optimally represent piecewise polynomial images. In the second part of this thesis we adapt this model to image restoration by changing the rate-distortion penalty to a description-length penalty. Moreover, one of the major drawbacks of this type of approximation is the computational complexity required to find a suitable subspace for each node of the quadtree. We address this issue by searching for a suitable subspace much more efficiently using the mathematics of updating matrix factorisations. Novel algorithms are developed to tackle the four problems previously mentioned. Simulation results indicate that we beat state of the art results when the original signal is in the model (e.g. depth images) and are competitive for natural images when the degradation is high.Open Acces