Variational Image Segmentation with Constraints

The research of Huizhu Pan addresses the problem of image segmentation with constraints though designing and solving various variational models. A novel constraint term is designed for the use of landmarks in image segmentation. Two region-based segmentation models were proposed where the segmentation contour passes through landmark points. A more stable and memory efficient solution to the self-repelling snakes model, a variational model with the topology preservation constraint, was also designed

Intelligent video surveillance

In the focus of this thesis are the new and modified algorithms for object detection, recognition and tracking within the context of video analytics. The manual video surveillance has been proven to have low effectiveness and, at the same time, high expense because of the need in manual labour of operators, which are additionally prone to erroneous decisions. Along with increase of the number of surveillance cameras, there is a strong need to push for automatisation of the video analytics. The benefits of this approach can be found both in military and civilian applications. For military applications, it can help in localisation and tracking of objects of interest. For civilian applications, the similar object localisation procedures can make the criminal investigations more effective, extracting the meaningful data from the massive video footage. Recently, the wide accessibility of consumer unmanned aerial vehicles has become a new threat as even the simplest and cheapest airborne vessels can carry some cargo that means they can be upgraded to a serious weapon. Additionally they can be used for spying that imposes a threat to a private life. The autonomous car driving systems are now impossible without applying machine vision methods. The industrial applications require automatic quality control, including non-destructive methods and particularly methods based on the video analysis. All these applications give a strong evidence in a practical need in machine vision algorithms for object detection, tracking and classification and gave a reason for writing this thesis. The contributions to knowledge of the thesis consist of two main parts: video tracking and object detection and recognition, unified by the common idea of its applicability to video analytics problems. The novel algorithms for object detection and tracking, described in this thesis, are unsupervised and have only a small number of parameters. The approach is based on rigid motion segmentation by Bayesian filtering. The Bayesian filter, which was proposed specially for this method and contributes to its novelty, is formulated as a generic approach, and then applied to the video analytics problems. The method is augmented with optional object coordinate estimation using plain two-dimensional terrain assumption which gives a basis for the algorithm usage inside larger sensor data fusion models. The proposed approach for object detection and classification is based on the evolving systems concept and the new Typicality-Eccentricity Data Analytics (TEDA) framework. The methods are capable of solving classical problems of data mining: clustering, classification, and regression. The methods are proposed in a domain-independent way and are capable of addressing shift and drift of the data streams. Examples are given for the clustering and classification of the imagery data. For all the developed algorithms, the experiments have shown sustainable results on the testing data. The practical applications of the proposed algorithms are carefully examined and tested

Variational methods and its applications to computer vision

Many computer vision applications such as image segmentation can be formulated in a ''variational'' way as energy minimization problems. Unfortunately, the computational task of minimizing these energies is usually difficult as it generally involves non convex functions in a space with thousands of dimensions and often the associated combinatorial problems are NP-hard to solve. Furthermore, they are ill-posed inverse problems and therefore are extremely sensitive to perturbations (e.g. noise). For this reason in order to compute a physically reliable approximation from given noisy data, it is necessary to incorporate into the mathematical model appropriate regularizations that require complex computations. The main aim of this work is to describe variational segmentation methods that are particularly effective for curvilinear structures. Due to their complex geometry, classical regularization techniques cannot be adopted because they lead to the loss of most of low contrasted details. In contrast, the proposed method not only better preserves curvilinear structures, but also reconnects some parts that may have been disconnected by noise. Moreover, it can be easily extensible to graphs and successfully applied to different types of data such as medical imagery (i.e. vessels, hearth coronaries etc), material samples (i.e. concrete) and satellite signals (i.e. streets, rivers etc.). In particular, we will show results and performances about an implementation targeting new generation of High Performance Computing (HPC) architectures where different types of coprocessors cooperate. The involved dataset consists of approximately 200 images of cracks, captured in three different tunnels by a robotic machine designed for the European ROBO-SPECT project.Open Acces

Information Extraction and Modeling from Remote Sensing Images: Application to the Enhancement of Digital Elevation Models

To deal with high complexity data such as remote sensing images presenting metric resolution over large areas, an innovative, fast and robust image processing system is presented. The modeling of increasing level of information is used to extract, represent and link image features to semantic content. The potential of the proposed techniques is demonstrated with an application to enhance and regularize digital elevation models based on information collected from RS images

Creating 3D city models from satellite imagery for integrated assessment and forecasting of solar energy

Buildings are the most prominent component in the urban environment. The geometric identification of urban buildings plays an important role in a range of urban applications, including 3D representations of buildings, energy consumption analysis, sustainable development, urban planning, risk assessment, and change detection. In particular, 3D building models can provide a comprehensive assessment of surfaces exposed to solar radiation. However, the identification of the available surfaces on urban structures and the actual locations which receive a sufficient amount of sunlight to increase installed power capacity (e.g. Photovoltaic systems) are crucial considerations for solar energy supply efficiency. Although considerable research has been devoted to detecting the rooftops of buildings, less attention has been paid to creating and completing 3D models of urban buildings. Therefore, there is a need to increase our understanding of the solar energy potential of the surfaces of building envelopes so we can formulate future adaptive energy policies for improving the sustainability of cities. The goal of this thesis was to develop a new approach to automatically model existing buildings for the exploitation of solar energy potential within an urban environment. By investigating building footprints and heights based on shadow information derived from satellite images, 3D city models were generated. Footprints were detected using a two level segmentation process: (1) the iterative graph cuts approach for determining building regions and (2) the active contour method and the adjusted-geometry parameters method for modifying the edges and shapes of the extracted building footprints. Building heights were estimated based on the simulation of artificial shadow regions using identified building footprints and solar information in the image metadata at pre-defined height increments. The difference between the actual and simulated shadow regions at every height increment was computed using the Jaccard similarity coefficient. The 3D models at the first level of detail were then obtained by extruding the building footprints based on their heights by creating image voxels and using the marching cube approach. In conclusion, 3D models of buildings can be generated solely from 2D data of the buildings’attributes in any selected urban area. The approach outperforms the past attempts, and mean error is reduced by at least 21%. Qualitative evaluations of the study illustrate that it is possible to achieve 3D building models based on satellite images with a mean error of less than 5 m. This comprehensive study allows for 3D city models to be generated in the absence of elevation attributes and additional data. Experiments revealed that this novel, automated method can be useful in a number of spatial analyses and urban sustainability applications

Rich probabilistic models for semantic labeling

Das Ziel dieser Monographie ist es die Methoden und Anwendungen des semantischen Labelings zu erforschen. Unsere Beiträge zu diesem sich rasch entwickelten Thema sind bestimmte Aspekte der Modellierung und der Inferenz in probabilistischen Modellen und ihre Anwendungen in den interdisziplinären Bereichen der Computer Vision sowie medizinischer Bildverarbeitung und Fernerkundung

Mesh generation using a correspondence distance field

The central tool of this work is a correspondence distance field to discrete surface points embedded within a quadtree data structure. The theory, development, and implementation of the distance field tool are described, and two main applications to two-dimensional mesh generation are presented with extension to three-dimensional capabilities in mind. First is a method for surface-oriented mesh generation from a sufficiently dense set of discrete surface points without connectivity information. Contour levels of distance from the body are specified and correspondences oriented normally to the contours are created. Regions of merging fronts inside and between objects are detected in the correspondence distance field and incorporated automatically. Second, the boundaries in a Voronoi diagram between specified coordinates are detected adaptively and used to make Delaunay tessellation. Tessellation of regions with holes is performed using ghost nodes. Images of meshed for each method are given for a sample set of test cases. Possible extensions, future work, and CFD applications are also discussed

Scene and crowd analysis using synthetic data generation with 3D quality improvements and deep network architectures

In this thesis, a scene analysis mainly focusing on vision-based techniques have been explored. The vision-based scene analysis techniques have a wide range of applications from surveillance, security to agriculture. A vision sensor can provide rich information about the environment such as colour, depth, shape, size and much more. This information can be further processed to have an in-depth knowledge of the scene such as type of environment, objects and distances. Hence, this thesis covers initially the background on human detection in particular pedestrian and crowd detection methods and introduces various vision-based techniques used in human detection. Followed by a detailed analysis of the use of synthetic data to improve the performance of state-of-the-art Deep Learning techniques and a multi-purpose synthetic data generation tool is proposed. The tool is a real-time graphics simulator which generates multiple types of synthetic data applicable for pedestrian detection, crowd density estimation, image segmentation, depth estimation, and 3D pose estimation. In the second part of the thesis, a novel technique has been proposed to improve the quality of the synthetic data. The inter-reflection also known as global illumination is a naturally occurring phenomena and is a major problem for 3D scene generation from an image. Thus, the proposed methods utilised a reverted ray-tracing technique to reduce the effect of inter-reflection problem and increased the quality of generated data. In addition, a method to improve the quality of the density map is discussed in the following chapter. The density map is the most commonly used technique to estimate crowds. However, the current procedure used to generate the map is not content-aware i.e., density map does not highlight the humans’ heads according to their size in the image. Thus, a novel method to generate a content-aware density map was proposed and demonstrated that the use of such maps can elevate the performance of an existing Deep Learning architecture. In the final part, a Deep Learning architecture has been proposed to estimate the crowd in the wild. The architecture tackled the challenging aspect such as perspective distortion by implementing several techniques like pyramid style inputs, scale aggregation method and self-attention mechanism to estimate a crowd density map and achieved state-of-the-art results at the time

Multi-perspective, Multi-modal Image Registration and Fusion

Multi-modal image fusion is an active research area with many civilian and military applications. Fusion is defined as strategic combination of information collected by various sensors from different locations or different types in order to obtain a better understanding of an observed scene or situation. Fusion of multi-modal images cannot be completed unless these two modalities are spatially aligned. In this research, I consider two important problems. Multi-modal, multi-perspective image registration and decision level fusion of multi-modal images. In particular, LiDAR and visual imagery. Multi-modal image registration is a difficult task due to the different semantic interpretation of features extracted from each modality. This problem is decoupled into three sub-problems. The first step is identification and extraction of common features. The second step is the determination of corresponding points. The third step consists of determining the registration transformation parameters. Traditional registration methods use low level features such as lines and corners. Using these features require an extensive optimization search in order to determine the corresponding points. Many methods use global positioning systems (GPS), and a calibrated camera in order to obtain an initial estimate of the camera parameters. The advantages of our work over the previous works are the following. First, I used high level-features, which significantly reduce the search space for the optimization process. Second, the determination of corresponding points is modeled as an assignment problem between a small numbers of objects. On the other side, fusing LiDAR and visual images is beneficial, due to the different and rich characteristics of both modalities. LiDAR data contain 3D information, while images contain visual information. Developing a fusion technique that uses the characteristics of both modalities is very important. I establish a decision-level fusion technique using manifold models