Hierarchically grouped 2D local features applied to edge contour localisation

One of the most significant research topics in computer vision is object detection. Most of the reported object detection results localise the detected object within a bounding box, but do not explicitly label the edge contours of the object. Since object contours provide a fundamental diagnostic of object shape, some researchers have initiated work on linear contour feature representations for object detection and localisation. However, linear contour feature-based localisation is highly dependent on the performance of linear contour detection within natural images, and this can be perturbed significantly by a cluttered background. In addition, the conventional approach to achieving rotation-invariant features is to rotate the feature receptive field to align with the local dominant orientation before computing the feature representation. Grid resampling after rotation adds extra computational cost and increases the total time consumption for computing the feature descriptor. Though it is not an expensive process if using current computers, it is appreciated that if each step of the implementation is faster to compute especially when the number of local features is increasing and the application is implemented on resource limited ”smart devices”, such as mobile phones, in real-time. Motivated by the above issues, a 2D object localisation system is proposed in this thesis that matches features of edge contour points, which is an alternative method that takes advantage of the shape information for object localisation. This is inspired by edge contour points comprising the basic components of shape contours. In addition, edge point detection is usually simpler to achieve than linear edge contour detection. Therefore, the proposed localization system could avoid the need for linear contour detection and reduce the pathological disruption from the image background. Moreover, since natural images usually comprise many more edge contour points than interest points (i.e. corner points), we also propose new methods to generate rotation-invariant local feature descriptors without pre-rotating the feature receptive field to improve the computational efficiency of the whole system. In detail, the 2D object localisation system is achieved by matching edge contour points features in a constrained search area based on the initial pose-estimate produced by a prior object detection process. The local feature descriptor obtains rotation invariance by making use of rotational symmetry of the hexagonal structure. Therefore, a set of local feature descriptors is proposed based on the hierarchically hexagonal grouping structure. Ultimately, the 2D object localisation system achieves a very promising performance based on matching the proposed features of edge contour points with the mean correct labelling rate of the edge contour points 0.8654 and the mean false labelling rate 0.0314 applied on the data from Amsterdam Library of Object Images (ALOI). Furthermore, the proposed descriptors are evaluated by comparing to the state-of-the-art descriptors and achieve competitive performances in terms of pose estimate with around half-pixel pose error

Articulated human tracking and behavioural analysis in video sequences

Recently, there has been a dramatic growth of interest in the observation and tracking of human subjects through video sequences. Arguably, the principal impetus has come from the perceived demand for technological surveillance, however applications in entertainment, intelligent domiciles and medicine are also increasing. This thesis examines human articulated tracking and the classi cation of human movement, rst separately and then as a sequential process. First, this thesis considers the development and training of a 3D model of human body structure and dynamics. To process video sequences, an observation model is also designed with a multi-component likelihood based on edge, silhouette and colour. This is de ned on the articulated limbs, and visible from a single or multiple cameras, each of which may be calibrated from that sequence. Second, for behavioural analysis, we develop a methodology in which actions and activities are described by semantic labels generated from a Movement Cluster Model (MCM). Third, a Hierarchical Partitioned Particle Filter (HPPF) was developed for human tracking that allows multi-level parameter search consistent with the body structure. This tracker relies on the articulated motion prediction provided by the MCM at pose or limb level. Fourth, tracking and movement analysis are integrated to generate a probabilistic activity description with action labels. The implemented algorithms for tracking and behavioural analysis are tested extensively and independently against ground truth on human tracking and surveillance datasets. Dynamic models are shown to predict and generate synthetic motion, while MCM recovers both periodic and non-periodic activities, de ned either on the whole body or at the limb level. Tracking results are comparable with the state of the art, however the integrated behaviour analysis adds to the value of the approach.Overseas Research Students Awards Scheme (ORSAS

Ein mobiler Serviceroboter zur Automatisierung der Probenahme und des Probenmanagements in einem biotechnologischen Pilotlabor

Scherer T. A mobile service robot for automisation of sample taking and sample management in a biotechnological pilot laboratory. Bielefeld (Germany): Bielefeld University; 2004.In biotechnologischen Laboratorien ist die Qualität der typischerweise pharmazeutischen Produkte ein wortwörtlich lebenswichtiges Ziel. Die Qualität der Zellkultivierungen wurde historisch nur durch off-line Messungen von physikalischen Prozessparametern wie pH und pO2 sichergestellt. Biologische Parameter wie die Zelldichte und -viabilität wurden nur off-line gemessen, weil das dazu notwendige Probenmanagement hochkomplizierte Manipulationen und Analysen beinhaltet und deshalb nicht automatisiert werden konnte. Es gibt zwar mehrere automatisierte Geräte, um einem Labortechniker zu assistieren, aber kein System, welches das gesamte Probenmanagement automatisiert. In dieser Arbeit wird ein neuer Typ von Serviceroboter präsentiert, der aus einem auf einer mobilen Plattform montierten Roboterarm besteht und diese Lücke schließt. Dieser Roboter muss eine ganze Reihe von Problemen bewältigen: Er muss seine Position im Labor bestimmen können (Lokalisation), er muss eine kollisionsfreie Bahn zu den beteiligten Geräten finden können (Bahnplanung mit Hindernisvermeidung), er darf bei seinen Bewegungen keine Menschen gefährden oder Laborausrüstung beschädigen (Kollisionsvermeidung), er muss die zu bedienenden Geräte erkennen und ihre Position präzise messen können (Bildverarbeitung), er muss sie bedienen können (Armsteuerung), er muss Objekte greifen können (Greifer und Finger) und er muss sie gefügig handhaben können, um sie nicht zu beschädigen (Kraftregelung). Er muss autonom sein, um nur die allernotwendigste Menge an Benutzereingriffen zu benötigen, und doch durch ein Laborsteuerprogramm kontrollierbar sein, um Eingriffe zu erlauben. Schließlich muss er einfach durch ungeschultes Personal zu warten sein. All diese Aspekte werden von dem in dieser Arbeit präsentierten neuen Robotersystem abgedeckt.In biotechnolgical laboratories, the quality of the typically pharmaceutical product is a literally life-important goal. Historically, the quality of the cell cultivations was ensured by on-line measurements of physical process parameters like pH and pO2 only. Biological parameters like cell density and viability were only measured off-line, because the necessary sample management involves highly complicated manipulations and analyses and could therefore not be automated. Various automated devices to assist a laboratory technician do exist, but so far no system to automate the entire sample management. In this work a novel type of service robot consisting of a robot arm mounted on a mobile platform is presented that closes this gap. This robot has to master a multitude of problems: It must be able to locate its position in the laboratory (localisation), it must be able to find a collision-free path to the involved devices (path planning with obstacle avoidance), it must not endanger humans or damage laboratory equipment while moving (collision avoidance), it must be able to recognize the devices to be manipulated and measure their precise position (computer vision), it must be able to manipulate them (arm control), it must be able to grasp objects (gripper and fingers) and it must be able to handle them with compliance in order to not damage them (force control). It must be autonomous in order to only require the least possible amount of user intervention, and yet controllable by a laboratory control program in order to allow intervention. Finally, it must be easily maintainable by non-expert personell. All these aspects are covered by the novel robot system presented in this thesis

Audio-coupled video content understanding of unconstrained video sequences

Unconstrained video understanding is a difficult task. The main aim of this thesis is to recognise the nature of objects, activities and environment in a given video clip using both audio and video information. Traditionally, audio and video information has not been applied together for solving such complex task, and for the first time we propose, develop, implement and test a new framework of multi-modal (audio and video) data analysis for context understanding and labelling of unconstrained videos. The framework relies on feature selection techniques and introduces a novel algorithm (PCFS) that is faster than the well-established SFFS algorithm. We use the framework for studying the benefits of combining audio and video information in a number of different problems. We begin by developing two independent content recognition modules. The first one is based on image sequence analysis alone, and uses a range of colour, shape, texture and statistical features from image regions with a trained classifier to recognise the identity of objects, activities and environment present. The second module uses audio information only, and recognises activities and environment. Both of these approaches are preceded by detailed pre-processing to ensure that correct video segments containing both audio and video content are present, and that the developed system can be made robust to changes in camera movement, illumination, random object behaviour etc. For both audio and video analysis, we use a hierarchical approach of multi-stage classification such that difficult classification tasks can be decomposed into simpler and smaller tasks. When combining both modalities, we compare fusion techniques at different levels of integration and propose a novel algorithm that combines advantages of both feature and decision-level fusion. The analysis is evaluated on a large amount of test data comprising unconstrained videos collected for this work. We finally, propose a decision correction algorithm which shows that further steps towards combining multi-modal classification information effectively with semantic knowledge generates the best possible results

3D Object Recognition Based On Constrained 2D Views

The aim of the present work was to build a novel 3D object recognition system capable of classifying man-made and natural objects based on single 2D views. The approach to this problem has been one motivated by recent theories on biological vision and multiresolution analysis. The project's objectives were the implementation of a system that is able to deal with simple 3D scenes and constitutes an engineering solution to the problem of 3D object recognition, allowing the proposed recognition system to operate in a practically acceptable time frame. The developed system takes further the work on automatic classification of marine phytoplank- (ons, carried out at the Centre for Intelligent Systems, University of Plymouth. The thesis discusses the main theoretical issues that prompted the fundamental system design options. The principles and the implementation of the coarse data channels used in the system are described. A new multiresolution representation of 2D views is presented, which provides the classifier module of the system with coarse-coded descriptions of the scale-space distribution of potentially interesting features. A multiresolution analysis-based mechanism is proposed, which directs the system's attention towards potentially salient features. Unsupervised similarity-based feature grouping is introduced, which is used in coarse data channels to yield feature signatures that are not spatially coherent and provide the classifier module with salient descriptions of object views. A simple texture descriptor is described, which is based on properties of a special wavelet transform. The system has been tested on computer-generated and natural image data sets, in conditions where the inter-object similarity was monitored and quantitatively assessed by human subjects, or the analysed objects were very similar and their discrimination constituted a difficult task even for human experts. The validity of the above described approaches has been proven. The studies conducted with various statistical and artificial neural network-based classifiers have shown that the system is able to perform well in all of the above mentioned situations. These investigations also made possible to take further and generalise a number of important conclusions drawn during previous work carried out in the field of 2D shape (plankton) recognition, regarding the behaviour of multiple coarse data channels-based pattern recognition systems and various classifier architectures. The system possesses the ability of dealing with difficult field-collected images of objects and the techniques employed by its component modules make possible its extension to the domain of complex multiple-object 3D scene recognition. The system is expected to find immediate applicability in the field of marine biota classification

Fundamental Study of Photoluminescence-Shape Relationship of Fluorescent Nanodiamonds using Machine Learning Assisted Correlative Transmission Electron Microscopy and Photoluminescence Microscopy Method

Luminescent nanoparticles have shown wide applications ranging from lighting, display, sensors, and biomedical diagnostics and imaging. Among these, fluorescent nanodiamonds (FNDs) containing nitrogen-vacancy (NV) color centers are posed as emerging materials particularly in biomedical and biological imaging applications due to their room-temperature emission, excellent photo- and chemical- stability, high bio-compatibility, and versatile functionalization potentials. The shape variation of nanoparticles has a decisive influence on their fluorescence. However, current relative studies are limited by the lack of reliable statistical analysis of nanoparticle shape and the difficulty of achieving a precise correlation between shape/structure and optical measurements of large numbers of individual nanoparticles. Therefore, new methods are urgently needed to overcome these challenges to assist in nanoparticle synthesis control and fluorescence performance optimization. In this thesis a new correlative TEM and photoluminescence (PL) microscopy (TEMPL) method has been developed that combines the measurements of the optical properties and the materials structure at the exact same particle and sample area, so that accurate correlation can be established to statistically study the FND morphology/structure and PL properties, at the single nanoparticle level. Moreover, machine learning based methods have been developed for categorizing the 2D and 3D shapes of a large number of nanoparticles generated in TEMPL method. This ML-assisted TEMPL method has been applied to understand the PL correlation with the size and shape of FNDs at the single particle level. In this thesis, a strong correlation between particle morphology and NV fluorescence in FND particles has been revealed: thin, flake-like particles produce enhanced fluorescence. The robustness of this trend is proven in FND with different surface oxidation treatments. This finding offers guidance for fluorescence-optimized sensing applications of FND, by controlling the shape of the particles in fabrication. Overall the TEMPL methodology developed in the thesis provides a versatile and general way to study the shape and fluorescence relationship of various nanoparticles and opens up the possibility of correlation methods between other characterisation techniques

Inferring Geodesic Cerebrovascular Graphs: Image Processing, Topological Alignment and Biomarkers Extraction

A vectorial representation of the vascular network that embodies quantitative features - location, direction, scale, and bifurcations - has many potential neuro-vascular applications. Patient-specific models support computer-assisted surgical procedures in neurovascular interventions, while analyses on multiple subjects are essential for group-level studies on which clinical prediction and therapeutic inference ultimately depend. This first motivated the development of a variety of methods to segment the cerebrovascular system. Nonetheless, a number of limitations, ranging from data-driven inhomogeneities, the anatomical intra- and inter-subject variability, the lack of exhaustive ground-truth, the need for operator-dependent processing pipelines, and the highly non-linear vascular domain, still make the automatic inference of the cerebrovascular topology an open problem. In this thesis, brain vessels’ topology is inferred by focusing on their connectedness. With a novel framework, the brain vasculature is recovered from 3D angiographies by solving a connectivity-optimised anisotropic level-set over a voxel-wise tensor field representing the orientation of the underlying vasculature. Assuming vessels joining by minimal paths, a connectivity paradigm is formulated to automatically determine the vascular topology as an over-connected geodesic graph. Ultimately, deep-brain vascular structures are extracted with geodesic minimum spanning trees. The inferred topologies are then aligned with similar ones for labelling and propagating information over a non-linear vectorial domain, where the branching pattern of a set of vessels transcends a subject-specific quantized grid. Using a multi-source embedding of a vascular graph, the pairwise registration of topologies is performed with the state-of-the-art graph matching techniques employed in computer vision. Functional biomarkers are determined over the neurovascular graphs with two complementary approaches. Efficient approximations of blood flow and pressure drop account for autoregulation and compensation mechanisms in the whole network in presence of perturbations, using lumped-parameters analog-equivalents from clinical angiographies. Also, a localised NURBS-based parametrisation of bifurcations is introduced to model fluid-solid interactions by means of hemodynamic simulations using an isogeometric analysis framework, where both geometry and solution profile at the interface share the same homogeneous domain. Experimental results on synthetic and clinical angiographies validated the proposed formulations. Perspectives and future works are discussed for the group-wise alignment of cerebrovascular topologies over a population, towards defining cerebrovascular atlases, and for further topological optimisation strategies and risk prediction models for therapeutic inference. Most of the algorithms presented in this work are available as part of the open-source package VTrails