Point-Based Medialness for Movement Computing

We introduce the idea of using a perception-based medial point description of a biological form (such as a 2D profile of a moving animal) as a basis for movement computing which delivers computational schemes to automatically annotate movement and be capable of producing meaningful qualitative descriptions. We distinguish interior from exterior shape representation. Interior medialness is used to characterise deformations from straightness, corners and necks, while exterior medialness identifies the main concavities and inlands which are useful to verify parts extent and reason about articulation and movement. We define an interior dominant point as a well localised peak value in medialness representation, while an exterior dominant point is evaluated by identifying a region of concavity sub-tended by a minimum angular support. Furthermore, significant convex points are extracted from the object's form to further characterise the elongation of parts. We propose that our evaluated feature points are sufficiently representative, as a basis for shape characterisation, to address many of the goals of movement computing

Movement description and gesture recognition for live media arts

The research aims to develop novel techniques able to recognise different sequential gestures, to the level where they will describe and compute articulated movements in real time. In the context of live media arts, the research outcomes would change the paradigm of creating, learning, performing, designing for live media arts, by giving feedback on performance after analysing, in real time, the streaming video of the performance

Medialness and the Perception of Visual Art

In this article we explore the practical use of medialness informed by perception studies as a representation and processing layer for describing a class of works of visual art. Our focus is towards the description of 2D objects in visual art, such as found in drawings, paintings, calligraphy, graffiti writing, where approximate boundaries or lines delimit regions associated to recognizable objects or their constitutive parts. We motivate this exploration on the one hand by considering how ideas emerging from the visual arts, cartoon animation and general drawing practice point towards the likely importance of medialness in guiding the interaction of the traditionally trained artist with the artifact. On the other hand, we also consider recent studies and results in cognitive science which point in similar directions in emphasizing the likely importance of medialness, an extension of the abstract mathematical representation known as ‘medial axis’ or ‘Voronoi graphs’, as a core feature used by humans in perceiving shapes in static or dynamic scenarios.We illustrate the use of medialness in computations performed with finished artworks as well as artworks in the process of being created, modified, or evolved through iterations. Such computations may be used to guide an artificial arm in duplicating the human creative performance or used to study in greater depth the finished artworks. Our implementations represent a prototyping of such applications of computing to art analysis and creation and remain exploratory. Our method also provides a possible framework to compare similar artworks or to study iterations in the process of producing a final preferred depiction, as selected by the artist

Swarm-based identification of animation key points from 2D-medialness maps

In this article we present the use of dispersive flies optimisation (DFO) for swarms of particles active on a medialness map — a 2D field representation of shape informed by perception studies. Optimising swarms activity permits to efficiently identify shape-based keypoints to automatically annotate movement and is capable of producing meaningful qualitative descriptions for animation applications. When taken together as a set, these keypoints represent the full body pose of a character in each processed frame. In addition, such keypoints can be used to embody the notion of the Line of Action (LoA), a well known classic technique from the Disney studios used to capture the overall pose of a character to be fleshed out. Keypoints along a medialness ridge are local peaks which are efficiently localised using DFO driven swarms. DFO is optimised in a way so that it does not need to scan every image pixel and always tend to converge at these peaks. A series of experimental trials on different animation characters in movement sequences confirms the promising performance of the optimiser over a simpler, currently-in-use brute-force approach

Point-based medialness for 2D shape description and identification

Abstract We propose a perception-based medial point description of a natural form (2D: static or in articulated movement) as a framework for a shape representation which can then be efficiently used in biological species identification and matching tasks. Medialness is defined by adapting and refining a definition first proposed in the cognitive science literature when studying the visual attention of human subjects presented with articulated biological 2D forms in movement, such as horses, dogs and humans (walking, running). In particular, special loci of high medialness for the interior of a form in movement, referred to as “hot spots”, prove most attractive to the human perceptual system. We propose an algorithmic process to identify such hot spots. In this article we distinguish exterior from interior shape representation. We further augment hot spots with extremities of medialness ridges identifying significant concavities (from outside) and convexities (from inside). Our representation is strongly footed in results from cognitive psychology, but also inspired by know-how in art and animation, and the algorithmic part is influenced by techniques from more traditional computer vision. A robust shape matching algorithm is designed that finds the most relevant targets from a database of templates by comparing feature points in a scale, rotation and translation invariant way. The performance of our method has been tested on several databases. The robustness of the algorithm is further tested by perturbing the data-set at different levels

Point-based Medialness for Animal and Plant Identification

Abstract We introduce the idea of using a perception-based medial point description [#kovacs1998medial] of a natural form (2D static or in movement) as a framework for a part-based shape representation which can then be efficiently used in biological species identification and matching tasks. The first step is one of fuzzy medialness measurements of 2D segmented objects from intensity images which emphasises main shape information characteristics of an object's parts (e.g. concavities and folds along a contour). We distinguish interior from exterior shape description. Interior medialness is used to characterise deformations from straightness, corners and necks, while exterior medialness identifies the main concavities and inlands which are useful to verify parts extent and reason about articulation and movement. In a second step we identify a set of characteristic features points built from three types. We define (i) an Interior dominant point as a well localised peak value in medialness representation, while (ii) an exterior dominant point is evaluated by identifying a region of concavity sub-tended by a minimum angular support. Furthermore, (iii) convex point are extracted from the form to further characterise the elongation of parts. Our evaluated feature points, together are sufficiently invariant to shape movement, where the articulation in moving objects are characterised by exterior dominant points. In the third step, a robust shape matching algorithm is designed that finds the most relevant targets from a database of templates by comparing the dominant feature points in a scale, rotation and translation invariant way (inspired by the SIFT method [#lowe2004distinctive]). The performance of our method has been tested on several databases. The robustness of the algorithm is further tested by perturbing the data-set at different scales

Point-based Medialness for Animal and Plant Identification

We introduce the idea of using a perception-based medial point description [9] of a natural form (2D static or in movement) as a framework for a part-based shape representation which can then be efficiently used in biological species identification and matching tasks. The first step is one of fuzzy medialness measurements of 2D segmented objects from intensity images which emphasises main shape information characteristics of an object’s parts (e.g. concavities and folds along a contour). We distinguish interior from exterior shape description. Interior medialness is used to characterise deformations from straightness, corners and necks, while exterior medialness identifies the main concavities and inlands which are useful to verify parts extent and reason about articulation and movement. In a second step we identify a set of characteristic features points built from three types. We define (i) an Interior dominant point as a well localised peak value in medialness representation, while (ii) an exterior dominant point is evaluated by identifying a region of concavity sub-tended by a minimum angular support. Furthermore, (iii) convex point are extracted from the form to further characterise the elongation of parts. Our evaluated feature points, together are sufficiently invariant to shape move ment, where the articulation in moving objects are characterised by exterior dominant points. In the third step, a robust shape matching algorithm is designed that finds the most relevant targets from a database of templates by comparing the dominant feature points in a scale, rotation and translation invariant way (inspired by the SIFT method [17]). The performance of our method has been tested on several databases. The robustness of the algorithm is further tested by perturbing the data-set at different scales

Registration and Analysis of Vascular Images

We have developed a method for rigidly aligning images of tubes. This paper presents an evaluation of the consistency of that method for three-dimensional images of human vasculature. Vascular images may contain alignment ambiguities, poorly corresponding vascular networks, and non-rigid deformations, yet the Monte Carlo experiments presented in this paper show that our method provides registrations with sub-voxel consistency in less than one minute. Our registration method builds on the principals of our ridges-and-widths tube modeling work; this registration method operates by aligning models of the tubes in a source image with subsequent target images. The registration method’s consistency results from incorporate multi-scale ridge and width measures into the model-image match metric. The method’s speed comes from the use of coarse-to-fine registration strategies that are directly enabled by our tube models and the model-image match metric. In this paper we also show that the method’s insensitivity to local, non-rigid deformations enables the visualization and quantification of the effects of such deformations

Blood Vessel Segmentation and shape analysis for quantification of Coronary Artery Stenosis in CT Angiography

This thesis presents an automated framework for quantitative vascular shape analysis of the coronary arteries, which constitutes an important and fundamental component of an automated image-based diagnostic system. Firstly, an automated vessel segmentation algorithm is developed to extract the coronary arteries based on the framework of active contours. Both global and local intensity statistics are utilised in the energy functional calculation, which allows for dealing with non-uniform brightness conditions, while evolving the contour towards to the desired boundaries without being trapped in local minima. To suppress kissing vessel artifacts, a slice-by-slice correction scheme, based on multiple regions competition, is proposed to identify and track the kissing vessels throughout the transaxial images of the CTA data. Based on the resulting segmentation, we then present a dedicated algorithm to estimate the geometric parameters of the extracted arteries, with focus on vessel bifurcations. In particular, the centreline and associated reference surface of the coronary arteries, in the vicinity of arterial bifurcations, are determined by registering an elliptical cross sectional tube to the desired constituent branch. The registration problem is solved by a hybrid optimisation method, combining local greedy search and dynamic programming, which ensures the global optimality of the solution and permits the incorporation of any hard constraints posed to the tube model within a natural and direct framework. In contrast with conventional volume domain methods, this technique works directly on the mesh domain, thus alleviating the need for image upsampling. The performance of the proposed framework, in terms of efficiency and accuracy, is demonstrated on both synthetic and clinical image data. Experimental results have shown that our techniques are capable of extracting the major branches of the coronary arteries and estimating the related geometric parameters (i.e., the centreline and the reference surface) with a high degree of agreement to those obtained through manual delineation. Particularly, all of the major branches of coronary arteries are successfully detected by the proposed technique, with a voxel-wise error at 0.73 voxels to the manually delineated ground truth data. Through the application of the slice-by-slice correction scheme, the false positive metric, for those coronary segments affected by kissing vessel artifacts, reduces from 294% to 22.5%. In terms of the capability of the presented framework in defining the location of centrelines across vessel bifurcations, the mean square errors (MSE) of the resulting centreline, with respect to the ground truth data, is reduced by an average of 62.3%, when compared with initial estimation obtained using a topological thinning based algorithm