Seeing Behind The Scene: Using Symmetry To Reason About Objects in Cluttered Environments

Rapid advances in robotic technology are bringing robots out of the controlled environments of assembly lines and factories into the unstructured and unpredictable real-life workspaces of human beings. One of the prerequisites for operating in such environments is the ability to grasp previously unobserved physical objects. To achieve this individual objects have to be delineated from the rest of the environment and their shape properties estimated from incomplete observations of the scene. This remains a challenging task due to the lack of prior information about the shape and pose of the object as well as occlusions in cluttered scenes. We attempt to solve this problem by utilizing the powerful concept of symmetry. Symmetry is ubiquitous in both natural and man-made environments. It reveals redundancies in the structure of the world around us and thus can be used in a variety of visual processing tasks. In this thesis we propose a complete pipeline for detecting symmetric objects and recovering their rotational and reflectional symmetries from 3D reconstructions of natural scenes. We begin by obtaining a multiple-view 3D pointcloud of the scene using the Kinect Fusion algorithm. Additionally a voxelized occupancy map of the scene is extracted in order to reason about occlusions. We propose two classes of algorithms for symmetry detection: curve based and surface based. Curve based algorithm relies on extracting and matching surface normal edge curves in the pointcloud. A more efficient surface based algorithm works by fitting symmetry axes/planes to the geometry of the smooth surfaces of the scene. In order to segment the objects we introduce a segmentation approach that uses symmetry as a global grouping principle. It extracts points of the scene that are consistent with a given symmetry candidate. To evaluate the performance of our symmetry detection and segmentation algorithms we construct a dataset of cluttered tabletop scenes with ground truth object masks and corresponding symmetries. Finally we demonstrate how our pipeline can be used by a mobile robot to detect and grasp objects in a house scenario

PRS-Net: planar reflective symmetry detection net for 3D models

In geometry processing, symmetry is a universal type of high-level structural information of 3D models and benefits many geometry processing tasks including shape segmentation, alignment, matching, and completion. Thus it is an important problem to analyze various symmetry forms of 3D shapes. Planar reflective symmetry is the most fundamental one. Traditional methods based on spatial sampling can be time-consuming and may not be able to identify all the symmetry planes. In this paper, we present a novel learning framework to automatically discover global planar reflective symmetry of a 3D shape. Our framework trains an unsupervised 3D convolutional neural network to extract global model features and then outputs possible global symmetry parameters, where input shapes are represented using voxels. We introduce a dedicated symmetry distance loss along with a regularization loss to avoid generating duplicated symmetry planes. Our network can also identify generalized cylinders by predicting their rotation axes. We further provide a method to remove invalid and duplicated planes and axes. We demonstrate that our method is able to produce reliable and accurate results. Our neural network based method is hundreds of times faster than the state-of-the-art methods, which are based on sampling. Our method is also robust even with noisy or incomplete input surfaces

Identification of metallic objects using spectral magnetic polarizability tensor signatures: Object characterisation and invariants

The early detection of terrorist threat objects, such as guns and knives, through improved metal detection, has the potential to reduce the number of attacks and improve public safety and security. To achieve this, there is considerable potential to use the fields applied and measured by a metal detector to discriminate between different shapes and different metals since, hidden within the field perturbation, is object characterisation information. The magnetic polarizability tensor (MPT) offers an economical characterisation of metallic objects that can be computed for different threat and non-threat objects and has an established theoretical background, which shows that the induced voltage is a function of the hidden object's MPT coefficients. In this article, we describe the additional characterisation information that measurements of the induced voltage over a range of frequencies offer compared with measurements at a single frequency. We call such object characterisations its MPT spectral signature. Then, we present a series of alternative rotational invariants for the purpose of classifying hidden objects using MPT spectral signatures. Finally, we include examples of computed MPT spectral signature characterisations of realistic threat and non-threat objects that can be used to train machine learning algorithms for classification purposes

Image processing for plastic surgery planning

This thesis presents some image processing tools for plastic surgery planning. In particular, it presents a novel method that combines local and global context in a probabilistic relaxation framework to identify cephalometric landmarks used in Maxillofacial plastic surgery. It also uses a method that utilises global and local symmetry to identify abnormalities in CT frontal images of the human body. The proposed methodologies are evaluated with the help of several clinical data supplied by collaborating plastic surgeons

Symmetry for face analysis.

Yuan Tianqiang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 51-55).Abstracts in English and Chinese.abstract --- p.iacknowledgments --- p.ivtable of contents --- p.vlist of figures --- p.viilist of tables --- p.ixChapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Reflectional Symmetry Detection --- p.1Chapter 1.2 --- Research Progress on Face Analysis --- p.2Chapter 1.2.1 --- Face Detection --- p.3Chapter 1.2.2 --- Face Alignment --- p.4Chapter 1.2.3 --- Face Recognition --- p.6Chapter 1.3 --- Organization of this thesis --- p.8Chapter Chapter 2 --- Local reflectional symmetry detection --- p.9Chapter 2.1 --- Proposed Method --- p.9Chapter 2.1.1 --- Symmetry measurement operator --- p.9Chapter 2.1.2 --- Potential regions selection --- p.10Chapter 2.1.3 --- Detection of symmetry axes --- p.11Chapter 2.2 --- Experiments --- p.13Chapter 2.2.1 --- Parameter setting and analysis --- p.13Chapter 2.2.2 --- Experimental Results --- p.14Chapter Chapter 3 --- Global perspective reflectional symmetry detection --- p.16Chapter 3.1 --- Introduction of camera models --- p.16Chapter 3.2 --- Property of Symmetric Point-Pair --- p.18Chapter 3.3 --- analysis and Experiment --- p.20Chapter 3.3.1 --- Confirmative Experiments --- p.20Chapter 3.3.2 --- Face shape generation with PSI --- p.22Chapter 3.3.3 --- Error Analysis --- p.24Chapter 3.3.4 --- Experiments of Pose Estimation --- p.25Chapter 3.4 --- Summary --- p.28Chapter Chapter 4 --- Pre-processing of face analysis --- p.30Chapter 4.1 --- Introduction of Hough Transform --- p.30Chapter 4.2 --- Eye Detection --- p.31Chapter 4.2.1 --- Coarse Detection --- p.32Chapter 4.2.2 --- Refine the eyes positions --- p.34Chapter 4.2.3 --- Experiments and Analysis --- p.35Chapter 4.3 --- Face Components Detection with GHT --- p.37Chapter 4.3.1 --- Parameter Analyses --- p.38Chapter 4 3.2 --- R-table Construction --- p.38Chapter 4.3.3 --- Detection Procedure and Voting Strategy --- p.39Chapter 4.3.4 --- Experiments and Analysis --- p.41Chapter Chapter 5 --- Pose estimation with face symmetry --- p.45Chapter 5.1 --- Key points selection --- p.45Chapter 5.2 --- Face Pose Estimation --- p.46Chapter 5.2.1 --- Locating eye corners --- p.46Chapter 5.2.2 --- Analysis and Summary --- p.47Chapter Chapter 6 --- Conclusions and future work --- p.49bibliography --- p.5