MonoPerfCap: Human Performance Capture from Monocular Video

We present the first marker-less approach for temporally coherent 3D performance capture of a human with general clothing from monocular video. Our approach reconstructs articulated human skeleton motion as well as medium-scale non-rigid surface deformations in general scenes. Human performance capture is a challenging problem due to the large range of articulation, potentially fast motion, and considerable non-rigid deformations, even from multi-view data. Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem. We tackle these challenges by a novel approach that employs sparse 2D and 3D human pose detections from a convolutional neural network using a batch-based pose estimation strategy. Joint recovery of per-batch motion allows to resolve the ambiguities of the monocular reconstruction problem based on a low dimensional trajectory subspace. In addition, we propose refinement of the surface geometry based on fully automatically extracted silhouettes to enable medium-scale non-rigid alignment. We demonstrate state-of-the-art performance capture results that enable exciting applications such as video editing and free viewpoint video, previously infeasible from monocular video. Our qualitative and quantitative evaluation demonstrates that our approach significantly outperforms previous monocular methods in terms of accuracy, robustness and scene complexity that can be handled.Comment: Accepted to ACM TOG 2018, to be presented on SIGGRAPH 201

Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification

Re-identification is generally carried out by encoding the appearance of a subject in terms of outfit, suggesting scenarios where people do not change their attire. In this paper we overcome this restriction, by proposing a framework based on a deep convolutional neural network, SOMAnet, that additionally models other discriminative aspects, namely, structural attributes of the human figure (e.g. height, obesity, gender). Our method is unique in many respects. First, SOMAnet is based on the Inception architecture, departing from the usual siamese framework. This spares expensive data preparation (pairing images across cameras) and allows the understanding of what the network learned. Second, and most notably, the training data consists of a synthetic 100K instance dataset, SOMAset, created by photorealistic human body generation software. Synthetic data represents a good compromise between realistic imagery, usually not required in re-identification since surveillance cameras capture low-resolution silhouettes, and complete control of the samples, which is useful in order to customize the data w.r.t. the surveillance scenario at-hand, e.g. ethnicity. SOMAnet, trained on SOMAset and fine-tuned on recent re-identification benchmarks, outperforms all competitors, matching subjects even with different apparel. The combination of synthetic data with Inception architectures opens up new research avenues in re-identification.Comment: 14 page

Cloaca Palace

In this thesis, I trace the compulsive fear of holes, known as trypophobia, from an uncontrollable obsession to a pleasurable preoccupation. The body’s physical porousness makes us receptive to our surroundings, allowing external matter in and destabilizing the boundaries of self and other. Matter invades us, encoding itself into our DNA and transforming humans into chimeric creatures. Through paintings and multi-media installations, I encourage viewers to reflect on their own bodies as a series of holes, vulnerable receptors to the world. I use the figure of a woman to personify a human hole which has been infected by the outside, giving her the power to infect others. I employ Luce Irigaray’s formation of mimicry to hyperbolize tropes of feminine performance, building a world where being hole is both offensive and defensive camouflage

Estimating Symmetry/Asymmetry in the Human Torso: A Novel Computational Method

Asymmetry in human body has largely been based on bilateral traits and/or subjective estimates, with potential usage in fields such as medicine, rehabilitation and apparel product design. In case of apparel, asymmetry in human body has been measured primarily by estimating differential linear measurement of bilateral traits. However, the characteristics of asymmetry can be better understood and be useful for clinicians and designers if it is quantified by considering the whole 3D surface. To address the prevailing issues in measuring asymmetry objectively, this research attempts to develop a novel method to quantify asymmetry that is robust, effective and non-invasive in operation. The method discussed here uses 3D scans of human torso to estimate asymmetry as a numerical index. Furthermore, using skeletal landmarks, twist and tilt measurements of the torsos are computed numerically. Together, these three measures can characterize the asymmetric/symmetric nature of a human torso. The approach taken in this research uses cross sections of torso to estimate local plane of symmetry that equi-divides a given cross section on the basis of its area, and connecting those planes to form a global surface that divides the torso volumetrically. The computational approach in estimating the area of cross section is based on the Green's theorem. The developed method was validated by both testing it on a known geometric model and by comparing the estimated index with subjective ratings by experts. This method has potential applications in various fields requiring characterizing asymmetry i.e., in case of scoliosis patients as diagnostic tool or an evaluation metric for rehabilitation efficiency, for body builders, and fashion models as an evaluation tool.Design, Housing and Merchandisin

Real time physics-based augmented fitting room using time-of-flight cameras

Ankara : The Department of Computer Engineering and the Graduate School of Engineering and Science of Bilkent University, 2013.Thesis (Master's) -- Bilkent University, 2013.Includes bibliographical references leaves 63-72.This thesis proposes a framework for a real-time physically-based augmented cloth tting environment. The required 3D meshes for the human avatar and apparels are modeled with speci c constraints. The models are then animated in real-time using input from a user tracked by a depth sensor. A set of motion lters are introduced in order to improve the quality of the simulation. The physical e ects such as inertia, external and forces and collision are imposed on the apparel meshes. The avatar and the apparels can be customized according to the user. The system runs in real-time on a high-end consumer PC with realistic rendering results.Gültepe, UmutM.S

Template based shape processing

As computers can only represent and process discrete data, information gathered from the real world always has to be sampled. While it is nowadays possible to sample many signals accurately and thus generate high-quality reconstructions (for example of images and audio data), accurately and densely sampling 3D geometry is still a challenge. The signal samples may be corrupted by noise and outliers, and contain large holes due to occlusions. These issues become even more pronounced when also considering the temporal domain. Because of this, developing methods for accurate reconstruction of shapes from a sparse set of discrete data is an important aspect of the computer graphics processing pipeline. In this thesis we propose novel approaches to including semantic knowledge into reconstruction processes using template based shape processing. We formulate shape reconstruction as a deformable template fitting process, where we try to fit a given template model to the sampled data. This approach allows us to present novel solutions to several fundamental problems in the area of shape reconstruction. We address static problems like constrained texture mapping and semantically meaningful hole-filling in surface reconstruction from 3D scans, temporal problems such as mesh based performance capture, and finally dynamic problems like the estimation of physically based material parameters of animated templates.Analoge Signale müssen digitalisiert werden um sie auf modernen Computern speichern und verarbeiten zu können. Für viele Signale, wie zum Beispiel Bilder oder Tondaten, existieren heutzutage effektive und effiziente Digitalisierungstechniken. Aus den so gewonnenen Daten können die ursprünglichen Signale hinreichend akkurat wiederhergestellt werden. Im Gegensatz dazu stellt das präzise und effiziente Digitalisieren und Rekonstruieren von 3D- oder gar 4D-Geometrie immer noch eine Herausforderung dar. So führen Verdeckungen und Fehler während der Digitalisierung zu Löchern und verrauschten Meßdaten. Die Erforschung von akkuraten Rekonstruktionsmethoden für diese groben digitalen Daten ist daher ein entscheidender Schritt in der Entwicklung moderner Verarbeitungsmethoden in der Computergrafik. In dieser Dissertation wird veranschaulicht, wie deformierbare geometrische Modelle als Vorlage genutzt werden können, um semantische Informationen in die robuste Rekonstruktion von 3D- und 4D Geometrie einfließen zu lassen. Dadurch wird es möglich, neue Lösungsansätze für mehrere grundlegenden Probleme der Computergrafik zu entwickeln. So können mit dieser Technik Löcher in digitalisierten 3D Modellen semantisch sinnvoll aufgefüllt, oder detailgetreue virtuelle Kopien von Darstellern und ihrer dynamischen Kleidung zu erzeugt werden

MONOCULAR POSE ESTIMATION AND SHAPE RECONSTRUCTION OF QUASI-ARTICULATED OBJECTS WITH CONSUMER DEPTH CAMERA

Quasi-articulated objects, such as human beings, are among the most commonly seen objects in our daily lives. Extensive research have been dedicated to 3D shape reconstruction and motion analysis for this type of objects for decades. A major motivation is their wide applications, such as in entertainment, surveillance and health care. Most of existing studies relied on one or more regular video cameras. In recent years, commodity depth sensors have become more and more widely available. The geometric measurements delivered by the depth sensors provide significantly valuable information for these tasks. In this dissertation, we propose three algorithms for monocular pose estimation and shape reconstruction of quasi-articulated objects using a single commodity depth sensor. These three algorithms achieve shape reconstruction with increasing levels of granularity and personalization. We then further develop a method for highly detailed shape reconstruction based on our pose estimation techniques. Our first algorithm takes advantage of a motion database acquired with an active marker-based motion capture system. This method combines pose detection through nearest neighbor search with pose refinement via non-rigid point cloud registration. It is capable of accommodating different body sizes and achieves more than twice higher accuracy compared to a previous state of the art on a publicly available dataset. The above algorithm performs frame by frame estimation and therefore is less prone to tracking failure. Nonetheless, it does not guarantee temporal consistent of the both the skeletal structure and the shape and could be problematic for some applications. To address this problem, we develop a real-time model-based approach for quasi-articulated pose and 3D shape estimation based on Iterative Closest Point (ICP) principal with several novel constraints that are critical for monocular scenario. In this algorithm, we further propose a novel method for automatic body size estimation that enables its capability to accommodate different subjects. Due to the local search nature, the ICP-based method could be trapped to local minima in the case of some complex and fast motions. To address this issue, we explore the potential of using statistical model for soft point correspondences association. Towards this end, we propose a unified framework based on Gaussian Mixture Model for joint pose and shape estimation of quasi-articulated objects. This method achieves state-of-the-art performance on various publicly available datasets. Based on our pose estimation techniques, we then develop a novel framework that achieves highly detailed shape reconstruction by only requiring the user to move naturally in front of a single depth sensor. Our experiments demonstrate reconstructed shapes with rich geometric details for various subjects with different apparels. Last but not the least, we explore the applicability of our method on two real-world applications. First of all, we combine our ICP-base method with cloth simulation techniques for Virtual Try-on. Our system delivers the first promising 3D-based virtual clothing system. Secondly, we explore the possibility to extend our pose estimation algorithms to assist physical therapist to identify their patients’ movement dysfunctions that are related to injuries. Our preliminary experiments have demonstrated promising results by comparison with the gold standard active marker-based commercial system. Throughout the dissertation, we develop various state-of-the-art algorithms for pose estimation and shape reconstruction of quasi-articulated objects by leveraging the geometric information from depth sensors. We also demonstrate their great potentials for different real-world applications

Variability in body size and shape of UK offshore workers: a cluster analysis approach.

Male UK offshore workers have enlarged dimensions compared with UK norms and knowledge of specific sizes and shapes typifying their physiques will assist a range of functions related to health and ergonomics. A representative sample of the UK offshore workforce (n=588) underwent 3D photonic scanning, from which 19 extracted dimensional measures were used in k-means cluster analysis to characterise physique groups. Of the 11 resulting clusters four somatotype groups were expressed: one cluster was muscular and lean, four had greater muscularity than adiposity, three had equal adiposity and muscularity and three had greater adiposity than muscularity. Some clusters appeared constitutionally similar to others, differing only in absolute size. These cluster centroids represent an evidence-base for future designs in apparel and other applications where body size and proportions affect functional performance. They also constitute phenotypic evidence providing insight into the ‘offshore culture’ which may underpin the enlarged dimensions of offshore workers

Supply chain network considerations for e-retail of luxury goods in Canada

Thesis (M. Eng. in Logistics)--Massachusetts Institute of Technology, Engineering Systems Division, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 85-89).The Internet has changed the way people purchase goods in the 21st century: many types of goods and services have become available online. Luxury goods followed this trend after an initial delay, primarily due to the nature of these goods. At the time of the preparation of this document, there were no proven guidelines for building the most successful e-retail website for luxury goods from the brand management and profitability perspectives. Ralph Lauren (RL) is an established American brand, well known for quality and consistent style in the following categories: clothing for women, men, and children; home goods, accessories; and fragrances. RL Corporation houses many labels that constitute premium and luxury offerings. RL currently sells through the Internet in many countries, in addition to countless company owned stores, 9 flagship stores, department stores and boutiques distributed around the world. To continue growth, RL wants to launch an e-retail website for Canada. This thesis aims to provide supply chain network considerations for the successful operation of the Canadian e-retail website for RL. These considerations stem from a careful look into potential luxury website characteristics that would meet the company objective of elevating the brand towards the luxury category. It is recommended that RL secure expansion capacity that will likely be necessary for B2C operation at its Toronto distribution center (DC). In addition, material handling equipment that will process a high volume of small orders should be placed in this DC. The Vancouver cross-docking facility could be expanded in the future as prompted by sales volume and coupled with a DC to cater to the West Coast of Canada. Also, it is recommended that advanced customer tracking systems and databases be employed, especially to determine high value customers for tailored offerings in the luxury segment.by Dilek Tansoy and Yi Linn Teo.M.Eng.in Logistic