Estimating posture-recognition performance in sensing garments using geometric wrinkle modeling

A fundamental challenge limiting information quality obtained from smart sensing garments is the influence of textile movement relative to limbs. We present and validate a comprehensive modeling and simulation framework to predict recognition performance in casual loose-fitting garments. A statistical posture and wrinkle-modeling approach is introduced to simulate sensor orientation errors pertained to local garment wrinkles. A metric was derived to assess fitting, the body-garment mobility. We validated our approach by analyzing simulations of shoulder and elbow rehabilitation postures with respect to experimental data using actual casual garments. Results confirmed congruent performance trends with estimation errors below 4% for all study participants. Our approach allows to estimate the impact of fitting before implementing a garment and performing evaluation studies with it. These simulations revealed critical design parameters for garment prototyping, related to performed body posture, utilized sensing modalities, and garment fitting. We concluded that our modeling approach can substantially expedite design and development of smart garments through early-stage performance analysis

Bimanual Interaction with Clothes. Topology, Geometry, and Policy Representations in Robots

Twardon L. Bimanual Interaction with Clothes. Topology, Geometry, and Policy Representations in Robots. Bielefeld: Universität Bielefeld; 2019.If anthropomorphic robots are to assist people with activities of daily living, they must be able to handle all kinds of everyday objects, including highly deformable ones such as garments. The present thesis begins with a detailed problem analysis of robotic interaction with and perception of clothes. We show that handling items of clothing is very challenging due to their complex dynamics and the vast number of degrees of freedom. As a result of our analysis, we obtain a topological, geometric, and functional description of garments that supports the development of reduced object and task representations. One of the key findings is that the boundary components, which typically correspond with the openings, characterize garments well, both in terms of their topology and their inherent purpose, namely dressing. We present a polygon-based and an interactive method for identifying boundary components using RGB-D vision with application to grasping. Moreover, we propose Active Boundary Component Models (ABCMs), a constraint-based framework for tracking garment openings with point clouds. It is often difficult to maintain an accurate representation of the objects involved in contact-rich interaction tasks such as dressing assistance. Therefore, our policy optimization approach to putting a knit cap on a styrofoam head avoids modeling the details of the garment and its deformations. The experimental results suggest that a heuristic performance measure that takes into account the amount of contact established between the two objects is suitable for the task

Data-driven robotic manipulation of cloth-like deformable objects : the present, challenges and future prospects

Manipulating cloth-like deformable objects (CDOs) is a long-standing problem in the robotics community. CDOs are flexible (non-rigid) objects that do not show a detectable level of compression strength while two points on the article are pushed towards each other and include objects such as ropes (1D), fabrics (2D) and bags (3D). In general, CDOs’ many degrees of freedom (DoF) introduce severe self-occlusion and complex state–action dynamics as significant obstacles to perception and manipulation systems. These challenges exacerbate existing issues of modern robotic control methods such as imitation learning (IL) and reinforcement learning (RL). This review focuses on the application details of data-driven control methods on four major task families in this domain: cloth shaping, knot tying/untying, dressing and bag manipulation. Furthermore, we identify specific inductive biases in these four domains that present challenges for more general IL and RL algorithms.Publisher PDFPeer reviewe

Measuring joint movement through garment-integrated wearable sensing

University of Minnesota Ph.D. dissertation. April 2015. Major: Computer Science. Advisor: Lucy Dunne. 1 computer file (PDF); xv, 154 pages.Wearable technology is generally interpreted as electronic devices with passive and/or active electronic components worn on the human body. A further sub-set of wearable technology includes devices that are equipped with sensing abilities for body movements or biosignals and computational power that allows for further analysis. Wearable devices can be distinguished by different levels of wearability: wearable devices integrated into clothing, which are an integral part of the clothes; and wearable devices put on as an accessory. This thesis introduces a novel approach to truly wearable sensing of body movement through novel garment-integrated sensors. It starts from an initial investigation of garment movement in order to quantify the effect that garment movement has on sensor accuracy in garment-integrated sensors; continues with the development and detailed characterization of garment-integrated sensors that use a stitched technique to create comfortable, soft sensors capable of sensing stretch and bend; and ends with a final evaluation of the proposed wearable solution for the specific case of knee joint monitoring in both the stretch and bend modalities

Semantics for virtual humans

Population of Virtual Worlds with Virtual Humans is increasing rapidly by people who want to create a virtual life parallel to the real one (i.e. Second Life). The evolution of technology is smoothly providing the necessary elements to increase realism within these virtual worlds by creating believable Virtual Humans. However, creating the amount of resources needed to succeed this believability is a difficult task, mainly because of the complexity of the creation process of Virtual Humans. Even though there are many existing available resources, their reusability is difficult because there is not enough information provided to evaluate if a model contains the desired characteristics to be reused. Additionally, the knowledge involved in the creation of Virtual Humans is not well known, nor well disseminated. There are several different creation techniques, different software components, and several processes to carry out before having a Virtual Human capable of populating a virtual environment. The creation of Virtual Humans involves: a geometrical representation with an internal control structure, the motion synthesis with different animation techniques, higher level controllers and descriptors to simulate human-like behavior such individuality, cognition, interaction capabilities, etc. All these processes require the expertise from different fields of knowledge such as mathematics, artificial intelligence, computer graphics, design, etc. Furthermore, there is neither common framework nor common understanding of how elements involved in the creation, development, and interaction of Virtual Humans features are done. Therefore, there is a need for describing (1) existing resources, (2) Virtual Human's composition and features, (3) a creation pipeline and (4) the different levels/fields of knowledge comprehended. This thesis presents an explicit representation of the Virtual Humans and their features to provide a conceptual framework that will interest to all people involved in the creation and development of these characters. This dissertation focuses in a semantic description of Virtual Humans. The creation of a semantic description involves gathering related knowledge, agreement among experts in the definition of concepts, validation of the ontology design, etc. In this dissertation all these procedures are presented, and an Ontology for Virtual Humans is described in detail together with the validations that conducted to the resulted ontology. The goal of creating such ontology is to promote reusability of existing resources; to create a shared knowledge of the creation and composition of Virtual Humans; and to support new research of the fields involved in the development of believable Virtual Humans and virtual environments. Finally, this thesis presents several developments that aim to demonstrate the ontology usability and reusability. These developments serve particularly to support the research on specialized knowledge of Virtual Humans, the population of virtual environments, and improve the believability of these characters

Characterizing the Influence of the Textile-Sensor Interface on Stitched Sensor Performance

University of Minnesota M.S.E.E. thesis. July 2019. Major: Electrical/Computer Engineering. Advisors: Lucy Dunne, Sarah Swisher. 1 computer file (PDF); vii, 155 pages.Textile-based strain sensors are first defined with examples of various sensing mechanisms and applications, focusing on on-body smart garments for biomonitoring. A current lack of research in the textile substrate influence on sensor performance is noted, with a thesis investigation outlined to highlight key variables that may be important for successful sensor design. Two conductive thread stitch-based strain sensors are chosen for the textile-based strain sensors and two fabric substrates (2-way and 4-way stretch) are used to investigate their influence on sensor performance. Part 1 investigates if fabric strain properties change due to the attachment of sensors and how the sensor performance changes due to fabric choice and attachment angle. Part 2 uses the recommendations for textile choice, stitch geometry of the sensor, and sensor placement based on Part 1 results to create a 3-sensor, 60° strain rosette. Between the two versions of rosettes fabricated, the 4-way fabric and chainstitch geometry, the strain rosette is proven to improve the overall sensor performance in predicting force, displacement, and force direction. This rosette is characterized and using machine learning model algorithms, model-fitted for future garment based strain sensing applications

3D human body modelling from range data

This thesis describes the design, implementation and application of an integrated and fully automated system for interpreting whole-body range data. The system is shown to be capable of generating complete surface models of human bodies, and robustly extracting anatomical features for anthropometry, with minimal intrusion on the subject. The ability to automate this process has enormous potential for personalised digital models in medicine, ergonomics, design and manufacture and for populating virtual environments. The techniques developed within this thesis now form the basis of a commercial product. However, the technical difficulties are considerable. Human bodies are highly varied and many of the features of interest are extremely subtle. The underlying range data is typically noisy and is sparse at occluded areas. In addressing these problems this thesis makes five main research contributions. Firstly, the thesis describes the design, implementation and testing of the whole integrated and automated system from scratch, starting at the image capture hardware. At each stage the tradeoffs between performance criteria are discussed, and experiments are described to test the processes developed. Secondly, a combined data-driven and model-based approach is described and implemented, for surface reconstruction from the raw data. This method addresses the whole body surface, including areas where body segments touch, and other occluded areas. The third contribution is a library of operators, designed specifically for shape description and measurement of the human body. The library provides high-level relational attributes, an "electronic tape measure" to extract linear and curvilinear measurements,as well as low-level shape information, such as curvature. Application of the library is demonstrated by building a large set of detectors to find anthropometric features, based on the ISO 8559 specification. Output is compared against traditional manual measurements and a detailed analysis is presented. The discrepancy between these sets of data is only a few per cent on most dimensions, and the system's reproducibility is shown to be similar to that of skilled manual measurers. The final contribution is that the mesh models and anthropometric features, produced by the system, have been used as a starting point to facilitate other research, Such as registration of multiple body images,draping clothing and advanced surface modelling techniques

Rekonstruktion, Analyse und Editierung dynamisch deformierter 3D-Oberflächen

Dynamically deforming 3D surfaces play a major role in computer graphics. However, producing time-varying dynamic geometry at ever increasing detail is a time-consuming and costly process, and so a recent trend is to capture geometry data directly from the real world. In the first part of this thesis, I propose novel approaches for this research area. These approaches capture dense dynamic 3D surfaces from multi-camera systems in a particularly robust and accurate way. This provides highly realistic dynamic surface models for phenomena like moving garments and bulging muscles. However, re-using, editing, or otherwise analyzing dynamic 3D surface data is not yet conveniently possible. To close this gap, the second part of this dissertation develops novel data-driven modeling and animation approaches. I first show a supervised data-driven approach for modeling human muscle deformations that scales to huge datasets and provides fine-scale, anatomically realistic deformations at high quality not attainable by previous methods. I then extend data-driven modeling to the unsupervised case, providing editing tools for a wider set of input data ranging from facial performance capture and full-body motion to muscle and cloth deformation. To this end, I introduce the concepts of sparsity and locality within a mathematical optimization framework. I also explore these concepts for constructing shape-aware functions that are useful for static geometry processing, registration, and localized editing.Dynamisch deformierbare 3D-Oberflächen spielen in der Computergrafik eine zentrale Rolle. Die Erstellung der für Computergrafik-Anwendungen benötigten, hochaufgelösten und zeitlich veränderlichen Oberflächengeometrien ist allerdings äußerst arbeitsintensiv. Aus dieser Problematik heraus hat sich der Trend entwickelt, Oberflächendaten direkt aus Aufnahmen der echten Welt zu erfassen. Dazu nötige 3D-Rekonstruktionsverfahren werden im ersten Teil der Arbeit entwickelt. Die vorgestellten, neuartigen Verfahren erlauben die Erfassung dynamischer 3D-Oberflächen aus Mehrkamera-Aufnahmen bei hoher Verlässlichkeit und Präzision. Auf diese Weise können detaillierte Oberflächenmodelle von Phänomenen wie in Bewegung befindliche Kleidung oder sich anspannende Muskeln erfasst werden. Aber auch die Wiederverwendung, Bearbeitung und Analyse derlei gewonnener 3D-Oberflächendaten ist aktuell noch nicht auf eine einfache Art und Weise möglich. Um diese Lücke zu schließen beschäftigt sich der zweite Teil der Arbeit mit der datengetriebenen Modellierung und Animation. Zunächst wird ein Ansatz für das überwachte Lernen menschlicher Muskel-Deformationen vorgestellt. Dieses neuartige Verfahren ermöglicht eine datengetriebene Modellierung mit besonders umfangreichen Datensätzen und liefert anatomisch-realistische Deformationseffekte. Es übertrifft damit die Genauigkeit früherer Methoden. Im nächsten Teil beschäftigt sich die Dissertation mit dem unüberwachten Lernen aus 3D-Oberflächendaten. Es werden neuartige Werkzeuge vorgestellt, die eine weitreichende Menge an Eingabedaten verarbeiten können, von aufgenommenen Gesichtsanimationen über Ganzkörperbewegungen bis hin zu Muskel- und Kleidungsdeformationen. Um diese Anwendungsbreite zu erreichen stützt sich die Arbeit auf die allgemeinen Konzepte der Spärlichkeit und Lokalität und bettet diese in einen mathematischen Optimierungsansatz ein. Abschließend zeigt die vorliegende Arbeit, wie diese Konzepte auch für die Konstruktion von oberflächen-adaptiven Basisfunktionen übertragen werden können. Dadurch können Anwendungen für die Verarbeitung, Registrierung und Bearbeitung statischer Oberflächenmodelle erschlossen werden