Feedback-based Fabric Strip Folding

Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.Comment: Submitted to IEEE/RSJ IROS201

Deep Learning of Force Manifolds from the Simulated Physics of Robotic Paper Folding

Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, such as imitation learning, have been used to address deformable material manipulation. These approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g., friction) properties. This article tackles a fundamental but difficult deformable manipulation task: forming a predefined fold in paper with only a single manipulator. A data-driven framework combining physically-accurate simulation and machine learning is used to train a deep neural network capable of predicting the external forces induced on the manipulated paper given a grasp position. We frame the problem using scaling analysis, resulting in a control framework robust against material and geometric changes. Path planning is then carried out over the generated "neural force manifold" to produce robot manipulation trajectories optimized to prevent sliding, with offline trajectory generation finishing 15$\times$ faster than previous physics-based folding methods. The inference speed of the trained model enables the incorporation of real-time visual feedback to achieve closed-loop sensorimotor control. Real-world experiments demonstrate that our framework can greatly improve robotic manipulation performance compared to state-of-the-art folding strategies, even when manipulating paper objects of various materials and shapes.Comment: Supplementary video is available on YouTube: https://youtu.be/k0nexYGy-P

Robotic system for garment perception and manipulation

Mención Internacional en el título de doctorGarments are a key element of people’s daily lives, as many domestic tasks -such as laundry-, revolve around them. Performing such tasks, generally dull and repetitive, implies devoting many hours of unpaid labor to them, that could be freed through automation. But automation of such tasks has been traditionally hard due to the deformable nature of garments, that creates additional challenges to the already existing when performing object perception and manipulation. This thesis presents a Robotic System for Garment Perception and Manipulation that intends to address these challenges. The laundry pipeline as defined in this work is composed by four independent -but sequential- tasks: hanging, unfolding, ironing and folding. The aim of this work is the automation of this pipeline through a robotic system able to work on domestic environments as a robot household companion. Laundry starts by washing the garments, that then need to be dried, frequently by hanging them. As hanging is a complex task requiring bimanipulation skills and dexterity, a simplified approach is followed in this work as a starting point, by using a deep convolutional neural network and a custom synthetic dataset to study if a robot can predict whether a garment will hang or not when dropped over a hanger, as a first step towards a more complex controller. After the garment is dry, it has to be unfolded to ease recognition of its garment category for the next steps. The presented model-less unfolding method uses only color and depth information from the garment to determine the grasp and release points of an unfolding action, that is repeated iteratively until the garment is fully spread. Before storage, wrinkles have to be removed from the garment. For that purpose, a novel ironing method is proposed, that uses a custom wrinkle descriptor to locate the most prominent wrinkles and generate a suitable ironing plan. The method does not require a precise control of the light conditions of the scene, and is able to iron using unmodified ironing tools through a force-feedback-based controller. Finally, the last step is to fold the garment to store it. One key aspect when folding is to perform the folding operation in a precise manner, as errors will accumulate when several folds are required. A neural folding controller is proposed that uses visual feedback of the current garment shape, extracted through a deep neural network trained with synthetic data, to accurately perform a fold. All the methods presented to solve each of the laundry pipeline tasks have been validated experimentally on different robotic platforms, including a full-body humanoid robot.La ropa es un elemento clave en la vida diaria de las personas, no sólo a la hora de vestir, sino debido también a que muchas de las tareas domésticas que una persona debe realizar diariamente, como hacer la colada, requieren interactuar con ellas. Estas tareas, a menudo tediosas y repetitivas, obligan a invertir una gran cantidad de horas de trabajo no remunerado en su realización, las cuales podrían reducirse a través de su automatización. Sin embargo, automatizar dichas tareas ha sido tradicionalmente un reto, debido a la naturaleza deformable de las prendas, que supone una dificultad añadida a las ya existentes al llevar a cabo percepción y manipulación de objetos a través de robots. Esta tesis presenta un sistema robótico orientado a la percepción y manipulación de prendas, que pretende resolver dichos retos. La colada es una tarea doméstica compuesta de varias subtareas que se llevan a cabo de manera secuencial. En este trabajo, se definen dichas subtareas como: tender, desdoblar, planchar y doblar. El objetivo de este trabajo es automatizar estas tareas a través de un sistema robótico capaz de trabajar en entornos domésticos, convirtiéndose en un asistente robótico doméstico. La colada comienza lavando las prendas, las cuales han de ser posteriormente secadas, generalmente tendiéndolas al aire libre, para poder realizar el resto de subtareas con ellas. Tender la ropa es una tarea compleja, que requiere de bimanipulación y una gran destreza al manipular la prenda. Por ello, en este trabajo se ha optado por abordar una versión simplicada de la tarea de tendido, como punto de partida para llevar a cabo investigaciones más avanzadas en el futuro. A través de una red neuronal convolucional profunda y un conjunto de datos de entrenamiento sintéticos, se ha llevado a cabo un estudio sobre la capacidad de predecir el resultado de dejar caer una prenda sobre un tendedero por parte de un robot. Este estudio, que sirve como primer paso hacia un controlador más avanzado, ha resultado en un modelo capaz de predecir si la prenda se quedará tendida o no a partir de una imagen de profundidad de la misma en la posición en la que se dejará caer. Una vez las prendas están secas, y para facilitar su reconocimiento por parte del robot de cara a realizar las siguientes tareas, la prenda debe ser desdoblada. El método propuesto en este trabajo para realizar el desdoble no requiere de un modelo previo de la prenda, y utiliza únicamente información de profundidad y color, obtenida mediante un sensor RGB-D, para calcular los puntos de agarre y soltado de una acción de desdoble. Este proceso es iterativo, y se repite hasta que la prenda se encuentra totalmente desdoblada. Antes de almacenar la prenda, se deben eliminar las posibles arrugas que hayan surgido en el proceso de lavado y secado. Para ello, se propone un nuevo algoritmo de planchado, que utiliza un descriptor de arrugas desarrollado en este trabajo para localizar las arrugas más prominentes y generar un plan de planchado acorde a las condiciones de la prenda. A diferencia de otros métodos existentes, este método puede aplicarse en un entorno doméstico, ya que no requiere de un contol preciso de las condiciones de iluminación. Además, es capaz de usar las mismas herramientas de planchado que usaría una persona sin necesidad de realizar modificaciones a las mismas, a través de un controlador que usa realimentación de fuerza para aplicar una presión constante durante el planchado. El último paso al hacer la colada es doblar la prenda para almacenarla. Un aspecto importante al doblar prendas es ejecutar cada uno de los dobleces necesarios con precisión, ya que cada error o desfase cometido en un doblez se acumula cuando la secuencia de doblado está formada por varios dobleces consecutivos. Para llevar a cabo estos dobleces con la precisión requerida, se propone un controlador basado en una red neuronal, que utiliza realimentación visual de la forma de la prenda durante cada operación de doblado. Esta realimentación es obtenida a través de una red neuronal profunda entrenada con un conjunto de entrenamiento sintético, que permite estimar la forma en 3D de la parte a doblar a través de una imagen monocular de la misma. Todos los métodos descritos en esta tesis han sido validados experimentalmente con éxito en diversas plataformas robóticas, incluyendo un robot humanoide.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Abderrahmane Kheddar.- Secretario: Ramón Ignacio Barber Castaño.- Vocal: Karinne Ramírez-Amar

Real-Time Numerical Simulation for Accurate Soft Tissues Modeling during Haptic Interaction

The simulation of fabrics physics and its interaction with the human body has been largely studied in recent years to provide realistic-looking garments and wears specifically in the entertainment business. When the purpose of the simulation is to obtain scientific measures and detailed mechanical properties of the interaction, the underlying physical models should be enhanced to obtain better simulation accuracy increasing the modeling complexity and relaxing the simulation timing constraints to properly solve the set of equations under analysis. However, in the specific field of haptic interaction, the desiderata are to have both physical consistency and high frame rate to display stable and coherent stimuli as feedback to the user requiring a tradeoff between accuracy and real-time interaction. This work introduces a haptic system for the evaluation of the fabric hand of specific garments either existing or yet to be produced in a virtual reality simulation. The modeling is based on the co-rotational Finite Element approach that allows for large displacements but the small deformation of the elements. The proposed system can be beneficial for the fabrics industry both in the design phase or in the presentation phase, where a virtual fabric portfolio can be shown to customers around the world. Results exhibit the feasibility of high-frequency real-time simulation for haptic interaction with virtual garments employing realistic mechanical properties of the fabric materials

Next generation mechanically deployable aero-decelerators for Mars entry

Current Mars entry vehicle technology is near its payload mass delivery limit. Mechanically deployable aero-decelerators are a next generation technology that would enable the future exploration of Mars, including human landing. Various concepts and architectures have been proposed over the years, with widely-varying mass assessments and limited technology development. A novel 6 degree-of-freedom entry trajectory simulator coupled with a structural model of the deployable elements, or ribs, has been developed and correlated against industry tools and flight data to investigate and optimise the design of mechanically deployable aero-decelerators. A major assumption of the simulator – that heatshield gores remain flat under rib deformation – has been investigated by testing ambient 3D woven carbon fabric for use as the flexible thermal protection system material, proving that conditioning can significantly improve the stiffness properties of the fabric. The design optimisation has revealed that, although deployable rib flexibility is beneficial in reducing mass and volume of the deployed ribs, an increase in peak heat flux will result. However, if mass savings from flexible ribs can be reallocated towards increasing the diameter of the entry vehicle, significant entry trajectory benefits can be gained. A set of general design principles for mechanically deployable aero-decelerators has been developed based on the optimisation investigations, including the recommendations to include at least 10 ribs to minimise drag reduction, and to increase the initial rib angle if rib flexibility is allowed to improve deceleration. In addition, the entry vehicle roll rate appears to be influenced by the number of deployable ribs. This roll instability of faceted entry vehicles is of significant concern, so a novel supersonic wind tunnel test methodology was developed to further investigate this hypothesis. The first experimental results imply that pitch and yaw attitude and instabilities are necessary pre-requisites to initiate roll during entry, and that an 8 rib test article rotates at faster rates than the 12 rib test article. Finally, a new functional relationship for the angular acceleration of entry vehicles has been proposed that it is hoped will inspire further investigations in this area.Open Acces

Adaptive Multi-Functional Space Systems for Micro-Climate Control

This report summarizes the work done during the Adaptive Multifunctional Systems for Microclimate Control Study held at the Caltech Keck Institute for Space Studies (KISS) in 2014-2015. Dr. Marco Quadrelli (JPL), Dr. James Lyke (AFRL), and Prof. Sergio Pellegrino (Caltech) led the Study, which included two workshops: the first in May of 2014, and another in February of 2015. The Final Report of the Study presented here describes the potential relevance of adaptive multifunctional systems for microclimate control to the missions outlined in the 2010 NRC Decadal Survey. The objective of the Study was to adapt the most recent advances in multifunctional reconfigurable and adaptive structures to enable a microenvironment control to support space exploration in extreme environments (EE). The technical goal was to identify the most efficient materials, architectures, structures and means of deployment/reconfiguration, system autonomy and energy management solutions needed to optimally project/generate a micro-environment around space assets. For example, compact packed thin-layer reflective structures unfolding to large areas can reflect solar energy, warming and illuminating assets such as exploration rovers on Mars or human habitats on the Moon. This novel solution is called an energy-projecting multifunctional system (EPMFS), which are composed of Multifunctional Systems (MFS) and Energy-Projecting Systems (EPS)

Design and Development of a Soft Robotic Gripper for Fabric Material Handling

Fabric and textile materials are widely used in many industrial applications, especially in automotive, aviation and consumer goods. Currently, there is no semi-automatic or automatic solution for rapid, effective, and reconfigurable pick and place activities for limp, air permeable flexible components in industry. The production of these light-weight flexible textile or composite fiber products highly rely on manual operations, which lead to high production costs, workplace safety issues, and process bottlenecks. As a bio-inspired novel technology, soft robotic grippers provide new opportunities for the automation of fabric handling tasks. In this research, the characteristics of fabric pick and place tasks using the clamping grippers are quantitatively investigated. Experiments on a carbon fiber fabric are performed with a collaborative robot to explore the damage, slippage, draping, and wrinkling during basic pick and place operations. Based on the experimental results, multiple soft robotic gripper configurations are developed, including a compliant glove set that can improve the performance of traditional rigid grippers, an elastomer-based soft gripper, and a linkage-based underactuated gripper. The gripper designs are analyzed and refined based on finite element simulation. Prototypes of the grippers are fabricated using a rapid tooling solution for an overmolding strategy to verify their functionality. Through the research, it is proven feasible to reliably perform flexible fabric handling operations using soft grippers with appropriate toolpath planning. Finite element simulation and additive manufacturing have shown to be useful tools during the gripper design and development procedure, and the methodologies developed and applied in this work should be expanded for more flexible material handling challenges