Sensing Highly Non-Rigid Objects with RGBD Sensors for Robotic Systems

The goal of this research is to enable a robotic system to manipulate clothing and other highly non-rigid objects using an RGBD sensor. The focus of this thesis is to define and test various algorithms / models that are used to solve parts of the laundry process (i.e. handling, classifying, sorting, unfolding, and folding). First, a system is presented for automatically extracting and classifying items in a pile of laundry. Using only visual sensors, the robot identifies and extracts items sequentially from the pile. When an item is removed and isolated, a model is captured of the shape and appearance of the object, which is then compared against a dataset of known items. The contributions of this part of the laundry process are a novel method for extracting articles of clothing from a pile of laundry, a novel method of classifying clothing using interactive perception, and a multi-layer approach termed L-M-H, more specifically L-C-S-H for clothing classification. This thesis describes two different approaches to classify clothing into categories. The first approach relies upon silhouettes, edges, and other low-level image measurements of the articles of clothing. Experiments from the first approach demonstrate the ability of the system to efficiently classify and label into one of six categories (pants, shorts, short-sleeve shirt, long-sleeve shirt, socks, or underwear). These results show that, on average, classification rates using robot interaction are 59% higher than those that do not use interaction. The second approach relies upon color, texture, shape, and edge information from 2D and 3D data within a local and global perspective. The multi-layer approach compartmentalizes the problem into a high (H) layer, multiple mid-level (characteristics(C), selection masks(S)) layers, and a low (L) layer. This approach produces \u27local\u27 solutions to solve the global classification problem. Experiments demonstrate the ability of the system to efficiently classify each article of clothing into one of seven categories (pants, shorts, shirts, socks, dresses, cloths, or jackets). The results presented in this paper show that, on average, the classification rates improve by +27.47% for three categories, +17.90% for four categories, and +10.35% for seven categories over the baseline system, using support vector machines. Second, an algorithm is presented for automatically unfolding a piece of clothing. A piece of cloth is pulled in different directions at various points of the cloth in order to flatten the cloth. The features of the cloth are extracted and calculated to determine a valid location and orientation in which to interact with it. The features include the peak region, corner locations, and continuity / discontinuity of the cloth. In this thesis, a two-stage algorithm is presented, introducing a novel solution to the unfolding / flattening problem using interactive perception. Simulations using 3D simulation software, and experiments with robot hardware demonstrate the ability of the algorithm to flatten pieces of laundry using different starting configurations. These results show that, at most, the algorithm flattens out a piece of cloth from 11.1% to 95.6% of the canonical configuration. Third, an energy minimization algorithm is presented that is designed to estimate the configuration of a deformable object. This approach utilizes an RGBD image to calculate feature correspondence (using SURF features), depth values, and boundary locations. Input from a Kinect sensor is used to segment the deformable surface from the background using an alpha-beta swap algorithm. Using this segmentation, the system creates an initial mesh model without prior information of the surface geometry, and it reinitializes the configuration of the mesh model after a loss of input data. This approach is able to handle in-plane rotation, out-of-plane rotation, and varying changes in translation and scale. Results display the proposed algorithm over a dataset consisting of seven shirts, two pairs of shorts, two posters, and a pair of pants. The current approach is compared using a simulated shirt model in order to calculate the mean square error of the distance from the vertices on the mesh model to the ground truth, provided by the simulation model

Komparasi Sistem Komunikasi Serial Multipoint Pada Robot Management Sampah Menggunakan I2C Dan SPI

Sampah merupakan masalah yang sering kita jumpai dalam kehidupan sehari-hari. Hal ini perlu diatasi karena setiap manusia pasti memproduksi sampah, disisi lain masyarakat tidak ingin berdekatan dengan sampah. Oleh karena itu dalam makalah ini penulis akan membahas bagian dari robot management sampah.Pada robot ini menggunakan 4 buah mikrokontroler sehingga diperlukan protokol komunikasi serial multipoint. Komunikasi ini harus dirancang dengan baik agar tidak terjadi kesalahan dalam pengiriman maupun penerimaan data serta kecepatan transfer data dapat maksimal sehingga informasi dari sensor selalu terbaca oleh robot.Empat buah mikrokontroler tersebut terdiri dari 1 Master dan 3 Slave. Fungsi masing-masing dari 3 mikrokontroler slave tersebut adalah untuk membaca sensor, mengendalikan motor kiri, dan mengendalikan motor kanan. Sedangkan fungsi dari mikrokontroler master adalah sebagai pemroses data dari slave pembaca sensor dan hasilnya di kirim ke slave pengendali motor kanan dan slave pengendali motor kiri. Dengan sistem seperti ini diharapkan waktu pemrosesan lebih cepat dibandingkan hanya menggunakan 1 mikrokontroler.Kata Kunci— Robot, Mikrokontroler, Serial, Multipoin

Komparasi Sistem Komunikasi Serial Multipoint pada Robot Management Sampah menggunakan I2C dan SPI

Sampah merupakan masalah yang sering kita jumpai dalam kehidupan sehari-hari. Hal ini perlu diatasi karena setiap manusia pasti memproduksi sampah, disisi lain masyarakat tidak ingin berdekatan dengan sampah. Oleh karena itu dalam makalah ini penulis akan membahas bagian dari robot management sampah.Pada robot ini menggunakan 4 buah mikrokontroler sehingga diperlukan protokol komunikasi serial multipoint. Komunikasi ini harus dirancang dengan baik agar tidak terjadi kesalahan dalam pengiriman maupun penerimaan data serta kecepatan transfer data dapat maksimal sehingga informasi dari sensor selalu terbaca oleh robot.Empat buah mikrokontroler tersebut terdiri dari 1 Master dan 3 Slave. Fungsi masing-masing dari 3 mikrokontroler slave tersebut adalah untuk membaca sensor, mengendalikan motor kiri, dan mengendalikan motor kanan. Sedangkan fungsi dari mikrokontroler master adalah sebagai pemroses data dari slave pembaca sensor dan hasilnya di kirim ke slave pengendali motor kanan dan slave pengendali motor kiri. Dengan sistem seperti ini diharapkan waktu pemrosesan lebih cepat dibandingkan hanya menggunakan 1 mikrokontroler.Kata Kunci— Robot, Mikrokontroler, Serial, Multipoin

Efficient and accurate stereo matching for cloth manipulation

Due to the recent development of robotic techniques, researching robots that can assist in everyday household tasks, especially robotic cloth manipulation has become popular in recent years. Stereo matching forms a crucial part of the robotic vision and aims to derive depth information from image pairs captured by the stereo cameras. Although stereo robotic vision is widely adopted for cloth manipulation robots in the research community, this remains a challenging research task. Robotic vision requires very accurate depth output in a relatively short timespan in order to successfully perform cloth manipulation in real-time. In this thesis, we mainly aim to develop a robotic stereo matching based vision system that is both efficient and effective for the task of robotic cloth manipulation. Effectiveness refers to the accuracy of the depth map generated from the stereo matching algorithms for the robot to grasp the required details to achieve the given task on cloth materials while efficiency emphasizes the required time for the stereo matching to process the images. With respect to efficiency, firstly, by exploring a variety of different hardware architectures such as multi-core CPU and graphic processors (GPU) to accelerate stereo matching, we demonstrate that the parallelised stereo-matching algorithm can be significantly accelerated, achieving 12X and 176X speed-ups respectively for multi-core CPU and GPU, compared with SISD (Single Instruction, Single Data) single-thread CPU. In terms of effectiveness, due to the fact that there are no cloth based testbeds with depth map ground-truths for evaluating the accuracy of stereo matching performance in this context, we created five different testbeds to facilitate evaluation of stereo matching in the context of cloth manipulation. In addition, we adapted a guided filtering algorithm into a pyramidical stereo matching framework that works directly for unrectified images, and evaluate its accuracy utilizing the created cloth testbeds. We demonstrate that our proposed approach is not only efficient, but also accurate and suits well to the characteristics of the task of cloth manipulations. This also shows that rather than relying on image rectification, directly applying stereo matching to unrectified images is effective and efficient. Finally, we further explore whether we can improve efficiency while maintaining reasonable accuracy for robotic cloth manipulations (i.e.~trading off accuracy for efficiency). We use a foveated matching algorithm, inspired by biological vision systems, and found that it is effective in trading off accuracy for efficiency, achieving almost the same level of accuracy for both cloth grasping and flattening tasks with two to three fold acceleration. We also demonstrate that with the robot we can use machine learning techniques to predict the optimal foveation level in order to accomplish the robotic cloth manipulation tasks successfully and much more efficiently. To summarize, in this thesis, we extensively study stereo matching, contributing to the long-term goal of developing effective ways for efficient whilst accurate robotic stereo matching for cloth manipulation

Automated handling of flexible materials

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 1988.Bibliography: leaf 186.by John Charles Briggs.M.S

Reconfigurable and transformational product design concepts and applications : a case study of innovative furniture

Tezin basılısı İstanbul Şehir Üniversitesi Kütüphanesi'ndedir.Multifarious design methods have been developed in engineering and art disciplines in the search of better quality, efficiency, cost-effectiveness, functionality, and novelty. Trans- formational and reconfigurable devices in different fields have become trendier after the 20th century. Customers expect both enhanced performances and reduced complexity level on their products. Compared to static state products with single function amongst furniture product category, the product items with transformational functionality and reconfigurablility possess higher ability for coping different customer needs and expecta- tions. For this case, research and development on innovative, novel product with specific design methodology is essential. In this paper, by reviewing and synthesizing previous research on transformational design and reconfigurable product systems, we carried out the integration of different structures and products, which are from various single state products. These processes are all under the transformational, flexible, reconfigurable design guidelines to obtain the overall function structure of the ideal integrated product.Declaration of Authorship ii Abstract iii Öz iv Acknowledgments vi List of Tables ix 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Product innovation and design . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Research Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Background and Literature Review 7 2.1 Design and production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Engineering design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Product design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.1 Universal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Design for Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.2.1 Methodology: assessing product flexibility . . . . . . . . . 13 2.3.3 Transformational Design . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3.3.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.3.2 Facilitators . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.3.4 Design by analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3.4.1 Definition of Analogy Design . . . . . . . . . . . . . . . . 30 2.3.4.2 Analogies Using Nature . . . . . . . . . . . . . . . . . . . 30 2.3.4.3 Biological Analogies for Design . . . . . . . . . . . . . . . 32 2.3.4.4 Types of similarity relationships . . . . . . . . . . . . . . 34 2.4 Reconfigurability and Reconfigurable Design . . . . . . . . . . . . . . . . . 35 2.4.1 Reconfigurable manufacturing system (RMS) . . . . . . . . . . . . 35 2.4.2 Reconfigurable modular robot machines (RMs) . . . . . . . . . . . 37 2.4.3 Reconfigurable product/system design . . . . . . . . . . . . . . . . 40 3 Identification of a case/product need: Case Study 43 3.1 Problem/Case Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.2 The need for integrated product . . . . . . . . . . . . . . . . . . . . . . . . 47 3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4 Integrated Product Design-Implementation of Design Methods 49 4.1 Identification of a design problem . . . . . . . . . . . . . . . . . . . . . . . 49 4.1.1 Target product identification . . . . . . . . . . . . . . . . . . . . . 49 4.1.2 Customer Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.1.3 Quality Function Development (QFD) Analysis . . . . . . . . . . . 56 4.2 Analysis and examples of target product . . . . . . . . . . . . . . . . . . 60 4.2.1 Market Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2.2 New Product Information in current market . . . . . . . . . . . . 65 4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5 Conceptual Design for Integrated Product 70 5.1 Initial Concept Design Ideas for Each Unit . . . . . . . . . . . . . . . . . . 70 5.1.1 Drawer Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.1.2 Valet Stand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5.1.3 Coat Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.1.4 Lamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.2 Generating Product Alternatives (Integration of New Ideas and Prototyping) 83 5.3 Analysis for Integrations (Materials, Cost, Dimension, Weight) . . . . . . 86 5.4 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6 Results, Conclusion, and Summary 92 6.1 Prototyping, Testing, and Validation . . . . . . . . . . . . . . . . . . . . . 92 6.1.1 Prototype Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.2 Testing and Validation . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.2 Summary and Recommended Future Work . . . . . . . . . . . . . . . . . . 97 A Survey Questionnaire 99 Bibliography 10

Following recipes with a cooking robot

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 141-143).In this thesis, we present BakeBot, a PR2 robot system that interprets natural language baking recipes into baking instructions which it follows to execute the recipe, from mise en place presentation of the ingredients through baking in a toaster oven. We developed parameterized motion primitives for baking. The motion primitives utilize the existing sensing and manipulation capabilities of the PR2 platform and also our new compliant control techniques to address environmental uncertainty. The system was first implemented as a static finite state machine, which was tested through 27 baking attempts, 16 of which successfully resulted in edible cookies. The system was then implemented as a dynamic state machine, in which the robot estimated the world state and planned sequences of motion primitives to follow the baking instructions inferred from the natural language recipe, which was tested thorough five baking attempts of two different recipes, all of which resulted in edible cookies.by Mario Attilio Bollini.S.M

The application of robotics to the assembly of flexible parts by sewing.

This thesis concerns the development of a robotic cell to perform assembly and handling operations on cloth.- A flexible automation approach was adopted, in which the robot was required to control the cloth panel during both handling and sewing operations, without the aid of hard automation attachments which might limit the flexibility of the system. The cell consisted of an adaptively controlled robot, a hierarchy of controllers, a conventional sewing machine, a two-fingered fabric steering end-effector, and several sensor systems. A technique was developed for producing a seam parallel to an edge of arbitrary contour, in which two cameras, a cloth tension sensor and the sewing machine's shaft encoder provided the sensory input. Two sensory servo control systems were required, one control system generated the robot's trajectory to maintain a small constant cloth tension, and the other directed the robot to manipulate the cloth panel to maintain a constant seam width. The design of the cloth tension control was based on the measured frequency response of the open loop system. The seam width control was designed using simulation studies, which accounted for the control transfer function, and nonlinearities such as camera pixel resolution, time delays and robot motion limitations. Several robotic handling techniques were developed, so that a cloth panel placed arbitrarily on the sewing table could be set up for an edge seaming operation, and the cloth could be rotated about the needle. The system's flexibility was demonstrated in the assembly of an irregularly shaped cloth panel, in which three adjacent sides were sewn up