Hands: Human to Robotic

Hands have for centuries been recognized as a fundamental tool for humans to gain an understanding of their environment and at the same time be able to manipulate it. In this presentation we will look at various studies made on the functionality and use of the human hand and examine the different approaches to analyzing and classifying human grasps and building a taxonomy of these grasps. We study the anatomy of the human hand, and examine experiments performed to understand the how gripping forces are applied when lifting objects, and the methods extraction of haptic information, by humans. We discuss issues involved in the building of electro-mechanical manipulators and some of the mathematics used in analyzing the suitability of a design. We look at one of the earliest designs of a computer controlled articulated gripper, as well as two of the most prevalent designs in today\u27s research world, the Stanford/JPL hand and the Utah/MIT had. Finally, we show why a more fundamental understanding of how human grasping works will help us design more useful manipulators

Active recognition and pose estimation of rigid and deformable objects in 3D space

Object recognition and pose estimation is a fundamental problem in computer vision and of utmost importance in robotic applications. Object recognition refers to the problem of recognizing certain object instances, or categorizing objects into specific classes. Pose estimation deals with estimating the exact position of the object in 3D space, usually expressed in Euler angles. There are generally two types of objects that require special care when designing solutions to the aforementioned problems: rigid and deformable. Dealing with deformable objects has been a much harder problem, and usually solutions that apply to rigid objects, fail when used for deformable objects due to the inherent assumptions made during the design. In this thesis we deal with object categorization, instance recognition and pose estimation of both rigid and deformable objects. In particular, we are interested in a special type of deformable objects, clothes. We tackle the problem of autonomously recognizing and unfolding articles of clothing using a dual manipulator. This problem consists of grasping an article from a random point, recognizing it and then bringing it into an unfolded state by a dual arm robot. We propose a data-driven method for clothes recognition from depth images using Random Decision Forests. We also propose a method for unfolding an article of clothing after estimating and grasping two key-points, using Hough Forests. Both methods are implemented into a POMDP framework allowing the robot to interact optimally with the garments, taking into account uncertainty in the recognition and point estimation process. This active recognition and unfolding makes our system very robust to noisy observations. Our methods were tested on regular-sized clothes using a dual-arm manipulator. Our systems perform better in both accuracy and speed compared to state-of-the-art approaches. In order to take advantage of the robotic manipulator and increase the accuracy of our system, we developed a novel approach to address generic active vision problems, called Active Random Forests. While state of the art focuses on best viewing parameters selection based on single view classifiers, we propose a multi-view classifier where the decision mechanism of optimally changing viewing parameters is inherent to the classification process. This has many advantages: a) the classifier exploits the entire set of captured images and does not simply aggregate probabilistically per view hypotheses; b) actions are based on learnt disambiguating features from all views and are optimally selected using the powerful voting scheme of Random Forests and c) the classifier can take into account the costs of actions. The proposed framework was applied to the same task of autonomously unfolding clothes by a robot, addressing the problem of best viewpoint selection in classification, grasp point and pose estimation of garments. We show great performance improvement compared to state of the art methods and our previous POMDP formulation. Moving from deformable to rigid objects while keeping our interest to domestic robotic applications, we focus on object instance recognition and 3D pose estimation of household objects. We are particularly interested in realistic scenes that are very crowded and objects can be perceived under severe occlusions. Single shot-based 6D pose estimators with manually designed features are still unable to tackle such difficult scenarios for a variety of objects, motivating the research towards unsupervised feature learning and next-best-view estimation. We present a complete framework for both single shot-based 6D object pose estimation and next-best-view prediction based on Hough Forests, the state of the art object pose estimator that performs classification and regression jointly. Rather than using manually designed features we propose an unsupervised feature learnt from depth-invariant patches using a Sparse Autoencoder. Furthermore, taking advantage of the clustering performed in the leaf nodes of Hough Forests, we learn to estimate the reduction of uncertainty in other views, formulating the problem of selecting the next-best-view. To further improve 6D object pose estimation, we propose an improved joint registration and hypotheses verification module as a final refinement step to reject false detections. We provide two additional challenging datasets inspired from realistic scenarios to extensively evaluate the state of the art and our framework. One is related to domestic environments and the other depicts a bin-picking scenario mostly found in industrial settings. We show that our framework significantly outperforms state of the art both on public and on our datasets. Unsupervised feature learning, although efficient, might produce sub-optimal features for our particular tast. Therefore in our last work, we leverage the power of Convolutional Neural Networks to tackled the problem of estimating the pose of rigid objects by an end-to-end deep regression network. To improve the moderate performance of the standard regression objective function, we introduce the Siamese Regression Network. For a given image pair, we enforce a similarity measure between the representation of the sample images in the feature and pose space respectively, that is shown to boost regression performance. Furthermore, we argue that our pose-guided feature learning using our Siamese Regression Network generates more discriminative features that outperform the state of the art. Last, our feature learning formulation provides the ability of learning features that can perform under severe occlusions, demonstrating high performance on our novel hand-object dataset. Concluding, this work is a research on the area of object detection and pose estimation in 3D space, on a variety of object types. Furthermore we investigate how accuracy can be further improved by applying active vision techniques to optimally move the camera view to minimize the detection error.Open Acces

Sensing Highly Non-Rigid Objects with RGBD Sensors for Robotic Systems

The goal of this research is to enable a robotic system to manipulate clothing and other highly non-rigid objects using an RGBD sensor. The focus of this thesis is to define and test various algorithms / models that are used to solve parts of the laundry process (i.e. handling, classifying, sorting, unfolding, and folding). First, a system is presented for automatically extracting and classifying items in a pile of laundry. Using only visual sensors, the robot identifies and extracts items sequentially from the pile. When an item is removed and isolated, a model is captured of the shape and appearance of the object, which is then compared against a dataset of known items. The contributions of this part of the laundry process are a novel method for extracting articles of clothing from a pile of laundry, a novel method of classifying clothing using interactive perception, and a multi-layer approach termed L-M-H, more specifically L-C-S-H for clothing classification. This thesis describes two different approaches to classify clothing into categories. The first approach relies upon silhouettes, edges, and other low-level image measurements of the articles of clothing. Experiments from the first approach demonstrate the ability of the system to efficiently classify and label into one of six categories (pants, shorts, short-sleeve shirt, long-sleeve shirt, socks, or underwear). These results show that, on average, classification rates using robot interaction are 59% higher than those that do not use interaction. The second approach relies upon color, texture, shape, and edge information from 2D and 3D data within a local and global perspective. The multi-layer approach compartmentalizes the problem into a high (H) layer, multiple mid-level (characteristics(C), selection masks(S)) layers, and a low (L) layer. This approach produces \u27local\u27 solutions to solve the global classification problem. Experiments demonstrate the ability of the system to efficiently classify each article of clothing into one of seven categories (pants, shorts, shirts, socks, dresses, cloths, or jackets). The results presented in this paper show that, on average, the classification rates improve by +27.47% for three categories, +17.90% for four categories, and +10.35% for seven categories over the baseline system, using support vector machines. Second, an algorithm is presented for automatically unfolding a piece of clothing. A piece of cloth is pulled in different directions at various points of the cloth in order to flatten the cloth. The features of the cloth are extracted and calculated to determine a valid location and orientation in which to interact with it. The features include the peak region, corner locations, and continuity / discontinuity of the cloth. In this thesis, a two-stage algorithm is presented, introducing a novel solution to the unfolding / flattening problem using interactive perception. Simulations using 3D simulation software, and experiments with robot hardware demonstrate the ability of the algorithm to flatten pieces of laundry using different starting configurations. These results show that, at most, the algorithm flattens out a piece of cloth from 11.1% to 95.6% of the canonical configuration. Third, an energy minimization algorithm is presented that is designed to estimate the configuration of a deformable object. This approach utilizes an RGBD image to calculate feature correspondence (using SURF features), depth values, and boundary locations. Input from a Kinect sensor is used to segment the deformable surface from the background using an alpha-beta swap algorithm. Using this segmentation, the system creates an initial mesh model without prior information of the surface geometry, and it reinitializes the configuration of the mesh model after a loss of input data. This approach is able to handle in-plane rotation, out-of-plane rotation, and varying changes in translation and scale. Results display the proposed algorithm over a dataset consisting of seven shirts, two pairs of shorts, two posters, and a pair of pants. The current approach is compared using a simulated shirt model in order to calculate the mean square error of the distance from the vertices on the mesh model to the ground truth, provided by the simulation model

Playful Materialities

Game culture and material culture have always been closely linked. Analog forms of rule-based play (ludus) would hardly be conceivable without dice, cards, and game boards. In the act of free play (paidia), children as well as adults transform simple objects into multifaceted toys in an almost magical way. Even digital play is suffused with material culture: Games are not only mediated by technical interfaces, which we access via hardware and tangible peripherals. They are also subject to material hybridization, paratextual framing, and processes of de-, and re-materialization

A Web-based user-oriented tool for universal kitchen design

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Architecture, 2002.Includes bibliographical references (leaves 93-94).Economic constraints to the professional design practice limit customized solutions to the very wealthy, and thus most of the kitchens in current development housing projects are still generic. With aging baby boomers and an increasing number of survivors of disability, diversified user needs require professional housing design to accommodate all different individuals, which challenges current design standards. Based on universal design principles, a user-oriented tool for universal home design provides more than a few homebuyers with a chance to take advantage of professional design. Kitchen layout design is taken as the starting point of the home design, since the kitchen has the most complex functional requirements and the most difficult barriers of any room in the house. The tool takes user needs as the design motivation and professional best design practice as the database. For the majority of lay people, this tool makes it possible for them to orient their own kitchen design through a guided searching of proper design strategies according to the user needs and preferences. It highlights the needs of particular users at different ages and physical conditions according to the universal design principles. A direct typological conversion is implemented to link user needs with design strategies. This paper is also the documentation of the universal home design tool project in the House_n Consortium at MIT. The paper takes the project as the main example for stating the possibility and feasibility of the new design methodology.by Xiaoyi Ma.S.M

Playful Materialities: The Stuff That Games Are Made Of

Game culture and material culture have always been closely linked. Analog forms of rule-based play (ludus) would hardly be conceivable without dice, cards, and game boards. In the act of free play (paidia), children as well as adults transform simple objects into multifaceted toys in an almost magical way. Even digital play is suffused with material culture: Games are not only mediated by technical interfaces, which we access via hardware and tangible peripherals. They are also subject to material hybridization, paratextual framing, and processes of de-, and re-materialization

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Playful Materialities

Game culture and material culture have always been closely linked. Analog forms of rule-based play (ludus) would hardly be conceivable without dice, cards, and game boards. In the act of free play (paidia), children as well as adults transform simple objects into multifaceted toys in an almost magical way. Even digital play is suffused with material culture: Games are not only mediated by technical interfaces, which we access via hardware and tangible peripherals. They are also subject to material hybridization, paratextual framing, and processes of de-, and re-materialization