Developmental Bayesian Optimization of Black-Box with Visual Similarity-Based Transfer Learning

We present a developmental framework based on a long-term memory and reasoning mechanisms (Vision Similarity and Bayesian Optimisation). This architecture allows a robot to optimize autonomously hyper-parameters that need to be tuned from any action and/or vision module, treated as a black-box. The learning can take advantage of past experiences (stored in the episodic and procedural memories) in order to warm-start the exploration using a set of hyper-parameters previously optimized from objects similar to the new unknown one (stored in a semantic memory). As example, the system has been used to optimized 9 continuous hyper-parameters of a professional software (Kamido) both in simulation and with a real robot (industrial robotic arm Fanuc) with a total of 13 different objects. The robot is able to find a good object-specific optimization in 68 (simulation) or 40 (real) trials. In simulation, we demonstrate the benefit of the transfer learning based on visual similarity, as opposed to an amnesic learning (i.e. learning from scratch all the time). Moreover, with the real robot, we show that the method consistently outperforms the manual optimization from an expert with less than 2 hours of training time to achieve more than 88% of success

Bagging by Learning to Singulate Layers Using Interactive Perception

Many fabric handling and 2D deformable material tasks in homes and industry require singulating layers of material such as opening a bag or arranging garments for sewing. In contrast to methods requiring specialized sensing or end effectors, we use only visual observations with ordinary parallel jaw grippers. We propose SLIP: Singulating Layers using Interactive Perception, and apply SLIP to the task of autonomous bagging. We develop SLIP-Bagging, a bagging algorithm that manipulates a plastic or fabric bag from an unstructured state, and uses SLIP to grasp the top layer of the bag to open it for object insertion. In physical experiments, a YuMi robot achieves a success rate of 67% to 81% across bags of a variety of materials, shapes, and sizes, significantly improving in success rate and generality over prior work. Experiments also suggest that SLIP can be applied to tasks such as singulating layers of folded cloth and garments. Supplementary material is available at https://sites.google.com/view/slip-bagging/

ShakingBot: Dynamic Manipulation for Bagging

Bag manipulation through robots is complex and challenging due to the deformability of the bag. Based on dynamic manipulation strategy, we propose a new framework, ShakingBot, for the bagging tasks. ShakingBot utilizes a perception module to identify the key region of the plastic bag from arbitrary initial configurations. According to the segmentation, ShakingBot iteratively executes a novel set of actions, including Bag Adjustment, Dual-arm Shaking, and One-arm Holding, to open the bag. The dynamic action, Dual-arm Shaking, can effectively open the bag without the need to account for the crumpled configuration.Then, we insert the items and lift the bag for transport. We perform our method on a dual-arm robot and achieve a success rate of 21/33 for inserting at least one item across various initial bag configurations. In this work, we demonstrate the performance of dynamic shaking actions compared to the quasi-static manipulation in the bagging task. We also show that our method generalizes to variations despite the bag's size, pattern, and color.Comment: Manipulating bag through robots to baggin

Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks

Rearranging and manipulating deformable objects such as cables, fabrics, and bags is a long-standing challenge in robotic manipulation. The complex dynamics and high-dimensional configuration spaces of deformables, compared to rigid objects, make manipulation difficult not only for multi-step planning, but even for goal specification. Goals cannot be as easily specified as rigid object poses, and may involve complex relative spatial relations such as "place the item inside the bag". In this work, we develop a suite of simulated benchmarks with 1D, 2D, and 3D deformable structures, including tasks that involve image-based goal-conditioning and multi-step deformable manipulation. We propose embedding goal-conditioning into Transporter Networks, a recently proposed model architecture for learning robotic manipulation that rearranges deep features to infer displacements that can represent pick and place actions. We demonstrate that goal-conditioned Transporter Networks enable agents to manipulate deformable structures into flexibly specified configurations without test-time visual anchors for target locations. We also significantly extend prior results using Transporter Networks for manipulating deformable objects by testing on tasks with 2D and 3D deformables. Supplementary material is available at https://berkeleyautomation.github.io/bags/.Comment: See https://berkeleyautomation.github.io/bags/ for project website and code; v2 corrects some BibTeX entries, v3 is ICRA 2021 version (minor revisions

Sensors for Robotic Hands: A Survey of State of the Art

Recent decades have seen significant progress in the field of artificial hands. Most of the surveys, which try to capture the latest developments in this field, focused on actuation and control systems of these devices. In this paper, our goal is to provide a comprehensive survey of the sensors for artificial hands. In order to present the evolution of the field, we cover five year periods starting at the turn of the millennium. At each period, we present the robot hands with a focus on their sensor systems dividing them into categories, such as prosthetics, research devices, and industrial end-effectors.We also cover the sensors developed for robot hand usage in each era. Finally, the period between 2010 and 2015 introduces the reader to the state of the art and also hints to the future directions in the sensor development for artificial hands

Prototyping for Research and Industry

In this thesis we want to present some of the activities carried out during the PhD studies held at the PhD School "L. da Vinci" in the period from January 2012 to December 2014. The activities were held in the fields of robotics and mechanical engineering, and the main theme was the prototyping of new concepts, as well as the activity of conceptual design in its different phases, from generation of the idea, to the realization and testing of prototypes. The conceptual design phase is of fundamental importance to structure the process of generation of new ideas. Sometimes it is a process that is carried out unconsciously by the inventor. Providing a tool that allows to guide him in the various stages of idea generation can lead to advantages that let the inventor to explore areas from which take inspiration, which otherwise would not have been taken into account. An aspect of fundamental importance in the development of new prototypes is a process that goes in the opposite direction of the idea generation phase. Initially the conceptual design tends to provide tools to generate as many ideas as possible, but at some point there is the need to select a limited number of cases to investigate. Through the selection phase, which can be structured at levels more or less structured, and more or less qualitative/quantitative, the inventor tends to identify, case by case, which are the ideas in which is worth investing time and resources, before moving to the following stages. Prototyping, as well as its previous phase, now commonly called pretotyping, are mandatory steps for those who want to develop any new idea. The success of the final product or service may depend from the analysis of the pretotype first, and of the prototype later, since it allows to detect limits and possible improvements of the concept before moving to the final implementation phase

Automated re-prefabrication system for buildings using robotics

Prefabrication has the advantages of simplicity, speed and economy but has been inflexible to changes in design which is a primary reason behind its limited market share in the construction industry. To tackle this drawback, this study presents a Robotic Prefabrication System (RPS) which employs a new concept called “re-fabrication”: the automatic disassembly of a prefabricated structure and its reconstruction according to a new design. The RPS consists of a software module and a hardware module. First, the software employs the 3D model of a prefabricated structure as input, and returns motor control command output to the hardware. There are two underlying algorithms developed in the software module. First, a novel algorithm automatically compares the old and new models and identifies the components which the two models do not have in common in order to enable disassembly of the original structure and its refabrication into the new design. In addition, an additional novel algorithm computes the optimal refabrication sequence to transform one model into another according to the differences identified. Meanwhile, the hardware module takes the motor control commands as input and executes the appropriate assembly/disassembly operations, and returns the desired refabricated structure in realtime. Validation tests on two lab-scaled prefabricated structures demonstrate that the system successfully generated the desired refabrication sequences and performed all assembly operations with acceptable placement precision

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion