Learning deep policies for physics-based robotic manipulation in cluttered real-world environments

This thesis presents a series of planners and learning algorithms for real-world manipulation in clutter. The focus is on interleaving real-world execution with look-ahead planning in simulation as an effective way to address the uncertainty arising from complex physics interactions and occlusions. We introduce VisualRHP, a receding horizon planner in the image space guided by a learned heuristic. VisualRHP generates, in closed-loop, prehensile and non-prehensile manipulation actions to manipulate a desired object in clutter while avoiding dropping obstacle objects off the edge of the manipulation surface. To acquire the heuristic of VisualRHP, we develop deep imitation learning and deep reinforcement learning algorithms specifically tailored for environments with complex dynamics and requiring long-term sequential decision making. The learned heuristic ensures generalization over different environment settings and transferability of manipulation skills to different desired objects in the real world. In the second part of this thesis, we integrate VisualRHP with a learnable object pose estimator to guide the search for an occluded desired object. This hybrid approach harnesses neural networks with convolution and recurrent structures to capture relevant information from the history of partial observation to guide VisualRHP future actions. We run an ablation study over the different component of VisualRHP and compare it with model-free and model-based alternatives. We run experiments in different simulation environments and real-world settings. The results show that by trading a small computation time for heuristic-guided look-ahead planning, VisualRHP delivers a more robust and efficient behaviour compared to alternative state-of-the-art approaches while still operating in near real-time

Cognition-enabled robotic wiping: Representation, planning, execution, and interpretation

Advanced cognitive capabilities enable humans to solve even complex tasks by representing and processing internal models of manipulation actions and their effects. Consequently, humans are able to plan the effect of their motions before execution and validate the performance afterwards. In this work, we derive an analog approach for robotic wiping actions which are fundamental for some of the most frequent household chores including vacuuming the floor, sweeping dust, and cleaning windows. We describe wiping actions and their effects based on a qualitative particle distribution model. This representation enables a robot to plan goal-oriented wiping motions for the prototypical wiping actions of absorbing, collecting and skimming. The particle representation is utilized to simulate the task outcome before execution and infer the real performance afterwards based on haptic perception. This way, the robot is able to estimate the task performance and schedule additional motions if necessary. We evaluate our methods in simulated scenarios, as well as in real experiments with the humanoid service robot Rollin’ Justin

Automated Planning of Whole-Body Motions for Everyday Household Chores with a Humanoid Service Robot

Wiping actions are required in many everyday household activities. Whether for skimming bread crumbs of a table top with a sponge or for collecting shards of a broken mug on the surface floor with a broom, future service robots are expected to master such manipulation tasks with high level of autonomy. In contrast to actions such as pick-and-place where immediate effects are observed, wiping tasks require a more elaborate planning process to achieve the desired outcome. The wiping motions have to autonomously adapt to different environment layouts and to the specifications of the robot and the tool involved. The work presented in this report proposes a strategy, called extended Semantic Directed Graph eSDG, for mapping wiping related semantic commands to joint motions of a humanoid robot. The medium (e.g. bread crumbs, water or shards of a broken mug) and the physical interaction parameters are represented in a qualitative model as particles distributed on a surface. eSDG combines information from the qualitative, the geometric state of the robot and the environment together with the specific semantic goal of the wiping task in order to generate executable and goal oriented Cartesian paths. The robot reachable workspace is used to reason about the wiping the partitioning of the tasks into smaller sub-tasks that can be executed from a static position of the robot base. For the eSDG path following problem a cascading structure of inverse kinematics solvers is developed, where the degree of freedom of the involved tool is exploited in favor of the wiping quality. The proposed approach is evaluated in a series of simulated scenarios. The results are validated by a real experiment on the humanoid robot Rollin’ Justin

Robotic Agents Representing, Reasoning, and Executing Wiping Tasks for Daily Household Chores

Universal robotic agents are envisaged to perform a wide range of manipulation tasks in everyday environments. A common action observed in many household chores is wiping, such as the absorption of spilled water with a sponge, skimming breadcrumbs off the dining table, or collecting shards of a broken mug using a broom. To cope with this versatility, the agents have to represent the tasks on a high level of abstraction. In this work, we propose to represent the medium in wiping tasks (e.g. water, breadcrumbs, or shards) as generic particle distribution. This representation enables us to represent wiping tasks as the desired state change of the particles, which allows the agent to reason about the effects of wiping motions in a qualitative manner. Based on this, we develop three prototypical wiping actions for the generic tasks of absorbing, collecting and skimming. The Cartesian wiping motions are resolved to joint motions exploiting the free degree of freedom of the involved tool. Furthermore, the workspace of the robotic manipulators is used to reason about the reachability of wiping motions. We evaluate our methods in simulated scenarios, as well as in a real experiment with the robotic agent Rollin' Justin

A novel xylene-free deparaffinization method for the extraction of proteins from human derived formalin-fixed paraffin embedded (FFPE) archival tissue blocks

Protein detection methods in formalin-fixed paraffin embedded (FFPE) tissue blocks are widely used in research and clinical setting in order to diagnose or to confirm a diagnosis of various types of diseases. Therefore, multiple protein extraction methods from FFPE tissue sections have been developed in this regard. However, the yield and the quality of proteins extracted from FFPE tissues are significantly reduced in blocks stored for longer periods of time. Regardless the protein extraction method used, tissue sections must be first deparaffinized with xylene, and then washed in serial dilutions of ethanol in order to remove the toxic organic solvent “xylene” and rehydrate the tissue. The objective of this study was first to develop a method to deparaffinize FFPE blocks that excludes the use of toxic solvent “xylene”. Second minimize the time required to perform the extraction. Here we describe a method where: • The entire paraffin embedded blocks are deparaffinized and rehydrated using only hot distilled water as a substitute for both xylene and ethanol • The entire procedure takes about 15 min • Deparaffinized blocks are immediately homogenized in lysis buffer, and the obtained lysate analyzed by Western blot. With this new modified technique, we were able to successfully detect actin and AKT proteins in lysates from blocks embedded in paraffin for up to 9 years

An optimized xylene-free protein extraction method adapted to formalin-fixed paraffin embedded tissue sections for western blot analysis

Deparaffinization of formalin-fixed paraffin embedded (FFPE) tissues with xylene currently remains a major challenge to the biomedical community. We developed an efficient xylene-free protocol to isolate proteins from archived FFPE human tissue sections. A total of 79 different types of FFPE tissue sections of 8 µm thickness were obtained from various archived FFPE specimens. Deparaffinization was conducted by gently washing each section with around 1 ml of hot distilled water (≈80°C). The deparaffinized tissues were homogenized in lysis buffer, and the isolated proteins were quantified and efficiently resolved using western blot analysis for the presence of Protein kinase B (PKB/AKT) and β-actin. Moreover, a significant amount of proteins was successfully isolated with an average of 2.31 µg/µl. The migration pattern of AKT and β-actin obtained from the specimens was similar to the positive control obtained from protein lysates prepared from in vitro cultured MDA231 cancer cell lines. AKT was successfully identified in all specimens, and β-actin protein was resolved with an efficiency higher than 80%. The entire extraction procedure requires only 20 minutes. This newly developed technique is an efficient, safe, cost-effective, and rapid method to isolate proteins from FFPE tissue sections adequate for molecular analysis