Improving Robotic Manipulation via Reachability, Tactile, and Spatial Awareness

Robotic grasping and manipulation remains an active area of research despite significant progress over the past decades. Many existing solutions still struggle to robustly handle difficult situations that a robot might encounter even in non-contrived settings.For example, grasping systems struggle when the object is not centrally located in the robot's workspace. Also, grasping in dynamic environments presents a unique set of challenges. A stable and feasible grasp can become infeasible as the object moves; this problem becomes pronounced when there are obstacles in the scene. This research is inspired by the observation that object-manipulation tasks like grasping, pick-and-place or insertion require different forms of awareness. These include reachability awareness -- being aware of regions that can be reached without self-collision or collision with surrounding objects; tactile awareness-- ability to feel and grasp objects just tight enough to prevent slippage or crushing the objects; and 3D awareness -- ability to perceive size and depth in ways that makes object manipulation possible. Humans use these capabilities to achieve a high level of coordination needed for object manipulation. In this work, we develop techniques that equip robots with similar sensitivities towards realizing a reliable and capable home-assistant robot. In this thesis we demonstrate the importance of reasoning about the robot's workspace to enable grasping systems handle more difficult settings such as picking up moving objects while avoiding surrounding obstacles. Our method encodes the notion of reachability and uses it to generate not just stable grasps but ones that are also achievable by the robot. This reachability-aware formulation effectively expands the useable workspace of the robot enabling the robot to pick up objects from difficult-to-reach locations. While recent vision-based grasping systems work reliably well achieving pickup success rate higher than 90\% in cluttered scenes, failure cases due to calibration error, slippage and occlusion were challenging. To address this, we develop a closed-loop tactile-based improvement that uses additional tactile sensing to deal with self-occlusion (a limitation of vision-based system) and adaptively tighten the robot's grip on the object-- making the grasping system tactile-aware and more reliable. This can be used as an add-on to existing grasping systems. This adaptive tactile-based approach demonstrates the effectiveness of closed-loop feedback in the final phase of the grasping process. To achieve closed-loop manipulation all through the manipulation process, we study the value of multi-view camera systems to improve learning-based manipulation systems. Using a multi-view Q-learning formulation, we develop a learned closed-loop manipulation algorithm for precise manipulation tasks that integrates inputs from multiple static RGB cameras to overcome self-occlusion and improve 3D understanding. To conclude, we discuss some opportunities/ directions for future work

A pneumatic conveyor robot for color detection and sorting

Despite numerous research works on conveyor robots, few works can be found on electropneumatic conveyor belt robots with two separated lines. The unique feature of this study is a combination of various systems to develop an electropneumatic robot. In this work, an automated and intelligent mechatronic conveyor system is designed and developed for transporting and positioning circular objects that can be used in the manufacturing and packaging industries. In addition to moving and positioning, timing can also be controlled on this conveyor belt robot. All control operations are handled by an electrical and programmable relay called a mini programmable logic controller (PLC), color sensor, gripper arm, and electronic switches. An electropneumatic system is used to control the robot for placing objects. The main goal of this study is to develop a novel 3D structural design which make the procedure unique for better efficiency and accuracy. The novelty of this work lies within the 3D design of two belts and assembly of all electropneumatic components which are helpful for manufacturing assembly lines. Also, TCS230 sensor and AVR microcontroller are used to identify the colors within the operation. The results show the accuracy of the developed system is reliable in terms of color and positioning detection. The system is able to work non-stop for more than 1 hour without any issues

Contemporary Robotics

This book book is a collection of 18 chapters written by internationally recognized experts and well-known professionals of the field. Chapters contribute to diverse facets of contemporary robotics and autonomous systems. The volume is organized in four thematic parts according to the main subjects, regarding the recent advances in the contemporary robotics. The first thematic topics of the book are devoted to the theoretical issues. This includes development of algorithms for automatic trajectory generation using redudancy resolution scheme, intelligent algorithms for robotic grasping, modelling approach for reactive mode handling of flexible manufacturing and design of an advanced controller for robot manipulators. The second part of the book deals with different aspects of robot calibration and sensing. This includes a geometric and treshold calibration of a multiple robotic line-vision system, robot-based inline 2D/3D quality monitoring using picture-giving and laser triangulation, and a study on prospective polymer composite materials for flexible tactile sensors. The third part addresses issues of mobile robots and multi-agent systems, including SLAM of mobile robots based on fusion of odometry and visual data, configuration of a localization system by a team of mobile robots, development of generic real-time motion controller for differential mobile robots, control of fuel cells of mobile robots, modelling of omni-directional wheeled-based robots, building of hunter- hybrid tracking environment, as well as design of a cooperative control in distributed population-based multi-agent approach. The fourth part presents recent approaches and results in humanoid and bioinspirative robotics. It deals with design of adaptive control of anthropomorphic biped gait, building of dynamic-based simulation for humanoid robot walking, building controller for perceptual motor control dynamics of humans and biomimetic approach to control mechatronic structure using smart materials

Motion Planning for Manipulation With Heuristic Search

Heuristic searches such as A* search are a popular means of finding least-cost plans due to their generality, strong theoretical guarantees on completeness and optimality, simplicity in implementation, and consistent behavior. In planning for robotic manipulation, however, these techniques are commonly thought of as impractical due to the high-dimensionality of the planning problem. As part of this thesis work, we have developed a heuristic search-based approach to motion planning for manipulation that does deal effectively with the high-dimensionality of the problem. In this thesis, I will present the approach together with its theoretical properties and show how to apply it to single-arm and dual-arm motion planning with upright constraints on a PR2 robot operating in non-trivial cluttered spaces. Then I will explain how we extended our approach to manipulation planning for n-arms with regrasping. In this work, the planner itself makes all of the discrete decisions, including which arm to use for the pickup and putdown, whether handoffs are necessary and how the object should be grasped at each step along the way. An extensive experimental analysis in both simulation and on a physical PR2 shows that, in terms of runtime, our approach is on par with some of the most common sampling-based approaches. This includes benchmarking our planning framework on two domains that we constructed that are common to manufacturing: pick-and-place of fast moving objects and the autonomous assembly of small objects. Between these applications, the planner exhibited fast planning times and the ability to robustly plan paths into and out of tight working environments that are common to assembly. The closing work of this thesis includes an exhaustive study of the natural tradeoff that occurs between planning efficiency versus solution quality for different values of the heuristic inflation factor. A comparison of the solution quality of our planner to paths computed by an asymptotically optimal approach given a great deal of time for path optimization is included as well. Finally, a set of experimental results are included that show that due to our approach\u27s deterministic cost-minimization, similar input tends to lead to similarity in the output. This kind of local consistency is important to the predictability of the robot\u27s motions and contributes to human-robot safety

From visuomotor control to latent space planning for robot manipulation

Deep visuomotor control is emerging as an active research area for robot manipulation. Recent advances in learning sensory and motor systems in an end-to-end manner have achieved remarkable performance across a range of complex tasks. Nevertheless, a few limitations restrict visuomotor control from being more widely adopted as the de facto choice when facing a manipulation task on a real robotic platform. First, imitation learning-based visuomotor control approaches tend to suffer from the inability to recover from an out-of-distribution state caused by compounding errors. Second, the lack of versatility in task definition limits skill generalisability. Finally, the training data acquisition process and domain transfer are often impractical. In this thesis, individual solutions are proposed to address each of these issues. In the first part, we find policy uncertainty to be an effective indicator of potential failure cases, in which the robot is stuck in out-of-distribution states. On this basis, we introduce a novel uncertainty-based approach to detect potential failure cases and a recovery strategy based on action-conditioned uncertainty predictions. Then, we propose to employ visual dynamics approximation to our model architecture to capture the motion of the robot arm instead of the static scene background, making it possible to learn versatile skill primitives. In the second part, taking inspiration from the recent progress in latent space planning, we propose a gradient-based optimisation method operating within the latent space of a deep generative model for motion planning. Our approach bypasses the traditional computational challenges encountered by established planning algorithms, and has the capability to specify novel constraints easily and handle multiple constraints simultaneously. Moreover, the training data comes from simple random motor-babbling of kinematically feasible robot states. Our real-world experiments further illustrate that our latent space planning approach can handle both open and closed-loop planning in challenging environments such as heavily cluttered or dynamic scenes. This leads to the first, to our knowledge, closed-loop motion planning algorithm that can incorporate novel custom constraints, and lays the foundation for more complex manipulation tasks