Bimanual regrasping from unimanual machine learning

Abstract — While unimanual regrasping has been studied ex-tensively, either by regrasping in-hand or by placing the object on a surface, bimanual regrasping has seen little attention. The recent popularity of simple end-effectors and dual-manipulator platforms makes bimanual regrasping an important behavior for service robots to possess. We solve the challenge of bimanual regrasping by casting it as an optimization problem, where the objective is to minimize execution time. The optimization problem is supplemented by image processing and a unimanual grasping algorithm based on machine learning that jointly identify two good grasping points on the object and the proper orientations for each end-effector. The optimization algorithm exploits this data by finding the proper regrasp location and orientation to minimize execution time. Influenced by human bimanual manipulation, the algorithm only requires a single stereo image as input. The efficacy of the method we propose is demonstrated on a dual manipulator torso equipped with Barrett WAM arms and Barrett Hands. I

Dynamics and Control of Whole Arm Grasps

In this paper we consider the dynamics and control of whole arm grasping systems. We develop a control scheme that employs a minimal set of inputs to control the trajectory of the system while using the surplus inputs to control the interaction forces in order to maintain the unilateral constraints at both rolling and sliding contacts. Since the number of surplus inputs is less than the number of output force variables, we propose a controller that controls the critical contact force components. We emphasize the dynamic models and algorithms for computing contact forces, which are crucial to the development of the control algorithms. Finally, we show how compliant contact models and a previously developed integrated simulation approach [14] are used to overcome the difficulties with uniqueness and existence of solutions. A planar whole arm manipulation system is used as an example to illustrate the basic ideas

Towards Developing Gripper to obtain Dexterous Manipulation

Artificial hands or grippers are essential elements in many robotic systems, such as, humanoid, industry, social robot, space robot, mobile robot, surgery and so on. As humans, we use our hands in different ways and can perform various maneuvers such as writing, altering posture of an object in-hand without having difficulties. Most of our daily activities are dependent on the prehensile and non-prehensile capabilities of our hand. Therefore, the human hand is the central motivation of grasping and manipulation, and has been explicitly studied from many perspectives such as, from the design of complex actuation, synergy, use of soft material, sensors, etc; however to obtain the adaptability to a plurality of objects along with the capabilities of in-hand manipulation of our hand in a grasping device is not easy, and not fully evaluated by any developed gripper. Industrial researchers primarily use rigid materials and heavy actuators in the design for repeatability, reliability to meet dexterity, precision, time requirements where the required flexibility to manipulate object in-hand is typically absent. On the other hand, anthropomorphic hands are generally developed by soft materials. However they are not deployed for manipulation mainly due to the presence of numerous sensors and consequent control complexity of under-actuated mechanisms that significantly reduce speed and time requirements of industrial demand. Hence, developing artificial hands or grippers with prehensile capabilities and dexterity similar to human like hands is challenging, and it urges combined contributions from multiple disciplines such as, kinematics, dynamics, control, machine learning and so on. Therefore, capabilities of artificial hands in general have been constrained to some specific tasks according to their target applications, such as grasping (in biomimetic hands) or speed/precision in a pick and place (in industrial grippers). Robotic grippers developed during last decades are mostly aimed to solve grasping complexities of several objects as their primary objective. However, due to the increasing demands of industries, many issues are rising and remain unsolved such as in-hand manipulation and placing object with appropriate posture. Operations like twisting, altering orientation of object within-hand, require significant dexterity of the gripper that must be achieved from a compact mechanical design at the first place. Along with manipulation, speed is also required in many robotic applications. Therefore, for the available speed and design simplicity, nonprehensile or dynamic manipulation is widely exploited. The nonprehensile approach however, does not focus on stable grasping in general. Also, nonprehensile or dynamic manipulation often exceeds robot\u2019s kinematic workspace, which additionally urges installation of high speed feedback and robust control. Hence, these approaches are inapplicable especially when, the requirements are grasp oriented such as, precise posture change of a payload in-hand, placing payload afterward according to a strict final configuration. Also, addressing critical payload such as egg, contacts (between gripper and egg) cannot be broken completely during manipulation. Moreover, theoretical analysis, such as contact kinematics, grasp stability cannot predict the nonholonomic behaviors, and therefore, uncertainties are always present to restrict a maneuver, even though the gripper is capable of doing the task. From a technical point of view, in-hand manipulation or within-hand dexterity of a gripper significantly isolates grasping and manipulation skills from the dependencies on contact type, a priory knowledge of object model, configurations such as initial or final postures and also additional environmental constraints like disturbance, that may causes breaking of contacts between object and finger. Hence, the property (in-hand manipulation) is important for a gripper in order to obtain human hand skill. In this research, these problems (to obtain speed, flexibility to a plurality of grasps, within-hand dexterity in a single gripper) have been tackled in a novel way. A gripper platform named Dexclar (DEXterous reConfigurable moduLAR) has been developed in order to study in-hand manipulation, and a generic spherical payload has been considered at the first place. Dexclar is mechanism-centric and it exploits modularity and reconfigurability to the aim of achieving within-hand dexterity rather than utilizing soft materials. And hence, precision, speed are also achievable from the platform. The platform can perform several grasps (pinching, form closure, force closure) and address a very important issue of releasing payload with final posture/ configuration after manipulation. By exploiting 16 degrees of freedom (DoF), Dexclar is capable to provide 6 DoF motions to a generic spherical or ellipsoidal payload. And since a mechanism is reliable, repeatable once it has been properly synthesized, precision and speed are also obtainable from them. Hence Dexclar is an ideal starting point to study within-hand dexterity from kinematic point of view. As the final aim is to develop specific grippers (having the above capabilities) by exploiting Dexclar, a highly dexterous but simply constructed reconfigurable platform named VARO-fi (VARiable Orientable fingers with translation) is proposed, which can be used as an industrial end-effector, as well as an alternative of bio-inspired gripper in many robotic applications. The robust four fingered VARO-fi addresses grasp, in-hand manipulation and release (payload with desired configuration) of plurality of payloads, as demonstrated in this thesis. Last but not the least, several tools and end-effectors have been constructed to study prehensile and non-prehensile manipulation, thanks to Bayer Robotic challenge 2017, where the feasibility and their potentiality to use them in an industrial environment have been validated. The above mentioned research will enhance a new dimension for designing grippers with the properties of dexterity and flexibility at the same time, without explicit theoretical analysis, algorithms, as those are difficult to implement and sometime not feasible for real system

Motion Planning for Manipulation With Heuristic Search

Heuristic searches such as A* search are a popular means of finding least-cost plans due to their generality, strong theoretical guarantees on completeness and optimality, simplicity in implementation, and consistent behavior. In planning for robotic manipulation, however, these techniques are commonly thought of as impractical due to the high-dimensionality of the planning problem. As part of this thesis work, we have developed a heuristic search-based approach to motion planning for manipulation that does deal effectively with the high-dimensionality of the problem. In this thesis, I will present the approach together with its theoretical properties and show how to apply it to single-arm and dual-arm motion planning with upright constraints on a PR2 robot operating in non-trivial cluttered spaces. Then I will explain how we extended our approach to manipulation planning for n-arms with regrasping. In this work, the planner itself makes all of the discrete decisions, including which arm to use for the pickup and putdown, whether handoffs are necessary and how the object should be grasped at each step along the way. An extensive experimental analysis in both simulation and on a physical PR2 shows that, in terms of runtime, our approach is on par with some of the most common sampling-based approaches. This includes benchmarking our planning framework on two domains that we constructed that are common to manufacturing: pick-and-place of fast moving objects and the autonomous assembly of small objects. Between these applications, the planner exhibited fast planning times and the ability to robustly plan paths into and out of tight working environments that are common to assembly. The closing work of this thesis includes an exhaustive study of the natural tradeoff that occurs between planning efficiency versus solution quality for different values of the heuristic inflation factor. A comparison of the solution quality of our planner to paths computed by an asymptotically optimal approach given a great deal of time for path optimization is included as well. Finally, a set of experimental results are included that show that due to our approach\u27s deterministic cost-minimization, similar input tends to lead to similarity in the output. This kind of local consistency is important to the predictability of the robot\u27s motions and contributes to human-robot safety

双腕アームロボットによる布被覆作業に関する研究

本研究の目的は，物体を布で包む作業（被覆作業）をモデル化し，ロボットによる被覆作業を実現させることである．本論文では「目標線」の概念に基づいて物体を布で包む作業（被覆作業）をモデル化することを提案した．これにより，まず人間が大まかな包み方を教示し，次に布と物体の形状から被覆作業を計画し，最終的にロボットの動作を生成し，ロボットによる被覆作業を実現した．近年，工場のロボット化が行われているが，ロボット化できない作業はまだまだ存在している．それらは，人間にしか行えないような巧みで複雑な作業，あるいは，ロボットより人間の方が効率的にできてしまうような作業である．そのような作業の１つとして，布を扱う作業が挙げられる．布を扱う作業の中には，布単体だけでなく，物体も一緒に取り扱っていく被覆作業が多く存在している．しかし，この被覆作業をロボットに指示するための有効な作業モデルは確立されていない．先行研究では，ロボットによる布操作の記述方法として，点，折り線や手先経路が用いられている．また，コンピュータグラフィクス分野では目標線という記述方法があり，これは被覆を表現するために用いられている．被覆作業をロボット化する上では，まず，実世界のロボットのために，汎用的な被覆モデルとして必要となる物体と布の関係や作業手順を，どのように記述すればいいのかという問題に直面する．このような点を考慮し，被覆作業に適した記述モデルを導入しなければならない．次に，そのような被覆のための作業記述を，実際のロボットにどのように入力すればいいのかという問題がある．煩雑な指示方法ではなく，実空間上で人間が考えている被覆作業を，直感的にロボットに指示できるのが望ましい．最後に，その作業記述から実際のロボットの動きをどのように生成すればよいのかという問題が現れてくる．ロボットが被覆作業を達成するためには，実際の手先軌道や干渉を回避するための動作を，状況に合わせて生成しなければならない．以上を踏まえて，本研究ではロボットによる被覆作業の課題に取り組んだ．具体的には以下の課題について取り組んだ．・布と物体の関係を適切に表す記述方法・直感的な被覆手順の指示方法・ロボットの動作軌道の生成方法まず，布と物体の関係を適切に表す記述方法について検討した．本研究では，コンピュータグラフィクス分野で用いられた目標線という記述方法を，実空間のロボットに導入することを提案した．この目標線は平面だけでなく曲面形状への指示が行いやすい．そして，物体のどこを布で包んでいくかという被覆の本質的な情報を自然に表せる利点を持つ．その中では，凹凸が存在するような物体に対しても被覆を行う場合があり，その凹凸を適切に処理して，作業を記述する必要がある．そこで，物体の埋めるべき凹部と埋めるべきでない凹部分を考慮し，凹凸へ適切な目標線指示を行うための局所凸という概念，及び局所凸生成方法を提案した．次に，直感的な被覆手順の指示方法について検討した．本研究では，人間の大まかな包む指示と被覆の関係を考え，物体と布のどこを重ね合わせるかという人間の被覆の意図を目標線として入力する方法を提案した．本研究は，作業指示を行う手の正確な３次元的な軌跡ではなく，手の軌跡とその軌跡が通過していく物体表面の関係に注目した．そして，デプスセンサとモーションキャプチャセンサを組合せた教示デバイスを用いて，人間の被覆の意図を抽出した．その中では，指示中の手振れの影響を小さくするための目標線逆走防止処理手法とスムージングと間引き処理を合わせた補正処理手法を提案した．最後に，ロボットの動作軌道の生成方法について検討した．本研究では，目標線と把持点から布の動きを表す手先経路を生成する方法と，その手先経路を実行するためのロボット動作の生成方法を提案した．実際のロボットを動かすためには，目標線だけでなく，手先経路や動作指令が必要であり，可動域や物体との干渉を考慮し，右手と左手を用いた布の持ち替えや持ち直しを行わなければならない．これらの情報を生成する上で，目標線が被覆の本質的な情報を保持している．そのため，手先経路・動作指令は自動的に生成可能である．動作生成手法の中では，各操作の布への重力の影響，動作ステップ数やロボットと布の位置関係を考慮した確実性を求め，それを基に生成された動作遷移グラフを用いて，最適な持ち替えや持ち直し操作の組み合わせを計画する方法を提案した．以上，本研究では，物体を布で包むという被覆作業について，ロボット化のための枠組みを提案した．さらに，各課題に対する提案方法を統合し，一連の被覆作業システムとして実装した．これにより，実際に人間の大まかな指示から，目標線を用いて布と物体の関係を記述し，そこから布の動きを表す手先経路，状況に合わせた最適なロボット動作を生成できるようになりロボットによる被覆作業が実現した．電気通信大学201