A Review of Haptic Feedback Teleoperation Systems for Micromanipulation and Microassembly

International audienceThis paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required

Haptic feedback in teleoperation in Micro-and Nano-Worlds.

International audienceRobotic systems have been developed to handle very small objects, but their use remains complex and necessitates long-duration training. Simulators, such as molecular simulators, can provide access to large amounts of raw data, but only highly trained users can interpret the results of such systems. Haptic feedback in teleoperation, which provides force-feedback to an operator, appears to be a promising solution for interaction with such systems, as it allows intuitiveness and flexibility. However several issues arise while implementing teleoperation schemes at the micro-nanoscale, owing to complex force-fields that must be transmitted to users, and scaling differences between the haptic device and the manipulated objects. Major advances in such technology have been made in recent years. This chapter reviews the main systems in this area and highlights how some fundamental issues in teleoperation for micro- and nano-scale applications have been addressed. The chapter considers three types of teleoperation, including: (1) direct (manipulation of real objects); (2) virtual (use of simulators); and (3) augmented (combining real robotic systems and simulators). Remaining issues that must be addressed for further advances in teleoperation for micro-nanoworlds are also discussed, including: (1) comprehension of phenomena that dictate very small object (< 500 micrometers) behavior; and (2) design of intuitive 3-D manipulation systems. Design guidelines to realize an intuitive haptic feedback teleoperation system at the micro-nanoscale level are proposed

Semi-autonomous scheme for pushing micro-objects

-In many microassembly applications, it is often desirable to position and orient polygonal micro-objects lying on a planar surface. Pushing micro-objects using point contact provides more flexibility and less complexity compared to pick and place operation. Due to the fact that in micro-world surface forces are much more dominant than inertial forces and these forces are distributed unevenly, pushing through the center of mass of the micro-object will not yield a pure translational motion. In order to translate a micro-object, the line of pushing should pass through the center of friction. In this paper, a semi-autonomous scheme based on hybrid vision/force feedback is proposed to push microobjects with human assistance using a custom built telemicromanipulation setup to achieve pure translational motion. The pushing operation is divided into two concurrent processes: In one process human operator who acts as an impedance controller alters the velocity of the pusher while in contact with the micro-object through scaled bilateral teleoperation with force feedback. In the other process, the desired line of pushing for the micro-object is determined continuously using visual feedback procedures so that it always passes through the varying center of friction. Experimental results are demonstrated to prove nanoNewton range force sensing, scaled bilateral teleoperation with force feedback and pushing microobjects

Scaled bilateral teleoperation using discrete-time sliding mode controller

In this paper, the design of a discrete-time slidingmode controller based on Lyapunov theory is presented along with a robust disturbance observer and is applied to a piezostage for high-precision motion. A linear model of a piezostage was used with nominal parameters to compensate the disturbance acting on the system in order to achieve nanometer accuracy. The effectiveness of the controller and disturbance observer is validated in terms of closed-loop position performance for nanometer references. The control structure has been applied to a scaled bilateral structure for the custom-built telemicromanipulation setup. A piezoresistive atomic force microscope cantilever with a built-in Wheatstone bridge is utilized to achieve the nanonewtonlevel interaction forces between the piezoresistive probe tip and the environment. Experimental results are provided for the nanonewton-range force sensing, and good agreement between the experimental data and the theoretical estimates has been demonstrated. Force/position tracking and transparency between the master and the slave has been clearly demonstrated after necessary scalin

Disturbance observer based bilateral control systems

Bilateral teleoperation is becoming one of the far reaching application areas of robotics science. Enabling a human operator the ability to reach and manipulate a remote location will be possible with the various applications of bilateral control. In that sense, ideal bilateral control allows extension of a person's sensing to a remote environment by a master slave structure. So, the coupled goals of bilateral control is to enforce the slave system track the motion generated on the master system and to reflect the forces from the slave system. This thesis investigates the current state of the art in bilateral teleoperation. For that purpose, design and analysis of bilateral control is made based on the use of disturbance observers. First, a known control structure is investigated in the context of acceleration control. Following this, a case study is made to show a different application of bilateral control, namely grasping force control. Performance improvement in bilateral control is also studied and correspondingly, a novel functional observer is proposed for better estimation of velocity, acceleration and disturbance. In the second half of the thesis, bilateral control with time delay is realized. Design is made via separating the position and force into two different loops. For position control under time delay, a previously proposed control scheme is used in which use of communication disturbance observer with convergence terms was discussed. Observation made about the divergence from the master reference under contact motion is analyzed and a model following control structure is proposed to eliminate the remaining disturbance from the slave plant. For force control under time delay, first the response of a local controller is analyzed. In order to improve the system transparency, a new method is proposed in which environment stiffness was used for force control loop rather than the delayed slave force. In this structure, estimation of environment stiffness was made via an indirect adaptive control scheme. The analyzed structures were also tested experimentally under a master slave system consisting of 1 DOF linear motors. Experiments show the validity of the contributions made for bilateral control with and without time delay

Online Recognition of Environment Properties by Using Bilateral Control

The topic of this thesis is identification of the mechanical impedance of an unknown environment. Through the use of bilateral control based on DOB and RFOB structures, position, speed and force information are gathered and analyzed while performing continuous contact with the environment. The nonlinear Hunt-Crossley model is preferred over the classic Kelvin-Voigt model. Particular attention is given to the precise recognition of contact and the detection of an occurring deformation.ope

Micromanipulation-force feedback pushing

In micromanipulation applications, it is often desirable to position and orient polygonal micro-objects lying on a planar surface. Pushing micro-objects using point contact provides more flexibility and less complexity compared to pick and place operation. Due to the fact that in micro-world surface forces are much more dominant than inertial forces and these forces are distributed unevenly, pushing through the center of mass of the micro-object will not yield a pure translational motion. In order to translate a micro-object, the line of pushing should pass through the center of friction. Moreover, due to unexpected nature of the frictional forces between the micro-object and substrate, the maximum force applied to the micro-object needs to be limited to prevent any damage either to the probe or micro-object. In this dissertation, a semi-autonomous manipulation scheme is proposed to push microobjects with human assistance using a custom built tele-micromanipulation setup to achieve pure translational motion. The pushing operation can be divided into two concurrent processes: In one process human operator who acts as an impedance controller to switch between force-position controllers and alters the velocity of the pusher while in contact with the micro-object through scaled bilateral teleoperation with force feedback. In the other process, the desired line of pushing for the micro-object is determined continuously so that it always passes through the varying center of friction. Visual feedback procedures are adopted to align the resultant velocity vector at the contact point to pass through the center of friction in order to achieve pure translational motion of the micro-object. Experimental results are demonstrated to prove the effectiveness of the proposed controller along with nanometer scale position control, nano-Newton range force sensing, scaled bilateral teleoperation with force feedback

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges

Bilateral control: a sliding mode control approach

Bilateral control is bi-directional control of force-position between two systems connected by a communication link. It is typically used for teleoperation with forcefeedback, such that the master system is handled by an operator. Motions of the operator are fed forward to the slave system, generally remote to the operator and forces encountered are fed back to the master system, enabling a telepresence of the operator in the remote environment. The necessity of bilateral control lies in its applicability to the tasks that cannot be handled by autonomous manipulators and/or reached by human beings. Main issues of consideration for bilateral control, namely transparency, scaling and time delay, are addressed and two discrete-time sliding-mode approaches are presented as solutions to highly transparent bilateral controllers that support scaling. First approach has a force-hybrid architecture, where the cascaded sliding mode hybrid force/position controller on the slave side reacts to the external forces directly. Therefore, it provides a protection (reflex) mechanism on the slave side to large external forces, that the operator cannot respond in time due to the time delay. Second approach has a decentralized nature. Virtual systems are devised by a linear transformation from the plant space to the task space and sliding mode control has been applied to those virtual systems, hence sides of bilateral control are interchangable. The decentralized structure of the controller makes it possible to generalize the problem to a coordination and/or cooperation of more than two plants. High precision has been achieved on experiments for both approaches designed and discussed in detail