Positioning and trajectory following tasks in microsystems using model free visual servoing

In this paper, we explore model free visual servoing algorithms by experimentally evaluating their performances for various tasks performed on a microassembly workstation developed in our lab. Model free or so called uncalibrated visual servoing does not need the system calibration (microscope-camera-micromanipulator) and the model of the observed scene. It is robust to parameter changes and disturbances. We tested its performance in point-to-point positioning and various trajectory following tasks. Experimental results validate the utility of model free visual servoing in microassembly tasks

Hybrid optical and magnetic manipulation of microrobots

Microrobotic systems have the potential to provide precise manipulation on cellular level for diagnostics, drug delivery and surgical interventions. These systems vary from tethered to untethered microrobots with sizes below a micrometer to a few microns. However, their main disadvantage is that they do not have the same capabilities in terms of degrees-of-freedom, sensing and control as macroscale robotic systems. In particular, their lack of on-board sensing for pose or force feedback, their control methods and interface for automated or manual user control are limited as well as their geometry has few degrees-of-freedom making three-dimensional manipulation more challenging. This PhD project is on the development of a micromanipulation framework that can be used for single cell analysis using the Optical Tweezers as well as a combination of optical trapping and magnetic actuation for recon gurable microassembly. The focus is on untethered microrobots with sizes up to a few tens of microns that can be used in enclosed environments for ex vivo and in vitro medical applications. The work presented investigates the following aspects of microrobots for single cell analysis: i) The microfabrication procedure and design considerations that are taken into account in order to fabricate components for three-dimensional micromanipulation and microassembly, ii) vision-based methods to provide 6-degree-offreedom position and orientation feedback which is essential for closed-loop control, iii) manual and shared control manipulation methodologies that take into account the user input for multiple microrobot or three-dimensional microstructure manipulation and iv) a methodology for recon gurable microassembly combining the Optical Tweezers with magnetic actuation into a hybrid method of actuation for microassembly.Open Acces

Model-based vs. model-free visual servoing: A Performance evaluation in microsystems

In this paper, model-based and model-free image based visual servoing (VS) approaches are implemented on a microassembly workstation, and their regulation and tracking performances are evaluated. A precise image based VS relies on computation of the image jacobian. In the model-based visual servoing, the image Jacobian is computed via calibrating the optical system. Precisely calibrated model based VS promises better positioning and tracking performance than the model-free approach. However, in the model-free approach, optical system calibration is not required due to the dynamic Jacobian estimation, thus it has the advantage of adapting to the different operating modes

Semi-autonomous scheme for pushing micro-objects

-In many microassembly applications, it is often desirable to position and orient polygonal micro-objects lying on a planar surface. Pushing micro-objects using point contact provides more flexibility and less complexity compared to pick and place operation. Due to the fact that in micro-world surface forces are much more dominant than inertial forces and these forces are distributed unevenly, pushing through the center of mass of the micro-object will not yield a pure translational motion. In order to translate a micro-object, the line of pushing should pass through the center of friction. In this paper, a semi-autonomous scheme based on hybrid vision/force feedback is proposed to push microobjects with human assistance using a custom built telemicromanipulation setup to achieve pure translational motion. The pushing operation is divided into two concurrent processes: In one process human operator who acts as an impedance controller alters the velocity of the pusher while in contact with the micro-object through scaled bilateral teleoperation with force feedback. In the other process, the desired line of pushing for the micro-object is determined continuously using visual feedback procedures so that it always passes through the varying center of friction. Experimental results are demonstrated to prove nanoNewton range force sensing, scaled bilateral teleoperation with force feedback and pushing microobjects

Microrobots for wafer scale microfactory: design fabrication integration and control.

Future assembly technologies will involve higher automation levels, in order to satisfy increased micro scale or nano scale precision requirements. Traditionally, assembly using a top-down robotic approach has been well-studied and applied to micro-electronics and MEMS industries, but less so in nanotechnology. With the bloom of nanotechnology ever since the 1990s, newly designed products with new materials, coatings and nanoparticles are gradually entering everyone’s life, while the industry has grown into a billion-dollar volume worldwide. Traditionally, nanotechnology products are assembled using bottom-up methods, such as self-assembly, rather than with top-down robotic assembly. This is due to considerations of volume handling of large quantities of components, and the high cost associated to top-down manipulation with the required precision. However, the bottom-up manufacturing methods have certain limitations, such as components need to have pre-define shapes and surface coatings, and the number of assembly components is limited to very few. For example, in the case of self-assembly of nano-cubes with origami design, post-assembly manipulation of cubes in large quantities and cost-efficiency is still challenging. In this thesis, we envision a new paradigm for nano scale assembly, realized with the help of a wafer-scale microfactory containing large numbers of MEMS microrobots. These robots will work together to enhance the throughput of the factory, while their cost will be reduced when compared to conventional nano positioners. To fulfill the microfactory vision, numerous challenges related to design, power, control and nanoscale task completion by these microrobots must be overcome. In this work, we study three types of microrobots for the microfactory: a world’s first laser-driven micrometer-size locomotor called ChevBot，a stationary millimeter-size robotic arm, called Solid Articulated Four Axes Microrobot (sAFAM), and a light-powered centimeter-size crawler microrobot called SolarPede. The ChevBot can perform autonomous navigation and positioning on a dry surface with the guidance of a laser beam. The sAFAM has been designed to perform nano positioning in four degrees of freedom, and nanoscale tasks such as indentation, and manipulation. And the SolarPede serves as a mobile workspace or transporter in the microfactory environment

Modeling, simulation and control of microrobots for the microfactory.

Future assembly technologies will involve higher levels of automation in order to satisfy increased microscale or nanoscale precision requirements. Traditionally, assembly using a top-down robotic approach has been well-studied and applied to the microelectronics and MEMS industries, but less so in nanotechnology. With the boom of nanotechnology since the 1990s, newly designed products with new materials, coatings, and nanoparticles are gradually entering everyone’s lives, while the industry has grown into a billion-dollar volume worldwide. Traditionally, nanotechnology products are assembled using bottom-up methods, such as self-assembly, rather than top-down robotic assembly. This is due to considerations of volume handling of large quantities of components, and the high cost associated with top-down manipulation requiring precision. However, bottom-up manufacturing methods have certain limitations, such as components needing to have predefined shapes and surface coatings, and the number of assembly components being limited to very few. For example, in the case of self-assembly of nano-cubes with an origami design, post-assembly manipulation of cubes in large quantities and cost-efficiency is still challenging. In this thesis, we envision a new paradigm for nanoscale assembly, realized with the help of a wafer-scale microfactory containing large numbers of MEMS microrobots. These robots will work together to enhance the throughput of the factory, while their cost will be reduced when compared to conventional nanopositioners. To fulfill the microfactory vision, numerous challenges related to design, power, control, and nanoscale task completion by these microrobots must be overcome. In this work, we study two classes of microrobots for the microfactory: stationary microrobots and mobile microrobots. For the stationary microrobots in our microfactory application, we have designed and modeled two different types of microrobots, the AFAM (Articulated Four Axes Microrobot) and the SolarPede. The AFAM is a millimeter-size robotic arm working as a nanomanipulator for nanoparticles with four degrees of freedom, while the SolarPede is a light-powered centimeter-size robotic conveyor in the microfactory. For mobile microrobots, we have introduced the world’s first laser-driven micrometer-size locomotor in dry environments, called ChevBot to prove the concept of the motion mechanism. The ChevBot is fabricated using MEMS technology in the cleanroom, following a microassembly step. We showed that it can perform locomotion with pulsed laser energy on a dry surface. Based on the knowledge gained with the ChevBot, we refined tits fabrication process to remove the assembly step and increase its reliability. We designed and fabricated a steerable microrobot, the SerpenBot, in order to achieve controllable behavior with the guidance of a laser beam. Through modeling and experimental study of the characteristics of this type of microrobot, we proposed and validated a new type of deep learning controller, the PID-Bayes neural network controller. The experiments showed that the SerpenBot can achieve closed-loop autonomous operation on a dry substrate

Novel estimation and control techniques in micromanipulation using vision and force feedback

With the recent advances in the fields of micro and nanotechnology, there has been growing interest for complex micromanipulation and microassembly strategies. Despite the fact that many commercially available micro devices such as the key components in automobile airbags, ink-jet printers and projection display systems are currently produced in a batch technique with little assembly, many other products such as read/write heads for hard disks and fiber optics assemblies require flexible precision assemblies. Furthermore, many biological micromanipulations such as invitro-fertilization, cell characterization and treatment rely on the ability of human operators. Requirement of high-precision, repeatable and financially viable operations in these tasks has given rise to the elimination of direct human involvement, and autonomy in micromanipulation and microassembly. In this thesis, a fully automated dexterous micromanipulation strategy based on vision and force feedback is developed. More specifically, a robust vision based control architecture is proposed and implemented to compensate errors due to the uncertainties about the position, behavior and shape of the microobjects to be manipulated. Moreover, novel estimators are designed to identify the system and to characterize the mechanical properties of the biological structures through a synthesis of concepts from the computer vision, estimation and control theory. Estimated mechanical parameters are utilized to reconstruct the imposed force on a biomembrane and to provide the adequate information to control the position, velocity and acceleration of the probe without damaging the cell/tissue during an injection task

Analysis of CAD Model-based Visual Tracking for Microassembly using a New Block Set for MATLAB/Simulink.

International audienceMicroassembly is an innovative alternative to the microfabrication process of MOEMS which is quite complex. It usually implies the use of microrobots controlled by an operator. The reliability of this approach has been already confirmed for the micro-optical technologies. However, the characterization of assemblies has shown that the operator is the main source of inaccuracies in the teleoperated microassembly. Therefore, there is a great interest in automating the microassembly process. One of the constraints of automation in microscale is the lack of high precision sensors capable to provide the full information about the object position. Thus, the usage of visual-based feedback represents a very promising approach allowing to automate the microassembly process. The purpose of this paper is to characterize the techniques of object position estimation based on the visual data, i.e. visual tracking techniques from the ViSP library. These algorithms allows to get the 3D object pose using a single view of the scene and the CAD model of the object. The performance of three main types of model-based trackers is analyzed and quantified: edge-based, texture-based and hybrid tracker. The problems of visual tracking in microscale are discussed. The control of the micromanipulation station used in the framework of our project is performed using a new Simulink block set. Experimental results are shown and demonstrate thepossibility to obtain the repeatability below 1 micrometer

Cyber-Physical Systems for Micro-/Nano-assembly Operations: a Survey

Abstract Purpose of Review Latest requirements of the global market force manufacturing systems to a change for a new production paradigm (Industry 4.0). Cyber-Physical Systems (CPS) appear as a solution to be deployed in different manufacturing fields, especially those with high added value and technological complexity, high product variants, and short time to market. In this sense, this paper aims at reviewing the introduction level of CPS technologies in micro/nano-manufacturing and how these technologies could cope with these challenging manufacturing requirements. Recent Findings The introduction of CPS is still in its infancy on many industrial applications, but it actually demonstrates its potential to support future manufacturing paradigm. However, only few research works in micro/nano-manufacturing considered CPS frameworks, since the concept barely appeared a decade ago. Summary Some contributions have revealed the potential of CPS technologies to improve manufacturing performance which may be scaled to the micro/nano-manufacturing. IoT-based frameworks with VR/AR technologies allow distributed and collaborative systems, or agent-based architectures with advance algorithm implementations that improve the flexibility and performance of micro-/nano-assembly operations. Future research of CPS in micro-/nano-assembly operations should be followed by more studies of its technical deployment showing its implications under other perspectives, i.e. sustainable, economic, and social point of views, to take full advance of all its features

Gluing free assembly of an advanced 3D structure using visual servoing.

International audienceThe paper deals with robotic assembly of 5 parts by their U-grooves to achieve stables 3D MEMS, without any use of soldering effect. The parts and their grooves measure 400 m 400 m 100 m 1.5 m and 100 m 100 m 100 m 1.5 m leading to an assembly clearance ranging from -3 and +3 m. Two visual servo approaches are used simultaneously: 2D visual servo for gripping and release of parts and 3D visual servo for displacement of parts. The results of experiments are presented and analyzed