Novel parameter estimation schemes in microsystems

This paper presents two novel estimation methods that are designed to enhance our ability of observing, positioning, and physically transforming the objects and/or biological structures in micromanipulation tasks. In order to effectively monitor and position the microobjects, an online calibration method with submicron precision via a recursive least square solution is presented. To provide the adequate information to manipulate the biological structures without damaging the cell or tissue during an injection, a nonlinear spring-mass-damper model is introduced and mechanical properties of a zebrafish embryo are obtained. These two methods are validated on a microassembly workstation and the results are evaluated quantitatively

Robotic Micromanipulation and Microassembly using Mono-view and Multi-scale visual servoing.

International audienceThis paper investigates sequential robotic micromanipulation and microassembly in order to build 3-D microsystems and devices. A mono-view and multiple scale 2-D visual control scheme is implemented for that purpose. The imaging system used is a photon video microscope endowed with an active zoom enabling to work at multiple scales. It is modelled by a non-linear projective method where the relation between the focal length and the zoom factor is explicitly established. A distributed robotic system (xy system, z system) with a twofingers gripping system is used in conjunction with the imaging system. The results of experiments demonstrate the relevance of the proposed approaches. The tasks were performed with the following accuracy: 1.4 m for the positioning error, and 0.5 for the orientation error

Workshop on "Robotic assembly of 3D MEMS".

Proceedings of a workshop proposed in IEEE IROS'2007.The increase of MEMS' functionalities often requires the integration of various technologies used for mechanical, optical and electronic subsystems in order to achieve a unique system. These different technologies have usually process incompatibilities and the whole microsystem can not be obtained monolithically and then requires microassembly steps. Microassembly of MEMS based on micrometric components is one of the most promising approaches to achieve high-performance MEMS. Moreover, microassembly also permits to develop suitable MEMS packaging as well as 3D components although microfabrication technologies are usually able to create 2D and "2.5D" components. The study of microassembly methods is consequently a high stake for MEMS technologies growth. Two approaches are currently developped for microassembly: self-assembly and robotic microassembly. In the first one, the assembly is highly parallel but the efficiency and the flexibility still stay low. The robotic approach has the potential to reach precise and reliable assembly with high flexibility. The proposed workshop focuses on this second approach and will take a bearing of the corresponding microrobotic issues. Beyond the microfabrication technologies, performing MEMS microassembly requires, micromanipulation strategies, microworld dynamics and attachment technologies. The design and the fabrication of the microrobot end-effectors as well as the assembled micro-parts require the use of microfabrication technologies. Moreover new micromanipulation strategies are necessary to handle and position micro-parts with sufficiently high accuracy during assembly. The dynamic behaviour of micrometric objects has also to be studied and controlled. Finally, after positioning the micro-part, attachment technologies are necessary

Robotic micromanipulation for microassembly : modelling by sequencial function chart and achievement by multiple scale visual servoings.

International audienceThe paper investigates robotic assembly by focusing on the manipulation of microparts. This task is formalized through the notion of basic tasks which are organized in a logical sequence represented by a function chart and interpreted as the model of the behavior of the experimental setup. The latter includes a robotic system, a gripping system, an imaging system, and a clean environment. The imaging system is a photon videomicroscope able to work at multiple scales. It is modelled by a linear projective model where the relation between the scale factor and the magnification or zoom is explicitly established. So, the usual visual control law is modified in order to take into account this relation. The manipulation of some silicon microparts (400 μm×400 μm×100 μm) by means of a distributed robotic system (xyθ system, ϕz system), a two-finger gripping system and a controllable zoom and focus videomicroscope shows the relevance of the concepts. The 30 % of failure rate comes mainly from the physical phenomena (electrostatic and capillary forces) instead of the accuracy of control or the occultations of microparts

MRF Stereo Matching with Statistical Estimation of Parameters

For about the last ten years, stereo matching in computer vision has been treated as a combinatorial optimization problem. Assuming that the points in stereo images form a Markov Random Field (MRF), a variety of combinatorial optimization algorithms has been developed to optimize their underlying cost functions. In many of these algorithms, the MRF parameters of the cost functions have often been manually tuned or heuristically determined for achieving good performance results. Recently, several algorithms for statistical, hence, automatic estimation of the parameters have been published. Overall, these algorithms perform well in labeling, but they lack in performance for handling discontinuity in labeling along the surface borders. In this dissertation, we develop an algorithm for optimization of the cost function with automatic estimation of the MRF parameters – the data and smoothness parameters. Both the parameters are estimated statistically and applied in the cost function with support of adaptive neighborhood defined based on color similarity. With the proposed algorithm, discontinuity handling with higher consistency than of the existing algorithms is achieved along surface borders. The data parameters are pre-estimated from one of the stereo images by applying a hypothesis, called noise equivalence hypothesis, to eliminate interdependency between the estimations of the data and smoothness parameters. The smoothness parameters are estimated applying a combination of maximum likelihood and disparity gradient constraint, to eliminate nested inference for the estimation. The parameters for handling discontinuities in data and smoothness are defined statistically as well. We model cost functions to match the images symmetrically for improved matching performance and also to detect occlusions. Finally, we fill the occlusions in the disparity map by applying several existing and proposed algorithms and show that our best proposed segmentation based least squares algorithm performs better than the existing algorithms. We conduct experiments with the proposed algorithm on publicly available ground truth test datasets provided by the Middlebury College. Experiments show that results better than the existing algorithms’ are delivered by the proposed algorithm having the MRF parameters estimated automatically. In addition, applying the parameter estimation technique in existing stereo matching algorithm, we observe significant improvement in computational time

Micromanipulation and Micro-Assembly Systems.

International audienceThe needs to manipulate micrometer sized objects keeps growing and concerns numerous and various fields like microsystems (MEMS1 and MOEMS2), micromechanics, optics, biology or pharmacy. The specificities of size, material, geometry and consistency of manipulated micro-objects, their surrounding, the kind of task to perform and the free size are all the more specific parameters that strongly influence the design and working of micromanipulation and micro-assembly systems. These systems are widely developing because they correspond both to industrial needs and really challenging scientific problematics. For these reasons, the present paper aimed at dealing with a review that mainly focuses on systems recently developed to assemble small series of microcomponents. The paper especially points out different solutions of carriers structures, gripping principles, sensors, other peri-microrobotic systems and control systems presenting the main solution and justifying their use and interest

Visual Servoing Schemes for Automatic Nanopositioning Under Scanning Electron Microscope.

International audienceThis paper presents two visual servoing approaches for nanopositioning in a scanning electron microscope (SEM). The first approach uses the total pixel intensities of an image as visual measurements for designing the control law. The positioning error and the platform control are directly linked with the intensity variations. The second approach is a frequency domain method that uses Fourier transform to compute the relative motion between images. In this case, the control law is designed to minimize the error i.e. the 2D motion between current and desired images by controlling the positioning platform movement. Both methods are validated at different experimental conditions for a task of positioning silicon microparts using a piezo-positioning platform. The obtained results demonstrate the efficiency and robustness of the developed methods

Design, fabrication and characterization of a flexible system based on thermal glue for in AIR and in SEM microassembly.

International audienceThis paper presents the design, fabrication and characterization of a device able to exchange the tip part (so-called the tools) of a two fingered microgripper. The principle of this tool changer is based on the use of a thermal glue whose state (liquid or solid) is changed by heating or cooling. Several kinds of pairs of tools have been designed. The suitable pair of tools can be chosen according to the size, shape and material of the object to manipulate. The tool changer enables one to perform a sequence of elementary micromanipulation tasks (i.e. an assembly sequence) by using only one gripper mounted on only one manipulator. The tool changer has been automated and successfully tested in air and in the vacuum chamber of a Scanning Electron Microscope (SEM). It brings flexibility to the micromanipulation cell and contributes to reduce the costs, the used space and experimentations time for micromanipulations in the SEM. The assembly of a ball bearing (the balls are 200 ¹m in diameter) has been successfully tested using the microgripper equipped with the tool changer in a SEM. This tool changer has been designed for a microgripper but can be easily adapted to lots of other kinds of systems

Micro-object pose estimation with sim-to-real transfer learning using small dataset

International audience<span style="color: rgb(34, 34, 34); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Oxygen-Sans, Ubuntu, Cantarell, &quot;Helvetica Neue&quot;, sans-serif; font-size: 18px;"&gtThree-dimensional (3D) pose estimation of micro/nano-objects isessential for the implementation of automatic manipulation inmicro/nano-robotic systems. However, out-of-plane pose estimationof a micro/nano-object is challenging, since the images aretypically obtained in 2D using a scanning electron microscope (SEM)or an optical microscope (OM). Traditional deep learning basedmethods require the collection of a large amount of labeled datafor model training to estimate the 3D pose of an object from amonocular image. Here we present a sim-to-real learning-to-matchapproach for 3D pose estimation of micro/nano-objects. Instead ofcollecting large training datasets, simulated data is generated toenlarge the limited experimental data obtained in practice, whilethe domain gap between the generated and experimental data isminimized via image translation based on a generative adversarialnetwork (GAN) model. A learning-to-match approach is used to mapthe generated data and the experimental data to a low-dimensionalspace with the same data distribution for different pose labels,which ensures effective feature embedding. Combining the labeleddata obtained from experiments and simulations, a new trainingdataset is constructed for robust pose estimation. The proposedmethod is validated with images from both SEM and OM, facilitatingthe development of closed-loop control of micro/nano-objects withcomplex shapes in micro/nano-robotic systems.</span&g

Overview of out of plane MEMS assembly techniques.

International audienceThis paper deals with a synthesis of the activities of the French FEMTO-ST institute in the field of robotic microassembly. It deals with the tridimensional assembly of objects whose typical size is from 10 microns to 400 microns. We are especially focusing on the automation of micro-assembly based on several principles. Closed loop control based on microvision has been studied and applied on the fully automatic assembly of several 400 microns objects. Force control has been also analyzed and is proposed for optical Microsystems assembly. At least, open loop trajectories of 40 microns objects with a throughput of 1800 unit per hour have been achieved. Scientific and technological aspects and industrial relevance will be presented