Novel estimation and control techniques in micromanipulation using vision and force feedback

With the recent advances in the fields of micro and nanotechnology, there has been growing interest for complex micromanipulation and microassembly strategies. Despite the fact that many commercially available micro devices such as the key components in automobile airbags, ink-jet printers and projection display systems are currently produced in a batch technique with little assembly, many other products such as read/write heads for hard disks and fiber optics assemblies require flexible precision assemblies. Furthermore, many biological micromanipulations such as invitro-fertilization, cell characterization and treatment rely on the ability of human operators. Requirement of high-precision, repeatable and financially viable operations in these tasks has given rise to the elimination of direct human involvement, and autonomy in micromanipulation and microassembly. In this thesis, a fully automated dexterous micromanipulation strategy based on vision and force feedback is developed. More specifically, a robust vision based control architecture is proposed and implemented to compensate errors due to the uncertainties about the position, behavior and shape of the microobjects to be manipulated. Moreover, novel estimators are designed to identify the system and to characterize the mechanical properties of the biological structures through a synthesis of concepts from the computer vision, estimation and control theory. Estimated mechanical parameters are utilized to reconstruct the imposed force on a biomembrane and to provide the adequate information to control the position, velocity and acceleration of the probe without damaging the cell/tissue during an injection task

Mechanical Manipulation and Characterization of Biological Cells

Mechanical manipulation and characterization of an individual biological cell is currently one of the most exciting research areas in the field of medical robotics. Single cell manipulation is an important process in intracytoplasmic sperm injection (ICSI), pro-nuclei DNA injection, gene therapy, and other biomedical areas. However, conventional cell manipulation requires long training and the success rate depends on the experience of the operator. The goal of this research is to address the drawbacks of conventional cell manipulation by using force and vision feedback for cell manipulation tasks. We hypothesize that force feedback plays an important role in cell manipulation and possibly helps in cell characterization. This dissertation will summarize our research on: 1) the development of force and vision feedback interface for cell manipulation, 2) human subject studies to evaluate the addition of force feedback for cell injection tasks, 3) the development of haptics-enabled atomic force microscope system for cell indentation tasks, 4) appropriate analytical model for characterizing the mechanical property of mouse embryonic stem cells (mESC) and 5) several indentation studies on mESC to determine the mechanical property of undifferentiated and early differentiating (6 days under differentiation conditions) mESC. Our experimental results on zebrafish egg cells show that a system with force feedback capability when combined with vision feedback can lead to potentially higher success rates in cell injection tasks. Using this information, we performed experiments on mESC using the AFM to understand their characteristics in the undifferentiated pluripotent state as well as early differentiating state. These experiments were done on both live as well as fixed cells to understand the correlation between the two during cell indentation studies. Our results show that the mechanical property of undifferentiated mESC differs from early differentiating (6th day) mESC in both live and fixed cells. Thus, we hypothesize that mechanical characterization studies will potentially pave the way for developing a high throughput system with force feedback capability, to understand and predict the differentiation path a particular pluripotent cell will follow. This finding could also be used to develop improved methods of targeted cellular differentiation of stem cells for therapeutic and regenerative medicine

Hybrid Automatic Visual Servoing Scheme using Defocus Information for 6-DoF Micropositioning

International audience— Direct photometric visual servoing uses only the pure image information as a visual feature, instead of using classic geometric features such as points or lines. It was demonstrated efficiently in 6 degrees of freedom (DoF) positioning. However, in micro-scale, using only image intensity as a visual feature performs unsatisfactorily in cases where the photometric variation is low, such as motions along vision sensor's focal axis under a high magnification. In order to improve the performance and accuracy in those cases, an approach using hybrid visual features is proposed in this paper. Image gradient is employed as a visual feature on z axis while image intensity is used on the other 5 DoFs to control the motion. A 6-DoF micro-positioning task is accomplished by this hybrid visual servoing scheme. The experimental results obtained on a parallel positioning micro-stage under a digital microscope show the robustness and efficiency of the proposed method