Hybrid optical and magnetic manipulation of microrobots

Microrobotic systems have the potential to provide precise manipulation on cellular level for diagnostics, drug delivery and surgical interventions. These systems vary from tethered to untethered microrobots with sizes below a micrometer to a few microns. However, their main disadvantage is that they do not have the same capabilities in terms of degrees-of-freedom, sensing and control as macroscale robotic systems. In particular, their lack of on-board sensing for pose or force feedback, their control methods and interface for automated or manual user control are limited as well as their geometry has few degrees-of-freedom making three-dimensional manipulation more challenging. This PhD project is on the development of a micromanipulation framework that can be used for single cell analysis using the Optical Tweezers as well as a combination of optical trapping and magnetic actuation for recon gurable microassembly. The focus is on untethered microrobots with sizes up to a few tens of microns that can be used in enclosed environments for ex vivo and in vitro medical applications. The work presented investigates the following aspects of microrobots for single cell analysis: i) The microfabrication procedure and design considerations that are taken into account in order to fabricate components for three-dimensional micromanipulation and microassembly, ii) vision-based methods to provide 6-degree-offreedom position and orientation feedback which is essential for closed-loop control, iii) manual and shared control manipulation methodologies that take into account the user input for multiple microrobot or three-dimensional microstructure manipulation and iv) a methodology for recon gurable microassembly combining the Optical Tweezers with magnetic actuation into a hybrid method of actuation for microassembly.Open Acces

PLANNING FOR AUTOMATED OPTICAL MICROMANIPULATION OF BIOLOGICAL CELLS

Optical tweezers (OT) can be viewed as a robot that uses a highly focused laser beam for precise manipulation of biological objects and dielectric beads at micro-scale. Using holographic optical tweezers (HOT) multiple optical traps can be created to allow several operations in parallel. Moreover, due to the non-contact nature of manipulation OT can be potentially integrated with other manipulation techniques (e.g. microfluidics, acoustics, magnetics etc.) to ensure its high throughput. However, biological manipulation using OT suffers from two serious drawbacks: (1) slow manipulation due to manual operation and (2) severe effects on cell viability due to direct exposure of laser. This dissertation explores the problem of autonomous OT based cell manipulation in the light of addressing the two aforementioned limitations. Microfluidic devices are well suited for the study of biological objects because of their high throughput. Integrating microfluidics with OT provides precise position control as well as high throughput. An automated, physics-aware, planning approach is developed for fast transport of cells in OT assisted microfluidic chambers. The heuristic based planner employs a specific cost function for searching over a novel state-action space representation. The effectiveness of the planning algorithm is demonstrated using both simulation and physical experiments in microfluidic-optical tweezers hybrid manipulation setup. An indirect manipulation approach is developed for preventing cells from high intensity laser. Optically trapped inert microspheres are used for manipulating cells indirectly either by gripping or pushing. A novel planning and control approach is devised to automate the indirect manipulation of cells. The planning algorithm takes the motion constraints of the gripper or pushing formation into account to minimize the manipulation time. Two different types of cells (Saccharomyces cerevisiae and Dictyostelium discoideum) are manipulated to demonstrate the effectiveness of the indirect manipulation approach

Haptic feedback in teleoperation in Micro-and Nano-Worlds.

International audienceRobotic systems have been developed to handle very small objects, but their use remains complex and necessitates long-duration training. Simulators, such as molecular simulators, can provide access to large amounts of raw data, but only highly trained users can interpret the results of such systems. Haptic feedback in teleoperation, which provides force-feedback to an operator, appears to be a promising solution for interaction with such systems, as it allows intuitiveness and flexibility. However several issues arise while implementing teleoperation schemes at the micro-nanoscale, owing to complex force-fields that must be transmitted to users, and scaling differences between the haptic device and the manipulated objects. Major advances in such technology have been made in recent years. This chapter reviews the main systems in this area and highlights how some fundamental issues in teleoperation for micro- and nano-scale applications have been addressed. The chapter considers three types of teleoperation, including: (1) direct (manipulation of real objects); (2) virtual (use of simulators); and (3) augmented (combining real robotic systems and simulators). Remaining issues that must be addressed for further advances in teleoperation for micro-nanoworlds are also discussed, including: (1) comprehension of phenomena that dictate very small object (< 500 micrometers) behavior; and (2) design of intuitive 3-D manipulation systems. Design guidelines to realize an intuitive haptic feedback teleoperation system at the micro-nanoscale level are proposed

A robot-assisted acoustofluidic end effector

Liquid manipulation is the foundation of most laboratory processes. For macroscale liquid handling, both do-it-yourself and commercial robotic systems are available; however, for microscale, reagents are expensive and sample preparation is difficult. Over the last decade, lab-on-a-chip (LOC) systems have come to serve for microscale liquid manipulation; however, lacking automation andmulti-functionality. Despite their potential synergies, each has grown separately and no suitable interface yet exists to link macro-level robotics with micro-level LOC or microfluidic devices. Here, we present a robot-assisted acoustofluidic end effector (RAEE) system, comprising a robotic arm and an acoustofluidic end effector, that combines robotics and microfluidic functionalities.We further carried out fluid pumping, particle and zebrafish embryo trapping, and mobile mixing of complex viscous liquids. Finally, we pre-programmed the RAEE to perform automatedmixing of viscous liquids in well plates, illustrating its versatility for the automatic execution of chemical processes

A robot-assisted acoustofluidic end effector

Liquid manipulation is the foundation of most laboratory processes. For macroscale liquid handling, both do-it-yourself and commercial robotic systems are available; however, for microscale, reagents are expensive and sample preparation is difficult. Over the last decade, lab-on-a-chip (LOC) systems have come to serve for microscale liquid manipulation; however, lacking automation and multi-functionality. Despite their potential synergies, each has grown separately and no suitable interface yet exists to link macro-level robotics with micro-level LOC or microfluidic devices. Here, we present a robot-assisted acoustofluidic end effector (RAEE) system, comprising a robotic arm and an acoustofluidic end effector, that combines robotics and microfluidic functionalities. We further carried out fluid pumping, particle and zebrafish embryo trapping, and mobile mixing of complex viscous liquids. Finally, we pre-programmed the RAEE to perform automated mixing of viscous liquids in well plates, illustrating its versatility for the automatic execution of chemical processes

OPTIMAL CONTROL OF OBJECTS ON THE MICRO- AND NANO-SCALE BY ELECTROKINETIC AND ELECTROMAGNETIC MANIPULATION: FOR BIO-SAMPLE PREPARATION, QUANTUM INFORMATION DEVICES AND MAGNETIC DRUG DELIVERY

In this thesis I show achievements for precision feedback control of objects inside micro-fluidic systems and for magnetically guided ferrofluids. Essentially, this is about doing flow control, but flow control on the microscale, and further even to nanoscale accuracy, to precisely and robustly manipulate micro and nano-objects (i.e. cells and quantum dots). Target applications include methods to miniaturize the operations of a biological laboratory (lab-on-a-chip), i.e. presenting pathogens to on-chip sensing cells or extracting cells from messy bio-samples such as saliva, urine, or blood; as well as non-biological applications such as deterministically placing quantum dots on photonic crystals to make multi-dot quantum information systems. The particles are steered by creating an electrokinetic fluid flow that carries all the particles from where they are to where they should be at each time step. The control loop comprises sensing, computation, and actuation to steer particles along trajectories. Particle locations are identified in real-time by an optical system and transferred to a control algorithm that then determines the electrode voltages necessary to create a flow field to carry all the particles to their next desired locations. The process repeats at the next time instant. I address following aspects of this technology. First I explain control and vision algorithms for steering single and multiple particles, and show extensions of these algorithms for steering in three dimensional (3D) spaces. Then I show algorithms for calculating power minimum paths for steering multiple particles in actuation constrained environments. With this microfluidic system I steer biological cells and nano particles (quantum dots) to nano meter precision. In the last part of the thesis I develop and experimentally demonstrate two dimensional (2D) manipulation of a single droplet of ferrofluid by feedback control of 4 external electromagnets, with a view towards enabling feedback control of magnetic drug delivery to reach deeper tumors in the long term. To this end, I developed and experimentally demonstrated an optimal control algorithm to effectively manipulate a single ferrofluid droplet by magnetic feedback control. This algorithm was explicitly designed to address the nonlinear and cross-coupled nature of dynamic magnetic actuation and to best exploit available electromagnetic forces for the applications of magnetic drug delivery

Microfluidics and Bio-MEMS for Next Generation Healthcare.

Ph.D. Thesis. University of Hawaiʻi at Mānoa 2018

Workshop on "Robotic assembly of 3D MEMS".

Proceedings of a workshop proposed in IEEE IROS'2007.The increase of MEMS' functionalities often requires the integration of various technologies used for mechanical, optical and electronic subsystems in order to achieve a unique system. These different technologies have usually process incompatibilities and the whole microsystem can not be obtained monolithically and then requires microassembly steps. Microassembly of MEMS based on micrometric components is one of the most promising approaches to achieve high-performance MEMS. Moreover, microassembly also permits to develop suitable MEMS packaging as well as 3D components although microfabrication technologies are usually able to create 2D and "2.5D" components. The study of microassembly methods is consequently a high stake for MEMS technologies growth. Two approaches are currently developped for microassembly: self-assembly and robotic microassembly. In the first one, the assembly is highly parallel but the efficiency and the flexibility still stay low. The robotic approach has the potential to reach precise and reliable assembly with high flexibility. The proposed workshop focuses on this second approach and will take a bearing of the corresponding microrobotic issues. Beyond the microfabrication technologies, performing MEMS microassembly requires, micromanipulation strategies, microworld dynamics and attachment technologies. The design and the fabrication of the microrobot end-effectors as well as the assembled micro-parts require the use of microfabrication technologies. Moreover new micromanipulation strategies are necessary to handle and position micro-parts with sufficiently high accuracy during assembly. The dynamic behaviour of micrometric objects has also to be studied and controlled. Finally, after positioning the micro-part, attachment technologies are necessary

Real-Time Path Planning for Automating Optical Tweezers based Particle Transport Operations

Optical tweezers (OT) have been developed to successfully trap, orient, and transport micro and nano scale components of many different sizes and shapes in a fluid medium. They can be viewed as robots made out of light. Components can be simply released from optical traps by switching off laser beams. By utilizing the principle of time sharing or holograms, multiple optical traps can perform several operations in parallel. These characteristics make optical tweezers a very promising technology for creating directed micro and nano scale assemblies. In the infra-red regime, they are useful in a large number of biological applications as well. This dissertation explores the problem of real-time path planning for autonomous OT based transport operations. Such operations pose interesting challenges as the environment is uncertain and dynamic due to the random Brownian motion of the particles and noise in the imaging based measurements. Silica microspheres having diameters between (1-20) µm are selected as model components. Offline simulations are performed to gather trapping probability data that serves as a measure of trap strength and reliability as a function of relative position of the particle under consideration with respect to the trap focus, and trap velocity. Simplified models are generated using Gaussian Radial Basis Functions to represent the data in a compact form. These metamodels can be queried at run-time to obtain estimated probability values accurately and efficiently. Simple trapping probability models are then utilized in a stochastic dynamic programming framework to compute optimum trap locations and velocities that minimizes the total, expected transport time by incorporating collision avoidance and recovery steps. A discrete version of an approximate partially observable Markov decision process algorithm, called the QMDP_NLTDV algorithm, is developed. Real-time performance is ensured by pruning the search space and enhancing convergence rates by introducing a non-linear value function. The algorithm is validated both using a simulator as well as a physical holographic tweezer set-up. Successful runs show that the automated planner is flexible, works well in reasonably crowded scenes, and is capable of transporting a specific particle to a given goal location by avoiding collisions either by circumventing or by trapping other freely diffusing particles. This technique for transporting individual particles is utilized within a decoupled and prioritized approach to move multiple particles simultaneously. An iterative version of a bipartite graph matching algorithm is also used to assign goal locations to target objects optimally. As in the case of single particle transport, simulation and some physical experiments are performed to validate the multi-particle planning approach

The development of optical nanomachines for studying molecules : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Mechatronics Engineering at Massey University, Palmerston North, New Zealand

Chapter 3 is ©2020 IEEE. Accepted manuscript is reprinted, with permission, from 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM). Chapter 5 is ©2022 IEEE. Accepted manuscript is reprinted, with permission, from 2022 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS).Optical tweezers have been used for a number of applications since their invention by Arthur Ashkin in 1986, and are particularly useful for biological and biophysical studies due to their exceptionally high spatial and force-based resolution. The same intense laser focus that allows light to be used as a tool for micro-nanoscale manipulation also has the potential to damage the objects being studied, and the extremely high force resolution is coupled with the limitation of very low forces. There is potential to overcome these drawbacks of optical manipulation through making use of another laser based technique: two-photon absorption polymerisation (TPAP). This thesis has brought these together to demonstrate the uses of optical nanomachines as helpful tools for optical tweezer studies. The project was highly interdisciplinary, concerning the intersection of optical trapping, 3D micromachine design and development, and DNA stretching. The thesis was based around the strategy of first developing microrobots and demonstrating their manipulation using optical tweezers, then adjusting the design for specific applications. Microlevers were developed for lever-assisted DNA stretching and amplification of optical forces. The influence of design features and TPAP parameters on microlever functionality was investigated; particularly the influence of overlapping area and presence of supports, and the effects of differently shaped "trapping handles". These features were important as lever functionality was tested in solutions of different ionic strength, and stable trapping of the levers was required for force amplification. DNA stretching was chosen as a target application for distanced-application of optical forces due to its status as a well-known and characterised example of single-molecule studies with optical tweezers. Amplification of optical forces was also seen as an application that could demonstrate the utility of optical micromachines, and microlevers with a 2:1 lever arm ratio were developed to produce consistent, two-fold amplification of optical forces, in a first for unsupported, pin-jointed optical microrobotics. It is hoped that in the future fully-remote, micromachine-assisted studies will extend optical tweezer studies of laser-sensitive subjects, as well as increasing the forces that can be applied, and the results obtained in this thesis are encouraging. All in all, the thesis confirms the potential of optical micromachines for aiding studies using optical tweezers, and demonstrates concrete progress in both design and application