Apprentissage par renforcement dans le cadre des processus décisionnels de Markov factorisés observables dans le désordre. Etude expérimentale du Q-Learning parallèle appliqué aux problèmes du labyrinthe et du New York Driving.

National audienceCet article présente les résultats expérimentaux obtenus avec une architecture originale permettant un apprentissage générique dans le cadre de processus décisionnels de Markov factorisés observables dans le désordre (PDMFOD). L'article décrit tout d'abord le cadre formel des PDMFOD puis le fonctionnement de l'algorithme, notamment le principe de parallélisation et l'attribution dynamique des récompenses. L'architecture est ensuite appliquée à deux problèmes de navigation, l'un dans un labyrinthe et l'autre dans un trafic routier (New York Driving). Les tests montrent que l'architecture permet effectivement d'apprendre une politique de décisions performante et générique malgré le nombre élevé de dimensions des espaces d'états des deux systèmes

Control of a particular Micro-Macro positioning system applied to cell Micromanipulation.

International audienceBiological research requires new tools for cell micromanipulations. Currently, biological cell sizes range from a few to hundreds of micrometers, their manipulation therefore belonging to the field of microrobotics. This paper presents a new wireless micromanipulation system which allows cells placed in a droplet of liquid to be pushed on a glass slide. The cell micropusher is a ferromagnetic object which follows the movement of a permanent magnet located under the glass slide. It has been proved in previous works that two kinds of micropusher movements can induce a movement of the pushed object: turning the micropusher around the contact point (rotation), or moving the micropusher in translation. Rotation allows an object to be placed with a precision below 1 μm ,but acts within a narrow range. Translation allows placement of an object with lower accuracy, but within a wide range. We propose a specific coarse-fine control strategy to push an object, with good precision, within a wide range. Furthermore, experimentation on polystyrene balls of 50 μm in diameter, and immature human oocytes of 150 μm in diameter are presented

Nanoforce estimation with Kalman filtering applied to a force sensor based on diamagnetic levitation.

International audienceNano force sensors based on passive diamagnetic levitation with a macroscopic seismic mass are a possible alternative to classical Atomic Force Microscopes when the force bandwidth to be measured is limited to a few Hertz. When an external unknown force is applied to the levitating seismic mass, this one acts as a transducer that converts this unknown input into a displacement that is the measured output signal. Because the little damped and long transient response of this kind of macroscopic transducer can not be neglected, it is then necessary to deconvolve the output to correctly estimate the unknown input force. The deconvolution approach proposed in this article is based on a Kalman filter that use an uncertain a priori model to represent the unknown nanoforce to be estimated. The main advantage of this approach is that the end-user can directly control the unavoidable trade-off that exists between the wished resolution on the estimatedforce and the response time of the estimation

Microforce sensor for microbiological applications based on a floating-magnetic principle.

International audienceIn this paper, we present the design of a new magentic nano and microforce sensor for microbiological applications. The sensing part of the sensor presents a naturally stable six degrees of freedom equilibrium state using the combination of upthrust buoyancy and magnetic force. The sensor allows force measurement without deformation of the sensing element using a feedback control loop and is able to measure the components, in the horizontal plan, of the external force applied. The measurement range varies between around ± 100 µN with a resolution of 20 nN and a linear output. The mechanical stiffness of the passive system is about 0.018 N.m−1(same order of magnitude than an AFM micro-cantilever). A complete static study and experimental validation of the used principle are presented in this paper

Trajectory modelling of a planar magnetic cell micropusher.

International audienceThe improving of the efficiency and the automation of biological cell technologies is a current high stake. One way is to build biological micro-factories which are able to perform a complete biotechnological processes automatically. This technology requires the development of new automatic cell transport system to feed work stations in microfactories. An original magnetic cell micropusher is described in this paper. The ferromagnetic pusher which is submerged in the biological medium follows the movement of a permanent magnet located in the air. This paper focuses on the modelling of the dynamic behaviour of the micropusher in function of the magnet trajectory. The generic model proposed is able to determine pusher trajectory according to the micropusher magnetic properties and the permanent magnet shape and properties. This simulation tool will allow to optimize and to study cell trajectory control in further works

Modelling and optimization of a floating triangular platform used for nano and microforces sensing.

International audienceThis paper presents the dynamic behaviour modelling of a horizontal triangular platform, used as the sensing part of a microforce sensor. This sensor is based on a magnetic and a buoyancy principle. A particular configuration used to obtain a linear model is presented. This linear model will be essential for the futur control of the system, in order to achieve force measurement without displacement of the platform. The determination of the platform position and orientation in the horizontal plane is done thanks to three laser range sensors. Sensors configuration provide a linear transformation between the three measured ranges and the position and orientation of the platform. Finally, an open loop result comparison is done between the linear state model and a more complex 3D non linear model

Nanoforce estimation based on Kalman filtering and applied to a force sensor using diamagnetic levitation

International audienceNanoforce sensors based on passive diamagnetic levitation with a macroscopic seismic mass are a possible alternative to classical Atomic Force Microscopes when the force bandwidth to be measured is limited to a few Hertz. When an external unknown force is applied to the levitating seismic mass, this one acts as a transducer that converts this unknown input into a displacement that is the measured output signal. Because the under-damped and long transient response of this kind of macroscopic transducer cannot be neglected for time-varying force measurement, it is then necessary to deconvolve the output to correctly estimate the unknown input force. The deconvolution approach proposed in this paper is based on a Kalman filter that use an uncertain a priori model to represent the unknown nanoforce to be estimated. The main advantage of this approach is that the end-user can directly control the unavoidable trade-off that exists between the wished resolution on the estimated force and the response time of the estimation

Analysis of the trade-off between resolution and bandwidth for a nanoforce sensor based on diamagnetic levitation.

International audienceNanoforce sensors based on passive diamagnetic levitation with a macroscopic seismic mass are a possible alternative to classical Atomic Force Microscopes when theforce bandwidth to be measured is limited to a few Hertz.When an external unknown force is applied to the levitating seismic mass, this one acts as a transducer that converts this unknown input into a displacement that is the measured output signal. Because the inertia effect due to the mass of such macroscopic transducers cannot be neglected for timevarying force measurement, it is necessary to deconvolve the displacement to correctly estimate the unknown input force.A deconvolution approach based on a Kalman filter and controlled by a scalar parameter has been recently proposed.The adjustement of this parameter leads to a trade-off that is analysed in this paper in term of resolution and bandwidth of the estimated force. Associated tools to help the end-user to set this parameter are also described

Modelling of a planar magnetic micropusher for biological cell manipulations.

International audienceThe improving of the efficiency and the automation of biological cell technologies is currently of great importance. One way is to build biological micro-factories which are able to perform complete biotechnological processes automatically. This technology requires the development of new automatic cell transport system to feed work stations in microfactories. An original magnetic cell micropusher is described in this paper. The ferromagnetic pusher which is submerged in the biological medium follows the movement of a permanent magnet located in the air. This paper focuses on the modelling of the dynamic behaviour of the micropusher according to the magnet trajectory. The generic model proposed is able to determine pusher trajectory according to the micropusher magnetic properties and the permanent magnet shape and properties. This simulation tool will permit to optimize and to study cell trajectory control in further works

Analysis of a passive microforce sensor based on magnetic spings and upthrust buoyancy.

International audienceThis paper is focused on the design and modeling of a new micro and nano force sensor using magnetic springs. The force sensitive part is a macroscopic horizontal rigid platform used as a oating seismic mass. This platform presents a naturally stable equilibrium state for its six degrees of freedom (dof) thanks to the combination of upthrust buoyancy and magnetic forces. This force sensor allows the measurement of the external horizontal force and the vertical torque applied to the platform. Thanks to the magnetic springs con guration used, the seismic mass presents a 0.02 N/m horizontal mechanical sti ness (similar to the sti ness of a thin AFM micro-cantilever). The measurement range typically varies between 50 N. The resolution depends on the displacement sensors used to measure the seismic mass displacement and on the environmental conditions (ground, liquid and air vibrations). In steady state, this displacement is proportional to the applied force. Resolution of less than 10 nN can be reached with the use of an anti-vibration table