Aberration-free calibration for 3D single molecule localization microscopy

We propose a straightforward sample-based technique to calibrate the axial detection in 3D single molecule localization microscopy (SMLM). Using microspheres coated with fluorescent molecules, the calibration curves of PSF-shaping- or intensity-based measurements can be obtained for any required depth range from a few hundreds of nanometers to several tens of microns. This experimental method takes into account the effect of the spherical aberration without requiring computational correction.Comment: 8 pages, 2 figures. Submitted to Optics Letters on October 12th, 201

Hysteretic Q-Learning : an algorithm for decentralized reinforcement learning in cooperative multi-agent teams.

International audienceMulti-agent systems (MAS) are a field of study of growing interest in a variety of domains such as robotics or distributed controls. The article focuses on decentralized reinforcement learning (RL) in cooperative MAS, where a team of independent learning robot (IL) try to coordinate their individual behavior to reach a coherent joint behavior. We assume that each robot has no information about its teammates'actions. To date, RL approaches for such ILs did not guarantee convergence to the optimal joint policy in scenarios where the coordination is difficult. We report an investigation of existing algorithms for the learning of coordination in cooperative MAS, and suggest a Q-Learning extension for ILs, called Hysteretic Q-Learning. This algorithm does not require any additional communication between robots. Its advantages are showing off and compared to other methods on various applications : bimatrix games, collaborative ball balancing task and pursuit domain

Choix de la fonction de renforcement et des valeurs initiales pour accélérer les problèmes d'Apprentissage par Renforcement de plus court chemin stochastique.

National audienceUn point important en apprentissage par renforcement (AR) est l'amélioration de la vitesse de convergence du processus d'apprentissage. Nous proposons dans cet article d'étudier l'influence de certains paramètres de l'AR sur la vitesse d'apprentissage. En effet, bien que les propriétés de convergence de l'AR ont été largement étudiées, peu de règles précises existent pour choisir correctement la fonction de renforcement et les valeurs initiales de la table Q. Notre méthode aide au choix de ces paramètres dans le cadre de problèmes de type goal-directed, c'est-à-dire dont l'objectif est d'atteindre un but en un minimum de temps. Nous développons une étude théorique et proposons ensuite des justifications expérimentales pour choisir d'une part la fonction de renforcement et d'autre part des valeurs initiales particulières de la table Q, basées sur une fonction d'influence

Reward function and initial values : Better choices for accelerated Goal-directed reinforcement learning.

International audienceAn important issue in Reinforcement Learning (RL) is to accelerate or improve the learning process. In this paper, we study the influence of some RL parameters over the learning speed. Indeed, although RL convergence properties have been widely studied, no precise rules exist to correctly choose the reward function and initial Q-values. Our method helps the choice of these RL parameters within the context of reaching a goal in a minimal time. We develop a theoretical study and also provide experimental justifications for choosing on the one hand the reward function, and on the other hand particular initial Q-values based on a goal bias function

A study of FMQ heuristic in cooperative multi-agent games.

International audienceThe article focuses on decentralized reinforcement learning (RL) in cooperative multi-agent games, where a team of independent learning agents (ILs) try to coordinate their individual actions to reach an optimal joint action. Within this framework, some algorithms based on Q-learning are proposed in recent works. Especially, we are interested in Distributed Q-learning which finds optimal policies in deterministic games, and in the Frequency Maximum Q value (FMQ) heuristic which is able in partially stochastic matrix games to distinguish if a poor reward received for the same action are due to either miscoordination or to the noisy reward function. Making this distinction is one of the main difficulties to solve stochastic games. Our objective is to find an algorithm able to switch over the updates according to a detection of the cause of noise. In this paper, a modified version of the FMQ heuristic is proposed which achieves this detection and the update adaptation. Moreover, this modified FMQ version is more robust and very easy to set

Un algorithme décentralisé d'apprentissage par renforcement multi-agents coopératifs : le Q-Learning Hystérétique.

National audienceNous nous intéressons aux techniques d'apprentissage par renforcement dans les systèmes multi-agents coopératifs. Nous présentons un nouvel algorithme pour agents indépendants qui permet d'apprendre l'action jointe optimale dans des jeux où la coordination est difficile. Nous motivons notre approche par le caractère décentralisé de cet algorithme qui ne nécessite aucune communication entre agents et des tables Q de taille indépendante du nombre d'agents. Des tests concluants sont de plus effectués sur des jeux coopératifs répétés, ainsi que sur un jeu de poursuite

The world of Independent learners is not Markovian.

International audienceIn multi-agent systems, the presence of learning agents can cause the environment to be non-Markovian from an agent's perspective thus violat- ing the property that traditional single-agent learning methods rely upon. This paper formalizes some known intuition about concurrently learning agents by providing formal conditions that make the environment non- Markovian from an independent (non-communicative) learner's perspec- tive. New concepts are introduced like the divergent learning paths and the observability of the e ects of others' actions. To illustrate the formal concepts, a case study is also presented. These ndings are signi cant because they both help to understand failures and successes of existing learning algorithms as well as being suggestive for future work

Autofocusing-based visual servoing : application to MEMS micromanipulation.

International audienceIn MEMS microassembly areas, different methods of automatic focusing are presented in the literature. All these methods have a common point. Thus, the current autofocusing methods for microscopes need to perform a scanning on all the vertical axis of the microscope in order to find the peak corresponding to the focus (sharpen image). Those methods are time consuming. Therefore, this paper presents an original method of autofocusing based on a velocity control approach which is developed and validated on real experiments

A new contactless conveyor system for handling clean and delicate products using induced air flows.

International audienceIn this paper, a new contactless conveyor system based on an original aerodynamic traction principle is described and experimented. This device is able to convey without any contact flat objects like silicon wafer, glass sheets or foodstufff thanks to an air cushion and induced air flows. A model of the system is established and the identification of the parameters is carried out. A closed-loop control is proposed for one dimension position control and position tracking. The PID-controller gives good performances for different reference signals. Its robustness to object change and perturbation rejection are also tested