Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.Comment: Paper submitted to Pattern Recognition Letter

Optimization of a range of 2D and 3D bulk forming processes by a meta-model assisted evolution strategy

International audienceAfter decades of developing efficient software, FORGE, for bulk metal forming simulation, its coupling with an optimization algorithm is considered in order to solve the actual engineering problem, the design problem, using the meta-model assisted evolution strategy proposed by Emmerich et al. The first application regards the shape optimization of a cylindrical preform to produce a crankshaft by closed die forging. It aims at minimizing the volume of the material while satisfying the filling of the dies. In the second application, the open die forging of an axisymmetric thick plate is considered. The objective is to optimize the initial geometry in order to minimize the material weight while overlapping a prescribed geometry at the end of forging. The third example regards a four-stepped wire drawing sequence. In order to prevent the formation of cracks, it is desired to minimize a damage criterion while keeping the targeted diameter of the final wire. The geometries of the four dies are optimized: reduction ratio, die angle and die length. The necessary calculations are run in parallel, by using both the parallel version of the software and by evaluating several designs at the same time

Geometric control of cardiomyogenic induction in human pluripotent stem cells

Although it has been observed that aggregate size affects cardiac development, an incomplete understanding of the cellular mechanisms underlying human pluripotent stem cell-derived cardiomyogenesis has limited the development of robust defined-condition cardiac cell generation protocols. Our objective was thus to elucidate cellular and molecular mechanisms underlying the endogenous control of human embryonic stem cell (hESC) cardiac tissue development, and to test the hypothesis that hESC aggregate size influences extraembryonic endoderm (ExE) commitment and cardiac inductive properties. hESC aggregates were generated with 100, 1000, or 4000 cells per aggregate using microwells. The frequency of endoderm marker (FoxA2 and GATA6)-expressing cells decreased with increasing aggregate size during early differentiation. Cardiogenesis was maximized in aggregates initiated from 1000 cells, with frequencies of 0.49±0.06 cells exhibiting a cardiac progenitor phenotype (KDRlow/C-KITneg) on day 5 and 0.24±0.06 expressing cardiac Troponin T on day 16. A direct relationship between ExE and cardiac differentiation efficiency was established by forming aggregates with varying ratios of SOX7 (a transcription factor required for ExE development) overexpressing or knockdown hESCs to unmanipulated hESCs. We demonstrate, in a defined, serum-free cardiac induction system, that robust and efficient cardiac differentiation is a function of endogenous ExE cell concentration, a parameter that can be directly modulated by controlling hESC aggregate size. © 2011 Mary Ann Liebert, Inc

Bipolar ablation for deep intra-myocardial circuits: human ex vivo development and in vivo experience.

To access publisher's full text version of this article click on the hyperlink at the bottom of the pageCurrent conventional ablation strategies for ventricular tachycardia (VT) aim to interrupt reentrant circuits by creating ablation lesions. However, the critical components of reentrant VT circuits may be located at deep intramural sites. We hypothesized that bipolar ablations would create deeper lesions than unipolar ablation in human hearts.Ablation was performed on nine explanted human hearts at the time of transplantation. Following explant, the hearts were perfused by using a Langendorff perfusion setup. For bipolar ablation, the endocardial catheter was connected to the generator as the active electrode and the epicardial catheter as the return electrode. Unipolar ablation was performed at 50 W with irrigation of 25 mL/min, with temperature limit of 50°C. Bipolar ablation was performed with the same settings. Subsequently, in a patient with an incessant septal VT, catheters were positioned on the septum from both the ventricles and radiofrequency was delivered with 40 W. In the explanted hearts, there were a total of nine unipolar ablations and four bipolar ablations. The lesion depth was greater with bipolar ablation, 14.8 vs. 6.1 mm (P < 0.01), but the width was not different (9.8 vs. 7.8 mm). All bipolar lesions achieved transmurality in contrast to the unipolar ablations. In the patient with a septal focus, bipolar ablation resulted in termination of VT with no inducible VTs.By using a bipolar ablation technique, we have demonstrated the creation of significantly deeper lesions without increasing the lesion width, compared with standard ablation. Further clinical trials are warranted to detail the risks of this technique

Omnipolarity applied to equi-spaced electrode array for ventricular tachycardia substrate mapping

Aims : Bipolar electrogram (BiEGM)-based substrate maps are heavily influenced by direction of a wavefront to the mapping bipole. In this study, we evaluate high-resolution, orientation-independent peak-to-peak voltage (Vpp) maps obtained with an equi-spaced electrode array and omnipolar EGMs (OTEGMs), measure its beat-to-beat consistency, and assess its ability to delineate diseased areas within the myocardium compared against traditional BiEGMs on two orientations: along (AL) and across (AC) array splines. Methods and results: The endocardium of the left ventricle of 10 pigs (three healthy and seven infarcted) were each mapped using an Advisor™ HD grid with a research EnSite Precision™ system. Cardiac magnetic resonance images with late gadolinium enhancement were registered with electroanatomical maps and were used for gross scar delineation. Over healthy areas, OTEGM Vpp values are larger than AL bipoles by 27% and AC bipoles by 26%, and over infarcted areas OTEGM Vpp values are 23% larger than AL bipoles and 27% larger than AC bipoles (P < 0.05). Omnipolar EGM voltage maps were 37% denser than BiEGM maps. In addition, OTEGM Vpp values are more consistent than bipolar Vpps showing less beat-by-beat variation than BiEGM by 39% and 47% over both infarcted and healthy areas, respectively (P < 0.01). Omnipolar EGM better delineate infarcted areas than traditional BiEGMs from both orientations. Conclusion: An equi-spaced electrode grid when combined with omnipolar methodology yielded the largest detectable bipolar-like voltage and is void of directional influences, providing reliable voltage assessment within infarcted and non-infarcted regions of the heart.This work was funded by Abbott Laboratories, St. Paul, MN, USA.S

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

International audienceIn this paper we address the problems of detecting objects of interest in a video and of estimating their locations, solely from the gaze directions of people present in the video. Objects can be indistinctly located inside or outside the camera field of view. We refer to this problem as extended gaze following. The contributions of the paper are the followings. First, we propose a novel spatial representation of the gaze directions adopting a top-view perspective. Second, we develop several convolutional encoder/decoder networks to predict object locations and compare them with heuristics and with classical learning-based approaches. Third, in order to train the proposed models, we generate a very large number of synthetic scenarios employing a probabilistic formulation. Finally, our methodology is empirically validated using a publicly available dataset

Pro-arrhythmic role of adrenergic spatial densities in the human atria: An in-silico study.

Chronic stress among young patients (≤ 45 years old) could result in autonomic dysfunction. Autonomic dysfunction could be exhibited via sympathetic hyperactivity, sympathetic nerve sprouting, and diffuse adrenergic stimulation in the atria. Adrenergic spatial densities could alter atrial electrophysiology and increase arrhythmic susceptibility. Therefore, we examined the role of adrenergic spatial densities in creating arrhythmogenic substrates in silico. We simulated three 25 cm2 atrial sheets with varying adrenergic spatial densities (ASD), activation rates, and external transmembrane currents. We measured their effects on spatial and temporal heterogeneity of action potential durations (APD) at 50% and 20%. Increasing ASD shortens overall APD, and maximum spatial heterogeneity (31%) is achieved at 15% ASD. The addition of a few (5% to 10%) adrenergic elements decreases the excitation threshold, below 18 μA/cm2, while ASDs greater than 10% increase their excitation threshold up to 22 μA/cm2. Increase in ASD during rapid activation increases APD50 and APD20 by 21% and 41%, respectively. Activation times of captured beats during rapid activation could change by as much as 120 ms from the baseline cycle length. Rapidly activated atrial sheets with high ASDs significantly increase temporal heterogeneity of APD50 and APD20. Rapidly activated atrial sheets with 10% ASD have a high likelihood (0.7 ± 0.06) of fragmenting otherwise uniform wavefronts due to the transient inexcitability of adrenergically stimulated elements, producing an effective functional block. The likelihood of wave fragmentation due to ASD highly correlates with the spatial variations of APD20 (ρ = 0.90, p = 0.04). Our simulations provide a novel insight into the contributions of ASD to spatial and temporal heterogeneities of APDs, changes in excitation thresholds, and a potential explanation for wave fragmentation in the human atria due to sympathetic hyperactivity. Our work may aid in elucidating an electrophysiological link to arrhythmia initiation due to chronic stress among young patients

Deep Reinforcement Learning for Audio-Visual Gaze Control

International audienceWe address the problem of audiovisual gaze control in the specific context of human-robot interaction, namely how controlled robot motions are combined with visual and acoustic observations in order to direct the robot head towards targets of interest. The paper has the following contributions: (i) a novel audiovisual fusion framework that is well suited for controlling the gaze of a robotic head; (ii) a reinforcement learning (RL) formulation for the gaze control problem, using a reward function based on the available temporal sequence of camera and microphone observations; and (iii) several deep architectures that allow to experiment with early and late fusion of audio and visual data. We introduce a simulated environment that enables us to learn the proposed deep RL model without the need of spending hours of tedious interaction. By thoroughly experimenting on a publicly available dataset and on a real robot, we provide empirical evidence that our method achieves state-of-the-art performance