A Constraint Programming Approach to Simultaneous Task Allocation and Motion Scheduling for Industrial Dual-Arm Manipulation Tasks

Modern lightweight dual-arm robots bring the physical capabilities to quickly take over tasks at typical industrial workplaces designed for workers. In times of mass-customization, low setup times including the instructing/specifying of new tasks are crucial to stay competitive. We propose a constraint programming approach to simultaneous task allocation and motion scheduling for such industrial manipulation and assembly tasks. The proposed approach covers dual-arm and even multi-arm robots as well as connected machines. The key concept are Ordered Visiting Constraints, a descriptive and extensible model to specify such tasks with their spatiotemporal requirements and task-specific combinatorial or ordering constraints. Our solver integrates such task models and robot motion models into constraint optimization problems and solves them efficiently using various heuristics to produce makespan-optimized robot programs. The proposed task model is robot independent and thus can easily be deployed to other robotic platforms. Flexibility and portability of our proposed model is validated through several experiments on different simulated robot platforms. We benchmarked our search strategy against a general-purpose heuristic. For large manipulation tasks with 200 objects, our solver implemented using Google's Operations Research tools and ROS requires less than a minute to compute usable plans.Comment: 8 pages, 8 figures, submitted to ICRA'1

A Computer-Aided Training (CAT) System for Short Track Speed Skating

Short track speed skating has become popular all over the world. The demands of a computer-aided training (CAT) system are booming due to this fact. However, the existing commercial systems for sports are highly dependent on expensive equipment and complicated hardware calibration. This dissertation presents a novel CAT system for tracking multiple skaters in short track skating competitions. Aiming at the challenges, we utilize global rink information to compensate camera motion and obtain the global spatial information of skaters; apply Random Forest to fuse multiple cues and predict the blobs for each of the skaters; and finally develop a silhouette and edge-based template matching and blob growing method to allocate each blob to corresponding skaters. The proposed multiple skaters tracking algorithm organically integrates multi-cue fusion, dynamic appearance modeling, machine learning, etc. to form an efficient and robust CAT system. The effectiveness and robustness of the proposed method are presented through experiments

Evolutionary algorithm for particle trajectory reconstruction within inhomogeneous magnetic field in the NA61/SHINE experiment at CERN SPS

In this paper, a novel probabilistic tracking method is proposed. It combines two competing models: (i) a discriminative one for background classification; and (ii) a generative one as a track model. The model competition, along with a combinatorial data association, shows good signal and background noise separation. Furthermore, a stochastic and derivative-free method is used for parameter optimization by means of the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). Finally, the applicability and performance of the particle trajectories reconstruction are shown. The algorithm is developed for NA61/SHINE data reconstruction purpose and therefore the method was tested on simulation data of the NA61/SHINE experiment

Single particle algorithms to reveal cellular nanodomain organization

Formation, maintenance and physiology of high-density protein-enriched organized nanodomains, first observed in electron microscopy images, remains challenging to investigate due to their small sizes. However, these regions regulate molecular trafficking, assembly and sorting required for higher cell functions, such as communication or plastic changes. Over the past ten years, super-resolution single-particle trajectories (SPTs) have been used to sample these sub-cellular environments at a nanometer resolution for both membrane and soluble proteins. We present here data analysis developments and algorithms that convert high-throughput molecular trajectories into maps of molecular density, diffusion and local drift organization. These approaches transform intrinsic trajectory properties into statistics of the underlying cellular organization. The automatic identification of large numbers of high-density regions allows quantifying their boundary location and organization, their stability over time and their ability to transiently retain molecules. To conclude recent automated algorithms can now be used to extract biophysical parameters of sub-cellular nanodomains over a large amount of trajectories.Comment: 6 fig

Multiple particle tracking in PEPT using Voronoi tessellations

An algorithm is presented which makes use of three-dimensional Voronoi tessellations to track up to 20 tracers using a PET scanner. The lines of response generated by the PET scanner are discretized into sets of equidistant points, and these are used as the input seeds to the Voronoi tessellation. For each line of response, the point with the smallest Voronoi region is located; this point is assumed to be the origin of the corresponding line of response. Once these origin points have been determined, any outliers are removed, and the remaining points are clustered using the DBSCAN algorithm. The centroid of each cluster is classified as a tracer location. Once the tracer locations are determined for each time frame in the experimental data set, a custom multiple target tracking algorithm is used to associate identical tracers from frame to frame. Since there are no physical properties to distinguish the tracers from one another, the tracking algorithm uses velocity and position to extrapolate the locations of existing tracers and match the next frame's tracers to the trajectories. A series of experiments were conducted in order to test the robustness, accuracy and computational performance of the algorithm. A measure of robustness is the chance of track loss, which occurs when the algorithm fails to match a tracer location with its trajectory, and the track is terminated. The chance of track loss increases with the number of tracers; the acceleration of the tracers; the time interval between successive frames; and the proximity of tracers to each other. In the case of two tracers colliding, the two tracks merge for a short period of time, before separating and become distinguishable again. Track loss also occurs when a tracer leaves the field of view of the scanner; on return it is treated as a new object. The accuracy of location of the algorithm was found to be slightly affected by tracer velocity, but is much more dependent on the distance between consecutive points on a line of response, and the number of lines of response used per time frame. A single tracer was located to within 1.26mm. This was compared to the widely accepted Birmingham algorithm, which located the same tracer to within 0.92mm. Precisions of between 1.5 and 2.0mm were easily achieved for multiple tracers. The memory usage and processing time of the algorithm are dependent on the number of tracers used in the experiment. It was found that the processing time per frame for 20 tracers was about 15s, and the memory usage was 400MB. Because of the high processing times, the algorithm as is is not feasible for practical use. However, the location phase of the algorithm is massively parallel, so the code can be adapted to significantly increase the efficiency

The Electrophysiology of Resting State fMRI Networks

Traditional research in neuroscience has studied the topography of specific brain functions largely by presenting stimuli or imposing tasks and measuring evoked brain activity. This paradigm has dominated neuroscience for 50 years. Recently, investigations of brain activity in the resting state, most frequently using functional magnetic resonance imaging (fMRI), have revealed spontaneous correlations within widely distributed brain regions known as resting state networks (RSNs). Variability in RSNs across individuals has found to systematically relate to numerous diseases as well as differences in cognitive performance within specific domains. However, the relationship between spontaneous fMRI activity and the underlying neurophysiology is not well understood. This thesis aims to combine invasive electrophysiology and resting state fMRI in human subjects to better understand the nature of spontaneous brain activity. First, we establish an approach to precisely coregister intra-cranial electrodes to fMRI data (Chapter 2). We then created a novel machine learning approach to define resting state networks in individual subjects (Chapter 3). This approach is validated with cortical stimulation in clinical electrocorticography (ECoG) patients (Chapter 4). Spontaneous ECoG data are then analyzed with respect to fMRI time-series and fMRI-defined RSNs in order to illustrate novel ECoG correlates of fMRI for both local field potentials and band-limited power (BLP) envelopes (Chapter 5). In Chapter 6, we show that the spectral specificity of these resting state ECoG correlates link classic brain rhythms with large-scale functional domains. Finally, in Chapter 7 we show that the frequencies and topographies of spontaneous ECoG correlations specifically recapitulate the spectral and spatial structure of task responses within individual subjects

Unsupervised deep learning of human brain diffusion magnetic resonance imaging tractography data

L'imagerie par résonance magnétique de diffusion est une technique non invasive permettant de connaître la microstructure organisationnelle des tissus biologiques. Les méthodes computationnelles qui exploitent la préférence orientationnelle de la diffusion dans des structures restreintes pour révéler les voies axonales de la matière blanche du cerveau sont appelées tractographie. Ces dernières années, diverses méthodes de tractographie ont été utilisées avec succès pour découvrir l'architecture de la matière blanche du cerveau. Pourtant, ces techniques de reconstruction souffrent d'un certain nombre de défauts dérivés d'ambiguïtés fondamentales liées à l'information orientationnelle. Cela a des conséquences dramatiques, puisque les cartes de connectivité de la matière blanche basées sur la tractographie sont dominées par des faux positifs. Ainsi, la grande proportion de voies invalides récupérées demeure un des principaux défis à résoudre par la tractographie pour obtenir une description anatomique fiable de la matière blanche. Des approches méthodologiques innovantes sont nécessaires pour aider à résoudre ces questions. Les progrès récents en termes de puissance de calcul et de disponibilité des données ont rendu possible l'application réussie des approches modernes d'apprentissage automatique à une variété de problèmes, y compris les tâches de vision par ordinateur et d'analyse d'images. Ces méthodes modélisent et trouvent les motifs sous-jacents dans les données, et permettent de faire des prédictions sur de nouvelles données. De même, elles peuvent permettre d'obtenir des représentations compactes des caractéristiques intrinsèques des données d'intérêt. Les approches modernes basées sur les données, regroupées sous la famille des méthodes d'apprentissage profond, sont adoptées pour résoudre des tâches d'analyse de données d'imagerie médicale, y compris la tractographie. Dans ce contexte, les méthodes deviennent moins dépendantes des contraintes imposées par les approches classiques utilisées en tractographie. Par conséquent, les méthodes inspirées de l'apprentissage profond conviennent au changement de paradigme requis, et peuvent ouvrir de nouvelles possibilités de modélisation, en améliorant ainsi l'état de l'art en tractographie. Dans cette thèse, un nouveau paradigme basé sur les techniques d'apprentissage de représentation est proposé pour générer et analyser des données de tractographie. En exploitant les architectures d'autoencodeurs, ce travail tente d'explorer leur capacité à trouver un code optimal pour représenter les caractéristiques des fibres de la matière blanche. Les contributions proposées exploitent ces représentations pour une variété de tâches liées à la tractographie, y compris (i) le filtrage et (ii) le regroupement efficace sur les résultats générés par d'autres méthodes, ainsi que (iii) la reconstruction proprement dite des fibres de la matière blanche en utilisant une méthode générative. Ainsi, les méthodes issues de cette thèse ont été nommées (i) FINTA (Filtering in Tractography using Autoencoders), (ii) CINTA (Clustering in Tractography using Autoencoders), et (iii) GESTA (Generative Sampling in Bundle Tractography using Autoencoders), respectivement. Les performances des méthodes proposées sont évaluées par rapport aux méthodes de l'état de l'art sur des données de diffusion synthétiques et des données de cerveaux humains chez l'adulte sain in vivo. Les résultats montrent que (i) la méthode de filtrage proposée offre une sensibilité et spécificité supérieures par rapport à d'autres méthodes de l'état de l'art; (ii) le regroupement des tractes dans des faisceaux est fait de manière consistante; et (iii) l'approche générative échantillonnant des tractes comble mieux l'espace de la matière blanche dans des régions difficiles à reconstruire. Enfin, cette thèse révèle les possibilités des autoencodeurs pour l'analyse des données des fibres de la matière blanche, et ouvre la voie à fournir des données de tractographie plus fiables.Abstract : Diffusion magnetic resonance imaging is a non-invasive technique providing insights into the organizational microstructure of biological tissues. The computational methods that exploit the orientational preference of the diffusion in restricted structures to reveal the brain's white matter axonal pathways are called tractography. In recent years, a variety of tractography methods have been successfully used to uncover the brain's white matter architecture. Yet, these reconstruction techniques suffer from a number of shortcomings derived from fundamental ambiguities inherent to the orientation information. This has dramatic consequences, since current tractography-based white matter connectivity maps are dominated by false positive connections. Thus, the large proportion of invalid pathways recovered remains one of the main challenges to be solved by tractography to obtain a reliable anatomical description of the white matter. Methodological innovative approaches are required to help solving these questions. Recent advances in computational power and data availability have made it possible to successfully apply modern machine learning approaches to a variety of problems, including computer vision and image analysis tasks. These methods model and learn the underlying patterns in the data, and allow making accurate predictions on new data. Similarly, they may enable to obtain compact representations of the intrinsic features of the data of interest. Modern data-driven approaches, grouped under the family of deep learning methods, are being adopted to solve medical imaging data analysis tasks, including tractography. In this context, the proposed methods are less dependent on the constraints imposed by current tractography approaches. Hence, deep learning-inspired methods are suit for the required paradigm shift, may open new modeling possibilities, and thus improve the state of the art in tractography. In this thesis, a new paradigm based on representation learning techniques is proposed to generate and to analyze tractography data. By harnessing autoencoder architectures, this work explores their ability to find an optimal code to represent the features of the white matter fiber pathways. The contributions exploit such representations for a variety of tractography-related tasks, including efficient (i) filtering and (ii) clustering on results generated by other methods, and (iii) the white matter pathway reconstruction itself using a generative method. The methods issued from this thesis have been named (i) FINTA (Filtering in Tractography using Autoencoders), (ii) CINTA (Clustering in Tractography using Autoencoders), and (iii) GESTA (Generative Sampling in Bundle Tractography using Autoencoders), respectively. The proposed methods' performance is assessed against current state-of-the-art methods on synthetic data and healthy adult human brain in vivo data. Results show that the (i) introduced filtering method has superior sensitivity and specificity over other state-of-the-art methods; (ii) the clustering method groups streamlines into anatomically coherent bundles with a high degree of consistency; and (iii) the generative streamline sampling technique successfully improves the white matter coverage in hard-to-track bundles. In summary, this thesis unlocks the potential of deep autoencoder-based models for white matter data analysis, and paves the way towards delivering more reliable tractography data