Fully Automatic Expression-Invariant Face Correspondence

We consider the problem of computing accurate point-to-point correspondences among a set of human face scans with varying expressions. Our fully automatic approach does not require any manually placed markers on the scan. Instead, the approach learns the locations of a set of landmarks present in a database and uses this knowledge to automatically predict the locations of these landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a template model and the newly available scan. To accurately fit the expression of the template to the expression of the scan, we use as template a blendshape model. Our algorithm was tested on a database of human faces of different ethnic groups with strongly varying expressions. Experimental results show that the obtained point-to-point correspondence is both highly accurate and consistent for most of the tested 3D face models

Visual analytics for relationships in scientific data

Domain scientists hope to address grand scientific challenges by exploring the abundance of data generated and made available through modern high-throughput techniques. Typical scientific investigations can make use of novel visualization tools that enable dynamic formulation and fine-tuning of hypotheses to aid the process of evaluating sensitivity of key parameters. These general tools should be applicable to many disciplines: allowing biologists to develop an intuitive understanding of the structure of coexpression networks and discover genes that reside in critical positions of biological pathways, intelligence analysts to decompose social networks, and climate scientists to model extrapolate future climate conditions. By using a graph as a universal data representation of correlation, our novel visualization tool employs several techniques that when used in an integrated manner provide innovative analytical capabilities. Our tool integrates techniques such as graph layout, qualitative subgraph extraction through a novel 2D user interface, quantitative subgraph extraction using graph-theoretic algorithms or by querying an optimized B-tree, dynamic level-of-detail graph abstraction, and template-based fuzzy classification using neural networks. We demonstrate our system using real-world workflows from several large-scale studies. Parallel coordinates has proven to be a scalable visualization and navigation framework for multivariate data. However, when data with thousands of variables are at hand, we do not have a comprehensive solution to select the right set of variables and order them to uncover important or potentially insightful patterns. We present algorithms to rank axes based upon the importance of bivariate relationships among the variables and showcase the efficacy of the proposed system by demonstrating autonomous detection of patterns in a modern large-scale dataset of time-varying climate simulation

Robust Hand Motion Capture and Physics-Based Control for Grasping in Real Time

Hand motion capture technologies are being explored due to high demands in the fields such as video game, virtual reality, sign language recognition, human-computer interaction, and robotics. However, existing systems suffer a few limitations, e.g. they are high-cost (expensive capture devices), intrusive (additional wear-on sensors or complex configurations), and restrictive (limited motion varieties and restricted capture space). This dissertation mainly focus on exploring algorithms and applications for the hand motion capture system that is low-cost, non-intrusive, low-restriction, high-accuracy, and robust. More specifically, we develop a realtime and fully-automatic hand tracking system using a low-cost depth camera. We first introduce an efficient shape-indexed cascaded pose regressor that directly estimates 3D hand poses from depth images. A unique property of our hand pose regressor is to utilize a low-dimensional parametric hand geometric model to learn 3D shape-indexed features robust to variations in hand shapes, viewpoints and hand poses. We further introduce a hybrid tracking scheme that effectively complements our hand pose regressor with model-based hand tracking. In addition, we develop a rapid 3D hand shape modeling method that uses a small number of depth images to accurately construct a subject-specific skinned mesh model for hand tracking. This step not only automates the whole tracking system but also improves the robustness and accuracy of model-based tracking and hand pose regression. Additionally, we also propose a physically realistic human grasping synthesis method that is capable to grasp a wide variety of objects. Given an object to be grasped, our method is capable to compute required controls (e.g. forces and torques) that advance the simulation to achieve realistic grasping. Our method combines the power of data-driven synthesis and physics-based grasping control. We first introduce a data-driven method to synthesize a realistic grasping motion from large sets of prerecorded grasping motion data. And then we transform the synthesized kinematic motion to a physically realistic one by utilizing our online physics-based motion control method. In addition, we also provide a performance interface which allows the user to act out before a depth camera to control a virtual object

Robust Hand Motion Capture and Physics-Based Control for Grasping in Real Time

Hand motion capture technologies are being explored due to high demands in the fields such as video game, virtual reality, sign language recognition, human-computer interaction, and robotics. However, existing systems suffer a few limitations, e.g. they are high-cost (expensive capture devices), intrusive (additional wear-on sensors or complex configurations), and restrictive (limited motion varieties and restricted capture space). This dissertation mainly focus on exploring algorithms and applications for the hand motion capture system that is low-cost, non-intrusive, low-restriction, high-accuracy, and robust. More specifically, we develop a realtime and fully-automatic hand tracking system using a low-cost depth camera. We first introduce an efficient shape-indexed cascaded pose regressor that directly estimates 3D hand poses from depth images. A unique property of our hand pose regressor is to utilize a low-dimensional parametric hand geometric model to learn 3D shape-indexed features robust to variations in hand shapes, viewpoints and hand poses. We further introduce a hybrid tracking scheme that effectively complements our hand pose regressor with model-based hand tracking. In addition, we develop a rapid 3D hand shape modeling method that uses a small number of depth images to accurately construct a subject-specific skinned mesh model for hand tracking. This step not only automates the whole tracking system but also improves the robustness and accuracy of model-based tracking and hand pose regression. Additionally, we also propose a physically realistic human grasping synthesis method that is capable to grasp a wide variety of objects. Given an object to be grasped, our method is capable to compute required controls (e.g. forces and torques) that advance the simulation to achieve realistic grasping. Our method combines the power of data-driven synthesis and physics-based grasping control. We first introduce a data-driven method to synthesize a realistic grasping motion from large sets of prerecorded grasping motion data. And then we transform the synthesized kinematic motion to a physically realistic one by utilizing our online physics-based motion control method. In addition, we also provide a performance interface which allows the user to act out before a depth camera to control a virtual object

IMPLEMENTATION OF REALTIME OBJECT TRACKING

Abstract: Emphasis is made on the development of techniques for tracking a user-selected object through digital image sequences in multimedia systems. The ability to track objects in sequences is useful in situations where the motion of objects is important. It is also useful when a moving object is difficult to see and needs to be highlighted. A tracking tool is also of benefit to the developers of a multimedia system to automatically 'mark up' a moving object so a user of the system can select it and manipulate it

Ultra-wideband indoor communications using optical technology

La communication ultra large bande (UWB) a attiré une énorme quantité de recherches ces dernières années, surtout après la présentation du masque spectral de US Federal Communications Commission (FCC). Les impulsions ultra-courtes permettent de très hauts débits de faible puissance tout en éliminant les interférences avec les systèmes existants à bande étroite. La faible puissance, cependant, limite la portée de propagation des radios UWB à quelques mètres pour la transmission sans fil à l’intérieur d’une pièce. En outre, des signaux UWB reçu sont étendus dans le temps en raison de la propagation par trajet multiple qui résulte en beaucoup d’interférence inter-symbole (ISI) à haut débit. Le monocycle Gaussien, l’impulsion la plus commune dans UWB, a une mauvaise couverture sous le masque de la FCC. Dans cette thèse, nous démontrons des transmet- teurs qui sont capables de générer des impulsions UWB avec une efficacité de puissance élevée. Une impulsion efficace résulte dans un rapport de signal à bruit (SNR) supérieur au récepteur en utilisant plus de la puissance disponible sous le masque spectral de la FCC. On produit les impulsions dans le domaine optique et utilise la fibre optique pour les transporter sur plusieurs kilomètres pour la distribution dans un réseau optique pas- sif. La fibre optique est très fiable pour le transport des signaux radio avec une faible consommation de puissance. On utilise les éléments simples comme un modulateur Mach-Zehnder ou un résonateur en anneau pour générer des impulsions, ce qui permet l’intégration dans le silicium. Compatible avec la technologie CMOS, la photonique sur silicium a un potentiel énorme pour abaisser le coût et l’encombrement des systèmes optiques. La photodétection convertit les impulsions optiques en impulsions électriques avant la transmission sur l’antenne du côté de l’utilisateur. La réponse fréquentielle de l’antenne déforme la forme d’onde de l’impulsion UWB. Nous proposons une technique d’optimisation non-linéaire qui prend en compte la distorsion d’antenne pour trouver des impulsions qui maximisent la puissance transmise, en respectant le masque spectral de la FCC. Nous travaillons avec trois antennes et concevons une impulsion unique pour chacune d’entre elle. L’amélioration de l’énergie des impulsions UWB améliore directement la SNR au récepteur. Les résultats de simulation montrent que les impulsions optimisées améliorent considérablement le taux d’erreur (BER) par rapport au monocycle Gaussien sous propagation par trajet multiple. Notre autre contribution est l’évaluation d’un filtre adapté pour recevoir efficacement des impulsions UWB. Le filtre adapté est synthétisé et fabriqué en technologie microstrip, en collaboration avec l’Université McGill comme un dispositif de bande interdite électromagnétique. La réponse fréquentielle du filtre adapté montre une ex- cellente concordance avec le spectre ciblé de l’impulsion UWB. Les mesures de BER confirment la performance supérieure du filtre adapté par rapport à un récepteur à conversion directe. Le canal UWB est très riche en trajet multiple conduisant à l’ISI à haut débit. Notre dernière contribution est l’étude de performance des récepteurs en simulant un système avec des conditions de canaux réalistes. Les résultats de la simulation montrent que la performance d’un tel système se dégrade de façon significative pour les hauts débits. Afin de compenser la forte ISI dans les taux de transfert de données en Gb/s, nous étudions l’algorithme de Viterbi (VA) avec un nombre limité d’états et un égaliseur DFE (decision feedback equalizer). Nous examinons le nombre d’états requis dans le VA, et le nombre de coefficients du filtre dans le DFE pour une transmission fiable de UWB en Gb/s dans les canaux en ligne de vue. L’évaluation par simulation de BER confirme que l’égalisation améliore considérablement les performances par rapport à la détection de symbole. La DFE a une meilleure performance par rapport à la VA en utilisant une complexité comparable. La DFE peut couvrir une plus grande mémoire de canal avec un niveau de complexité relativement réduit.Ultra-wideband (UWB) communication has attracted an enormous amount of research in recent years, especially after the introduction of the US Federal Communications Commission (FCC) spectral mask. Ultra-short pulses allow for very high bit-rates while low power eliminates interference with existing narrowband systems. Low power, however, limits the propagation range of UWB radios to a few meters for indoors wireless transmission. Furthermore, received UWB signals are spread in time because of multipath propagation which results in high intersymbol interference at high data rates. Gaussian monocycle, the most commonly employed UWB pulse, has poor coverage under the FCC mask. In this thesis we demonstrate transmitters capable of generating UWB pulses with high power efficiency at Gb/s bit-rates. An efficient pulse results in higher signal-to-noise ratio (SNR) at the receiver by utilizing most of the available power under the FCC spectral mask. We generate the pulses in the optical domain and use optical fiber to transport the pulses over several kilometers for distribution in a passive optical network. Optical fiber is very reliable for transporting radio signals with low power consumption. We use simple elements such as a Mach Zehnder modulator or a ring resonator for pulse shaping, allowing for integration in silicon. Being compatible with CMOS technology, silicon photonics has huge potential for lowering the cost and bulkiness of optical systems. Photodetection converts the pulses to the electrical domain before antenna transmission at the user side. The frequency response of UWB antennas distorts the UWB waveforms. We pro- pose a nonlinear optimization technique which takes into account antenna distortion to find pulses that maximize the transmitted power, while respecting the FCC spectral mask. We consider three antennas and design a unique pulse for each. The energy improvement in UWB pulses directly improves the receiver SNR. Simulation results show that optimized pulses have a significant bit error rate (BER) performance improvement compared to the Gaussian monocycle under multipath propagation. Our other contribution is evaluating a matched filter to receive efficiently designed UWB pulses. The matched filter is synthesized and fabricated in microstrip technology in collaboration with McGill University as an electromagnetic bandgap device. The frequency response of the matched filter shows close agreement with the target UWB pulse spectrum. BER measurements confirm superior performance of the matched filter compared to a direct conversion receiver. The UWB channel is very rich in multipath leading to ISI at high bit rates. Our last contribution is investigating the performance of receivers by simulating a system employing realistic channel conditions. Simulation results show that the performance of such system degrades significantly for high data rates. To compensate the severe ISI at gigabit rates, we investigate the Viterbi algorithm (VA) with a limited number of states and the decision feedback equalizer (DFE). We examine the required number of states in the VA, and the number of taps in the DFE for reliable Gb/s UWB trans- mission for line-of-sight channels. Non-line-of-sight channels were also investigated at lower speeds. BER simulations confirm that equalization considerably improves the performance compared to symbol detection. The DFE results in better performance compared to the VA when using comparable complexity as the DFE can cover greater channel memory with a relatively low complexity level

Proceedings of the Linux Audio Conference 2018

These proceedings contain all papers presented at the Linux Audio Conference 2018. The conference took place at c-base, Berlin, from June 7th - 10th, 2018 and was organized in cooperation with the Electronic Music Studio at TU Berlin

DirectMHP: Direct 2D Multi-Person Head Pose Estimation with Full-range Angles

Existing head pose estimation (HPE) mainly focuses on single person with pre-detected frontal heads, which limits their applications in real complex scenarios with multi-persons. We argue that these single HPE methods are fragile and inefficient for Multi-Person Head Pose Estimation (MPHPE) since they rely on the separately trained face detector that cannot generalize well to full viewpoints, especially for heads with invisible face areas. In this paper, we focus on the full-range MPHPE problem, and propose a direct end-to-end simple baseline named DirectMHP. Due to the lack of datasets applicable to the full-range MPHPE, we firstly construct two benchmarks by extracting ground-truth labels for head detection and head orientation from public datasets AGORA and CMU Panoptic. They are rather challenging for having many truncated, occluded, tiny and unevenly illuminated human heads. Then, we design a novel end-to-end trainable one-stage network architecture by joint regressing locations and orientations of multi-head to address the MPHPE problem. Specifically, we regard pose as an auxiliary attribute of the head, and append it after the traditional object prediction. Arbitrary pose representation such as Euler angles is acceptable by this flexible design. Then, we jointly optimize these two tasks by sharing features and utilizing appropriate multiple losses. In this way, our method can implicitly benefit from more surroundings to improve HPE accuracy while maintaining head detection performance. We present comprehensive comparisons with state-of-the-art single HPE methods on public benchmarks, as well as superior baseline results on our constructed MPHPE datasets. Datasets and code are released in https://github.com/hnuzhy/DirectMHP.Comment: 13 page