Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables 3D human pose estimation using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall.Comment: 12 pages, Accepted at Eurographics 201

LiveCap: Real-time Human Performance Capture from Monocular Video

We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video. We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned template model is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting non-linear optimization problems per-frame are solved with specially-tailored data-parallel Gauss-Newton solvers. In order to achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques, while being orders of magnitude faster

Paroxysmal eye–head movements in Glut1 deficiency syndrome

Objective:To describe a characteristic paroxysmal eye–head movement disorder that occurs in infants with Glut1 deficiency syndrome (Glut1 DS).Methods:We retrospectively reviewed the medical charts of 101 patients with Glut1 DS to obtain clinical data about episodic abnormal eye movements and analyzed video recordings of 18 eye movement episodes from 10 patients.Results:A documented history of paroxysmal abnormal eye movements was found in 32/101 patients (32%), and a detailed description was available in 18 patients, presented here. Episodes started before age 6 months in 15/18 patients (83%), and preceded the onset of seizures in 10/16 patients (63%) who experienced both types of episodes. Eye movement episodes resolved, with or without treatment, by 6 years of age in 7/8 patients with documented long-term course. Episodes were brief (usually &lt;5 minutes). Video analysis revealed that the eye movements were rapid, multidirectional, and often accompanied by a head movement in the same direction. Eye movements were separated by clear intervals of fixation, usually ranging from 200 to 800 ms. The movements were consistent with eye–head gaze saccades. These movements can be distinguished from opsoclonus by the presence of a clear intermovement fixation interval and the association of a same-direction head movement.Conclusions:Paroxysmal eye–head movements, for which we suggest the term aberrant gaze saccades, are an early symptom of Glut1 DS in infancy. Recognition of the episodes will facilitate prompt diagnosis of this treatable neurodevelopmental disorder.</jats:sec

An empirical study of Conv-TasNet

Conv-TasNet is a recently proposed waveform-based deep neural network that achieves state-of-the-art performance in speech source separation. Its architecture consists of a learnable encoder/decoder and a separator that operates on top of this learned space. Various improvements have been proposed to Conv-TasNet. However, they mostly focus on the separator, leaving its encoder/decoder as a (shallow) linear operator. In this paper, we conduct an empirical study of Conv-TasNet and propose an enhancement to the encoder/decoder that is based on a (deep) non-linear variant of it. In addition, we experiment with the larger and more diverse LibriTTS dataset and investigate the generalization capabilities of the studied models when trained on a much larger dataset. We propose cross-dataset evaluation that includes assessing separations from the WSJ0-2mix, LibriTTS and VCTK databases. Our results show that enhancements to the encoder/decoder can improve average SI-SNR performance by more than 1 dB. Furthermore, we offer insights into the generalization capabilities of Conv-TasNet and the potential value of improvements to the encoder/decoder.Comment: In proceedings of ICASSP202

Gender Differences in COVID-19 Related Attitudes and Behavior: Evidence from a Panel Survey in Eight OECD Countries

Using original data from two waves of a survey conducted in March and April 2020 in eight OECD countries (N = 21,649), we show that women are more likely to see COVID-19 as a very serious health problem, to agree with restraining public policy measures adopted in response to it, and to comply with them. Gender differences in attitudes and behavior are substantial in all countries, robust to controlling for a large set of sociodemographic, employment, psychological, and behavioral factors, and only partially mitigated for individuals who cohabit or have direct exposure to COVID-19. The results are not driven by differential social desirability bias. They carry important implications for the spread of the pandemic and may contribute to explain gender differences in vulnerability to it

Adding tightly-integrated task scheduling acceleration to a RISC-V multi-core processor

Task Parallelism is a parallel programming model that provides code annotation constructs to outline tasks and describe how their pointer parameters are accessed so that they might be executed in parallel, and asynchronously, by a runtime capable of inferring and honoring their data dependence relationships. It is supported by several parallelization frameworks, as OpenMP and StarSs. Overhead related to automatic dependence inference and to the scheduling of ready-to-run tasks is a major performance limiting factor of Task Parallel systems. To amortize this overhead, programmers usually trade the higher parallelism that could be leveraged from finer-grained work partitions for the higher runtime-efficiency of coarser-grained work partitions. Such problems are even more severe for systems with many cores, as the task spawning frequency required for preserving cores from starvation grows linearly with their number. To mitigate these problems, researchers have designed hardware accelerators to improve runtime performance. Nevertheless, the high CPU-accelerator communication overheads of these solutions hampered their gains. We thus propose a RISC-V based architecture that minimizes communication overhead between the HW Task Scheduler and the CPU by allowing Task Scheduling software to directly interact with the former through custom instructions. Empirical evaluation of the architecture is made possible by an FPGA prototype featuring an eight-core Linux-capable Rocket Chip implementing such instructions. To evaluate the prototype performance, we both (1) adapted Nanos, a mature Task Scheduling runtime, to benefit from the new task-scheduling-accelerating instructions; and (2) developed Phentos, a new HW-accelerated light weight Task Scheduling runtime. Our experiments show that task parallel programs using Nanos-RV --- the Nanos version ported to our system --- are on average 2.13 times faster than those being serviced by baseline Nanos, while programs running on Phentos are 13.19 times faster, considering geometric means. Using eight cores, Nanos-RV is able to deliver speedups with respect to serial execution of up to 5.62 times, while Phentos produces speedups of up to 5.72 times.This work was supported by the Spanish Government (projects SEV-2015-0493 and TIN2015-65316-P), the Generalitat de Catalunya (2017-SGR-1414 and 2017-SGR1328), FAPESP (grants 2017/02682-2, 2018/00687-0, and 2014/25694-8), CNPq (grant 408782/2016-1), and CAPES.Peer ReviewedPostprint (author's final draft

Structuring chemical space: similarity-based characterization of the PubChem Database

The ensemble of conceivable molecules is referred to as the Chemical Space. In this article we describe a hierarchical version of the Affinity Propagation (AP) clustering algorithm and apply it to analyze the LINGO‐based similarity matrix of a 500 000‐molecule subset of the PubChem database, which contains more than 19 million compounds. The combination of two highly efficient methods, namely the AP clustering algorithm and LINGO‐based molecular similarity calculations, allows the unbiased analysis of large databases. Hierarchical clustering generates a numerical diagonalization of the similarity matrix. The target‐independent, intrinsic structure of the database , derived without any previous information on the physical or biological properties of the compounds, maps together molecules experimentally shown to bind the same biological target or to have similar physical propertie