Capture, Learning, and Synthesis of 3D Speaking Styles

Audio-driven 3D facial animation has been widely explored, but achieving realistic, human-like performance is still unsolved. This is due to the lack of available 3D datasets, models, and standard evaluation metrics. To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers. We then train a neural network on our dataset that factors identity from facial motion. The learned model, VOCA (Voice Operated Character Animation) takes any speech signal as input - even speech in languages other than English - and realistically animates a wide range of adult faces. Conditioning on subject labels during training allows the model to learn a variety of realistic speaking styles. VOCA also provides animator controls to alter speaking style, identity-dependent facial shape, and pose (i.e. head, jaw, and eyeball rotations) during animation. To our knowledge, VOCA is the only realistic 3D facial animation model that is readily applicable to unseen subjects without retargeting. This makes VOCA suitable for tasks like in-game video, virtual reality avatars, or any scenario in which the speaker, speech, or language is not known in advance. We make the dataset and model available for research purposes at http://voca.is.tue.mpg.de.Comment: To appear in CVPR 201

Spectroscopic measurement of cortical nitric oxide release induced by ascending activation

[Abstract] The transition from sleep to the awake state is regulated by the activation of subcortical nuclei of the brainstem (BS) and basal forebrain (BF), releasing acetylcholine and glutamate throughout the cortex and inducing a tonic state of neural activity. It has been suggested that such activation is also mediated by the massive and diffuse cortical release of nitric oxide (NO). In this work we have combined the spectroscopic measurement of NO levels in the somatosensory cortex of the cat through its marker methemoglobin, as well as two other hemodynamic markers (oxyhemoglobin – oxyHb – and deoxyhemoglobin – deoxyHb), together with the electrical stimulation of BS and BF – to induce an experimental transition from a sleep-like state to an awake-like mode. The results show an increase of NO levels either after BS or BF activation. The response induced by BS stimulation was biphasic in the three studied markers, and lasted for up to 30 s. The changes induced by BF were monophasic lasting for up to 20 s. The systemic blockade of NO production abolished the observed responses to BS whereas responses to BF stimulation were much less affected. These results indicate a crucial role for NO in the neuronal activation induced by the ascending systems.Galicia. Consellería de Economía e Industria; INCITE09137272P

Responses of primate LGN cells to moving stimuli involve a constant background modulation by feedback from area MT

The feedback connections from the cortical motion area middle temporal (MT), to layer 6 of the primary visual cortex (V1), have the capacity to drive a cascaded feedback influence from the layer 6 cortico-geniculate cells back to the lateral geniculate nucleus (LGN) relay cells. This introduces the possibility of a re-entrant motion signal affecting the relay of the retinal input through the LGN to the visual cortex. The question is whether the response of LGN cells to moving stimuli involves a component derived from this feedback. By producing a reversible focal pharmacological block of the activity of an MT direction column we show the presence of such an influence from MT on the responses of magno, parvo and koniocellular cells in the macaque LGN. The pattern of effect in the LGN reflects the direction bias of the MT location inactivated. This suggests a moving stimulus is captured by iterative interactions in the circuit formed by visual cortical areas and visual thalamus

The effects of waveform and current direction on the efficacy and test–retest reliability of transcranial magnetic stimulation

[Abstract] The pulse waveform and current direction of transcranial magnetic stimulation (TMS) influence its interactions with the neural substrate; however, their role in the efficacy and reliability of single- and paired-pulse TMS measures is not fully understood. We investigated how pulse waveform and current direction affect the efficacy and test–retest reliability of navigated, single- and paired-pulse TMS measures. 23 healthy adults (aged 18–35 years) completed two identical TMS sessions, assessing resting motor threshold (RMT), motor-evoked potentials (MEPs), cortical silent period (cSP), short- and long-interval intra-cortical inhibition (SICI and LICI), and intracortical facilitation (ICF) using either monophasic posterior–anterior (monoPA; n = 9), monophasic anterior–posterior (monoAP; n = 7), or biphasic (biAP-PA; n = 7) pulses. Averages of each TMS measure were compared across the three groups and intraclass correlation coefficients were calculated to assess test–retest reliability. RMT was the lowest and cSP was the longest with biAP-PA pulses, whereas MEP latency was the shortest with monoPA pulses. SICI and LICI had the largest effect with monoPA pulses, whereas only monoAP and biAP-PA pulses resulted in significant ICF. MEP amplitude was more reliable with either monoPA or monoAP than with biAP-PA pulses. LICI was the most reliable with monoAP pulses, whereas ICF was the most reliable with biAP-PA pulses. Waveform/current direction influenced RMT, MEP latency, cSP, SICI, LICI, and ICF, as well as the reliability of MEP amplitude, LICI, and ICF. These results show the importance of considering TMS pulse parameters for optimizing the efficacy and reliability of TMS neurophysiologic measures

Stride: a flexible software platform for high-performance ultrasound computed tomography

BACKGROUND AND OBJECTIVE: Advanced ultrasound computed tomography techniques like full-waveform inversion are mathematically complex and orders of magnitude more computationally expensive than conventional ultrasound imaging methods. This computational and algorithmic complexity, and a lack of open-source libraries in this field, represent a barrier preventing the generalised adoption of these techniques, slowing the pace of research, and hindering reproducibility. Consequently, we have developed Stride, an open-source Python library for the solution of large-scale ultrasound tomography problems. METHODS: On one hand, Stride provides high-level interfaces and tools for expressing the types of optimisation problems encountered in medical ultrasound tomography. On the other, these high-level abstractions seamlessly integrate with high-performance wave-equation solvers and with scalable parallelisation routines. The wave-equation solvers are generated automatically using Devito, a domain-specific language, and the parallelisation routines are provided through the custom actor-based library Mosaic. RESULTS: We demonstrate the modelling accuracy achieved by our wave-equation solvers through a comparison (1) with analytical solutions for a homogeneous medium, and (2) with state-of-the-art modelling software applied to a high-contrast, complex skull section. Additionally, we show through a series of examples how Stride can handle realistic numerical and experimental tomographic problems, in 2D and 3D, and how it can scale robustly from a local multi-processing environment to a multi-node high-performance cluster. CONCLUSIONS: Stride enables researchers to rapidly and intuitively develop new imaging algorithms and to explore novel physics without sacrificing performance and scalability. This will lead to faster scientific progress in this field and will significantly ease clinical translation

cGMP-Dependent Protein Kinase Type I Is Implicated in the Regulation of the Timing and Quality of Sleep and Wakefulness

Many effects of nitric oxide (NO) are mediated by the activation of guanylyl cyclases and subsequent production of the second messenger cyclic guanosine-3′,5′-monophosphate (cGMP). cGMP activates cGMP-dependent protein kinases (PRKGs), which can therefore be considered downstream effectors of NO signaling. Since NO is thought to be involved in the regulation of both sleep and circadian rhythms, we analyzed these two processes in mice deficient for cGMP-dependent protein kinase type I (PRKG1) in the brain. Prkg1 mutant mice showed a strikingly altered distribution of sleep and wakefulness over the 24 hours of a day as well as reductions in rapid-eye-movement sleep (REMS) duration and in non-REM sleep (NREMS) consolidation, and their ability to sustain waking episodes was compromised. Furthermore, they displayed a drastic decrease in electroencephalogram (EEG) power in the delta frequency range (1–4 Hz) under baseline conditions, which could be normalized after sleep deprivation. In line with the re-distribution of sleep and wakefulness, the analysis of wheel-running and drinking activity revealed more rest bouts during the activity phase and a higher percentage of daytime activity in mutant animals. No changes were observed in internal period length and phase-shifting properties of the circadian clock while chi-squared periodogram amplitude was significantly reduced, hinting at a less robust oscillator. These results indicate that PRKG1 might be involved in the stabilization and output strength of the circadian oscillator in mice. Moreover, PRKG1 deficiency results in an aberrant pattern, and consequently a reduced quality, of sleep and wakefulness, possibly due to a decreased wake-promoting output of the circadian system impinging upon sleep

Virtual Reality as a Tool for Evaluation of Repetitive Rhythmic Movements in the Elderly and Parkinson's Disease Patients

This work presents an immersive Virtual Reality (VR) system to evaluate, and potentially treat, the alterations in rhythmic hand movements seen in Parkinson's disease (PD) and the elderly (EC), by comparison with healthy young controls (YC). The system integrates the subjects into a VR environment by means of a Head Mounted Display, such that subjects perceive themselves in a virtual world consisting of a table within a room. In this experiment, subjects are presented in 1st person perspective, so that the avatar reproduces finger tapping movements performed by the subjects. The task, known as the finger tapping test (FT), was performed by all three subject groups, PD, EC and YC. FT was carried out by each subject on two different days (sessions), one week apart. In each FT session all subjects performed FT in the real world (FTREAL) and in the VR (FTVR); each mode was repeated three times in randomized order. During FT both the tapping frequency and the coefficient of variation of inter-tap interval were registered. FTVR was a valid test to detect differences in rhythm formation between the three groups. Intra-class correlation coefficients (ICC) and mean difference between days for FTVR (for each group) showed reliable results. Finally, the analysis of ICC and mean difference between FTVR vs FTREAL, for each variable and group, also showed high reliability. This shows that FT evaluation in VR environments is valid as real world alternative, as VR evaluation did not distort movement execution and detects alteration in rhythm formation. These results support the use of VR as a promising tool to study alterations and the control of movement in different subject groups in unusual environments, such as during fMRI or other imaging studies

Vasomotion and Neurovascular Coupling in the Visual Thalamus In Vivo

Spontaneous contraction and relaxation of arteries (and in some instances venules) has been termed vasomotion and has been observed in an extensive variety of tissues and species. However, its functions and underlying mechanisms are still under discussion. We demonstrate that in vivo spectrophotometry, measured simultaneously with extracellular recordings at the same locations in the visual thalamus of the cat, reveals vasomotion, measured as an oscillation (0.14hz) in the recorded oxyhemoglobin (OxyHb) signal, which appears spontaneously in the microcirculation and can last for periods of hours. During some non-oscillatory periods, maintained sensory stimulation evokes vasomotion lasting ∼30s, resembling an adaptive vascular phenomenon. This oscillation in the oxyhaemoblobin signal is sensitive to pharmacological manipulation: it is inducible by chloralose anaesthesia and it can be temporarily blocked by systemic administration of adrenaline or acetylcholine (ACh). During these oscillatory periods, neurovascular coupling (i.e. the relationship between local neural activity and the rate of blood supply to that location) appears significantly altered. This raises important questions with regard to the interpretation of results from studies currently dependent upon a linear relationship between neural activity and blood flow, such as neuroimaging