End-to-end Audiovisual Speech Activity Detection with Bimodal Recurrent Neural Models

Speech activity detection (SAD) plays an important role in current speech processing systems, including automatic speech recognition (ASR). SAD is particularly difficult in environments with acoustic noise. A practical solution is to incorporate visual information, increasing the robustness of the SAD approach. An audiovisual system has the advantage of being robust to different speech modes (e.g., whisper speech) or background noise. Recent advances in audiovisual speech processing using deep learning have opened opportunities to capture in a principled way the temporal relationships between acoustic and visual features. This study explores this idea proposing a \emph{bimodal recurrent neural network} (BRNN) framework for SAD. The approach models the temporal dynamic of the sequential audiovisual data, improving the accuracy and robustness of the proposed SAD system. Instead of estimating hand-crafted features, the study investigates an end-to-end training approach, where acoustic and visual features are directly learned from the raw data during training. The experimental evaluation considers a large audiovisual corpus with over 60.8 hours of recordings, collected from 105 speakers. The results demonstrate that the proposed framework leads to absolute improvements up to 1.2% under practical scenarios over a VAD baseline using only audio implemented with deep neural network (DNN). The proposed approach achieves 92.7% F1-score when it is evaluated using the sensors from a portable tablet under noisy acoustic environment, which is only 1.0% lower than the performance obtained under ideal conditions (e.g., clean speech obtained with a high definition camera and a close-talking microphone).Comment: Submitted to Speech Communicatio

Speech-driven Animation with Meaningful Behaviors

Conversational agents (CAs) play an important role in human computer interaction. Creating believable movements for CAs is challenging, since the movements have to be meaningful and natural, reflecting the coupling between gestures and speech. Studies in the past have mainly relied on rule-based or data-driven approaches. Rule-based methods focus on creating meaningful behaviors conveying the underlying message, but the gestures cannot be easily synchronized with speech. Data-driven approaches, especially speech-driven models, can capture the relationship between speech and gestures. However, they create behaviors disregarding the meaning of the message. This study proposes to bridge the gap between these two approaches overcoming their limitations. The approach builds a dynamic Bayesian network (DBN), where a discrete variable is added to constrain the behaviors on the underlying constraint. The study implements and evaluates the approach with two constraints: discourse functions and prototypical behaviors. By constraining on the discourse functions (e.g., questions), the model learns the characteristic behaviors associated with a given discourse class learning the rules from the data. By constraining on prototypical behaviors (e.g., head nods), the approach can be embedded in a rule-based system as a behavior realizer creating trajectories that are timely synchronized with speech. The study proposes a DBN structure and a training approach that (1) models the cause-effect relationship between the constraint and the gestures, (2) initializes the state configuration models increasing the range of the generated behaviors, and (3) captures the differences in the behaviors across constraints by enforcing sparse transitions between shared and exclusive states per constraint. Objective and subjective evaluations demonstrate the benefits of the proposed approach over an unconstrained model.Comment: 13 pages, 12 figures, 5 table

s-Processing in the Galactic Disk. I. Super-Solar Abundances of Y, Zr, La, Ce in Young Open Clusters

In a recent study, based on homogeneous barium abundance measurements in open clusters, a trend of increasing [Ba/Fe] ratios for decreasing cluster age was reported. We present here further abundance determinations, relative to four other elements hav- ing important s-process contributions, with the aim of investigating whether the growth found for [Ba/Fe] is or not indicative of a general property, shared also by the other heavy elements formed by slow neutron captures. In particular, we derived abundances for yttrium, zirconium, lanthanum and cerium, using equivalent widths measurements and the MOOG code. Our sample includes 19 open clusters of different ages, for which the spectra were obtained at the ESO VLT telescope, using the UVES spectrometer. The growth previously suggested for Ba is confirmed for all the elements analyzed in our study. This fact implies significant changes in our views of the Galactic chemical evolution for elements beyond iron. Our results necessarily require that very low-mass AGB stars (M < 1.5M\odot) produce larger amounts of s-process elements (hence acti- vate the 13 C-neutron source more effectively) than previously expected. Their role in producing neutron-rich elements in the Galactic disk has been so far underestimated and their evolution and neutron-capture nucleosynthesis should now be reconsidered.Comment: ApJ accepte

Seed weight variation of wyoming sagebrush in Northern Nevada

Seed size is a crucial plant trait that may potentially affect not only immediate seedling success but also the subsequent generation. We examined variation in seed weight of Wyoming sagebrush (Artemisia tridentata ssp. wyomingensis Beetle and Young), an excellent candidate species for rangeland restoration. The working hypothesis was that a major fraction of spatial and temporal variability in seed size (weight) of Wyoming sagebrush could be explained by variations in mean monthly temperatures and precipitation. Seed collection was conducted at Battle Mountain and Eden Valley sites in northern Nevada, USA, during November of 2002 and 2003. Frequency distributions of seed weight varied from leptokurtic to platykurtic, and from symmetry to skewness to the right for both sites and years. Mean seed weight varied by a factor of 1.4 between locations and years. Mean seed weight was greater (P0.05) in all study situations. Simple linear regression showed that monthly precipitation (March to November) explained 85% of the total variation in mean seed weight ( P=0.079). Since the relationship between mean monthly temperature (June-November) and mean seed weight was not significant (r2=0.00, P=0.431), this emphasizes the importance of precipitation as an important determinant of mean seed weight. Our results suggest that the precipitation regime to which the mother plant is exposed can have a significant effect on sizes of seeds produced. Hence, seasonal changes in water availability would tend to alter size distributions of produced offspring.Fil: Busso, Carlos Alberto. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Centro de Recursos Naturales Renovables de la Zona Semiárida. Universidad Nacional del Sur. Centro de Recursos Naturales Renovables de la Zona Semiárida; ArgentinaFil: Perryman, Barry L.. University of Nevada; Estados Unido

Semi-Supervised Speech Emotion Recognition with Ladder Networks

Speech emotion recognition (SER) systems find applications in various fields such as healthcare, education, and security and defense. A major drawback of these systems is their lack of generalization across different conditions. This problem can be solved by training models on large amounts of labeled data from the target domain, which is expensive and time-consuming. Another approach is to increase the generalization of the models. An effective way to achieve this goal is by regularizing the models through multitask learning (MTL), where auxiliary tasks are learned along with the primary task. These methods often require the use of labeled data which is computationally expensive to collect for emotion recognition (gender, speaker identity, age or other emotional descriptors). This study proposes the use of ladder networks for emotion recognition, which utilizes an unsupervised auxiliary task. The primary task is a regression problem to predict emotional attributes. The auxiliary task is the reconstruction of intermediate feature representations using a denoising autoencoder. This auxiliary task does not require labels so it is possible to train the framework in a semi-supervised fashion with abundant unlabeled data from the target domain. This study shows that the proposed approach creates a powerful framework for SER, achieving superior performance than fully supervised single-task learning (STL) and MTL baselines. The approach is implemented with several acoustic features, showing that ladder networks generalize significantly better in cross-corpus settings. Compared to the STL baselines, the proposed approach achieves relative gains in concordance correlation coefficient (CCC) between 3.0% and 3.5% for within corpus evaluations, and between 16.1% and 74.1% for cross corpus evaluations, highlighting the power of the architecture

Isotope Anomalies in the Fe-group Elements in Meteorites and Connection to Nucleosynthesis in AGB Stars

We study the effects of neutron captures in AGB stars on \oq Fe-group\cqb elements, with an emphasis on Cr, Fe, and Ni. These elements show anomalies in $^{54}$Cr, $^{58}$Fe, and $^{64}$Ni in solar-system materials, which are commonly attributed to SNe. However, as large fractions of the interstellar medium (ISM) were reprocessed in AGB stars, these elements were reprocessed, too. We calculate the effects of such reprocessing on Cr, Fe, and Ni through 1.5\msb and 3\msb AGB models, adopting solar and 1/3 solar metallicities. All cases produce excesses of $^{54}$Cr, $^{58}$Fe, and $^{64}$Ni, while the other isotopes are little altered; hence, the observations may be explained by AGB processing. The results are robust and not dependent on the detailed initial isotopic composition. Consequences for other \oq Fe group\cqb elements are then explored. They include $^{50}$Ti excesses, and some production of $^{46,47,49}$Ti. In many circumstellar condensates, Ti quantitatively reflects these effects of AGB neutron captures. Scatter in the data results from small variations (granularity) in the isotopic composition of the local ISM. For Si, the main effects are instead due to variations in the local ISM from different SNe sources. The problem of Ca is discussed, particularly with regard to $^{48}$Ca. The measured data are usually represented assuming terrestrial values for $^{42}$Ca/$^{44}$Ca. Materials processed in AGB stars or sources with variable initial $^{42}$Ca/$^{44}$Ca ratios can give apparent $^{48}$Ca excesses/deficiencies, attributed to SNe. The broader issue of Galactic Chemical Evolution is also discussed in view of the isotopic granularity in the ISM. \end{abstract

Theoretical Estimates of Stellar e-Captures. I. The half-life of 7Be in Evolved Stars

The Li enrichment in the Universe still presents various puzzles to astrophysics. One open issue is that of obtaining estimates for the rate of e-captures on 7Be, for T and rho conditions different from solar. This is important to model the Galactic nucleosynthesis of Li. In this framework, we present a new theoretical method for calculating the e-capture rate in conditions typical of evolved stars. We show how our approach compares with state-of-the-art techniques for solar conditions, where various estimates are available. Our computations include: i) "traditional" calculations of the electronic density at the nucleus, to which the e-capture rate for 7Be is proportional, for different theoretical approaches including the Thomas--Fermi, Poisson--Boltzmann and Debye--Hueckel (DH) models of screening, ii) a new computation, based on a formalism that goes beyond the previous ones, adopting a mean-field "adiabatic" approximation to the scattering process. The results obtained with our approach as well as with the traditional ones and their differences are discussed in some detail, starting from solar conditions, where our method and the DH model converge to the same solution. We then analyze the applicability of the various models to a rather broad range of T and rho values, embracing those typical of red giant stars. We find that, over a wide region of the parameter space explored, the DH approximation does not stand, and the more general method we suggest is preferable. We then briefly reanalyze the 7Li abundances in RGB and AGB stars of the Galactic Disk using the new Be-decay rate. We also underline that the different values of the electron density at the nucleus we find should induce effects on electron screening (for p-captures on Li itself, as well as for other nuclei) so that our new approach might have wide astrophysical consequences.Comment: Astrophts. Journal Feb. 1, 201

New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators

Currently available asymptotic results in the literature suggest that matching estimators have higher variance than reweighting estimators. The extant literature comparing the finite sample properties of matching to specific reweighting estimators, however, has concluded that reweighting performs far worse than even the simplest matching estimator. We resolve this puzzle. We show that the findings from the finite sample analyses are not inconsistent with asymptotic analysis, but are very specific to particular choices regarding the implementation of reweighting, and fail to generalize to settings likely to be encountered in actual empirical practice. In the DGPs studied here, reweighting typically outperforms propensity score matching.treatment effects, propensity score, semiparametric efficiency

Infrared Properties Of AGB Stars: from Existing Databases to Antarctic Surveys

We present here a study of the Infrared properties of Asymptotic Giant Branch stars (hereafter AGB) based on existing databases, mainly from space-borne experiments. Preliminary results about C and S stars are discussed, focusing on the topics for which future Infrared surveys from Antarctica will be crucial. This kind of surveys will help in making more quantitative our knowledge of the last evolutionary stages of low mass stars, especially for what concerns luminosities and mass loss.Comment: 6 pages, 3 figures. Contribution from the 1st ARENA Conference on "Large Astronomical Infrastructures at CONCORDIA, prospects and constraints for Antarctic Optical/IR Astronomy" held 16-19 October 2006 at Roscoff, Franc