Articulatory Copy Synthesis Based on the Speech Synthesizer VocalTractLab

Articulatory copy synthesis (ACS), a subarea of speech inversion, refers to the reproduction of natural utterances and involves both the physiological articulatory processes and their corresponding acoustic results. This thesis proposes two novel methods for the ACS of human speech using the articulatory speech synthesizer VocalTractLab (VTL) to address or mitigate the existing problems of speech inversion, such as non-unique mapping, acoustic variation among different speakers, and the time-consuming nature of the process. The first method involved finding appropriate VTL gestural scores for given natural utterances using a genetic algorithm. It consisted of two steps: gestural score initialization and optimization. In the first step, gestural scores were initialized using the given acoustic signals with speech recognition, grapheme-to-phoneme (G2P), and a VTL rule-based method for converting phoneme sequences to gestural scores. In the second step, the initial gestural scores were optimized by a genetic algorithm via an analysis-by-synthesis (ABS) procedure that sought to minimize the cosine distance between the acoustic features of the synthetic and natural utterances. The articulatory parameters were also regularized during the optimization process to restrict them to reasonable values. The second method was based on long short-term memory (LSTM) and convolutional neural networks, which were responsible for capturing the temporal dependence and the spatial structure of the acoustic features, respectively. The neural network regression models were trained, which used acoustic features as inputs and produced articulatory trajectories as outputs. In addition, to cover as much of the articulatory and acoustic space as possible, the training samples were augmented by manipulating the phonation type, speaking effort, and the vocal tract length of the synthetic utterances. Furthermore, two regularization methods were proposed: one based on the smoothness loss of articulatory trajectories and another based on the acoustic loss between original and predicted acoustic features. The best-performing genetic algorithms and convolutional LSTM systems (evaluated in terms of the difference between the estimated and reference VTL articulatory parameters) obtained average correlation coefficients of 0.985 and 0.983 for speaker-dependent utterances, respectively, and their reproduced speech achieved recognition accuracies of 86.25% and 64.69% for speaker-independent utterances of German words, respectively. When applied to German sentence utterances, as well as English and Mandarin Chinese word utterances, the neural network based ACS systems achieved recognition accuracies of 73.88%, 52.92%, and 52.41%, respectively. The results showed that both of these methods not only reproduced the articulatory processes but also reproduced the acoustic signals of reference utterances. Moreover, the regularization methods led to more physiologically plausible articulatory processes and made the estimated articulatory trajectories be more articulatorily preferred by VTL, thus reproducing more natural and intelligible speech. This study also found that the convolutional layers, when used in conjunction with batch normalization layers, automatically learned more distinctive features from log power spectrograms. Furthermore, the neural network based ACS systems trained using German data could be generalized to the utterances of other languages

Recent Results on Balanced Symmetric Boolean Functions

In this paper we prove all balanced symmetric Boolean functions of fixed degree are trivial when the number of variables grows large enough. We also present the nonexistence of trivial balanced elementary symmetric Boolean functions except for $n=l\cdot2^{t+1}-1$ and $d=2^t$, where $t$ and $l$ are any positive integers, which shows Cusick\u27s conjecture for balanced elementary symmetric Boolean functions is exactly the conjecture that all balanced elementary symmetric Boolean functions are trivial balanced. In additional, we obtain an integer $n_0$, which depends only on $d$, that Cusick\u27s conjecture holds for any $n>n_0$

A Keypoint Based Enhancement Method for Audio Driven Free View Talking Head Synthesis

Audio driven talking head synthesis is a challenging task that attracts increasing attention in recent years. Although existing methods based on 2D landmarks or 3D face models can synthesize accurate lip synchronization and rhythmic head pose for arbitrary identity, they still have limitations, such as the cut feeling in the mouth mapping and the lack of skin highlights. The morphed region is blurry compared to the surrounding face. A Keypoint Based Enhancement (KPBE) method is proposed for audio driven free view talking head synthesis to improve the naturalness of the generated video. Firstly, existing methods were used as the backend to synthesize intermediate results. Then we used keypoint decomposition to extract video synthesis controlling parameters from the backend output and the source image. After that, the controlling parameters were composited to the source keypoints and the driving keypoints. A motion field based method was used to generate the final image from the keypoint representation. With keypoint representation, we overcame the cut feeling in the mouth mapping and the lack of skin highlights. Experiments show that our proposed enhancement method improved the quality of talking-head videos in terms of mean opinion score

Class I histone deacetylases (HDAC1-3) are histone lysine delactylases

Lysine L-lactylation [K(L-la)] is a newly discovered histone mark stimulated under conditions of high glycolysis, such as the Warburg effect. K(L-la) is associated with functions that are different from the widely studied histone acetylation. While K(L-la) can be introduced by the acetyltransferase p300, histone delactylases enzymes remained unknown. Here, we report the systematic evaluation of zinc- and nicotinamide adenine dinucleotide-dependent histone deacetylases (HDACs) for their ability to cleave ε-N-L-lactyllysine marks. Our screens identified HDAC1-3 and SIRT1-3 as delactylases in vitro. HDAC1-3 show robust activity toward not only K(L-la) but also K(D-la) and diverse short-chain acyl modifications. We further confirmed the de-L-lactylase activity of HDACs 1 and 3 in cells. Together, these data suggest that histone lactylation is installed and removed by regulatory enzymes as opposed to spontaneous chemical reactivity. Our results therefore represent an important step toward full characterization of this pathway's regulatory elements

Crystal structure of rhodopsin bound to arrestin by femtosecond X-ray laser.

G-protein-coupled receptors (GPCRs) signal primarily through G proteins or arrestins. Arrestin binding to GPCRs blocks G protein interaction and redirects signalling to numerous G-protein-independent pathways. Here we report the crystal structure of a constitutively active form of human rhodopsin bound to a pre-activated form of the mouse visual arrestin, determined by serial femtosecond X-ray laser crystallography. Together with extensive biochemical and mutagenesis data, the structure reveals an overall architecture of the rhodopsin-arrestin assembly in which rhodopsin uses distinct structural elements, including transmembrane helix 7 and helix 8, to recruit arrestin. Correspondingly, arrestin adopts the pre-activated conformation, with a ∼20° rotation between the amino and carboxy domains, which opens up a cleft in arrestin to accommodate a short helix formed by the second intracellular loop of rhodopsin. This structure provides a basis for understanding GPCR-mediated arrestin-biased signalling and demonstrates the power of X-ray lasers for advancing the frontiers of structural biology

Soil’s Hidden Power : The Stable Soil Organic Carbon Pool Controls the Burden of Persistent Organic Pollutants in Background Soils

Persistent organic pollutants (POPs) tend to accumulate in cold regions by cold condensation and global distillation. Soil organic matter is the main storage compartment for POPs in terrestrial ecosystems due to deposition and repeated air–surface exchange processes. Here, physicochemical properties and environmental factors were investigated for their role in influencing POPs accumulation in soils of the Tibetan Plateau and Antarctic and Arctic regions. The results showed that the soil burden of most POPs was closely coupled to stable mineral-associated organic carbon (MAOC). Combining the proportion of MAOC and physicochemical properties can explain much of the soil distribution characteristics of the POPs. The background levels of POPs were estimated in conjunction with the global soil database. It led to the proposition that the stable soil carbon pools are key controlling factors affecting the ultimate global distribution of POPs, so that the dynamic cycling of soil carbon acts to counteract the cold-trapping effects. In the future, soil carbon pool composition should be fully considered in a multimedia environmental model of POPs, and the risk of secondary release of POPs in soils under conditions such as climate change can be further assessed with soil organic carbon models

Genome editing reveals dmrt1 as an essential male sex-determining gene in Chinese tongue sole (Cynoglossus semilaevis)

Chinese tongue sole is a marine fish with ZW sex determination. Genome sequencing suggested that the Z-linked dmrt1 is a putative male determination gene, but direct genetic evidence is still lacking. Here we show that TALEN of dmrt1 efficiently induced mutations of this gene. The ZZ dmrt1 mutant fish developed ovary-like testis, and the spermatogenesis was disrupted. The female-related genes foxl2 and cyp19a1a were significantly increased in the gonad of the ZZ dmrt1 mutant. Conversely, the male-related genes Sox9a and Amh were significantly decreased. The dmrt1 deficient ZZ fish grew much faster than ZZ male control. Notably, we obtained an intersex ZW fish with a testis on one side and an ovary on the other side. This fish was chimeric for a dmrt1 mutation in the ovary, and wild-type dmrt1 in the testis. Our data provide the first functional evidence that dmrt1 is a male determining gene in tongue sole

Direct and indirect effects of climate on richness drive the latitudinal diversity gradient in forest trees

Data accessibility statement: Full census data are available upon reasonable request from the ForestGEO data portal, http://ctfs.si.edu/datarequest/ We thank Margie Mayfield, three anonymous reviewers and Jacob Weiner for constructive comments on the manuscript. This study was financially supported by the National Key R&D Program of China (2017YFC0506100), the National Natural Science Foundation of China (31622014 and 31570426), and the Fundamental Research Funds for the Central Universities (17lgzd24) to CC. XW was supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB3103). DS was supported by the Czech Science Foundation (grant no. 16-26369S). Yves Rosseel provided us valuable suggestions on using the lavaan package conducting SEM analyses. Funding and citation information for each forest plot is available in the Supplementary Information Text 1.Peer reviewedPostprin

Articulatory Copy Synthesis Based on the Speech Synthesizer VocalTractLab

Articulatory copy synthesis (ACS), a subarea of speech inversion, refers to the reproduction of natural utterances and involves both the physiological articulatory processes and their corresponding acoustic results. This thesis proposes two novel methods for the ACS of human speech using the articulatory speech synthesizer VocalTractLab (VTL) to address or mitigate the existing problems of speech inversion, such as non-unique mapping, acoustic variation among different speakers, and the time-consuming nature of the process. The first method involved finding appropriate VTL gestural scores for given natural utterances using a genetic algorithm. It consisted of two steps: gestural score initialization and optimization. In the first step, gestural scores were initialized using the given acoustic signals with speech recognition, grapheme-to-phoneme (G2P), and a VTL rule-based method for converting phoneme sequences to gestural scores. In the second step, the initial gestural scores were optimized by a genetic algorithm via an analysis-by-synthesis (ABS) procedure that sought to minimize the cosine distance between the acoustic features of the synthetic and natural utterances. The articulatory parameters were also regularized during the optimization process to restrict them to reasonable values. The second method was based on long short-term memory (LSTM) and convolutional neural networks, which were responsible for capturing the temporal dependence and the spatial structure of the acoustic features, respectively. The neural network regression models were trained, which used acoustic features as inputs and produced articulatory trajectories as outputs. In addition, to cover as much of the articulatory and acoustic space as possible, the training samples were augmented by manipulating the phonation type, speaking effort, and the vocal tract length of the synthetic utterances. Furthermore, two regularization methods were proposed: one based on the smoothness loss of articulatory trajectories and another based on the acoustic loss between original and predicted acoustic features. The best-performing genetic algorithms and convolutional LSTM systems (evaluated in terms of the difference between the estimated and reference VTL articulatory parameters) obtained average correlation coefficients of 0.985 and 0.983 for speaker-dependent utterances, respectively, and their reproduced speech achieved recognition accuracies of 86.25% and 64.69% for speaker-independent utterances of German words, respectively. When applied to German sentence utterances, as well as English and Mandarin Chinese word utterances, the neural network based ACS systems achieved recognition accuracies of 73.88%, 52.92%, and 52.41%, respectively. The results showed that both of these methods not only reproduced the articulatory processes but also reproduced the acoustic signals of reference utterances. Moreover, the regularization methods led to more physiologically plausible articulatory processes and made the estimated articulatory trajectories be more articulatorily preferred by VTL, thus reproducing more natural and intelligible speech. This study also found that the convolutional layers, when used in conjunction with batch normalization layers, automatically learned more distinctive features from log power spectrograms. Furthermore, the neural network based ACS systems trained using German data could be generalized to the utterances of other languages

Articulatory Copy Synthesis Based on the Speech Synthesizer VocalTractLab

Articulatory copy synthesis (ACS), a subarea of speech inversion, refers to the reproduction of natural utterances and involves both the physiological articulatory processes and their corresponding acoustic results. This thesis proposes two novel methods for the ACS of human speech using the articulatory speech synthesizer VocalTractLab (VTL) to address or mitigate the existing problems of speech inversion, such as non-unique mapping, acoustic variation among different speakers, and the time-consuming nature of the process. The first method involved finding appropriate VTL gestural scores for given natural utterances using a genetic algorithm. It consisted of two steps: gestural score initialization and optimization. In the first step, gestural scores were initialized using the given acoustic signals with speech recognition, grapheme-to-phoneme (G2P), and a VTL rule-based method for converting phoneme sequences to gestural scores. In the second step, the initial gestural scores were optimized by a genetic algorithm via an analysis-by-synthesis (ABS) procedure that sought to minimize the cosine distance between the acoustic features of the synthetic and natural utterances. The articulatory parameters were also regularized during the optimization process to restrict them to reasonable values. The second method was based on long short-term memory (LSTM) and convolutional neural networks, which were responsible for capturing the temporal dependence and the spatial structure of the acoustic features, respectively. The neural network regression models were trained, which used acoustic features as inputs and produced articulatory trajectories as outputs. In addition, to cover as much of the articulatory and acoustic space as possible, the training samples were augmented by manipulating the phonation type, speaking effort, and the vocal tract length of the synthetic utterances. Furthermore, two regularization methods were proposed: one based on the smoothness loss of articulatory trajectories and another based on the acoustic loss between original and predicted acoustic features. The best-performing genetic algorithms and convolutional LSTM systems (evaluated in terms of the difference between the estimated and reference VTL articulatory parameters) obtained average correlation coefficients of 0.985 and 0.983 for speaker-dependent utterances, respectively, and their reproduced speech achieved recognition accuracies of 86.25% and 64.69% for speaker-independent utterances of German words, respectively. When applied to German sentence utterances, as well as English and Mandarin Chinese word utterances, the neural network based ACS systems achieved recognition accuracies of 73.88%, 52.92%, and 52.41%, respectively. The results showed that both of these methods not only reproduced the articulatory processes but also reproduced the acoustic signals of reference utterances. Moreover, the regularization methods led to more physiologically plausible articulatory processes and made the estimated articulatory trajectories be more articulatorily preferred by VTL, thus reproducing more natural and intelligible speech. This study also found that the convolutional layers, when used in conjunction with batch normalization layers, automatically learned more distinctive features from log power spectrograms. Furthermore, the neural network based ACS systems trained using German data could be generalized to the utterances of other languages