A Unified Compression Framework for Efficient Speech-Driven Talking-Face Generation

Virtual humans have gained considerable attention in numerous industries, e.g., entertainment and e-commerce. As a core technology, synthesizing photorealistic face frames from target speech and facial identity has been actively studied with generative adversarial networks. Despite remarkable results of modern talking-face generation models, they often entail high computational burdens, which limit their efficient deployment. This study aims to develop a lightweight model for speech-driven talking-face synthesis. We build a compact generator by removing the residual blocks and reducing the channel width from Wav2Lip, a popular talking-face generator. We also present a knowledge distillation scheme to stably yet effectively train the small-capacity generator without adversarial learning. We reduce the number of parameters and MACs by 28$\times$ while retaining the performance of the original model. Moreover, to alleviate a severe performance drop when converting the whole generator to INT8 precision, we adopt a selective quantization method that uses FP16 for the quantization-sensitive layers and INT8 for the other layers. Using this mixed precision, we achieve up to a 19$\times$ speedup on edge GPUs without noticeably compromising the generation quality.Comment: MLSys Workshop on On-Device Intelligence, 2023; Demo: https://huggingface.co/spaces/nota-ai/compressed_wav2li

Atomic-Scale Scanning of Domain Network in the Ferroelectric HfO2 Thin Film

Ferroelectric HfO2-based thin films have attracted much interest in the utilization of ferroelectricity at the nanoscale for next-generation electronic devices. However, the structural origin and stabilization mechanism of the ferroelectric phase are not understood because the film is typically nanocrystalline with active yet stochastic ferroelectric domains. Here, electron microscopy is used to map the in-plane domain network structures of epitaxially grown ferroelectric Y:HfO2 films in atomic resolution. The ferroelectricity is confirmed in free-standing Y:HfO2 films, allowing for investigating the structural origin for their ferroelectricity by 4D-STEM, high-resolution STEM, and iDPC-STEM. At the grain boundaries of <111>-oriented Pca21 orthorhombic grains, a high-symmetry mixed-(R3m, Pnm21) phase is induced, exhibiting enhanced polarization due to in-plane compressive strain. Nanoscale Pca21 orthorhombic grains and their grain boundaries with mixed-(R3m, Pnm21) phases of higher symmetry cooperatively determine the ferroelectricity of the Y:HfO2 film. It is also found that such ferroelectric domain networks emerge when the film thickness is beyond a finite value. Furthermore, in-plane mapping of oxygen positions overlaid on ferroelectric domains discloses that polarization is suppressed at vertical domain walls, while it is active when domains are aligned horizontally with subangstrom domain walls. In addition, randomly distributed 180° charged domain walls are confined by spacer layers

Machine Learning-Based Anomaly Detection on Seawater Temperature Data with Oversampling

This study deals with a method for anomaly detection in seawater temperature data using machine learning methods with oversampling techniques. Data were acquired from 2017 to 2023 using a Conductivity–Temperature–Depth (CTD) system in the Pacific Ocean, Indian Ocean, and Sea of Korea. The seawater temperature data consist of 1414 profiles including 1218 normal and 196 abnormal profiles. This dataset has an imbalance problem in which the amount of abnormal data is insufficient compared to that of normal data. Therefore, we generated abnormal data with oversampling techniques using duplication, uniform random variable, Synthetic Minority Oversampling Technique (SMOTE), and autoencoder (AE) techniques for the balance of data class, and trained Interquartile Range (IQR)-based, one-class support vector machine (OCSVM), and Multi-Layer Perceptron (MLP) models with a balanced dataset for anomaly detection. In the experimental results, the F1 score of the MLP showed the best performance at 0.882 in the combination of learning data, consisting of 30% of the minor data generated by SMOTE. This result is a 71.4%-point improvement over the F1 score of the IQR-based model, which is the baseline of this study, and is 1.3%-point better than the best-performing model among the models without oversampling data

Fabrication of Micro-Patterned Chip with Controlled Thickness for High-Throughput Cryogenic Electron Microscopy

© 2022 JoVE Journal of Visualized Experiments.A major limitation for the efficient and high-throughput structure analysis of biomolecules using cryogenic electron microscopy (cryo-EM) is the difficulty of preparing cryo-EM samples with controlled ice thickness at the nanoscale. The silicon (Si)-based chip, which has a regular array of micro-holes with graphene oxide (GO) window patterned on a thickness-controlled silicon nitride (SixNy) film, has been developed by applying microelectromechanical system (MEMS) techniques. UV photolithography, chemical vapor deposition, wet and dry etching of the thin film, and drop-casting of 2D nanosheet materials were used for mass-production of the micro-patterned chips with GO windows. The depth of the micro-holes is regulated to control the ice thickness on-demand, depending on the size of the specimen for cryo-EM analysis. The favorable affinity of GO toward biomolecules concentrates the biomolecules of interest within the micro-hole during cryo-EM sample preparation. The micro-patterned chip with GO windows enables high-throughput cryo-EM imaging of various biological molecules, as well as inorganic nanomaterials.11Nsciescopu