3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models

Acquiring the virtual equivalent of exhibits, such as sculptures, in virtual reality (VR) museums, can be labour-intensive and sometimes infeasible. Deep learning based 3D reconstruction approaches allow us to recover 3D shapes from 2D observations, among which single-view-based approaches can reduce the need for human intervention and specialised equipment in acquiring 3D sculptures for VR museums. However, there exist two challenges when attempting to use the well-researched human reconstruction methods: limited data availability and domain shift. Considering sculptures are usually related to humans, we propose our unsupervised 3D domain adaptation method for adapting a single-view 3D implicit reconstruction model from the source (real-world humans) to the target (sculptures) domain. We have compared the generated shapes with other methods and conducted ablation studies as well as a user study to demonstrate the effectiveness of our adaptation method. We also deploy our results in a VR application

On the Design Fundamentals of Diffusion Models: A Survey

Diffusion models are generative models, which gradually add and remove noise to learn the underlying distribution of training data for data generation. The components of diffusion models have gained significant attention with many design choices proposed. Existing reviews have primarily focused on higher-level solutions, thereby covering less on the design fundamentals of components. This study seeks to address this gap by providing a comprehensive and coherent review on component-wise design choices in diffusion models. Specifically, we organize this review according to their three key components, namely the forward process, the reverse process, and the sampling procedure. This allows us to provide a fine-grained perspective of diffusion models, benefiting future studies in the analysis of individual components, the applicability of design choices, and the implementation of diffusion models

Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models

Generating realistic motions for digital humans is a core but challenging part of computer animations and games, as human motions are both diverse in content and rich in styles. While the latest deep learning approaches have made significant advancements in this domain, they mostly consider motion synthesis and style manipulation as two separate problems. This is mainly due to the challenge of learning both motion contents that account for the inter-class behaviour and styles that account for the intra-class behaviour effectively in a common representation. To tackle this challenge, we propose a denoising diffusion probabilistic model solution for styled motion synthesis. As diffusion models have a high capacity brought by the injection of stochasticity, we can represent both inter-class motion content and intra-class style behaviour in the same latent. This results in an integrated, end-to-end trained pipeline that facilitates the generation of optimal motion and exploration of content-style coupled latent space. To achieve high-quality results, we design a multi-task architecture of diffusion model that strategically generates aspects of human motions for local guidance. We also design adversarial and physical regulations for global guidance. We demonstrate superior performance with quantitative and qualitative results and validate the effectiveness of our multi-task architecture

Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment

Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency, completeness, and prosody, previous efforts typically only model one aspect (e.g., accuracy) at one granularity (e.g., at the phoneme-level). In this work, we explore modeling multi-aspect pronunciation assessment at multiple granularities. Specifically, we train a Goodness Of Pronunciation feature-based Transformer (GOPT) with multi-task learning. Experiments show that GOPT achieves the best results on speechocean762 with a public automatic speech recognition (ASR) acoustic model trained on Librispeech.Comment: Accepted at ICASSP 2022. Code at https://github.com/YuanGongND/gopt Interactive Colab demo at https://colab.research.google.com/github/YuanGongND/gopt/blob/master/colab/GOPT_GPU.ipynb . ICASSP 202

Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient

Recently, methods for skeleton-based human activity recognition have been shown to be vulnerable to adversarial attacks. However, these attack methods require either the full knowledge of the victim (i.e. white-box attacks), access to training data (i.e. transfer-based attacks) or frequent model queries (i.e. black-box attacks). All their requirements are highly restrictive, raising the question of how detrimental the vulnerability is. In this paper, we show that the vulnerability indeed exists. To this end, we consider a new attack task: the attacker has no access to the victim model or the training data or labels, where we coin the term hard no-box attack. Specifically, we first learn a motion manifold where we define an adversarial loss to compute a new gradient for the attack, named skeleton-motion-informed (SMI) gradient. Our gradient contains information of the motion dynamics, which is different from existing gradient-based attack methods that compute the loss gradient assuming each dimension in the data is independent. The SMI gradient can augment many gradient-based attack methods, leading to a new family of no-box attack methods. Extensive evaluation and comparison show that our method imposes a real threat to existing classifiers. They also show that the SMI gradient improves the transferability and imperceptibility of adversarial samples in both no-box and transfer-based black-box settings.Comment: Camera-ready version for ICCV 202

Denoising Diffusion Probabilistic Models for Styled Walking Synthesis

Generating realistic motions for digital humans is time-consuming for many graphics applications. Data-driven motion synthesis approaches have seen solid progress in recent years through deep generative models. These results offer high-quality motions but typically suffer in motion style diversity. For the first time, we propose a framework using the denoising diffusion probabilistic model (DDPM) to synthesize styled human motions, integrating two tasks into one pipeline with increased style diversity compared with traditional motion synthesis methods. Experimental results show that our system can generate high-quality and diverse walking motions

Epidural combined optical and electrical stimulation induces high-specificity activation of target muscles in spinal cord injured rats

IntroductionEpidural electrical stimulation (EES) has been shown to improve motor dysfunction after spinal cord injury (SCI) by activating residual locomotor neural networks. However, the stimulation current often spreads excessively, leading to activation of non-target muscles and reducing the accuracy of stimulation regulation.ObjectivesNear-infrared nerve stimulation (nINS) was combined with EES to explore its regulatory effect on lower limb muscle activity in spinal-cord-transected rats.MethodsIn this study, stimulation electrodes were implanted into the rats’ L3–L6 spinal cord segment with T8 cord transected. Firstly, a series of EES parameters (0.2–0.6 mA and 20–60 Hz) were tested to determine those that specifically regulate the tibialis anterior (TA) and medial gastrocnemius (MG). Subsequently, to determine the effect of combined optical and electrical stimulation, near-infrared laser with a wavelength of 808 nm was used to irradiate the L3–L6 spinal cord segment while EES was performed. The amplitude of electromyography (EMG), the specific activation intensity of the target muscle, and the minimum stimulus current intensity to induce joint movement (motor threshold) under a series of optical stimulation parameters (power: 0.0–2.0 W; pulse width: 0–10 ms) were investigated and analyzed.ResultsEES stimulation with 40 Hz at the L3 and L6 spinal cord segments specifically activated TA and MG, respectively. High stimulation intensity (>2 × motor threshold) activated non-target muscles, while low stimulation frequency (<20 Hz) produced intermittent contraction. Compared to electrical stimulation alone (0.577 ± 0.081 mV), the combined stimulation strategy could induce stronger EMG amplitude of MG (1.426 ± 0.365 mV) after spinal cord injury (p < 0.01). The combined application of nINS effectively decreased the EES-induced motor threshold of MG (from 0.237 ± 0.001 mA to 0.166 ± 0.028 mA, p < 0.001). Additionally, the pulse width (PW) of nINS had a slight impact on the regulation of muscle activity. The EMG amplitude of MG only increased by ~70% (from 3.978 ± 0.240 mV to 6.753 ± 0.263 mV) when the PW increased by 10-fold (from 1 to 10 ms).ConclusionThe study demonstrates the feasibility of epidural combined electrical and optical stimulation for highly specific regulation of muscle activity after SCI, and provides a new strategy for improving motor dysfunction caused by SCI

Metabolomics reveals the response of hydroprimed maize to mitigate the impact of soil salinization

Soil salinization is a major environmental stressor hindering global crop production. Hydropriming has emerged as a promising approach to reduce salt stress and enhance crop yields on salinized land. However, a better mechanisitic understanding is required to improve salt stress tolerance. We used a biochemical and metabolomics approach to study the effect of salt stress of hydroprimed maize to identify the types and variation of differentially accumulated metabolites. Here we show that hydropriming significantly increased catalase (CAT) activity, soluble sugar and proline content, decreased superoxide dismutase (SOD) activity and peroxide (H2O2) content. Conversely, hydropriming had no significant effect on POD activity, soluble protein and MDA content under salt stress. The Metabolite analysis indicated that salt stress significantly increased the content of 1278 metabolites and decreased the content of 1044 metabolites. Ethisterone (progesterone) was the most important metabolite produced in the roots of unprimed samples in response to salt s tress. Pathway enrichment analysis indicated that flavone and flavonol biosynthesis, which relate to scavenging reactive oxygen species (ROS), was the most significant metabolic pathway related to salt stress. Hydropriming significantly increased the content of 873 metabolites and significantly decreased the content of 1313 metabolites. 5-Methyltetrahydrofolate, a methyl donor for methionine, was the most important metabolite produced in the roots of hydroprimed samples in response to salt stress. Plant growth regulator, such as melatonin, gibberellin A8, estrone, abscisic acid and brassinolide involved in both treatment. Our results not only verify the roles of key metabolites in resisting salt stress, but also further evidence that flavone and flavonol biosynthesis and plant growth regulator relate to salt tolerance

Metabolomics reveals the response of hydroprimed maize to mitigate the impact of soil salinization

Soil salinization is a major environmental stressor hindering global crop production. Hydropriming has emerged as a promising approach to reduce salt stress and enhance crop yields on salinized land. However, a better mechanisitic understanding is required to improve salt stress tolerance. We used a biochemical and metabolomics approach to study the effect of salt stress of hydroprimed maize to identify the types and variation of differentially accumulated metabolites. Here we show that hydropriming significantly increased catalase (CAT) activity, soluble sugar and proline content, decreased superoxide dismutase (SOD) activity and peroxide (H2O2) content. Conversely, hydropriming had no significant effect on POD activity, soluble protein and MDA content under salt stress. The Metabolite analysis indicated that salt stress significantly increased the content of 1278 metabolites and decreased the content of 1044 metabolites. Ethisterone (progesterone) was the most important metabolite produced in the roots of unprimed samples in response to salt s tress. Pathway enrichment analysis indicated that flavone and flavonol biosynthesis, which relate to scavenging reactive oxygen species (ROS), was the most significant metabolic pathway related to salt stress. Hydropriming significantly increased the content of 873 metabolites and significantly decreased the content of 1313 metabolites. 5-Methyltetrahydrofolate, a methyl donor for methionine, was the most important metabolite produced in the roots of hydroprimed samples in response to salt stress. Plant growth regulator, such as melatonin, gibberellin A8, estrone, abscisic acid and brassinolide involved in both treatment. Our results not only verify the roles of key metabolites in resisting salt stress, but also further evidence that flavone and flavonol biosynthesis and plant growth regulator relate to salt tolerance