Lossy Image Compression with Conditional Diffusion Models

Denoising diffusion models have recently marked a milestone in high-quality image generation. One may thus wonder if they are suitable for neural image compression. This paper outlines an end-to-end optimized image compression framework based on a conditional diffusion model, drawing on the transform-coding paradigm. Besides the latent variables inherent to the diffusion process, this paper introduces an additional discrete "content" latent variable to condition the denoising process on. This variable is equipped with a hierarchical prior for entropy coding. The remaining "texture" latent variables characterizing the diffusion process are synthesized (either stochastically or deterministically) at decoding time. We furthermore show that the performance can be tuned toward perceptual metrics of interest. Our extensive experiments involving five datasets and 16 image perceptual quality assessment metrics show that our approach not only compares favorably in terms of rate and perceptual distortion tradeoffs but also shows robust performance under all metrics while other baselines show less consistent behavior.Comment: Accepted at the ECCV 2022 Workshop on Uncertainty Quantification for Computer Visio

Neural Volumetric Memory for Visual Locomotion Control

Legged robots have the potential to expand the reach of autonomy beyond paved roads. In this work, we consider the difficult problem of locomotion on challenging terrains using a single forward-facing depth camera. Due to the partial observability of the problem, the robot has to rely on past observations to infer the terrain currently beneath it. To solve this problem, we follow the paradigm in computer vision that explicitly models the 3D geometry of the scene and propose Neural Volumetric Memory (NVM), a geometric memory architecture that explicitly accounts for the SE(3) equivariance of the 3D world. NVM aggregates feature volumes from multiple camera views by first bringing them back to the ego-centric frame of the robot. We test the learned visual-locomotion policy on a physical robot and show that our approach, which explicitly introduces geometric priors during training, offers superior performance than more na\"ive methods. We also include ablation studies and show that the representations stored in the neural volumetric memory capture sufficient geometric information to reconstruct the scene. Our project page with videos is https://rchalyang.github.io/NVM .Comment: CVPR 2023 Highlight. Our project page with videos is https://rchalyang.github.io/NV

An analysis of the new developments and dilemmas of Chinese comic adaptations from the perspective of cultural resonance in the new era ——The example of “White Snake” series of Light Chaser Animation

The film series “White Snake” of Light Chaser Animation is an attempt to internationalize Chinese comics in terms of cultural elements, characterization and core spirit.However, in the midst of continuous innovation, Chinese comics also face new dilemmas, and how to get out of this dilemma is the key to further development of Chinese comics in current era. By using both documentary and case study methods, I review the literature on the “White Snake” series to understand how the film has been received by audiences in China and the overseas market. What’s more, to gain a comprehensive understanding of the current state of adaptation of Chinese comics base on the various aspects of the adaptation and innovation of “White Snake” in the literature of others.Then, in the part of the study on new dilemmas, I use the method of comparative thinking to compare “White Snake” with the success of foreign films called “Kung Fu Panda” and “Dragon Ball” respectively, and come to the conclusion that national comics already have good innovation in terms of cultural elements and characterization, but in order to further develop Chinese comics, we need to combine the core spirit with the current trend of the times, and through the localization of foreign culture to arouse the cultural resonance of the new era

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers

We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimensional depth sensor inputs. While learning-based locomotion has made great advances using RL, most methods still rely on domain randomization for training blind agents that generalize to challenging terrains. Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL method for quadrupedal locomotion that leverages a Transformer-based model for fusing proprioceptive states and visual observations. We evaluate our method in challenging simulated environments with different obstacles and uneven terrain. We show that our method obtains significant improvements over policies with only proprioceptive state inputs, and that Transformer-based models further improve generalization across environments. Our project page with videos is at https://RchalYang.github.io/LocoTransformer .Comment: Our project page with videos is at https://RchalYang.github.io/LocoTransforme

Chronic lateral ankle instability using anterior tibiofibular ligament distal fascicle transfer augmentation repair: an anatomical, biomechanical, and histological study

Background: The transfer of the anterior tibiofibular ligament distal fascicle (ATiFL-DF) for the augmentation repair of the anterior talofibular ligament (ATFL) shows potential as a surgical technique. However, evidences on the benefits and disadvantages of this method in relation to ankle joint function are lacking.Purpose: This study aimed to provide comprehensive experimental data to validate the feasibility of ATiFL-DF transfer augmentation repair of the ATFL.Methods: This study included 50 embalmed ankle specimens to measure various morphological features, such as length, width, thickness, and angle, for evaluating similarities between the ATiFL-DF and ATFL. Furthermore, 24 fresh-frozen ankle specimens were examined for biomechanical testing of the ATiFL-DF transfer augmented repair of the ATFL. Finally, 12 pairs of ATiFL-DF and ATFL tissues from fresh-frozen ankle specimens were treated with gold chloride staining to analyze mechanoreceptor densities.Results: Anatomical studies found that the lengths and thicknesses of the ATFL and ATiFL-DF are similar. Biomechanical outcomes showed that performing ATiFL-DF transfer for ATFL repair can improve the stability of the talus and ankle joints. This is evident from the results of the anterior drawer, axial load, and ultimate failure load tests. However, performing ATiFL-DF transfer may compromise the stability of the distal tibiofibular joint, based on the Cotton and axial load tests at an external rotation of 5°. Analysis of the histological findings revealed that mechanoreceptor densities for four types of mechanoreceptors were comparable between the ATiFL-DF and ATFL groups.Conclusion: ATiFL-DF transfer is a viable method for augmenting ATFL repair. This technique helps to improve the stability of the talus and ankle joints while compensating for proprioception loss. Although ATiFL-DF transfer augmented repair of the ATFL may negatively affect the stability of the distal tibiofibular joint, this procedure can enhance the stability of the talus and ankle joints