High-Fidelity Eye Animatable Neural Radiance Fields for Human Face

Face rendering using neural radiance fields (NeRF) is a rapidly developing research area in computer vision. While recent methods primarily focus on controlling facial attributes such as identity and expression, they often overlook the crucial aspect of modeling eyeball rotation, which holds importance for various downstream tasks. In this paper, we aim to learn a face NeRF model that is sensitive to eye movements from multi-view images. We address two key challenges in eye-aware face NeRF learning: how to effectively capture eyeball rotation for training and how to construct a manifold for representing eyeball rotation. To accomplish this, we first fit FLAME, a well-established parametric face model, to the multi-view images considering multi-view consistency. Subsequently, we introduce a new Dynamic Eye-aware NeRF (DeNeRF). DeNeRF transforms 3D points from different views into a canonical space to learn a unified face NeRF model. We design an eye deformation field for the transformation, including rigid transformation, e.g., eyeball rotation, and non-rigid transformation. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our model is capable of generating high-fidelity images with accurate eyeball rotation and non-rigid periocular deformation, even under novel viewing angles. Furthermore, we show that utilizing the rendered images can effectively enhance gaze estimation performance.Comment: Under revie

Category-Level 6D Object Pose Estimation with Flexible Vector-Based Rotation Representation

In this paper, we propose a novel 3D graph convolution based pipeline for category-level 6D pose and size estimation from monocular RGB-D images. The proposed method leverages an efficient 3D data augmentation and a novel vector-based decoupled rotation representation. Specifically, we first design an orientation-aware autoencoder with 3D graph convolution for latent feature learning. The learned latent feature is insensitive to point shift and size thanks to the shift and scale-invariance properties of the 3D graph convolution. Then, to efficiently decode the rotation information from the latent feature, we design a novel flexible vector-based decomposable rotation representation that employs two decoders to complementarily access the rotation information. The proposed rotation representation has two major advantages: 1) decoupled characteristic that makes the rotation estimation easier; 2) flexible length and rotated angle of the vectors allow us to find a more suitable vector representation for specific pose estimation task. Finally, we propose a 3D deformation mechanism to increase the generalization ability of the pipeline. Extensive experiments show that the proposed pipeline achieves state-of-the-art performance on category-level tasks. Further, the experiments demonstrate that the proposed rotation representation is more suitable for the pose estimation tasks than other rotation representations.Comment: revised from CVPR2021 paper FS-NET. arXiv admin note: substantial text overlap with arXiv:2103.0705

CNx-modified Fe3O4 as Pt nanoparticle support for the oxygen reduction reaction

A novel electrocatalyst support material, nitrogendoped carbon (CNx)-modified Fe3O4 (Fe3O4-CNx), was synthesized through carbonizing a polypyrrole-Fe3O4 hybridized precursor. Subsequently, Fe3O4-CNx-supported Pt (Pt/Fe3O4-CNx) nanocomposites were prepared by reducing Pt precursor in ethylene glycol solution and evaluated for the oxygen reduction reaction (ORR). The Pt/Fe3O4-CNx catalysts were characterized by X-ray diffraction, Raman spectra, X-ray photoelectron spectroscopy, scanning electron microscopy, and transmission electron microscopy. The electrocatalytic activity and stability of the as-prepared electrocatalysts toward ORR were studied by cyclic voltammetry and steady-state polarization measurements. The results showed that Pt/ Fe3O4-CNx catalysts exhibited superior catalytic performance for ORR to the conventional Pt/C and Pt/C-CNx catalysts.Web of Scienc

Targeted synthesis of an electroactive organic framework

A new strategy for targeted design and synthesis of an electroactive microporous organic molecular sieve (JUC-Z2) is described. Experiment demonstrated that such a targeted synthesis approach to achieve phenyl-phenyl coupling was a controllable process and predominately generated two-dimensional polymer sheets, significantly different from the traditional chemical or electrochemical oxidation methods to prepare conducting polymers. Successive self-assembly leads to a lamellar organic framework comprised of stacked polymer sheets with an hcb topology. JUC-Z2 was found to have a well-defined uniform micropore distribution (similar to 1.2 nm), a large surface area (BET = 2081 m(2) g(-1)) and high physicochemical stability (> 440 degrees C). After doping with I(2), JUC-Z2 exhibits typical p-type semiconductive properties. As the first example of an electroactive organic framework, JUC-Z2 possesses a unique ability of electrochemical ion recognition, arising from the synergistic function of the uniform micropores and the N-atom redox site.State Basic Research Project[2011CB808703]; NSFC[91022030, 20771041, 20773101, 20833005]; "111'' project[B07016]; Ministry of Science and Technology[2006DFA41190]; Jilin Science and Technology Department[20106021

The Ninth Visual Object Tracking VOT2021 Challenge Results

acceptedVersionPeer reviewe

A deep image classification model based on prior feature knowledge embedding and application in medical diagnosis

Aiming at the problem of image classification with insignificant morphological structural features, strong target correlation, and low signal-to-noise ratio, combined with prior feature knowledge embedding, a deep learning method based on ResNet and Radial Basis Probabilistic Neural Network (RBPNN) is proposed model. Taking ResNet50 as a visual modeling network, it uses feature pyramid and self-attention mechanism to extract appearance and semantic features of images at multiple scales, and associate and enhance local and global features. Taking into account the diversity of category features, channel cosine similarity attention and dynamic C-means clustering algorithms are used to select representative sample features in different category of sample subsets to implicitly express prior category feature knowledge, and use them as the kernel centers of radial basis probability neurons (RBPN) to realize the embedding of diverse prior feature knowledge. In the RBPNN pattern aggregation layer, the outputs of RBPN are selectively summed according to the category of the kernel center, that is, the subcategory features are combined into category features, and finally the image classification is implemented based on Softmax. The functional module of the proposed method is designed specifically for image characteristics, which can highlight the significance of local and structural features of the image, form a non-convex decision-making area, and reduce the requirements for the completeness of the sample set. Applying the proposed method to medical image classification, experiments were conducted based on the brain tumor MRI image classification public dataset and the actual cardiac ultrasound image dataset, and the accuracy rate reached 85.82% and 83.92% respectively. Compared with the three mainstream image classification models, the performance indicators of this method have been significantly improved