Learning-Based Facial Attribute Estimation and Manipulation

Facial attribute analysis plays a crucial role in various fields such as surveillance, entertainment, healthcare, and human-computer interaction. The advert of deep neural networks has sparked a growing interest in learning-based facial attribute analysis. This dissertation focuses on learning-based facial attribute analysis, encompassing facial attribute estimation and manipulation tasks. We focused on solving challenges including addressing data scarcity in supervised facial attribute estimation, handling facial attribute manipulation in high-resolution images, efficiently disentangling targeted attributes from others, and training a facial attribute manipulator with datasets from a small number of subjects.The dissertation is structured around three key facial attribute categories: head orientations, gaze directions, and facial action units. The first part delves into improving appearance-based gaze estimation by considering person-dependent anatomical variations and accounting for ocular countering-rolling (OCR) responses, resulting in a more efficient and accurate method. The second part introduces ReDirTrans, a portable network designed for gaze redirection in high-resolution face images. By focusing on latent-to-latent translation, ReDirTrans enables precise gaze and head pose redirection while preserving other attributes, expanding its applicability beyond limited ranges of faces. The final part presents AUEditNet, a model for manipulating facial action unit intensities. This addresses challenges posed by data scarcity by effectively disentangling attributes and identity within a limited subject pool. AUEditNet demonstrates superior accuracy in editing AU intensities across 12 AUs, showcasing its potential for fine-grained facial attribute manipulation.Overall, this dissertation contributes novel methodologies in learning-based facial attribute analysis, paving the way for enhanced performance and versatility across various real-world applications

ReDirTrans: Latent-to-Latent Translation for Gaze and Head Redirection

Learning-based gaze estimation methods require large amounts of training data with accurate gaze annotations. Facing such demanding requirements of gaze data collection and annotation, several image synthesis methods were proposed, which successfully redirected gaze directions precisely given the assigned conditions. However, these methods focused on changing gaze directions of the images that only include eyes or restricted ranges of faces with low resolution (less than $128\times128$) to largely reduce interference from other attributes such as hairs, which limits application scenarios. To cope with this limitation, we proposed a portable network, called ReDirTrans, achieving latent-to-latent translation for redirecting gaze directions and head orientations in an interpretable manner. ReDirTrans projects input latent vectors into aimed-attribute embeddings only and redirects these embeddings with assigned pitch and yaw values. Then both the initial and edited embeddings are projected back (deprojected) to the initial latent space as residuals to modify the input latent vectors by subtraction and addition, representing old status removal and new status addition. The projection of aimed attributes only and subtraction-addition operations for status replacement essentially mitigate impacts on other attributes and the distribution of latent vectors. Thus, by combining ReDirTrans with a pretrained fixed e4e-StyleGAN pair, we created ReDirTrans-GAN, which enables accurately redirecting gaze in full-face images with $1024\times1024$ resolution while preserving other attributes such as identity, expression, and hairstyle. Furthermore, we presented improvements for the downstream learning-based gaze estimation task, using redirected samples as dataset augmentation

Efficient IoT Inference via Context-Awareness

While existing strategies to execute deep learning-based classification on low-power platforms assume the models are trained on all classes of interest, this paper posits that adopting context-awareness i.e. narrowing down a classification task to the current deployment context consisting of only recent inference queries can substantially enhance performance in resource-constrained environments. We propose a new paradigm, CACTUS, for scalable and efficient context-aware classification where a micro-classifier recognizes a small set of classes relevant to the current context and, when context change happens (e.g., a new class comes into the scene), rapidly switches to another suitable micro-classifier. CACTUS features several innovations, including optimizing the training cost of context-aware classifiers, enabling on-the-fly context-aware switching between classifiers, and balancing context switching costs and performance gains via simple yet effective switching policies. We show that CACTUS achieves significant benefits in accuracy, latency, and compute budget across a range of datasets and IoT platforms.Comment: 12 pages, 8 figure

AUEditNet: Dual-Branch Facial Action Unit Intensity Manipulation with Implicit Disentanglement

Facial action unit (AU) intensity plays a pivotal role in quantifying fine-grained expression behaviors, which is an effective condition for facial expression manipulation. However, publicly available datasets containing intensity annotations for multiple AUs remain severely limited, often featuring a restricted number of subjects. This limitation places challenges to the AU intensity manipulation in images due to disentanglement issues, leading researchers to resort to other large datasets with pretrained AU intensity estimators for pseudo labels. In addressing this constraint and fully leveraging manual annotations of AU intensities for precise manipulation, we introduce AUEditNet. Our proposed model achieves impressive intensity manipulation across 12 AUs, trained effectively with only 18 subjects. Utilizing a dual-branch architecture, our approach achieves comprehensive disentanglement of facial attributes and identity without necessitating additional loss functions or implementing with large batch sizes. This approach offers a potential solution to achieve desired facial attribute editing despite the dataset's limited subject count. Our experiments demonstrate AUEditNet's superior accuracy in editing AU intensities, affirming its capability in disentangling facial attributes and identity within a limited subject pool. AUEditNet allows conditioning by either intensity values or target images, eliminating the need for constructing AU combinations for specific facial expression synthesis. Moreover, AU intensity estimation, as a downstream task, validates the consistency between real and edited images, confirming the effectiveness of our proposed AU intensity manipulation method

Dietary Supplementation of Astaxanthin Improved the Growth Performance, Antioxidant Ability and Immune Response of Juvenile Largemouth Bass (Micropterus salmoides) Fed High-Fat Diet

High-fat diet (HFD) usually induces oxidative stress and astaxanthin is regarded as an excellent anti-oxidant. An 8-week feeding trial was conducted to investigate the effects of dietary astaxanthin supplementation on growth performance, lipid metabolism, antioxidant ability, and immune response of juvenile largemouth bass (Micropterus salmoides) fed HFD. Four diets were formulated: the control diet (10.87% lipid, C), high-fat diet (18.08% lipid, HF), and HF diet supplemented with 75 and 150 mg kg−1 astaxanthin (HFA1 and HFA2, respectively). Dietary supplementation of astaxanthin improved the growth of fish fed HFD, also decreased hepatosomatic index and intraperitoneal fat ratio of fish fed HFD, while having no effect on body fat. Malondialdehyde content and superoxide dismutase activity were increased in fish fed HFD, astaxanthin supplementation in HFD decreased the oxidative stress of fish. The supplementation of astaxanthin in HFD also reduced the mRNA levels of Caspase 3, Caspase 9, BAD, and IL15. These results suggested that dietary astaxanthin supplementation in HFD improved the growth performance, antioxidant ability and immune response of largemouth bass.publishedVersio