Towards Racially Unbiased Skin Tone Estimation via Scene Disambiguation

Virtual facial avatars will play an increasingly important role in immersive communication, games and the metaverse, and it is therefore critical that they be inclusive. This requires accurate recovery of the appearance, represented by albedo, regardless of age, sex, or ethnicity. While significant progress has been made on estimating 3D facial geometry, albedo estimation has received less attention. The task is fundamentally ambiguous because the observed color is a function of albedo and lighting, both of which are unknown. We find that current methods are biased towards light skin tones due to (1) strongly biased priors that prefer lighter pigmentation and (2) algorithmic solutions that disregard the light/albedo ambiguity. To address this, we propose a new evaluation dataset (FAIR) and an algorithm (TRUST) to improve albedo estimation and, hence, fairness. Specifically, we create the first facial albedo evaluation benchmark where subjects are balanced in terms of skin color, and measure accuracy using the Individual Typology Angle (ITA) metric. We then address the light/albedo ambiguity by building on a key observation: the image of the full scene -- as opposed to a cropped image of the face -- contains important information about lighting that can be used for disambiguation. TRUST regresses facial albedo by conditioning both on the face region and a global illumination signal obtained from the scene image. Our experimental results show significant improvement compared to state-of-the-art methods on albedo estimation, both in terms of accuracy and fairness. The evaluation benchmark and code will be made available for research purposes at https://trust.is.tue.mpg.de.Comment: Camera-Ready version, accepted at ECCV202

FLARE: Fast Learning of Animatable and Relightable Mesh Avatars

Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. While 3D meshes enable efficient processing and are highly portable, they lack realism in terms of shape and appearance. Neural representations, on the other hand, are realistic but lack compatibility and are slow to train and render. Our key insight is that it is possible to efficiently learn high-fidelity 3D mesh representations via differentiable rendering by exploiting highly-optimized methods from traditional computer graphics and approximating some of the components with neural networks. To that end, we introduce FLARE, a technique that enables the creation of animatable and relightable mesh avatars from a single monocular video. First, we learn a canonical geometry using a mesh representation, enabling efficient differentiable rasterization and straightforward animation via learned blendshapes and linear blend skinning weights. Second, we follow physically-based rendering and factor observed colors into intrinsic albedo, roughness, and a neural representation of the illumination, allowing the learned avatars to be relit in novel scenes. Since our input videos are captured on a single device with a narrow field of view, modeling the surrounding environment light is non-trivial. Based on the split-sum approximation for modeling specular reflections, we address this by approximating the pre-filtered environment map with a multi-layer perceptron (MLP) modulated by the surface roughness, eliminating the need to explicitly model the light. We demonstrate that our mesh-based avatar formulation, combined with learned deformation, material, and lighting MLPs, produces avatars with high-quality geometry and appearance, while also being efficient to train and render compared to existing approaches.Comment: 15 pages, Accepted: ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 202

I M Avatar: Implicit Morphable Head Avatars from Videos

Traditional morphable face models provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photo-realism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose-related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray tracing and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods

Long-term chromospheric activity in southern M dwarfs: Gl 229 A and Gl 752 A

Several late-type stars present activity cycles similar to that of the Sun. However, these cycles have been mostly studied in F to K stars. Due to their small intrinsic brightness, M dwarfs are not usually the targets of long-term observational studies of stellar activity, and their long-term variability is generally not known. In this work, we study the long-term activity of two M dwarf stars: Gl 229 A (M1/2) and Gl 752 A (M2.5). We employ medium resolution echelle spectra obtained at the 2.15 m telescope at the Argentinian observatory CASLEO between the years 2000 and 2010 and photometric observations obtained from the ASAS database. We analyzed Ca \II K line-core fluxes and the mean V magnitude with the Lomb-Scargle periodogram, and we obtain possible activity cycles of $\sim$4 yr and $\sim$7 yr for Gl 229 A and Gl 752 A respectively.Comment: Accepted for publication by Astronomical Journal (AJ

The UV surface habitability of Proxima <i>b</i>: first experiments revealing probable life survival to stellar flares

Abstract We use a new interdisciplinary approach to study the UV surface habitability of Proxima b under quiescent and flaring stellar conditions. We assumed planetary atmospheric compositions based on CO2 and N2 and surface pressures from 100 to 5000 mbar. Our results show that the combination of these atmospheric compositions and pressures provide enough shielding from the most damaging UV wavelengths, expanding the ”UV-protective” planetary atmospheric compositions beyond ozone. Additionally, we show that the UV radiation reaching the surface of Proxima b during quiescent conditions would be negligible from the biological point of view, even without an atmosphere. Given that high UV fluxes could challenge the existence of life, then, we experimentally tested the effect that flares would have on microorganisms in a ”worst case scenario” (no UV-shielding). Our results show the impact that a typical flare and a superflare would have on life: when microorganisms receive very high fluences of UVC, such as those expected to reach the surface of Proxima b after a typical flare or a superflare, a fraction of the population is able to survive. Our study suggests that life could cope with highly UV irradiated environments in exoplanets under conditions that cannot be found on Earth

Estimation and inference under economic restrictions

Estimation of economic relationships often requires imposition of constraints such as positivity or monotonicity on each observation. Methods to impose such constraints, however, vary depending upon the estimation technique employed. We describe a general methodology to impose (observation-specific) constraints for the class of linear regression estimators using a method known as constraint weighted bootstrapping. While this method has received attention in the nonparametric regression literature, we show how it can be applied for both parametric and nonparametric estimators. A benefit of this method is that imposing numerous constraints simultaneously can be performed seamlessly. We apply this method to Norwegian dairy farm data to estimate both unconstrained and constrained parametric and nonparametric models