3,536 research outputs found
Smart information desk system with voice assistant for universities
This article aims to develop a smart information desk system through a smart mirror for universities. It is a mirror with extra capabilities of displaying answers for academic inquiries such as asking about the lecturers’ office numbers and hours, exams dates and times on the mirror surface. In addition, the voice recognition feature was used to answer spoken inquiries in audio responds to serve all types of users including disabled ones. Furthermore, the system showed general information such as date, weather, time and the university map. The smart mirror was connected to an outdoor camera to monitor the traffics at the university entrance gate. The system was implemented on a Raspberry Pi 4 model B connected to a two-way mirror and an infrared (IR) touch frame. The results of this study helped to overcome the problem of the information desk absence in the university. Therefore, it helped users to save their time and effort in making requests for important academic information
MasqueArray: Automatic makeup selector/applicator
Discusses the design of a computer which selects and applies makeup
Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality
Social presence, the feeling of being there with a real person, will fuel the
next generation of communication systems driven by digital humans in virtual
reality (VR). The best 3D video-realistic VR avatars that minimize the uncanny
effect rely on person-specific (PS) models. However, these PS models are
time-consuming to build and are typically trained with limited data
variability, which results in poor generalization and robustness. Major sources
of variability that affects the accuracy of facial expression transfer
algorithms include using different VR headsets (e.g., camera configuration,
slop of the headset), facial appearance changes over time (e.g., beard,
make-up), and environmental factors (e.g., lighting, backgrounds). This is a
major drawback for the scalability of these models in VR. This paper makes
progress in overcoming these limitations by proposing an end-to-end
multi-identity architecture (MIA) trained with specialized augmentation
strategies. MIA drives the shape component of the avatar from three cameras in
the VR headset (two eyes, one mouth), in untrained subjects, using minimal
personalized information (i.e., neutral 3D mesh shape). Similarly, if the PS
texture decoder is available, MIA is able to drive the full avatar
(shape+texture) robustly outperforming PS models in challenging scenarios. Our
key contribution to improve robustness and generalization, is that our method
implicitly decouples, in an unsupervised manner, the facial expression from
nuisance factors (e.g., headset, environment, facial appearance). We
demonstrate the superior performance and robustness of the proposed method
versus state-of-the-art PS approaches in a variety of experiments
DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery
Despite the enhanced realism and immersion provided by VR headsets, users
frequently encounter adverse effects such as digital eye strain (DES), dry eye,
and potential long-term visual impairment due to excessive eye stimulation from
VR displays and pressure from the mask. Recent VR headsets are increasingly
equipped with eye-oriented monocular cameras to segment ocular feature maps.
Yet, to compute the incident light stimulus and observe periocular condition
alterations, it is imperative to transform these relative measurements into
metric dimensions. To bridge this gap, we propose a lightweight framework
derived from the U-Net 3+ deep learning backbone that we re-optimised, to
estimate measurable periocular depth maps. Compatible with any VR headset
equipped with an eye-oriented monocular camera, our method reconstructs
three-dimensional periocular regions, providing a metric basis for related
light stimulus calculation protocols and medical guidelines. Navigating the
complexities of data collection, we introduce a Dynamic Periocular Data
Generation (DPDG) environment based on UE MetaHuman, which synthesises
thousands of training images from a small quantity of human facial scan data.
Evaluated on a sample of 36 participants, our method exhibited notable efficacy
in the periocular global precision evaluation experiment, and the pupil
diameter measurement
3D Human Face Reconstruction and 2D Appearance Synthesis
3D human face reconstruction has been an extensive research for decades due to its wide applications, such as animation, recognition and 3D-driven appearance synthesis. Although commodity depth sensors are widely available in recent years, image based face reconstruction are significantly valuable as images are much easier to access and store.
In this dissertation, we first propose three image-based face reconstruction approaches according to different assumption of inputs.
In the first approach, face geometry is extracted from multiple key frames of a video sequence with different head poses. The camera should be calibrated under this assumption.
As the first approach is limited to videos, we propose the second approach then focus on single image. This approach also improves the geometry by adding fine grains using shading cue. We proposed a novel albedo estimation and linear optimization algorithm in this approach.
In the third approach, we further loose the constraint of the input image to arbitrary in the wild images. Our proposed approach can robustly reconstruct high quality model even with extreme expressions and large poses.
We then explore the applicability of our face reconstructions on four interesting applications: video face beautification, generating personalized facial blendshape from image sequences, face video stylizing and video face replacement. We demonstrate great potentials of our reconstruction approaches on these real-world applications. In particular, with the recent surge of interests in VR/AR, it is increasingly common to see people wearing head-mounted displays. However, the large occlusion on face is a big obstacle for people to communicate in a face-to-face manner. Our another application is that we explore hardware/software solutions for synthesizing the face image with presence of HMDs. We design two setups (experimental and mobile) which integrate two near IR cameras and one color camera to solve this problem. With our algorithm and prototype, we can achieve photo-realistic results.
We further propose a deep neutral network to solve the HMD removal problem considering it as a face inpainting problem. This approach doesn\u27t need special hardware and run in real-time with satisfying results
BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction
We propose BareSkinNet, a novel method that simultaneously removes makeup and
lighting influences from the face image. Our method leverages a 3D morphable
model and does not require a reference clean face image or a specified light
condition. By combining the process of 3D face reconstruction, we can easily
obtain 3D geometry and coarse 3D textures. Using this information, we can infer
normalized 3D face texture maps (diffuse, normal, roughness, and specular) by
an image-translation network. Consequently, reconstructed 3D face textures
without undesirable information will significantly benefit subsequent
processes, such as re-lighting or re-makeup. In experiments, we show that
BareSkinNet outperforms state-of-the-art makeup removal methods. In addition,
our method is remarkably helpful in removing makeup to generate consistent
high-fidelity texture maps, which makes it extendable to many realistic face
generation applications. It can also automatically build graphic assets of face
makeup images before and after with corresponding 3D data. This will assist
artists in accelerating their work, such as 3D makeup avatar creation.Comment: accepted at PG202
- …