Photo-realistic face synthesis and reenactment with deep generative models

The advent of Deep Learning has led to numerous breakthroughs in the field of Computer Vision. Over the last decade, a significant amount of research has been undertaken towards designing neural networks for visual data analysis. At the same time, rapid advancements have been made towards the direction of deep generative modeling, especially after the introduction of Generative Adversarial Networks (GANs), which have shown particularly promising results when it comes to synthesising visual data. Since then, considerable attention has been devoted to the problem of photo-realistic human face animation due to its wide range of applications, including image and video editing, virtual assistance, social media, teleconferencing, and augmented reality. The objective of this thesis is to make progress towards generating photo-realistic videos of human faces. To that end, we propose novel generative algorithms that provide explicit control over the facial expression and head pose of synthesised subjects. Despite the major advances in face reenactment and motion transfer, current methods struggle to generate video portraits that are indistinguishable from real data. In this work, we aim to overcome the limitations of existing approaches, by combining concepts from deep generative networks and video-to-video translation with 3D face modelling, and more specifically by capitalising on prior knowledge of faces that is enclosed within statistical models such as 3D Morphable Models (3DMMs). In the first part of this thesis, we introduce a person-specific system that performs full head reenactment using ideas from video-to-video translation. Subsequently, we propose a novel approach to controllable video portrait synthesis, inspired from Implicit Neural Representations (INR). In the second part of the thesis, we focus on person-agnostic methods and present a GAN-based framework that performs video portrait reconstruction, full head reenactment, expression editing, novel pose synthesis and face frontalisation.Open Acces

Publication Bias in Meta-Analysis: Confidence Intervals for Rosenthal's Fail-Safe Number

The purpose of the present paper is to assess the efficacy of confidence intervals for Rosenthal's fail-safe number. Although Rosenthal's estimator is highly used by researchers, its statistical properties are largely unexplored. First of all, we developed statistical theory which allowed us to produce confidence intervals for Rosenthal's fail-safe number.This was produced by discerning whether the number of studies analysed in a meta-analysis is fixed or random. Each case produces different variance estimators. For a given number of studies and a given distribution, we provided five variance estimators. Confidence intervals are examined with a normal approximation and a nonparametric bootstrap. The accuracy of the different confidence interval estimates was then tested by methods of simulation under different distributional assumptions. The half normal distribution variance estimator has the best probability coverage. Finally, we provide a table of lower confidence intervals for Rosenthal's estimator.Comment: Published in the International Scholarly Research Notices in December 201

Space-Constrained Massive MIMO: Hitting the Wall of Favorable Propagation

The recent development of the massive multiple-input multiple-output (MIMO) paradigm, has been extensively based on the pursuit of favorable propagation: in the asymptotic limit, the channel vectors become nearly orthogonal and inter-user interference tends to zero [1]. In this context, previous studies have considered fixed inter-antenna distance, which implies an increasing array aperture as the number of elements increases. Here, we focus on a practical, space-constrained topology, where an increase in the number of antenna elements in a fixed total space imposes an inversely proportional decrease in the inter-antenna distance. Our analysis shows that, contrary to existing studies, inter-user interference does not vanish in the massive MIMO regime, thereby creating a saturation effect on the achievable rate

Image Quality Assessment of a CMOS/Gd 2 O 2 S:Pr,Ce,F X-Ray Sensor

The aim of the present study was to examine the image quality performance of a CMOS digital imaging optical sensor coupled to custom made gadolinium oxysulfide powder scintillators, doped with praseodymium, cerium, and fluorine (Gd 2 O 2 S:Pr,Ce,F). The screens, with coating thicknesses 35.7 and 71.2 mg/cm 2 , were prepared in our laboratory from Gd 2 O 2 S:Pr,Ce,F powder (Phosphor Technology, Ltd.) by sedimentation on silica substrates and were placed in direct contact with the optical sensor. Image quality was determined through single index (information capacity, IC) and spatial frequency dependent parameters, by assessing the Modulation Transfer Function (MTF) and the Normalized Noise Power Spectrum (NNPS). The MTF was measured using the slanted-edge method. The CMOS sensor/Gd 2 O 2 S:Pr,Ce,F screens combinations were irradiated under the RQA-5 (IEC 62220-1) beam quality. The detector response function was linear for the exposure range under investigation. Under the general radiography conditions, both Gd 2 O 2 S:Pr,Ce,F screen/CMOS combinations exhibited moderate imaging properties, in terms of IC, with previously published scintillators, such as CsI:Tl, Gd 2 O 2 S:Tb, and Gd 2 O 2 S:Eu

Development of peptide inhibitor nanoparticles (PINPs) for treatment of Alzheimer’s Disease

Purpose: To investigate the best carrier technology for our β-amyloid (Aβ) aggregation inhibitors by developing three types of liposomes (a) plain liposomes, (b) MAL-PEG liposomes, and finally the combination of retro-inverted peptide RI-OR2- TAT (Ac-rGffvlkGrrrrqrrkkrGyc-NH2) attached onto the surface of MAL-PEG liposomes, creating Peptide Inhibitor Nanoparticles (PINPs) of three different sizes (50, 100 and 200 nm). In addition, these nanoliposomes (NLPs) (with particular focus on PINPs) were examined for their ability to affect Aβ aggregation, and to protect against Aβ cytotoxicity. Methods: The creation of NLPs was carried out by the use of a mini extruder, while the elution of PINPs from a size exclusion column was assessed by Dynamic Light Scattering (DLS). The quantification of peptide bound to liposomes was determined by bicinchoninic acid (BCA) assay, while phospholipid content was quantified by Wako phospholipid assay. The effects of the different types of liposomes on Aβ toxicity and viability of SHSY-5Y neuronal cells were examined by MTS assay, whereas effects on Aβ aggregation were determined by Thioflavin-T (Th-T) assay. In addition, a cell penetration assay was carried out in order to examine the ability of liposomes to penetrate into neuroblastoma SHSY-5Y cells. Results: Low concentrations of PINPs 0.1 μM inhibited Aβ aggregation and toxicity in vitro. MAL-PEG liposomes and PINPs were able to penetrate into neuroblastoma SHSY-5Y cells and were also more stable than simple liposomes. Stability means the ability of liposomes to keep their size and their shape stable for long time. In addition, the three types of liposomes were not toxic towards SHSY-5Y neuroblastoma cells. Cytotoxicity is the quality of being toxic to cells. So, none of the three types of our liposomes showed any negative effect on the viability towards SHSY-5Y neuroblastoma cells. Conclusion: NLPs are an ideal carrier for our aggregation inhibitors because they affect Aβ aggregation and toxicity at low doses, and according to other data generated by our group, can cross the Blood Brain Barrier (BBB)

Dynamic Neural Portraits

We present Dynamic Neural Portraits, a novel approach to the problem of full-head reenactment. Our method generates photo-realistic video portraits by explicitly controlling head pose, facial expressions and eye gaze. Our proposed architecture is different from existing methods that rely on GAN-based image-to-image translation networks for transforming renderings of 3D faces into photo-realistic images. Instead, we build our system upon a 2D coordinate-based MLP with controllable dynamics. Our intuition to adopt a 2D-based representation, as opposed to recent 3D NeRF-like systems, stems from the fact that video portraits are captured by monocular stationary cameras, therefore, only a single viewpoint of the scene is available. Primarily, we condition our generative model on expression blendshapes, nonetheless, we show that our system can be successfully driven by audio features as well. Our experiments demonstrate that the proposed method is 270 times faster than recent NeRF-based reenactment methods, with our networks achieving speeds of 24 fps for resolutions up to 1024 x 1024, while outperforming prior works in terms of visual quality.Comment: In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 202

Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control

We present Free-HeadGAN, a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance, without relying on strong statistical priors of the face, such as 3D Morphable Models. Apart from 3D pose and facial expressions, our method is capable of fully transferring the eye gaze, from a driving actor to a source identity. Our complete pipeline consists of three components: a canonical 3D key-point estimator that regresses 3D pose and expression-related deformations, a gaze estimation network and a generator that is built upon the architecture of HeadGAN. We further experiment with an extension of our generator to accommodate few-shot learning using an attention mechanism, in case more than one source images are available. Compared to the latest models for reenactment and motion transfer, our system achieves higher photo-realism combined with superior identity preservation, while offering explicit gaze control