ICface: Interpretable and Controllable Face Reenactment Using GANs

This paper presents a generic face animator that is able to control the pose and expressions of a given face image. The animation is driven by human interpretable control signals consisting of head pose angles and the Action Unit (AU) values. The control information can be obtained from multiple sources including external driving videos and manual controls. Due to the interpretable nature of the driving signal, one can easily mix the information between multiple sources (e.g. pose from one image and expression from another) and apply selective post-production editing. The proposed face animator is implemented as a two-stage neural network model that is learned in a self-supervised manner using a large video collection. The proposed Interpretable and Controllable face reenactment network (ICface) is compared to the state-of-the-art neural network-based face animation techniques in multiple tasks. The results indicate that ICface produces better visual quality while being more versatile than most of the comparison methods. The introduced model could provide a lightweight and easy to use tool for a multitude of advanced image and video editing tasks.Comment: Accepted in WACV-202

Human Factors Considerations in System Design

Human factors considerations in systems design was examined. Human factors in automated command and control, in the efficiency of the human computer interface and system effectiveness are outlined. The following topics are discussed: human factors aspects of control room design; design of interactive systems; human computer dialogue, interaction tasks and techniques; guidelines on ergonomic aspects of control rooms and highly automated environments; system engineering for control by humans; conceptual models of information processing; information display and interaction in real time environments

Analysis of Signal Decomposition and Stain Separation methods for biomedical applications

Nowadays, the biomedical signal processing and classification and medical image interpretation play an essential role in the detection and diagnosis of several human diseases. The problem of high variability and heterogeneity of information, which is extracted from digital data, can be addressed with signal decomposition and stain separation techniques which can be useful approaches to highlight hidden patterns or rhythms in biological signals and specific cellular structures in histological color images, respectively. This thesis work can be divided into two macro-sections. In the first part (Part I), a novel cascaded RNN model based on long short-term memory (LSTM) blocks is presented with the aim to classify sleep stages automatically. A general workflow based on single-channel EEG signals is developed to enhance the low performance in staging N1 sleep without reducing the performances in the other sleep stages (i.e. Wake, N2, N3 and REM). In the same context, several signal decomposition techniques and time-frequency representations are deployed for the analysis of EEG signals. All extracted features are analyzed by using a novel correlation-based timestep feature selection and finally the selected features are fed to a bidirectional RNN model. In the second part (Part II), a fully automated method named SCAN (Stain Color Adaptive Normalization) is proposed for the separation and normalization of staining in digital pathology. This normalization system allows to standardize digitally, automatically and in a few seconds, the color intensity of a tissue slide with respect to that of a target image, in order to improve the pathologist’s diagnosis and increase the accuracy of computer-assisted diagnosis (CAD) systems. Multiscale evaluation and multi-tissue comparison are performed for assessing the robustness of the proposed method. In addition, a stain normalization based on a novel mathematical technique, named ICD (Inverse Color Deconvolution) is developed for immunohistochemical (IHC) staining in histopathological images. In conclusion, the proposed techniques achieve satisfactory results compared to state-of-the-art methods in the same research field. The workflow proposed in this thesis work and the developed algorithms can be employed for the analysis and interpretation of other biomedical signals and for digital medical image analysis

Electroencephalography (EEG), electromyography (EMG) and eye-tracking for astronaut training and space exploration

The ongoing push to send humans back to the Moon and to Mars is giving rise to a wide range of novel technical solutions in support of prospective astronaut expeditions. Against this backdrop, the European Space Agency (ESA) has recently launched an investigation into unobtrusive interface technologies as a potential answer to such challenges. Three particular technologies have shown promise in this regard: EEG-based brain-computer interfaces (BCI) provide a non-invasive method of utilizing recorded electrical activity of a user's brain, electromyography (EMG) enables monitoring of electrical signals generated by the user's muscle contractions, and finally, eye tracking enables, for instance, the tracking of user's gaze direction via camera recordings to convey commands. Beyond simply improving the usability of prospective technical solutions, our findings indicate that EMG, EEG, and eye-tracking could also serve to monitor and assess a variety of cognitive states, including attention, cognitive load, and mental fatigue of the user, while EMG could furthermore also be utilized to monitor the physical state of the astronaut. In this paper, we elaborate on the key strengths and challenges of these three enabling technologies, and in light of ESA's latest findings, we reflect on their applicability in the context of human space flight. Furthermore, a timeline of technological readiness is provided. In so doing, this paper feeds into the growing discourse on emerging technology and its role in paving the way for a human return to the Moon and expeditions beyond the Earth's orbit

An eeg based study of unintentional sleep onset

Ph.DDOCTOR OF PHILOSOPH

The role of visual information in the steering behaviour of young and adult bicyclists

In a first series of experiments, the visual behaviour during different steering tasks, and under different constraints, was investigated in an indoor environment. Young learner, and experienced adult bicyclists were asked to steer through narrow lanes, a curved lane, and a slalom. Participants directed their gaze to the future path about one to two seconds ahead, and moved forward using optokinetic nystagmus-like eye movements. Both cycling speed and task demand were found to affect the visual behaviour of bicyclists. Although these shifts of visual attention were in line with earlier findings in pedestrians and car drivers, they did not seem to be entirely in line with the two-level model of steering behaviour. Therefore, a redefined version of this model was proposed as the ‘gaze constraints model for steering’. During a simple linear steering task, the visual behaviour of children (between 6 and 12 years of age) was similar to that of adults. However, in a more demanding slalom task children adopted a different visual-motor strategy. Whereas adults made more use of anticipatory fixations and often looked at the functional space between two cones, children mainly focussed on the upcoming cone. These findings suggest that adults plan their route through the slalom whereas children focus on steering around one cone at the time. In a second series of experiments, the distribution of visual attention was investigated in an actual traffic environment and the influence of a low quality cycling track on visual behaviour was studied. Results showed that children direct their gaze more to the environment and less to the path than adults. However, both adults and children made an apparent shift of visual attention from distant environmental regions towards more proximate road properties on the low quality cycling track. In general, the current thesis provides insights into how visual attention of young and adult bicyclists is distributed during different steering tasks and how this is affected by individual, task, and environmental constraints. Based on the current results, a gaze constraints model for steering was proposed. Furthermore, it seems that children adapted their visual behaviour to their limited capabilities, but that children’s visual behaviour changes in a similar way to changing task constraints as the visual behaviour of adults. These findings suggest that traffic rules, road infrastructure and traffic education should take into account the limited capabilities of children. However, it should be noted that this work only focussed on the lane-keeping task. Future research should therefore study the integration of these findings in the visual control of other traffic tasks such as hazard perception. A better understanding of the development of information processing of young learner bicyclists could potentially lead to better traffic education and more appropriate road infrastructure. Additionally, a new fixation-by-fixation analysis method to analyze head-mounted eye tracking data was tested in this thesis. This method was found to be a good alternative to the time-consuming frame-by-frame method, provided that the areas of interest were large, and the analysis is done over an extended period of time

Combining Smart Material Platforms and New Computational Tools to Investigate Cell Motility Behavior and Control

Cell-extracellular matrix (ECM) interactions play a critical role in regulating important biological phenomena, including morphogenesis, tissue repair, and disease states. In vivo, cells are subjected to various mechanical, chemical, and electrical cues to collectively guide their functionality within a specific microenvironment. To better understand the mechanisms regulating cell adhesive, differentiation, and motility dynamics, researchers have developed in vitro platforms to synthetically mimic native tissue responses. While important information about cell-ECM interactions have been revealed using these systems, a knowledge gap currently exists regarding how cell responses in static environments relate to the dynamic cell-ECM interaction behaviors observed in vivo. Advances at the intersection of materials science, biophysics, and cell biology have recently enabled the production of dynamic ECM mimics where cells can be exposed to controlled mechanical, electrical or chemical cues to directly decouple cell-ECM related behaviors from cell-cell or cell-environmental factors. Utilization of these dynamic synthetic biomaterials will enable discovery of novel mechanisms fundamental in tissue development, homeostasis, repair, and disease. In this dissertation, the primary goal was to evaluate how mechanical changes in the ECM regulate cell motility and polarization responses. This was accomplished through two major aims: 1) by developing a modular image processing tool that could be applied in complex synthetic in vitro microenvironments to asses cell motility dynamics, and 2) to utilize that tool to advance understanding of mechanobiology and mechanotransduction processes associated with development, wound healing, and disease progression. Therefore, the first portion of this thesis (Chapters 2 and 3) dealt with proof of concept for our newly developed automated cell tracking system, termed ACTIVE (automated contour-based tracking for in vitro environments), while the second portion of this thesis (Chapter 4-7) addressed applying this system in multiple experimental designs to synthesize new knowledge regarding cell-ECM or cell-cell interactions. In Chapter 1, we introduced why cell-ECM interactions are essential for in vivo processes and highlighted the current state of the literature. In Chapter 2, we demonstrated that ACTIVE could achieve greater than 95% segmentation accuracy at multiple cell densities, while improving two-body cell-cell interaction error by up to 43%. In Chapter 3 we showed that ACTIVE could be applied to reveal subtle differences in fibroblast motility atop static wrinkled or static non-wrinkled surfaces at multiple cell densities. In Chapters 4 and 5, we characterized fibroblast motility and intracellular reorganization atop a dynamic shape memory polymer biomaterial, focusing on the role of the Rho-mediated pathway in the observed responses. We then utilized ACTIVE to identify differences in subpopulation dynamics of monoculture versus co-culture endothelial and smooth muscle cells (Chapter 6). In Chapter 7, we applied ACTIVE to investigate E. coli biofilm formation atop poly(dimethylsiloxane) surfaces with varying stiffness and line patterns. Finally, we presented a summary and future work in Chapter 8. Collectively, this work highlights the capabilities of the newly developed ACTIVE tracking system and demonstrates how to synthesize new information about mechanobiology and mechanotransduction processes using dynamic biomaterial platforms