Machine learning for automatic analysis of affective behaviour

The automated analysis of affect has been gaining rapidly increasing attention by researchers over the past two decades, as it constitutes a fundamental step towards achieving next-generation computing technologies and integrating them into everyday life (e.g. via affect-aware, user-adaptive interfaces, medical imaging, health assessment, ambient intelligence etc.). The work presented in this thesis focuses on several fundamental problems manifesting in the course towards the achievement of reliable, accurate and robust affect sensing systems. In more detail, the motivation behind this work lies in recent developments in the field, namely (i) the creation of large, audiovisual databases for affect analysis in the so-called ''Big-Data`` era, along with (ii) the need to deploy systems under demanding, real-world conditions. These developments led to the requirement for the analysis of emotion expressions continuously in time, instead of merely processing static images, thus unveiling the wide range of temporal dynamics related to human behaviour to researchers. The latter entails another deviation from the traditional line of research in the field: instead of focusing on predicting posed, discrete basic emotions (happiness, surprise etc.), it became necessary to focus on spontaneous, naturalistic expressions captured under settings more proximal to real-world conditions, utilising more expressive emotion descriptions than a set of discrete labels. To this end, the main motivation of this thesis is to deal with challenges arising from the adoption of continuous dimensional emotion descriptions under naturalistic scenarios, considered to capture a much wider spectrum of expressive variability than basic emotions, and most importantly model emotional states which are commonly expressed by humans in their everyday life. In the first part of this thesis, we attempt to demystify the quite unexplored problem of predicting continuous emotional dimensions. This work is amongst the first to explore the problem of predicting emotion dimensions via multi-modal fusion, utilising facial expressions, auditory cues and shoulder gestures. A major contribution of the work presented in this thesis lies in proposing the utilisation of various relationships exhibited by emotion dimensions in order to improve the prediction accuracy of machine learning methods - an idea which has been taken on by other researchers in the field since. In order to experimentally evaluate this, we extend methods such as the Long Short-Term Memory Neural Networks (LSTM), the Relevance Vector Machine (RVM) and Canonical Correlation Analysis (CCA) in order to exploit output relationships in learning. As it is shown, this increases the accuracy of machine learning models applied to this task. The annotation of continuous dimensional emotions is a tedious task, highly prone to the influence of various types of noise. Performed real-time by several annotators (usually experts), the annotation process can be heavily biased by factors such as subjective interpretations of the emotional states observed, the inherent ambiguity of labels related to human behaviour, the varying reaction lags exhibited by each annotator as well as other factors such as input device noise and annotation errors. In effect, the annotations manifest a strong spatio-temporal annotator-specific bias. Failing to properly deal with annotation bias and noise leads to an inaccurate ground truth, and therefore to ill-generalisable machine learning models. This deems the proper fusion of multiple annotations, and the inference of a clean, corrected version of the ``ground truth'' as one of the most significant challenges in the area. A highly important contribution of this thesis lies in the introduction of Dynamic Probabilistic Canonical Correlation Analysis (DPCCA), a method aimed at fusing noisy continuous annotations. By adopting a private-shared space model, we isolate the individual characteristics that are annotator-specific and not shared, while most importantly we model the common, underlying annotation which is shared by annotators (i.e., the derived ground truth). By further learning temporal dynamics and incorporating a time-warping process, we are able to derive a clean version of the ground truth given multiple annotations, eliminating temporal discrepancies and other nuisances. The integration of the temporal alignment process within the proposed private-shared space model deems DPCCA suitable for the problem of temporally aligning human behaviour; that is, given temporally unsynchronised sequences (e.g., videos of two persons smiling), the goal is to generate the temporally synchronised sequences (e.g., the smile apex should co-occur in the videos). Temporal alignment is an important problem for many applications where multiple datasets need to be aligned in time. Furthermore, it is particularly suitable for the analysis of facial expressions, where the activation of facial muscles (Action Units) typically follows a set of predefined temporal phases. A highly challenging scenario is when the observations are perturbed by gross, non-Gaussian noise (e.g., occlusions), as is often the case when analysing data acquired under real-world conditions. To account for non-Gaussian noise, a robust variant of Canonical Correlation Analysis (RCCA) for robust fusion and temporal alignment is proposed. The model captures the shared, low-rank subspace of the observations, isolating the gross noise in a sparse noise term. RCCA is amongst the first robust variants of CCA proposed in literature, and as we show in related experiments outperforms other, state-of-the-art methods for related tasks such as the fusion of multiple modalities under gross noise. Beyond private-shared space models, Component Analysis (CA) is an integral component of most computer vision systems, particularly in terms of reducing the usually high-dimensional input spaces in a meaningful manner pertaining to the task-at-hand (e.g., prediction, clustering). A final, significant contribution of this thesis lies in proposing the first unifying framework for probabilistic component analysis. The proposed framework covers most well-known CA methods, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locality Preserving Projections (LPP) and Slow Feature Analysis (SFA), providing further theoretical insights into the workings of CA. Moreover, the proposed framework is highly flexible, enabling novel CA methods to be generated by simply manipulating the connectivity of latent variables (i.e. the latent neighbourhood). As shown experimentally, methods derived via the proposed framework outperform other equivalents in several problems related to affect sensing and facial expression analysis, while providing advantages such as reduced complexity and explicit variance modelling.Open Acces

Structure and Function Analysis of the Mammalian ATP-Binding Cassette Transporters, ABCB1 and ABCB4.

PhDMammalian ABC (ATP-binding cassette) transporters are integral membrane proteins that translocate allocrites across biological membranes using ATP as a substrate. ABCB1 is a polyspecific efflux pump which can confer multidrug resistance in cancer. ABCB1 is also expressed in a variety of normal tissues where it functions to prevent the accumulation of toxic allocrites. Direct inhibition of ABCB1 can therefore have detrimental effects on patients. Identification of ABCB1-interacting partners that influence trafficking or function would therefore provide alternative targets for therapy which may be cell- or tissue-type specific. The “split-ubiquitin” yeast two-hybrid system, that can detect protein:protein interactions at the plasma membrane, was used to screen for ABCB1-interactors in a human liver library. All candidates isolated from the screen interacted with ABCB1 in a non-specific manner when subjected to strict testing. ABCB4, a close relative of ABCB1, is expressed primarily at the hepatocyte canalicular membrane where it flops phosphatidylcholine (PC) into the outer leaflet for extraction by bile salts. Many ABCB4 non-synonymous mutations have been linked to cholestatic liver diseases in humans, but data confirming an impact on ABCB4 function is lacking. Transient expression of wild-type (WT) ABCB4 in tissue culture has proved difficult because the protein is toxic to HEK293T cells. However, co-expression of the phosphatidylserine flippase ATP8B1 (FIC1) and its accessory protein CDC50A allowed the cells to tolerate ABCB4. To investigate the impact of SNPs on ABCB4 function, equivalent changes were introduced into the ABCB4 cDNA for transient expression in the presence or absence of ATP8B1/CDC50A. ABCB4 expression and targeting to the plasma membrane were monitored by western analysis and confocal microscopy, respectively, and, by “feeding” the transfected cells [methyl-3H]choline, PC efflux to added bile salt acceptor was measured. By thus mimicking the situation at the canalicular membrane I report the preliminary characterisation of nine variants of ABCB4 that have been linked to cholestatic liver disease

Senior medical students’ awareness of radiation risks from common diagnostic imaging examinations

Background: Senior medical students represent future physicians who commonly refer patients for diagnostic imaging studies that may involve ionizing radiation. The radiology curriculum at the University of British Columbia provides students with broad-based knowledge about common imaging examinations. The purpose of this study was to investigate students’ awareness of radiation exposures and risks.Methods: An anonymous multiple-choice cross-sectional questionnaire was distributed to final year medical students to assess knowledge of radiation from common diagnostic examinations and radiation-related risks following completion of the longitudinal radiology curriculum, carried out over the four years of medical training.Results: Sixty-three of 192 eligible students participated (33% response rate). The majority felt that knowledge of radiation doses of common imaging examinations is somewhat or very important; however, only 12% (N = 8) routinely discuss radiation-related risks with patients. While all respondents recognized children as most sensitive to the effects of radiation, only 24% (N = 15) correctly identified gonads as the most radiation-sensitive tissue. Almost all respondents recognized ultrasound and MRI as radiation free modalities. Respondents who correctly identified the relative dose of common imaging examinations in chest x-ray equivalents varied from 3-77% (N = 2 – 49); the remaining responses were largely underestimates. Finally, 44% (N = 28) correctly identified the excess risk of a fatal cancer from an abdominal CT in an adult, while the remainder underestimated this risk.Conclusion: Medical students acknowledge the importance of radiation-related issues to patient care. While almost all students are familiar with radiation-free modalities, many are not familiar with, and commonly underestimate, the relative doses and risks of common imaging studies. This may expose patients to increasing imaging investigations and exposure to radiation hazards

Vancouver simplified grading system with computed tomographic angiography for blunt aortic injury

ObjectiveDelineation of blunt aortic injury by computed tomographic angiography guides management of this potentially fatal injury. Two existing grading systems are problematic to apply and not linked to outcomes. A simplified computed tomographic angiography–based grading system, linked to clinical outcomes, was developed, and feasibility and reliability were evaluated.MethodsRetrospective review was performed of all blunt aortic injury cases presenting to a single provincial quaternary referral center designated for blunt aortic injury management between 2001 and 2009. Management, associated injuries, hospital survival, and cause of death were determined. Initial computed tomographic angiography was reviewed, and injuries were graded according to the new Vancouver simplified grading system by 2 study authors. Three additional trauma radiologists then graded the aortic injuries with the 2 existing systems and the simplified system. Interrater reliability was determined.ResultsForty-eight patients were identified. Two had minimal aortic injury (grade I), 7 had an intimal flap larger than 1 cm (grade II), 32 had traumatic pseudoaneurysm (grade III), 6 had active contrast extravasation (grade IV), and 1 could not be rated. Survivals were 100%, 90%, and 33% for grades I and II, III, and IV, respectively. Of grade III injuries, 14% were medically managed, 68% repaired endovascularly, and 18% repaired with open surgery. Interrater correlation was best with the simplified score, with only 0.5% of cases unable to be classified.ConclusionsThe Vancouver simplified blunt aortic injury grading system is easy to use and correlates with clinical outcomes. Prospective external validation is required

The effect of task on visual attention in interactive virtual environments

Virtual environments for gaming and simulation provide dynamic and adaptive experiences, but, despite advances in multisensory interfaces, these are still primarily visual experiences. To support real-time dynamic adaptation, interactive virtual environments could implement techniques to predict and manipulate human visual attention. One promising way of developing such techniques is to base them on psychophysical observations, an approach that requires a sound understanding of visual attention allocation. Understanding how this allocation of visual attention changes depending on a user’s task offers clear benefits in developing these techniques and improving virtual environment design. With this aim, we investigated the effect of task on visual attention in interactive virtual environments. We recorded fixation data from participants completing freeview, search, and navigation tasks in three different virtual environments. We quantified visual attention differences between conditions by identifying the predictiveness of a low-level saliency model and its corresponding color, intensity, and orientation feature-conspicuity maps, as well as measuring fixation center bias, depth, duration, and saccade amplitude. Our results show that task does affect visual attention in virtual environments. Navigation relies more than search or freeview on intensity conspicuity to allocate visual attention. Navigation also produces fixations that are more central, longer, and deeper into the scenes. Further, our results suggest that it is difficult to distinguish between freeview and search tasks. These results provide important guidance for designing virtual environments for human interaction, as well as identifying future avenues of research for developing “attention-aware” virtual worlds

Glycopeptide resistance in Enterococcus spp. and coagulase-negative staphylococci from hospitalised patients in Germany: occurrence, characteristics and dalbavancin susceptibility

Objectives: The aim of this study was to evaluate the occurrence of glycopeptide resistance in enterococci and coagulase-negative staphylococci (CoNS) and to determine the susceptibilities of the identified glycopeptide-resistant isolates to dalbavancin. Methods: Twenty-two medical laboratories participated in the study conducted in 2016/17 by the Paul-Ehrlich-Society for Chemotherapy. Each laboratory was asked to collect 30 Enterococcus spp. (limited to Enterococcus faecalis and Enterococcus faecium) and 30 CoNS isolates consecutively from hospitalised patients with a proven or suspected infection. Results: A total of 1285 isolates were collected, comprising 364 E. faecalis, 291 E. faecium and 630 CoNS. No E. faecalis isolates (0%) but 76 E. faecium isolates (26.1%) were vancomycin-resistant, of which 21 showed the VanA type and 55 the VanB type. The proportion of vancomycin-resistant strains among E. faecium isolates from patients in intensive care units (21.6%) was significantly lower than that from patients on regular wards (30.5%). Among the CoNS, 67 isolates (10.6%) were teicoplanin-resistant but none were vancomycin-resistant, with resistance only detected in Staphylococcus epidermidis (12.2%), Staphylococcus haemolyticus (17.9%) and Staphylococcus hominis (13.2%). Dalbavancin at ≤0.25 mg/L inhibited all VanB-type enterococci and 95.5% of teicoplanin-resistant CoNS. Conclusion: The level of glycopeptide resistance in E. faecalis remains very low in Germany but achieved 26% in E. faecium and was >10% in CoNS. Dalbavancin appears to be a feasible option for treating infections caused by VanB-type vancomycin-resistant E. faecium and teicoplanin-resistant CoNS.Peer Reviewe

Novel Anti-Infective Compounds from Marine Bacteria

As a result of the continuous evolution of microbial pathogens towards antibiotic-resistance, there have been demands for the development of new and effective antimicrobial compounds. Since the 1960s, the scientific literature has accumulated many publications about novel pharmaceutical compounds produced by a diverse range of marine bacteria. Indeed, marine micro-organisms continue to be a productive and successful focus for natural products research, with many newly isolated compounds possessing potentially valuable pharmacological activities. In this regard, the marine environment will undoubtedly prove to be an increasingly important source of novel antimicrobial metabolites, and selective or targeted approaches are already enabling the recovery of a significant number of antibiotic-producing micro-organisms. The aim of this review is to consider advances made in the discovery of new secondary metabolites derived from marine bacteria, and in particular those effective against the so called “superbugs”, including methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin resistant enterococci (VRE), which are largely responsible for the increase in numbers of hospital acquired, i.e., nosocomial, infections

The Immunological Synapse: a Dynamic Platform for Local Signaling

The immunological synapse (IS) as a concept has evolved from a static view of the junction between T cells and their antigen-presenting cell partners. The entire process of IS formation and extinction is now known to entail a dynamic reorganization of membrane domains and proteins within and adjacent to those domains. Discussion The entire process is also intricately tied to the motility machinery—both as that machinery directs “scanning” prior to T-cell receptor engagement and as it is appropriated during the ongoing developments at the IS. While the synapse often remains dynamic in order to encourage surveillance of new antigen-presenting surfaces, cytoskeletal forces also regulate the development of signals, likely including the assembly of ion channels. In both neuronal and immunological synapses, localized Ca 2+ signals and accumulation or depletion of ions in microdomains accompany the concentration of signaling molecules in the synapse. Such spatiotemporal signaling in the synapse greatly accelerates kinetics and provides essential checkpoints to validate effective cell–cell communication