Recognition of affect in the wild using deep neural networks

In this paper we utilize the first large-scale "in-the-wild" (Aff-Wild) database, which is annotated in terms of the valence-arousal dimensions, to train and test an end-to-end deep neural architecture for the estimation of continuous emotion dimensions based on visual cues. The proposed architecture is based on jointly training convolutional (CNN) and recurrent neural network (RNN) layers, thus exploiting both the invariant properties of convolutional features, while also modelling temporal dynamics that arise in human behaviour via the recurrent layers. Various pre-trained networks are used as starting structures which are subsequently appropriately fine-tuned to the Aff-Wild database. Obtained results show premise for the utilization of deep architectures for the visual analysis of human behaviour in terms of continuous emotion dimensions and analysis of different types of affect

Deep neural network augmentation: generating faces for affect analysis

This paper presents a novel approach for synthesizing facial affect; either in terms of the six basic expressions (i.e., anger, disgust, fear, joy, sadness and surprise), or in terms of valence (i.e., how positive or negative is an emotion) and arousal (i.e., power of the emotion activation). The proposed approach accepts the following inputs:(i) a neutral 2D image of a person; (ii) a basic facial expression or a pair of valence-arousal (VA) emotional state descriptors to be generated, or a path of affect in the 2D VA space to be generated as an image sequence. In order to synthesize affect in terms of VA, for this person, 600,000 frames from the 4DFAB database were annotated. The affect synthesis is implemented by fitting a 3D Morphable Model on the neutral image, then deforming the reconstructed face and adding the inputted affect, and blending the new face with the given affect into the original image. Qualitative experiments illustrate the generation of realistic images, when the neutral image is sampled from fifteen well known lab-controlled or in-the-wild databases, including Aff-Wild, AffectNet, RAF-DB; comparisons with generative adversarial networks (GANs) show the higher quality achieved by the proposed approach. Then, quantitative experiments are conducted, in which the synthesized images are used for data augmentation in training deep neural networks to perform affect recognition over all databases; greatly improved performances are achieved when compared with state-of-the-art methods, as well as with GAN-based data augmentation, in all cases

Weakly-supervised mesh-convolutional hand reconstruction in the wild

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark. The dataset and additional resources are available at https://arielai.com/mesh_hands

Aff-Wild: Valence and Arousal ‘in-the-wild’ Challenge

The Affect-in-the-Wild (Aff-Wild) Challenge proposes a new comprehensive benchmark for assessing the performance of facial affect/behaviour analysis/understanding 'in-the-wild'. The Aff-wild benchmark contains about 300 videos (over 2,000 minutes of data) annotated with regards to valence and arousal, all captured 'in-the-wild' (the main source being Youtube videos). The paper presents the database description, the experimental set up, the baseline method used for the Challenge and finally the summary of the performance of the different methods submitted to the Affect-in-the-Wild Challenge for Valence and Arousal estimation. The challenge demonstrates that meticulously designed deep neural networks can achieve very good performance when trained with in-the-wild data

Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond

Automatic understanding of human affect using visual signals is of great importance in everyday human–machine interac- tions. Appraising human emotional states, behaviors and reactions displayed in real-world settings, can be accomplished using latent continuous dimensions (e.g., the circumplex model of affect). Valence (i.e., how positive or negative is an emo- tion) and arousal (i.e., power of the activation of the emotion) constitute popular and effective representations for affect. Nevertheless, the majority of collected datasets this far, although containing naturalistic emotional states, have been captured in highly controlled recording conditions. In this paper, we introduce the Aff-Wild benchmark for training and evaluating affect recognition algorithms. We also report on the results of the First Affect-in-the-wild Challenge (Aff-Wild Challenge) that was recently organized in conjunction with CVPR 2017 on the Aff-Wild database, and was the first ever challenge on the estimation of valence and arousal in-the-wild. Furthermore, we design and extensively train an end-to-end deep neural architecture which performs prediction of continuous emotion dimensions based on visual cues. The proposed deep learning architecture, AffWildNet, includes convolutional and recurrent neural network layers, exploiting the invariant properties of convolutional features, while also modeling temporal dynamics that arise in human behavior via the recurrent layers. The AffWildNet produced state-of-the-art results on the Aff-Wild Challenge. We then exploit the AffWild database for learning features, which can be used as priors for achieving best performances both for dimensional, as well as categorical emo- tion recognition, using the RECOLA, AFEW-VA and EmotiW 2017 datasets, compared to all other methods designed for the same goal. The database and emotion recognition models are available at http://ibug.doc.ic.ac.uk/resources/first-affect-wild-challenge

Widespread expression of erythropoietin receptor in brain and its induction by injury

Erythropoietin (EPO) exerts potent neuroprotective, neuroregenerative and procognitive functions. However, unequivocal demonstration of erythropoietin receptor (EPOR) expression in brain cells has remained difficult since previously available anti-EPOR antibodies (EPOR-AB) were unspecific. We report here a new, highly specific, polyclonal rabbit EPOR-AB directed against different epitopes in the cytoplasmic tail of human and murine EPOR and its characterization by mass spectrometric analysis of immuno-precipitated endogenous EPOR, Western blotting, immunostaining and flow cytometry. Among others, we applied genetic strategies including overexpression, Lentivirus-mediated conditional knockout of EpoR and tagged proteins, both on cultured cells and tissue sections, as well as intracortical implantation of EPOR-transduced cells to verify specificity. We show examples of EPOR expression in neurons, oligodendroglia, astrocytes and microglia. Employing this new EPOR-AB with double-labeling strategies, we demonstrate membrane expression of EPOR as well as its localization in intracellular compartments such as the Golgi apparatus. Moreover, we show injury-induced expression of EPOR. In mice, a stereotactically applied stab wound to the motor cortex leads to distinct EpoR expression by reactive GFAP-expressing cells in the lesion vicinity. In a patient suffering from epilepsy, neurons and oligodendrocytes of the hippocampus strongly express EPOR. To conclude, this new analytical tool will allow neuroscientists to pinpoint EPOR expression in cells of the nervous system and to better understand its role in healthy conditions, including brain development, as well as under pathological circumstances, such as upregulation upon distress and injury

Analytical Validation and Clinical Qualification of a New Immunohistochemical Assay for Androgen Receptor Splice Variant-7 Protein Expression in Metastatic Castration-resistant Prostate Cancer.

Background The androgen receptor splice variant-7 (AR-V7) has been implicated in the development of castration-resistant prostate cancer (CRPC) and resistance to abiraterone and enzalutamide.Objective To develop a validated assay for detection of AR-V7 protein in tumour tissue and determine its expression and clinical significance as patients progress from hormone-sensitive prostate cancer (HSPC) to CRPC.Design, setting, and participants Following monoclonal antibody generation and validation, we retrospectively identified patients who had HSPC and CRPC tissue available for AR-V7 immunohistochemical (IHC) analysis.Outcome measurements and statistical analysis Nuclear AR-V7 expression was determined using IHC H score (HS) data. The change in nuclear AR-V7 expression from HSPC to CRPC and the association between nuclear AR-V7 expression and overall survival (OS) was determined.Results and limitations Nuclear AR-V7 expression was significantly lower in HSPC (median HS 50, interquartile range [IQR] 17.5-90) compared to CRPC (HS 135, IQR 80-157.5; p<0.0001), and in biopsy tissue taken before (HS 80, IQR 30-136.3) compared to after (HS 140, IQR 105-167.5; p=0.007) abiraterone or enzalutamide treatment. Lower nuclear AR-V7 expression at CRPC biopsy was associated with longer OS (hazard ratio 1.012, 95% confidence interval 1.004-1.020; p=0.003). While this monoclonal antibody primarily binds to AR-V7 in PC biopsy tissue, it may also bind to other proteins.Conclusions We provide the first evidence that nuclear AR-V7 expression increases with emerging CRPC and is prognostic for OS, unlike antibody staining for the AR N-terminal domain. These data indicate that AR-V7 is important in CRPC disease biology; agents targeting AR splice variants are needed to test this hypothesis and further improve patient outcome from CRPC.Patient summary In this study we found that levels of the protein AR-V7 were higher in patients with advanced prostate cancer. A higher level of AR-V7 identifies a group of patients who respond less well to certain prostate cancer treatments and live for a shorter period of time

Robust statistical frontalization of human and animal faces

The unconstrained acquisition of facial data in real-world conditions may result in face images with significant pose variations, illumination changes, and occlusions, affecting the performance of facial landmark localization and recognition methods. In this paper, a novel method, robust to pose, illumination variations, and occlusions is proposed for joint face frontalization and landmark localization. Unlike the state-of-the-art methods for landmark localization and pose correction, where large amount of manually annotated images or 3D facial models are required, the proposed method relies on a small set of frontal images only. By observing that the frontal facial image of both humans and animals, is the one having the minimum rank of all different poses, a model which is able to jointly recover the frontalized version of the face as well as the facial landmarks is devised. To this end, a suitable optimization problem is solved, concerning minimization of the nuclear norm (convex surrogate of the rank function) and the matrix ℓ1 norm accounting for occlusions. The proposed method is assessed in frontal view reconstruction of human and animal faces, landmark localization, pose-invariant face recognition, face verification in unconstrained conditions, and video inpainting by conducting experiment on 9 databases. The experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods for the target problems

A phase I dose-escalation study of enzalutamide in combination with the AKT inhibitor AZD5363 (capivasertib) in patients with metastatic castration-resistant prostate cancer.

Background Activation of the PI3K/AKT/mTOR pathway through loss of phosphatase and tensin homolog (PTEN) occurs in approximately 50% of patients with metastatic castration-resistant prostate cancer (mCRPC). Recent evidence suggests that combined inhibition of the androgen receptor (AR) and AKT may be beneficial in mCRPC with PTEN loss.Patients and methods mCRPC patients who previously failed abiraterone and/or enzalutamide, received escalating doses of AZD5363 (capivasertib) starting at 320 mg twice daily (b.i.d.) given 4 days on and 3 days off, in combination with enzalutamide 160 mg daily. The co-primary endpoints were safety/tolerability and determining the maximum tolerated dose and recommended phase II dose; pharmacokinetics, antitumour activity, and exploratory biomarker analysis were also evaluated.Results Sixteen patients were enrolled, 15 received study treatment and 13 were assessable for dose-limiting toxicities (DLTs). Patients were treated at 320, 400, and 480 mg b.i.d. dose levels of capivasertib. The recommended phase II dose identified for capivasertib was 400 mg b.i.d. with 1/6 patients experiencing a DLT (maculopapular rash) at this level. The most common grade ≥3 adverse events were hyperglycemia (26.7%) and rash (20%). Concomitant administration of enzalutamide significantly decreased plasma exposure of capivasertib, though this did not appear to impact pharmacodynamics. Three patients met the criteria for response (defined as prostate-specific antigen decline ≥50%, circulating tumour cell conversion, and/or radiological response). Responses were seen in patients with PTEN loss or activating mutations in AKT, low or absent AR-V7 expression, as well as those with an increase in phosphorylated extracellular signal-regulated kinase (pERK) in post-exposure samples.Conclusions The combination of capivasertib and enzalutamide is tolerable and has antitumour activity, with all responding patients harbouring aberrations in the PI3K/AKT/mTOR pathway.Clinical trial number NCT02525068

Genomics of lethal prostate cancer at diagnosis and castration resistance.

The genomics of primary prostate cancer differ from those of metastatic castration-resistant prostate cancer (mCRPC). We studied genomic aberrations in primary prostate cancer biopsies from patients who developed mCRPC, also studying matching, same-patient, diagnostic, and mCRPC biopsies following treatment. We profiled 470 treatment-naive prostate cancer diagnostic biopsies and, for 61 cases, mCRPC biopsies, using targeted and low-pass whole-genome sequencing (n = 52). Descriptive statistics were used to summarize mutation and copy number profile. Prevalence was compared using Fisher's exact test. Survival correlations were studied using log-rank test. TP53 (27%) and PTEN (12%) and DDR gene defects (BRCA2 7%; CDK12 5%; ATM 4%) were commonly detected. TP53, BRCA2, and CDK12 mutations were markedly more common than described in the TCGA cohort. Patients with RB1 loss in the primary tumor had a worse prognosis. Among 61 men with matched hormone-naive and mCRPC biopsies, differences were identified in AR, TP53, RB1, and PI3K/AKT mutational status between same-patient samples. In conclusion, the genomics of diagnostic prostatic biopsies acquired from men who develop mCRPC differ from those of the nonlethal primary prostatic cancers. RB1/TP53/AR aberrations are enriched in later stages, but the prevalence of DDR defects in diagnostic samples is similar to mCRPC