Sparse Linear Concept Discovery Models

The recent mass adoption of DNNs, even in safety-critical scenarios, has shifted the focus of the research community towards the creation of inherently intrepretable models. Concept Bottleneck Models (CBMs) constitute a popular approach where hidden layers are tied to human understandable concepts allowing for investigation and correction of the network's decisions. However, CBMs usually suffer from: (i) performance degradation and (ii) lower interpretability than intended due to the sheer amount of concepts contributing to each decision. In this work, we propose a simple yet highly intuitive interpretable framework based on Contrastive Language Image models and a single sparse linear layer. In stark contrast to related approaches, the sparsity in our framework is achieved via principled Bayesian arguments by inferring concept presence via a data-driven Bernoulli distribution. As we experimentally show, our framework not only outperforms recent CBM approaches accuracy-wise, but it also yields high per example concept sparsity, facilitating the individual investigation of the emerging concepts.Comment: Accepted @ ICCVW CLVL 202

Local Competition and Stochasticity for Adversarial Robustness in Deep Learning

This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each model layer, as the units are organized in blocks where only one unit generates a non-zero output. The main operating principle of the introduced units lies on stochastic arguments, as the network performs posterior sampling over competing units to select the winner. We combine these LWTA arguments with tools from the field of Bayesian non-parametrics, specifically the stick-breaking construction of the Indian Buffet Process, to allow for inferring the sub-part of each layer that is essential for modeling the data at hand. Then, inference is performed by means of stochastic variational Bayes. We perform a thorough experimental evaluation of our model using benchmark datasets. As we show, our method achieves high robustness to adversarial perturbations, with state-of-the-art performance in powerful adversarial attack schemes.Comment: Accepted AISTATS 2021. arXiv admin note: text overlap with arXiv:2006.1062

A Deep Learning Approach for Dynamic Balance Sheet Stress Testing

In the aftermath of the financial crisis, supervisory authorities have considerably improved their approaches in performing financial stress testing. However, they have received significant criticism by the market participants due to the methodological assumptions and simplifications employed, which are considered as not accurately reflecting real conditions. First and foremost, current stress testing methodologies attempt to simulate the risks underlying a financial institution's balance sheet by using several satellite models, making their integration a really challenging task with significant estimation errors. Secondly, they still suffer from not employing advanced statistical techniques, like machine learning, which capture better the nonlinear nature of adverse shocks. Finally, the static balance sheet assumption, that is often employed, implies that the management of a bank passively monitors the realization of the adverse scenario, but does nothing to mitigate its impact. To address the above mentioned criticism, we introduce in this study a novel approach utilizing deep learning approach for dynamic balance sheet stress testing. Experimental results give strong evidence that deep learning applied in big financial/supervisory datasets create a state of the art paradigm, which is capable of simulating real world scenarios in a more efficient way.Comment: Preprint submitted to Journal of Forecastin

A map of transcriptional heterogeneity and regulatory variation in human microglia.

Microglia, the tissue-resident macrophages of the central nervous system (CNS), play critical roles in immune defense, development and homeostasis. However, isolating microglia from humans in large numbers is challenging. Here, we profiled gene expression variation in primary human microglia isolated from 141 patients undergoing neurosurgery. Using single-cell and bulk RNA sequencing, we identify how age, sex and clinical pathology influence microglia gene expression and which genetic variants have microglia-specific functions using expression quantitative trait loci (eQTL) mapping. We follow up one of our findings using a human induced pluripotent stem cell-based macrophage model to fine-map a candidate causal variant for Alzheimer's disease at the BIN1 locus. Our study provides a population-scale transcriptional map of a critically important cell for human CNS development and disease

Modified penetrance of coding variants by cis-regulatory variation contributes to disease risk

Coding variants represent many of the strongest associations between genotype and phenotype; however, they exhibit interindividual differences in effect, termed 'variable penetrance'. Here, we study how cis-regulatory variation modifies the penetrance of coding variants. Using functional genomic and genetic data from the Genotype-Tissue Expression Project (GTEx), we observed that in the general population, purifying selection has depleted haplotype combinations predicted to increase pathogenic coding variant penetrance. Conversely, in cancer and autism patients, we observed an enrichment of penetrance increasing haplotype configurations for pathogenic variants in disease-implicated genes, providing evidence that regulatory haplotype configuration of coding variants affects disease risk. Finally, we experimentally validated this model by editing a Mendelian single-nucleotide polymorphism (SNP) using CRISPR/Cas9 on distinct expression haplotypes with the transcriptome as a phenotypic readout. Our results demonstrate that joint regulatory and coding variant effects are an important part of the genetic architecture of human traits and contribute to modified penetrance of disease-causing variants.Peer reviewe

Engineering and characterisation of chimeric monoclonal antibody 806 (ch806) for targeted immunotherapy of tumours expressing de2-7 EGFR or amplified EGFR

We report the generation of a chimeric monoclonal antibody (ch806) with specificity for an epitope on the epidermal growth factor receptor (EGFR) that is different from that targeted by all other anti-EGFR therapies. Ch806 antibody is reactive to both de2-7 and overexpressed wild-type (wt) EGFR but not native EGFR expressed in normal tissues at physiological levels. Ch806 was stably expressed in CHO (DHFR −/−) cells and purified for subsequent characterisation and validated for use in preliminary immunotherapy investigations. Ch806 retained the antigen binding specificity and affinity of the murine parental antibody. Furthermore, ch806 displayed enhanced antibody-dependent cellular cytotoxicity against target cells expressing the 806 antigen in the presence of human effector cells. Ch806 was successfully radiolabelled with both iodine-125 and indium-111 without loss of antigen binding affinity or specificity. The radioimmunoconjugates were stable in the presence of human serum at 37°C for up to 9 days and displayed a terminal half-life (T1/2β) of approximately 78 h in nude mice. Biodistribution studies undertaken in BALB/c nude mice bearing de2-7 EGFR-expressing or amplified EGFR-expressing xenografts revealed that 125I-labelled ch806 failed to display any significant tumour retention. However, specific and prolonged tumour localisation of' 111In-labelled ch806 was demonstrated with uptake of 31%ID g−1 and a tumour to blood ratio of 5 : 1 observed at 7 days postinjection. In vivo therapy studies with ch806 demonstrated significant antitumour effects on established de2-7 EGFR xenografts in BALB/c nude mice compared to control, and both murine 806 and the anti-EGFR 528 antibodies. These results support a potential therapeutic role of ch806 in the treatment of suitable EGFR-expressing tumours, and warrants further investigation of the potential of ch806 as a therapeutic agent