Predicting the Efficiency of Interferon Therapy for Multiple Sclerosis using Genotype-based Machine Learning Models

Despite extensive research, the pathogenesis of various autoimmune diseases still remains partly unresolved. For example, the cause of multiple sclerosis (MS), one of the most common neurodegenerative autoimmune diseases, is still unknown and treatment approaches are limited. In most cases, interferon-β is an effective medication for MS Hartung et al. (2013); Sitzer and Steinmetz (2011). However, within time a large percentage of the patients treated with interferon-β produce binding antibodies (BABs) or neutralizing antibodies (NABs) which either bind or neutralize interferon-β and lead to therapy failure Creeke and Farrell (2013). The aim of this thesis is to predict therapy response for interferon-β therapy by analyzing patients' genotypes. The data of MS patients treated with interferon-β as well as data on antibody development subsequent to medication and the genotype information were provided by the Neurological Department of Klinikum Rechts der Isar, Munich. We analyzed the data with a machine learning approach and discovered candidate genes that may be involved in antibody production in response to interferon-β treatment and might lead to a better understanding of the underlying molecular mechanism. So far the HLA-DRB1 gene and the SNP rs9272105, localized in close proximity to the HLA-DQA1 gene on chromosome 6, have been associated with antibody production against interferon-β Barbosa et al. (2006); Buck et al. (2011); Buck and Hemmer (2014); Hoffmann et al. (2008); Link et al. (2014); Soelberg Sorensen (2008); Weber et al. (2012). The SNPs rs4961252, localized on chromosome 8, and rs5743810, within the TLR6 gene on chromosome 4, also showed genome-wide significance, yet the latter was only the case in males whereas not in females Weber et al. (2012); Buck and Hemmer (2014); Enevold et al. (2010). In this project, prediction models were created using machine learning techniques through the use of Support Vector Machines (SVMs). I wanted to go beyond single SNP effects and include SNP x SNP interactions in order to create a model based on candidate SNPs to predict a patient's response to medication for treatment of MS. Compared to other machine learning techniques, SVMs have the advantage of also accounting for SNP x SNP interactions. In order to keep the number of SNP variants manageable for the SVM calculations, I partitioned the data in gene-wise subsets. For each gene-wise dataset, prediction models containing the SNPs that were ranked by their ability to predict antibody production were generated. These calculations resulted in a list of significant genes including the predictive features (SNPs). From these results I was able to identify the SNPs that achieved the best performance. The results included HLA genes as well as the HCG23 and BTNL2 genes in close proximity on chromosome 6 to reveal significance. The SNP rs34784936, localized within the HLA region, achieved the best single SNP performance. Genome-wide, we found 78 genes with significant results based on 315 SNPs. Of those, only the most relevant 166 SNPs need to be included in the final prediction model, since at that point the performance of the pruning calculation reaches its maximum. It is important to note that only a small set of selected genotype information of an individual patient is needed to predict therapy response. The identified genes associated with antibody production against interferon-β require further investigation. Übersetzte Kurzfassung: Trotz intensiver Forschung ist die Pathogenese verschiedener neurologischer Krankheiten bislang noch teils ungeklärt. So ist beispielsweise die Ätiologie der Multiplen Sklerose, einer der häufigsten neurodegenerativen Autoimmunkrankheiten, noch nicht vollständig bekannt und Therapieansätze sind nur eingeschränkt verfügbar. In den meisten Fällen stellt Interferon-β eine effektive Therapieoption dar Hartung et al. (2013); Sitzer and Steinmetz (2011). Dennoch entwickeln eine bedeutsame Anzahl der Patienten bindende Antikörper (BABs) oder neutralisierende Antikörper (NABs), die das Medikament binden bzw. neutralisieren und damit zu Therapieversagen führen Creeke and Farrell (2013). Ziel dieser Arbeit war die Entwicklung eines auf Genotypen basierenden Vorhersagemodells, anhand dessen die Wahrscheinlichkeit der Antikörperbildung auf Interferon-β Medikation schon vor Therapiebeginn abgeschätzt werden kann. Darüber hinaus könnte man mögliche Kandidaten Gene identifizieren, anhand derer dann auf ein besseres Verständnis der molekularen Mechanismen gehofft werden kann, die dieser Krankheit und der Produktion von Antikörpern zugrunde liegen. Nach aktuellen Forschungsergebnissen liefern das Gen HLA-DRB1, sowie der SNP rs9272105, welcher in der Nähe des Genes HLA-DQA1 auf Chromosom 6 lokalisiert ist, erste Hinweise auf eine Assoziation von Antikörperproduktion als Reaktion auf eine Interferon-β Therapie Barbosa et al. (2006); Buck et al. (2011); Buck and Hemmer (2014); Hoffmann et al. (2008); Link et al. (2014); Soelberg Sorensen (2008); Weber et al. (2012). Auch die SNPs rs4961252 auf Chromosom 8 und rs5743810, welcher innerhalb des Gens TLR6 auf Chromosom 4 liegt, zeigten genomweite Signifikanz in Zusammenhang mit der Produktion von Antikörpern gegen Interferon-β letzterer jedoch nur bei männlichen Patienten Weber et al. (2012); Buck and Hemmer (2014); Enevold et al. (2010). Mit der Fragestellung, ob anhand von genetischen Prädikatoren eine Vorhersage getroffen werden kann, wurden uns sowohl die Genotypen als auch die Daten zum Antikörpertiter gegen Interferon-β von der neurologischen Abteilung des Klinikums Rechts der Isar, München zur Verfügung gestellt. Diese Dissertation beinhaltet die Entwicklung eines Vorhersagemodels zur Antikörperproduktion gegen Interferon-β unter Berücksichtigung von SNP x SNP Interaktionen. Support Vector Machines ist eine Methode des maschinellen Lernens, die im Gegensatz zu anderen Methoden in der Lage ist solche Interaktionen zu berücksichtigen. Dadurch geht dieses Modell über bisherige Forschungsansätze hinaus, die sich auf die Analyse von Einzel-SNP-Assoziationen oder maximal paarweisen Epistasiseffekten stützen. Um die mögliche Anzahl der miteinbezogenen SNPs für eine SVM Berechnung nicht zu überschreiten, wurden die Genotypen genweise nach Gengrenzen aufgeteilt. Für jedes Gen wurde ein Vorhersagemodel erstellt, das die zugeordneten SNPs entsprechend ihres Einflusses bezüglich einer Vorhersage zur Produktion von Antikörpern einstuft. Als Resultat ergab sich eine Liste signifikanter Gene mit den jeweils vorhersagerelevanten SNPs. Dadurch war es möglich, die vorhersagekräftigsten SNPs zu bestimmen. Sowohl einige HLA Gene, aber auch die unmittelbar benachbarten Gene HCG23 und BTNL2 auf Chromosom 6 konnten als signifikant ermittelt werden. In den genomweiten Resultaten fanden sich 78 signifikante Gene mit 315 relevanten SNPs. Das endgültige Modell nutzt davon die 166 besten SNPs, welche die beste Vorhersage lieferten, da zu diesem Zeitpunkt bereits das Maximum der Vorhersage erreicht werden kann. Wesentlich ist, dass für die zukünftige Anwendung dieses Modells nur ein ausgewählter Anteil der Genotypen eines Patienten zur Vorhersage benötigt wird. Dafür könnte man spezielle Tests entwickeln, die nur die im Modell verwendeten SNPs benötigen und somit relativ einfach und kostengünstig durchzuführen wären. Die identifizierten Gene sollten hinsichtlich ihrer Bedeutung weiter untersucht werden

NFormer: Robust Person Re-identification with Neighbor Transformer

Person re-identification aims to retrieve persons in highly varying settings across different cameras and scenarios, in which robust and discriminative representation learning is crucial. Most research considers learning representations from single images, ignoring any potential interactions between them. However, due to the high intra-identity variations, ignoring such interactions typically leads to outlier features. To tackle this issue, we propose a Neighbor Transformer Network, or NFormer, which explicitly models interactions across all input images, thus suppressing outlier features and leading to more robust representations overall. As modelling interactions between enormous amount of images is a massive task with lots of distractors, NFormer introduces two novel modules, the Landmark Agent Attention, and the Reciprocal Neighbor Softmax. Specifically, the Landmark Agent Attention efficiently models the relation map between images by a low-rank factorization with a few landmarks in feature space. Moreover, the Reciprocal Neighbor Softmax achieves sparse attention to relevant -- rather than all -- neighbors only, which alleviates interference of irrelevant representations and further relieves the computational burden. In experiments on four large-scale datasets, NFormer achieves a new state-of-the-art. The code is released at \url{https://github.com/haochenheheda/NFormer}.Comment: 8 pages, 7 figures, CVPR2022 poste

Technical Change and Industrial Dynamics as Evolutionary Processes

This work prepared for B. Hall and N. Rosenberg (eds.) Handbook of Innovation, Elsevier (2010), lays out the basic premises of this research and review and integrate much of what has been learned on the processes of technological evolution, their main features and their effects on the evolution of industries. First, we map and integrate the various pieces of evidence concerning the nature and structure of technological knowledge the sources of novel opportunities, the dynamics through which they are tapped and the revealed outcomes in terms of advances in production techniques and product characteristics. Explicit recognition of the evolutionary manners through which technological change proceed has also profound implications for the way economists theorize about and analyze a number of topics central to the discipline. One is the theory of the firm in industries where technological and organizational innovation is important. Indeed a large literature has grown up on this topic, addressing the nature of the technological and organizational capabilities which business firms embody and the ways they evolve over time. Another domain concerns the nature of competition in such industries, wherein innovation and diffusion affect growth and survival probabilities of heterogeneous firms, and, relatedly, the determinants of industrial structure. The processes of knowledge accumulation and diffusion involve winners and losers, changing distributions of competitive abilities across different firms, and, with that, changing industrial structures. Both the sector-specific characteristics of technologies and their degrees of maturity over their life cycles influence the patterns of industrial organization ? including of course size distributions, degrees of concentration, relative importance of incumbents and entrants, etc. This is the second set of topics which we address. Finally, in the conclusions, we briefly flag some fundamental aspects of economic growth and development as an innovation driven evolutionary process.Innovation, Technological paradigms, Technological regimes and trajectories, Evolution, Learning, Capability-based theories of the firm, Selection, Industrial dynamics, Emergent properties, Endogenous growth

Pre-processing of tandem mass spectra using machine learning methods

Protein identification has been more helpful than before in the diagnosis and treatment of many diseases, such as cancer, heart disease and HIV. Tandem mass spectrometry is a powerful tool for protein identification. In a typical experiment, proteins are broken into small amino acid oligomers called peptides. By determining the amino acid sequence of several peptides of a protein, its whole amino acid sequence can be inferred. Therefore, peptide identification is the first step and a central issue for protein identification. Tandem mass spectrometers can produce a large number of tandem mass spectra which are used for peptide identification. Two issues should be addressed to improve the performance of current peptide identification algorithms. Firstly, nearly all spectra are noise-contaminated. As a result, the accuracy of peptide identification algorithms may suffer from the noise in spectra. Secondly, the majority of spectra are not identifiable because they are of too poor quality. Therefore, much time is wasted attempting to identify these unidentifiable spectra. The goal of this research is to design spectrum pre-processing algorithms to both speedup and improve the reliability of peptide identification from tandem mass spectra. Firstly, as a tandem mass spectrum is a one dimensional signal consisting of dozens to hundreds of peaks, and majority of peaks are noisy peaks, a spectrum denoising algorithm is proposed to remove most noisy peaks of spectra. Experimental results show that our denoising algorithm can remove about 69% of peaks which are potential noisy peaks among a spectrum. At the same time, the number of spectra that can be identified by Mascot algorithm increases by 31% and 14% for two tandem mass spectrum datasets. Next, a two-stage recursive feature elimination based on support vector machines (SVM-RFE) and a sparse logistic regression method are proposed to select the most relevant features to describe the quality of tandem mass spectra. Our methods can effectively select the most relevant features in terms of performance of classifiers trained with the different number of features. Thirdly, both supervised and unsupervised machine learning methods are used for the quality assessment of tandem mass spectra. A supervised classifier, (a support vector machine) can be trained to remove more than 90% of poor quality spectra without removing more than 10% of high quality spectra. Clustering methods such as model-based clustering are also used for quality assessment to cancel the need for a labeled training dataset and show promising results

Isometry and convexity in dimensionality reduction

The size of data generated every year follows an exponential growth. The number of data points as well as the dimensions have increased dramatically the past 15 years. The gap between the demand from the industry in data processing and the solutions provided by the machine learning community is increasing. Despite the growth in memory and computational power, advanced statistical processing on the order of gigabytes is beyond any possibility. Most sophisticated Machine Learning algorithms require at least quadratic complexity. With the current computer model architecture, algorithms with higher complexity than linear O(N) or O(N logN) are not considered practical. Dimensionality reduction is a challenging problem in machine learning. Often data represented as multidimensional points happen to have high dimensionality. It turns out that the information they carry can be expressed with much less dimensions. Moreover the reduced dimensions of the data can have better interpretability than the original ones. There is a great variety of dimensionality reduction algorithms under the theory of Manifold Learning. Most of the methods such as Isomap, Local Linear Embedding, Local Tangent Space Alignment, Diffusion Maps etc. have been extensively studied under the framework of Kernel Principal Component Analysis (KPCA). In this dissertation we study two current state of the art dimensionality reduction methods, Maximum Variance Unfolding (MVU) and Non-Negative Matrix Factorization (NMF). These two dimensionality reduction methods do not fit under the umbrella of Kernel PCA. MVU is cast as a Semidefinite Program, a modern convex nonlinear optimization algorithm, that offers more flexibility and power compared to iv KPCA. Although MVU and NMF seem to be two disconnected problems, we show that there is a connection between them. Both are special cases of a general nonlinear factorization algorithm that we developed. Two aspects of the algorithms are of particular interest: computational complexity and interpretability. In other words computational complexity answers the question of how fast we can find the best solution of MVU/NMF for large data volumes. Since we are dealing with optimization programs, we need to find the global optimum. Global optimum is strongly connected with the convexity of the problem. Interpretability is strongly connected with local isometry1 that gives meaning in relationships between data points. Another aspect of interpretability is association of data with labeled information. The contributions of this thesis are the following: 1. MVU is modified so that it can scale more efficient. Results are shown on 1 million speech datasets. Limitations of the method are highlighted. 2. An algorithm for fast computations for the furthest neighbors is presented for the first time in the literature. 3. Construction of optimal kernels for Kernel Density Estimation with modern convex programming is presented. For the first time we show that the Leave One Cross Validation (LOOCV) function is quasi-concave. 4. For the first time NMF is formulated as a convex optimization problem 5. An algorithm for the problem of Completely Positive Matrix Factorization is presented. 6. A hybrid algorithm of MVU and NMF the isoNMF is presented combining advantages of both methods. 7. The Isometric Separation Maps (ISM) a variation of MVU that contains classification information is presented. 8. Large scale nonlinear dimensional analysis on the TIMIT speech database is performed. 9. A general nonlinear factorization algorithm is presented based on sequential convex programming. Despite the efforts to scale the proposed methods up to 1 million data points in reasonable time, the gap between the industrial demand and the current state of the art is still orders of magnitude wide.Ph.D.Committee Chair: David Anderson; Committee Co-Chair: Alexander Gray; Committee Member: Anthony Yezzi; Committee Member: Hongyuan Zha; Committee Member: Justin Romberg; Committee Member: Ronald Schafe

cii Student Papers - 2021

In this collection of papers, we, the Research Group Critical Information Infrastructures (cii) from the Karlsruhe Institute of Technology, present nine selected student research articles contributing to the design, development, and evaluation of critical information infrastructures. During our courses, students mostly work in groups and deal with problems and issues related to sociotechnical challenges in the realm of (critical) information systems. Student papers came from four different cii courses, namely Emerging Trends in Digital Health, Emerging Trends in Internet Technologies, Critical Information Infrastructures, and Digital Health in the winter term of 2020 and summer term of 2021

Gender and Inclusion Toolbox: Participatory Research in Climate Change and Agriculture

This manual is a resource and toolbox for NGO practitioners and programme designers interested in diagnostic and action research for gender sensitive and socially inclusive climate change programmes in the rural development context. It is meant to be an easy to use manual, increasing the research capacity, skills and knowledge of its users. Integrating gender and social differentiation frameworks should ideally begin from the start of the programme cycle and be coordinated throughout research, design, implementation, and monitoring and evaluation phases. The data gathered using this toolbox supports this programme work. While the manual emphasizes participatory and qualitative approaches, many of the activities and tools can produce quantitative data. Each chapter features a bundle of research tools intended to be used sequentially. However, we know that each organization has its diverse needs. The chapters are in modular format so that teams can assemble their own research toolbox specific to their needs

Timely processing of big data in collaborative large-scale distributed systems

Today’s Big Data phenomenon, characterized by huge volumes of data produced at very high rates by heterogeneous and geographically dispersed sources, is fostering the employment of large-scale distributed systems in order to leverage parallelism, fault tolerance and locality awareness with the aim of delivering suitable performances. Among the several areas where Big Data is gaining increasing significance, the protection of Critical Infrastructure is one of the most strategic since it impacts on the stability and safety of entire countries. Intrusion detection mechanisms can benefit a lot from novel Big Data technologies because these allow to exploit much more information in order to sharpen the accuracy of threats discovery. A key aspect for increasing even more the amount of data at disposal for detection purposes is the collaboration (meant as information sharing) among distinct actors that share the common goal of maximizing the chances to recognize malicious activities earlier. Indeed, if an agreement can be found to share their data, they all have the possibility to definitely improve their cyber defenses. The abstraction of Semantic Room (SR) allows interested parties to form trusted and contractually regulated federations, the Semantic Rooms, for the sake of secure information sharing and processing. Another crucial point for the effectiveness of cyber protection mechanisms is the timeliness of the detection, because the sooner a threat is identified, the faster proper countermeasures can be put in place so as to confine any damage. Within this context, the contributions reported in this thesis are threefold * As a case study to show how collaboration can enhance the efficacy of security tools, we developed a novel algorithm for the detection of stealthy port scans, named R-SYN (Ranked SYN port scan detection). We implemented it in three distinct technologies, all of them integrated within an SR-compliant architecture that allows for collaboration through information sharing: (i) in a centralized Complex Event Processing (CEP) engine (Esper), (ii) in a framework for distributed event processing (Storm) and (iii) in Agilis, a novel platform for batch-oriented processing which leverages the Hadoop framework and a RAM-based storage for fast data access. Regardless of the employed technology, all the evaluations have shown that increasing the number of participants (that is, increasing the amount of input data at disposal), allows to improve the detection accuracy. The experiments made clear that a distributed approach allows for lower detection latency and for keeping up with higher input throughput, compared with a centralized one. * Distributing the computation over a set of physical nodes introduces the issue of improving the way available resources are assigned to the elaboration tasks to execute, with the aim of minimizing the time the computation takes to complete. We investigated this aspect in Storm by developing two distinct scheduling algorithms, both aimed at decreasing the average elaboration time of the single input event by decreasing the inter-node traffic. Experimental evaluations showed that these two algorithms can improve the performance up to 30%. * Computations in online processing platforms (like Esper and Storm) are run continuously, and the need of refining running computations or adding new computations, together with the need to cope with the variability of the input, requires the possibility to adapt the resource allocation at runtime, which entails a set of additional problems. Among them, the most relevant concern how to cope with incoming data and processing state while the topology is being reconfigured, and the issue of temporary reduced performance. At this aim, we also explored the alternative approach of running the computation periodically on batches of input data: although it involves a performance penalty on the elaboration latency, it allows to eliminate the great complexity of dynamic reconfigurations. We chose Hadoop as batch-oriented processing framework and we developed some strategies specific for dealing with computations based on time windows, which are very likely to be used for pattern recognition purposes, like in the case of intrusion detection. Our evaluations provided a comparison of these strategies and made evident the kind of performance that this approach can provide