Improved integration of information to reduce subsurface model bias

Subsurface modeling deals with data-related issues like cognitive and sampling biases, and model-related challenges including statistical assumptions, misspecification, and algorithmic biases. These challenges introduce four critical implications during subsurface modeling. Firstly, subsurface sampling is subject to sampling bias, which compromises statistical representativeness. Secondly, analog selection methodologies rely on multivariate statistics and expert judgment that overlook spatial information and data dimensionality. Thirdly, subsurface inferential workflows that utilize dimensionality reduction seldom provide repeatable frameworks that maintain model stability and are invariant to Euclidean transformations. Lastly, deep learning methods for dimensionality reduction, characterized as black-box models, lack interpretability and robust evaluation metrics, increasing susceptibility to algorithmic bias. Consequently, neglecting these challenges in subsurface modeling could lead to erroneous predictions, inconsistent inferences, diminished model reliability, and suboptimal decision-making that impacts project economics. This dissertation integrates information within subsurface models to reduce model bias and significantly improve their accuracy, robustness, and generalizability. First, I create spatial declustering methods to debias spatial datasets with single and multiscale preferential sampling in stationary populations. Second, I introduce a novel geostatistics-based machine learning method for identifying subsurface resource analogs that integrate spatial information in subsurface datasets with high dimensionality. Next, I efficiently combine machine learning and computational geometry methods to stabilize lower dimensional spaces for uncertainty quantification and interpretation. Finally, I create a methodology to assess, evaluate, and interpret the stability of deep learning latent feature spaces. These novel methodologies demonstrate the importance of improved techniques for information integration in subsurface modeling and show better results over naïve methods. This results in objective sampling debiasing in spatial stationary populations with single or multiple data scales, improving statistical representativity. Also, the results show better generalization and accurate identification of spatial analogs in high-dimensional datasets. Moreover, the methods yield Euclidean transformation-invariant lower-dimensional spaces, ensuring unique and repeatable solutions that improve model reliability and interpretability, for rational comparisons. Finally, the results indicate that deep learning models for dimensionality reduction exhibit algorithmic biases and instabilities, including sample, structural, and inferential instability, affecting their reliability and interpretability. Together, these innovations ultimately reduce model bias and significantly improve subsurface modeling.Petroleum and Geosystems Engineerin

Clustering techniques for base station coordination in a wireless cellular system

A lo largo de este Proyecto Fin de Carrera, propondremos mejoras para futuros sistemas de comunicaciones móviles mediante un estudio detallado de la coordinación entre estaciones base en sistemas celulares basados en MIMO. Este proyecto se compone de dos partes fundamentales. Por un lado, nos centraremos en técnicas de procesado de señal para MIMO como filtrado y precodificación lineales en el dominio espacial. Partiendo de los últimos desarrollos en dicho ámbito, se han desarrollado precodificadores de mínimo error cuadrático medio que incluyen restricciones de máxima potencia transmitida por celda. Además, se ha propuesto un concepto novedoso consistente en la introducción de una nueva formulación que, además de minimizar el error cuadrático medio en el interior de cada agrupación de celdas (cluster ), trata de mantener la interferencia entre clusters en niveles suficientemente bajos. Durante la segunda parte, analizaremos el impacto que la agrupación de celdas en clusters, que define qué estaciones base pueden ser coordinadas entre sí , tiene en el rendimiento global del sistema. Se ha estudiado la aplicabilidad de técnicas de agrupamiento dentro del aprendizaje máquina, dando como resultado un conjunto de nuevos algoritmos que han sido desarrollados adaptando algoritmos de agrupamiento de propósito general ya existentes al problema de crear una partición del conjunto de celdas de acuerdo a las condiciones de propagación de señal existentes en el sistema en un determinado instante. Todas nuestras contribuciones se han verificado mediante la simulación de un sistema de comunicaciones móviles basado en modelos de propagación de señal del 3GPP para LTE. De acuerdo a los resultados obtenidos, las técnicas propuestas a lo largo de este proyecto proporcionan un aumento considerable de la media y la mediana de las tasas por usuario respecto a soluciones ya existentes. La idea de introducir la reducción de interferencia entre clusters en la formulación de los precodifiadores MMSE mejora dramáticamente el rendimiento en sistemas celulares MIMO al ser comparados con precodifiadores de Wiener tradicionales. Por otro lado, nuestros algoritmos de agrupamiento dinámico de estaciones base exhiben un notable aumento de las tasas por usuario a la vez que emplean clusters de menor tamaño con respecto a soluciones existentes basadas en particiones estáticas del conjunto de celdas en el sistema. _______________________________________________________________________________________________________________________________In this project, we attempt to provide enhancements for future mobile communications systems by carrying out a throughout study of base-station coordination in cellular MIMO systems. Our work can be divided in two main blocks. During the first part, we focus our attention on linear MIMO signal processing techniques such as linear spatial precoding and linear spatial ltering. Starting from the state-of-the-art in that area of knowledge, we have developed novel MMSE precoders which include per-cell power constraints and a new formulation which, apart from minimizing the intra-cluster MSE, tries to keep inter-cluster interference at low levels. In the second part, we focus on the study of the impact the particular mapping of cells to clusters in the cellular system has on the overall performance of the mobile communication radio access network. The applicability of existing clustering algorithms in the fi eld of machine learning has been studied, resulting in a set of novel algorithms that we developed by adapting existing general-purpose clustering solutions for the problem of dynamically partitioning a set of cells according to the instantaneous signal propagation conditions. All our contributions have been exhaustively tested by simulation of a cellular mobile communication system based on 3GPP signal propagation models for LTE. According to the results obtained, the techniques proposed along this project provide a remarkable increase of both the average and median user rates in the system with respect to previous existing solutions. The inter-cluster interference-awareness we introduced in the formulation of MMSE precoders dramatically increases the performance in cellular coordinated MIMO when comparing it with traditional Wiener precoders. On the other hand, our dynamic base-station clustering has been shown to signi catively enhance the user rates while using smaller clusters that existing solutions based on static partitions of the base-station deployment.Ingeniería de Telecomunicació

Improving Representation Learning for Deep Clustering and Few-shot Learning

The amounts of data in the world have increased dramatically in recent years, and it is quickly becoming infeasible for humans to label all these data. It is therefore crucial that modern machine learning systems can operate with few or no labels. The introduction of deep learning and deep neural networks has led to impressive advancements in several areas of machine learning. These advancements are largely due to the unprecedented ability of deep neural networks to learn powerful representations from a wide range of complex input signals. This ability is especially important when labeled data is limited, as the absence of a strong supervisory signal forces models to rely more on intrinsic properties of the data and its representations. This thesis focuses on two key concepts in deep learning with few or no labels. First, we aim to improve representation quality in deep clustering - both for single-view and multi-view data. Current models for deep clustering face challenges related to properly representing semantic similarities, which is crucial for the models to discover meaningful clusterings. This is especially challenging with multi-view data, since the information required for successful clustering might be scattered across many views. Second, we focus on few-shot learning, and how geometrical properties of representations influence few-shot classification performance. We find that a large number of recent methods for few-shot learning embed representations on the hypersphere. Hence, we seek to understand what makes the hypersphere a particularly suitable embedding space for few-shot learning. Our work on single-view deep clustering addresses the susceptibility of deep clustering models to find trivial solutions with non-meaningful representations. To address this issue, we present a new auxiliary objective that - when compared to the popular autoencoder-based approach - better aligns with the main clustering objective, resulting in improved clustering performance. Similarly, our work on multi-view clustering focuses on how representations can be learned from multi-view data, in order to make the representations suitable for the clustering objective. Where recent methods for deep multi-view clustering have focused on aligning view-specific representations, we find that this alignment procedure might actually be detrimental to representation quality. We investigate the effects of representation alignment, and provide novel insights on when alignment is beneficial, and when it is not. Based on our findings, we present several new methods for deep multi-view clustering - both alignment and non-alignment-based - that out-perform current state-of-the-art methods. Our first work on few-shot learning aims to tackle the hubness problem, which has been shown to have negative effects on few-shot classification performance. To this end, we present two new methods to embed representations on the hypersphere for few-shot learning. Further, we provide both theoretical and experimental evidence indicating that embedding representations as uniformly as possible on the hypersphere reduces hubness, and improves classification accuracy. Furthermore, based on our findings on hyperspherical embeddings for few-shot learning, we seek to improve the understanding of representation norms. In particular, we ask what type of information the norm carries, and why it is often beneficial to discard the norm in classification models. We answer this question by presenting a novel hypothesis on the relationship between representation norm and the number of a certain class of objects in the image. We then analyze our hypothesis both theoretically and experimentally, presenting promising results that corroborate the hypothesis

Automated Detection of Anomalous Patterns in Validation Scores for Protein X-Ray Structure Models

Structural bioinformatics is a subdomain of data mining focused on identifying structural patterns relevant to functional attributes in repositories of biological macromolecular structure models. This research focused on structures determined via x-ray crystallography and deposited in the Protein Data Bank (PDB). Protein structures deposited in the PDB are products of experimental processes, and only approximately model physical reality. Structural biologists address accuracy and precision concerns via community-enforced consensus standards of accepted practice for proper building, refinement, and validation of models. Validation scores are quantitative partial indicators of the likelihood that a model contains serious systematic errors. The PDB recently convened a panel of experts, which placed renewed emphasis on troubling anomalies among deposited structure models. This study set out to detect such anomalies. I hypothesized that community consensus standards would be evident in patterns of validation scores, and deviations from those standards would appear as unusual combinations of validation scores. Validation attributes were extracted from PDB entry headers and multiple software tools (e.g., WhatCheck, SFCheck, and MolProbity). Independent component analysis (ICA) was used for attribute transformation to increase contrast between inliers and outliers. Unusual patterns were sought in regions of locally low density in the space of validation score profiles, using a novel standardization of Local Outlier Factor (LOF) scores. Validation score profiles associated with the most extreme outlier scores were demonstrably anomalous according to domain theory. Among these were documented fabrications, possible annotation errors, and complications in the underlying experimental data. Analysis of deep inliers revealed promising support for the hypothesized link between consensus standard practices and common validation score values. Unfortunately, with numerical anomaly detection methods that operate simultaneously on numerous continuous-valued attributes, it is often quite difficult to know why a case gets a particular outlier score. Therefore, I hypothesized that IF-THEN rules could be used to post-process outlier scores to make them comprehensible and explainable. Inductive rule extraction was performed using RIPPER. Results were mixed, but they represent a promising proof of concept. The methods explored are general and applicable beyond this problem. Indeed, they could be used to detect structural anomalies using physical attributes

Model-based deep autoencoders for characterizing discrete data with application to genomic data analysis

Deep learning techniques have achieved tremendous successes in a wide range of real applications in recent years. For dimension reduction, deep neural networks (DNNs) provide a natural choice to parameterize a non-linear transforming function that maps the original high dimensional data to a lower dimensional latent space. Autoencoder is a kind of DNNs used to learn efficient feature representation in an unsupervised manner. Deep autoencoder has been widely explored and applied to analysis of continuous data, while it is understudied for characterizing discrete data. This dissertation focuses on developing model-based deep autoencoders for modeling discrete data. A motivating example of discrete data is the count data matrix generated by single-cell RNA sequencing (scRNA-seq) technology which is widely used in biological and medical fields. scRNA-seq promises to provide higher resolution of cellular differences than bulk RNA sequencing and has helped researchers to better understand complex biological questions. The recent advances in sequencing technology have enabled a dramatic increase in the throughput to thousands of cells for scRNA-seq. However, analysis of scRNA-seq data remains a statistical and computational challenge. A major problem is the pervasive dropout events obscuring the discrete matrix with prevailing \u27false\u27 zero count observations, which is caused by the shallow sequencing depth per cell. To make downstream analysis more effective, imputation, which recovers the missing values, is often conducted as the first step in preprocessing scRNA-seq data. Several imputation methods have been proposed. Of note is a deep autoencoder model, which proposes to explicitly characterize the count distribution, over-dispersion, and sparsity of scRNA-seq data using a zero-inflated negative binomial (ZINB) model. This dissertation introduces a model-based deep learning clustering model ? scDeepCluster for clustering analysis of scRNA-seq data. The scDeepCluster is a deep autoencoder which simultaneously learns feature representation and clustering via explicit modeling of scRNA-seq data generation using the ZINB model. Based on testing extensive simulated datasets and real datasets from different representative single-cell sequencing platforms, scDeepCluster outperformed several state-of-the-art methods under various clustering performance metrics and exhibited improved scalability, with running time increasing linearly with the sample size. Although this model-based deep autoencoder approach has demonstrated superior performance, it is over-permissive in defining ZINB model space, which can lead to an unidentifiable model and make results unstable. Next, this dissertation proposes to impose a regularization that takes dropout events into account. The regularization uses a differentiable categorical distribution - Gumbel-Softmax to explicitly model the dropout events, and minimizes the Maximum Mean Discrepancy (MMD) between the reconstructed randomly masked matrix and the raw count matrix. Imputation analyses showed that the proposed regularized model-based autoencoder significantly outperformed the vanilla model-based deep autoencoder

Proceedings. 23. Workshop Computational Intelligence, Dortmund, 5. - 6. Dezember 2013

Dieser Tagungsband enthält die Beiträge des 23. Workshops Computational Intelligence des Fachausschusses 5.14 der VDI/VDE-Gesellschaft für Mess- und Automatisierungstechnik (GMA), der vom 5. - 6. Dezember 2013 in Dortmund stattgefunden hat. Im Fokus stehen Methoden, Anwendungen und Tools für Fuzzy-Systeme, Künstliche Neuronale Netze, Evolutionäre Algorithmen und Data-Mining-Verfahren

Statistical Methods for Conservation and Alignment Quality in Proteins

Construction of multiple sequence alignments is a fundamental task in Bioinformatics. Multiple sequence alignments are used as a prerequisite in many Bioinformatics methods, and subsequently the quality of such methods can be critically dependent on the quality of the alignment. However, automatic construction of a multiple sequence alignment for a set of remotely related sequences does not always provide biologically relevant alignments.Therefore, there is a need for an objective approach for evaluating the quality of automatically aligned sequences. The profile hidden Markov model is a powerful approach in comparative genomics. In the profile hidden Markov model, the symbol probabilities are estimated at each conserved alignment position. This can increase the dimension of parameter space and cause an overfitting problem. These two research problems are both related to conservation. We have developed statistical measures for quantifying the conservation of multiple sequence alignments. Two types of methods are considered, those identifying conserved residues in an alignment position, and those calculating positional conservation scores. The positional conservation score was exploited in a statistical prediction model for assessing the quality of multiple sequence alignments. The residue conservation score was used as part of the emission probability estimation method proposed for profile hidden Markov models. The results of the predicted alignment quality score highly correlated with the correct alignment quality scores, indicating that our method is reliable for assessing the quality of any multiple sequence alignment. The comparison of the emission probability estimation method with the maximum likelihood method showed that the number of estimated parameters in the model was dramatically decreased, while the same level of accuracy was maintained. To conclude, we have shown that conservation can be successfully used in the statistical model for alignment quality assessment and in the estimation of emission probabilities in the profile hidden Markov models.Siirretty Doriast

Software Development Effort Estimation Using Regression Fuzzy Models

Software effort estimation plays a critical role in project management. Erroneous results may lead to overestimating or underestimating effort, which can have catastrophic consequences on project resources. Machine-learning techniques are increasingly popular in the field. Fuzzy logic models, in particular, are widely used to deal with imprecise and inaccurate data. The main goal of this research was to design and compare three different fuzzy logic models for predicting software estimation effort: Mamdani, Sugeno with constant output and Sugeno with linear output. To assist in the design of the fuzzy logic models, we conducted regression analysis, an approach we call regression fuzzy logic. State-of-the-art and unbiased performance evaluation criteria such as standardized accuracy, effect size and mean balanced relative error were used to evaluate the models, as well as statistical tests. Models were trained and tested using industrial projects from the International Software Benchmarking Standards Group (ISBSG) dataset. Results showed that data heteroscedasticity affected model performance. Fuzzy logic models were found to be very sensitive to outliers. We concluded that when regression analysis was used to design the model, the Sugeno fuzzy inference system with linear output outperformed the other models.Comment: This paper has been accepted in January 2019 in Computational Intelligence and Neuroscience Journal (In Press

Tulkittavia menetelmiä ja työkaluja bayesiläiseen mallinvalintaan

This thesis discusses interpretability in model selection. It considers some of the central themes of interpretable models and introduces a new tool, shinyproj, to improve interpretability in variable selection. shinyproj is a new R package for interpretable Bayesian model selection for generalised linear models. shinyproj emphasises a modern workflow for variable selection, in which the properties of the models are examined iteratively with a guidance of an efficient variable selection algorithm. The need for the package is motivated especially by the increasing demands for transparent and interpretable models, which are also discussed in this thesis. The problem is that in order to increase the performance of the model, one often has to increase the complexity of the model, which in turn will often reduce the interpretability of the model. shinyproj combines an existing R package for projection predictive variable selection with an interface that allows the user to explore the model space and make informed and efficient tradeoffs between the accuracy and the interpretability of the model. While the current functionality of the package does not constitute a conclusive solution to the problem, it serves as a proof-of-concept and likely a good basis for future improvements.Tämä työ tarkastelee tulkittavaa bayesiläistä mallinvalintaa. Yhtäältä työssä tarkastellaan tekijöitä, jotka tekevät malleista tulkittavia, mutta toisaalta työssä esitetään myös uusi työkalu, shinyproj, joka tekee lisäksi itse mallinvalintaprosessista ymmärrettävän. shinyproj on uusi R paketti tulkittavaan bayesiläiseen mallinvalintaan yleisestetyille lineaarimalleille (generalized linear models). shinyproj korostaa moderneja työskentelytapoja, jossa mallin ominaisuuksia tarkastellaan iteratiivisesti tehokkaan muuttujanvalinta-algoritmin tuella. Tulkittavien ja ymmärrettävien mallien tarve on noussut erityisesti viime aikoina, kun yhtäältä malleja käytetään enemmän ja enemmän osana päätöksentekoa, mutta toisaalta juuri siitä syystä malleilta vaaditaan myös läpinäkyvyyttä ja tulkittavuutta. Ongelmana on pohjimmiltaan se, että mitä paremmin mallin halutaan suoriutuvan, sitä monipuolisempi ja yksityiskohtaisempi sen on vääjäämättä oletava. Monipuolisuus ja yksityiskohtaisuus taas tekevät mallista väkisinkin vaikeammin tulkittavan ja ymmärrettävän. shinyproj yhdistää olemassaolevan tehokkaan parametrien projisoimiseen perustuvan muuttujanvalinta-paketin yksinkertaiseen graafiseen käyttöliittymään, joka helpottaa malli-avaruuden läpikäymistä ja siten mahdollistaa informoitujen ja tehokkaiden vaihtokauppojen tekemisen mallin suorituskyvyn ja tulkittavuuden välillä. Vaikka nykyisellään paketti ei ratkaisekkaan tyhjentävästi kaikkia tulkittavaan mallinvalintaan liittyviä ongelmia, se tarjoaa siihen yhden käyttökelpoisen ratkaisun ja toimii esimerkkinä siitä, minkälaisia ratkaisuja ongelmaan voi tulevaisuudessa tarjota