17 research outputs found
Probabilistic Modeling of Process Systems with Application to Risk Assessment and Fault Detection
Three new methods of joint probability estimation (modeling), a maximum-likelihood maximum-entropy method, a constrained maximum-entropy method, and a copula-based method called the rolling pin (RP) method, were developed. Compared to many existing probabilistic modeling methods such as Bayesian networks and copulas, the developed methods yield models that have better performance in terms of flexibility, interpretability and computational tractability. These methods can be used readily to model process systems and perform risk analysis and fault detection at steady state conditions, and can be coupled with appropriate mathematical tools to develop dynamic probabilistic models. Also, a method of performing probabilistic inference using RP-estimated joint probability distributions was introduced; this method is superior to Bayesian networks in several aspects. The RP method was also applied successfully to identify regression models that have high level of flexibility and are appealing in terms of computational costs.Ph.D., Chemical Engineering -- Drexel University, 201
Understanding Data Manipulation and How to Leverage it To Improve Generalization
Augmentations and other transformations of data, either in the input or latent space, are a critical component of modern machine learning systems. While these techniques are widely used in practice and known to provide improved generalization in many cases, it is still unclear how data manipulation impacts learning and generalization. To take a step toward addressing the problem, this thesis focuses on understanding and leveraging data augmentation and alignment for improving machine learning performance and transfer. In the first part of the thesis, we establish a novel theoretical framework to understand how data augmentation (DA) impacts learning in linear regression and classification tasks. The results demonstrate how the augmented transformed data spectrum plays a key role in characterizing the behavior of different augmentation strategies, especially in the overparameterized regime. The tools developed in this aim provide simple guidelines to build new augmentation strategies and a simple framework for comparing the generalization of different types of DA. In the second part of the thesis, we demonstrate how latent data alignment can be used to tackle the domain transfer problem, where training and testing datasets vary in distribution. Our algorithm builds upon joint clustering and data-matching through optimal transport, and outperforms the pure matching algorithm baselines in both synthetic and real datasets. Extension of the generalization analysis and algorithm design for data augmentation and alignment for nonlinear models such as artificial neural networks and random feature models are discussed. This thesis provides tools and analyses for better data manipulation design, which benefit both supervised and unsupervised learning schemes.Ph.D
Conditional Quantile Processes based on Series or Many Regressors
Quantile regression (QR) is a principal regression method for analyzing the
impact of covariates on outcomes. The impact is described by the conditional
quantile function and its functionals. In this paper we develop the
nonparametric QR-series framework, covering many regressors as a special case,
for performing inference on the entire conditional quantile function and its
linear functionals. In this framework, we approximate the entire conditional
quantile function by a linear combination of series terms with
quantile-specific coefficients and estimate the function-valued coefficients
from the data. We develop large sample theory for the QR-series coefficient
process, namely we obtain uniform strong approximations to the QR-series
coefficient process by conditionally pivotal and Gaussian processes. Based on
these strong approximations, or couplings, we develop four resampling methods
(pivotal, gradient bootstrap, Gaussian, and weighted bootstrap) that can be
used for inference on the entire QR-series coefficient function.
We apply these results to obtain estimation and inference methods for linear
functionals of the conditional quantile function, such as the conditional
quantile function itself, its partial derivatives, average partial derivatives,
and conditional average partial derivatives. Specifically, we obtain uniform
rates of convergence and show how to use the four resampling methods mentioned
above for inference on the functionals. All of the above results are for
function-valued parameters, holding uniformly in both the quantile index and
the covariate value, and covering the pointwise case as a by-product. We
demonstrate the practical utility of these results with an example, where we
estimate the price elasticity function and test the Slutsky condition of the
individual demand for gasoline, as indexed by the individual unobserved
propensity for gasoline consumption.Comment: 131 pages, 2 tables, 4 figure