Stacking-based Deep Neural Network: Deep Analytic Network on Convolutional Spectral Histogram Features

Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.Comment: 5 page

Improving Face Recognition Performance Using a Hierarchical Bayesian Model

Over the past two decades, face recognition research has shot to the forefront due to its increased demand in security and commercial applications. Many facial feature extraction techniques for the purpose of recognition have been developed, some of which have also been successfully installed and used. Principal Component Analysis (PCA), also popularly called as Eigenfaces has been used successfully and also is a de facto standard. Linear generative models such as Principal Component Analysis (PCA) and Independent Component Analysis (ICA) find a set of basis images and represent the faces as a linear combination of these basis functions. These models make certain assumptions about the data which limit the type of structure they can capture. This thesis is mainly based on the hierarchical Bayesian model developed by Yan Karklin of Carnegie Mellon University. His research was mainly focused on natural signals like natural images and speech signals in which he showed that for such signals, latent variables exhibit residual dependencies and non-stationary statistics. He built his model atop ICA and this hierarchical model could capture more abstract and invariant properties of the data. We apply the same hierarchical model on facial images to extract features which can result in an improved recognition performance over already existing baseline approaches. We use Kernelized Fisher Discriminant Analysis (KFLD) as our baseline as it is superior to PCA in a way that it produces well separated classes even under variations in facial expression and lighting. We conducted extensive experiments on the GreyFERET database and tested the performance on test sets with varying facial expressions. The results demonstrate the increase in performance that was expected

Learning Stable and Robust Linear Parameter-Varying State-Space Models

This paper presents two direct parameterizations of stable and robust linear parameter-varying state-space (LPV-SS) models. The model parametrizations guarantee a priori that for all parameter values during training, the allowed models are stable in the contraction sense or have their Lipschitz constant bounded by a user-defined value γ . Furthermore, since the parametrizations are direct, the models can be trained using unconstrained optimization. The fact that the trained models are of the LPV-SS class makes them useful for, e.g., further convex analysis or controller design. The effectiveness of the approach is demonstrated on an LPV identification problem

Learning Stable and Robust Linear Parameter-Varying State-Space Models

This paper presents two direct parameterizations of stable and robust linear parameter-varying state-space (LPV-SS) models. The model parametrizations guarantee a priori that for all parameter values during training, the allowed models are stable in the contraction sense or have their Lipschitz constant bounded by a user-defined value γ . Furthermore, since the parametrizations are direct, the models can be trained using unconstrained optimization. The fact that the trained models are of the LPV-SS class makes them useful for, e.g., further convex analysis or controller design. The effectiveness of the approach is demonstrated on an LPV identification problem

Learning Stable and Robust Linear Parameter-Varying State-Space Models

This paper presents two direct parameterizations of stable and robust linear parameter-varying state-space (LPV-SS) models. The model parametrizations guarantee a priori that for all parameter values during training, the allowed models are stable in the contraction sense or have their Lipschitz constant bounded by a user-defined value $\gamma$. Furthermore, since the parametrizations are direct, the models can be trained using unconstrained optimization. The fact that the trained models are of the LPV-SS class makes them useful for, e.g., further convex analysis or controller design. The effectiveness of the approach is demonstrated on an LPV identification problem.Comment: Accepted for the 62nd IEEE Conference on Decision and Control (CDC2023

Algorithmic Advances for the Design and Analysis of Randomized Experiments

Randomized experiments are the gold standard for investigating the causal effect of treatment on a population. In this dissertation, we present algorithmic advances for three different problems arising in the design and analysis of randomized experiments: covariate balancing, variance estimation, and bipartite experiments. In the first chapter, we describe an inherent trade-off between covariate balancing and robustness, which we formulate as a distributional discrepancy problem. In order to navigate this trade-off, we present the Gram–Schmidt Walk Design which is based on the recent discrepancy algorithm of Bansal, Dadush, Garg, and Lovett (2019). By tightening the algorithmic analysis, we derive bounds on the mean squared error of the Horvitz–Thompson estimator under this design in terms of a ridge regression of the outcomes on the covariates, which we interpret as regression by design. We carry out further analysis including tail bounds on effect estimator, methods for constructing confidence intervals, and an extension of the design which accommodates non-linear responses via kernel methods. In the second chapter, we study the problem of estimating the variance of treat- ment effect estimators under interference. It is well-known that unbiased variance estimation is impossible without strong assumptions on the outcomes, due to the fundamental problem of causal inference. Thus, we study a class of conservative es- timators which are based on variance bounds. We identify conditions under which the variance bounds themselves are admissible and provide a general algorithmic framework to construct admissible variance bounds, according to the experimenter’s preferences and prior substantive knowledge. In the final chapter, we present methodology for the newly proposed bipartite experimental framework, where units which receive treatment are distinct from units on which outcomes are measured, and the two are connected via a bipartite graph. We investigate a linear exposure-response assumption which allows more complex interactions. We propose the Exposure Re-weighted Linear (ERL) estimator which we show is unbiased in finite samples and consistent and asymptotically normal in large samples provided the bipartite graph is sufficiently sparse. We provide a variance estimator which facilitates confidence intervals based on the normal approximation. Finally, we present Exposure-Design, a correlation clustering based design for improving precision of the ERL estimator