Analysis of Models for Longitudinal and Clustered Binary Data

This dissertation deals with modeling and statistical analysis of longitudinal and clustered binary data. Such data consists of observations on a dichotomous response variable generated from multiple time or cluster points, that exhibit either decaying correlation or equi-correlated dependence. The current literature addresses modeling the dependence using an appropriate correlation structure, but ignores the feasible bounds on the correlation parameter imposed by the marginal means. The first part of this dissertation deals with two multivariate probability models, the first order Markov chain model and the multivariate probit model, that adhere to the feasible bounds on the correlation. For both the models we obtain maximum likelihood estimates for the regression and correlation parameters, and study both asymptotic and small-sample properties of the estimates. Through simulations we compare the efficiency of the two methods and demonstrate that neither is uniformly superior over the other. The second part of this dissertation deals with marginal models, an alternative to multivariate probability models. We discuss the generalized estimating equations and the quadratic inference function methods for estimating the regression parameter in marginal models. Relative efficiency calculations show these methods when compared to the likelihood estimates could result in significant loss in efficiency for highly correlated data. We also propose a modified quadratic inference function method and demonstrate through efficiency calculations this is an improvement of the original quadratic inference function approach. The final part of this dissertation deals with methods for constructing higher order Markov chain models using copulas

Semistochastic Quadratic Bound Methods

Partition functions arise in a variety of settings, including conditional random fields, logistic regression, and latent gaussian models. In this paper, we consider semistochastic quadratic bound (SQB) methods for maximum likelihood inference based on partition function optimization. Batch methods based on the quadratic bound were recently proposed for this class of problems, and performed favorably in comparison to state-of-the-art techniques. Semistochastic methods fall in between batch algorithms, which use all the data, and stochastic gradient type methods, which use small random selections at each iteration. We build semistochastic quadratic bound-based methods, and prove both global convergence (to a stationary point) under very weak assumptions, and linear convergence rate under stronger assumptions on the objective. To make the proposed methods faster and more stable, we consider inexact subproblem minimization and batch-size selection schemes. The efficacy of SQB methods is demonstrated via comparison with several state-of-the-art techniques on commonly used datasets.Comment: 11 pages, 1 figur

Bankruptcy Prediction of Small and Medium Enterprises Using a Flexible Binary Generalized Extreme Value Model

We introduce a binary regression accounting-based model for bankruptcy prediction of small and medium enterprises (SMEs). The main advantage of the model lies in its predictive performance in identifying defaulted SMEs. Another advantage, which is especially relevant for banks, is that the relationship between the accounting characteristics of SMEs and response is not assumed a priori (e.g., linear, quadratic or cubic) and can be determined from the data. The proposed approach uses the quantile function of the generalized extreme value distribution as link function as well as smooth functions of accounting characteristics to flexibly model covariate effects. Therefore, the usual assumptions in scoring models of symmetric link function and linear or pre-specied covariate-response relationships are relaxed. Out-of-sample and out-of-time validation on Italian data shows that our proposal outperforms the commonly used (logistic) scoring model for different default horizons

Various Distributional Results For Ratios Of Quadratic Forms With Applications To Serial Correlation

Quadratic forms in normal vectors are central building blocks in statistics, and ratios of quadratic forms arise in a variety of contexts. They are connected, for example, to regression and analysis of variance problems associated with linear models and to correlation analysis.;The matrices of the quadratic forms may be for example positive definite, positive semidefinite or indefinite; the normal vectors may be central or noncentral, nonsingular or singular. All these cases are considered for a single quadratic form as well as for the ratio of two quadratic forms. The distributional results also apply to bilinear forms, sums of quadratic and bilinear forms and the ratios thereof.;The inverse Mellin transform technique is used to obtain a representation of the density function of a nonnegative definite quadratic form. The density function of an indefinite quadratic form is then obtained in terms of Whittaker\u27s function. Approximate density functions based on Patnaik and Pearson\u27s approximations are also proposed. Computable expressions are then derived for the corresponding distribution functions. Three techniques are proposed for determining the distribution of ratios of quadratic forms: (1) the problem is transformed so that the results obtained for indefinite quadratic forms may be used; (2) the inverse Mellin transform technique is applied directly to the ratio in order to obtain the exact density function in terms of generalized hypergeometric functions; these techniques apply to ratios of quadratic forms which are not necessarily independently distributed; the moments which are used to approximate the distribution are also given; (3) the density function is obtained by differentiation; this approach requires the independence of the quadratic forms. These theoretical results are illustrated via several examples, and then corroborated by simulations. For the lag-k serial correlation coefficient, the exact distribution function and an approximation are obtained. The cumulants of the serial covariance are derived in two ways: by introducing a special operator and by solving a system of second order difference equations

Demand Forecasting for Food Production Using Machine Learning Algorithms: A Case Study of University Refectory

Accurate food demand forecasting is one of the critical aspects of successfully managing restaurants, cafeterias, canteens, and refectories. This paper aims to develop demand forecasting models for a university refectory. Our study focused on the development of Machine Learning-based forecasting models which take into account the calendar effect and meal ingredients to predict the heavy demand for food within a limited timeframe (e.g., lunch) and without pre-booking. We have developed eighteen prediction models gathered under five main techniques. Three Artificial Neural Network models (i.e., Feed Forward, Function Fitting, and Cascade Forward), four Gauss Process Regression models (i.e., Rational Quadratic, Squared Exponential, Matern 5/2, and Exponential), six Support Vector Regression models (i.e., Linear, Quadratic, Cubic, Fine Gaussian, Medium Gaussian, and Coarse Gaussian), three Regression Tree models (i.e., Fine, Medium, and Coarse), two Ensemble Decision Tree (EDT) models (i.e., Boosted and Bagged) and one Linear Regression model were applied. When evaluated in terms of method diversity, prediction performance, and application area, to the best of our knowledge, this study offers a different contribution from previous studies. The EDT Boosted model obtained the best prediction performance (i.e., Mean Squared Error = 0,51, Mean Absolute Erro = 0,50, and R = 0,96)

Non-linear projection to latent structures

PhD ThesisThis Thesis focuses on the study of multivariate statistical regression techniques which have been used to produce non-linear empirical models of chemical processes, and on the development of a novel approach to non-linear Projection to Latent Structures regression. Empirical modelling relies on the availability of process data and sound empirical regression techniques which can handle variable collinearities, measurement noise, unknown variable and noise distributions and high data set dimensionality. Projection based techniques, such as Principal Component Analysis (PCA) and Projection to Latent Structures (PLS), have been shown to be appropriate for handling such data sets. The multivariate statistical projection based techniques of PCA and linear PLS are described in detail, highlighting the benefits which can be gained by using these approaches. However, many chemical processes exhibit severely nonlinear behaviour and non-linear regression techniques are required to develop empirical models. The derivation of an existing quadratic PLS algorithm is described in detail. The procedure for updating the model parameters which is required by the quadratic PLS algorithms is explored and modified. A new procedure for updating the model parameters is presented and is shown to perform better the existing algorithm. The two procedures have been evaluated on the basis of the performance of the corresponding quadratic PLS algorithms in modelling data generated with a strongly non-linear mathematical function and data generated with a mechanistic model of a benchmark pH neutralisation system. Finally a novel approach to non-linear PLS modelling is then presented combining the general approximation properties of sigmoid neural networks and radial basis function networks with the new weights updating procedure within the PLS framework. These algorithms are shown to outperform existing neural network PLS algorithms and the quadratic PLS approaches. The new neural network PLS algorithms have been evaluated on the basis of their performance in modelling the same data used to compare the quadratic PLS approaches.Strang Studentship European project ESPRIT PROJECT 22281 (PROGNOSIS) Centre for Process Analysis, Chemometrics and Control