Fast ML estimation for the mixture of factor analyzers via an ECM algorithm

In this brief, we propose a fast expectation conditional maximization (ECM) algorithm for maximum-likelihood (ML) estimation of mixtures of factor analyzers (MFA). Unlike the existing expectation-maximization (EM) algorithms such as the EM in Ghahramani and Hinton, 1996, and the alternating ECM (AECM) in McLachlan and Peel, 2003, where the missing data contains component-indicator vectors as well as latent factors, the missing data in our ECM consists of component-indicator vectors only. The novelty of our algorithm is that closed-form expressions in all conditional maximization (CM) steps are obtained explicitly, instead of resorting to numerical optimization methods. As revealed by experiments, the convergence of our ECM is substantially faster than EM and AECM regardless of whether assessed by central processing unit (CPU) time or number of iterations. © 2008 IEEE.published_or_final_versio

(Non) Linear Regression Modeling

We will study causal relationships of a known form between random variables. Given a model, we distinguish one or more dependent (endogenous) variables Y = (Y1, . . . , Yl), l ∈ N, which are explained by a model, and independent (exogenous, explanatory) variables X = (X1, . . . ,Xp), p ∈ N, which explain or predict the dependent variables by means of the model. Such relationships and models are commonly referred to as regression models. --

Mixed-integer linear programming for computing optimal experimental designs

We show that the optimal exact design of experiment on a finite design space can be computed via mixed-integer linear programming (MILP) for a wide class of optimality criteria, including the criteria of A-, I-, G- and MV-optimality. The key idea of the MILP formulation is the McCormick relaxation, which critically depends on finite interval bounds for the elements of the covariance matrix corresponding to an optimal exact design. We provide both analytic and algorithmic constructions of such bounds. Finally, we demonstrate some unique advantages of the MILP approach and illustrate its performance in selected experimental design settings

Development of the D-Optimality-Based Coordinate-Exchange Algorithm for an Irregular Design Space and the Mixed-Integer Nonlinear Robust Parameter Design Optimization

Robust parameter design (RPD), originally conceptualized by Taguchi, is an effective statistical design method for continuous quality improvement by incorporating product quality into the design of processes. The primary goal of RPD is to identify optimal input variable level settings with minimum process bias and variation. Because of its practicality in reducing inherent uncertainties associated with system performance across key product and process dimensions, the widespread application of RPD techniques to many engineering and science fields has resulted in significant improvements in product quality and process enhancement. There is little disagreement among researchers about Taguchi\u27s basic philosophy. In response to apparent mathematical flaws surrounding his original version of RPD, researchers have closely examined alternative approaches by incorporating well-established statistical methods, particularly the response surface methodology (RSM), while accepting the main philosophy of his RPD concepts. This particular RSM-based RPD method predominantly employs the central composite design technique with the assumption that input variables are quantitative on a continuous scale. There is a large number of practical situations in which a combination of input variables is of real-valued quantitative variables on a continuous scale and qualitative variables such as integer- and binary-valued variables. Despite the practicality of such cases in real-world engineering problems, there has been little research attempt, if any, perhaps due to mathematical hurdles in terms of inconsistencies between a design space in the experimental phase and a solution space in the optimization phase. For instance, the design space associated with the central composite design, which is perhaps known as the most effective response surface design for a second-order prediction model, is typically a bounded convex feasible set involving real numbers due to its inherent real-valued axial design points; however, its solution space may consist of integer and real values. Along the lines, this dissertation proposes RPD optimization models under three different scenarios. Given integer-valued constraints, this dissertation discusses why the Box-Behnken design is preferred over the central composite design and other three-level designs, while maintaining constant or nearly constant prediction variance, called the design rotatability, associated with a second-order model. Box-Behnken design embedded mixed integer nonlinear programming models are then proposed. As a solution method, the Karush-Kuhn-Tucker conditions are developed and the sequential quadratic integer programming technique is also used. Further, given binary-valued constraints, this dissertation investigates why neither the central composite design nor the Box-Behnken design is effective. To remedy this potential problem, several 0-1 mixed integer nonlinear programming models are proposed by laying out the foundation of a three-level factorial design with pseudo center points. For these particular models, we use standard optimization methods such as the branch-and-bound technique, the outer approximation method, and the hybrid nonlinear based branch-and-cut algorithm. Finally, there exist some special situations during the experimental phase where the situation may call for reducing the number of experimental runs or using a reduced regression model in fitting the data. Furthermore, there are special situations where the experimental design space is constrained, and therefore optimal design points should be generated. In these particular situations, traditional experimental designs may not be appropriate. D-optimal experimental designs are investigated and incorporated into nonlinear programming models, as the design region is typically irregular which may end up being a convex problem. It is believed that the research work contained in this dissertation is the initial examination in the related literature and makes a considerable contribution to an existing body of knowledge by filling research gaps