Contributions to the planning and analysis of factorial designs

The thesis is structured in five articles. In the first article, two methods are compared to analyze the significance of the effects: the Lenth method and the one based on the estimation of the variance of the effects from interactions that can be considered negligible from scratch. For the most common factorial designs and in a set of scenarios that seek to reflect the situations that the experimenter can find in practice, simulation techniques are used to identify the errors that are committed with each method. Based on the analysis of the results obtained, we recommend in which situations it is more appropriate to use one method or the other. The second article analyzes the problem of estimating the results of experiments that could not be performed based on the expression of the interactions that can be considered negligible. The variance of the estimated values depends on what these values are and also on the interactions considered negligible. All possible encountered situations are analyzed and tables are presented with the values that can be estimated with minimum variance depending on the type of design and the contrasts available to perform the estimates. The third article deals with the same problem as the second but analyzing the impact of the estimate not on the variance of the estimated response values but on the variance of the effects and also on the correlations among them. The analysis of all the situations that can be given in the most common designs, allows us to make recommendations about what experiments should be skipped in the case that, due to time or budgetary constraints, all runs indicated by the factorial design cannot be executed.La tesi està estructurada en cinc articles. En el primer es comparen dos mètodes per analitzar la significació dels efectes: el mètode de Lenth i el basat en l’estimació de la variància dels efectes a partir de les interacciones que a priori es poden considerar nul·les. Pels dissenys factorials més habituals i en un conjunt d’escenaris que pretenen reflectir les situacions que l’experimentador es pots trobar a la pràctica, es fan servir tècniques de simulació per identificar els errors que es cometen amb cada mètode. A partir de l’anàlisi dels resultats obtinguts es recomana en quina situació és més adequat fer servir un mètode o l’altre. El segon article analitza el problema d’estimar els resultats d’experiments que no s’han pogut realitzar a partir de l’expressió de les interaccions que es poden considerar negligibles. La variància dels valors estimats depèn de quins siguin aquests valors i també de les interaccions considerades negligibles. S’analitzen totes les situacions que es poden donar i es presenten unes taules amb els valors que es poden estimar amb variància mínima segons el tipus de disseny i de quins siguin els contrastos disponibles per fer les estimacions. El tercer article aborda el mateix problema que el segon però analitzant l’impacte de l’estimació no en la variància dels valors estimats sinó en la variància dels efectes i també en les correlacions entre ells. L’anàlisi de totes les situacions que es poden donar en els dissenys més habituals permet fer recomanacions sobre quins experiments convé deixar de fer en el cas que per restriccions de temps o pressupostaries no es puguin fer tots el que indica el disseny factorial. Fer servir resultats estimats en comptes de fer els experiments, té conseqüències negatives encara que es facin servir les millors opcions possibles. Aquestes conseqüències negatives s’estudien en el quart article i fan referencia a l’augment de la probabilitat d’error, tant de tipus I com de tipus II, en l’anàlisi de la significació dels efectes. Aquest article aporta un conjunt de gràfics que representen les probabilitats d’error amb valors estimats en front el cas que s’hagin fet tots els experiments. Finalment, els cinquè article té una estructura similar al primer. Es compara el mètode de Lenth, el més popular i el que apareix més sovint en els paquets de software estadístic, amb el mètode de Box-Meyer. Aquest últim és un mètode bayesià que pràcticament no es fa servir, segurament per la seva dificultat tant a nivell conceptual com de càlcul. Però aquestes dificultats perden rellevància amb l’ús generalitzat dels ordinadors. L’article posa de manifest que en molts casos el mètode de Box-Meyer dona millor resultats que el de Lenth

SAS Macros for Analysis of Unreplicated 2^k and 2^k-p Designs with a Possible Outlier

Many techniques have been proposed for judging the significance of effects in unreplicated 2^k and 2^k-p designs. However, relatively few methods have been proposed for analyzing unreplicated designs with possible outliers. Outliers can be a major impediment to valid interpretation of data from unreplicated designs. This paper presents SAS macros which automate a manual method for detecting an outlier and performing an analysis of data from an unreplicated 2^k or 2^k-p design when an outlier is present. This method was originally suggested by Cuthbert Daniel and is based on the normal or half normal plot of effects. This automated version was shown in simulation studies to perform better than other procedures proposed to do the same thing.

Design of Experiments for Screening

The aim of this paper is to review methods of designing screening experiments, ranging from designs originally developed for physical experiments to those especially tailored to experiments on numerical models. The strengths and weaknesses of the various designs for screening variables in numerical models are discussed. First, classes of factorial designs for experiments to estimate main effects and interactions through a linear statistical model are described, specifically regular and nonregular fractional factorial designs, supersaturated designs and systematic fractional replicate designs. Generic issues of aliasing, bias and cancellation of factorial effects are discussed. Second, group screening experiments are considered including factorial group screening and sequential bifurcation. Third, random sampling plans are discussed including Latin hypercube sampling and sampling plans to estimate elementary effects. Fourth, a variety of modelling methods commonly employed with screening designs are briefly described. Finally, a novel study demonstrates six screening methods on two frequently-used exemplars, and their performances are compared

Minimum aberration designs for discrete choice experiments

A discrete choice experiment (DCE) is a survey method that givesinsight into individual preferences for particular attributes.Traditionally, methods for constructing DCEs focus on identifyingthe individual effect of each attribute (a main effect). However, aninteraction effect between two attributes (a two-factor interaction)better represents real-life trade-offs, and provides us a better understandingof subjects’ competing preferences. In practice it is oftenunknown which two-factor interactions are significant. To address theuncertainty, we propose the use of minimum aberration blockeddesigns to construct DCEs. Such designs maximize the number ofmodels with estimable two-factor interactions in a DCE with two-levelattributes. We further extend the minimum aberration criteria toDCEs with mixed-level attributes and develop some general theoreticalresults

Consequences of using estimated response values from negligible interactions in factorial designs

This article analyzes the increase in the probability of committing type I and type II errors in assessing the significance of the effects when some properly selected runs have not been carried out and their responses have been estimated from the interactions considered null from scratch. This is done by simulating the responses from known models that represent a wide variety of practical situations that the experimenter will encounter; the responses considered to be missing are then estimated and the significance of the effects is assessed. Through comparison with the parameters of the model, the errors are then identified. To assess the significance of the effects when there are missing values, the Box-Meyer method has been used. The conclusions are that 1 missing value in 8 run designs and up to 3 missing values in 16 run designs experiments can be estimated without hardly any notable increase in the probability of error when assessing the significance of the effects.Peer ReviewedPostprint (author's final draft

Using simulation studies to evaluate statistical methods

Simulation studies are computer experiments that involve creating data by pseudorandom sampling. The key strength of simulation studies is the ability to understand the behaviour of statistical methods because some 'truth' (usually some parameter/s of interest) is known from the process of generating the data. This allows us to consider properties of methods, such as bias. While widely used, simulation studies are often poorly designed, analysed and reported. This tutorial outlines the rationale for using simulation studies and offers guidance for design, execution, analysis, reporting and presentation. In particular, this tutorial provides: a structured approach for planning and reporting simulation studies, which involves defining aims, data-generating mechanisms, estimands, methods and performance measures ('ADEMP'); coherent terminology for simulation studies; guidance on coding simulation studies; a critical discussion of key performance measures and their estimation; guidance on structuring tabular and graphical presentation of results; and new graphical presentations. With a view to describing recent practice, we review 100 articles taken from Volume 34 of Statistics in Medicine that included at least one simulation study and identify areas for improvement.Comment: 31 pages, 9 figures (2 in appendix), 8 tables (1 in appendix

SAS Macros for Analysis of Unreplicated 2^k and 2^(k-p) Designs with a Possible Outlier

Many techniques have been proposed for judging the significance of effects in unreplicated 2^k and 2^(k-p) designs. However, relatively few methods have been proposed for analyzing unreplicated designs with possible outliers. Outliers can be a major impediment to valid interpretation of data from unreplicated designs. This paper presents SAS macros which automate a manual method for detecting an outlier and performing an analysis of data from an unreplicated 2^k and 2^(k-p) design when an outlier is present. This method was originally suggested by Cuthbert Daniel and is based on the normal or half normal plot of effects. This automated version was shown in simulation studies to perform better than other procedures proposed to do the same thing

Contributions to the planning and analysis of factorial designs

Tesi per compendi de publicacions.The thesis is structured in five articles. In the first article, two methods are compared to analyze the significance of the effects: the Lenth method and the one based on the estimation of the variance of the effects from interactions that can be considered negligible from scratch. For the most common factorial designs and in a set of scenarios that seek to reflect the situations that the experimenter can find in practice, simulation techniques are used to identify the errors that are committed with each method. Based on the analysis of the results obtained, we recommend in which situations it is more appropriate to use one method or the other. The second article analyzes the problem of estimating the results of experiments that could not be performed based on the expression of the interactions that can be considered negligible. The variance of the estimated values depends on what these values are and also on the interactions considered negligible. All possible encountered situations are analyzed and tables are presented with the values that can be estimated with minimum variance depending on the type of design and the contrasts available to perform the estimates. The third article deals with the same problem as the second but analyzing the impact of the estimate not on the variance of the estimated response values but on the variance of the effects and also on the correlations among them. The analysis of all the situations that can be given in the most common designs, allows us to make recommendations about what experiments should be skipped in the case that, due to time or budgetary constraints, all runs indicated by the factorial design cannot be executed.La tesi està estructurada en cinc articles. En el primer es comparen dos mètodes per analitzar la significació dels efectes: el mètode de Lenth i el basat en l’estimació de la variància dels efectes a partir de les interacciones que a priori es poden considerar nul·les. Pels dissenys factorials més habituals i en un conjunt d’escenaris que pretenen reflectir les situacions que l’experimentador es pots trobar a la pràctica, es fan servir tècniques de simulació per identificar els errors que es cometen amb cada mètode. A partir de l’anàlisi dels resultats obtinguts es recomana en quina situació és més adequat fer servir un mètode o l’altre. El segon article analitza el problema d’estimar els resultats d’experiments que no s’han pogut realitzar a partir de l’expressió de les interaccions que es poden considerar negligibles. La variància dels valors estimats depèn de quins siguin aquests valors i també de les interaccions considerades negligibles. S’analitzen totes les situacions que es poden donar i es presenten unes taules amb els valors que es poden estimar amb variància mínima segons el tipus de disseny i de quins siguin els contrastos disponibles per fer les estimacions. El tercer article aborda el mateix problema que el segon però analitzant l’impacte de l’estimació no en la variància dels valors estimats sinó en la variància dels efectes i també en les correlacions entre ells. L’anàlisi de totes les situacions que es poden donar en els dissenys més habituals permet fer recomanacions sobre quins experiments convé deixar de fer en el cas que per restriccions de temps o pressupostaries no es puguin fer tots el que indica el disseny factorial. Fer servir resultats estimats en comptes de fer els experiments, té conseqüències negatives encara que es facin servir les millors opcions possibles. Aquestes conseqüències negatives s’estudien en el quart article i fan referencia a l’augment de la probabilitat d’error, tant de tipus I com de tipus II, en l’anàlisi de la significació dels efectes. Aquest article aporta un conjunt de gràfics que representen les probabilitats d’error amb valors estimats en front el cas que s’hagin fet tots els experiments. Finalment, els cinquè article té una estructura similar al primer. Es compara el mètode de Lenth, el més popular i el que apareix més sovint en els paquets de software estadístic, amb el mètode de Box-Meyer. Aquest últim és un mètode bayesià que pràcticament no es fa servir, segurament per la seva dificultat tant a nivell conceptual com de càlcul. Però aquestes dificultats perden rellevància amb l’ús generalitzat dels ordinadors. L’article posa de manifest que en molts casos el mètode de Box-Meyer dona millor resultats que el de Lenth.Postprint (published version

Regularities in the Augmentation of Fractional Factorial Designs

Two-level factorial experiments are widely used in experimental design because they are simple to construct and interpret while also being efficient. However, full factorial designs for many factors can quickly become inefficient, time consuming, or expensive and therefore fractional factorial designs are sometimes preferable since they provide information on effects of interest and can be performed in fewer experimental runs. The disadvantage of using these designs is that when using fewer experimental runs, information about effects of interest is sometimes lost. Although there are methods for selecting fractional designs so that the number of runs is minimized while the amount of information provided is maximized, sometimes the design must be augmented with a follow-up experiment to resolve ambiguities. Using a fractional factorial design augmented with an optimal follow-up design allows for many factors to be studied using only a small number of additional experimental runs, compared to the full factorial design, without a loss in the amount of information that can be gained about the effects of interest. This thesis looks at discovering regularities in the number of follow-up runs that are needed to estimate all aliased effects in the model of interest for 4-, 5-, 6-, and 7-factor resolution III and IV fractional factorial experiments. From this research it was determined that for all of the resolution IV designs, four or fewer (typically three) augmented runs would estimate all of the aliased effects in the model of interest. In comparison, all of the resolution III designs required seven or eight follow-up runs to estimate all of the aliased effects of interest. It was determined that D-optimal follow-up experiments were significantly better with respect to run size economy versus fold-over and semi-foldover designs for (i) resolution IV designs and (ii) designs with larger run sizes