AIC-type Theory-Based Model Selection for Structural Equation Models

Structural equation modeling (SEM) software commonly report information criteria, like the AIC, for the model under investigation and for the unconstrained/saturated model. With these criteria, (non-)nested models can be compared. This comes down to evaluating equalities (e.g., setting some paths equal or to 0). These criteria cannot evaluate inequality restrictions on the parameters, while the AIC-type criterion called GORICA can. For example, GORICA can evaluate the hypothesis stating that one predictor has more (standardized) strength than some other predictors. This paper illustrates inequality-constrained hypothesis-evaluation in SEM models using the GORICA (in R). Examples will be presented for confirmatory factor analysis, latent regression, and multigroup latent regression

How to Evaluate Theory-Based Hypotheses in Meta-Analysis Using an AIC-Type Criterion

Meta-analysis techniques allow researchers to aggregate effect sizes—like standardized mean difference(s), correlation(s), or odds ratio(s)—of different studies. This leads to overall effect-size estimates and their confidence intervals. Additionally, researchers can aim for theory development or theory evaluation. That is, researchers may not only be interested in these overall estimates but also in a specific ordering or size of them, which then reflects a theory. Researchers may have expectations regarding the ordering of standardized mean differences or about the (ranges of) sizes of an odds ratio or Hedges’ g. Such theory-based hypotheses most probably contain inequality constraints and can be evaluated with the Akaike’s information criterion type (i.e., AIC-type) confirmatory model selection criterion called generalized order-restricted information criterion (GORICA). This paper introduces and illustrates how the GORICA can be applied to meta-analyzed estimates. Additionally, it compares the use of the GORICA to that of classical null hypothesis testing and the AIC, that is, the use of theory-based hypotheses versus null hypotheses. By using the GORICA, researchers from all types of fields (e.g., psychology, sociology, political science, biomedical science, and medicine) can quantify the support for theory-based hypotheses specified a priori. This leads to increased statistical power, because of (i) the use of theory-based hypotheses (cf. one-sided vs. two-sided testing) and (ii) the use of meta-analyzed results (that are based on multiple studies which increase the combined sample size). The quantification of support and the power increase aid in, for instance, evaluating and developing theories and, therewith, developing evidence-based treatments and policy

How to Evaluate Causal Dominance Hypotheses in Lagged Effects Models

The (Random Intercept) Cross-Lagged Panel Model ((RI-)CLPM) is increasingly used in psychology and related fields to assess the longitudinal relationship of two or more variables on each other. Researchers are interested in the question which of the lagged effects is causally dominant receives considerable attention. However, currently used methods do not allow for the evaluation of causal dominance hypotheses. This paper will show the performance of the Generalized Order-Restricted Information Criterion Approximation (GORICA), an extension of Akaike’s Information Criterion (AIC), in the context of causal dominance hypotheses using a simulation study. The GORICA proves to be an adequate method to evaluate causal dominance in lagged effects models

Drawing Conclusions from Cross-Lagged Relationships: Re-Considering the Role of the Time-Interval

The cross-lagged panel model (CLPM), a discrete-time (DT) SEM model, is frequently used to gather evidence for (reciprocal) Granger-causal relationships when lacking an experimental design. However, it is well known that CLPMs can lead to different parameter estimates depending on the time-interval of observation. Consequently, this can lead to researchers drawing conflicting conclusions regarding the sign and/or dominance of relationships. Multiple authors have suggested the use of continuous-time models to address this issue. In this article, we demonstrate the exact circumstances under which such conflicting conclusions occur. Specifically, we show that such conflicts are only avoided in general in the case of bivariate, stable, nonoscillating, first-order systems, when comparing models with uniform time-intervals between observations. In addition, we provide a range of tools, proofs, and guidelines regarding the comparison of discrete- and continuous-time parameter estimates

Public health workforce capacity building: the use of quality assurance indicators for the improvement in public health programmes

Agenţia Europeană pentru Acreditare în Sănătate Publică (APHEA), Bruxelles, Belgia, Școala de Sănătate Publică Charité-Universitätsmedizin, Berlin, Germania, Departamentul Sănătate Publică și Pediatrie, Școala de Medicină, Universitatea din Torino, Italia, Facultatea de Sănătate Publică, Universitatea Lituaniană de Știinţe în Sănătate, Kaunas, Lituania, Comitetul de Acreditare, Agenţia Europeană pentru Acreditare în Sănătate Publică (APHEA), Bruxelles, Belgia, Școala de Sănătate Publică Charité-Universitätsmedizin, Berlin, GermaniaRezumat În 2011, a fost fondată Agenţia Europeană pentru Acreditare în Sănătate Publică (APHEA). Această agenţie a fost o culminaţie a celor peste 25 de ani de activitate pentru îmbunătăţirea calităţii activităţii Asociaţiei Şcolilor de Sănătate Publică din Regiunea Europeană (ASPHER). Acreditarea a înlocuit un instrument anterior, numit evaluare inter pares (evalure colegială), care a fost utilizat între 2001 și 2006, pentru a ajuta la dezvoltarea școlilor și programelor din regiunea Europei Centrale și de Est. În 2012, APHEA, ASPHER și școlile partenere au utilizat noile criterii de acreditare pentru a evalua trei școli din Regiunea Europeană care au demonstrat că, doar cu mici ajustări, există potenţial de a încadra îmbunătăţirea calităţii în procesul de acreditare. Abstract In 2011, the Agency for Public Health Education Accreditation (APHEA) was launched. This agency was a culmination of over 25 years activity on quality improvement by the Association of Schools of Public Health in the European Region (ASPHER). Accreditation superseded a previous tool called a PEER review, which was used between 2001 and 2006 to help in the development of schools and programmes in the Central Eastern European Region. In 2012, APHEA, ASPHER and partner schools used the new accreditation criteria to review three schools throughout the European Region which proved that, with small adjustments, there was a potential to incorporate a quality Improvement framework around the accreditation process

Evaluation of inequality constrained hypotheses using a generalization of the AIC

In the social and behavioral sciences, it is often not interesting to evaluate the null hypothesis by means of a p-value. Researchers are often more interested in quantifying the evidence in the data (as opposed to using p-values) with respect to their own expectations represented by equality and/or inequality constrained hypotheses (as opposed to the null hypothesis). This article proposes an Akaike-type information criterion (AIC; Akaike, 1973, 1974) called the generalized order-restricted information criterion approximation (GORICA) that evaluates (in)equality constrained hypotheses under a very broad range of statistical models. The results of five simulation studies provide empirical evidence showing that the performance of the GORICA on selecting the best hypothesis out of a set of (in)equality constrained hypotheses is convincing. To illustrate the use of the GORICA, the expectations of researchers are investigated in a logistic regression, multilevel regression, and structural equation model

Evaluation of inequality constrained hypotheses using a generalization of the AIC

In the social and behavioral sciences, it is often not interesting to evaluate the null hypothesis by means of a p-value. Researchers are often more interested in quantifying the evidence in the data (as opposed to using p-values) with respect to their own expectations represented by equality and/or inequality constrained hypotheses (as opposed to the null hypothesis). This article proposes an Akaike-type information criterion (AIC; Akaike, 1973, 1974) called the generalized order-restricted information criterion approximation (GORICA) that evaluates (in)equality constrained hypotheses under a very broad range of statistical models. The results of five simulation studies provide empirical evidence showing that the performance of the GORICA on selecting the best hypothesis out of a set of (in)equality constrained hypotheses is convincing. To illustrate the use of the GORICA, the expectations of researchers are investigated in a logistic regression, multilevel regression, and structural equation model.</p

Evaluating a Theory-Based Hypothesis Against Its Complement Using an AIC-Type Information Criterion With an Application to Facial Burn Injury

An information criterion (IC) like the Akaike IC (AIC), can be used to select the best hypothesis from a set of competing theory-based hypotheses. An IC developed to evaluate theory-based order-restricted hypotheses is the Generalized Order-Restricted Information Criterion (GORIC). Like for any IC, the values themselves are not interpretable but only comparable. To improve the interpretation regarding the strength, GORIC weights and related evidence ratios can be computed. However, if the unconstrained hypothesis (the default) is used as competing hypothesis, the evidence ratio is not affected by sample-size nor effect-size in case the hypothesis of interest is (also) in agreement with the data. In practice, this means that in such a case strong support for the order-restricted hypothesis is not reflected by a high evidence ratio. Therefore, we introduce the evaluation of an order-restricted hypothesis against its complement using the GORIC (weights). We show how to compute the GORIC value for the complement, which cannot be achieved by current methods. In a small simulation study, we show that the evidence ratio for the order-restricted hypothesis versus the complement increases for larger samples and/or effect-sizes, while the evidence ratio for the order-restricted hypothesis versus the unconstrained hypothesis remains bounded. An empirical example about facial burn injury illustrates our method and shows that using the complement as competing hypothesis results in much more support for the hypothesis of interest than using the unconstrained hypothesis as competing hypothesis

Relationship quality in higher education and the interplay with student engagement and loyalty

Background: To date, studies that have investigated the bonds between students and their institution have emphasized the importance of student–staff relationships. Measuring the quality of those relationships (i.e., relationship quality) appears to help with investigating the relational ties students have with their higher education institutions. Growing interest has arisen in further investigating relationship quality in higher education, as it might predict students’ involvement with the institution (e.g., student engagement and student loyalty). So far, most studies have used a cross-sectional design, so that causality could not be determined. Aims: The aim of this longitudinal study was twofold. First, we investigated the temporal ordering of the relation between the relationship quality dimensions of trust (in benevolence and honesty) and affect (satisfaction, affective commitment, and affective conflict). Second, we examined the ordering of the paths between relationship quality, student engagement, and student loyalty. Our objectives were to gain a deeper understanding of the relationship quality construct in higher education and its later outcomes. Sample: Participants (N = 1649) were students from three Dutch higher education institutions who were studying in a technology economics or social sciences program. Methods: Longitudinal data from two time points were used to evaluate two types of cross-lagged panel models. In the first analysis, we could not assume measurement invariance for affective conflict over time. Therefore, we tested an alternative model without affective conflict, using the latent variables of trust and affect, the student engagement dimensions and student loyalty. In the second type of model, we investigated the manifest variables of relationship quality, student engagement, and student loyalty. The hypotheses were tested by evaluating simultaneous comparisons between estimates. Results: Results indicated that the relation between relationship quality at Time 1 with student engagement and loyalty at Time 2 was stronger than the reverse ordering in the first model. In the second model, results indicated that cross-lagged relations between trust in benevolence and trust in honesty at Time 1 and affective commitment, affective conflict, and satisfaction at Time 2 were more likely than the reverse ordering. Furthermore, cross-lagged relations from relationship quality at Time 1 to student engagement and student loyalty at Time 2 also supported our hypothesis. Conclusions: This study contributes to the existing higher education literature, indicating that students’ trust in the quality of their relationship with faculty/staff is essential for developing students’ affective commitment and satisfaction and for avoiding conflict over time. Second, relationship quality factors positively influence students’ engagement in their studies and their loyalty towards the institution. A relational approach to establishing (long-lasting) bonds with students appears to be fruitful as an approach for educational psychologists and for practitioners’ guidance and strategies. Recommendations are made for future research to further examine relationship quality in higher education in Europe and beyond

IsoGeneGUI: Multiple Approaches for Dose-Response Analysis of Microarray Data Using R

The analysis of transcriptomic experiments with ordered covariates, such as dose-response data, has become a central topic in bioinformatics, in particular in omics studies. Consequently, multiple R packages on CRAN and Bioconductor are designed to analyse microarray data from various perspectives under the assumption of order restriction. We introduce the new R package IsoGene Graphical User Interface (IsoGeneGUI), an extension of the original IsoGene package that includes methods from most of available R packages designed for the analysis of order restricted microarray data, namely orQA, ORIClust, goric and ORCME. The methods included in the new IsoGeneGUI range from inference and estimation to model selection and clustering tools. The IsoGeneGUI is not only the most complete tool for the analysis of order restricted microarray experiments available in R but also it can be used to analyse other types of dose-response data. The package provides all the methods in a user friendly fashion, so analyses can be implemented by users with limited knowledge of R programming