External validity in healthy public policy: application of the RE-AIM tool to the field of housing improvement

<b>Background</b><p></p> Researchers and publishers have called for improved reporting of external validity items and for testing of existing tools designed to assess reporting of items relevant to external validity. Few tools are available and most of this work has been done within the field of health promotion.<p></p> <b>Methods</b><p></p> We tested a tool assessing reporting of external validity items which was developed by Green & Glasgow on 39 studies assessing the health impacts of housing improvement. The tool was adapted to the topic area and criteria were developed to define the level of reporting, e.g. “some extent”. Each study was assessed by two reviewers.<p></p> <b>Results</b><p></p> The tool was applicable to the studies but some items required considerable editing to facilitate agreement between the two reviewers. Levels of reporting of the 17 external validity items were low (mean 6). The most commonly reported items related to outcomes. Details of the intervention were poorly reported. Study characteristics were not associated with variation in reporting.<p></p> <b>Conclusions</b><p></p> The Green & Glasgow tool was useful to assess reporting of external validity items but required tailoring to the topic area. In some public health evaluations the hypothesised impact is dependent on the intervention effecting change, e.g. improving socio-economic conditions. In such studies data confirming the function of the intervention may be as important as details of the components and implementation of the intervention

From the lab to the field: envelopes, dictators and manners

Results are reported of the first natural field experiment on the dictator game, where subjects are unaware that they participate in an experiment. In contrast to predictions of the standard economic model, dictators show a large degree of pro-social behavior. This paper builds a bridge from the laboratory to the field to explore how predictive findings from the laboratory are for the field. External validity is remarkably high. In all experiments, subjects display an equally high amount of pro-social behavior, whether they are students or not, participate in a laboratory or not, or are aware that they participate in an experiment or not.altruism, natural field experiment, external validity

Gluodynamics in External Field: A Test of the Dual Superconductor Picture

We study gluodynamics in an external Abelian electromagnetic field within the dual superconductor approach. We show that the SU(2) gluodynamics should possess a deconfining phase transition at zero temperature at certain value of the external field. A dual superconductor model for the SU(3) gauge theory in external field predicts a rich phase structure containing confinement, asymmetric confinement and deconfinement phases. These results can be used to check the validity of the dual superconductor description of gluodynamics in external fields. We also discuss the gauge-independence of the obtained results.Comment: Talk given at ``Confinement 2003'', RIKEN, Wako, Japan, July 21-24, 2003; 5 pages, 1 figure, uses ws-procs9x6.cl

Internal and External Validity of Sluggish Cognitive Tempo in Young Adolescents with ADHD

Adolescents with Sluggish Cognitive Tempo (SCT) show symptoms of slowness, mental confusion, excessive daydreaming, low motivation, and drowsiness/sleepiness. Although many symptoms of SCT reflect internalizing states, no study has evaluated the utility of self-report of SCT in an ADHD sample. Further, it remains unclear whether SCT is best conceptualized as a unidimensional or multidimensional construct. In a sample of 262 adolescents comprehensively diagnosed with ADHD, the present study evaluated the dimensionality of a SCT scale and compared CFA and bifactor model fits for parent- and self-report versions. Analyses revealed the three-factor bifactor model to be the best fitting model. In addition, SCT factors predicted social and academic impairment and internalizing symptoms. Therefore, SCT as a multidimensional construct appears to have clinical utility in predicting impairment. Also, multiple reporters should be used, as they predicted different areas of functioning and were not invariant, suggesting that each rater adds unique information

Improving the external validity of clinical trials: the case of multiple chronic conditions

The U.S. Department of Health and Human Services vision and strategic framework on multiple chronic conditions (MCCs) incorporates recommendations designed to facilitate research that will improve our knowledge about interventions and systems that will benefit individuals with MCCs (or multimorbidity). The evidence base supporting the management of patients with MCCs will be built both through intervention trials specifically designed to address multimorbidity and identification of MCCs in participants across the clinical trial range. This article specifically focuses on issues relating to external validity with specific reference to trials involving patients with MCCs. The exclusion of such patients from clinical trials has been well documented. Randomized control trials (RCTs) are considered the “gold standard” of evidence, but may have drawbacks in relation to external validity, particularly in relation to multimorbidity. It may, therefore, be necessary to consider a broader range of research methods that can provide converging evidence on intervention effects to address MCCs. Approaches can also be taken to increase the usefulness of RCTs in general for providing evidence to inform multimorbidity management. Additional improvements to RCTs would include better reporting of inclusion and exclusion criteria and participant characteristics in relation to MCCs. New trials should be considered in terms of how they will add to the existing evidence base and should inform how interventions may work in different settings and patient groups. Research on treatments and interventions for patients with MCCs is badly needed. It is important that this research includes patient-centered measures and that generalizability issues be explicitly addressed.Journal of Comorbidity 2013;3(2)30–3

External Validity: From Do-Calculus to Transportability Across Populations

The generalizability of empirical findings to new environments, settings or populations, often called "external validity," is essential in most scientific explorations. This paper treats a particular problem of generalizability, called "transportability," defined as a license to transfer causal effects learned in experimental studies to a new population, in which only observational studies can be conducted. We introduce a formal representation called "selection diagrams" for expressing knowledge about differences and commonalities between populations of interest and, using this representation, we reduce questions of transportability to symbolic derivations in the do-calculus. This reduction yields graph-based procedures for deciding, prior to observing any data, whether causal effects in the target population can be inferred from experimental findings in the study population. When the answer is affirmative, the procedures identify what experimental and observational findings need be obtained from the two populations, and how they can be combined to ensure bias-free transport.Comment: Published in at http://dx.doi.org/10.1214/14-STS486 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org). arXiv admin note: text overlap with arXiv:1312.748

Experiments in the Social Sciences: The relationship between External and Internal Validity

The article identifies a latent debate in the recent literature on the role and worth of experiments in economics and other social sciences concerning the relationship between the external and the internal validity of experimental designs. Our work identifies two incompatible views regarding the relationship between internal and external validity of experiments. While in the methodological literature references to the idea that there is a trade-off between the internal and external validity of experiments abound, this view coexists with the position stating that internal validity is rather a prerequisite of external validity. By identifying the contours of this implicit debate in the recent methodological literature around the use of experiments in the social sciences we call attention upon a series of insufficiently conceptualized issues regarding the central notions of internal and external validity and we question the standard view positing a trade-off between the two. This article stands against common associations of internal validity and external validity with the distinction between field and laboratory experiments and assesses critically the arguments that link the artificiality of experimental settings done in the laboratory with the purported trade-off between internal and external validity