Methods for Population Adjustment with Limited Access to Individual Patient Data: A Review and Simulation Study

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in effective sample size and precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.Comment: 73 pages (34 are supplementary appendices and references), 8 figures, 2 tables. Full article (following Round 4 of minor revisions). arXiv admin note: text overlap with arXiv:2008.0595

Correcting Knowledge Base Assertions

The usefulness and usability of knowledge bases (KBs) is often limited by quality issues. One common issue is the presence of erroneous assertions, often caused by lexical or semantic confusion. We study the problem of correcting such assertions, and present a general correction framework which combines lexical matching, semantic embedding, soft constraint mining and semantic consistency checking. The framework is evaluated using DBpedia and an enterprise medical KB

Some Econometric Evidence on the Effectiveness of Active Labour Market Programmes in East Germany

In this paper we summarise our previous results on the effectiveness of different kinds of labour market training programmes as well as employment programmes in East Germany after unification. All the studies use the microeconometric evaluation approach and are based on different types of matching estimators. We find some positive earnings effect for on-the-job training and also some positive employment effects for employment programmes. No such effects appear for public sector sponsored (off-the-job) training programmes. Generally, the scope of such analysis is very much hampered by the insufficient quality and quantity of the data available for East Germany. Although in particular the results for public sector sponsored training programmes raise serious doubts about the effectiveness of these programmes, any definite policy conclusion from this and other studies about active labour market policy in East Germany would probably be premature.http://deepblue.lib.umich.edu/bitstream/2027.42/39702/3/wp318.pd

Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure

Statistical Matching of Administrative and Survey Data : An Application to Wealth Inequality Analysis

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Using population representative survey data from the German Socio-Economic Panel (SOEP) and administrative pension records from the Statutory Pension Insurance, the authors compare four statistical matching techniques to complement survey information on net worth with social security wealth (SSW) information from the administrative records. The unique properties of the linked data allow for a straight control of the quality of matches under each technique. Based on various evaluation criteria, Mahalanobis distance matching performs best. Exploiting the advantages of the newly assembled data, the authors include SSW in a wealth inequality analysis. Despite its quantitative relevance, SSW is thus far omitted from such analyses because adequate micro data are lacking. The inclusion of SSW doubles the level of net worth and decreases inequality by almost 25 percent. Moreover, the results reveal striking differences along occupational lines.Hans Böckler-Foundation, 2006-835-4, Erstellung und Analyse einer konsistenten Geld- und Realvermögensverteilungsrechnung für Personen und Haushalte 2002 und 2007 unter Berücksichtigung der personellen Einkommensverteilun