Meta-evaluations in government and government institutions: A case study example from the Australian Centre for International Agricultural Research

In this paper we draw on impact assessment work of the Australian Centre for International Agricultural Research (ACIAR) to present an example of meta-evaluation – an evaluation of evaluations – in an agricultural research, development and extension setting. We explore quality issues relating to evaluation studies in the context of government institutions. Program evaluation standards (PES) are divided into categories of utility, feasibility, propriety and accuracy to provide a framework for the meta-evaluation. The PES are presented as a universal measure of evaluation study quality. The intent of using them here is to judge the adequacy of PES as a universal quality measure or meta-evaluation base and to extract useful insights from ACIAR program evaluation activities when developing a meta-evaluation model for the Limpopo Department of Agriculture (LDA). Our meta-evaluation is undertaken of 63 impact assessment reports. First, the literature guiding the conduct of a meta-evaluation is reviewed. Second, an assessment (the meta-evaluation) of the evaluation studies is carried out for 19 sampled reports from a population of relevant reports fitting the dimension of the analysis, and results are presented and discussed. Also, lessons learned are presented, using the framework provided by the meta-evaluation criteria. Third, taking into account the lessons learned, implications are drawn for a proposed systematic meta-evaluation of the LDA. Finally, we conclude that all the PES cannot be equally emphasized in a meta-evaluation model. At ACIAR, 70% of the standards were at least partially addressed. Therefore, we succeeded in using the PES in judging the ACIAR evaluation quality. As such, they can be an important base when developing an evaluation model but should be applied in a contextualized manner.Meta-evaluation, Evaluation Quality, Program Evaluation Standards, Evaluation Model, Australian Centre for International Agricultural Research, Limpopo Department of Agriculture (South Africa), International Development,

The prevalence of autism spectrum disorders in China: A comprehensive meta-analysis

There are conflicting prevalence estimates of autism spectrum disorders (ASDs) in mainland China (China thereafter). This study is a comprehensive meta-analysis of the pooled prevalence of ASDs in the general population in China. Study investigators independently conducted a systematic literature search of the following databases: PubMed, EMBASE, PsycINFO, China National Knowledge Infrastructure, Chinese biomedical literature service system, and Wan Fang. Studies reporting prevalence of ASDs and autism in Chinese population were identified and analysed using the Comprehensive Meta-Analysis program with the random effects model. Forty-four studies were included in the meta-analysis comprising 2,337,321 subjects of whom 46.66 % were females. The mean age of subjects ranged from 1.6 to 8 years. Based on diagnostic criteria the pooled prevalence of ASDs was 39.23 per 10,000 (95% CI: 28.44-50.03 per 10,000, I2=89.2%); specifically, the prevalence of autism was 10.18 per 10,000 (95% CI: 8.46-11.89 per 10,000, I2=92.5%). Subgroup analyses revealed significant difference in the prevalence of ASDs between genders (72.77 per 10,000 in males vs. 16.45 per 10,000 in females). In conclusion, the prevalence of ASDs and autism in China was found generally lower than those reported in other countries. Further studies are needed to clarify the variation in prevalence

A random effects variance shift model for detecting and accommodating outliers in meta-analysis.

BACKGROUND: Meta-analysis typically involves combining the estimates from independent studies in order to estimate a parameter of interest across a population of studies. However, outliers often occur even under the random effects model. The presence of such outliers could substantially alter the conclusions in a meta-analysis. This paper proposes a methodology for identifying and, if desired, downweighting studies that do not appear representative of the population they are thought to represent under the random effects model. METHODS: An outlier is taken as an observation (study result) with an inflated random effect variance. We used the likelihood ratio test statistic as an objective measure for determining whether observations have inflated variance and are therefore considered outliers. A parametric bootstrap procedure was used to obtain the sampling distribution of the likelihood ratio test statistics and to account for multiple testing. Our methods were applied to three illustrative and contrasting meta-analytic data sets. RESULTS: For the three meta-analytic data sets our methods gave robust inferences when the identified outliers were downweighted. CONCLUSIONS: The proposed methodology provides a means to identify and, if desired, downweight outliers in meta-analysis. It does not eliminate them from the analysis however and we consider the proposed approach preferable to simply removing any or all apparently outlying results. We do not however propose that our methods in any way replace or diminish the standard random effects methodology that has proved so useful, rather they are helpful when used in conjunction with the random effects model

Using Meta Analysis for Benefits Transfer: Theory and Practice

Meta-analysis, or the "study of studies", attempts to statistically measure systematic relationships between reported valuation estimates for an environmental good or service and attributes of the study that generated the estimates including valuation methods, human population and sample characteristics, and characteristics of the good or service itself. In this paper, we discuss the general theory behind and practice of the emerging use of meta-analysis for benefits transfer. If carefully conducted following systematic protocols for model development, data collection, and data analysis and interpretation, we believe that meta-analysis may prove to be a useful tool for benefits transfer in particular applications. However, before widespread application of this method, more convergent validity tests are needed. One of the greatest strengths of using meta-analysis for benefits transfer is the ability to combine and summarize large amounts of information from previous studies. This strength can also lead to one of the greatest weaknesses of this method which is the loss of important valuation details across time and space in the aggregation process. Thus, application of this method to policy questions and issues should always proceed with caution.

Association between bacterial vaginosis and cervical intraepithelial neoplasia: systematic review and meta-analysis

Objective: Bacterial vaginosis (BV), the most common vaginal disorder among women of reproductive age, has been suggested as co-factor in the development of cervical cancer. Previous studies examining the relationship between BV and cervical intra-epithelial neoplasia (CIN) provided inconsistent and conflicting results. The aim of this study is to clarify the association between these two conditions. Methods: A systematic review and meta-analysis were conducted to summarize published literature on the association between BV and cervical pre-cancerous lesions. An extensive search of electronic databases Medline (Pubmed) and Web of Science was performed. The key words 'bacterial vaginosis' and 'bacterial infections and vaginitis' were used in combination with 'cervical intraepithelial neoplasia', 'squamous intraepithelial lesions', 'cervical lesions', 'cervical dysplasia', and 'cervical screening'. Eligible studies required a clear description of diagnostic methods used for detecting both BV and cervical precancerous lesions. Publications were included if they either reported odds ratios (OR) and corresponding 95% confidence intervals (CI) representing the magnitude of association between these two conditions, or presented data that allowed calculation of the OR. Results: Out of 329 articles, 17 cross-sectional and 2 incidence studies were selected. In addition, two studies conducted in The Netherlands, using the national KOPAC system, were retained. After testing for heterogeneity and publication bias, meta-analysis and meta-regression were performed, using a random effects model. Although heterogeneity among studies was high (chi(2) = 164.7, p < 0.01, I-2 = 88.5), a positive association between BV and cervical pre-cancerous lesions was found, with an overall estimated odds ratio of 1.51 (95% CI, 1.24-1.83). Meta-regression analysis could not detect a significant difference between studies based on BV diagnosis, CIN diagnosis or study population. Conclusions: Although most studies were cross-sectional and heterogeneity was high, this meta-analysis confirms a connection between BV and CIN

Tempo and Mode of Evolution in the Tangled Nature Model

We study the Tangled Nature model of macro evolution and demonstrate that the co-evolutionary dynamics produces an increasingly correlated core of well occupied types. At the same time the entire configuration of types becomes increasing de-correlated. This finding is related to ecosystem evolution. The systems level dynamics of the model is subordinated to intermittent transitions between meta-stable states. We improve on previous studies of the statistics of the transition times and show that the fluctuations in the offspring probability decreases with number of transitions. The longtime adaptation, as seen by an increasing population size is demonstrated to be related to the convexity of the offspring probability. We explain how the models behaviour is a mathematical reflection of Darwin's concept of adaptation of profitable variations.Comment: 6 pages, 5 figure

Meta-Potentiation: Neuro-Astroglial Interactions Supporting Perceptual Consciousness

Conscious perceptual processing involves the sequential activation of cortical networks at several brain locations, and the onset of oscillatory synchrony affecting the same neuronal population. How do the earlier activated circuits sustain their excitation to synchronize with the later ones? We call such a sustaining process &#x22;meta-potentiation&#x22;, and propose that it depends on neuro-astroglial interactions. In our proposed model, attentional cholinergic and stimulus-related glutamatergic inputs to astroglia elicit the release of astroglial glutamate to bind with neuronal NMDA receptors containing the NR2B subunit. Once calcium channels are open, slow inward currents activate the CaM/CaMKII complex to phosphorylate AMPA receptors in a population of neurons connected with the astrocyte, thus amplifying the local excitatory pattern to participate in a larger synchronized assembly that supports consciousness

Invasive species management in two-patch environments: Agricultural damage control in the raccoon (procyon lotor) problem, Hokkaido, Japan

We develop discrete-time models for analyzing the long run equilibrium outcomes on invasive species management in two-patch environments with migration. In particular, the focus is upon a situation where removal operations for invasive species are implemented only in one patch (controlled patch). The new features of the model are that (i) asymmetry in density dependent migration is considered, which may originate from impact of harvesting as well as heterogeneous habitat conditions, and (ii) the effect of density-dependent catchability is well-taken to account for the nature that required effort level to remove one individual may rise as the existing population decreases. The model is applied for agricultural damage control in the raccoon problem that has occurred in Hokkaido, Japan. Numerical illustration demonstrates that the long run equilibrium outcomes highly depend on the degree of asymmetry in migration as well as the sensitivity of catchability in response to a change in the population size of invasive species. Furthermore, we characterize the conditions under which the economically optimal effort levels are qualitatively affected by the above two factors and aiming at local extermination of invasive species in controlled patch is justified.catchability, meta-population, local extermination, removal effort, density dependent migration