Meta-analysis: Neither quick nor easy

BACKGROUND: Meta-analysis is often considered to be a simple way to summarize the existing literature. In this paper we describe how a meta-analysis resembles a conventional study, requiring a written protocol with design elements that parallel those of a record review. METHODS: The paper provides a structure for creating a meta-analysis protocol. Some guidelines for measurement of the quality of papers are given. A brief overview of statistical considerations is included. Four papers are reviewed as examples. The examples generally followed the guidelines we specify in reporting the studies and results, but in some of the papers there was insufficient information on the meta-analysis process. CONCLUSIONS: Meta-analysis can be a very useful method to summarize data across many studies, but it requires careful thought, planning and implementation

A meta-analysis of catalytic literature data reveals property-performance correlations for the OCM reaction

Decades of catalysis research have created vast amounts of experimental data. Within these data, new insights into property-performance correlations are hidden. However, the incomplete nature and undefined structure of the data has so far prevented comprehensive knowledge extraction. We propose a meta-analysis method that identifies correlations between a catalyst’s physico-chemical properties and its performance in a particular reaction. The method unites literature data with textbook knowledge and statistical tools. Starting from a researcher’s chemical intuition, a hypothesis is formulated and tested against the data for statistical significance. Iterative hypothesis refinement yields simple, robust and interpretable chemical models. The derived insights can guide new fundamental research and the discovery of improved catalysts. We demonstrate and validate the method for the oxidative coupling of methane (OCM). The final model indicates that only well-performing catalysts provide under reaction conditions two independent functionalities, i.e. a thermodynamically stable carbonate and a thermally stable oxide support

A Family of Experiments to Evaluate the Effects of Mindfulness on Software Engineering Students: The MetaMind Dataset

Context: Software Engineering students are often excellent developers although they may occasionally encounter difficulties with certain tasks such as conceptual modelling, in which mental clarity plays an essential role. With several years of experience in practising mindfulness (a meditation technique that calm the mind to see with clarity), we hypothesise that several weeks of continued mindfulness practise may increase the performance of students regarding conceptual modelling since the proven benefits of mindfulness include increased concentration and mental clarity. In order to ascertain this hypothesis, a family of controlled experiments, involving 130 students, has been carried out at the University of Seville over three consecutive academic years. Subsequent to the analysis of the individual experiments, a meta—analysis was conducted. Aim: This chapter helps understand how various datasets have been generated and traces their evolution across the family of experiments and the resulting meta—analysis. Not only does the process include the integration of datasets into a single target dataset with the complete set of data, but it also improves the structure of the dataset and adapts it to several statistical analysis tools. Method: Data collection was manually carried out during the individual experiments by means of gathering the scores of the students attained in conceptual modelling exercises. In order to obtain the complete but also simple dataset of the family of experiments, certain dataset columns were necessarily selected and renamed. New relevant information was also added to form a conclusive meta—analysis. The dataset was adapted to the type and format of SPSS for the execution of a further analysis, complementary to the main study in R. The MetaMind dataset is the set of datasets involved in the present research. The MetaMind dataset has become a helpful resource for the independent reproducibility of the results, to ascertain the evolution of the process across a family of experiments, and to enable external replications of the controlled experiment.Ministerio de Ciencia e Innovación HORATIO (RTI2018-101204–B–C21)Junta de Andalucía APOLO (US-1264651)Junta de Andalucía EKIPMENTPLUS (P18–FR–2895

The molecular basis of antigenic variation among A(H9N2) avian influenza viruses

Avian influenza A(H9N2) viruses are an increasing threat to global poultry production and, through zoonotic infection, to human health where they are considered viruses with pandemic potential. Vaccination of poultry is a key element of disease control in endemic countries, but vaccine effectiveness is persistently challenged by the emergence of antigenic variants. Here we employed a combination of techniques to investigate the genetic basis of H9N2 antigenic variability and evaluate the role of different molecular mechanisms of immune escape. We systematically tested the influence of published H9N2 monoclonal antibody escape mutants on chicken antisera binding, determining that many have no significant effect. Substitutions introducing additional glycosylation sites were a notable exception, though these are relatively rare among circulating viruses. To identify substitutions responsible for antigenic variation in circulating viruses, we performed an integrated meta-analysis of all published H9 haemagglutinin sequences and antigenic data. We validated this statistical analysis experimentally and allocated several new residues to H9N2 antigenic sites, providing molecular markers that will help explain vaccine breakdown in the field and inform vaccine selection decisions. We find evidence for the importance of alternative mechanisms of immune escape, beyond simple modulation of epitope structure, with substitutions increasing glycosylation or receptor-binding avidity, exhibiting the largest impacts on chicken antisera binding. Of these, meta-analysis indicates avidity regulation to be more relevant to the evolution of circulating viruses, suggesting that a specific focus on avidity regulation is required to fully understand the molecular basis of immune escape by influenza, and potentially other viruses

Consensus and meta-analysis regulatory networks for combining multiple microarray gene expression datasets

Microarray data is a key source of experimental data for modelling gene regulatory interactions from expression levels. With the rapid increase of publicly available microarray data comes the opportunity to produce regulatory network models based on multiple datasets. Such models are potentially more robust with greater confidence, and place less reliance on a single dataset. However, combining datasets directly can be difficult as experiments are often conducted on different microarray platforms, and in different laboratories leading to inherent biases in the data that are not always removed through pre-processing such as normalisation. In this paper we compare two frameworks for combining microarray datasets to model regulatory networks: pre- and post-learning aggregation. In pre-learning approaches, such as using simple scale-normalisation prior to the concatenation of datasets, a model is learnt from a combined dataset, whilst in post-learning aggregation individual models are learnt from each dataset and the models are combined. We present two novel approaches for post-learning aggregation, each based on aggregating high-level features of Bayesian network models that have been generated from different microarray expression datasets. Meta-analysis Bayesian networks are based on combining statistical confidences attached to network edges whilst Consensus Bayesian networks identify consistent network features across all datasets. We apply both approaches to multiple datasets from synthetic and real (Escherichia coli and yeast) networks and demonstrate that both methods can improve on networks learnt from a single dataset or an aggregated dataset formed using a standard scale-normalisation

Forecasting and prequential validation for time varying meta-elliptical distributions

We consider forecasting and prequential (predictive sequential) validation of meta-elliptical distributions with time varying parameters. Using the weak prequential principle of Dawid, we conduct model validation avoiding nuisance parameter problems. Results rely on the structure of meta-elliptical distributions and we allow for discontinuities in the marginals and time varying parameters. We illustrate the ideas of the paper using a large data set of 16 commodity prices

Aspects of Objective Priors and Computations for Bayesian Modelling

Bayesian statistics is flourishing nowadays not only because it provides ways to combine prior knowledge with statistical models but also because many algorithms have become available to sample from the resulting posterior distributions. However, how to specify a good objective prior can be very difficult. This is largely because ignorance does not have a unique definition. For sampling from posterior distributions, Markov Chain Monte Carlo (MCMC) methods are main tools. However, as statistical models become more and more sophisticated, there is a need for more efficient MCMC methods than the traditional ones. For objective prior specifications, we present a new principle to express ignorance through the global distance structure. This principle allows us to assign the prior weight to points in parameter space according to their correspondences to the statistical models displayed in the structure of the global distance. This method is applied to simple problems such as location family, scale family and location-scale family. It is also applied to the one-way random effect model which attracts considerable interest from many researchers. The method considered here allows us to avoid the dependency of the priors on the experimental design, which has been seriously disputed, and enables the resulting prior to reflect how the models change with respect to the population and not the collected samples. Of MCMC methods for sampling from posterior distributions, the Hamiltonian Monte Carlo (HMC) method is one that has the potential to avoid random-walk behaviour. It does so by exploiting ideas from Hamiltonian dynamics. Its performance, however, depends on the choice of step-size which is required by this method when numerically solving the Hamiltonian equations. We propose an algorithm, which we call HMC with stochastic step-size, to automatically tune the step-size by exploiting the local curvature information. We also present a meta-algorithm which includes HMC, HMC with stochastic step-size and the ordinary Metropolis-Hastings algorithm as a special case. Finally, we come to a sophisticated hierarchical model developed for analysing the exco-toxicology data. We present ways to obtain more informative posterior samples by embedding the marginalized approach and advanced samplers into the entire Gibbs structure of the modified MCMCglmm algorithm provided by Craig (2013). The combination of the marginalized approach and HMC with stochastic step-size is found to be the best choice among a range of methods for the challenging problem of sampling the hyper-parameters in the model