Meta-analysis methods for synthesizing treatment effects in multisite studies: hierarchical linear modeling (HLM) perspective

The objectives of the present mixed-effects meta-analytic application are to provide practical guidelines to: (a) Calculate treatment effect sizes from multiple sites; (b) Calculate the overall mean of the site effect sizes and their variances; (c) Model the heterogeneity in these site treatment effects as a function of site and program characteristics plus unexplained random error using Hierarchical Linear Modeling (HLM); (d) Improve the ability of multisite evaluators and policy makers to reach sound conclusions about the effectiveness of educational and social interventions based on multisite evaluations; and (e) Illustrate the proposed methodology by applying these methods to real multi-site research data. Accessed 58,759 times on https://pareonline.net from June 23, 2003 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Terminating Sequential Delphi Survey Data Collection

The Delphi survey technique is an iterative mail or electronic (e-mail or web-based) survey method used to obtain agreement or consensus among a group of experts in a specific field on a particular issue through a well-designed and systematic multiple sequential rounds of survey administrations. Each of the multiple rounds of the Delphi survey administration is augmented with continuous summary feedback of aggregated responses from the same group of experts. Statistical methods to analyze data from the Delphi surveys to make decisions for terminating subsequent Delphi data collection are needed to ensure that (a) stability of the responses of the panel of experts is reached; and (b) termination of the rounds of the Delphi survey administration is based on sound statistical results. The present study presents an overview of the parametric and nonparametric statistical methods that can be used to analyze the structured Delphi survey data to make decisions about terminating the sequential Delphi survey data collection. Accessed 9,961 times on https://pareonline.net from January 23, 2012 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Analyzing The Effect Of Top Management Support On Information System (IS) Performance Across Organizations And Industries Using Hierarchical Linear Modeling

Top management support has long been conceivable as an important factor for the success of IS projects. Due to the hierarchical nature of an organization, a cross-level interaction can occur among nested levels. Thus, using inappropriate statistical analysis can cause misleading results and lost of information. This study provides two contributions to the IS research. First, Hierarchical Linear Modeling (HLM) was used to explain the cross-level interaction between organizational level and industry level. Second, unlike other studies focusing on an organizational level, this study considers top management support at the industry level and examines the mediating role of top management support between the two levels

Which Is the Best Parametric Statistical Method For Analyzing Delphi Data?

This study compares the three parametric statistical methods: coefficient of variation, Pearson correlation coefficient, and F-test to obtain reliability in a Delphi study that involved more than 100 participants. The results of this study indicated that coefficient of variation was the best procedure to obtain reliability in such a study

Is Self-Confidence in Teaching Multidimensional or Unidimensional? An Exploratory Study

This study examined the factor structure of a 12-item Self-Confidence in Teaching Scale. Two samples of teacher candidates participated: 1,176 students (80% females) entering a teacher preparation program and 861 candidates (77% females) completing their final semester in the program. Even though the scale was deliberately designed to assess candidates\u27 self-confidence in executing 12 distinct teaching roles (as confirmed by a panel of judges), principal axis factor analyses yielded only one factor with an eigenvalue greater than 1.0. Also, similar patterns of factor loadings were found across all subsample contrasts that were considered (e.g., females vs. males; entry-vs. exit-level candidates). These results suggest that the measure of self-confidence in teaching considered in this investigation is a unidimensional rather than a multidimensional scale

Measuring elimination of podoconiosis, endemicity classifications, case definition and targets: an international Delphi exercise

BACKGROUND Podoconiosis is one of the major causes of lymphoedema in the tropics. Nonetheless, currently there are no endemicity classifications or elimination targets to monitor the effects of interventions. This study aimed at establishing case definitions and indicators that can be used to assess endemicity, elimination and clinical outcomes of podoconiosis. METHODS This paper describes the result of a Delphi technique used among 28 experts. A questionnaire outlining possible case definitions, endemicity classifications, elimination targets and clinical outcomes was developed. The questionnaire was distributed to experts working on podoconiosis and other neglected tropical diseases in two rounds. The experts rated the importance of case definitions, endemic classifications, elimination targets and the clinical outcome measures. Median and mode were used to describe the central tendency of expert responses. The coefficient of variation was used to describe the dispersals of expert responses. RESULTS Consensus on definitions and indicators for assessing endemicity, elimination and clinical outcomes of podoconiosis directed at policy makers and health workers was achieved following the two rounds of Delphi approach among the experts. CONCLUSIONS Based on the two Delphi rounds we discuss potential indicators and endemicity classification of this disabling disease, and the ongoing challenges to its elimination in countries with the highest prevalence. Consensus will help to increase effectiveness of podoconiosis elimination efforts and ensure comparability of outcome data

An exploration of tutors’ experiences of facilitating problem-based learning. Part 2: Implications for the facilitation of problem-based learning

YesThis paper is the second of two parts exploring a study that was undertaken to investigate the role of the tutor in facilitating problem-based learning (PBL). The first part focussed on the methodological underpinnings of the study. This paper aims to focus on the findings of the study and their implications for the facilitation of PBL. Six essential themes emerged from the findings that described the facilitation role. The tutors believed that their facilitation role was essentially structured around the decision of when to intervene and how to intervene in the PBL process. Modelling and non-verbal communication were seen as essential strategies for the facilitator. Underpinning these decisions was the need to trust in the philosophy of PBL. However, within many of the themes, there was a divergence of opinion as to how the role should actually be undertaken. Despite this, these findings have implications for the future role of PBL facilitators in Health Professional Education

Bivariate random-effects meta-analysis and the estimation of between-study correlation

BACKGROUND: When multiple endpoints are of interest in evidence synthesis, a multivariate meta-analysis can jointly synthesise those endpoints and utilise their correlation. A multivariate random-effects meta-analysis must incorporate and estimate the between-study correlation (ρ(B)). METHODS: In this paper we assess maximum likelihood estimation of a general normal model and a generalised model for bivariate random-effects meta-analysis (BRMA). We consider two applied examples, one involving a diagnostic marker and the other a surrogate outcome. These motivate a simulation study where estimation properties from BRMA are compared with those from two separate univariate random-effects meta-analyses (URMAs), the traditional approach. RESULTS: The normal BRMA model estimates ρ(B )as -1 in both applied examples. Analytically we show this is due to the maximum likelihood estimator sensibly truncating the between-study covariance matrix on the boundary of its parameter space. Our simulations reveal this commonly occurs when the number of studies is small or the within-study variation is relatively large; it also causes upwardly biased between-study variance estimates, which are inflated to compensate for the restriction on [Formula: see text] (B). Importantly, this does not induce any systematic bias in the pooled estimates and produces conservative standard errors and mean-square errors. Furthermore, the normal BRMA is preferable to two normal URMAs; the mean-square error and standard error of pooled estimates is generally smaller in the BRMA, especially given data missing at random. For meta-analysis of proportions we then show that a generalised BRMA model is better still. This correctly uses a binomial rather than normal distribution, and produces better estimates than the normal BRMA and also two generalised URMAs; however the model may sometimes not converge due to difficulties estimating ρ(B). CONCLUSION: A BRMA model offers numerous advantages over separate univariate synthesises; this paper highlights some of these benefits in both a normal and generalised modelling framework, and examines the estimation of between-study correlation to aid practitioners