Development and evaluation of an instrument for the critical appraisal of randomized controlled trials of natural products

<p>Abstract</p> <p>Background</p> <p>The efficacy of natural products (NPs) is being evaluated using randomized controlled trials (RCTs) with increasing frequency, yet a search of the literature did not identify a widely accepted critical appraisal instrument developed specifically for use with NPs. The purpose of this project was to develop and evaluate a critical appraisal instrument that is sufficiently rigorous to be used in evaluating RCTs of conventional medicines, and also has a section specific for use with single entity NPs, including herbs and natural sourced chemicals.</p> <p>Methods</p> <p>Three phases of the project included: 1) using experts and a Delphi process to reach consensus on a list of items essential in describing the identity of an NP; 2) compiling a list of non-NP items important for evaluating the quality of an RCT using systematic review methodology to identify published instruments and then compiling item categories that were part of a validated instrument and/or had empirical evidence to support their inclusion and 3) conducting a field test to compare the new instrument to a published instrument for usefulness in evaluating the quality of 3 RCTs of a NP and in applying results to practice.</p> <p>Results</p> <p>Two Delphi rounds resulted in a list of 15 items essential in describing NPs. Seventeen item categories fitting inclusion criteria were identified from published instruments for conventional medicines. The new assessment instrument was assembled based on content of the two lists and the addition of a Reviewer's Conclusion section. The field test of the new instrument showed good criterion validity. Participants found it useful in translating evidence from RCTs to practice.</p> <p>Conclusion</p> <p>A new instrument for the critical appraisal of RCTs of NPs was developed and tested. The instrument is distinct from other available assessment instruments for RCTs of NPs in its systematic development and validation. The instrument is ready to be used by pharmacy students, health care practitioners and academics and will continue to be refined as required.</p

'Allocation concealment': the evolution and adoption of a methodological term.

Random assignment of individual participants in clinical trials entails two steps: (i) generating an unbiased treatment allocation schedule; and (ii) applying the schedule without foreknowledge of upcoming allocations. These two steps were implicit in the famous randomized trial of streptomycin for pulmonary tuberculosis in 1948, and were recognized explicitly in some early books on controlled trials. However, half a century later, no widely accepted term denoting the process of concealing upcoming allocations had been adopted. In 1983 Thomas Chalmers and colleagues termed that process “randomization blinding,” and showed that blinded randomization and unblinded randomization were associated with differing estimates of treatment effects; however, their terminology was subsequently rarely used. In the mid-1990s we suggested that the term “allocation concealment” would be preferable to “blinded randomization,” particularly to avoid terminology that might be confused with blinding of treatments after random allocation. After controlling for more factors than had been accounted for by Chalmers and colleagues, we demonstrated an association between allocation concealment and estimates of treatment effects. Moreover, as further indication of bias, inadequately concealed trials displayed more heterogeneity than adequately concealed trials. Notably, our modeling and methodological approach to examine the associations between trial quality and estimates of treatment effects has gained recognition and achieved replication. A PubMed search for the term “allocation concealment” between 1972 and 1993 in “any field” yielded no instances, compared with 1471 between 1995 and 2016. Google Scholar found 25 matches before 1994 and over 30,000 matches after. Although the term might still be improved to avoid occasional misconceptions about its meaning, we assume that it has been widely adopted by authors and editors because they find the term useful

The Alberta Health Technology Assessment (HTA) Ambassador Program: The Development of a Contextually Relevant, Multidisciplinary Clinical Practice Guideline for Non-specific Low Back Pain: A Review

Purpose: To describe the development of a contextually relevant multidisciplinary clinical practice guideline (CPG) for non-specific low back pain (LBP) and to discuss its value to the management of LBP and the practice of physiotherapy