A systematic review of the use of an expertise-based randomised controlled trial design

Acknowledgements JAC held a Medical Research Council UK methodology (G1002292) fellowship, which supported this research. The Health Services Research Unit, Institute of Applied Health Sciences (University of Aberdeen), is core-funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. Views express are those of the authors and do not necessarily reflect the views of the funders.Peer reviewedPublisher PD

Assessment of learning curves in complex surgical interventions: a consecutive case-series study

Background: Surgical interventions are complex, which complicates their rigorous assessment through randomised clinical trials. An important component of complexity relates to surgeon experience and the rate at which the required level of skill is achieved, known as the learning curve. There is considerable evidence that operator performance for surgical innovations will change with increasing experience. Such learning effects complicate evaluations; the start of the trial might be delayed, resulting in loss of surgeon equipoise or, if an assessment is undertaken before performance has stabilised, the true impact of the intervention may be distorted. Methods: Formal estimation of learning parameters is necessary to characterise the learning curve, model its evolution and adjust for its presence during assessment. Current methods are either descriptive or model the learning curve through three main features: the initial skill level, the learning rate and the final skill level achieved. We introduce a fourth characterising feature, the duration of the learning period, which provides an estimate of the point at which learning has stabilised. We propose a two-phase model to estimate formally all four learning curve features. Results: We demonstrate that the two-phase model can be used to estimate the end of the learning period by incorporating a parameter for estimating the duration of learning. This is achieved by breaking down the model into a phase describing the learning period and one describing cases after the final skill level is reached, with the break point representing the length of learning. We illustrate the method using cardiac surgery data. Conclusions: This modelling extension is useful as it provides a measure of the potential cost of learning an intervention and enables statisticians to accommodate cases undertaken during the learning phase and assess the intervention after the optimal skill level is reached. The limitations of the method and implications for the optimal timing of a definitive randomised controlled trial are also discussed

Key issues in recruitment to randomised controlled trials with very different interventions: a qualitative investigation of recruitment to the SPARE trial (CRUK/07/011)

<p>Abstract</p> <p>Background</p> <p>Recruitment to randomised controlled trials (RCTs) with very different treatment arms is often difficult. The ProtecT (Prostate testing for cancer and Treatment) study successfully used qualitative research methods to improve recruitment and these methods were replicated in five other RCTs facing recruitment difficulties. A similar qualitative recruitment investigation was undertaken in the SPARE (Selective bladder Preservation Against Radical Excision) feasibility study to explore reasons for low recruitment and attempt to improve recruitment rates by implementing changes suggested by qualitative findings.</p> <p>Methods</p> <p>In Phase I of the investigation, reasons for low levels of recruitment were explored through content analysis of RCT documents, thematic analysis of interviews with trial staff and recruiters, and conversation analysis of audio-recordings of recruitment appointments. Findings were presented to the trial management group and a plan of action was agreed. In Phase II, changes to design and conduct were implemented, with training and feedback provided for recruitment staff.</p> <p>Results</p> <p>Five key challenges to trial recruitment were identified in Phase I: (a) Investigators and recruiters had considerable difficulty articulating the trial design in simple terms; (b) The recruitment pathway was complicated, involving staff across different specialties/centres and communication often broke down; (c) Recruiters inadvertently used 'loaded' terminology such as 'gold standard' in study information, leading to unbalanced presentation; (d) Fewer eligible patients were identified than had been anticipated; (e) Strong treatment preferences were expressed by potential participants and trial staff in some centres. In Phase II, study information (patient information sheet and flowchart) was simplified, the recruitment pathway was focused around lead recruiters, and training sessions and 'tips' were provided for recruiters. Issues of patient eligibility were insurmountable, however, and the independent Trial Steering Committee advised closure of the SPARE trial in February 2010.</p> <p>Conclusions</p> <p>The qualitative investigation identified the key aspects of trial design and conduct that were hindering recruitment, and a plan of action that was acceptable to trial investigators and recruiters was implemented. Qualitative investigations can thus be used to elucidate challenges to recruitment in trials with very different treatment arms, but require sufficient time to be undertaken successfully.</p> <p>Trial Registration</p> <p>CRUK/07/011; <a href="http://www.controlled-trials.com/ISRCTN61126465">ISRCTN61126465</a></p

Developing a placebo-controlled trial in surgery:issues of design, acceptability and feasibility

BACKGROUND: Surgical placebos are controversial. This in-depth study explored the design, acceptability, and feasibility issues relevant to designing a surgical placebo-controlled trial for the evaluation of the clinical and cost effectiveness of arthroscopic lavage for the management of people with osteoarthritis of the knee in the UK. METHODS: Two surgeon focus groups at a UK national meeting for orthopaedic surgeons and one regional surgeon focus group (41 surgeons); plenary discussion at a UK national meeting for orthopaedic anaesthetists (130 anaesthetists); three focus groups with anaesthetists (one national, two regional; 58 anaesthetists); two focus groups with members of the patient organisation Arthritis Care (7 participants); telephone interviews with people on consultant waiting lists from two UK regional centres (15 participants); interviews with Chairs of UK ethics committees (6 individuals); postal surveys of members of the British Association of Surgeons of the Knee (382 surgeons) and members of the British Society of Orthopaedic Anaesthetists (398 anaesthetists); two centre pilot (49 patients assessed). RESULTS: There was widespread acceptance that evaluation of arthroscopic lavage had to be conducted with a placebo control if scientific rigour was not to be compromised. The choice of placebo surgical procedure (three small incisions) proved easier than the method of anaesthesia (general anaesthesia). General anaesthesia, while an excellent mimic, was more intrusive and raised concerns among some stakeholders and caused extensive discussion with local decision-makers when seeking formal approval for the pilot.Patients were willing to participate in a pilot with a placebo arm; although some patients when allocated to surgery became apprehensive about the possibility of receiving placebo, and withdrew. Placebo surgery was undertaken successfully. CONCLUSIONS: Our study illustrated the opposing and often strongly held opinions about surgical placebos, the ethical issues underpinning this controversy, and the challenges that exist even when ethics committee approval has been granted. It showed that a placebo-controlled trial could be conducted in principle, albeit with difficulty. It also highlighted that not only does a placebo-controlled trial in surgery have to be ethically and scientifically acceptable but that it also must be a feasible course of action. The place of placebo-controlled surgical trials more generally is likely to be limited and require specific circumstances to be met. Suggested criteria are presented. TRIAL REGISTRATION NUMBER: The trial was assigned ISRCTN02328576 through http://controlled-trials.com/ in June 2006. The first patient was randomised to the pilot in July 2007

IDEAL framework for surgical innovation 2: observational studies in the exploration and assessment stages.

The IDEAL framework describes the stages of evaluation for surgical innovations. This paper considers the role of observational studies in the exploration and assessment stages. At the exploration stage, the surgical intervention is usually more widely used, and observational studies should collect prospective data from multiple surgeons, deal with factors such as case mix and learning, and prepare for a definitive evaluation at the next stage of assessment. Although a randomised controlled trial is preferable, a high quality observational study would be acceptable if a randomised trial is not feasible or, on rare occasions, deemed unnecessary