Identifying potential indicators to measure the outcome of translational cancer research: a mixed methods approach

International audienceAbstractBackgroundIn a context where there is an increasing demand to evaluate the outcome of bio-medical research, our work aims to develop a set of indicators to measure the impact of translational cancer research. The objective of our study was to explore the scope and issues of translational research relevant to evaluation, explore the views of researchers on the evaluation of oncological translational research, and select indicators measuring the outcomes and outputs of translational research in oncology by consensus.MethodsSemi-structured interviews amongst 23 researchers involved in translational cancer research were conducted and analysed using thematic analysis. A two-round modified Delphi survey of 35 participants with similar characteristics was then performed followed by a physical meeting. Participants rated the feasibility and validity of 60 indicators. The physical meeting was held to discuss the methodology of the new indicators.ResultsThe main themes emerging from the interviews included a common definition for translational research but disagreements about the exact scope and limits of this research, the importance of multidisciplinarity and collaboration for the success of translational research, the disadvantages that translational research faces in current evaluation systems, the relative lack of pertinence of existing indicators, and propositions to measure translational cancer research in terms of clinical applications and patient outcomes. A total of 35 participants took part in the first round survey and 12 in the second round. The two-round survey helped us select a set of 18 indicators, including four that seemed to be particularly adapted to measure translational cancer research impact on health service research (number of biomarkers identified, generation of clinical guidelines, citation of research in clinical guidelines, and citation of research in public health guidelines). The feedback from participants helped refine the methodology and definition of indicators not commonly used.ConclusionIndicators need to be accepted by stakeholders under evaluation. This study helped the selection and refinement of indicators considered as the most relevant by researchers in translational cancer research. The feasibility and validity of those indicators will be tested in a scientometric study

A Delphi process to optimize quality and performance of drug evaluation in neonates

Background Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. The objective of this work was to identify the major criteria considered necessary for selecting neonatal intensive care units that are able to perform drug evaluations competently. Methodology and Main Findings This Delphi process was conducted with an international multidisciplinary panel of 25 experts from 13 countries, selected to be part of two committees (a scientific committee and an expert committee), in order to validate criteria required to perform drug evaluation in neonates. Eighty six items were initially selected and classified under 7 headings: “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6). Sixty-one items were retained and headings were rearranged after the first round, 34 were selected after the second round. A third round was required to validate 13 additional items. The final set includes 47 items divided under 5 headings. Conclusion A set of 47 relevant criteria will help to NICUs that want to implement, conduct or participate in drug trials within a neonatal network identify important issues to be aware of. Summary Points 1) Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. 2) The present Delphi study was conducted with an international multidisciplinary panel of 25 experts from 13 countries and aims to identify the major criteria considered necessary for selecting neonatal intensive care units (NICUs) that are able to perform drug evaluations competently. 3) Of the 86 items initially selected and classified under 7 headings - “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6) - 47 items were selected following a three rounds Delphi process. 4) The present consensus will help NICUs to implement, conduct or participate in drug trials within a neonatal network

Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review

OBJECTIVE: Delphi technique is a structured process commonly used to developed healthcare quality indicators, but there is a little recommendation for researchers who wish to use it. This study aimed 1) to describe reporting of the Delphi method to develop quality indicators, 2) to discuss specific methodological skills for quality indicators selection 3) to give guidance about this practice. METHODOLOGY AND MAIN FINDING: Three electronic data bases were searched over a 30 years period (1978-2009). All articles that used the Delphi method to select quality indicators were identified. A standardized data extraction form was developed. Four domains (questionnaire preparation, expert panel, progress of the survey and Delphi results) were assessed. Of 80 included studies, quality of reporting varied significantly between items (9% for year's number of experience of the experts to 98% for the type of Delphi used). Reporting of methodological aspects needed to evaluate the reliability of the survey was insufficient: only 39% (31/80) of studies reported response rates for all rounds, 60% (48/80) that feedback was given between rounds, 77% (62/80) the method used to achieve consensus and 57% (48/80) listed quality indicators selected at the end of the survey. A modified Delphi procedure was used in 49/78 (63%) with a physical meeting of the panel members, usually between Delphi rounds. Median number of panel members was 17(Q1:11; Q3:31). In 40/70 (57%) studies, the panel included multiple stakeholders, who were healthcare professionals in 95% (38/40) of cases. Among 75 studies describing criteria to select quality indicators, 28 (37%) used validity and 17(23%) feasibility. CONCLUSION: The use and reporting of the Delphi method for quality indicators selection need to be improved. We provide some guidance to the investigators to improve the using and reporting of the method in future surveys

Méthodes de sélection et de suivi d'indicateurs en Santé (application en obstétrique)

PARIS7-Bibliothèque centrale (751132105) / SudocSudocFranceF

Quality indicators for continuous monitoring to improve maternal and infant health in maternity departments: a modified Delphi survey of an international multidisciplinary panel.

OBJECTIVE: Measuring the quality of inpatient obstetrical care using quality indicators is becoming increasingly important for both patients and healthcare providers. However, there is no consensus about which measures are optimal. We describe a modified Delphi method to identify a set of indicators for continuously monitoring the quality of maternity care by healthcare professionals. METHODOLOGY AND MAIN FINDINGS: An international French-speaking multidisciplinary panel comprising 22 obstetricians-gynaecologists, 12 midwives, and 1 paediatrician assessed potential indicators extracted from a medical literature search, using a two-round Delphi procedure followed by a physical meeting. Each panellist rated each indicator based on validity and feasibility. In the first round, 35 panellists from 5 countries and 20 maternity units evaluated 26 indicators including 15 related to the management of the overall population of pregnant women, 3 to the management of women followed from the first trimester of pregnancy, 2 to the management of low-risk pregnant women, and 6 to the management of neonates. 25 quality indicators were kept for next step. In the second round, 27 (27/35: 77%) panellists selected 17 indicators; the remaining 8 indicators were discussed during a physical meeting. The final set comprised 18 indicators. CONCLUSION: A multidisciplinary panel selected indicators that reflect the quality of obstetrical care. This set of indicators could be used to assess and monitor obstetrical care, with the goal of improving the quality of care in maternity units

POPI (Pediatrics: Omission of Prescriptions and Inappropriate prescriptions): development of a tool to identify inappropriate prescribing.

INTRODUCTION:Rational prescribing for children is an issue for all countries and has been inadequately studied. Inappropriate prescriptions, including drug omissions, are one of the main causes of medication errors in this population. Our aim is to develop a screening tool to identify omissions and inappropriate prescriptions in pediatrics based on French and international guidelines. METHODS:A selection of diseases was included in the tool using data from social security and hospital statistics. A literature review was done to obtain criteria which could be included in the tool called POPI. A 2-round-Delphi consensus technique was used to establish the content validity of POPI; panelists were asked to rate their level of agreement with each proposition on a 9-point Likert scale and add suggestions if necessary. RESULTS:108 explicit criteria (80 inappropriate prescriptions and 28 omissions) were obtained and submitted to a 16-member expert panel (8 pharmacists, 8 pediatricians hospital-based -50%- or working in community -50%-). Criteria were categorized according to the main physiological systems (gastroenterology, respiratory infections, pain, neurology, dermatology and miscellaneous). Each criterion was accompanied by a concise explanation as to why the practice is potentially inappropriate in pediatrics (including references). Two round of Delphi process were completed via an online questionnaire. 104 out of the 108 criteria submitted to experts were selected after 2 Delphi rounds (79 inappropriate prescriptions and 25 omissions). DISCUSSION CONCLUSION:POPI is the first screening-tool develop to detect inappropriate prescriptions and omissions in pediatrics based on explicit criteria. Inter-user reliability study is necessary before using the tool, and prospective study to assess the effectiveness of POPI is also necessary

Exploring how non-inferiority and equivalence are assessed in paediatrics: a systematic review

International audienc

Construction of reference criteria to admit patients to intermediate care units in France: a Delphi survey of intensivists, anaesthesiologists and emergency medicine practitioners (first part of the UNISURC project)

Objectives No consensus criteria describe the medical eligibility of the patients to intermediate care units (IMCUs). In this first part of the UNISURC project, we aimed to develop criteria based on a consensus of physicians from the main specialties involved in IMCU admission decisions.Design We selected criteria from IMCU literature, scoring systems and intensive care unit nursing workload. We submitted these criteria to a panel of experts in a Delphi survey. We used a two-round Delphi survey procedure to assess the validity and feasibility of each criterion.Setting Medical practitioners in either public or private French institutions and proposed by the national scientific societies of anaesthesiology, emergency medicine and intensive care. The Delphi rounds took place in 2015–2016.Outcome measures Validity and feasibility of the proposed criteria; uniformity of the judgement across the primary specialty and the hospital category of the responders.Results The criteria submitted to vote were classified as one of: chronic factor (CF); acute factor (AF); specific pathway (SP); nursing activity (NA) and hospital environment (HE). Of 189 experts invited, 81 (41%) responded to the first round and 62 of them (76%) responded to the second round. A definite selection of 63 items was made, distributed across 6 CF, 18 AF, 31 SP, 3 NA and 5 HE. Validity and feasibility were influenced by the specialty or the public/private status of the institution of the responders for a few items.Conclusion We created a set of 63 consensus criteria with acceptable validity and feasibility to assess the medical eligibility of the patients to IMCUs. The second part of the UNISURC project will assess the distribution of each criterion in a prospective multicentre national cohort.Trial registration number NCT02590172

Validity and feasibility scores assigned during the first and second Delphi rounds.

<p>From 1 to 26: indicators included in the first round.</p><p>From 27 to 32: new indicators added by panellists for evaluation in the second round.</p><p>“Status: included” means that the indicator was included in the second round without any change; “status: modified” means the indicator was included in the second round after being modified; “status: deleted” means the indicator was not included in the second round; and “status: added” means the indicator was added for inclusion in the second round on the suggestion of the panellists.</p><p><b>#21</b>. Brachial plexus palsy was rated as valid but was not selected after discussion with the researchers and a review of the comments by the panellists. It was considered relatively rare and ill-suited to continuous CUSUM-chart monitoring.</p><p><b>#24</b>., <b>#25</b>., <b>#26</b>. Apgar to 9 at 5 min was considered to high and not relevant for neonatal outcomes.</p><p>ICU: intensive care unit; NICU: neonatal intensive care unit; Wk: weeks of amenorrhoea.</p