Educational interventions to improve people's understanding of key concepts in assessing the effects of health interventions: a systematic review

Abstract Background Health information is readily accessible but is of variable quality. General knowledge about how to assess whether claims about health interventions are trustworthy is not common, so people’s health decisions can be ill-informed, unnecessarily costly and even unsafe. This review aims to identify and evaluate studies of educational interventions designed to improve people’s understanding of key concepts for evaluating claims about the effects of health interventions. Methods/Design We searched multiple electronic databases and sources of grey literature. Inclusion criteria included all study types that included a comparison, any participants (except health professionals or health professional students) and educational interventions aimed at improving people’s understanding of one or more of the key concepts considered necessary for assessing health intervention claims. Knowledge and/or understanding of concepts or skills relevant to evaluating health information were our primary outcome measures. Secondary outcomes included behaviour, confidence, attitude and satisfaction with the educational interventions. Two authors independently screened search results, assessed study eligibility and risk of bias and extracted data. Results were summarised using descriptive synthesis. Results Among 24 eligible studies, 14 were randomised trials and 10 used other study designs. There was heterogeneity across study participants, settings and educational intervention type, content and delivery. The risk of bias was high in at least one domain for all randomised studies. Most studies measured outcomes immediately after the educational intervention, with few measuring later. In most of the comparisons, measures of knowledge and skills were better among those who had received educational interventions than among controls, and some of these differences were statistically significant. The effects on secondary outcomes were inconsistent. Conclusions Educational interventions to improve people’s understanding of key concepts for evaluating health intervention claims can improve people’s knowledge and skills, at least in the short term. Effects on confidence, attitude and behaviour are uncertain. Many of the studies were at moderate or greater risk of bias. Improvements in study quality, consistency of outcome measures and measures of longer-term effects are needed to improve confidence in estimates of the effects of educational interventions to improve people’s understanding of key concepts for evaluating health intervention claims. Systematic review registration PROSPERO CRD4201603310

Wikipedia editing and information literacy: A case study

Purpose: This paper aims to evaluate the success of a Wikipedia editing assessment designed to improve the information literacy skills of a cohort of first-year undergraduate health sciences students. Design/methodology/approach: In this action research case study (known hereafter as “the project” to differentiate this action research from the students’ own research), students researched, wrote and published Wikipedia articles on Australia-centric health topics. Students were given a pre- and post-test to assess levels of self-confidence in finding, evaluating and referencing information. Student work was also analysed in terms of article length and quantity and the type of information sources used. Findings: Tests revealed that students’ self-confidence in their information literacy skills improved overall. Analysis of student work revealed that students wrote longer articles and incorporated more references than expected. References used were of appropriate quality relevant to the article despite minimal instructions. Originality/value: There are few studies that investigate information literacy development through Wikipedia editing in Australian universities. This study shows that Wikipedia editing is an effective way to carry out student assessment prior to essay writing and an innovative platform to improve information literacy skills in undergraduate students

Use of the Research Readiness Self-Assessment (RRSA) to Evaluate Medical Students’ Competencies in Finding and Evaluating Online Health Information (Poster Presented at MHSLA Annual Conference)

To maintain awareness of current medical evidence and clinical practice guidelines in order to provide the best possible patient care, physicians must be able to locate, critically evaluate, synthesize, and make clinical decisions based on health information from multiple digital sources. Concerningly, however, medical students and junior doctors have been found to exhibit poor information literacy skills, including an inability to conduct efficient literature searches, find randomized controlled trials and systematic reviews, or evaluate study quality. Here, we describe our medical library’s experience in administering the Health version of the Research Readiness Self-Assessment (RRSA) tool to two cohorts of first-year medical students to evaluate their perceived and actual competencies in finding and critically evaluating online health information. We also describe our creation of online training modules that aim to improve medical students’ health information literacy skills, including one on using Bloom’s taxonomy to ask great research questions and another on deciding when to use various types of information sources (e.g., textbooks, journals, point-of-care clinical decision tools) to answer different types of health-related questions. Our preliminary findings suggest that first-year medical students are better at evaluating health information than they are at finding health information, although both skills show room for improvement. These findings will help guide our development of additional educational interventions to improve medical students’ research readiness

Principles and practice of public health surveillance

Public health surveillance is the systematic, ongoing assessment of the health of a community including the timely collection, analysis, interpretation, dissemination and subsequent use of data. The book presents an organized approach to planning, developing, implementing, and evaluating public health surveillance systems. Chapters include: planning; data sources; system management and data quality control; analyzing surveillance data; special statistical issues; communication; evaluation; ethical issues; legal issues; use of computers; state and local issues; and surveillance in developing countries. The book is intended to serve as a desk reference for public health practitioners and as a text for students in public health.PB9 3-10 1129I: Introduction -- II: Planning a surveillance system -- III: Sources of routinely collected data for surveillance -- IV: Management of the surveillance system and quality control of data -- V: Analyzing and interpreting surveillance data -- VI: Special analytic issues -- VII: Communicating information for action -- VIII: Evaluating public health surveillance -- IX: Ethical issues -- X:Public health surveillance and the law -- XI: Computerizing public health surveillance systems -- XII: State and local issues in surveillance -- XIII: Important surveillance issues in developing countries -- Tables and figures.1992874

Science and Technology Resources on the Internet Maps for Health Metrics: An Epidemiology Resource Webliography

Maps have a long history of being used as sources to track disease outbreaks, link causes and effects of disease, combat misinformation, present ideas and improve patient care. This webliography is a compendium of thematic maps, including health metrics, risk factors, infectious diseases, cancers, chronic diseases, and psychiatric disorders. Maps were gathered after evaluating data for reliability and currency. These selective epidemiology resources may aid public health professionals, medical practitioners, researchers, and librarians, who serve an information-seeking clientele interested in health-related quality of life

Evaluation of complex integrated care programmes: the approach in North West London

Background: Several local attempts to introduce integrated care in the English National Health Service have been tried, with limited success. The Northwest London Integrated Care Pilot attempts to improve the quality of care of the elderly and people with diabetes by providing a novel integration process across primary, secondary and social care organisations. It involves predictive risk modelling, care planning, multidisciplinary management of complex cases and an information technology tool to support information sharing. This paper sets out the evaluation approach adopted to measure its effect. Study design: We present a mixed methods evaluation methodology. It includes a quantitative approach measuring changes in service utilization, costs, clinical outcomes and quality of care using routine primary and secondary data sources. It also contains a qualitative component, involving observations, interviews and focus groups with patients and professionals, to understand participant experiences and to understand the pilot within the national policy context. Theory and discussion: This study considers the complexity of evaluating a large, multi-organisational intervention in a changing healthcare economy. We locate the evaluation within the theory of evaluation of complex interventions. We present the specific challenges faced by evaluating an intervention of this sort, and the responses made to mitigate against them. Conclusions: We hope this broad, dynamic and responsive evaluation will allow us to clarify the contribution of the pilot, and provide a potential model for evaluation of other similar interventions. Because of the priority given to the integrated agenda by governments internationally, the need to develop and improve strong evaluation methodologies remains strikingly important

Oral health services in prison settings: A global scoping review of availability, accessibility, and model of delivery

: This review aimed at evaluating the state of availability, accessibility and model of delivery of oral health services in prisons, globally. Five databases of peer-reviewed literature and potential sources of grey literature were systematically searched. Inclusion criteria encompassed oral health papers related to prisons globally, with exclusion of certain article types. Selection involved independent evaluations by two researchers, followed by quality assessment. Data on the availability of oral health interventions in prisons came from 18 countries, while information on the model of delivery of the services is scarce. In addition, two sets of individual and organizational barriers toward oral health service uptake in prisons were revealed and discussed in the text. Lack of oral health services in prisons affects people living in prisons and jeopardizes their reintegration. Urgent and concrete international actions are required to ensure the availability, accessibility, and quality of oral health services among people living in prisons

Are interventions for improving the quality of services provided by specialized drug shops effective in sub-Saharan Africa? A systematic review of the literature.

PURPOSE: We set out to determine effectiveness of interventions for improving the quality of services provided by specialized drug shops in sub-Saharan Africa. DATA SOURCES: We searched PubMed, CAB Abstracts, Web of Science, PsycINFO and Eldis databases and websites for organizations such as WHO and Management Sciences for Health. Finally, we searched manually through the references of retrieved articles. STUDY SELECTION: Our search strategy included randomized trials, time-series studies and before and after studies evaluating six interventions; education, peer review, reorganizing administrative structures, incentives, regulation and legislation. DATA EXTRACTION: We extracted information on design features, participants, interventions and outcomes assessed studies for methodological quality, and extracted results, all using uniform checklists. RESULTS OF DATA SYNTHESIS: We obtained 10 studies, all implementing educational interventions. Outcome measures were heterogeneous and included knowledge, communication and dispensing practices. Education improved knowledge across studies, but gave mixed results on communication between sellers and clients, dispensing of appropriate treatments and referring of patients to health facilities. Profit incentives appeared to constrain behaviour change in certain instances, although cases of shops adopting practices at the expense of sales revenue were also reported. CONCLUSION: Evidence suggests that knowledge and practices of pharmacies and drug shops can be improved across a range of diseases and countries/regions, although variations were reported across studies. Profit incentives appear to bear some influence on the level of success of interventions. More work is required to extend the geographical base of evidence, investigate cost-effectiveness and evaluate sustainability of interventions over periods longer than 1 year

Electronic immunization data collection systems: application of an evaluation framework

BACKGROUND: Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. METHODS: The Centers for Disease Control and Prevention’s Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. RESULTS: Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers’ immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. CONCLUSIONS: Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage

Identifying and appraising promising sources of UK clinical, health and social care data for use by NICE

This report aimed to aid the National Institute of Health and Care Excellence (NICE) in identifying opportunities for greater use of real-world data within its work. NICE identified five key ways in which real-world data was currently informing its work, or could do so in the future through: (i) researching the effectiveness of interventions or practice in real-world (UK) settings (ii) auditing the implementation of guidance (iii) providing information on resource use and evaluating the potential impact of guidance (iv) providing epidemiological information (v) providing information on current practice to inform the development of NICE quality standards. This report took a broad definition of ‘real-world’ data and created a map of UK sources, informed by a number of experts in real-world data, as well as a literature search, to highlight where some of the opportunities may lie for NICE within its clinical, public health and social care remit. The report was commissioned by the NICE, although the findings are likely to be of wider interest to a range of stakeholders interested in the role of real-world data in informing clinical, social care and public health decision-making. Most of the issues raised surrounding the use and appraisal of real-world data are likely to be generic, although the choice of datasets that were profiled in-depth reflected the interests of NICE. We discovered 275 sources that were named as real-world data sources for clinical, social care or public health investigation, 233 of which were deemed as active. The real-world data landscape therefore is highly complex and heterogeneous and composed of sources with different purposes, structures and collection methods. Some real-world data sources are purposefully either set-up or re-developed to enhance their data linkages and to examine the presence/absence/effectiveness of integrated patient care; however, such sources are in the minority. Furthermore, the small number of real-world data sources that are designed to enable the monitoring of care across providers, or at least have the capability to do so at a national level, have been utilised infrequently for this purpose in the literature. Data that offer the capacity to monitor transitions between health and social care do not currently exist at a national level, despite the increasing recognition of the interdependency between these sectors. Among the data sources we included, it was clear that no one data source represented a panacea for NICE’s real world data needs. This does highlight the merits and importance of data linkage projects and is suggestive of a need to triangulate evidence across different data, particularly in order to understand the feasibility and impact of guidance. There exists no overall catalogue or repository of real-world data sources for health, public health and social care, and previous initiatives aimed at creating such a resource have not been maintained. As much as there is a need for enhanced usage of the data, there is also a need for taking stock, integration, standardisation, and quality assurance of different sources. This research highlights a need for a systematic approach to creating an inventory of sources with detailed metadata and the funding to maintain this resource. This would represent an essential first step to support future initiatives aimed at enhancing the use of real-world data