Clinical Trial Results Summary for Laypersons: A User Testing Study

Objective: To apply “user testing” to maximize readability and acceptability of a Clinical Trial Results Laypersons Summary—a new European requirement. Methods: “User testing” (using questionnaire and semistructured interview) assessed whether people could find and understand key points. Findings were used to improve content and design, prior to retesting. Participants had a range of levels of health literacy and there was a higher education group. Participants accessed the summary on screen. In round 1 we tested 12 points of information. In round 2 a revised summary addressing round 1 findings was tested, leading to a third final version. Results: In round 1, 2 of 12 points of information did not reach the target and interviews raised further format and content issues (some distracting technical explanations and inability to find or understand the 2 main study purposes). These findings informed revisions for the version tested in round 2, with 2 different points not reaching the target (inclusion criteria relating to duration of seasonal allergies and how researchers found out about participants’ symptoms). Identified problems in both rounds were addressed and reflected in the final version. Despite improvements, participants did not consistently understand that summaries were intended for the public, or to only interpret results of single trials in the context of additional trials. All readers, including those with higher education, found the clear and straightforward language acceptable. Conclusions: Applying “user testing” resulted in a largely health-literate summary suitable for people across a range of backgrounds

Developing alternative over-the-counter medicine label formats: How do they compare when evaluated by consumers?

Background: In recent years, the Australian Therapeutic Goods Administration (TGA) has proposed implementing a standardized over-the-counter (OTC) medicine label. However, there were mixed consumer opinions regarding a label proposed in 2012 and limited evidence demonstrating the usability of the revised (2014) format. Objective: To develop and examine the usability of alternative OTC medicine label formats for standardization, and explore consumer perspectives on the labels. Materials and methods: Four alternative labels were developed for the exemplar medicine diclofenac. One was based on the Medicine Information label proposed by the TGA (‘Medicine Information’), one was based on the U.S. Drug Facts label (‘Drug Facts’), and two were based on suggestions proposed by consumers in the earlier needs analysis phase of this research (referred to as the ‘Medicine Facts’ and ‘Consumer Desires’ label formats). Five cohorts of 10 participants were recruited. Each cohort was assigned to user test one of the alternative labels or an existing label for a proprietary diclofenac product (which acted as a comparator) for diagnostic purposes. Each participant then provided feedback on all 5 labels. Each interview consisted of the administration of a user testing questionnaire, measuring consumers’ ability to find and understand key points of information, and a semi-structured interview exploring consumer perspectives. Results: Overall, all 4 alternative label formats supported consumers’ ability to find and understand key points. The existing comparator label was the poorer label with respect to participants’ ability to find and understand key points. Factors such as perceived usability, color, design, content, and/or content ordering impacted consumer preferences. The ‘Consumer Desires’ or ‘Drug Facts’ label formats were most often preferred by consumers for use as the standardized OTC label over the TGA proposed format. Conclusions: All alternative label formats demonstrated satisfactory usability and could be considered for use in OTC label standardization. User testing of OTC labels and consumer feedback received as part of the testing process can assist in the refinement of OTC labeling to ensure that implemented policies are evidence-based

Do Patients Use a Headline Section in a Leaflet to Find Key Information About Their Medicines? Findings From a User-Test Study

Background: In the European Union (EU), all medicines are mandated to be provided with a patient information leaflet (PIL). Many patients express concerns about the length and complexity of some PILs, and this can be a disincentive for patients to read the PILS. In order to address this, the UK’s regulatory body (Medicine and Healthcare products Regulatory Agency [MHRA]) suggested leaflets might include a headline section—information presented prominently at the beginning of a leaflet that summarizes key safety messages about a drug. Objective: To explore the extent to which readers used a headline section in a PIL, using a form of diagnostic testing called user-testing, which examines how readers find and understand key information. Methods: The study used a cross-sectional design to user-test a PIL with a headline section in a target sample of 20 participants. Participants were provided with an exemplar PIL, and the performance of the PIL was evaluated by a questionnaire and semistructured interview. Results: The results showed that a headline section was used just over one-third of the time (39%); 90% of participants used the headline section to find information when they initially began the user-test. The qualitative findings suggested that the participants valued the presence of the headline section. Conclusion: The research suggests there does not appear to be any negative impact from including a headline section in a PIL, and it is a technique that is highly valued by the consumers of medicines information

Consumer interpretation of ramipril and clopidogrel medication risk information – Implications for risk communication strategies

Purpose: Side effects and side-effect risk information can be provided using written medicine information. However, challenges exist in effectively communicating this information to consumers. This study aimed to explore broad consumer profiles relevant to ramipril and clopidogrel side-effect risk information interpretation. Methods: Three focus groups were conducted (n=18 consumers) exploring consumer perspectives, understanding and treatment decision making in response to ramipril and clopidogrel written medicine information leaflets containing side effects and side-effect risk information. All discussions were audio recorded, transcribed verbatim, and analyzed to explore consumer profiles pertaining to side-effect risk appraisal. Results: Three consumer profiles emerged: glass half-empty, glass half-full, and middle-of-the-road consumers, highlighting the influence of perceived individual susceptibility, interpretation of side-effect risk information, and interindividual differences, on consumers’ understanding of side-effect risk information. All profiles emphasized the importance of gaining an understanding of individual side-effect risk when taking medicines. Conclusion: Written side-effect risk information is not interpreted uniformly by consumers. Consumers formulated their own construct of individual susceptibility to side effects. Health care professionals should consider how consumers interpret side-effect risk information and its impact on medication use. Existing risk communication strategies should be evaluated in light of these profiles to determine their effectiveness in conveying information

User Testing to Improve Retrieval and Comprehension of Information in Guidelines to Improve Medicines Safety

Objective: The aim of the study was to investigate the effectiveness of user testing for improving healthcare professionals’ retrieval and comprehension of information in medicines guidelines. Methods: The United Kingdom’s Injectable Medicines Guide was selected as a case study. This gives guidance to nurses on preparing and administering intravenous medicines on hospital wards, in line with standard UK practice. Three rounds of user testing were completed with 10 hospital nurses per round, using the Injectable Medicines Guide for voriconazole and aminophylline. Participants used the guidelines to answer 17 questions related to the administration of these medicines. Answers were scored for “finding” and “understanding” the required information. Semistructured interviews explored participants’ opinions of guideline content, design, and wording, with responses analyzed thematically. The guidelines were revised between rounds. Results: In round 1, 8 of 17 questions were answered correctly by all participants. Participants had difficulty with dose, dilution, administration rate, and adverse effects questions. Revisions included new subsections and increased calculation support. In round 2, 14 of 17 questions were answered correctly by all participants. Difficulty persisted with dose and administration rate questions and further revisions made. In round 3, 15 of 17 questions were answered correctly by all participants. Across all rounds, participants considered appropriate subheadings and information order as important for fast location of information. Specific, detailed, and practical instructions were perceived as important to improve understandability and usefulness. Conclusions: Key information in medicines guidelines may not be found and/or understood by healthcare professionals. User testing increased information retrieval and comprehension and could have an important role in improving the safety of medicines use

European Public Assessment Report (EPAR) summaries for the public: are they fit for purpose? A user-testing study.

OBJECTIVES: Apply 'user testing' methodology to test the readability of a European Public Assessment Report (EPAR) summary-which describes how the decision was made by the European Medicines Agency to approve a medicine. DESIGN: User testing uses mixed methods (questionnaire and semistructured interview), applied iteratively, to assess document performance-can people find and understand key points of information. SETTING AND PARTICIPANTS: Testing was undertaken with 40 members of the public in four consecutive rounds of 10. Inclusion criteria, matched across rounds, included range of ages and educational attainment. TESTED DOCUMENTS: In round 1 we tested 19 key points of information in a printed version of the EPAR summary for Bondronat (a cancer medicine). This was then revised to address the findings, and tested in round 2. In round 3 we tested the summary on-screen, and in round 4, tested a revised on-screen version, after addressing findings from both rounds 1 and 3. PRIMARY OUTCOME MEASURE: The target followed European guidance for medicine leaflets: for each point of information 90% of participants should be able to find, and of those, 90% able to show understanding of the point. RESULTS: For the original EPAR summary, 6 of the 19 points of information reached the target (both paper-based and on-screen). After revisions to format and content, using good practice in information writing and design, 14 and 16 points, respectively, met the target. The problems related to both finding (dependent on layout, headings and design) and understanding (words and sentences used, as well as design). We devised a new heading structure, increased use of bullet points, replaced difficult and technical words and divided long sentences. CONCLUSIONS: People had difficulty finding and understanding key messages in the summary, but user testing identified the problems, and application of good practice resulted in a revised format which performed well

Patients’ use of information about medicine side effects in relation to experiences of suspected adverse drug reactions

Background Adverse drug reactions (ADRs) are common, and information about medicines is increasingly widely available to the public. However, relatively little work has explored how people use medicines information to help them assess symptoms that may be suspected ADRs. Objective Our objective was to determine how patients use patient information leaflets (PILs) or other medicines information sources and whether information use differs depending on experiences of suspected ADRs. Method This was a cross-sectional survey conducted in six National Health Service (NHS) hospitals in North West England involving medical in-patients taking at least two regular medicines prior to admission. The survey was administered via a questionnaire and covered use of the PIL and other medicines information sources, perceived knowledge about medicines risks/ADRs, experiences of suspected ADRs, plus demographic information. Results Of the 1,218 respondents to the survey, 18.8 % never read the PIL, whilst 6.5 % only do so if something unexpected happens. Educational level was related to perceived knowledge about medicines risks, but not to reading the PIL or seeking further information about medicines risks. Over half the respondents (56.0 %) never sought more information about possible side effects of medicines. A total of 57.2 % claimed they had experienced a suspected ADR. Of these 85.9 % were either very sure or fairly sure this was a reaction to a medicine. Over half of those experiencing a suspected ADR (53.8 %) had read the PIL, of whom 36.2 % did so before the suspected ADR occurred, the remainder afterwards. Reading the PIL helped 84.8 % of these respondents to decide they had experienced an ADR. Educational level, general knowledge of medicines risks and number of regular medicines used all increased the likelihood of experiencing an ADR. Conclusion More patients should be encouraged to read the PIL supplied with medicines. The results support the view that most patients feel knowledgeable about medicines risks and suspected ADRs and value information about side effects, but that reading about side effects in PILs or other medicines information sources does not lead to experiences of suspected ADRs

Dataset for "User testing to improve retrieval and comprehension of information in guidelines to improve medicines safety"

This dataset describes a series of user tests designed to improve the retrieval and comprehension of information by nurses from the NHS Injectable Medicines Guide. It includes the research protocol, data collection tools, interview transcriptions and thematic analysis, participant demographics and numerical outcome data.Nvivo version 11 used for thematic analysis

User-testing guidelines to improve the safety of intravenous medicines administration: a randomised in situ simulation study

Background: User-testing and subsequent modification of clinical guidelines increases health professionals’ information retrieval and comprehension. No study has investigated whether this results in safer care. Objective: To compare the frequency of medication errors when administering an intravenous medicine using the current National Health Service Injectable Medicines Guide (IMG) versus an IMG version revised with user-testing. Method: Single-blind, randomised parallel group in situ simulation. Participants were on-duty nurses/midwives who regularly prepared intravenous medicines. Using a training manikin in their clinical area, participants administered a voriconazole infusion, a high-risk medicine requiring several steps to prepare. They were randomised to use current IMG guidelines or IMG guidelines revised with user-testing. Direct observation was used to time the simulation and identify errors. Participant confidence was measured using a validated instrument. The primary outcome was the percentage of simulations with at least one moderate-severe IMG-related error, with error severity classified by an expert panel. Results: In total, 133 participants were randomised to current guidelines and 140 to user-tested guidelines. Fewer moderate-severe IMG-related errors occurred with the user-tested guidelines (n=68, 49%) compared with current guidelines (n=79, 59%), but this difference was not statistically significant (risk ratio: 0.82; 95% CI 0.66 to 1.02). Significantly more simulations were completed without any IMG-related errors with the user-tested guidelines (n=67, 48%) compared with current guidelines (n=26, 20%) (risk ratio: 2.46; 95% CI 1.68 to 3.60). Median simulation completion time was 1.6 min (95% CI 0.2 to 3.0) less with the user-tested guidelines. Participants who used user-tested guidelines reported greater confidence. Conclusion: User-testing injectable medicines guidelines reduces the number of errors and the time taken to prepare and administer intravenous medicines, while increasing staff confidence. Trial registration number: researchregistry5275

User-testing guidelines to improve the safety of intravenous medicines administration: a randomised in situ simulation study

Background: User-testing and subsequent modification of clinical guidelines increases health professionals’ information retrieval and comprehension. No study has investigated whether this results in safer care. Objective: To compare the frequency of medication errors when administering an intravenous medicine using the current National Health Service Injectable Medicines Guide (IMG) versus an IMG version revised with user-testing. Method: Single-blind, randomised parallel group in situ simulation. Participants were on-duty nurses/midwives who regularly prepared intravenous medicines. Using a training manikin in their clinical area, participants administered a voriconazole infusion, a high-risk medicine requiring several steps to prepare. They were randomised to use current IMG guidelines or IMG guidelines revised with user-testing. Direct observation was used to time the simulation and identify errors. Participant confidence was measured using a validated instrument. The primary outcome was the percentage of simulations with at least one moderatesevere IMG-related error, with error severity classified by an expert panel. Results: In total, 133 participants were randomised to current guidelines and 140 to user-tested guidelines. Fewer moderate-severe IMG-related errors occurred with the user-tested guidelines (n=68, 49%) compared with current guidelines (n=79, 59%), but this difference was not statistically significant (risk ratio: 0.82; 95% CI 0.66 to 1.02). Significantly more simulations were completed without any IMG-related errors with the usertested guidelines (n=67, 48%) compared with current guidelines (n=26, 20%) (risk ratio: 2.46; 95% CI 1.68 to 3.60). Median simulation completion time was 1.6min (95% CI 0.2 to 3.0) less with the user-tested guidelines. Participants who used user-tested guidelines reported greater confidence. Conclusion: User-testing injectable medicines guidelines reduces the number of errors and the time taken to prepare and administer intravenous medicines, while increasing staff confidence. Trial registration: number researchregistry5275