Understanding reliability of the observer-reported communication ability measure within Angelman syndrome through the lens of generalizability theory

Abstract Aims Caregivers rate improved communication ability as one of the most desired outcomes for successful interventions for individuals with Angelman syndrome (AS). When measuring communication ability in clinical trials, the reliability of such measures is critical for detecting significant changes over time. This study examined the reliability of the Observed-Reported Communication Ability (ORCA) measure completed by caregivers of individuals with AS. Methods The ORCA measure was completed by 249 caregivers with 170 caregivers completing the ORCA measure again after 5–12 days. Generalizability theory was used to examine the following sources of measurement error in ORCA scores: concepts, subdomains, assessment points, and the interactions among those facets and the object of measurement: communication ability. Three generalizability studies were conducted to understand the reliability of the ORCA measure for different measurement designs. Decision studies were carried out to demonstrate the optimization of measurement procedures of the ORCA measure. Results G and Phi coefficients of the original measurement design exceeded the 0.80 threshold considered sufficiently reliable to make relative and absolute decisions about the communication ability of individuals with AS based on their caregivers’ observed scores. The optimization procedures indicated that increasing the number of communication concepts and/or assessment points leads to more reliable estimates of communication. Conclusion The ORCA measure was able to reliably distinguish different levels of communication ability among individuals with AS. Multiple assessment points and or more concepts would provide more precise estimates of an individual’s communication ability but at the cost of survey fatigue

Patient experience of spontaneous intracranial hypotension (SIH): qualitative interviews for concept elicitation

Abstract Background & objectives Spontaneous intracranial hypotension (SIH) is an underdiagnosed and debilitating condition caused by a spinal cerebrospinal fluid (CSF) leak. Although SIH can lead to substantial morbidity and disability, little data exists about patients’ perspectives. Without hearing directly from patients, our understanding of the full experience of having SIH is limited, as is our ability to identify and use appropriate patient-reported outcome measures (PROMs) within clinical care and research. The purpose of this study was to conduct qualitative interviews with confirmed SIH patients to fully describe their experiences and identify relevant concepts to measure. Methods Patients were recruited from an SIH specialty clinic at a large, U.S.-based healthcare center. Patients undergoing an initial consultation who were ≥ 18 years old, English-speaking, met the International Classification of Headache Disorders-3 criteria for SIH, and had a brain MRI with contrast that was positive for SIH were eligible to participate. During semi-structured qualitative interviews with a trained facilitator, participants were asked to describe their current SIH symptoms, how their experiences with SIH had changed over time, and the aspects of SIH that they found most bothersome. Analysts reviewed the data, created text summaries, and wrote analytic reports. Results Fifteen participants completed interviews. Common symptoms reported by patients included headache, tinnitus, ear fullness/pressure/pain, and neck or interscapular pain. Patients reported that their symptoms worsened over the course of their day and with activity. The most bothersome aspect of SIH was disruption to daily activities and limits to physical activities/exercise, which were severe. With regard to symptoms, the most bothersome and impactful included physical pain and discomfort (including headache), as well as fatigue. Conclusions Patients reported a diverse set of symptoms that were attributed to SIH, with devastating impacts on functioning and high levels of disability. Researchers considering use of PROMs for SIH should consider inclusion of both symptom scales and aspects of functioning, and future work should focus on evaluating the validity of existing measures for this patient population using rigorous qualitative and quantitative methods in diverse samples. Additionally, these data can be used to assist clinicians in understanding the impacts of SIH on patients

The future of the provision process for mobility assistive technology: a survey of providers

<p><b>Purpose:</b> The purpose of this study was to evaluate the opinions of providers of mobility assistive technologies to help inform a research agenda and set priorities.</p> <p><b>Materials and methods:</b> This survey study was anonymous and gathered opinions of individuals who participate in the process to provide wheelchairs and other assistive technologies to clients. Participants were asked to rank the importance of developing various technologies and rank items against each other in terms of order of importance. Participants were also asked to respond to several open-ended questions or statements.</p> <p><b>Results:</b> A total of 161 providers from 35 states within the USA consented to participation and completed the survey.</p> <p><b>Conclusions:</b> This survey revealed themes of advanced wheelchair design, assistive robotics and intelligent systems, human machine interfaces and smart device applications. It also outlined priorities for researchers to provide continuing education to clients and providers. These themes will be used to develop research and development priorities.Implications for Rehabilitation</p><p>• Research in advanced wheelchair design is needed to facilitate travel and environmental access with wheelchairs and to develop alternative power sources for wheelchairs.• New assistive robotics and intelligent systems are needed to help wheelchairs overcome obstacles or self-adjust, assist wheelchair navigation in the community, assist caregivers and transfers, and aid ambulation.• Innovations in human machine interfaces may help advance the control of mobility devices and robots with the brain, eye movements, facial gesture recognition or other systems.• Development of new smart devices is needed for better control of the environment, monitoring activity and promoting healthy behaviours.</p><p></p> <p>• Research in advanced wheelchair design is needed to facilitate travel and environmental access with wheelchairs and to develop alternative power sources for wheelchairs.• New assistive robotics and intelligent systems are needed to help wheelchairs overcome obstacles or self-adjust, assist wheelchair navigation in the community, assist caregivers and transfers, and aid ambulation.• Innovations in human machine interfaces may help advance the control of mobility devices and robots with the brain, eye movements, facial gesture recognition or other systems.• Development of new smart devices is needed for better control of the environment, monitoring activity and promoting healthy behaviours.</p

Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory

OBJECTIVE: We identified challenges and solutions to using electronic health record (EHR) systems for the design and conduct of pragmatic research. MATERIALS AND METHODS: Since 2012, the Health Care Systems Research Collaboratory has served as the resource coordinating center for 21 pragmatic clinical trial demonstration projects. The EHR Core working group invited these demonstration projects to complete a written semistructured survey and used an inductive approach to review responses and identify EHR-related challenges and suggested EHR enhancements. RESULTS: We received survey responses from 20 projects and identified 21 challenges that fell into 6 broad themes: (1) inadequate collection of patient-reported outcome data, (2) lack of structured data collection, (3) data standardization, (4) resources to support customization of EHRs, (5) difficulties aggregating data across sites, and (6) accessing EHR data. DISCUSSION: Based on these findings, we formulated 6 prerequisites for PCTs that would enable the conduct of pragmatic research: (1) integrate the collection of patient-centered data into EHR systems, (2) facilitate structured research data collection by leveraging standard EHR functions, usable interfaces, and standard workflows, (3) support the creation of high-quality research data by using standards, (4) ensure adequate IT staff to support embedded research, (5) create aggregate, multidata type resources for multisite trials, and (6) create re-usable and automated queries. CONCLUSION: We are hopeful our collection of specific EHR challenges and research needs will drive health system leaders, policymakers, and EHR designers to support these suggestions to improve our national capacity for generating real-world evidence

Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory

Embedded pragmatic clinical trials (ePCTs) play a vital role in addressing current population health problems, and their use of electronic health record (EHR) systems promises efficiencies that will increase the speed and volume of relevant and generalizable research. However, as the number of ePCTs using EHR-derived data grows, so does the risk that research will become more vulnerable to biases due to differences in data capture and access to care for different subsets of the population, thereby propagating inequities in health and the healthcare system. We identify 3 challenges-incomplete and variable capture of data on social determinants of health, lack of representation of vulnerable populations that do not access or receive treatment, and data loss due to variable use of technology-that exacerbate bias when working with EHR data and offer recommendations and examples of ways to actively mitigate bias