QualDash: Adaptable Generation of Visualisation Dashboards for Healthcare Quality Improvement

YesAdapting dashboard design to different contexts of use is an open question in visualisation research. Dashboard designers often seek to strike a balance between dashboard adaptability and ease-of-use, and in hospitals challenges arise from the vast diversity of key metrics, data models and users involved at different organizational levels. In this design study, we present QualDash, a dashboard generation engine that allows for the dynamic configuration and deployment of visualisation dashboards for healthcare quality improvement (QI). We present a rigorous task analysis based on interviews with healthcare professionals, a co-design workshop and a series of one-on-one meetings with front line analysts. From these activities we define a metric card metaphor as a unit of visual analysis in healthcare QI, using this concept as a building block for generating highly adaptable dashboards, and leading to the design of a Metric Specification Structure (MSS). Each MSS is a JSON structure which enables dashboard authors to concisely configure unit-specific variants of a metric card, while offloading common patterns that are shared across cards to be preset by the engine. We reflect on deploying and iterating the design of QualDash in cardiology wards and pediatric intensive care units of five NHS hospitals. Finally, we report evaluation results that demonstrate the adaptability, ease-of-use and usefulness of QualDash in a real-world scenario

Planning the Radiology Workforce for Cancer Diagnostics

YesThe publication of the Delivery plan for tackling the COVID-10 backlog of elective care (NHSE/I, 2022:5) contained a number of ambitions, including that, by March 2024, 75% of patients who have been urgently referred by their GP for suspected cancer are diagnosed or have had cancer ruled out within 28 days. By March 2025, waits of longer than a year for elective care should be eliminated and 95% of patients needing a diagnostic test should receive it within six weeks. The report acknowledged the need to grow the workforce to achieve these ambitions and ensure a timely cancer diagnosis, while also proposing the use of digital technology and data systems to free up capacity. To assist West Yorkshire National Health Service (NHS) organisations to meet these ambitions, this report presents the findings of a ‘deep dive’ that focuses on the role of radiology in meeting the ambitions of providing timely cancer diagnosis. Aims 1. To understand current and projected demand for radiology expertise in cancer diagnosis in West Yorkshire. 2. To understand the current and projected radiology workforce in West Yorkshire and determine the gap between the projected radiology workforce and the required radiology workforce. 3. To identify possible solutions to assist in providing the radiology workforce required for West Yorkshire and explore their acceptability and potential impact. Methods A range of sources of data and methods were utilised. We examined publicly available quantitative data concerning cancer waiting times and diagnostic waiting times and activity and used this to forecast future cancer waiting times and diagnostic waiting times and activity. We examined data from Health Education England (HEE) regarding radiologists’ and radiographers’ workforce profile data for West Yorkshire, the number of radiologists completing training, and the number of radiographers graduating, and data submitted by West Yorkshire Trusts to HEE regarding their plans for growing their radiology and radiographer workforce. Interviews (N=15) conducted with radiology service managers, university academics and key strategic and operational stakeholders delivering radiology services were used to understand the current and future issues around strategic workforce planning, workforce changes and transformation, workforce roles and skills, training and education and service changes. A rapid review of the literature examining the impacts of artificial intelligence (AI) on the workload of radiology services was also undertaken. To put this work in context, we also reviewed relevant policy documents and reports. Alongside this, we consulted with the Yorkshire Imaging Collaborative (YIC) and the West Yorkshire Cancer Alliance (WYCA) and attended a series of workshops run by the Yorkshire Imaging Collaborative. Results Overall, the findings show that demand for radiology services is increasing and that both cancer waiting times and the waiting times for diagnostic tests increased, with a concurrent downward trend in activity that, if all else stays the same, is forecast to continue up to 2025. The cancer waiting times data indicate that patients were waiting longer and that their needs were not being met. Moreover, 3 the proportion of people treated within accepted cancer waiting times decreased both nationally and within the West Yorkshire region from 2013. This was exacerbated by COVID-19 which caused a further decrease nationally and for the West Yorkshire region. National data for waiting times for all diagnostic tests show a significant decline between 2006 and 2008, with a decrease in median waiting times from just under 6.0 weeks to approximately 2.0 weeks. Overall, waiting times remained stable until late 2020 when they started to rise with the longest median waiting times at just over 8.0 weeks in mid-2020. The total number of people waiting for radiology tests nationally is decreasing and is predicted to continue to do so, while in West Yorkshire the number of people waiting for radiology tests decreased until 2020 but has since been on an upward trend which is predicted to continue. Nationally, the total number of radiology tests is on an upward trend that is predicted to continue, while in West Yorkshire activity has been decreasing since well before COVID-19 and is predicted to continue to do so. Data examining the current and future workforce showed that the national figures for the total radiology and radiography workforce are small relative to other health professional groups. In West Yorkshire, 265 radiologists and 926 radiographers were employed, and staff turnover was generally low. Trusts’ forecasts for the number of radiologists and radiographers they believe they need suggest a 16% increase in the number of radiologists in post between March 2022 and March 2027 and a 25% increase in the number of radiographers in post. The numbers of radiographers and radiologists being trained in West Yorkshire suggest that this is feasible. Interview data identified a number of main themes and associated issues: delivering diagnostic cancer targets, strategic workforce planning, workforce roles and skills, service transformation, recruitment and retention, universities, artificial intelligence, collaboration, and international recruitment. Across all themes, some reoccurring issues were identified: a lack of staff, increased demands, a lack of capacity in terms of space and staff, a lack of strategic workforce planning with a focus on operational or financial plans. Respondents proposed potential solutions to some of the issues raised that included: new ways of working, upskilling, developing current and emerging roles, Community Diagnostic Centres (CDCs), greater collaboration between NHS Trusts, universities, CDCs, imaging academies and networks and the private sector, and the international recruitment of radiologists and radiographers to address workforce gaps. The rapid review findings helped to identify a number of potential benefits of use of AI in radiology, including contributing to improved workflow efficacy and efficiency of radiology services. However, this is dependent on the nature of the work and the AI function. As a result of faster AI reading, radiologists may be able to focus more on high-risk, complex reading tasks. AI can support automation of image segmentation and classification and aid the diagnostic confidence of less experienced radiologists. Respondents’ views on AI were mixed. There was acknowledgement that AI was already used to support radiology service delivery and both the benefits and problems associated were identified. The implications of AI for radiologists’ and radiographers’ roles were discussed in terms of changing work, AI being used to support or in some cases substitute radiologists and radiographers, and the need for the radiology workforce to adapt to the technological change whilst maintaining a caring servic

Institutional use of National Clinical Audits by healthcare providers

Rationale, aims, and objectives Healthcare systems worldwide devote significant resources towards collecting data to support care quality assurance and improvement. In the United Kingdom, National Clinical Audits are intended to contribute to these objectives by providing public reports of data on healthcare treatment and outcomes, but their potential for quality improvement in particular is not realized fully among healthcare providers. Here, we aim to explore this outcome from the perspective of hospital boards and their quality committees: an under‐studied area, given the emphasis in previous research on the audits' use by clinical teams. Methods We carried out semi‐structured, qualitative interviews with 54 staff in different clinical and management settings in five English National Health Service hospitals about their use of NCA data, and the circumstances that supported or constrained such use. We used Framework Analysis to identify themes within their responses. Results We found that members and officers of hospitals' governing bodies perceived an imbalance between the benefits to their institutions from National Clinical Audits and the substantial resources consumed by participating in them. This led some to question the audits' legitimacy, which could limit scope for improvements based on audit data, proposed by clinical teams. Conclusions Measures to enhance the audits' perceived legitimacy could help address these limitations. These include audit suppliers moving from an emphasis on cumulative, retrospective reports to real‐time reporting, clearly presenting the “headline” outcomes important to institutional bodies and staff. Measures may also include further negotiation between hospitals, suppliers and their commissioners about the nature and volume of data the latter are expected to collect; wider use by hospitals of routine clinical data to populate audit data fields; and further development of interactive digital technologies to help staff explore and report audit data in meaningful ways

A scoping review: Strategic workforce planning in health and social care

YesAim This aim of this scoping review was to undertake a detailed review of the pertinent literature examining strategic workforce planning in the health and social care sectors. The scoping review was tasked to address the following three questions: 1. How is strategic health and social care workforce planning currently undertaken? 2. What models, methods, and tools are available for supporting strategic health and social care workforce planning? 3. What are the most effective methods for strategic health and social care workforce planning? Methods The scoping review utilised the five-stage scoping review framework proposed by Arksey and O’Malley (2005). This includes identifying the research question; identifying relevant studies; study selection; charting the data and collating, summarizing, and reporting the results. The search included a range of databases and key search terms included “workforce” OR “human resource*” OR “personnel” OR “staff*”. Relevant documents were selected through initially screening titles and abstracts, followed by full text screening of potentially relevant documents. Results The search returned 6105 unique references. Based on title and abstract screening, 654 were identified as potentially relevant. Screening of full texts resulted in 115 items of literature being included in the synthesis. Both national and international literature covers strategic workforce planning, with all continents represented, but with a preponderance from high income nations. The emphasis in the literature is mainly on the healthcare workforce, with few items on social care. Medical and dental workforces are the predominate groups covered in the literature, although nursing and midwifery are also discussed. Other health and social care workers are less represented. A variety of categories of workforce planning methods are noted in the literature that range from determining the workforce using supply and demand, practitioner to population ratios, needs based approach, the utilisation of methods such as horizon scanning, modelling, and scenario planning, together with mathematical and statistical modelling. Several of the articles and websites include specific workforce planning models that are nationally and internationally recognised, e.g., the workload indicators of staffing needs (WISN), Star model and the Six Step Methodology. These models provide a series of steps to help with workforce planning and tend to take a more strategic view of the process. Some of the literature considers patient safety and quality in relation to safe staffing numbers and patient acuity. The health and social care policies reviewed include broad actions to address workforce planning, staff shortages or future service developments and advocate a mixture of developing new roles, different ways of working, flexibility, greater integrated working and enhanced used of digital technology. However, the policies generally do not include workforce models or guidance about how to achieve these measures. Overall, there is an absence in the literature of studies that evaluate what are the most effective methods for strategic health and social care planning. Recommendations The literature suggests the need for the implementation of a strategic approach to workforce planning, utilising a needs-based approach, including horizon scanning and scenarios. This could involve adoption of a recognised workforce planning model that incorporates the strategic elements required for workforce planning and a ‘one workforce’ approach across health and social care

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

YesDashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard's perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement.This research was funded by the National Institute for Health Research Health Services and Delivery Research Program (project #16/04/06)

Design and evaluation of an interactive quality dashboard for national clinical audit data: a realist evaluation

YesBackground: National audits aim to reduce variations in quality by stimulating quality improvement. However, varying provider engagement with audit data means that this is not being realised. Aim: The aim of the study was to develop and evaluate a quality dashboard (i.e. QualDash) to support clinical teams’ and managers’ use of national audit data. Design: The study was a realist evaluation and biography of artefacts study. Setting: The study involved five NHS acute trusts. Methods and results: In phase 1, we developed a theory of national audits through interviews. Data use was supported by data access, audit staff skilled to produce data visualisations, data timeliness and quality, and the importance of perceived metrics. Data were mainly used by clinical teams. Organisational-level staff questioned the legitimacy of national audits. In phase 2, QualDash was co-designed and the QualDash theory was developed. QualDash provides interactive customisable visualisations to enable the exploration of relationships between variables. Locating QualDash on site servers gave users control of data upload frequency. In phase 3, we developed an adoption strategy through focus groups. ‘Champions’, awareness-raising through e-bulletins and demonstrations, and quick reference tools were agreed. In phase 4, we tested the QualDash theory using a mixed-methods evaluation. Constraints on use were metric configurations that did not match users’ expectations, affecting champions’ willingness to promote QualDash, and limited computing resources. Easy customisability supported use. The greatest use was where data use was previously constrained. In these contexts, report preparation time was reduced and efforts to improve data quality were supported, although the interrupted time series analysis did not show improved data quality. Twenty-three questionnaires were returned, revealing positive perceptions of ease of use and usefulness. In phase 5, the feasibility of conducting a cluster randomised controlled trial of QualDash was assessed. Interviews were undertaken to understand how QualDash could be revised to support a region-wide Gold Command. Requirements included multiple real-time data sources and functionality to help to identify priorities. Conclusions: Audits seeking to widen engagement may find the following strategies beneficial: involving a range of professional groups in choosing metrics; real-time reporting; presenting ‘headline’ metrics important to organisational-level staff; using routinely collected clinical data to populate data fields; and dashboards that help staff to explore and report audit data. Those designing dashboards may find it beneficial to include the following: ‘at a glance’ visualisation of key metrics; visualisations configured in line with existing visualisations that teams use, with clear labelling; functionality that supports the creation of reports and presentations; the ability to explore relationships between variables and drill down to look at subgroups; and low requirements for computing resources. Organisations introducing a dashboard may find the following strategies beneficial: clinical champion to promote use; testing with real data by audit staff; establishing routines for integrating use into work practices; involving audit staff in adoption activities; and allowing customisation. Limitations: The COVID-19 pandemic stopped phase 4 data collection, limiting our ability to further test and refine the QualDash theory. Questionnaire results should be treated with caution because of the small, possibly biased, sample. Control sites for the interrupted time series analysis were not possible because of research and development delays. One intervention site did not submit data. Limited uptake meant that assessing the impact on more measures was not appropriate. Future work: The extent to which national audit dashboards are used and the strategies national audits use to encourage uptake, a realist review of the impact of dashboards, and rigorous evaluations of the impact of dashboards and the effectiveness of adoption strategies should be explored. Study registration: This study is registered as ISRCTN18289782.This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 10, No. 12. See the NIHR Journals Library website for further project information

Creative Visualisation Opportunities Workshops: A Case Study in Population Health Management

NoPopulation Health Management (PHM) relies on the analysis of data from several sources to account for the complex interaction of factors that contribute to the health and well-being of a population, while considering biases and inequalities across sub-populations. Visualisation is emerging as an essential tool for insight generation from data shared and linked across services including healthcare, education, housing, policing, etc. However, visualisation design is challenged by poor data connectivity and quality, high dimensionality and complexity of real-world routinely collected data, in addition to the heterogeneity of users’ backgrounds and tasks. The Creative Visualisation Opportunities (CVO) framework provides a structured approach for working with diverse communities of visualisation stakeholders and defines a set of participatory activities for the effective elicitation of requirements and visualisation design alternatives. We conducted three workshops, applying variations of the CVO framework, with over one hundred participants from the PHM domain, including clinicians, researchers, government and private sector representatives, and local communities. In this paper, we present the results of preliminary analysis of these activities and report on the perceived impact of visualisation in this domain from a stakeholders’ perspective. We report real-world successes and limitations of applying the framework in different formats (through online and in-person workshops), and reflect on lessons learned for task analysis and visualisation design in the PHM domain

ITEA—interactive trajectories and events analysis: exploring sequences of spatio-temporal events in movement data

Widespread use of GPS and similar technologies makes it possible to collect extensive amounts of trajectory data. These data sets are essential for reasonable decision making in various application domains. Additional information, such as events taking place along a trajectory, makes data analysis challenging, due to data size and complexity. We present an integrated solution for interactive visual analysis and exploration of events along trajectories data. Our approach supports analysis of event sequences at three different levels of abstraction, namely spatial, temporal, and events themselves. Customized views as well as standard views are combined to form a coordinated multiple views system. In addition to trajectories and events, we include on-the-fly derived data in the analysis. We evaluate our integrated solution using the IEEE VAST 2015 Challenge data set. A successful detection and characterization of malicious activity indicate the usefulness and efficiency of the presented approach.Fil: Cibulski, Lena. Zentrum für Virtual Reality und Visualisierung; Austria. University of Magdeburg; AlemaniaFil: Gračanin, Denis. Virginia Tech University; Estados UnidosFil: Diehl, Alexandra. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; ArgentinaFil: Splechtna, Rainer. Zentrum für Virtual Reality und Visualisierung; AustriaFil: Elshehaly, Mai. University of Maryland; Estados UnidosFil: Delrieux, Claudio Augusto. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca; Argentina. Universidad Nacional del Sur; ArgentinaFil: Matković, Krešimir. Zentrum für Virtual Reality und Visualisierung; Austri