Drop, Denzel L Q

Gommers, Diederik

Jung, Christian

van Genderen, Michel E

EUR Research Repository

EUR Research Information PortalExtended reality in critically ill patientsPublished in:Intensive Care MedicinePublication status and date:Published: 01/08/2024DOI (link to publisher):10.1007/s00134-024-07511-7Document VersionPublisher's PDF, also known as Version of recordDocument License/Available under:Article 25fa Dutch Copyright ActCitation for the published version (APA):Drop, D. L. Q., Jung, C., Gommers, D., & van Genderen, M. E. (2024). Extended reality in critically ill patients: not yet readyfor take-off! Intensive Care Medicine, 50(8), 1365-1367. https://doi.org/10.1007/s00134-024-07511-7Link to publication on the EUR Research Information PortalTerms and Conditions of UseExcept as permitted by the applicable copyright law, you may not reproduce or make this material available to any third partywithout the prior written permission from the copyright holder(s). Copyright law allows the following uses of this materialwithout prior permission:            • you may download, save and print a copy of this material for your personal use only;            • you may share the EUR portal link to this material.In case the material is published with an open access license (e.g. a Creative Commons (CC) license), other uses may beallowed. Please check the terms and conditions of the specific license.Take-down policyIf you believe that this material infringes your copyright and/or any other intellectual property rights, you may request itsremoval by contacting us at the following email address: openaccess.library@eur.nl. Please provide us with all the relevantinformation, including the reasons why you believe any of your rights have been infringed. In case of a legitimate complaint,we will make the material inaccessible and/or remove it from the website.Intensive Care Medhttps://doi.org/10.1007/s00134-024-07511-7LETTERExtended reality in critically ill patients: not yet ready for take-off!Denzel L. Q. Drop1 , Christian Jung2,3 , Diederik Gommers1  and Michel E. van Genderen1* © 2024 Springer-Verlag GmbH Germany, part of Springer NatureDear Editor,Over the past decades, there has been an exponential rise of publications of virtual reality (VR)-related studies in healthcare [1]. Against the backdrop of this increasing research growth and use of VR-related studies, currently also known as extended reality (XR)-related studies, it is important to closely monitor these advancements and constantly be aware of the maturity of these devices within the medical field. Such awareness and oversight enables the development and clinical implementation, thereby optimizing users and patients’ safety.The National Aeronautics and Space Administration (NASA) established a general concept of nine technol-ogy readiness levels (TRLs), which has been widely adopted to assess the maturity level of a technology and its readiness for implementation, after which this general concept has been translated to classify the maturity of artificial intelligence (AI) models with means to imple-ment them at the bedside [2]. Although the NASA and AI-specific TRLs are, to our knowledge, the existing clas-sifications that best evaluate the development of clinical technologies, these include several stages that do not directly apply to studies focused on XR. This discrepancy prompted the development of a TRL specifically designed for evaluating the maturity of clinical research involving XR technologies. In this study, we, therefore, set out to translate the NASA TRL into a clinically applicable scale for XR research and subsequently identified the levels of maturity of all recent XR-related intensive care unit (ICU) studies to determine its clinical applicability.Four members (DD, CJ, MvG, and HV) of the RATE-XR steering group iterated translations of the NASA TRLs into a clinically applicable scale, achieving unani-mous consensus (see Table  1). Following this, three researchers (DD, CJ, and MvG) applied the XR-specific TRL on studies that were identified in the largest and most recent systematic review regarding XR-related ICU studies [3]. Initial disagreements in the scoring of two articles, particularly concerning the definitions of TRLs seven and eight, were resolved through consensus. The Fleiss’ kappa score for the overall interrater agreement in scoring these articles was 0.949, indicating a very high strength of agreement.The classification of clinical readiness levels for inter-ventions based on XR are presented in Table 1. We thor-oughly analyzed a total of 60 articles, extracted from the systematic review of Kanschik et  al. [3]. Three pro-tocols and 19 educational studies aimed at providers were explicitly excluded from our analysis to maintain a clear focus on patient-centered interventions. This left a pool of 38 clinical XR studies that specifically investigate patient-related outcomes such as delirium, pain, psycho-logical symptoms, and sleep, providing a comprehensive view of the current state of XR applications in patient care. Detailed descriptions of the interventions and their targeted outcomes are provided in Supplementary Tables 1 and 2. 11 articles (29%) were being classified at the development and testing phases (readiness level 1 to 5) and 27 articles (71%) at the clinical testing phases (readiness level 6 to 8). Notably, no studies that involved the full-scale implementation of XR in the clinical work-flow were found (level 9).Since XR is becoming an increasingly important tool, a critical appraisal of this technology is important to assess clinical readiness [4]. The small minority of clini-cally implemented projects identified, either outcome evaluation or implementation, arguably represents a gap *Correspondence:  m.vangenderen@erasmusmc.nl 1 Department of Adult Intensive Care, Erasmus MC, University Medical Center Rotterdam, (internal postaddress – Room Ne-403), Doctor Molewaterplein 40, 3015 GD Rotterdam, The NetherlandsFull author information is available at the end of the articleTable 1 Clinical extended reality application readiness levelsTRL Technology readiness level, NASA American National Aeronautics and Space Administration*Percentages are rounded; therefore, the total sum is > 100%TRLNASA definitionClinical (XR) definitionExplanationExamplesPapers*1Basic principles observed and reportedClinical problem identificationMedical literature is searched and a lack of available effective methods for a clinical problem is identifiedLetters and reviews of the literature con-cluding a gap of available interventions, tools, or devices8%2Technology concept and/or application formulatedProposal of an application using XRAn idea for the development of an applica-tion using XR for a clinical problem is proposed, based on potential beneficial identified outcomesLetters and reviews proposing an idea for an explicit XR driven applicationRegistration of proposals prior to develop-ment3%3Analytical and experimental critical function and/or characteristic proof-of-conceptXR application developmentThe process of developing an application using XR. In this pre-clinical phase, the main focus is on the technical features and content of the application being developedManuscripts describing the creation of a specific XR driven application3%4Component and/or breadboard validation in laboratory environmentTesting of the software of the XR applica-tionThe software part of the XR application will be tested, without using the actual hardwareTesting of the application using conven-tional methods (e.g., computer screens)16%5Component and/or breadboard validation in relevant environmentTesting of the software combined with the hardware of the XR applicationThe software of the XR application is tested using the XR hardware device in a small group of healthy volunteers within the development group, in a non-clinical set-ting, before safety studies are performedTesting of the application using the XR device0%6System/subsystem model or prototype demonstration in a relevant environment (ground or space)Clinical XR application testingThe developed application is applied on healthy volunteers in a clinical setting (representative of the target population (based on for example a diagnosis or specific hospital department admission) for which it is has been developed)Early phase studies on healthy volunteers (e.g., case reports/series and interven-tional prospective cohort studies)8%7System prototype demonstration in a space environmentOutcome orientationPrimary the safety, feasibility, or non-effec-tiveness outcomes are tested in the target population. Secondarily, effectiveness of the intervention can be studiedFeasibility or pilot studies on clinical patients (e.g., case reports/series, interventional prospective cohort studies & randomized controlled trials)53%8Actual system completed and ‘’flight qualified’’ through test and demonstration (ground or space)Outcome evaluationEffectiveness outcomes are studied as primary aim in the target population. Additionally feasibility, safety, or other non-effectiveness outcomes can be studiedEffectiveness trials comparing relevant outcomes to a control group (randomized controlled trials)11%9Actual system ‘’flight proven’’ through suc-cessful mission operationsXR application implementationThe XR application is proven to be effective in a research setting and is now imple-mented in the regular workflow. The long-term effectiveness and workability are monitoredLong-term follow-up studies after workflow integration0%that we need to bridge to move from testing to clinical application in a structured fashion. With the current framework and initiation of the reporting guideline for early stage clinical evaluation of XR-based intervention trials (RATE-XR), we hope to establish this structure and encourage future randomized trials to bridge this gap but also to take into account the different steps that are needed to move to the bedside [5].Supplementary InformationThe online version contains supplementary material available at https:// doi. org/ 10. 1007/ s00134- 024- 07511-7.Author details1 Department of Adult Intensive Care, Erasmus MC, University Medical Center Rotterdam, (internal postaddress – Room Ne-403), Doctor Molewaterplein 40, 3015 GD Rotterdam, The Netherlands. 2 Medical Faculty and University Hospital of Düsseldorf, Cardiovascular Research Institute Düsseldorf (CARID), Heinrich-Heine University Düsseldorf, 40225 Düsseldorf, Germany. 3 Medi-cal Faculty, Department of Cardiology, Pulmonology and Vascular Medicine, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany. AcknowledgementsWe thank Freek van Polen (Sync-VR), Lucas M. Fleuren, and Johan H. Vlake for their comments and help in defining TRL criteria.Author contributionsThe manuscript was written by DD. MvG and CJ conceived the idea and edited the manuscript. DG critically reviewed and edited the manuscript. All the authors approved the final manuscript and had final responsibility for the decision to submit for publication.Data availabilityNot applicable.DeclarationsConflicts of interestThe authors declare that they have no conflicts of interest.Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims in pub-lished maps and institutional affiliations.Accepted: 2 June 2024References 1. Yeung AWK, Tosevska A, Klager E, Eibensteiner F, Laxar D, Stoyanov J, Glisic M, Zeiner S, Kulnik ST, Crutzen R, Kimberger O, Kletecka-Pulker M, Atanasov AG, Willschke H (2021) Virtual and augmented reality applica-tions in medicine: analysis of the scientific literature. J Med Internet Res 23:e25499 2. Fleuren LM, Thoral P, Shillan D, Ercole A, Elbers PWG, Right Data Right Now C (2020) Machine learning in intensive care medicine: ready for take-off? Intensive Care Med 46:1486–1488 3. Kanschik D, Bruno RR, Wolff G, Kelm M, Jung C (2023) Virtual and augmented reality in intensive care medicine: a systematic review. Ann Intensive Care 13:81 4. Bruno RR, Bruining N, Jung C, VR-ICU Study group (2022) Virtual reality in intensive care. Intensive Care Med 48:1227–1229 5. Vlake JH, van Bommel J, Riva G, Wiederhold BK, Cipresso P, Rizzo AS, Botella C, Hooft L, Bienvenu OJ, Geerts B, Wils EJ, Gommers D, van Gen-deren ME (2023) Reporting the early stage clinical evaluation of virtual-reality-based intervention trials: RATE-VR. Nat Med 29:12–13

Extended reality in critically ill patients:not yet ready for take-off!

Abstract

Similar works

Full text

Available Versions

EUR Research Repository