Apple Vision Pro for Healthcare: "The Ultimate Display"? -- Entering the Wonderland of Precision Medicine

At the Worldwide Developers Conference (WWDC) in June 2023, Apple introduced the Vision Pro. The Vision Pro is a Mixed Reality (MR) headset, more specifically it is a Virtual Reality (VR) device with an additional Video See-Through (VST) capability. The VST capability turns the Vision Pro also into an Augmented Reality (AR) device. The AR feature is enabled by streaming the real world via cameras to the (VR) screens in front of the user's eyes. This is of course not unique and similar to other devices, like the Varjo XR-3. Nevertheless, the Vision Pro has some interesting features, like an inside-out screen that can show the headset wearers' eyes to "outsiders" or a button on the top, called "Digital Crown", that allows you to seamlessly blend digital content with your physical space by turning it. In addition, it is untethered, except for the cable to the battery, which makes the headset more agile, compared to the Varjo XR-3. This could actually come closer to the "Ultimate Display", which Ivan Sutherland had already sketched in 1965. Not available to the public yet, like the Ultimate Display, we want to take a look into the crystal ball in this perspective to see if it can overcome some clinical challenges that - especially - AR still faces in the medical domain, but also go beyond and discuss if the Vision Pro could support clinicians in essential tasks to spend more time with their patients.Comment: This is a Preprint under CC BY. This work was supported by NIH/NIAID R01AI172875, NIH/NCATS UL1 TR001427, the REACT-EU project KITE and enFaced 2.0 (FWF KLI 1044). B. Puladi was funded by the Medical Faculty of the RWTH Aachen University as part of the Clinician Scientist Program. C. Gsaxner was funded by the Advanced Research Opportunities Program from the RWTH Aachen Universit

Application of mixed reality to ultrasound-guided femoral arterial cannulation during real-time practice in cardiac interventions

Producción CientíficaMixed reality opens interesting possibilities as it allows physicians to interact with both, the real physical and the virtual computer-generated environment and objects, in a powerful way. A mixed reality system, based in the HoloLens 2 glasses, has been developed to assist cardiologists in a quite complex interventional procedure: the ultrasound-guided femoral arterial cannulations, during real-time practice in interventional cardiology. The system is divided into two modules, the transmitter module, responsible for sending medical images to HoloLens 2 glasses, and the receiver module, hosted in the HoloLens 2, which renders those medical images, allowing the practitioner to watch and manage them in a 3D environment. The system has been successfully used, between November 2021 and August 2022, in up to 9 interventions by 2 different practitioners, in a large public hospital in central Spain. The practitioners using the system confirmed it as easy to use, reliable, real-time, reachable, and cost-effective, allowing a reduction of operating times, a better control of typical errors associated to the interventional procedure, and opening the possibility to use the medical imagery produced in ubiquitous e-learning. These strengths and opportunities were only nuanced by the risk of potential medical complications emerging from system malfunction or operator errors when using the system (e.g., unexpected momentary lag). In summary, the proposed system can be taken as a realistic proof of concept of how mixed reality technologies can support practitioners when performing interventional and surgical procedures during real-time daily practice.Junta de Castilla y León - Gerencia Regional de Salud (SACyL) (grant number GRS 2275/A/2020)Instituto de Salud Carlos III (grant number DTS21/00158)Publicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCL

Respiratory Compensated Robot for Liver Cancer Treatment: Design, Fabrication, and Benchtop Characterization

Hepatocellular carcinoma (HCC) is one of the leading causes of cancer-related death in the world. Radiofrequency ablation (RFA) is an effective method for treating tumors less than 5 cm. However, manually placing the RFA needle at the site of the tumor is challenging due to the complicated respiratory induced motion of the liver. This paper presents the design, fabrication, and benchtop characterization of a patient mounted, respiratory compensated robotic needle insertion platform to perform percutaneous needle interventions. The robotic platform consists of a 4-DoF dual-stage cartesian platform used to control the pose of a 1-DoF needle insertion module. The active needle insertion module consists of a 3D printed flexible fluidic actuator capable of providing a step-like, grasp-insert-release actuation that mimics the manual insertion procedure. Force characterization of the needle insertion module indicates that the device is capable of producing 22.6 ± 0.40 N before the needle slips between the grippers. Static phantom targeting experiments indicate a positional error of 1.14 ± 0.30 mm and orientational error of 0.99° ± 0.36°. Static ex-vivo porcine liver targeting experiments indicate a positional error of 1.22 ± 0.31 mm and orientational error of 1.16° ± 0.44°. Dynamic targeting experiments with the proposed active motion compensation in dynamic phantom and ex-vivo porcine liver show 66.3% and 69.6% positional accuracy improvement, respectively. Future work will continue to develop this platform with the long-term goal of applying the system to RFA for HCC