Participatory Development of a 3D Telemedicine system during Covid: the future of remote consultations

Background: The Covid pandemic brought the need for more realistic remote consultations into focus. 2D telemedicine solutions fail to replicate the fluency or authenticity of in-person consultations. This research reports on an international collaboration on the participatory development and first validated clinical use of a novel, real-time 360-degree 3D Telemedicine system worldwide. Development of the system - leveraging Microsoft's HoloportationTM communication technology – commenced at Canniesburn Plastic Surgery Unit, Glasgow in March 2020. Methods: Research followed VR CORE guidelines on development of Digital Health trials, placing patients at the heart of the development process. This consisted of three separate studies - a clinician feedback study (23 clinicians, Nov-Dec 2020), a patient feedback study (26 patients, Jul-Oct 2021), and a cohort study focusing on safety and reliability (40 patients, Oct 2021 - Mar 2022). “Lose, Keep and Change” feedback prompts were used to engage patients in the development process and guide incremental improvements. Results: Participatory testing demonstrated improved patient metrics with 3D in comparison to 2D Telemedicine, including validated measures of satisfaction (p<0.0001), realism or ‘presence’ (Single Item Presence scale, p<0.0001), and quality (Telehealth Usability Questionnaire, p=0.0002). Safety and clinical concordance (95%) of 3D Telemedicine with a face-to-face consultation were equivalent or exceeded estimates for 2D Telemedicine. Conclusions: One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. These data provide the first evidence that HoloportationTM communication technology brings 3D telemedicine closer to this goal than a 2D equivalent

A cross-platform solution for light field based 3D telemedicine

Current telehealth services are dominated by conventional 2D video conferencing systems, which are limited in their capabilities in providing a satisfactory communication experience due to the lack of realism. The "immersiveness" provided by 3D technologies has the potential to promote telehealth services to a wider range of applications. However, conventional stereoscopic 3D technologies are deficient in many aspects, including low resolution and the requirement for complicated multi-camera setup and calibration, and special glasses. The advent of light field (LF) photography enables us to record light rays in a single shot and provide glasses-free 3D display with continuous motion parallax in a wide viewing zone, which is ideally suited for 3D telehealth applications. As far as our literature review suggests, there have been no reports of 3D telemedicine systems using LF technology. In this paper, we propose a cross-platform solution for a LF-based 3D telemedicine system. Firstly, a novel system architecture based on LF technology is established, which is able to capture the LF of a patient, and provide an immersive 3D display at the doctor site. For 3D modeling, we further propose an algorithm which is able to convert the captured LF to a 3D model with a high level of detail. For the software implementation on different platforms (i.e., desktop, web-based and mobile phone platforms), a cross-platform solution is proposed. Demo applications have been developed for 2D/3D video conferencing, 3D model display and edit, blood pressure and heart rate monitoring, and patient data viewing functions. The demo software can be extended to multi-discipline telehealth applications, such as tele-dentistry, tele-wound and tele-psychiatry. The proposed 3D telemedicine solution has the potential to revolutionize next-generation telemedicine technologies by providing a high quality immersive tele-consultation experience