Health-5G: A Mixed Reality-Based System for Remote Medical Assistance in Emergency Situations

Mixed reality is the combination of virtual and augmented reality to interactively and believably merge physical and computer-generated environments. This paper discusses the design of Health5G, a scalable mixed reality-based system that facilitates and supports emergency response by medical emergency teams. Health-5G is supported by a distributed architecture divided into four interrelated applications responsible for advanced computer-human interaction, effective real-time videoconference, medical device integration, and communication infrastructure, respectively. The mixed reality layer is provided by the headset Microsoft Hololens 2™. Health-5G is based on scenarios in which emergency personnel wear mixed reality glasses that can transmit audio, video, and data streams bidirectionally over a 5G network to medical specialists stationed in a hospital at any distance. Thanks to Health-5G, the specialist will be able to access the emergency team’s point of view at any time and provide verbal and visual instructions, including gestures and positioning of graphical markers in 3D space. In this way, emergency personnel can provide the best possible care to the patient without having to wait for them to arrive at the hospital, saving a lot of time in scenarios where every second can make a difference. Health-5G also addresses the integration of medical devices and the collection of the patient’s medical data in a scalable way through optical character recognition. A case study is discussed where Health-5G is used to attend a patient in the street suffering from syncope due to third-degree atrioventricular block. Latency and performance tests over a 5G network are also discussed. To the best of our knowledge, there is no comprehensive solution in the literature that provides all the capabilities offered by Health-5G in terms of functionality and advanced interaction mechanisms within the context of remote, immersive support in emergency situations

PAMTEL-RT: Web-based Multimedia Platform for Tele-Assistance of Pediatric Health Emergencies in Real Time in Training Centers

[EN] In this paper, PAMTEL-RT, a web-based platform to provide remote health professional support and guidance during the initial assistance of emergency situations is presented. In particular, this platform focuses on the pediatric population and has been specifically designed to be used by non-healthcare professionals in training centers, such as educational or sport centers. It allows pediatricians in hospitals to use Information and Communication Technologies to assist or guide, in real-time, training personnel in charge of children when any emergency situation may arise. A direct and secure real-time connection between the medical and training centers is established, allowing sharing multiple media, such as audio, video, images, documents, text and data. Since the implementation relies on standard web-based technologies, cross-network, cross-platform and cross-device support are ensured. At first, an opinion and acceptance survey was conducted to collect requirements from some potential users to consider their opinions when designing and developing the platform. In line with the obtained results, the architecture of the platform has been designed, and a prototype has been developed and preliminarily evaluated through 12 drills/simulations in 7 education and sports centers. In the evaluation, 24 training personnel were involved, who considered that its usability was excellent and that it would have applicability in their centers. Furthermore, they considered that the included functionalities in the platform are appropriate and believed that the use of PAMTEL-RT could be highly effective in the early management of extreme emergency situations, and, in some cases, in avoiding tragic consequences.This work has been partially funded by the Universitat Politecnica de Valencia (UPV) and the Foundation for the Promotion of Health and Biomedical Research of Valencia Region (FISABIO), under their R&D&I Support Program with references UPV-FISABIO-2017-002-940. Authors also want to thank educational and sport institutions involved in the drills: Gregori Maians and Esclavas del Sagrado Corazón de Jesús Schools in Gandia city; Batoi and Pare Victoria Schools in Alcoi city; and U.E. Gandia Football Club, and C.A. Safor Delikia and C.C. El Garbí athletics clubs, in Gandia cityBoronat, F.; Escrivá, P.; Salvador-Llàcer, P.; Pareja, F.; Pastor, J. (2021). PAMTEL-RT: Web-based Multimedia Platform for Tele-Assistance of Pediatric Health Emergencies in Real Time in Training Centers. Multimedia Systems. 27(2):205-227. https://doi.org/10.1007/s00530-020-00730-0205227272Tomines, A.: Pediatric telehealth: approaches by specialty and implications for general pediatric care. Adv. Pediatr. 66, 55–85 (2019). https://doi.org/10.1016/j.yapd.2019.04.005European Commission. A healthy approach. Technology for personalized, preventative healthcare. ICT Research: the policy perspective. Publications Office of the European Union, 2010. 24 pp. ISBN 978-92-79-16085–1. (2010). https://doi.org/10.2759/33350EXPH (EXpert Panel on effective ways of investing in Health), Assessing the impact of digital transformation of health services, 20 November 2018. https://ec.europa.eu/health/expert_panel/sites/expertpanel/files/docsdir/022_digitaltransformation_en.pdf (2020). Accessed May 2020Olson, C.A., McSwain, S.D., Curfman, A.L., et al.: The current pediatric telehealth landscape. Pediatrics 141(3), e20172334 (2018)Burke, B.L., Jr., Hall, R.W.: Telemedicine: pediatric applications. Pediatrics 136(1), e293-308 (2015). https://doi.org/10.1542/peds.2015-1517McConnochie, K.M., Tan, J., Wood, N.E., et al.: Acute illness utilization patterns before and after telemedicine in childcare for inner-city children: a cohort study. Telemed. J. E Health 13(4), 381–390 (2007)McConnochie, K.M., Wood, N.E., Herendeen, N.E., et al.: Telemedicine in urban and suburban childcare and elementary schools lightens family burdens. Telemed. J. E Health 16(5), 533–542 (2010). https://doi.org/10.1089/tmj.2009.0138Utidjian, L., Abramson, E.: Pediatric telehealth: opportunities and challenges. Pediatr Clin. N. Am. 63(2), 367–378 (2016). https://doi.org/10.1016/j.pcl.2015.11.006Qubty, W., Patniyot, I., Gelfand, A.: Telemedicine in a pediatric headache clinic: a prospective survey. Neurology. 90(19), e1702–e1705 (2018). https://doi.org/10.1212/WNL.0000000000005482Stavas, N., Shea, J., Keddem, S., et al.: Perceptions of caregivers and adolescents of the use of telemedicine for the child sexual abuse examination. Child Abuse Negl. 85, 47–57 (2018)Vyas, S., Murren-Boezem, J., Solo-Josephson, P.: Analysis of a pediatric telemedicine program. Prog. Telemed. J. E Health (2018). https://doi.org/10.1089/tmj.2017.0281Bala, N., Price, S.N., Horan, C.M., et al.: Use of telehealth to enhance care in a family-centered childhood obesity intervention. Clin. Pediatr. 58(7), 789–797 (2019)mHealth Economics 2017. Current Status and Future Trends in Mobile Health: Research 2 Guidance. http://www.uzelf.org/wp-content/uploads/2017/12/R2G-mHealth-Developer-Economics-2017-Status-And-Trends.pdf (2020). Accessed May 2020Cisotto, G., Casarin, E., Tomasin, S.: Requirements and enablers of advanced healthcare services over future cellular systems. IEEE Commun. Mag. 58(3), 76–81 (2020). https://doi.org/10.1109/MCOM.001.1900349Morosi, S., Jayousi, S., Mucchi, L., et al.: Medical Tele-Monitoring and Tele-Assistance for Diabetics Patients by Means of 5G Cellular Networks, in Proceedings of BodyNets 2019. In: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 297. Springer, Cham (2019)Montagud, M., Boronat, F., Belda, J., et al.: Use of Web Components to Develop Interactive, Customizable and Multi-Device Video Consumption Platforms, Applications and Usability of Interactive TV, pp. 26–43. Springer, Berlin (2016)Arntzen, I., Bjorch, N., Daoust, F.: Media Synchronization on the Web. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Pauwels, D., van der Hooft, J., Petrangeli, S., et al.: A Web-based framework for fast synchronization of live video players, IFIP/IEEE Symposium on Integrated Network and Service Management, Lisbon, 2017, pp. 524–530 (2017)Rainer, B., Petscharnig, S., Timmerer, C.: Merge and forward: a self-organized inter-destination media synchronization scheme for adaptive media streaming over HTTP. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Montagud, M., Boronat, F., Stokking, H., et al.: Inter-destination multimedia synchronization: schemes, use cases and standardization. Multimedia Syst. J. 18(6), 459–482 (2012). https://doi.org/10.1007/s00530-012-0278-9Montagud, M., Boronat, F., Stokking, H., et al.: Design development and assessment of control schemes for IDMS in a standardized RTCP-based solution. COMNET 70(9), 240–259 (2014). https://doi.org/10.1016/j.comnet.2014.06.004Dynamic Adaptive Streaming Over HTTP (DASH): Part 1: media presentation description and segment formats, ISO/IEC Standard 23009-1: 2012 (2012)Gómez, D., Boronat, F., Montagud, M., et al.: End-to-end DASH platform including a network-based and client-based adaptive quality switching module. In: Proc. Int. Conf. Multimedia Syst., Klagenfurt, Austria, Art. no. 38 (2016)Melvin, H., Shannon, J., Stanton, K.: Time, frequency and phase synchronization for multimedia—basics, issues, developments and opportunities. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Boronat, F., Lloret, J., García, M.: Multimedia group and inter-stream synchronization techniques: a comparative study. Inf. Syst. 34(1), 108–131 (2009). https://doi.org/10.1016/j.is.2008.05.001Montagud, M., Boronat, F., Pastor, J., et al.: Web-based platform for a customizable and synchronized presentation of subtitles in single- and multi-screen scenarios. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-08955-xSu, Y., Yang, Y., Lu, M., et al.: Smooth control of adaptive media playout for video streaming. IEEE TMM 1(7), 1331–1339 (2009)Rainer, B., Timmerer, C.: Self-Organized inter-destination multimedia synchronization for adaptive media streaming. In: Proceedings of the 22nd ACM international conference on Multimedia (MM '14). ACM, New York, NY, USA, pp. 327–336 (2014)Montagud, M., Boronat, F., Roig, B., et al.: How to perform AMP? Cubic adjustments for improving the QoE. Comput. Commun. 103, 61–73 (2017). https://doi.org/10.1016/j.comcom.2017.01.017Hossfeld, T., Seufert, M., Hirth, M., et al.: Quantification of YouTube QoE via crowdsourcing. IEEE ISM (2011). https://doi.org/10.1109/ISM.2011.87Park, S., Kim, J.: An adaptive media playout for intra-media synchronization of networked-video applications. J. Vis. Commun. Image Represent. 19(2), 106–120 (2008)Brooke, J.: Sus-a quick and dirty usability scale. In the book “Usability evaluation in industry”, issue 194, pages 189–194, Taylor and Francis Ed (1996)Bangor, A., Kortum, P., Miller, J.: Determining what individual sus scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114–123 (2009

Towards fostering the role of 5G networks in the field of digital health

A typical healthcare system needs further participation with patient monitoring, vital signs sensors and other medical devices. Healthcare moved from a traditional central hospital to scattered patients. Healthcare systems receive help from emerging technology innovations such as fifth generation (5G) communication infrastructure: internet of things (IoT), machine learning (ML), and artificial intelligence (AI). Healthcare providers benefit from IoT capabilities to comfort patients by using smart appliances that improve the healthcare level they receive. These IoT smart healthcare gadgets produce massive data volume. It is crucial to use very high-speed communication networks such as 5G wireless technology with the increased communication bandwidth, data transmission efficiency and reduced communication delay and latency, thus leading to strengthen the precise requirements of healthcare big data utilities. The adaptation of 5G in smart healthcare networks allows increasing number of IoT devices that supplies an augmentation in network performance. This paper reviewed distinctive aspects of internet of medical things (IoMT) and 5G architectures with their future and present sides, which can lead to improve healthcare of patients in the near future

Video Relay Service for Deaf people using WebRTC

This paper reports on an experimental open source video relay service prototype that helps Deaf people communicate with hearing people by accessing a third party sign language interpreter on a mobile device. Deaf people are disadvantaged in many ways when communicating with the hearing world in real world scenarios, such as hospital visits and in cases of emergency. When possible, Deaf people can enlist the assistance of a family member, community worker or sign language interpreter to assist with such scenarios, however this assistance is pre-arranged and Deaf people would prefer on-the-fly assistance. Our application will assist Deaf people to contact any available sign language interpreter to facilitate communication between the Deaf person and a hearing person using a split screen model, effectively creating a three-way conversation between the Deaf person, the hearing person and the sign language interpreter. The prototype was developed using the WebRTC platform, with JavaScript for browser operability and hardware platform independence. Our hope is that the research can be used to persuade mobile network operators of the need for free or heavily discounted data connection to relay services for Deaf mobile customers

Использование VoIP в задачах удаленного консультирования пациентов

Использование современных технологий наряду с адаптирующейся нормативной базой позволяет повысить доступность медицинской помощи. В работе рассматриваются вопросы обеспечения двухсторонней аудио- и видеосвязи с организацией хранения формируемой истории болезни.The use of modern technology, along with an adaptive regulatory framework, increases the availability of medical care. The paper considers the issues of providing two-way audio and video communications with the organization of storage of the formed medical history

Access visits using video communication

An online video communication system is presented that enables Occupational Therapists (OTs) assess patient homes for assistive technology needs before acute care discharge to ensure appropriate independence and recovery conditions. Explorations under multiple conditions revealed perspectives from OTs and volunteer facilitators. Preliminary key findings and insights are reported

Robot Assisted Surgical Ward Rounds: Virtually Always There

Background:  While an explosion in technological sophistication has revolutionized surgery within the operating theatre, delivery of surgical ward-based care has seen little innovation.  Use of telepresence allowing off-site clinicians communicate with patients has been largely restricted to outpatient settings or use of complex, expensive, static devices.  We designed a prospective study to ascertain feasibility and face validity of a remotely controlled mobile audiovisual drone (LUCY) to access inpatients.  This device is, uniquely, lightweight, freely mobile and emulates ‘human’ interaction by swiveling and adjusting height to patients’ eye-level.   Methods: Robot-assisted ward rounds(RASWR) were conducted over 3 months. A remotely located consultant surgeon communicated with patients/bedside teams via encrypted audiovisual telepresence robot (DoubleRoboticstm, California USA).  Likert-scale satisfaction questionnaires, incorporating free-text sections for mixed-methods data collection, were disseminated to patient and staff volunteers following RASWRs.  The same cohort completed a linked questionnaire following conventional (gold-standard) rounds, acting as control group. Data were paired, and non-parametric analysis performed.  Results: RASWRs are feasible (>90% completed without technical difficulty). The RASWR(n=52 observations) demonstrated face validity with strong correlations (r>0.7; Spearman, p-value <0.05) between robotic and conventional ward rounds among patients and staff on core themes, including dignity/confidentiality/communication/satisfaction with management plan. Patients (96.08%, n=25) agreed RASWR were a satisfactory alternative when consultant physical presence was not possible. There was acceptance of nursing/NCHD cohort (100% (n=11) willing to regularly partake in RASWR).  Conclusion: RASWRs receive high levels of patient and staff acceptance, and offer a valid alternative to conventional ward rounds when a consultant cannot be physically present

Augmented reality as a telemedicine platform for remote procedural training

Traditionally, rural areas in many countries are limited by a lack of access to health care due to the inherent challenges associated with recruitment and retention of healthcare professionals. Telemedicine, which uses communication technology to deliver medical services over distance, is an economical and potentially effective way to address this problem. In this research, we develop a new telepresence application using an augmented reality (AR) system. We explore the use of the Microsoft HoloLens to facilitate and enhance remote medical training. Intrinsic advantages of AR systems enable remote learners to perform complex medical procedures such as Point of Care Ultrasound (PoCUS) without visual interference. This research uses the HoloLens to capture the first-person view of a simulated rural emergency room (ER) through mixed reality capture (MRC) and serves as a novel telemedicine platform with remote pointing capabilities. The mentor's hand gestures are captured using a Leap Motion and virtually displayed in the AR space of the HoloLens. To explore the feasibility of the developed platform, twelve novice medical trainees were guided by a mentor through a simulated ultrasound exploration in a trauma scenario, as part of a pilot user study. The study explores the utility of the system from the trainees, mentor, and objective observers' perspectives and compares the findings to that of a more traditional multi-camera telemedicine solution. The results obtained provide valuable insight and guidance for the development of an AR-supported telemedicine platform