Human Pose Estimation on Privacy-Preserving Low-Resolution Depth Images

Human pose estimation (HPE) is a key building block for developing AI-based context-aware systems inside the operating room (OR). The 24/7 use of images coming from cameras mounted on the OR ceiling can however raise concerns for privacy, even in the case of depth images captured by RGB-D sensors. Being able to solely use low-resolution privacy-preserving images would address these concerns and help scale up the computer-assisted approaches that rely on such data to a larger number of ORs. In this paper, we introduce the problem of HPE on low-resolution depth images and propose an end-to-end solution that integrates a multi-scale super-resolution network with a 2D human pose estimation network. By exploiting intermediate feature-maps generated at different super-resolution, our approach achieves body pose results on low-resolution images (of size 64x48) that are on par with those of an approach trained and tested on full resolution images (of size 640x480).Comment: Published at MICCAI-201

ST(OR)2: Spatio-Temporal Object Level Reasoning for Activity Recognition in the Operating Room

Surgical robotics holds much promise for improving patient safety and clinician experience in the Operating Room (OR). However, it also comes with new challenges, requiring strong team coordination and effective OR management. Automatic detection of surgical activities is a key requirement for developing AI-based intelligent tools to tackle these challenges. The current state-of-the-art surgical activity recognition methods however operate on image-based representations and depend on large-scale labeled datasets whose collection is time-consuming and resource-expensive. This work proposes a new sample-efficient and object-based approach for surgical activity recognition in the OR. Our method focuses on the geometric arrangements between clinicians and surgical devices, thus utilizing the significant object interaction dynamics in the OR. We conduct experiments in a low-data regime study for long video activity recognition. We also benchmark our method againstother object-centric approaches on clip-level action classification and show superior performance

Interventional 3D Augmented Reality in Orthopedic, Trauma and Vascular Surgery

The Medical Education, Training and Computer Assisted Interventions (METRICS) Laboratory aims to integrate novel mixed-reality technologies with application in computer assisted interventions. We showcase two technologies with specific aims at optimizing surgical workflow and minimizing radiation exposure in orthopedic, trauma, and vascular surgeries. The first is an Augmented Reality C-arm fluoroscope, which provides intuitive real-time visualization by accurately overlaying X-ray to video images. The second is a ‘Desired-views’ user interface which resolves the challenges involved in the optimal control of C-arm fluoroscopes for their constant repositioning during surgery by either the interventionalist or the surgical team

Realidad aumentada para la enseñanza y concientización de los efectos del COVID-19 en la salud

Para reducir los casos de contagio del nuevo coronavirus en la población, se plantea una solución basada en realidad aumentada que permita informar y concientizar dando una explicación y simulación de los efectos del COVID-19 en los órganos del cuerpo humano en 3 escenarios: un niño con asma de entre 6 a 8 años, un joven de entre 20 y 30 años sin comorbilidades y un adulto mayor con obesidad grado 2. Para ello se realizó una aplicación de realidad aumentada que muestra estos efectos clasificados por etapas de esta nueva enfermedad: leve, moderada y grave. Luego se realizó una prueba piloto para evaluar la usabilidad de la aplicación. Los resultados de esta prueba determinaron que la aplicación es entendible y atractiva, sin embargo, aún puede ser mejorada. Se realizaron mejoras en la aplicación basadas en los resultados de esta primera prueba y se aplicó una experimentación a un grupo distinto de 70 personas de entre 20 y 40 años, primero se realizó una encuesta de conocimiento sobre COVID-19 antes de interactuar con la aplicación, luego los encuestados interactuaron con esta y finalmente realizaron una encuesta de conocimiento y de concientización. En esta segunda encuesta, se encontró evidencia significativa ya que se obtuvo un promedio de 26.8% de mejora en las respuestas correctas de los participantes antes y después de interactuar con la aplicación. La aplicación cumplió con sus objetivos y se mejorará en base a los resultados de las encuestas

Usability framework for mobile augmented reality language learning

After several decades since its introduction, the existing ISO9241-11 usability framework is still vastly used in Mobile Augmented Reality (MAR) language learning. The existing framework is generic and can be applied to diverse emerging technologies such as electronic and mobile learning. However, technologies like MAR have interaction properties that are significantly unique and require different usability processes. Hence, implementing the existing framework on MAR can lead to non-optimized, inefficient, and ineffective outcomes. Furthermore, state-of-the-art analysis models such as machine learning are not apparent in MAR usability studies, despite evidence of positive outcomes in other learning technologies. In recent MAR learning studies, machine learning benefits such as problem identification and prioritization were non-existent. These setbacks could slow down the advancement of MAR language learning, which mainly aims to improve language proficiency among MAR users, especially in English communication. Therefore, this research proposed the Usability Framework for MAR (UFMAR) that addressed the currently identified research problems and gaps in language learning. UFMAR introduced an improved data collection method called Individual Interaction Clustering-based Usability Measuring Instrument (IICUMI), followed by a machine learning-driven analysis model called Clustering-based Usability Prioritization Analysis (CUPA) and a prioritization quantifier called Usability Clustering Prioritization Model (UCPM). UFMAR showed empirical evidence of significantly improving usability in MAR, capitalizing on its unique interaction properties. UFMAR enhanced the existing framework with new abilities to systematically identify and prioritize MAR usability issues. Through the experimental results of UFMAR, it was found that the IICUMI method was 50% more effective, while CUPA and UCPM were 57% more effective than the existing framework. The outcome through UFMAR also produced 86% accuracy in analysis results and was 79% more efficient in framework implementation. UFMAR was validated through three cycles of the experimental processes, with triangulation through expert reviews, to be proven as a fitting framework for MAR language learning

Surgical Data Science - from Concepts toward Clinical Translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process