3D Medical Collaboration Technology to Enhance Emergency Healthcare

Two-dimensional (2D) videoconferencing has been explored widely in the past 15–20 years to support collaboration in healthcare. Two issues that arise in most evaluations of 2D videoconferencing in telemedicine are the difficulty obtaining optimal camera views and poor depth perception. To address these problems, we are exploring the use of a small array of cameras to reconstruct dynamic three-dimensional (3D) views of a remote environment and of events taking place within. The 3D views could be sent across wired or wireless networks to remote healthcare professionals equipped with fixed displays or with mobile devices such as personal digital assistants (PDAs). The remote professionals’ viewpoints could be specified manually or automatically (continuously) via user head or PDA tracking, giving the remote viewers head-slaved or hand-slaved virtual cameras for monoscopic or stereoscopic viewing of the dynamic reconstructions. We call this idea remote 3D medical collaboration. In this article we motivate and explain the vision for 3D medical collaboration technology; we describe the relevant computer vision, computer graphics, display, and networking research; we present a proof-of-concept prototype system; and we present evaluation results supporting the general hypothesis that 3D remote medical collaboration technology could offer benefits over conventional 2D videoconferencing in emergency healthcare

Realistic and interactive high-resolution 4D environments for real-time surgeon and patient interaction

Copyright © 2016 John Wiley & Sons, Ltd. Background: Remote consultations that are realistic enough to be useful medically offer considerable clinical, logistical and cost benefits. Despite advances in virtual reality and vision hardware and software, these benefits are currently often unrealised. Method: The proposed approach combines high spatial and temporal resolution 3D and 2D machine vision with virtual reality techniques, in order to develop new environments and instruments that will enable realistic remote consultations and the generation of new types of useful clinical data. Results: New types of clinical data have been generated for skin analysis and respiration measurement; and the combination of 3D with 2D data was found to offer potential for the generation of realistic virtual consultations. Conclusion: An innovative combination of high resolution machine vision data and virtual reality online methods, promises to provide advanced functionality and significant medical benefits, particularly in regions where populations are dispersed or access to clinicians is limited. Copyright © 2016 John Wiley & Sons, Ltd

Enabling Distributed Collaboration among Heterogeneous Devices

ABSTRACT In this paper we describe a new class of collaborative scientific applications that incorporate heterogeneous devices, such as shared supercomputing or visualization resources, personal computers, and mobile devices, present some classes of collaboration tasks that could profitably make use an application infrastructure that connects heterogeneous devices, and identify some particular applications that fall into these classes. We also describe in greater detail one potential application, natural science field research that employs sensor networks. We also discuss some of the problems that need to be addressed in building an application that allows such heterogeneous device collaboration and some benefits to digital science that could be realized by building collaborative applications in this fashion

Illuminating Collaboration in Emergency Helath Care Situations: Paramedic-Physician Collaboration and 3D Telepresence Technology

Introduction. This paper focuses on paramedics' perspectives regarding paramedic-physician collaboration today, and their perspectives regarding the potential of 3D telepresence technology in the future. Method. Interviews were conducted with forty practicing paramedics. Analysis. The interview data were analysed using open and axial coding. An agreement of 0.82 using Cohen's kappa inter-coder reliability measure was reached. After coding was completed themes and relationships among codes were synthesised using topic memos. Results. Paramedics expressed concern about the lack of respect and trust exhibited towards them by other medical professionals. They discussed how they paint the picture for physicians and the importance of the physician trusting the paramedic. They further reported 3D telepresence technology would make their work visible in ways not previously possible. They also reported the technology would require additional training, changes to existing financial models used in emergency health care, and increased access to physicians. Conclusions. Teaching collaboration skills and strategies to physicians and paramedics could benefit their collaboration today, and increase their readiness to effectively use collaboration technologies in the future

Illuminating collaboration in emergency health care situations:Paramedic-physician collaboration and 3D telepresence technology

Introduction. This paper focuses on paramedics\u27 perspectives regarding paramedic-physician collaboration today, and their perspectives regarding the potential of 3D telepresence technology in the future. Method. Interviews were conducted with forty practicing paramedics. Analysis. The interview data were analysed using open and axial coding. An agreement of 0.82 using Cohen\u27s kappa inter-coder reliability measure was reached. After coding was completed themes and relationships among codes were synthesised using topic memos. Results. Paramedics expressed concern about the lack of respect and trust exhibited towards them by other medical professionals. They discussed how they paint the picture for physicians and the importance of the physician trusting the paramedic. They further reported 3D telepresence technology would make their work visible in ways not previously possible. They also reported the technology would require additional training, changes to existing financial models used in emergency health care, and increased access to physicians. Conclusions. Teaching collaboration skills and strategies to physicians and paramedics could benefit their collaboration today, and increase their readiness to effectively use collaboration technologies in the future

Exploring the potential of video technologies for collaboration in emergency medical care: Part I. Information sharing

We are investigating the potential of 3D telepresence, or televideo, technology to support collaboration among geographically separated medical personnel in trauma emergency care situations. 3D telepresence technology has the potential to provide richer visual information than current 2D video conferencing techniques. This may be of benefit in diagnosing and treating patients in emergency situations where specialized medical expertise is not locally available. The 3D telepresence technology does not yet exist and there is a need to understand its potential before resources are spent on its development and deployment. This poses a complex challenge. How can we evaluate the potential impact of a technology within complex, dynamic work contexts when the technology does not yet exist? To address this challenge we conducted an experiment with a post-test, between-subjects design that takes the medical situation and context into account. In the experiment we simulated an emergency medical situation involving practicing paramedics and physicians, collaborating remotely via two conditions: with today’s 2D videoconferencing and a 3D telepresence proxy. In this paper we examine information sharing between the attending paramedic and collaborating physician. Postquestionnaire data illustrate that the information provided by the physician was perceived to be more useful by the paramedic in the 3D proxy condition than the 2D condition. However, data pertaining to the quality of interaction and trust between the collaborating physician and paramedic show mixed results. Post-interview data help explain these results

Modified belief propagation for reconstruction of office environments

Belief Propagation (BP) is an algorithm that has found broad application in many areas of computer science. The range of these areas includes Error Correcting Codes, Kalman filters, particle filters, and -- most relevantly -- stereo computer vision. Many of the currently best algorithms for stereo vision benchmarks, e.g. the Middlebury dataset, use Belief Propagation. This dissertation describes improvements to the core algorithm to improve its applicability and usefulness for computer vision applications. A Belief Propagation solution to a computer vision problem is commonly based on specification of a Markov Random Field that it optimizes. Both Markov Random Fields and Belief Propagation have at their core some definition of nodes and neighborhoods' for each node. Each node has a subset of the other nodes defined to be its neighborhood. In common usages for stereo computer vision, the neighborhoods are defined as a pixel's immediate four spatial neighbors. For any given node, this neighborhood definition may or may not be correct for the specific scene. In a setting with video cameras, I expand the neighborhood definition to include corresponding nodes in temporal neighborhoods in addition to spatial neighborhoods. This amplifies the problem of erroneous neighborhood assignments. Part of this dissertation addresses the erroneous neighborhood assignment problem. Often, no single algorithm is always the best. The Markov Random Field formulation appears amiable to integration of other algorithms: I explore that potential here by integrating priors from independent algorithms. This dissertation makes core improvements to BP such that it is more robust to erroneous neighborhood assignments, is more robust in regions with inputs that are near-uniform, and can be biased in a sensitive manner towards higher level priors. These core improvements are demonstrated by the presented results: application to office environments, real-world datasets, and benchmark datasets

The Extraction and Use of Image Planes for Three-dimensional Metric Reconstruction

The three-dimensional (3D) metric reconstruction of a scene from two-dimensional images is a fundamental problem in Computer Vision. The major bottleneck in the process of retrieving such structure lies in the task of recovering the camera parameters. These parameters can be calculated either through a pattern-based calibration procedure, which requires an accurate knowledge of the scene, or using a more flexible approach, known as camera autocalibration, which exploits point correspondences across images. While pattern-based calibration requires the presence of a calibration object, autocalibration constraints are often cast into nonlinear optimization problems which are often sensitive to both image noise and initialization. In addition, autocalibration fails for some particular motions of the camera. To overcome these problems, we propose to combine scene and autocalibration constraints and address in this thesis (a) the problem of extracting geometric information of the scene from uncalibrated images, (b) the problem of obtaining a robust estimate of the affine calibration of the camera, and (c) the problem of upgrading and refining the affine calibration into a metric one. In particular, we propose a method for identifying the major planar structures in a scene from images and another method to recognize parallel pairs of planes whenever these are available. The identified parallel planes are then used to obtain a robust estimate of both the affine and metric 3D structure of the scene without resorting to the traditional error prone calculation of vanishing points. We also propose a refinement method which, unlike existing ones, is capable of simultaneously incorporating plane parallelism and perpendicularity constraints in the autocalibration process. Our experiments demonstrate that the proposed methods are robust to image noise and provide satisfactory results