Utilizing image guided surgery for user interaction in medical augmented reality

The graphical overlay of additional medical information over the patient during a surgical procedure has long been considered one of the most promising applications of augmented reality. While many experimental systems for augmented reality in medicine have reached an advanced state and can deliver high-quality augmented video streams, they usually depend heavily on specialized dedicated hardware. Such dedicated system components, which originally have been designed for engineering applications or VR research, often are ill-suited for use in the clinical practice. We have described a novel medical augmented reality application, which is based almost exclusively on existing, commercially available, and certified medical equipment. In our system, a so-called image guided surgery device is used for tracking a webcam, which delivers the digital video stream of the physical scene that is augmented with the virtual information. In this paper, we show how the capability of the image guided surgery system for tracking surgical instruments can be harnessed for user interaction. Our method enables the user to define points and freely drawn shapes in 3-d and provides selectable menu items, which can be located in immediate proximity to the patient. This eliminates the need for conventional touchscreen- or mouse-based user interaction without requiring additional dedicated hardware like dedicated tracking systems or specialized 3-d input devices. Thus the surgeon can directly interact with the system, without the help of additional personnel. We demonstrate our new input method with an application for creating operation plan sketches directly on the patient in an augmented view

筐体のひずみを利用した入力インタフェース

This paper proposes a new input method that provides a multi-valued, linear output according touserʼs pressure on surfaces of a small, portable information terminal. Our proposed interface usesstrain gage sensors to detect microscopic strain caused by fi ngertip pressure on the device. First, astrain detection circuit was verifi ed using a simple aluminum plate sample with 2 strain gages. Next,to verify operation of the proposed interface, an experimental prototype was manufactured usingstrain gage sensors on each side surface of an aluminum box and 2-axis strain detect circuits. Theexperiments to control screen image displayed on a LCD were conducted by 6 users. As the results,the users could operate the image by pushing the side surfaces of the terminal device. The strength ofuserʼs pressure applied to the scroll speed, and the direction of the load related to the motion directionof the image. With the proposed interface, the information terminal units do not require a specifi c area,thickness, or position for input button-switches in the chassis.This paper proposes a new input method that provides a multi-valued, linear output according touserʼs pressure on surfaces of a small, portable information terminal. Our proposed interface usesstrain gage sensors to detect microscopic strain caused by fi ngertip pressure on the device. First, astrain detection circuit was verifi ed using a simple aluminum plate sample with 2 strain gages. Next,to verify operation of the proposed interface, an experimental prototype was manufactured usingstrain gage sensors on each side surface of an aluminum box and 2-axis strain detect circuits. Theexperiments to control screen image displayed on a LCD were conducted by 6 users. As the results,the users could operate the image by pushing the side surfaces of the terminal device. The strength ofuserʼs pressure applied to the scroll speed, and the direction of the load related to the motion directionof the image. With the proposed interface, the information terminal units do not require a specifi c area,thickness, or position for input button-switches in the chassis

Usability improvement of public transit application through mental model and user journey

Using a mobile application that is featured with local public transit information can greatly improve user action, as well as assisting travelers to have a better experience while taking public transit services. Crowd environment inside a public transit is one of the most frequent causes that lead difficulties to passengers when they engaging their mobile devices to access their apps or in some conditions. Finding crucial information such as stops points and remaining time to interchanges point when switching to another line becomes more difficult in those conditions especially for foreigners who new in a particular region. This study presents the combination of a mental model and a usability approach to construct a user journey map that led to new insights on user's experiences and challenges when utilizing their mobile local transit application. This valuable information is a part of the elicitation process to propose an alternative interaction method to enhance the usability and travel experience of the public transit app. The experimental results indicate that in contrast to the existing mobile transit app, the proposed interface with the utilization of a wearable device could considerably enhance user action when trying to reach the desired location in terms of total time and performance. It implies that the proposed solution, which works through the mental model and user journey is able to intuitively enhance the public transit app usability

Object-Oriented Component-based Design using Behavioral Contracts: Application to Railway Systems

In this report, we propose a formal approach for the design of object-oriented component-based systems using behavioral contracts. This formalism merges interface automata describingcommunication protocols of components with the semantics of their operations. On grounds ofconsistency with the object-oriented paradigms, we revisit the notions of incremental design andindependent implementability of interface automata by novel definitions of components compatibility,composition, and refinement. Our work is illustrated by a design case study of CBTC railway systems