Automatic alignment of surgical videos using kinematic data

Over the past one hundred years, the classic teaching methodology of "see one, do one, teach one" has governed the surgical education systems worldwide. With the advent of Operation Room 2.0, recording video, kinematic and many other types of data during the surgery became an easy task, thus allowing artificial intelligence systems to be deployed and used in surgical and medical practice. Recently, surgical videos has been shown to provide a structure for peer coaching enabling novice trainees to learn from experienced surgeons by replaying those videos. However, the high inter-operator variability in surgical gesture duration and execution renders learning from comparing novice to expert surgical videos a very difficult task. In this paper, we propose a novel technique to align multiple videos based on the alignment of their corresponding kinematic multivariate time series data. By leveraging the Dynamic Time Warping measure, our algorithm synchronizes a set of videos in order to show the same gesture being performed at different speed. We believe that the proposed approach is a valuable addition to the existing learning tools for surgery.Comment: Accepted at AIME 201

NOViSE: a virtual natural orifice transluminal endoscopic surgery simulator

Purpose: Natural Orifice Transluminal Endoscopic Surgery (NOTES) is a novel technique in minimally invasive surgery whereby a flexible endoscope is inserted via a natural orifice to gain access to the abdominal cavity, leaving no external scars. This innovative use of flexible endoscopy creates many new challenges and is associated with a steep learning curve for clinicians. Methods: We developed NOViSE - the first force-feedback enabled virtual reality simulator for NOTES training supporting a flexible endoscope. The haptic device is custom built and the behaviour of the virtual flexible endoscope is based on an established theoretical framework – the Cosserat Theory of Elastic Rods. Results: We present the application of NOViSE to the simulation of a hybrid trans-gastric cholecystectomy procedure. Preliminary results of face, content and construct validation have previously shown that NOViSE delivers the required level of realism for training of endoscopic manipulation skills specific to NOTES Conclusions: VR simulation of NOTES procedures can contribute to surgical training and improve the educational experience without putting patients at risk, raising ethical issues or requiring expensive animal or cadaver facilities. In the context of an experimental technique, NOViSE could potentially facilitate NOTES development and contribute to its wider use by keeping practitioners up to date with this novel surgical technique. NOViSE is a first prototype and the initial results indicate that it provides promising foundations for further development

A Suture Training System with Synchronized Force, Motion and Video Data Collection

Suturing is a common surgical task where surgeons stitch a particular tissue. There is an increasing demand for a tool to objectively quantify and train surgical skills. Suturing is particularly difficult to teach due to various multi-modal aspects involved in the task including applied forces, hand motion and optimal time for suturing. Towards quantifying the task of suturing, a platform is required to capture force, motion and video data while performing surgical suturing. This objective data can potentially be used to evaluate performance of a trainee and provide feedback regarding improving suturing skill. In the previous prototype of the platform, 3 key issues faced were synchronization of the three sensors, inadequate construction of the platform and the lack of a framework for image processing towards real-time assessment of suturing skill. In order to improve the platform, the aforementioned issues have been addressed in specific ways. The data collected in the system is synchronized in real-time along with a video recording for image processing and the noise due to the platform is considerably reduced by making modification to the platform construction. Data was collected on the platform with 15 novice participants. Initial analysis validates the synchronization of the sensor data. In the future, the suture skill of experts and novices will be analyzed using meaningful metrics and machine learning algorithms. This work has the potential of enabling objective and structured training and evaluation for next generation surgeons

Integrating user-centred design in the development of a silent speech interface based on permanent magnetic articulography

Abstract: A new wearable silent speech interface (SSI) based on Permanent Magnetic Articulography (PMA) was developed with the involvement of end users in the design process. Hence, desirable features such as appearance, port-ability, ease of use and light weight were integrated into the prototype. The aim of this paper is to address the challenges faced and the design considerations addressed during the development. Evaluation on both hardware and speech recognition performances are presented here. The new prototype shows a com-parable performance with its predecessor in terms of speech recognition accuracy (i.e. ~95% of word accuracy and ~75% of sequence accuracy), but significantly improved appearance, portability and hardware features in terms of min-iaturization and cost

Computer- and robot-assisted Medical Intervention

Medical robotics includes assistive devices used by the physician in order to make his/her diagnostic or therapeutic practices easier and more efficient. This chapter focuses on such systems. It introduces the general field of Computer-Assisted Medical Interventions, its aims, its different components and describes the place of robots in that context. The evolutions in terms of general design and control paradigms in the development of medical robots are presented and issues specific to that application domain are discussed. A view of existing systems, on-going developments and future trends is given. A case-study is detailed. Other types of robotic help in the medical environment (such as for assisting a handicapped person, for rehabilitation of a patient or for replacement of some damaged/suppressed limbs or organs) are out of the scope of this chapter.Comment: Handbook of Automation, Shimon Nof (Ed.) (2009) 000-00

Distributed artificial intelligence in a virtual reality setting: a case study

Artificial intelligence, or Al, is a fascinating area of research. Al refers to the attempt to create a computer program, known as an intelligent agent, which can think and operate in a complex, changing environment. Many problems have been discovered in attempting to develop these intelligent agents, including the simulation of learning, planning, and natural language understanding. While researching these issues, a new branch of Al research has developed. Researchers discovered that there are other problems associated with how intelligent agents work together to solve problems. This field of research has become known as distributed artificial intelligence, or DAI