Parsing human skeletons in an operating room

Multiple human pose estimation is an important yet challenging problem. In an Operating Room (OR) environment, the 3D body poses of surgeons and medical staff can provide important clues for surgical workflow analysis. For that purpose, we propose an algorithm for localizing and recovering body poses of multiple human in an OR environment under a multi-camera setup. Our model builds on 3D Pictorial Structures (3DPS) and 2D body part localization across all camera views, using Convolutional Neural Networks (ConvNets). To evaluate our algorithm, we introduce a dataset captured in a real OR environment. Our dataset is unique, challenging and publicly available with annotated ground truths. Our proposed algorithm yields to promising pose estimation results on this dataset

The Iceberg Phenomenon: As Soon as One Technological Problem in NOTES Is Solved, the Next One Appears!

Purpose. Though already proclaimed about 7 years ago, natural orifice transluminal endoscopic surgery (NOTES) is still in its early stages. A multidisciplinary working team tried to analyze the technical obstacles and identify potential solutions. Methods. After a comprehensive review of the literature, a group of 3 surgeons, 1 gastroenterologist, 10 engineers, and 1 representative of biomedical industry defined the most important deficiencies within the system and then compiled as well as evaluated innovative technologies that could be used to help overcome these problems. These technologies were classified with regard to the time needed for their implementation and associated hindrances, where priority is based on the level of impact and significance that it would make. Results. Both visualization and actuation require significant improvement. Advanced illumination, mist elimination, image stabilization, view extension, 3-dimensional stereoscopy, and augmented reality are feasible options and could optimize visual information. Advanced mechatronic platforms with miniaturized, powerful actuators, and intuitive human–machine interfaces could optimize dexterity, as long as enabling technologies are used. The latter include depth maps in real time, precise navigation, fast pattern recognition, partial autonomy, and cognition systems. Conclusion. The majority of functional deficiencies that still exist in NOTES platforms could be overcome by a broad range of already existing or emerging enabling technologies. To combine them in an optimal manner, a permanent dialogue between researchers and clinicians is mandatory

Acoustic signal analysis of instrument-tissue interaction for minimally invasive interventions

PURPOSE Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. METHODS In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument-tissue interaction. RESULTS The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument-tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. CONCLUSION Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback