Acoustic signal analysis of instrument-tissue interaction for minimally invasive interventions

PURPOSE Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. METHODS In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument-tissue interaction. RESULTS The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument-tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. CONCLUSION Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback

Telemedical percussion: objectifying a fundamental clinical examination technique for telemedicine

Purpose!#!While demand for telemedicine is increasing, patients are currently restricted to tele-consultation for the most part. Fundamental diagnostics like the percussion still require the in person expertize of a physician. To meet today's challenges, a transformation of the manual percussion into a standardized, digital version, ready for telemedical execution is required.!##!Methods!#!In conjunction with a comprehensive telemedical diagnostic system, in which patients can get examined by a remote-physician, a series of three robotic end-effectors for mechanical percussion were developed. Comprising a motor, a magnetic and a pneumatic-based version, the devices strike a pleximeter to perform the percussion. Emitted sounds were captured using a microphone-equipped stethoscope. The 84 recordings were further integrated into a survey in order to classify lung and non-lung samples.!##!Results!#!The study with 21 participants comprised physicians, medical students and non-medical-related raters in equal parts. With 71.4% correctly classified samples, the ventral motorized device prevailed. While the result is significantly better compared to a manual or pneumatic percussion in this very setup, it only has a small edge over the magnetic devices. In addition, for all ventral versions non-lung regions were rather correctly identified than lung regions.!##!Conclusion!#!The overall setup proves the feasibility of a telemedical percussion. Despite the fact, that produced sounds differ compared to today's manual technique, the study shows that a standardized mechanical percussion has the potential to improve the gold standard's accuracy. While further extensive medical evaluation is yet to come, the system paves the way for future uncompromised remote examinations

COVID-19 and beyond: development of a comprehensive telemedical diagnostic framework

Purpose!#!During the COVID-19 pandemic, a threatening bottleneck of medical staff arose due to a shortage of trained caregivers, who became infected while working with infectious patients. While telemedicine is rapidly evolving in the fields of teleconsultation and telesurgery, proper telediagnostic systems are not yet available, although the demand for contactless patient-doctor interaction is increasing.!##!Methods!#!In this project, the current limitations were addressed by developing a comprehensive telediagnostic system. Therefore, medical examinations have been assessed in collaboration with medical experts. Subsequently, a framework was developed, satisfying the relevant constraints of medical-, technical-, and hygienic- aspects in order to transform in-person examinations into a contactless procedure. Diagnostic steps were classified into three groups: assisted procedures carried out by the patient, teleoperated examination methods, and adoptions of conventional methods.!##!Results!#!The Telemedical Diagnostic Framework was implemented, resulting in a functional proof of concept, where potentially infectious patients could undergo a full medical examination. The system comprises, e.g., a naso-pharyngeal swab, an inspection of the oral cavity, auscultation, percussion, and palpation, based on robotic end-effectors. The physician is thereby connected using a newly developed user-interface and a lead robot, with force feedback control, that enables precise movements with the follower robot on the patient's side.!##!Conclusion!#!Our concept proves the feasibility of a fully telediagnostic system, that consolidates available technology and new developments to an efficient solution enabling safe patient-doctor interaction. Besides infectious situations, this solution can also be applied to remote areas

sj-docx-1-dhj-10.1177_20552076231225084 - Supplemental material for Toward telemedical diagnostics—clinical evaluation of a robotic examination system for emergency patients

Supplemental material, sj-docx-1-dhj-10.1177_20552076231225084 for Toward telemedical diagnostics—clinical evaluation of a robotic examination system for emergency patients by Maximilian Berlet, Jonas Fuchtmann, Roman Krumpholz, Abdeldjallil Naceri, Daniela Macari, Christoph Jähne-Schon, Sami Haddadin, Helmut Friess, Hubertus Feussner and Dirk Wilhelm in DIGITAL HEALTH</p