Sonification as a reliable alternative to conventional visual surgical navigation

Despite the undeniable advantages of image-guided surgical assistance systems in terms of accuracy, such systems have not yet fully met surgeons' needs or expectations regarding usability, time efficiency, and their integration into the surgical workflow. On the other hand, perceptual studies have shown that presenting independent but causally correlated information via multimodal feedback involving different sensory modalities can improve task performance. This article investigates an alternative method for computer-assisted surgical navigation, introduces a novel four-DOF sonification methodology for navigated pedicle screw placement, and discusses advanced solutions based on multisensory feedback. The proposed method comprises a novel four-DOF sonification solution for alignment tasks in four degrees of freedom based on frequency modulation synthesis. We compared the resulting accuracy and execution time of the proposed sonification method with visual navigation, which is currently considered the state of the art. We conducted a phantom study in which 17 surgeons executed the pedicle screw placement task in the lumbar spine, guided by either the proposed sonification-based or the traditional visual navigation method. The results demonstrated that the proposed method is as accurate as the state of the art while decreasing the surgeon's need to focus on visual navigation displays instead of the natural focus on surgical tools and targeted anatomy during task execution

Acoustic process monitoring in laser beam welding

Structure-borne acoustic emission (AE) measurement shows major advantages regarding quality assurance and process control in industrial applications. In this paper, laser beam welding of steel and aluminum was carried out under varying process parameters (welding speed, focal position) in order to provide data by means of structure-borne AE and simultaneously high-speed video recordings. The analysis is based on conventionally (e.g. filtering, autocorrelation, spectrograms) as well as machine learning methods (convolutional neural nets) and showed promising results with respect to the use of structure-borne AE for process monitoring using the example of spatter formation

Required but disguised: Environmental signals in limestone-marl alternations

The nature of rhythmic carbonate-rich successions such as limestone^marl alternations has been, and still is, subject to controversy. The possibility of an entirely diagenetic origin for the rhythmic calcareous alternations is discarded by most authors. One problem with an entirely diagenetic, self-organized development of limestone^marl alternations is the fact that limestone and marl beds in many examples are laterally continuous over hundreds of meters or even kilometers. In an entirely self-organized system, lateral coupling would be very limited; thus one would expect that, rather than laterally continuous beds, randomly distributed elongate nodules would form. We address the origin of limestone^marl alternations using a computer model that simulates differential diagenesis of rhythmic calcareous successions. The setup uses a cellular automaton model to test whether laterally extensive, rhythmic calcareous alternations could develop from homogeneous sediments in a process of self-organization. Our model is a strong simplification of early diagenesis in fine-grained, partly calcareous sediments. It includes the relevant key mechanisms to the question whether an external trigger is required in order to obtain laterally extensive limestone^ marl alternations. Our model shows that diagenetic self-organization alone is not sufficient to produce laterally extensive, correlatable beds. Although an external control on bedding formation could be considered to have solved the problem as commonly assumed, we here suggest an interesting third possibility: the rhythmic alternations were formed through the interaction of both an external trigger and diagenetic self-organization. In particular we observe that a very limited external trigger, either in time or amplitude, readily forms correlatable beds in our otherwise diagenetic model. Remarkably, the resulting rhythmites often do not mirror the external trigger in a one-to-one fashion and may differ in phase, frequency and number of couplets. Therefore, the interpretation of calcareous rhythmites as a one-to-one archive of climate fluctuations may be misleading. Parameters independent of diagenetic alteration should be considered for unequivocal interpretation

Real world music object recognition

We present solutions to two of the most pressing issues in contemporary optical music recognition (OMR).We improve recognition accuracy on low-quality, real-world (i.e. containing ageing, lighting, or dirt artefacts among others) input data and provide confidence-rated model outputs to enable efficient human post-processing. Specifically, we present (i) a sophisticated input augmentation scheme that can reduce the gap between sanitised benchmarks and realistic tasks through a combination of synthetic data and noisy perturbations of real-world documents; (ii) an adversarial discriminative domain adaptation method that can be employed to improve the performance of OMR systems on low-quality data; (iii) a combination of model ensembles and prediction fusion, which generates trustworthy confidence ratings for each prediction. We evaluate our contributions on a newly created test set consisting of manually annotated pages of varying real-world quality, sourced from International Music Score Library Project (IMSLP) / the Petrucci Music Library. With the presented data augmentation scheme, we achieve a doubling in detection performance from 36.0% to 73.3% on noisy real-world data compared to state-of-the-art training. This result is then combined with robust confidence ratings paving the way forOMR to be deployed in the realworld. Additionally, we showthe merits of unsupervised adversarial domain adaptation for OMR raising the 36.0% baseline to 48.9%. All our code and data are freely available at: https://github.com/raember/s2anet/tree/TISMIR_publication

MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision

Prior to the deep learning era, shape was commonly used to describe the objects. Nowadays, state-of-the-art (SOTA) algorithms in medical imaging are predominantly diverging from computer vision, where voxel grids, meshes, point clouds, and implicit surface models are used. This is seen from numerous shape-related publications in premier vision conferences as well as the growing popularity of ShapeNet (about 51,300 models) and Princeton ModelNet (127,915 models). For the medical domain, we present a large collection of anatomical shapes (e.g., bones, organs, vessels) and 3D models of surgical instrument, called MedShapeNet, created to facilitate the translation of data-driven vision algorithms to medical applications and to adapt SOTA vision algorithms to medical problems. As a unique feature, we directly model the majority of shapes on the imaging data of real patients. As of today, MedShapeNet includes 23 dataset with more than 100,000 shapes that are paired with annotations (ground truth). Our data is freely accessible via a web interface and a Python application programming interface (API) and can be used for discriminative, reconstructive, and variational benchmarks as well as various applications in virtual, augmented, or mixed reality, and 3D printing. Exemplary, we present use cases in the fields of classification of brain tumors, facial and skull reconstructions, multi-class anatomy completion, education, and 3D printing. In future, we will extend the data and improve the interfaces. The project pages are: https://medshapenet.ikim.nrw/ and https://github.com/Jianningli/medshapenet-feedbackComment: 16 page

The DLR MiroSurge surgical robotic demonstrator

This chapter gives a brief overview of the DLR MiroSurge versatile surgical robotic demonstrator, its major components, mechatronic design and control paradigms. Central component is the robot arm Miro based on intelligent mechatronic technology developed at DLR. The system is adapted to various medical applications by attaching various instruments, e.g., the dedicated instrument MICA, and a surgeon workstation providing HD 3D vision and haptic feedback. Furthermore, a selection of current research topics concerning novel MIRS applications and system components are introduced