2,440 research outputs found
SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge
Surgical tool segmentation and action recognition are fundamental building
blocks in many computer-assisted intervention applications, ranging from
surgical skills assessment to decision support systems. Nowadays,
learning-based action recognition and segmentation approaches outperform
classical methods, relying, however, on large, annotated datasets. Furthermore,
action recognition and tool segmentation algorithms are often trained and make
predictions in isolation from each other, without exploiting potential
cross-task relationships. With the EndoVis 2022 SAR-RARP50 challenge, we
release the first multimodal, publicly available, in-vivo, dataset for surgical
action recognition and semantic instrumentation segmentation, containing 50
suturing video segments of Robotic Assisted Radical Prostatectomy (RARP). The
aim of the challenge is twofold. First, to enable researchers to leverage the
scale of the provided dataset and develop robust and highly accurate
single-task action recognition and tool segmentation approaches in the surgical
domain. Second, to further explore the potential of multitask-based learning
approaches and determine their comparative advantage against their single-task
counterparts. A total of 12 teams participated in the challenge, contributing 7
action recognition methods, 9 instrument segmentation techniques, and 4
multitask approaches that integrated both action recognition and instrument
segmentation. The complete SAR-RARP50 dataset is available at:
https://rdr.ucl.ac.uk/projects/SARRARP50_Segmentation_of_surgical_instrumentation_and_Action_Recognition_on_Robot-Assisted_Radical_Prostatectomy_Challenge/19109
Pitch-Informed Instrument Assignment using a Deep Convolutional Network with Multiple Kernel Shapes.
This paper proposes a deep convolutional neural network
for performing note-level instrument assignment. Given a
polyphonic multi-instrumental music signal along with its
ground truth or predicted notes, the objective is to assign
an instrumental source for each note. This problem is addressed as a pitch-informed classification task where each
note is analysed individually. We also propose to utilise
several kernel shapes in the convolutional layers in order
to facilitate learning of timbre-discriminative feature maps.
Experiments on the MusicNet dataset using 7 instrument
classes show that our approach is able to achieve an average F-score of 0.904 when the original multi-pitch annotations are used as the pitch information for the system, and
that it also excels if the note information is provided using
third-party multi-pitch estimation algorithms. We also include ablation studies investigating the effects of the use of
multiple kernel shapes and comparing different input representations for the audio and the note-related information
Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes
This paper proposes a deep convolutional neural network for performing
note-level instrument assignment. Given a polyphonic multi-instrumental music
signal along with its ground truth or predicted notes, the objective is to
assign an instrumental source for each note. This problem is addressed as a
pitch-informed classification task where each note is analysed individually. We
also propose to utilise several kernel shapes in the convolutional layers in
order to facilitate learning of efficient timbre-discriminative feature maps.
Experiments on the MusicNet dataset using 7 instrument classes show that our
approach is able to achieve an average F-score of 0.904 when the original
multi-pitch annotations are used as the pitch information for the system, and
that it also excels if the note information is provided using third-party
multi-pitch estimation algorithms. We also include ablation studies
investigating the effects of the use of multiple kernel shapes and comparing
different input representations for the audio and the note-related information.Comment: 4 figures, 4 tables and 7 pages. Accepted for publication at ISMIR
Conference 202
Pitch-informed instrument assignment using a deep convolutional network with multiple kernel shapes
This paper proposes a deep convolutional neural network for performing note-level instrument assignment. Given a polyphonic multi-instrumental music signal along with its ground truth or predicted notes, the objective is to assign an instrumental source for each note. This problem is addressed as a pitch-informed classification task where each note is analysed individually. We also propose to utilise several kernel shapes in the convolutional layers in order to facilitate learning of timbre-discriminative feature maps. Experiments on the MusicNet dataset using 7 instrument classes show that our approach is able to achieve an average F-score of 0.904 when the original multi-pitch annotations are used as the pitch information for the system, and that it also excels if the note information is provided using third-party multi-pitch estimation algorithms. We also include ablation studies investigating the effects of the use of multiple kernel shapes and comparing different input representations for the audio and the note-related information
- …