1,912 research outputs found
Optical Music Recognition: State of the Art and Major Challenges
Optical Music Recognition (OMR) is concerned with transcribing sheet music into a machine-readable format. The transcribed copy should allow musicians to compose, play and edit music by taking a picture of a music sheet. Complete transcription of sheet music would also enable more efficient archival. OMR facilitates examining sheet music statistically or searching for patterns of notations, thus helping use cases in digital musicology too. Recently, there has been a shift in OMR from using conventional computer vision techniques towards a deep learning approach. In this paper, we review relevant works in OMR, including fundamental methods and significant outcomes, and highlight different stages of the OMR pipeline. These stages often lack standard input and output representation and standardised evaluation. Therefore, comparing different approaches and evaluating the impact of different processing methods can become rather complex. This paper provides recommendations for future work, addressing some of the highlighted issues and represents a position in furthering this important field of research
Proceedings of the 4th International Workshop on Reading Music Systems
The International Workshop on Reading Music Systems (WoRMS) is a workshop
that tries to connect researchers who develop systems for reading music, such
as in the field of Optical Music Recognition, with other researchers and
practitioners that could benefit from such systems, like librarians or
musicologists.
The relevant topics of interest for the workshop include, but are not limited
to: Music reading systems; Optical music recognition; Datasets and performance
evaluation; Image processing on music scores; Writer identification; Authoring,
editing, storing and presentation systems for music scores; Multi-modal
systems; Novel input-methods for music to produce written music; Web-based
Music Information Retrieval services; Applications and projects; Use-cases
related to written music.
These are the proceedings of the 4th International Workshop on Reading Music
Systems, held online on Nov. 18th 2022.Comment: Proceedings edited by Jorge Calvo-Zaragoza, Alexander Pacha and Elona
Shatr
Understanding Optical Music Recognition
For over 50 years, researchers have been trying to teach computers to read music notation, referred to as Optical Music Recognition (OMR). However, this field is still difficult to access for new researchers, especially those without a significant musical background: Few introductory materials are available, and, furthermore, the field has struggled with defining itself and building a shared terminology. In this work, we address these shortcomings by (1) providing a robust definition of OMR and its relationship to related fields, (2) analyzing how OMR inverts the music encoding process to recover the musical notation and the musical semantics from documents, and (3) proposing a taxonomy of OMR, with most notably a novel taxonomy of applications. Additionally, we discuss how deep learning affects modern OMR research, as opposed to the traditional pipeline. Based on this work, the reader should be able to attain a basic understanding of OMR: its objectives, its inherent structure, its relationship to other fields, the state of the art, and the research opportunities it affords
Integration of Language Models in Sequence to Sequence Optical Music Recognition Systems
El present projecte és un estudi del potencial d'integrar per mitjà de diverses tècniques un model de llenguatge a un sistema de Reconeixement Òptic de Partitures (OMR) basat en una arquitectura Sequence to Sequence. L'objectiu és millorar el rendiment del model sobre partitures manuscrites antigues, que són especialment complexes d'interpretar a causa del seu elevat grau de variabilitat i les distorsions que solen incorporar.The following project is a study of the potential of integrating a language model into a Sequence to Sequence-based Optical Music Recognition (OMR) system through various techniques. The goal is to improve the performance of the model on handwritten old music scores, whose interpretation is particularly error-prone due to their high degree of variability and distortion.El presente proyecto es un estudio del potencial de integrar por medio de varias técnicas un modelo de lenguaje a un sistema de Reconocimiento Óptico de Partituras (OMR) basado en una arquitectura Sequence to Sequence. El objetivo es mejorar el rendimiento del modelo sobre partituras manuscritas antiguas, que son especialmente complicadas de interpretar a causa de su elevado grado de variabilidad y las distorsiones que suelen incorporar
Applying Automatic Translation for Optical Music Recognition’s Encoding Step
Optical music recognition is a research field whose efforts have been mainly focused, due to the difficulties involved in its processes, on document and image recognition. However, there is a final step after the recognition phase that has not been properly addressed or discussed, and which is relevant to obtaining a standard digital score from the recognition process: the step of encoding data into a standard file format. In this paper, we address this task by proposing and evaluating the feasibility of using machine translation techniques, using statistical approaches and neural systems, to automatically convert the results of graphical encoding recognition into a standard semantic format, which can be exported as a digital score. We also discuss the implications, challenges and details to be taken into account when applying machine translation techniques to music languages, which are very different from natural human languages. This needs to be addressed prior to performing experiments and has not been reported in previous works. We also describe and detail experimental results, and conclude that applying machine translation techniques is a suitable solution for this task, as they have proven to obtain robust results.This work was supported by the Spanish Ministry HISPAMUS project TIN2017-86576-R, partially funded by the EU, and by the Generalitat Valenciana through project GV/2020/030
GraphVid: It Only Takes a Few Nodes to Understand a Video
We propose a concise representation of videos that encode perceptually
meaningful features into graphs. With this representation, we aim to leverage
the large amount of redundancies in videos and save computations. First, we
construct superpixel-based graph representations of videos by considering
superpixels as graph nodes and create spatial and temporal connections between
adjacent superpixels. Then, we leverage Graph Convolutional Networks to process
this representation and predict the desired output. As a result, we are able to
train models with much fewer parameters, which translates into short training
periods and a reduction in computation resource requirements. A comprehensive
experimental study on the publicly available datasets Kinetics-400 and Charades
shows that the proposed method is highly cost-effective and uses limited
commodity hardware during training and inference. It reduces the computational
requirements 10-fold while achieving results that are comparable to
state-of-the-art methods. We believe that the proposed approach is a promising
direction that could open the door to solving video understanding more
efficiently and enable more resource limited users to thrive in this research
field.Comment: Accepted to ECCV2022 (Oral
- …