Handwritten Music Recognition for Mensural notation with convolutional recurrent neural networks

[EN] Optical Music Recognition is the technology that allows computers to read music notation, which is also referred to as Handwritten Music Recognition when it is applied over handwritten notation. This technology aims at efficiently transcribing written music into a representation that can be further processed by a computer. This is of special interest to transcribe the large amount of music written in early notations, such as the Mensural notation, since they represent largely unexplored heritage for the musicological community. Traditional approaches to this problem are based on complex strategies with many explicit rules that only work for one particular type of manuscript. Machine learning approaches offer the promise of generalizable solutions, based on learning from just labelled examples. However, previous research has not achieved sufficiently acceptable results for handwritten Mensural notation. In this work we propose the use of deep neural networks, namely convolutional recurrent neural networks, which have proved effective in other similar domains such as handwritten text recognition. Our experimental results achieve, for the first time, recognition results that can be considered effective for transcribing handwritten Mensural notation, decreasing the symbol-level error rate of previous approaches from 25.7% to 7.0%. (C) 2019 Elsevier B.V. All rights reserved.First author thanks the support from the Spanish Ministry "HISPAMUS" project (TIN2017-86576-R), partially funded by the EU. The other authors were supported by the European Union's H2020 grant "Recognition and Enrichment of Archival Documents" (Ref. 674943), by the BBVA Foundacion through the 2017-2018 and 2018-2019 Digital Humanities research grants "Carabela" and "HistWeather - Dos Siglos de Datos Cilmaticos", and by EU JPICH project "HOME - History Of Medieval Europe"(Spanish PEICTI Ref. PCI2018-093122).Calvo-Zaragoza, J.; Toselli, AH.; Vidal, E. (2019). Handwritten Music Recognition for Mensural notation with convolutional recurrent neural networks. Pattern Recognition Letters. 128:115-121. https://doi.org/10.1016/j.patrec.2019.08.021S11512112

Finding What You Need, and Knowing What You Can Find: Digital Tools for Palaeographers in Musicology and Beyond

This chapter examines three projects that provide musicologists with a range of resources for managing and exploring their materials: DIAMM (Digital Image Archive of Medieval Music), CMME (Computerized Mensural Music Editing) and the software Gamera. Since 1998, DIAMM has been enhancing research of scholars worldwide by providing them with the best possible quality of digital images. In some cases these images are now the only access that scholars are permitted, since the original documents are lost or considered too fragile for further handling. For many sources, however, simply creating a very high-resolution image is not enough: sources are often damaged by age, misuse (usually Medieval ‘vandalism’), or poor conservation. To deal with damaged materials the project has developed methods of digital restoration using mainstream commercial software, which has revealed lost data in a wide variety of sources. The project also uses light sources ranging from ultraviolet to infrared in order to obtain better readings of erasures or material lost by heat or water damage. The ethics of digital restoration are discussed, as well as the concerns of the document holders. CMME and a database of musical sources and editions, provides scholars with a tool for making fluid editions and diplomatic transcriptions: without the need for a single fixed visual form on a printed page, a computerized edition system can utilize one editor’s transcription to create any number of visual forms and variant versions. Gamera, a toolkit for building document image recognition systems created by Ichiro Fujinaga is a broad recognition engine that grew out of music recognition, which can be adapted and developed to perform a number of tasks on both music and non-musical materials. Its application to several projects is discussed

Into the Wide – Into the Deep: Manuscript Research in the Digital Age. Introduction

Manuscript research is a wide field of scholarship which is integrated in core disciplines such as history, philology, or library science. Yet manuscript research is also crucial in other fields such as archaeology, history of arts, musicology or Egyptology, to name but a few. For all these disciplines, manuscripts are fundamental sources. There are different approaches to different types of manuscripts, but questions and perspectives, methodologies and tools are often quite similar. Innovations and new research strategies from one discipline can be transferred to and adopted by others. This article is an introduction to the second volume of the anthology "Codicology and Palaeography in the Digital Age" and gives an overview of current aspects in the field of manuscript studies in both theory and practice by showing the relatedness of the contributions to the volume at hand as well as its predecessor. The texts are roughly assigned to five interrelated areas of manuscript research: (I) the photographic capturing of the manuscript surface, (II) the description of the manuscript for a catalogue, (III) the scientific examination of material aspects, (IV) the analysis of the script and (V) the deep encoding of the text itself

Identifying music documents in a collection of images

Digital libraries and search engines are now well-equipped to find images of documents based on queries. Many images of music scores are now available, often mixed up with textual documents and images. For example, using the Google “images” search feature, a search for “Beethoven” will return a number of scores and manuscripts as well as pictures of the composer. In this paper we report on an investigation into methods to mechanically determine if a particular document is indeed a score, so that the user can specify that only musical scores should be returned. The goal is to find a minimal set of features that can be used as a quick test that will be applied to large numbers of documents. A variety of filters were considered, and two promising ones (run-length ratios and Hough transform) were evaluated. We found that a method based around run-lengths in vertical scans (RL) that out-performs a comparable algorithm using the Hough transform (HT). On a test set of 1030 images, RL achieved recall and precision of 97.8% and 88.4% respectively while HT achieved 97.8% and 73.5%. In terms of processor time, RL was more than five times as fast as HT

The Use of Seneca’s Texts in Antonii Radyvylovskyi’s Sermons

In this paper, through the example of Antonii Radyvylovskyi’s work, I examine the impact of Seneca’s texts on the philosophical component of Ukrainian church sermons from the Baroque period. The objective of this study is to investigate Radyvylovskyi’s use of Seneca’s texts in his own writing. The result should help better understand the ideological influence of ancient philosophy on the formation of the national philosophical tradition of the Baroque epoch. The contents of ideological borrowings from Seneca’s texts and the mechanisms of their use are traced. A list of Seneca’s texts from which Radyvylovskyi quotes is provided. It is also shown that Radyvylovskyi uses Seneca’s authority in his moral teachings and philosophical thinking about the characteristics of human nature. We conclude by commenting on Radyvylovskyi’s creative use of Seneca’s ideas and the significant philosophical component of his written legacy

MS-059: Papers of Charles H. Glatfelter (Class of 1946)

This collection consists of research notes, sources, and manuscripts for A Salutary Influence: Gettysburg College, 1832-1985, written by Dr. Charles H. Glatfelter. It also contains committee papers, department chair files, and faculty manuals and papers. Special Collections and College Archives Finding Aids are discovery tools used to describe and provide access to our holdings. Finding aids include historical and biographical information about each collection in addition to inventories of their content. More information about our collections can be found on our website http://www.gettysburg.edu/special_collections/collections/.https://cupola.gettysburg.edu/findingaidsall/1054/thumbnail.jp

Multi-task Layout Analysis of Handwritten Musical Scores

[EN] Document Layout Analysis (DLA) is a process that must be performed before attempting to recognize the content of handwritten musical scores by a modern automatic or semiautomatic system. DLA should provide the segmentation of the document image into semantically useful region types such as staff, lyrics, etc. In this paper we extend our previous work for DLA of handwritten text documents to also address complex handwritten music scores. This system is able to perform region segmentation, region classification and baseline detection in an integrated manner. Several experiments were performed in two different datasets in order to validate this approach and assess it in different scenarios. Results show high accuracy in such complex manuscripts and very competent computational time, which is a good indicator of the scalability of the method for very large collections.This work was partially supported by the Universitat Politecnica de Valencia under grant FPI-420II/899, a 2017-2018 Digital Humanities research grant of the BBVA Foundation for the project Carabela, the History Of Medieval Europe (HOME) project (Ref.: PCI2018-093122) and through the EU project READ (Horizon-2020 program, grant Ref. 674943). NVIDIA Corporation kindly donated the Titan X GPU used for this research.Quirós, L.; Toselli, AH.; Vidal, E. (2019). Multi-task Layout Analysis of Handwritten Musical Scores. Springer. 123-134. https://doi.org/10.1007/978-3-030-31321-0_11S123134Burgoyne, J.A., Ouyang, Y., Himmelman, T., Devaney, J., Pugin, L., Fujinaga, I.: Lyric extraction and recognition on digital images of early music sources. In: Proceedings of the 10th International Society for Music Information Retrieval Conference, vol. 10, pp. 723–727 (2009)Calvo-Zaragoza, J., Toselli, A.H., Vidal, E.: Probabilistic music-symbol spotting in handwritten scores. In: 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 558–563, August 2018Calvo-Zaragoza, J., Zhang, K., Saleh, Z., Vigliensoni, G., Fujinaga, I.: Music document layout analysis through machine learning and human feedback. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 02, pp. 23–24, November 2017Calvo-Zaragoza, J., Castellanos, F.J., Vigliensoni, G., Fujinaga, I.: Deep neural networks for document processing of music score images. Appl. Sci. 8(5), 654 (2018). (2076-3417)Calvo-Zaragoza, J., Toselli, A.H., Vidal, E.: Handwritten music recognition for mensural notation: formulation, data and baseline results. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1081–1086. IEEE (2017)Campos, V.B., Calvo-Zaragoza, J., Toselli, A.H., Ruiz, E.V.: Sheet music statistical layout analysis. In: 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 313–318. IEEE (2016)Castellanos, F.J., Calvo-Zaragoza, J., Vigliensoni, G., Fujinaga, I.: Document analysis of music score images with selectional auto-encoders. In: 19th International Society for Music Information Retrieval Conference, pp. 256–263 (2018)Grüning, T., Labahn, R., Diem, M., Kleber, F., Fiel, S.: READ-BAD: a new dataset and evaluation scheme for baseline detection in archival documents. CoRR abs/1705.03311 (2017). http://arxiv.org/abs/1705.03311Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations (ICLR) (2015)Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Quirós, L.: Multi-task handwritten document layout analysis. ArXiv e-prints, 1806.08852 (2018). https://arxiv.org/abs/1806.08852Quirós, L., Bosch, V., Serrano, L., Toselli, A.H., Vidal, E.: From HMMs to RNNs: computer-assisted transcription of a handwritten notarial records collection. In: 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 116–121. IEEE, August 2018Rebelo, A., Fujinaga, I., Paszkiewicz, F., Marcal, A.R., Guedes, C., Cardoso, J.S.: Optical music recognition: state-of-the-art and open issues. Int. J. Multimed. Inf. Retrieval 1(3), 173–190 (2012)Sánchez, J.A., Romero, V., Toselli, A.H., Villegas, M., Vidal, E.: ICDAR2017 competition on handwritten text recognition on the READ dataset. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1383–1388. IEEE (2017)Suzuki, S., et al.: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985

Hybrid hidden Markov models and artificial neural networks for handwritten music recognition in mensural notation

In this paper, we present a hybrid approach using hidden Markov models (HMM) and artificial neural networks to deal with the task of handwritten Music Recognition in mensural notation. Previous works have shown that the task can be addressed with Gaussian density HMMs that can be trained and used in an end-to-end manner, that is, without prior segmentation of the symbols. However, the results achieved using that approach are not sufficiently accurate to be useful in practice. In this work, we hybridize HMMs with deep multilayer perceptrons (MLPs), which lead to remarkable improvements in optical symbol modeling. Moreover, this hybrid architecture maintains important advantages of HMMs such as the ability to properly model variable-length symbol sequences through segmentation-free training, and the simplicity and robustness of combining optical models with N-gram language models, which provide statistical a priori information about regularities in musical symbol concatenation observed in the training data. The results obtained with the proposed hybrid MLP-HMM approach outperform previous works by a wide margin, achieving symbol-level error rates around 26%, as compared with about 40% reported in previous works