Understanding Optical Music Recognition

For over 50 years, researchers have been trying to teach computers to read music notation, referred to as Optical Music Recognition (OMR). However, this field is still difficult to access for new researchers, especially those without a significant musical background: Few introductory materials are available, and, furthermore, the field has struggled with defining itself and building a shared terminology. In this work, we address these shortcomings by (1) providing a robust definition of OMR and its relationship to related fields, (2) analyzing how OMR inverts the music encoding process to recover the musical notation and the musical semantics from documents, and (3) proposing a taxonomy of OMR, with most notably a novel taxonomy of applications. Additionally, we discuss how deep learning affects modern OMR research, as opposed to the traditional pipeline. Based on this work, the reader should be able to attain a basic understanding of OMR: its objectives, its inherent structure, its relationship to other fields, the state of the art, and the research opportunities it affords

Automatic concept learning via information lattices

Concept learning is about distilling interpretable rules and concepts from data, a prelude to more advanced knowledge discovery and problem solving in creative domains such as art and science. While concept learning is pervasive in humans, current artificial intelligent (AI) systems are mostly good at either applying human-distilled rules (rule-based AI) or capturing patterns in a task-driven fashion (pattern recognition), but not at learning concepts in a human-interpretable way similar to human-induced rules and theory. This thesis introduces a new learning problem---Automatic Concept Learning (ACL)---targeting self-explanation and self-exploration as the two principal pursuits; correspondingly, it proposes a new learning model---Information Lattice Learning (ILL)---combining computational abstraction and probabilistic rule learning as the two principal components. Woven around the core idea of abstraction, the entire ACL framework is presented as a generalization of Shannon’s information lattice that further brings learning into the picture. The core idea of abstraction is cast as a hierarchical, interpretable, data-free, and task-free clustering problem, seeded from universal priors such as simple algebra and symmetries. The main body of the thesis comprises three self-contained yet close-knit parts: theory, algorithms, and applications. The theory part presents the mathematical exposition of ACL, formalizing the key notions of abstraction, concept, probabilistic rule, and further the entire concept learning problem. The goal is to lay down a solid path towards algorithmic means that are computationally feasible, reliable, and human-interpretable. The algorithms part presents the computational development of ACL, that is, ILL. It puts together computational abstraction and statistical learning in the same algorithmic picture, creating a bridge that connects deductive (rule-based) and inductive (data-driven) approaches in AI. Aiming for human interpretability and model transparency in particular, ILL in many ways mimics human learning. This includes mechanism-driven abstraction generation, as well as a "teacher-student" loop that can distill customizable traces of rules for data summarization and data explanation. The applications part recapitulates the theory and algorithms through concrete examples. Music is used for demonstration and automatic music concept learning is thoroughly studied. This part details the implementation of MUS-ROVER, an automatic music theorist that distills music composition rules from sheet music. To better support music ACL and music AI in general, the twin system MUS-NET is built as a crowdsourcing platform for making and serving digital sheet music data sets