Automatic concept learning via information lattices

Concept learning is about distilling interpretable rules and concepts from data, a prelude to more advanced knowledge discovery and problem solving in creative domains such as art and science. While concept learning is pervasive in humans, current artificial intelligent (AI) systems are mostly good at either applying human-distilled rules (rule-based AI) or capturing patterns in a task-driven fashion (pattern recognition), but not at learning concepts in a human-interpretable way similar to human-induced rules and theory. This thesis introduces a new learning problem---Automatic Concept Learning (ACL)---targeting self-explanation and self-exploration as the two principal pursuits; correspondingly, it proposes a new learning model---Information Lattice Learning (ILL)---combining computational abstraction and probabilistic rule learning as the two principal components. Woven around the core idea of abstraction, the entire ACL framework is presented as a generalization of Shannon’s information lattice that further brings learning into the picture. The core idea of abstraction is cast as a hierarchical, interpretable, data-free, and task-free clustering problem, seeded from universal priors such as simple algebra and symmetries. The main body of the thesis comprises three self-contained yet close-knit parts: theory, algorithms, and applications. The theory part presents the mathematical exposition of ACL, formalizing the key notions of abstraction, concept, probabilistic rule, and further the entire concept learning problem. The goal is to lay down a solid path towards algorithmic means that are computationally feasible, reliable, and human-interpretable. The algorithms part presents the computational development of ACL, that is, ILL. It puts together computational abstraction and statistical learning in the same algorithmic picture, creating a bridge that connects deductive (rule-based) and inductive (data-driven) approaches in AI. Aiming for human interpretability and model transparency in particular, ILL in many ways mimics human learning. This includes mechanism-driven abstraction generation, as well as a "teacher-student" loop that can distill customizable traces of rules for data summarization and data explanation. The applications part recapitulates the theory and algorithms through concrete examples. Music is used for demonstration and automatic music concept learning is thoroughly studied. This part details the implementation of MUS-ROVER, an automatic music theorist that distills music composition rules from sheet music. To better support music ACL and music AI in general, the twin system MUS-NET is built as a crowdsourcing platform for making and serving digital sheet music data sets