Propositionalisation of multi-instance data using random forests

Multi-instance learning is a generalisation of attribute-value learning where examples for learning consist of labeled bags (i.e. multi-sets) of instances. This learning setting is more computationally challenging than attribute-value learning and a natural fit for important application areas of machine learning such as classification of molecules and image classification. One approach to solve multi-instance learning problems is to apply propositionalisation, where bags of data are converted into vectors of attribute-value pairs so that a standard propositional (i.e. attribute-value) learning algorithm can be applied. This approach is attractive because of the large number of propositional learning algorithms that have been developed and can thus be applied to the propositionalised data. In this paper, we empirically investigate a variant of an existing propositionalisation method called TLC. TLC uses a single decision tree to obtain propositionalised data. Our variant applies a random forest instead and is motivated by the potential increase in robustness that this may yield. We present results on synthetic and real-world data from the above two application domains showing that it indeed yields increased classification accuracy when applying boosting and support vector machines to classify the propositionalised data

A Comparison of Multi-instance Learning Algorithms

Motivated by various challenging real-world applications, such as drug activity prediction and image retrieval, multi-instance (MI) learning has attracted considerable interest in recent years. Compared with standard supervised learning, the MI learning task is more difficult as the label information of each training example is incomplete. Many MI algorithms have been proposed. Some of them are specifically designed for MI problems whereas others have been upgraded or adapted from standard single-instance learning algorithms. Most algorithms have been evaluated on only one or two benchmark datasets, and there is a lack of systematic comparisons of MI learning algorithms. This thesis presents a comprehensive study of MI learning algorithms that aims to compare their performance and find a suitable way to properly address different MI problems. First, it briefly reviews the history of research on MI learning. Then it discusses five general classes of MI approaches that cover a total of 16 MI algorithms. After that, it presents empirical results for these algorithms that were obtained from 15 datasets which involve five different real-world application domains. Finally, some conclusions are drawn from these results: (1) applying suitable standard single-instance learners to MI problems can often generate the best result on the datasets that were tested, (2) algorithms exploiting the standard asymmetric MI assumption do not show significant advantages over approaches using the so-called collective assumption, and (3) different MI approaches are suitable for different application domains, and no MI algorithm works best on all MI problems

Learning and Interpreting Multi-Multi-Instance Learning Networks

We introduce an extension of the multi-instance learning problem where examples are organized as nested bags of instances (e.g., a document could be represented as a bag of sentences, which in turn are bags of words). This framework can be useful in various scenarios, such as text and image classification, but also supervised learning over graphs. As a further advantage, multi-multi instance learning enables a particular way of interpreting predictions and the decision function. Our approach is based on a special neural network layer, called bag-layer, whose units aggregate bags of inputs of arbitrary size. We prove theoretically that the associated class of functions contains all Boolean functions over sets of sets of instances and we provide empirical evidence that functions of this kind can be actually learned on semi-synthetic datasets. We finally present experiments on text classification, on citation graphs, and social graph data, which show that our model obtains competitive results with respect to accuracy when compared to other approaches such as convolutional networks on graphs, while at the same time it supports a general approach to interpret the learnt model, as well as explain individual predictions.Comment: JML

Rerepresenting and Restructuring Domain Theories: A Constructive Induction Approach

Theory revision integrates inductive learning and background knowledge by combining training examples with a coarse domain theory to produce a more accurate theory. There are two challenges that theory revision and other theory-guided systems face. First, a representation language appropriate for the initial theory may be inappropriate for an improved theory. While the original representation may concisely express the initial theory, a more accurate theory forced to use that same representation may be bulky, cumbersome, and difficult to reach. Second, a theory structure suitable for a coarse domain theory may be insufficient for a fine-tuned theory. Systems that produce only small, local changes to a theory have limited value for accomplishing complex structural alterations that may be required. Consequently, advanced theory-guided learning systems require flexible representation and flexible structure. An analysis of various theory revision systems and theory-guided learning systems reveals specific strengths and weaknesses in terms of these two desired properties. Designed to capture the underlying qualities of each system, a new system uses theory-guided constructive induction. Experiments in three domains show improvement over previous theory-guided systems. This leads to a study of the behavior, limitations, and potential of theory-guided constructive induction.Comment: See http://www.jair.org/ for an online appendix and other files accompanying this articl

Graphical Models and Symmetries : Loopy Belief Propagation Approaches

Whenever a person or an automated system has to reason in uncertain domains, probability theory is necessary. Probabilistic graphical models allow us to build statistical models that capture complex dependencies between random variables. Inference in these models, however, can easily become intractable. Typical ways to address this scaling issue are inference by approximate message-passing, stochastic gradients, and MapReduce, among others. Exploiting the symmetries of graphical models, however, has not yet been considered for scaling statistical machine learning applications. One instance of graphical models that are inherently symmetric are statistical relational models. These have recently gained attraction within the machine learning and AI communities and combine probability theory with first-order logic, thereby allowing for an efficient representation of structured relational domains. The provided formalisms to compactly represent complex real-world domains enable us to effectively describe large problem instances. Inference within and training of graphical models, however, have not been able to keep pace with the increased representational power. This thesis tackles two major aspects of graphical models and shows that both inference and training can indeed benefit from exploiting symmetries. It first deals with efficient inference exploiting symmetries in graphical models for various query types. We introduce lifted loopy belief propagation (lifted LBP), the first lifted parallel inference approach for relational as well as propositional graphical models. Lifted LBP can effectively speed up marginal inference, but cannot straightforwardly be applied to other types of queries. Thus we also demonstrate efficient lifted algorithms for MAP inference and higher order marginals, as well as the efficient handling of multiple inference tasks. Then we turn to the training of graphical models and introduce the first lifted online training for relational models. Our training procedure and the MapReduce lifting for loopy belief propagation combine lifting with the traditional statistical approaches to scaling, thereby bridging the gap between statistical relational learning and traditional statistical machine learning

Developing Novices' Professional Scripts for Teaching: An Investigation of Teacher Education Practice

This dissertation is my effort to better understand how teacher educators teach professional, anti-racist teaching practice to novice teachers. I argue that one important way to interrupt systemic racism in schooling is to design teacher education that both teaches novice teachers what anti-racist practice is and helps them gain some initial skill with how to enact it. I develop a conceptual tool, professional scripts for teaching, to identify, parse, and, in this study, teach the anti-racist teaching practice of assigning competence (Cohen, 1973; Cohen, Lotan, Scarloss, & Arellano, 1999; Featherstone et al., 2011) to novices. Professional scripts for teaching help to define what “counts” as acceptable professional practice by describing patterns of practice that reflect anti-racist professional ethics to bound the work of teaching. Professional scripts for teaching foreground the relationships among teachers’ professional ethics, decision-making, and in-the-moment patterns of practice. This study comprises a first-person inquiry (Ball, 2000) into my teacher education practice for using professional scripts for teaching to teach assigning competence to a group of novice teachers. First-person inquiry is a form of qualitative case study closely related to methodological approaches such as action research, teacher narratives, and reflection in or on teaching, which demand an intentional and disciplined marrying of the enactment of practice with the analysis of practice. I investigated my own work to use professional scripts for teaching to design and teach the practice of assigning competence in a secondary methods course. I used my teacher education practice as the site of inquiry because the kind of teacher education work that I sought to study is different from what is most commonly practiced in the field. I designed, delivered, and analyzed a practice-focused teaching methods course for a group of novice secondary English Language Arts teachers in an alternative certification program. I examined transcripts of bi-weekly planning meetings held in collaboration with another teacher educator, course materials generated across the semester, videos and transcripts of class sessions, and written reflections on instruction composed immediately after each class session to answer: What is involved in the work for a teacher educator to translate anti-racist practice from the research literature into a professional script for teaching that can be taught in practice-based teacher education? I identify four endemic requirements of practice-based teacher education work aimed at anti-racist practice: (1) the importance of forming productive pedagogical relationships with novices in order to teach anti-racist practice; (2) the need to connect instruction in the practice to the professional ethics of the practice; (3) the requirement to develop decompositions of focal practices that both capture their complexity and reflect enactment; and (4) managing challenges associated with designing meaningful approximations of focal practices. The work to move from ideas about teaching anti-racist practice to the teaching of anti-racist practice is not straightforward, even for a teacher educator who has relevant knowledge, experience, and commitments to do such work. Some of the complexity arises from common features of programmatic contexts that perpetuate practices that are rooted in structural racism and can interfere with teacher educators’ efforts to teach anti-racist practice. Some of the complexity stems from the inherent difficulty of making anti-racist practice accessible to novices in practice-based teacher education. I offer what I have learned as a possible resource for other teacher educators involved in this necessary and difficult work.PHDEducational StudiesUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144076/1/rgadd_1.pd

Towards a conceptual framework of enterprise support for pro-environmental small and medium-sized enterprises: A contextualised review of diverse knowledge domains

Whilst there are well-established bodies of knowledge about enterprise support and the role of entrepreneurial learning for SMEs (Small and medium-sized enterprises) in general and a growing body of evidence relating to environmental capabilities, green/eco-innovation, sustainable supply chains and green skills for SMEs in particular, there is little empirical and peer reviewed literature that address approaches to enterprise support specifically focussed on the needs of the growing number of pro-environmental SMEs. This study undertakes a contextualised review of diverse knowledge domains to identify the key features of enterprise support for pro-environmental SMEs. In doing so, the paper plots the knowledge journey of experienced academic programme providers, from the initial design of an enterprise support programme for pro-environmental SMEs, through a thematic review of academic, grey and other related literature and finally presents a propositional and normative conceptual framework that proposes eight key features of enterprise support for pro-environmental SMEs. The resulting ‘framework for action’ aims to offer a practical tool for providers of pro-environmental enterprise support to review and improve their own provision, an analytical frame for other researchers in this field and a benchmark for SMEs seeking guidance on their pathway to net-zero business performance

Interpretation of Natural-language Robot Instructions: Probabilistic Knowledge Representation, Learning, and Reasoning

A robot that can be simply told in natural language what to do -- this has been one of the ultimate long-standing goals in both Artificial Intelligence and Robotics research. In near-future applications, robotic assistants and companions will have to understand and perform commands such as set the table for dinner'', make pancakes for breakfast'', or cut the pizza into 8 pieces.'' Although such instructions are only vaguely formulated, complex sequences of sophisticated and accurate manipulation activities need to be carried out in order to accomplish the respective tasks. The acquisition of knowledge about how to perform these activities from huge collections of natural-language instructions from the Internet has garnered a lot of attention within the last decade. However, natural language is typically massively unspecific, incomplete, ambiguous and vague and thus requires powerful means for interpretation. This work presents PRAC -- Probabilistic Action Cores -- an interpreter for natural-language instructions which is able to resolve vagueness and ambiguity in natural language and infer missing information pieces that are required to render an instruction executable by a robot. To this end, PRAC formulates the problem of instruction interpretation as a reasoning problem in first-order probabilistic knowledge bases. In particular, the system uses Markov logic networks as a carrier formalism for encoding uncertain knowledge. A novel framework for reasoning about unmodeled symbolic concepts is introduced, which incorporates ontological knowledge from taxonomies and exploits semantically similar relational structures in a domain of discourse. The resulting reasoning framework thus enables more compact representations of knowledge and exhibits strong generalization performance when being learnt from very sparse data. Furthermore, a novel approach for completing directives is presented, which applies semantic analogical reasoning to transfer knowledge collected from thousands of natural-language instruction sheets to new situations. In addition, a cohesive processing pipeline is described that transforms vague and incomplete task formulations into sequences of formally specified robot plans. The system is connected to a plan executive that is able to execute the computed plans in a simulator. Experiments conducted in a publicly accessible, browser-based web interface showcase that PRAC is capable of closing the loop from natural-language instructions to their execution by a robot