What do You Want to do Today? : Relevant-Information Bookkeeping in Goal-Oriented Behaviour

We extend existing models and methods for the informational treatment of the perception-action loop to the case of goaloriented behaviour and introduce the notion of relevant goal information as the amount of information an agent necessarily has to maintain about its goal. Starting from the hypothesis that organisms use information economically, we study the structure of this information and how goal-information parsimony can guide behaviour. It is shown how these methods lead to a general definition and quantification of sub-goals and how the biologically motivated hypothesis of information parsimony gives rise to the emergence of behavioural properties such as least-commitment and goal-concealing

A literature survey of active machine learning in the context of natural language processing

Active learning is a supervised machine learning technique in which the learner is in control of the data used for learning. That control is utilized by the learner to ask an oracle, typically a human with extensive knowledge of the domain at hand, about the classes of the instances for which the model learned so far makes unreliable predictions. The active learning process takes as input a set of labeled examples, as well as a larger set of unlabeled examples, and produces a classifier and a relatively small set of newly labeled data. The overall goal is to create as good a classifier as possible, without having to mark-up and supply the learner with more data than necessary. The learning process aims at keeping the human annotation effort to a minimum, only asking for advice where the training utility of the result of such a query is high. Active learning has been successfully applied to a number of natural language processing tasks, such as, information extraction, named entity recognition, text categorization, part-of-speech tagging, parsing, and word sense disambiguation. This report is a literature survey of active learning from the perspective of natural language processing

Characterization of the signal sequence binding domain of Ffh by genetics and comparative analysis

The signal recognition particle (SRP) is a ribonucleoprotein complex whose components are highly conserved throughout all three domains of life, where it functions to target proteins to extracytoplasmic locations. In Escherichia coli, the SRP is comprised of a single essential protein (Ffh) in complex with a 4.5S RNA species. To better understand how the structure of Ffh contributes to its function, we have used genetic approaches to isolate and characterize new ffh mutants altered in two distinct domains of the protein. Both domains of interest have been implicated as being important for binding to hydrophobic signal peptides of membrane proteins. These studies include using a random sequence approach to identify amino acids important for activity of the finger loop domain. The finger loop was identified from structural analysis as a ∼ 20 amino acid domain with the unusual properties of being both hydrophobic and exposed near the surface of Ffh. Approximately 1% of the random sequences were able to replace the FL domain of Ffh. Bioinformatic analysis of the random sequences revealed that all of the complementing sequences followed a trend of high hydrophobicity at the amino-terminus that decreased towards the carboxy end. These observations were validated by observing that mutants that deviated from this trend rendered Ffh nonfunctional. Mutants were characterized by growth rates that allowed the sequences to be grouped into three functional classes. Secondary and tertiary structure predictions suggested that the products of the random sequences lack extensive secondary structure, which is consistent with the role of the finger loop in binding a variety of ligands. To address the importance of the conserved methionine residues at the carboxy-terminal region (M-domain) in SRP function, we combined phylogenetic comparisons with functional studies, including replacing methionine residues within the M-domain with other residues that varied in hydrophobicity, side chain flexibility and charge. These studies revealed that, in E. coli, the M-domain of Ffh was able to tolerate substitutions of five different hydrophobic amino acids including valine, phenylalanine, tyrosine, tryptophan and isoleucine for the conserved methionine residues found in helix αM4 and the extreme C-terminus. Phylogenetic comparisons of microorganisms with varying optimal growth temperatures revealed methionine residues were substituted with amino acids containing less flexible side chains. Interestingly, we observed that mutants containing less flexible residues were able to support cell viability at higher growth temperatures better than at lower temperatures. Phylogenetic comparisons also revealed three positions where methionine is highly conserved. We show that replacing all of the methionine residues, except these three highly conserved residues, with valine yielded a functional SRP. In contrast to predicted results, these studies reveal that the M-domain of Ffh is highly flexible in content and that methionines are not absolutely required for SRP function. Collectively, these efforts have contributed to our understanding of SRP function by identifying key features essential for the function of the signal sequence binding domain of the Ffh protein component of SRP

Bootstrapping Named Entity Annotation by Means of Active Machine Learning: A Method for Creating Corpora

This thesis describes the development and in-depth empirical investigation of a method, called BootMark, for bootstrapping the marking up of named entities in textual documents. The reason for working with documents, as opposed to for instance sentences or phrases, is that the BootMark method is concerned with the creation of corpora. The claim made in the thesis is that BootMark requires a human annotator to manually annotate fewer documents in order to produce a named entity recognizer with a given performance, than would be needed if the documents forming the basis for the recognizer were randomly drawn from the same corpus. The intention is then to use the created named en- tity recognizer as a pre-tagger and thus eventually turn the manual annotation process into one in which the annotator reviews system-suggested annotations rather than creating new ones from scratch. The BootMark method consists of three phases: (1) Manual annotation of a set of documents; (2) Bootstrapping – active machine learning for the purpose of selecting which document to an- notate next; (3) The remaining unannotated documents of the original corpus are marked up using pre-tagging with revision. Five emerging issues are identified, described and empirically investigated in the thesis. Their common denominator is that they all depend on the real- ization of the named entity recognition task, and as such, require the context of a practical setting in order to be properly addressed. The emerging issues are related to: (1) the characteristics of the named entity recognition task and the base learners used in conjunction with it; (2) the constitution of the set of documents annotated by the human annotator in phase one in order to start the bootstrapping process; (3) the active selection of the documents to annotate in phase two; (4) the monitoring and termination of the active learning carried out in phase two, including a new intrinsic stopping criterion for committee-based active learning; and (5) the applicability of the named entity recognizer created during phase two as a pre-tagger in phase three. The outcomes of the empirical investigations concerning the emerging is- sues support the claim made in the thesis. The results also suggest that while the recognizer produced in phases one and two is as useful for pre-tagging as a recognizer created from randomly selected documents, the applicability of the recognizer as a pre-tagger is best investigated by conducting a user study involving real annotators working on a real named entity recognition task

Regret Minimization in MDPs with Options without Prior Knowledge

International audienceThe option framework integrates temporal abstraction into the reinforcement learning model through the introduction of macro-actions (i.e., options). Recent works leveraged the mapping of Markov decision processes (MDPs) with options to semi-MDPs (SMDPs) and introduced SMDP-versions of exploration-exploitation algorithms (e.g., RMAX-SMDP and UCRL-SMDP) to analyze the impact of options on the learning performance. Nonetheless, the PAC-SMDP sample complexity of RMAX-SMDP can hardly be translated into equivalent PAC-MDP theoretical guarantees, while the regret analysis of UCRL-SMDP requires prior knowledge of the distributions of the cumulative reward and duration of each option, which are hardly available in practice. In this paper, we remove this limitation by combining the SMDP view together with the inner Markov structure of options into a novel algorithm whose regret performance matches UCRL-SMDP's up to an additive regret term. We show scenarios where this term is negligible and the advantage of temporal abstraction is preserved. We also report preliminary empirical results supporting the theoretical findings

Sensory prediction mechanisms in action

When we produce an action we generate predictions about the sensory consequences that are likely to ensue. This thesis tests a series of claims about the functional contribution these predictions make to perception, the role that such predictions play in processing the reactions of others, and the range of sensory inputs that these prediction mechanisms operate over. Chapter 1 outlines the theoretical background to each of these claims, alongside the previous literature that motivates subsequent experiments. The first three empirical chapters focus on claims about the functional role of sensory predictions during action: that they act to ‘cancel’ perception of expected action outcomes. Chapter 2 investigates this hypothesis in the context an intensity judgement task, Chapter 3 tests the hypothesis in the context of a signal detection task and Chapter 4 assess how predictions generated during action influence multivariate measures of visual brain activity, recorded via functional magnetic resonance imaging. Chapter 5 investigates the claim that sensory predictions during action support the processing of imitative reactions in others. Two psychophysical experiments are reported which investigate whether sensory predictions generated during action have temporal properties needed to support processing of others’ reactions. Chapter 6 investigates whether sensory predictions generated during action influence the ‘when’ - as well as the ‘what’ - of perception. Four psychophysical experiments investigate whether the temporal features of executed actions are incorporated into duration perception. Chapters 7 and 8 report preliminary investigations into the mechanism underlying these effects. Chapter 7 assesses whether these influences arise through a mechanism that is primarily tuned to biological action outcomes. Chapter 8 investigates whether these effects arise as a result of statistical learning about the relationship between actions and outcomes. Chapter 9 summarises the studies presented in the thesis, and outlines their implications for thinking about sensory prediction during action

Non-parametric modeling in non-intrusive load monitoring

Non-intrusive Load Monitoring (NILM) is an approach to the increasingly important task of residential energy analytics. Transparency of energy resources and consumption habits presents opportunities and benefits at all ends of the energy supply-chain, including the end-user. At present, there is no feasible infrastructure available to monitor individual appliances at a large scale. The goal of NILM is to provide appliance monitoring using only the available aggregate data, side-stepping the need for expensive and intrusive monitoring equipment. The present work showcases two self-contained, fully unsupervised NILM solutions: the first featuring non-parametric mixture models, and the second featuring non-parametric factorial Hidden Markov Models with explicit duration distributions. The present implementation makes use of traditional and novel constraints during inference, showing marked improvement in disaggregation accuracy with very little effect on computational cost, relative to the motivating work. To constitute a complete unsupervised solution, labels are applied to the inferred components using a Res-Net-based deep learning architecture. Although this preliminary approach to labelling proves less than satisfactory, it is well-founded and several opportunities for improvement are discussed. Both methods, along with the labelling network, make use of block-filtered data: a steady-state representation that removes transient behaviour and signal noise. A novel filter to achieve this steady-state representation that is both fast and reliable is developed and discussed at length. Finally, an approach to monitor the aggregate for novel events during deployment is developed under the framework of Bayesian surprise. The same non-parametric modelling can be leveraged to examine how the predictive and transitional distributions change given new windows of observations. This framework is also shown to have potential elsewhere, such as in regularizing models against over-fitting, which is an important problem in existing supervised NILM

From 'tree' based Bayesian networks to mutual information classifiers : deriving a singly connected network classifier using an information theory based technique

For reasoning under uncertainty the Bayesian network has become the representation of choice. However, except where models are considered 'simple' the task of construction and inference are provably NP-hard. For modelling larger 'real' world problems this computational complexity has been addressed by methods that approximate the model. The Naive Bayes classifier, which has strong assumptions of independence among features, is a common approach, whilst the class of trees is another less extreme example. In this thesis we propose the use of an information theory based technique as a mechanism for inference in Singly Connected Networks. We call this a Mutual Information Measure classifier, as it corresponds to the restricted class of trees built from mutual information. We show that the new approach provides for both an efficient and localised method of classification, with performance accuracies comparable with the less restricted general Bayesian networks. To improve the performance of the classifier, we additionally investigate the possibility of expanding the class Markov blanket by use of a Wrapper approach and further show that the performance can be improved by focusing on the class Markov blanket and that the improvement is not at the expense of increased complexity. Finally, the two methods are applied to the task of diagnosing the 'real' world medical domain, Acute Abdominal Pain. Known to be both a different and challenging domain to classify, the objective was to investigate the optiniality claims, in respect of the Naive Bayes classifier, that some researchers have argued, for classifying in this domain. Despite some loss of representation capabilities we show that the Mutual Information Measure classifier can be effectively applied to the domain and also provides a recognisable qualitative structure without violating 'real' world assertions. In respect of its 'selective' variant we further show that the improvement achieves a comparable predictive accuracy to the Naive Bayes classifier and that the Naive Bayes classifier's 'overall' performance is largely due the contribution of the majority group Non-Specific Abdominal Pain, a group of exclusion