Confidence-Based Feature Acquisition

Confidence-based Feature Acquisition (CFA) is a novel, supervised learning method for acquiring missing feature values when there is missing data at both training (learning) and test (deployment) time. To train a machine learning classifier, data is encoded with a series of input features describing each item. In some applications, the training data may have missing values for some of the features, which can be acquired at a given cost. A relevant JPL example is that of the Mars rover exploration in which the features are obtained from a variety of different instruments, with different power consumption and integration time costs. The challenge is to decide which features will lead to increased classification performance and are therefore worth acquiring (paying the cost). To solve this problem, CFA, which is made up of two algorithms (CFA-train and CFA-predict), has been designed to greedily minimize total acquisition cost (during training and testing) while aiming for a specific accuracy level (specified as a confidence threshold). With this method, it is assumed that there is a nonempty subset of features that are free; that is, every instance in the data set includes these features initially for zero cost. It is also assumed that the feature acquisition (FA) cost associated with each feature is known in advance, and that the FA cost for a given feature is the same for all instances. Finally, CFA requires that the base-level classifiers produce not only a classification, but also a confidence (or posterior probability)

Exploring Apprenticeship Learning for Player Modelling in Interactive Narratives

In this paper we present an early Apprenticeship Learning approach to mimic the behaviour of different players in a short adaption of the interactive fiction Anchorhead. Our motivation is the need to understand and simulate player behaviour to create systems to aid the design and personalisation of Interactive Narratives (INs). INs are partially observable for the players and their goals are dynamic as a result. We used Receding Horizon IRL (RHIRL) to learn players' goals in the form of reward functions, and derive policies to imitate their behaviour. Our preliminary results suggest that RHIRL is able to learn action sequences to complete a game, and provided insights towards generating behaviour more similar to specific players.Comment: Extended Abstracts of the 2019 Annual Symposium on Computer-Human Interaction in Play (CHI Play

Schistosoma mansoni-specific immune responses and allergy in Uganda.

Low allergy-related disease (ARD) prevalence in low-income countries may be partly attributed to helminth infections. In the Schistosoma mansoni (Sm)-endemic Lake Victoria islands (Uganda), we recently observed positive helminth-allergy associations, despite low ARD prevalence. To understand how Sm-induced cytokine and antibody profiles might influence allergic response profiles in this population, we assessed Schistosoma worm (SWA)- and egg antigen (SEA)-specific Th1 (IFN-γ), Th2 (IL-5, IL-13) and regulatory (IL-10) cytokine profiles (n = 407), and total (n = 471), SWA-, SEA- and allergen (house dust mite [HDM] and cockroach)-specific (as)IgE and IgG4 profiles (n = 2117) by ELISA. Wheeze was inversely associated with SWA-specific IFN-γ (P < .001) and IL-10 (P = .058), and SEA-specific IL-5 (P = .004). Conversely, having a detectable asIgE response was positively associated with SWA-specific IL-5 (P = .006) and IL-10 (P < .001). Total, SWA-, SEA- and allergen-specific IgE and IgG4 responses were higher among Sm Kato-Katz positive (SmKK+) and skin prick test (SPT)+ individuals compared to SmKK- and SPT- individuals. However, total and asIgG4/IgE ratios were lower among SPT+ and wheezing individuals. We conclude that, in this population, helminth-induced antibody and cytokine responses may underlie individual positive helminth-atopy associations, while the overall IgG4-IgE balance may contribute to the low overall prevalence of clinical allergies in such settings

Skewed Exposure to Environmental Antigens Complements Hygiene Hypothesis in Explaining the Rise of Allergy

The Hygiene Hypothesis has been recognized as an important cornerstone to explain the sudden increase in the prevalence of asthma and allergic diseases in modernized culture. The recent epidemic of allergic diseases is in contrast with the gradual implementation of Homo sapiens sapiens to the present-day forms of civilization. This civilization forms a gradual process with cumulative effects on the human immune system, which co-developed with parasitic and commensal Helminths. The clinical manifestation of this epidemic, however, became only visible in the second half of the twentieth century. In order to explain these clinical effects in terms of the underlying IgE-mediated reactions to innocuous environmental antigens, the low biodiversity of antigens in the domestic environment plays a pivotal role. The skewing of antigen exposure as a cumulative effect of reducing biodiversity in the immediate human environment as well as in changing food habits, provides a sufficient and parsimonious explanation for the rise in allergic diseases in a highly developed and helminth-free modernized culture. Socio-economic tendencies that incline towards a further reduction of environmental biodiversity may provide serious concern for future health. This article explains that the “Hygiene Hypothesis”, the “Old Friends Hypothesis”, and the “Skewed Antigen Exposure Hypothesis” are required to more fully explain the rise of allergy in modern societies

Multi-Source Option-Based Policy Transfer

Reinforcement learning algorithms are very effective at learning policies (mappings from states to actions) for specific well defined tasks, thereby allowing an agent to learn how to behave without extensive deliberation. However, if an agent must complete a novel variant of a task that is similar to, but not exactly the same as, a previous version for which it has already learned a policy, learning must begin anew and there is no benefit to having previously learned anything. To address this challenge, I introduce novel approaches for policy transfer. Policy transfer allows the agent to follow the policy of a previously solved, but different, task (called a source task) while it is learning a new task (called a target task). Specifically, I introduce option-based policy transfer (OPT). OPT enables policy transfer by encapsulating the policy for a source task in an option Sutton, Precup, & Singh 1999), which allows the agent to treat the policy of a source task as if it were a primitive action. A significant advantage of this approach is that if there are multiple source tasks, an option can be created for each of them, thereby enabling the agent to transfer knowledge from multiple sources and to combine their knowledge in useful ways. Moreover, this approach allows the agent to learn in which states of the world each source task is most applicable. OPT's approach to constructing and learning with options that represent source tasks allows OPT to greatly outperform existing policy transfer approaches. Additionally, OPT can utilize source tasks that other forms of transfer learning for reinforcement learning cannot. Challenges for policy transfer include identifying sets of source tasks that would be useful for a target task and providing mappings between the state and action spaces of source and target tasks. That is, it may not be useful to transfer from all previously solved source tasks. If a source task has a different state or action space than the target task, then a mapping between these spaces must be provided. To address these challenges, I introduce object-oriented OPT (OO-OPT), which leverages object-oriented MDP (OO-MDP) (Diuk, Cohen, & Littman 2008) state representations to automatically detect related tasks and redundant source tasks, and to provide multiple useful state and action space mappings between tasks. I also introduce methods to adapt value function approximation techniques (which are useful when the state space of a task is very large or continuous) to the unique state representation of OO-MDPs