Learning Time Dependent Choice

We explore questions dealing with the learnability of models of choice over time. We present a large class of preference models defined by a structural criterion for which we are able to obtain an exponential improvement over previously known learning bounds for more general preference models. This in particular implies that the three most important discounted utility models of intertemporal choice - exponential, hyperbolic, and quasi-hyperbolic discounting - are learnable in the PAC setting with VC dimension that grows logarithmically in the number of time periods. We also examine these models in the framework of active learning. We find that the commonly studied stream-based setting is in general difficult to analyze for preference models, but we provide a redeeming situation in which the learner can indeed improve upon the guarantees provided by PAC learning. In contrast to the stream-based setting, we show that if the learner is given full power over the data he learns from - in the form of learning via membership queries - even very naive algorithms significantly outperform the guarantees provided by higher level active learning algorithms

A Simulation Approach to Dynamic Portfolio Choice with an Application to Learning About Return Predictability

We present a simulation-based method for solving discrete-time portfolio choice problems involving non-standard preferences, a large number of assets with arbitrary return distribution, and, most importantly, a large number of state variables with potentially path-dependent or non-stationary dynamics. The method is flexible enough to accommodate intermediate consumption, portfolio constraints, parameter and model uncertainty, and learning. We first establish the properties of the method for the portfolio choice between a stock index and cash when the stock returns are either iid or predictable by the dividend yield. We then explore the problem of an investor who takes into account the predictability of returns but is uncertain about the parameters of the data generating process. The investor chooses the portfolio anticipating that future data realizations will contain useful information to learn about the true parameter values.

The neuronal and molecular basis of quinine-dependent bitter taste signaling in Drosophila larvae.

The sensation of bitter substances can alert an animal that a specific type of food is harmful and should not be consumed. However, not all bitter compounds are equally toxic and some may even be beneficial in certain contexts. Thus, taste systems in general may have a broader range of functions than just in alerting the animal. In this study we investigate bitter sensing and processing in Drosophila larvae using quinine, a substance perceived by humans as bitter. We show that behavioral choice, feeding, survival, and associative olfactory learning are all directly affected by quinine. On the cellular level, we show that 12 gustatory sensory receptor neurons that express both GR66a and GR33a are required for quinine-dependent choice and feeding behavior. Interestingly, these neurons are not necessary for quinine-dependent survival or associative learning. On the molecular receptor gene level, the GR33a receptor, but not GR66a, is required for quinine-dependent choice behavior. A screen for gustatory sensory receptor neurons that trigger quinine-dependent choice behavior revealed that a single GR97a receptor gene expressing neuron located in the peripheral terminal sense organ is partially necessary and sufficient. For the first time, we show that the elementary chemosensory system of the Drosophila larva can serve as a simple model to understand the neuronal basis of taste information processing on the single cell level with respect to different behavioral outputs

Sequential Gaussian Processes for Online Learning of Nonstationary Functions

Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertainty quantification. However, the typical GP regression model suffers from several drawbacks: i) Conventional GP inference scales $O(N^{3})$ with respect to the number of observations; ii) updating a GP model sequentially is not trivial; and iii) covariance kernels often enforce stationarity constraints on the function, while GPs with non-stationary covariance kernels are often intractable to use in practice. To overcome these issues, we propose an online sequential Monte Carlo algorithm to fit mixtures of GPs that capture non-stationary behavior while allowing for fast, distributed inference. By formulating hyperparameter optimization as a multi-armed bandit problem, we accelerate mixing for real time inference. Our approach empirically improves performance over state-of-the-art methods for online GP estimation in the context of prediction for simulated non-stationary data and hospital time series data

When students can choose easy, medium, or hard homework problems

We investigate student-chosen, multi-level homework in our Integrated Learning Environment for Mechanics [1] built using the LON-CAPA [2] open-source learning system. Multi-level refers to problems categorized as easy, medium, and hard. Problem levels were determined a priori based on the knowledge needed to solve them [3]. We analyze these problems using three measures: time-per-problem, LON-CAPA difficulty, and item difficulty measured by item response theory. Our analysis of student behavior in this environment suggests that time-per-problem is strongly dependent on problem category, unlike either score-based measures. We also found trends in student choice of problems, overall effort, and efficiency across the student population. Allowing students choice in problem solving seems to improve their motivation; 70% of students worked additional problems for which no credit was given.National Science Foundation (U.S.) (Grant PHY-0757931)National Science Foundation (U.S.) (Grant DUE-1044294

Uncertainty in action-value estimation affects both action choice and learning rate of the choice behaviors of rats

The estimation of reward outcomes for action candidates is essential for decision making. In this study, we examined whether and how the uncertainty in reward outcome estimation affects the action choice and learning rate. We designed a choice task in which rats selected either the left-poking or right-poking hole and received a reward of a food pellet stochastically. The reward probabilities of the left and right holes were chosen from six settings (high, 100% vs. 66%; mid, 66% vs. 33%; low, 33% vs. 0% for the left vs. right holes, and the opposites) in every 20–549 trials. We used Bayesian Q-learning models to estimate the time course of the probability distribution of action values and tested if they better explain the behaviors of rats than standard Q-learning models that estimate only the mean of action values. Model comparison by cross-validation revealed that a Bayesian Q-learning model with an asymmetric update for reward and non-reward outcomes fit the choice time course of the rats best. In the action-choice equation of the Bayesian Q-learning model, the estimated coefficient for the variance of action value was positive, meaning that rats were uncertainty seeking. Further analysis of the Bayesian Q-learning model suggested that the uncertainty facilitated the effective learning rate. These results suggest that the rats consider uncertainty in action-value estimation and that they have an uncertainty-seeking action policy and uncertainty-dependent modulation of the effective learning rate

Familiares y amigos [10th grade]

This unit addresses two enduring understandings: cultures evolve over time and who we become is dependent on where we live. Students will demonstrate mastery of knowledge and skills through the creation of an illustrated brochure for a summer study abroad program that compares a Spanish-speaking city or region of their choice to New Orleans. The unit addresses all five categories of National Standards in Foreign Language Education (Communication, Culture, Connections, Comparisons, and Communities), and features a variety of cooperative and communicative learning strategies

Learning spatial aversion is sensory-specific in the hematophagous insect Rhodnius prolixus

Even though innate behaviors are essential for assuring quick responses to expected stimuli, experience-dependent behavioral plasticity confers an advantage when unexpected conditions arise. As being rigidly responsive to too many stimuli can be biologically expensive, adapting preferences to time-dependent relevant environmental conditions provide a cheaper and wider behavioral reactivity. According to their specific life habits, animals prioritize different sensory modalities to maximize environment exploitation. Besides, when mediating learning processes, the salience of a stimulus usually plays a relevant role in determining the intensity of an association. Then, sensory prioritization might reflect an heterogeneity in the cognitive abilities of an individual. Here, we analyze in the kissing bug Rhodnius prolixus if stimuli from different sensory modalities generate different cognitive capacities under an operant aversive paradigm. In a 2-choice walking arena, by registering the spatial distribution of insects over an experimental arena, we evaluated firstly the innate responses of bugs confronted to mechanical (rough substrate), visual (green light), thermal (32°C heated plate), hygric (humidified substrate), gustatory (sodium chloride), and olfactory (isobutyric acid) stimuli. In further experimental series bugs were submitted to an aversive operant conditioning by pairing each stimulus with a negative reinforcement. Subsequent tests allowed us to analyze if the innate behaviors were modulated by such previous aversive experience. In our experimental setup mechanical and visual stimuli were neutral, the thermal cue was attractive, and the hygric, gustatory and olfactory ones were innately aversive. After the aversive conditioning, responses to the mechanical, the visual, the hygric and the gustatory stimuli were modulated while responses to the thermal and the olfactory stimuli remained rigid. We present evidences that the spatial learning capacities of R. prolixus are dependent on the sensory modality of the conditioned stimulus, regardless their innate valence (i.e., neutral, attractive, or aversive). These differences might be given by the biological relevance of the stimuli and/or by evolutionary aspects of the life traits of this hematophagous insect.Fil: Minoli, Sebastian. Universidad de Buenos Aires; ArgentinaFil: Cano, Agustina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Pontes, Gina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Magallanes, Amorina. Universidad de Buenos Aires; ArgentinaFil: Roldán, Nahuel. Universidad de Buenos Aires; ArgentinaFil: Barrozo, Romina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentin