Belief dynamics extraction

Animal behavior is not driven simply by its current observa-tions, but is strongly influenced by internal states. Estimatingthe structure of these internal states is crucial for understand-ing the neural basis of behavior. In principle, internal statescan be estimated by inverting behavior models, as in inversemodel-based Reinforcement Learning. However, this requirescareful parameterization and risks model-mismatch to the ani-mal. Here we take a data-driven approach to infer latent statesdirectly from observations of behavior, using a partially ob-servable switching semi-Markov process. This process has twoelements critical for capturing animal behavior: it captures non-exponential distribution of times between observations, andtransitions between latent states depend on the animal’s actions,features that require more complex non-markovian models torepresent. To demonstrate the utility of our approach, we applyit to the observations of a simulated optimal agent performinga foraging task, and find that latent dynamics extracted by themodel has correspondences with the belief dynamics of theagent. Finally, we apply our model to identify latent states inthe behaviors of monkey performing a foraging task, and findclusters of latent states that identify periods of time consistentwith expectant waiting. This data-driven behavioral model willbe valuable for inferring latent cognitive states, and thereby formeasuring neural representations of those states

Extracting the time-dependent transmission rate from infection data via solution of an inverse ODE problem

The transmission rate of many acute infectious diseases varies significantly in time, but the underlying mechanisms are usually uncertain. They may include seasonal changes in the environment, contact rate, immune system response, etc. The transmission rate has been thought difficult to measure directly. We present a new algorithm to compute the time-dependent transmission rate directly from prevalence data, which makes no assumptions about the number of susceptible or vital rates. The algorithm follows our complete and explicit solution of a mathematical inverse problem for SIR-type transmission models. We prove that almost any infection profile can be perfectly fitted by an SIR model with variable transmission rate. This clearly shows a serious danger of overfitting such transmission models. We illustrate the algorithm with historic UK measles data and our observations support the common belief that measles transmission was predominantly driven by school contacts

Integrative Use of Information Extraction, Semantic Matchmaking and Adaptive Coupling Techniques in Support of Distributed Information Processing and Decision-Making

In order to press maximal cognitive benefit from their social, technological and informational environments, military coalitions need to understand how best to exploit available information assets as well as how best to organize their socially-distributed information processing activities. The International Technology Alliance (ITA) program is beginning to address the challenges associated with enhanced cognition in military coalition environments by integrating a variety of research and development efforts. In particular, research in one component of the ITA ('Project 4: Shared Understanding and Information Exploitation') is seeking to develop capabilities that enable military coalitions to better exploit and distribute networked information assets in the service of collective cognitive outcomes (e.g. improved decision-making). In this paper, we provide an overview of the various research activities in Project 4. We also show how these research activities complement one another in terms of supporting coalition-based collective cognition

Belief Tree Search for Active Object Recognition

Active Object Recognition (AOR) has been approached as an unsupervised learning problem, in which optimal trajectories for object inspection are not known and are to be discovered by reducing label uncertainty measures or training with reinforcement learning. Such approaches have no guarantees of the quality of their solution. In this paper, we treat AOR as a Partially Observable Markov Decision Process (POMDP) and find near-optimal policies on training data using Belief Tree Search (BTS) on the corresponding belief Markov Decision Process (MDP). AOR then reduces to the problem of knowledge transfer from near-optimal policies on training set to the test set. We train a Long Short Term Memory (LSTM) network to predict the best next action on the training set rollouts. We sho that the proposed AOR method generalizes well to novel views of familiar objects and also to novel objects. We compare this supervised scheme against guided policy search, and find that the LSTM network reaches higher recognition accuracy compared to the guided policy method. We further look into optimizing the observation function to increase the total collected reward of optimal policy. In AOR, the observation function is known only approximately. We propose a gradient-based method update to this approximate observation function to increase the total reward of any policy. We show that by optimizing the observation function and retraining the supervised LSTM network, the AOR performance on the test set improves significantly.Comment: IROS 201

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents. Our method is based on a graph-structured variational recurrent neural network (Graph-VRNN), which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states. We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.Comment: ICLR 2019 camera read

Bayesian model predictive control: Efficient model exploration and regret bounds using posterior sampling

Tight performance specifications in combination with operational constraints make model predictive control (MPC) the method of choice in various industries. As the performance of an MPC controller depends on a sufficiently accurate objective and prediction model of the process, a significant effort in the MPC design procedure is dedicated to modeling and identification. Driven by the increasing amount of available system data and advances in the field of machine learning, data-driven MPC techniques have been developed to facilitate the MPC controller design. While these methods are able to leverage available data, they typically do not provide principled mechanisms to automatically trade off exploitation of available data and exploration to improve and update the objective and prediction model. To this end, we present a learning-based MPC formulation using posterior sampling techniques, which provides finite-time regret bounds on the learning performance while being simple to implement using off-the-shelf MPC software and algorithms. The performance analysis of the method is based on posterior sampling theory and its practical efficiency is illustrated using a numerical example of a highly nonlinear dynamical car-trailer system

Trading Frequency and Volatility Clustering

Volatility clustering, with autocorrelations of the hyperbolic decay rate, is unquestionably one of the most important stylized facts of financial time series. This paper presents a market microstructure model, that is able to generate volatility clustering with hyperbolic autocorrelations through traders with multiple trading frequencies using Bayesian information updating in an incomplete market. The model illustrates that signal extraction, which is induced by multiple trading frequency, can increase the persistence of the volatility of returns. Furthermore, we show that the local temporal memory of the underlying time series of returns and their volatility varies greatly varies with the number of traders in the marketTrading frequency, Volatility clustering, Signal extraction, Hyperbolic decay

Resilience in Ecology and Belief

Replaced with revised version of paper 06/18/08. Former title: Non-Linearity in Belief and Environmental Risk DynamicsBelief dynamics, ecological hysteresis, water scarcity, groundwater dependent ecosystems, threshold effects, Environmental Economics and Policy, Resource /Energy Economics and Policy, Risk and Uncertainty,