Stochastic models of evidence accumulation in changing environments

Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable model of evidence accumulation when the correct option changes in time. Our analysis shows that ideal observers discount prior evidence at a rate determined by the volatility of the environment, and the dynamics of evidence accumulation is governed by the information gained over an average environmental epoch. A plausible neural implementation of an optimal observer in a changing environment shows that, in contrast to previous models, neural populations representing alternate choices are coupled through excitation. Our work builds a bridge between statistical decision making in volatile environments and stochastic nonlinear dynamics.Comment: 26 pages, 7 figure

An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context

One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluctuated bandwidth in congestion control. Existing works on adaptation to unknown environment contexts either assume the contexts are the same for the whole episode or assume the context variables are Markovian. However, in many real-world applications, the environment context usually stays stable for a stochastic period and then changes in an abrupt and unpredictable manner within an episode, resulting in a segment structure, which existing works fail to address. To leverage the segment structure of piecewise stable context in real-world applications, in this paper, we propose a \textit{\textbf{Se}gmented \textbf{C}ontext \textbf{B}elief \textbf{A}ugmented \textbf{D}eep~(SeCBAD)} RL method. Our method can jointly infer the belief distribution over latent context with the posterior over segment length and perform more accurate belief context inference with observed data within the current context segment. The inferred belief context can be leveraged to augment the state, leading to a policy that can adapt to abrupt variations in context. We demonstrate empirically that SeCBAD can infer context segment length accurately and outperform existing methods on a toy grid world environment and Mujuco tasks with piecewise-stable context.Comment: NeurIPS 202

Data-driven fault diagnosis of awind farm benchmark model

The fault diagnosis of wind farms has been proven to be a challenging task, and motivates the research activities carried out through this work. Therefore, this paper deals with the fault diagnosis of a wind park benchmark model, and it considers viable solutions to the problem of earlier fault detection and isolation. The design of the fault indicator involves data-driven approaches, as they can represent effective tools for coping with poor analytical knowledge of the system dynamics, noise, uncertainty, and disturbances. In particular, the proposed data-driven solutions rely on fuzzy models and neural networks that are used to describe the strongly nonlinear relationships between measurement and faults. The chosen architectures rely on nonlinear autoregressive with exogenous input models, as they can represent the dynamic evolution of the system over time. The developed fault diagnosis schemes are tested by means of a high-fidelity benchmark model that simulates the normal and the faulty behaviour of a wind farm installation. The achieved performances are also compared with those of a model-based approach relying on nonlinear differential geometry tools. Finally, a Monte-Carlo analysis validates the robustness and reliability of the proposed solutions against typical parameter uncertainties and disturbances.The fault diagnosis of wind farms has been proven to be a challenging task, and motivates the research activities carried out through this work. Therefore, this paper deals with the fault diagnosis of a wind park benchmark model, and it considers viable solutions to the problem of earlier fault detection and isolation. The design of the fault indicator involves data-driven approaches, as they can represent effective tools for coping with poor analytical knowledge of the system dynamics, noise, uncertainty, and disturbances. In particular, the proposed data-driven solutions rely on fuzzy models and neural networks that are used to describe the strongly nonlinear relationships between measurement and faults. The chosen architectures rely on nonlinear autoregressive with exogenous input models, as they can represent the dynamic evolution of the system over time. The developed fault diagnosis schemes are tested by means of a high-fidelity benchmark model that simulates the normal and the faulty behaviour of a wind farm installation. The achieved performances are also compared with those of a model-based approach relying on nonlinear differential geometry tools. Finally, a Monte-Carlo analysis validates the robustness and reliability of the proposed solutions against typical parameter uncertainties and disturbances

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Variable selection and sensitivity analysis using dynamic trees, with an application to computer code performance tuning

We investigate an application in the automatic tuning of computer codes, an area of research that has come to prominence alongside the recent rise of distributed scientific processing and heterogeneity in high-performance computing environments. Here, the response function is nonlinear and noisy and may not be smooth or stationary. Clearly needed are variable selection, decomposition of influence, and analysis of main and secondary effects for both real-valued and binary inputs and outputs. Our contribution is a novel set of tools for variable selection and sensitivity analysis based on the recently proposed dynamic tree model. We argue that this approach is uniquely well suited to the demands of our motivating example. In illustrations on benchmark data sets, we show that the new techniques are faster and offer richer feature sets than do similar approaches in the static tree and computer experiment literature. We apply the methods in code-tuning optimization, examination of a cold-cache effect, and detection of transformation errors.Comment: Published in at http://dx.doi.org/10.1214/12-AOAS590 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Statistical Computations Underlying the Dynamics of Memory Updating

Psychophysical and neurophysiological studies have suggested that memory is not simply a carbon copy of our experience: Memories are modified or new memories are formed depending on the dynamic structure of our experience, and specifically, on how gradually or abruptly the world changes. We present a statistical theory of memory formation in a dynamic environment, based on a nonparametric generalization of the switching Kalman filter. We show that this theory can qualitatively account for several psychophysical and neural phenomena, and present results of a new visual memory experiment aimed at testing the theory directly. Our experimental findings suggest that humans can use temporal discontinuities in the structure of the environment to determine when to form new memory traces. The statistical perspective we offer provides a coherent account of the conditions under which new experience is integrated into an old memory versus forming a new memory, and shows that memory formation depends on inferences about the underlying structure of our experience.Templeton FoundationAlfred P. Sloan Foundation (Fellowship)National Science Foundation (U.S.) (NSF Graduate Research Fellowship)National Institute of Mental Health (U.S.) (NIH Award Number R01MH098861

Data-driven techniques for the fault diagnosis of a wind turbine benchmark

This paper deals with the fault diagnosis of wind turbines and investigates viable solutions to the problem of earlier fault detection and isolation. The design of the fault indicator, i.e., the fault estimate, involves data-driven approaches, as they can represent effective tools for coping with poor analytical knowledge of the system dynamics, together with noise and disturbances. In particular, the proposed data-driven solutions rely on fuzzy systems and neural networks that are used to describe the strongly nonlinear relationships between measurement and faults. The chosen architectures rely on nonlinear autoregressive models with exogenous input, as they can represent the dynamic evolution of the system along time. The developed fault diagnosis schemes are tested by means of a high-fidelity benchmark model that simulates the normal and the faulty behaviour of a wind turbine. The achieved performances are also compared with those of other model-based strategies from the related literature. Finally, a Monte-Carlo analysis validates the robustness and the reliability of the proposed solutions against typical parameter uncertainties and disturbances.This paper deals with the fault diagnosis of wind turbines and investigates viable solutions to the problem of earlier fault detection and isolation. The design of the fault indicator, i.e., the fault estimate, involves data-driven approaches, as they can represent effective tools for coping with poor analytical knowledge of the system dynamics, together with noise and disturbances. In particular, the proposed data-driven solutions rely on fuzzy systems and neural networks that are used to describe the strongly nonlinear relationships between measurement and faults. The chosen architectures rely on nonlinear autoregressive models with exogenous input, as they can represent the dynamic evolution of the system along time. The developed fault diagnosis schemes are tested by means of a high-fidelity benchmark model that simulates the normal and the faulty behaviour of a wind turbine. The achieved performances are also compared with those of other model-based strategies from the related literature. Finally, a Monte-Carlo analysis validates the robustness and the reliability of the proposed solutions against typical parameter uncertainties and disturbances