Empowerment for Continuous Agent-Environment Systems

This paper develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowemerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, e.g., it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this paper is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte-Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning

Local information transfer as a spatiotemporal filter for complex systems

We present a measure of local information transfer, derived from an existing averaged information-theoretical measure, namely transfer entropy. Local transfer entropy is used to produce profiles of the information transfer into each spatiotemporal point in a complex system. These spatiotemporal profiles are useful not only as an analytical tool, but also allow explicit investigation of different parameter settings and forms of the transfer entropy metric itself. As an example, local transfer entropy is applied to cellular automata, where it is demonstrated to be a novel method of filtering for coherent structure. More importantly, local transfer entropy provides the first quantitative evidence for the long-held conjecture that the emergent traveling coherent structures known as particles (both gliders and domain walls, which have analogues in many physical processes) are the dominant information transfer agents in cellular automata.Comment: 12 page

A framework for the local information dynamics of distributed computation in complex systems

The nature of distributed computation has often been described in terms of the component operations of universal computation: information storage, transfer and modification. We review the first complete framework that quantifies each of these individual information dynamics on a local scale within a system, and describes the manner in which they interact to create non-trivial computation where "the whole is greater than the sum of the parts". We describe the application of the framework to cellular automata, a simple yet powerful model of distributed computation. This is an important application, because the framework is the first to provide quantitative evidence for several important conjectures about distributed computation in cellular automata: that blinkers embody information storage, particles are information transfer agents, and particle collisions are information modification events. The framework is also shown to contrast the computations conducted by several well-known cellular automata, highlighting the importance of information coherence in complex computation. The results reviewed here provide important quantitative insights into the fundamental nature of distributed computation and the dynamics of complex systems, as well as impetus for the framework to be applied to the analysis and design of other systems.Comment: 44 pages, 8 figure

Evidence for surprise minimization over value maximization in choice behavior

Classical economic models are predicated on the idea that the ultimate aim of choice is to maximize utility or reward. In contrast, an alternative perspective highlights the fact that adaptive behavior requires agents' to model their environment and minimize surprise about the states they frequent. We propose that choice behavior can be more accurately accounted for by surprise minimization compared to reward or utility maximization alone. Minimizing surprise makes a prediction at variance with expected utility models; namely, that in addition to attaining valuable states, agents attempt to maximize the entropy over outcomes and thus 'keep their options open'. We tested this prediction using a simple binary choice paradigm and show that human decision-making is better explained by surprise minimization compared to utility maximization. Furthermore, we replicated this entropy-seeking behavior in a control task with no explicit utilities. These findings highlight a limitation of purely economic motivations in explaining choice behavior and instead emphasize the importance of belief-based motivations

Peripheral Administration of a Humanized Anti-PrP Antibody Blocks Alzheimer's Disease Aβ Synaptotoxicity.

Alzheimer's disease (AD) is associated with pathological assembly states of amyloid-β protein (Aβ). Aβ-related synaptotoxicity can be blocked by anti-prion protein (PrP) antibodies, potentially allowing therapeutic targeting of this aspect of AD neuropathogenesis. Here, we show that intravascular administration of a high-affinity humanized anti-PrP antibody to rats can prevent the plasticity-disrupting effects induced by exposure to soluble AD brain extract. These results provide an in vivo proof of principle for such a therapeutic strategy

Integrated information increases with fitness in the evolution of animats

One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular since organismal functional complexity is not well-defined. We present here several candidate measures that quantify information and integration, and study their dependence on fitness as an artificial agent ("animat") evolves over thousands of generations to solve a navigation task in a simple, simulated environment. We compare the ability of these measures to predict high fitness with more conventional information-theoretic processing measures. As the animat adapts by increasing its "fit" to the world, information integration and processing increase commensurately along the evolutionary line of descent. We suggest that the correlation of fitness with information integration and with processing measures implies that high fitness requires both information processing as well as integration, but that information integration may be a better measure when the task requires memory. A correlation of measures of information integration (but also information processing) and fitness strongly suggests that these measures reflect the functional complexity of the animat, and that such measures can be used to quantify functional complexity even in the absence of fitness data.Comment: 27 pages, 8 figures, one supplementary figure. Three supplementary video files available on request. Version commensurate with published text in PLoS Comput. Bio

Sensitive detection of Aβ protofibrils by proximity ligation - relevance for Alzheimer's disease

<p>Abstract</p> <p>Background</p> <p>Protein aggregation plays important roles in several neurodegenerative disorders. For instance, insoluble aggregates of phosphorylated tau and of Aβ peptides are cornerstones in the pathology of Alzheimer's disease. Soluble protein aggregates are therefore potential diagnostic and prognostic biomarkers for their cognate disorders. Detection of the aggregated species requires sensitive tools that efficiently discriminate them from monomers of the same proteins. Here we have established a proximity ligation assay (PLA) for specific and sensitive detection of Aβ protofibrils via simultaneous recognition of three identical determinants present in the aggregates. PLA is a versatile technology in which the requirement for multiple target recognitions is combined with the ability to translate signals from detected target molecules to amplifiable DNA strands, providing very high specificity and sensitivity.</p> <p>Results</p> <p>For specific detection of Aβ protofibrils we have used a monoclonal antibody, mAb158, selective for Aβ protofibrils in a modified PLA, where the same monoclonal antibody was used for the three classes of affinity reagents required in the assay. These reagents were used for detection of soluble Aβ aggregates in solid-phase reactions, allowing detection of just 0.1 pg/ml Aβ protofibrils, and with a dynamic range greater than six orders of magnitude. Compared to a sandwich ELISA setup of the same antibody the PLA increases the sensitivity of the Aβ protofibril detection by up to 25-fold. The assay was used to measure soluble Aβ aggregates in brain homogenates from mice transgenic for a human allele predisposing to Aβ aggregation.</p> <p>Conclusions</p> <p>The proximity ligation assay is a versatile analytical technology for proteins, which can provide highly sensitive and specific detection of Aβ aggregates - and by implication other protein aggregates of relevance in Alzheimer's disease and other neurodegenerative disorders.</p

Active Inference: A Process Theory

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence—or minimizing variational free energy—we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes’ optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton’s principle of least action

Rationally Designed Turn Promoting Mutation in the Amyloid-β Peptide Sequence Stabilizes Oligomers in Solution

Enhanced production of a 42-residue beta amyloid peptide (Aβ42) in affected parts of the brain has been suggested to be the main causative factor for the development of Alzheimer's Disease (AD). The severity of the disease depends not only on the amount of the peptide but also its conformational transition leading to the formation of oligomeric amyloid-derived diffusible ligands (ADDLs) in the brain of AD patients. Despite being significant to the understanding of AD mechanism, no atomic-resolution structures are available for these species due to the evanescent nature of ADDLs that hinders most structural biophysical investigations. Based on our molecular modeling and computational studies, we have designed Met35Nle and G37p mutations in the Aβ42 peptide (Aβ42Nle35p37) that appear to organize Aβ42 into stable oligomers. 2D NMR on the Aβ42Nle35p37 peptide revealed the occurrence of two β-turns in the V24-N27 and V36-V39 stretches that could be the possible cause for the oligomer stability. We did not observe corresponding NOEs for the V24-N27 turn in the Aβ21–43Nle35p37 fragment suggesting the need for the longer length amyloid peptide to form the stable oligomer promoting conformation. Because of the presence of two turns in the mutant peptide which were absent in solid state NMR structures for the fibrils, we propose, fibril formation might be hindered. The biophysical information obtained in this work could aid in the development of structural models for toxic oligomer formation that could facilitate the development of therapeutic approaches to AD