Adaptation to criticality through organizational invariance in embodied agents

Many biological and cognitive systems do not operate deep within one or other regime of activity. Instead, they are poised at critical points located at phase transitions in their parameter space. The pervasiveness of criticality suggests that there may be general principles inducing this behaviour, yet there is no well-founded theory for understanding how criticality is generated at a wide span of levels and contexts. In order to explore how criticality might emerge from general adaptive mechanisms, we propose a simple learning rule that maintains an internal organizational structure from a specific family of systems at criticality. We implement the mechanism in artificial embodied agents controlled by a neural network maintaining a correlation structure randomly sampled from an Ising model at critical temperature. Agents are evaluated in two classical reinforcement learning scenarios: the Mountain Car and the Acrobot double pendulum. In both cases the neural controller appears to reach a point of criticality, which coincides with a transition point between two regimes of the agent's behaviour. These results suggest that adaptation to criticality could be used as a general adaptive mechanism in some circumstances, providing an alternative explanation for the pervasive presence of criticality in biological and cognitive systems.Comment: arXiv admin note: substantial text overlap with arXiv:1704.0525

Adaptation to criticality through organizational invariance in embodied agents

Many biological and cognitive systems do not operate deep within one or other regime of activity. Instead, they are poised at critical points located at phase transitions in their parameter space. The pervasiveness of criticality suggests that there may be general principles inducing this behaviour, yet there is no well-founded theory for understanding how criticality is generated at a wide span of levels and contexts. In order to explore how criticality might emerge from general adaptive mechanisms, we propose a simple learning rule that maintains an internal organizational structure from a specific family of systems at criticality. We implement the mechanism in artificial embodied agents controlled by a neural network maintaining a correlation structure randomly sampled from an Ising model at critical temperature. Agents are evaluated in two classical reinforcement learning scenarios: the Mountain Car and the Acrobot double pendulum. In both cases the neural controller appears to reach a point of criticality, which coincides with a transition point between two regimes of the agent''s behaviour. These results suggest that adaptation to criticality could be used as a general adaptive mechanism in some circumstances, providing an alternative explanation for the pervasive presence of criticality in biological and cognitive systems

Exploring Criticality as a Generic Adaptive Mechanism

The activity of many biological and cognitive systems is not poised deep within a specific regime of activity. Instead, they operate near points of critical behavior located at the boundary between different phases. Certain authors link some of the properties of criticality with the ability of living systems to generate autonomous or intrinsically generated behavior. However, these claims remain highly speculative. In this paper, we intend to explore the connection between criticality and autonomous behavior through conceptual models that show how embodied agents may adapt themselves toward critical points. We propose to exploit maximum entropy models and their formal descriptions of indicators of criticality to present a learning model that drives generic agents toward critical points. Specifically, we derive such a learning model in an embodied Boltzmann machine by implementing a gradient ascent rule that maximizes the heat capacity of the controller in order to make the network maximally sensitive to external perturbations. We test and corroborate the model by implementing an embodied agent in the Mountain Car benchmark test, which is controlled by a Boltzmann machine that adjusts its weights according to the model. We find that the neural controller reaches an apparent point of criticality, which coincides with a transition point of the behavior of the agent between two regimes of behavior, maximizing the synergistic information between its sensors and the combination of hidden and motor neurons. Finally, we discuss the potential of our learning model to answer questions about the connection between criticality and the capabilities of living systems to autonomously generate intrinsic constraints on their behavior. We suggest that these "critical agents" are able to acquire flexible behavioral patterns that are useful for the development of successful strategies in different contexts.Research was supported in part by the Spanish National Programme for Fostering Excellence in Scientific and Technical Research project PSI2014-62092-EXP and by the project TIN2016-80347-R funded by the Spanish Ministry of Economy and Competitiveness. MA was supported by the UPV/EHU postdoctoral training program ESPDOC17/17

Exploring Criticality as a Generic Adaptive Mechanism

The activity of many biological and cognitive systems is not poised deep within a specific regime of activity. Instead, they operate near points of critical behavior located at the boundary between different phases. Certain authors link some of the properties of criticality with the ability of living systems to generate autonomous or intrinsically generated behavior. However, these claims remain highly speculative. In this paper, we intend to explore the connection between criticality and autonomous behavior through conceptual models that show how embodied agents may adapt themselves toward critical points. We propose to exploit maximum entropy models and their formal descriptions of indicators of criticality to present a learning model that drives generic agents toward critical points. Specifically, we derive such a learning model in an embodied Boltzmann machine by implementing a gradient ascent rule that maximizes the heat capacity of the controller in order to make the network maximally sensitive to external perturbations. We test and corroborate the model by implementing an embodied agent in the Mountain Car benchmark test, which is controlled by a Boltzmann machine that adjusts its weights according to the model. We find that the neural controller reaches an apparent point of criticality, which coincides with a transition point of the behavior of the agent between two regimes of behavior, maximizing the synergistic information between its sensors and the combination of hidden and motor neurons. Finally, we discuss the potential of our learning model to answer questions about the connection between criticality and the capabilities of living systems to autonomously generate intrinsic constraints on their behavior. We suggest that these "critical agents" are able to acquire flexible behavioral patterns that are useful for the development of successful strategies in different contexts

When to be critical? Performance and evolvability in different regimes of neural Ising agents

It has long been hypothesized that operating close to the critical state is beneficial for natural, artificial and their evolutionary systems. We put this hypothesis to test in a system of evolving foraging agents controlled by neural networks that can adapt agents' dynamical regime throughout evolution. Surprisingly, we find that all populations that discover solutions, evolve to be subcritical. By a resilience analysis, we find that there are still benefits of starting the evolution in the critical regime. Namely, initially critical agents maintain their fitness level under environmental changes (for example, in the lifespan) and degrade gracefully when their genome is perturbed. At the same time, initially subcritical agents, even when evolved to the same fitness, are often inadequate to withstand the changes in the lifespan and degrade catastrophically with genetic perturbations. Furthermore, we find the optimal distance to criticality depends on the task complexity. To test it we introduce a hard and simple task: for the hard task, agents evolve closer to criticality whereas more subcritical solutions are found for the simple task. We verify that our results are independent of the selected evolutionary mechanisms by testing them on two principally different approaches: a genetic algorithm and an evolutionary strategy. In summary, our study suggests that although optimal behaviour in the simple task is obtained in a subcritical regime, initializing near criticality is important to be efficient at finding optimal solutions for new tasks of unknown complexity.Comment: arXiv admin note: substantial text overlap with arXiv:2103.1218

Self-organized criticality, plasticity and sensorimotor coupling. Explorations with a neurorobotic model in a behavioural preference task

During the last two decades, analysis of 1/ƒ noise in cognitive science has led to a consider- able progress in the way we understand the organization of our mental life. However, there is still a lack of specific models providing explanations of how 1/ƒ noise is generated in cou- pled brain-body-environment systems, since existing models and experiments typically tar- get either externally observable behaviour or isolated neuronal systems but do not address the interplay between neuronal mechanisms and sensorimotor dynamics. We present a conceptual model of a minimal neurorobotic agent solving a behavioural task that makes it possible to relate mechanistic (neurodynamic) and behavioural levels of description. The model consists of a simulated robot controlled by a network of Kuramoto oscillators with ho- meostatic plasticity and the ability to develop behavioural preferences mediated by sensori- motor patterns. With only three oscillators, this simple model displays self-organized criticality in the form of robust 1/ƒ noise and a wide multifractal spectrum. We show that the emergence of self-organized criticality and 1/ƒ noise in our model is the result of three simul- taneous conditions: a) non-linear interaction dynamics capable of generating stable collec- tive patterns, b) internal plastic mechanisms modulating the sensorimotor flows, and c) strong sensorimotor coupling with the environment that induces transient metastable neuro- dynamic regimes. We carry out a number of experiments to show that both synaptic plastici- ty and strong sensorimotor coupling play a necessary role, as constituents of self-organized criticality, in the generation of 1/ƒ noise. The experiments also shown to be useful to test the robustness of 1/ƒ scaling comparing the results of different techniques. We finally discuss the role of conceptual models as mediators between nomothetic and mechanistic models and how they can inform future experimental research where self-organized critically in- cludes sensorimotor coupling among the essential interaction-dominant process giving rise to 1/ƒ noise

Interaction dynamics and autonomy in cognitive systems

The concept of autonomy is of crucial importance for understanding life and cognition. Whereas cellular and organismic autonomy is based in the self-production of the material infrastructure sustaining the existence of living beings as such, we are interested in how biological autonomy can be expanded into forms of autonomous agency, where autonomy as a form of organization is extended into the behaviour of an agent in interaction with its environment (and not its material self-production). In this thesis, we focus on the development of operational models of sensorimotor agency, exploring the construction of a domain of interactions creating a dynamical interface between agent and environment. We present two main contributions to the study of autonomous agency: First, we contribute to the development of a modelling route for testing, comparing and validating hypotheses about neurocognitive autonomy. Through the design and analysis of specific neurodynamical models embedded in robotic agents, we explore how an agent is constituted in a sensorimotor space as an autonomous entity able to adaptively sustain its own organization. Using two simulation models and different dynamical analysis and measurement of complex patterns in their behaviour, we are able to tackle some theoretical obstacles preventing the understanding of sensorimotor autonomy, and to generate new predictions about the nature of autonomous agency in the neurocognitive domain. Second, we explore the extension of sensorimotor forms of autonomy into the social realm. We analyse two cases from an experimental perspective: the constitution of a collective subject in a sensorimotor social interactive task, and the emergence of an autonomous social identity in a large-scale technologically-mediated social system. Through the analysis of coordination mechanisms and emergent complex patterns, we are able to gather experimental evidence indicating that in some cases social autonomy might emerge based on mechanisms of coordinated sensorimotor activity and interaction, constituting forms of collective autonomous agency

Exploring Criticality as a Generic Adaptive Mechanism

The activity of many biological and cognitive systems is not poised deep within a specific regime of activity. Instead, they operate near points of critical behavior located at the boundary between different phases. Certain authors link some of the properties of criticality with the ability of living systems to generate autonomous or intrinsically generated behavior. However, these claims remain highly speculative. In this paper, we intend to explore the connection between criticality and autonomous behavior through conceptual models that show how embodied agents may adapt themselves toward critical points. We propose to exploit maximum entropy models and their formal descriptions of indicators of criticality to present a learning model that drives generic agents toward critical points. Specifically, we derive such a learning model in an embodied Boltzmann machine by implementing a gradient ascent rule that maximizes the heat capacity of the controller in order to make the network maximally sensitive to external perturbations. We test and corroborate the model by implementing an embodied agent in the Mountain Car benchmark test, which is controlled by a Boltzmann machine that adjusts its weights according to the model. We find that the neural controller reaches an apparent point of criticality, which coincides with a transition point of the behavior of the agent between two regimes of behavior, maximizing the synergistic information between its sensors and the combination of hidden and motor neurons. Finally, we discuss the potential of our learning model to answer questions about the connection between criticality and the capabilities of living systems to autonomously generate intrinsic constraints on their behavior. We suggest that these “critical agents” are able to acquire flexible behavioral patterns that are useful for the development of successful strategies in different contexts

Divergent Criticality – A Mechanism of Neural Function for Perception and Learning

The natural world presents opportunities to all organisms as they compete for the biological-value afforded to them through their ecological engagement. This presents two fundamental requirements for perceiving such opportunities: to be able to recognise value and learning how to access new value. Though many theoretical accounts of how we might achieve such selectionist ends have been explored – how ‘perception’ and ‘learning’ resonate with life’s challenges and opportunities, to date, no explanation has yet been able to naturalise such perception adequately in the Universal laws that govern our existence – not only for explaining the human experience of the world, but in exploring the true nature of our perception. This thesis explores our perceptions of engaging with the world and seeks to explain how the demands of our experiences resonate with the efficient functioning of our brain. It proposes, that in a world of challenge and opportunity, rather than the efficient functioning of our neural resources, it is, instead, the optimising of ‘learning’ that is selected for, as an evolutionary priority. Building on existing literature in the fields of Phenomenology, Free Energy and Neuroscience, this thesis considers perception and learning as synonymous with the cognitive constructs of an ‘attention’ tuned for learning optimisation, and explores the processes of learning in neural function. It addresses the philosophical issues of how an individual’s perception of subjective experiences, might provide some empirical objectivity in proposing a ‘Tolerance’ hypothesis. This is a relative definition able to coordinate a ‘perception of experience’ in terms of an learning-function, grounded in free-energy theory (the laws of physics) and the ecological dynamics of a spontaneous or ‘self- organising’ mechanism – Divergent Criticality. The methodology incorporated three studies: Pilot, Developmental and Exploratory. Over the three studies, Divergent Criticality was tested by developing a functional Affordance measure to address the Research Question – are perceptions as affective-cognitions made aware as reflecting the agential mediation of a self-regulating, optimal learning mechanism? Perception questionnaires of Situational Interest and Self-concept were used in Study One and Study Two to investigate their suitability in addressing the Research Question. Here, Factor Analysis and Structural Equation Modelling assessed the validity and reliability of these measures, developing robust questionnaires and a research design for testing Divergent Criticality. In Study Three, the Divergent Criticality hypothesis was found to be significant, supporting that a Divergent Criticality mechanism is in operation: When individuals are engaging with dynamic ecological challenges, perception is affective in accordance with Tolerance Optimisation, demonstrating that a Divergent Criticality mechanism is driving individuals to the limits of their Effectivity – an optimal learning state which is fundamental to life and naturalised in Universal laws

The Ising model as a discovery tool in cognitive sciences

In this contribution we will explore how the Ising model can connect different physical phenomena with the cognitive sciences. For decades, physicists have pursued the use ideas from statistical mechanics to capture the collective phenomena of life. Biological systems have a subtle structure that is not described neither by ordered crystals nor disordered gases. Moreover, these states are far-from-equilibrium, being maintained by a constant flow of energy and matter through the system. There are special states for a functional living system and, at the same time, their activity cannot rely on fine-tuning the parameters of the system. Of the many ideas originated in statistical physics that have been suggested to characterize these states, perhaps the most suggestive and speculative is the idea of self-organized criticality. The theory of self-organized criticality originated in models of inanimate objects (sand mountains, earthquakes, etc.) [1], [2], but then the theory was to include biological systems through the analysis of simple toy models [3]. A simple model the evolution of interacting species can self-organize into a critical state where the quiescent period is interrupted by avalanches of all sizes [4], describing a behavior similar to the idea of punctuated equilibrium in evolution [5]. Similarly, the brain has been suggested to be in a self-organized critical state at the boundary between being nearly dead and becoming completely epileptic [6]. References [1] P. Bak, C. Tang, and K. Wiesenfeld, ‘Self-organized criticality’, Physical review A, vol. 38, no. 1, p. 364, 1988. [2] P. Bak, C. Tang, and K. Wiesenfeld, ‘Self-organized criticality: An explanation of the 1/f noise’, Phys. Rev. Lett., vol. 59, no. 4, pp. 381–384, Jul. 1987, doi: 10.1103/PhysRevLett.59.381. [3] P. Bak, How nature works: the science of self-organized criticality. New York: Copernicus, 1996. [4] P. Bak and K. Sneppen, ‘Punctuated equilibrium and criticality in a simple model of evolution’, Phys. Rev. Lett., vol. 71, no. 24, pp. 4083–4086, Dec. 1993, doi: 10.1103/PhysRevLett.71.4083. [5] S. J. Gould and N. Eldredge, ‘Punctuated Equilibria: The Tempo and Mode of Evolution Reconsidered’, Paleobiology, vol. 3, no. 2, pp. 115–151, 1977. [6] M. Usher, M. Stemmler, and Z. Olami, ‘Dynamic Pattern Formation Leads to 1/f Noise in Neural Populations’, Phys. Rev. Lett., vol. 74, no. 2, pp. 326–329, Jan. 1995, doi: 10.1103/PhysRevLett.74.326