Distributed control design for underwater vehicles

The vast majority of control applications are based on non-interacting decentralized control designs. Because of their single-loop structure, these controllers cannot suppress interactions of the system. It would be useful to tackle the undesirable effects of the interactions at the design stage. A novel model predictive control scheme based on Nash optimality is presented to achieve this goal. In this algorithm, the control problem is decomposed into that of several small-coupled mixed integer optimisation problems. The relevant computational convergence, closed-loop performance and the effect of communication failures on the closed-loop behaviour are analysed. Simulation results are presented to illustrate the effectiveness and practicality of the proposed control algorithm

Trade-Off Geometries and Frequency-Dependent Selection

Life-history evolution is determined by the interplay between natural selection and adaptive constraints. The classical approach to studying constrained life-history evolution - Richard Levin's geometric comparison of fitness sets and adaptive functions - is applicable when selection pressures are frequency-independent. Here we extend this widely used tool to frequency-dependent selection. Such selection pressures very with a population's phenotypic composition, and are increasingly recognized as ubiquitous. Under frequency dependence, two independent properties have to be distinguished: evolutionary stability (an evolutionary stable strategy cannot be invaded once established) and convergence stability (only a convergence stable strategy can be attained through small, selectively advantageous steps). Combination of both properties results in four classes of possible evolutionary outcomes. We introduce a geometric mode of analysis that enables us to predict, for any bivariate selection problem, (1) evolutionary outcomes induced by trade-offs of given shape, (ii) shapes of trade-offs required for given evolutionary outcomes, (iii) the set of all evolutionary outcomes trade-off can induce, (iv) effects of ecological parameters on evolutionary outcomes independent of trade-off shape

Free Energy, Value, and Attractors

It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter

Information-theoretic Reasoning in Distributed and Autonomous Systems

The increasing prevalence of distributed and autonomous systems is transforming decision making in industries as diverse as agriculture, environmental monitoring, and healthcare. Despite significant efforts, challenges remain in robustly planning under uncertainty. In this thesis, we present a number of information-theoretic decision rules for improving the analysis and control of complex adaptive systems. We begin with the problem of quantifying the data storage (memory) and transfer (communication) within information processing systems. We develop an information-theoretic framework to study nonlinear interactions within cooperative and adversarial scenarios, solely from observations of each agent's dynamics. This framework is applied to simulations of robotic soccer games, where the measures reveal insights into team performance, including correlations of the information dynamics to the scoreline. We then study the communication between processes with latent nonlinear dynamics that are observed only through a filter. By using methods from differential topology, we show that the information-theoretic measures commonly used to infer communication in observed systems can also be used in certain partially observed systems. For robotic environmental monitoring, the quality of data depends on the placement of sensors. These locations can be improved by either better estimating the quality of future viewpoints or by a team of robots operating concurrently. By robustly handling the uncertainty of sensor model measurements, we are able to present the first end-to-end robotic system for autonomously tracking small dynamic animals, with a performance comparable to human trackers. We then solve the issue of coordinating multi-robot systems through distributed optimisation techniques. These allow us to develop non-myopic robot trajectories for these tasks and, importantly, show that these algorithms provide guarantees for convergence rates to the optimal payoff sequence

Complex evolutionary systems in behavioral finance

Traditional finance is built on the rationality paradigm. This chapter discusses simple models from an alternative approach in which financial markets are viewed as complex evolutionary systems. Agents are boundedly rational and base their investment decisions upon market forecasting heuristics. Prices and beliefs about future prices co-evolve over time with mutual feedback. Strategy choice is driven by evolutionary selection, so that agents tend to adopt strategies that were successful in the past. Calibration of "simple complexity models" with heterogeneous expectations to real financial market data and laboratory experiments with human subjects are also discussed.

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain