Companion website of the paper "Cooperative control of environmental extremes by artificial intelligent agents"

Humans have been able to tackle biosphere complexities by acting as ecosystem engineers, profoundly changing the flows of matter, energy and information. This includes major innovations that allowed to reduce and control the impact of extreme events. Modelling the evolution of such adaptive dynamics can be challenging given the potentially large number of individual and environmental variables involved. This paper shows how to address this problem by using fire as the source of external, bursting and wide fluctuations. We implement a simulated environment where fire propagates on a spatial landscape and a group of artificial agents learn how to harvest and exploit trees while avoiding the damaging effects of fire spreading. The agents need to solve a conflict to reach a group-level optimal state: while tree harvesting reduces the propagation of fires, it also reduces the availability of resources provided by trees. It is shown that the system displays two major evolutionary innovations that end up in an ecological engineering strategy that favours high biomass along with the suppression of large fires. The implications for potential A.I. management of complex ecosystems are discussed.</p

Behaviors obtained with quasimetric and dynamic programming methods with different discount factors.

<p>Starting from the initial stable state, both methods lead to the objective but with different trajectories.</p

Simple systems where the quasimetric and the dynamic programming methods are not equivalent.

<p>(A) Example of non deterministic systems where the quasi-distance differs from the value function.Arrows indicate possible actions with their associated transition probabilities and costs. Dotted arrow represents action and dashed arrows action , both allowed in state . (B) Example with a prison state . Starting from to the goal we can choose between two actions. Action in dotted leads to with a low cost but then with the risk to fall from to with a probability . is a risky state. Action in dashed leads to the goal with a probability but with a high cost .</p

Comparison of computation time for the under-actuated pendulum example.

<p>Comparison of computation time for the under-actuated pendulum example.</p

Results obtained for the non-holonomic system.

<p>(A) Accessibility volume of the Dubins car obtained with geometrical methods (adapted from <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0083411#pone.0083411-Laumond1" target="_blank">[42]</a>). (B) Discrete accessibility volume obtained for the described probabilistic case using the quasimetric method (axes aligned similarly). (C) Average trajectory for 500 simulations of 50 timesteps obtained starting at with goal at with a stochastic simulation and a drawn policy. Red curves are the optimal trajectories for a deterministic system.</p

Example of a simple <i>probabilistic</i> maze of size where dynamic programming and quasimetric methods are equivalent.

<p>(A) S is the starting state and G the goal. The red wall cannot be traversed (transition probability ) while gray ones can be considered as <i>probabilistic doors</i> with transition probability . 5 actions are considered: not moving, going east, west, south and north. (B) Quasi-distance obtained for the probabilistic maze example with a constant cost function () and corresponding policy. White arrows represent the optimal policy from position S to G. Black arrows represent the optimal policy to reach G from other positions.</p

Comparison of obtained value function, quasi-distance and policies for the inverted pendulum system.

<p>(A) Value function obtained with undiscounted value iteration. (B) quasi-distance. (C) Policy obtained with undiscounted value iteration. (D) Policy obtained with the quasi-distance (most probable policy).</p

Quasi-distances and Value function for example 2A.

<p>Quasi-distances and Value function for example 2A.</p