Reinforcement Learning Agents acquire Flocking and Symbiotic Behaviour in Simulated Ecosystems

In nature, group behaviours such as flocking as well as cross-species symbiotic partnerships are observed in vastly different forms and circumstances. We hypothesize that such strategies can arise in response to generic predator-prey pressures in a spatial environment with range-limited sensation and action. We evaluate whether these forms of coordination can emerge by independent multi-agent reinforcement learning in simple multiple-species ecosystems. In contrast to prior work, we avoid hand-crafted shaping rewards, specific actions, or dynamics that would directly encourage coordination across agents. Instead we test whether coordination emerges as a consequence of adaptation without encouraging these specific forms of coordination, which only has indirect benefit. Our simulated ecosystems consist of a generic food chain involving three trophic levels: apex predator, mid-level predator, and prey. We conduct experiments on two different platforms, a 3D physics engine with tens of agents as well as in a 2D grid world with up to thousands. The results clearly confirm our hypothesis and show substantial coordination both within and across species. To obtain these results, we leverage and adapt recent advances in deep reinforcement learning within an ecosystem training protocol featuring homogeneous groups of independent agents from different species (sets of policies), acting in many different random combinations in parallel habitats. The policies utilize neural network architectures that are invariant to agent individuality but not type (species) and that generalize across varying numbers of observed other agents. While the emergence of complexity in artificial ecosystems have long been studied in the artificial life community, the focus has been more on individual complexity and genetic algorithms or explicit modelling, and less on group complexity and reinforcement learning emphasized in this article. Unlike what the name and intuition suggests, reinforcement learning adapts over evolutionary history rather than a life-time and is here addressing the sequential optimization of fitness that is usually approached by genetic algorithms in the artificial life community. We utilize a shift from procedures to objectives, allowing us to bring new powerful machinery to bare, and we see emergence of complex behaviour from a sequence of simple optimization problems

Ultra-Rapid Warming Yields High Survival of Mouse Oocytes Cooled to −196°C in Dilutions of a Standard Vitrification Solution

Intracellular ice is generally lethal. One way to avoid it is to vitrify cells; that is, to convert cell water to a glass rather than to ice. The belief has been that this requires both the cooling rate and the concentration of glass-inducing solutes be very high. But high solute concentrations can themselves be damaging. However, the findings we report here on the vitrification of mouse oocytes are not in accord with the first belief that cooling needs to be extremely rapid. The important requirement is that the warming rate be extremely high. We subjected mouse oocytes in the vitrification solution EAFS 10/10 to vitrification procedures using a broad range of cooling and warming rates. Morphological survivals exceeded 80% when they were warmed at the highest rate (117,000°C/min) even when the prior cooling rate was as low as 880°C/min. Functional survival was >81% and 54% with the highest warming rate after cooling at 69,000 and 880°C/min, respectively. Our findings are also contrary to the second belief. We show that a high percentage of mouse oocytes survive vitrification in media that contain only half the usual concentration of solutes, provided they are warmed extremely rapidly; that is, >100,000°C/min. Again, the cooling rate is of less consequence

Rectification of the Water Permeability in COS-7 Cells at 22, 10 and 0°C

The osmotic and permeability parameters of a cell membrane are essential physico-chemical properties of a cell and particularly important with respect to cell volume changes and the regulation thereof. Here, we report the hydraulic conductivity, Lp, the non-osmotic volume, Vb, and the Arrhenius activation energy, Ea, of mammalian COS-7 cells. The ratio of Vb to the isotonic cell volume, Vc iso, was 0.29. Ea, the activation energy required for the permeation of water through the cell membrane, was 10,700, and 12,000 cal/mol under hyper- and hypotonic conditions, respectively. Average values for Lp were calculated from swell/shrink curves by using an integrated equation for Lp. The curves represented the volume changes of 358 individually measured cells, placed into solutions of nonpermeating solutes of 157 or 602 mOsm/kg (at 0, 10 or 22°C) and imaged over time. Lp estimates for all six combinations of osmolality and temperature were calculated, resulting in values of 0.11, 0.21, and 0.10 µm/min/atm for exosmotic flow and 0.79, 1.73 and 1.87 µm/min/atm for endosmotic flow (at 0, 10 and 22°C, respectively). The unexpected finding of several fold higher Lp values for endosmotic flow indicates highly asymmetric membrane permeability for water in COS-7. This phenomenon is known as rectification and has mainly been reported for plant cell, but only rarely for animal cells. Although the mechanism underlying the strong rectification found in COS-7 cells is yet unknown, it is a phenomenon of biological interest and has important practical consequences, for instance, in the development of optimal cryopreservation

The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex

Is visual cortex made up of general-purpose information processing machinery, or does it consist of a collection of specialized modules? If prior knowledge, acquired from learning a set of objects is only transferable to new objects that share properties with the old, then the recognition system’s optimal organization must be one containing specialized modules for different object classes. Our analysis starts from a premise we call the invariance hypothesis: that the computational goal of the ventral stream is to compute an invariant-to-transformations and discriminative signature for recognition. The key condition enabling approximate transfer of invariance without sacrificing discriminability turns out to be that the learned and novel objects transform similarly. This implies that the optimal recognition system must contain subsystems trained only with data from similarly-transforming objects and suggests a novel interpretation of domain-specific regions like the fusiform face area (FFA). Furthermore, we can define an index of transformation-compatibility, computable from videos, that can be combined with information about the statistics of natural vision to yield predictions for which object categories ought to have domain-specific regions in agreement with the available data. The result is a unifying account linking the large literature on view-based recognition with the wealth of experimental evidence concerning domain-specific regions.National Science Foundation (U.S.). Science and Technology Center (Award CCF-1231216)National Science Foundation (U.S.) (Grant NSF-0640097)National Science Foundation (U.S.) (Grant NSF-0827427)United States. Air Force Office of Scientific Research (Grant FA8650-05-C-7262)Eugene McDermott Foundatio

The influence of external factors on bacteriophages—review

The ability of bacteriophages to survive under unfavorable conditions is highly diversified. We summarize the influence of different external physical and chemical factors, such as temperature, acidity, and ions, on phage persistence. The relationships between a phage’s morphology and its survival abilities suggested by some authors are also discussed. A better understanding of the complex problem of phage sensitivity to external factors may be useful not only for those interested in pharmaceutical and agricultural applications of bacteriophages, but also for others working with phages