All macroscopic quantum states are fragile and hard to prepare

We study the effect of local decoherence on arbitrary quantum states. Adapting techniques developed in quantum metrology, we show that the action of generic local noise processes -- though arbitrarily small -- always yields a state whose Quantum Fisher Information (QFI) with respect to local observables is linear in system size N, independent of the initial state. This implies that all macroscopic quantum states, which are characterized by a QFI that is quadratic in N, are fragile under decoherence, and cannot be maintained if the system is not perfectly isolated. We also provide analytical bounds on the effective system size, and show that the effective system size scales as the inverse of the noise parameter p for small p for all the noise channels considered, making it increasingly difficult to generate macroscopic or even mesoscopic quantum states. In turn, we also show that the preparation of a macroscopic quantum state, with respect to a conserved quantity, requires a device whose QFI is already at least as large as the one of the desired state. Given that the preparation device itself is classical and not a perfectly isolated macroscopic quantum state, the preparation device needs to be quadratically bigger than the macroscopic target state

Optimal foraging strategies can be learned

The foraging behavior of animals is a paradigm of target search in nature. Understanding which foraging strategies are optimal and how animals learn them are central challenges in modeling animal foraging. While the question of optimality has wide-ranging implications across fields such as economy, physics, and ecology, the question of learnability is a topic of ongoing debate in evolutionary biology. Recognizing the interconnected nature of these challenges, this work addresses them simultaneously by exploring optimal foraging strategies through a reinforcement learning framework. To this end, we model foragers as learning agents. We first prove theoretically that maximizing rewards in our reinforcement learning model is equivalent to optimizing foraging efficiency. We then show with numerical experiments that, in the paradigmatic model of non-destructive search, our agents learn foraging strategies which outperform the efficiency of some of the best known strategies such as L\'evy walks. These findings highlight the potential of reinforcement learning as a versatile framework not only for optimizing search strategies but also to model the learning process, thus shedding light on the role of learning in natural optimization processes.Comment: 12 pages (6 main text) and 9 figures. Codes and tutorials available at: https://gorkamunoz.github.io/rl_opts

Automated Gadget Discovery in Science

In recent years, reinforcement learning (RL) has become increasingly successful in its application to science and the process of scientific discovery in general. However, while RL algorithms learn to solve increasingly complex problems, interpreting the solutions they provide becomes ever more challenging. In this work, we gain insights into an RL agent's learned behavior through a post-hoc analysis based on sequence mining and clustering. Specifically, frequent and compact subroutines, used by the agent to solve a given task, are distilled as gadgets and then grouped by various metrics. This process of gadget discovery develops in three stages: First, we use an RL agent to generate data, then, we employ a mining algorithm to extract gadgets and finally, the obtained gadgets are grouped by a density-based clustering algorithm. We demonstrate our method by applying it to two quantum-inspired RL environments. First, we consider simulated quantum optics experiments for the design of high-dimensional multipartite entangled states where the algorithm finds gadgets that correspond to modern interferometer setups. Second, we consider a circuit-based quantum computing environment where the algorithm discovers various gadgets for quantum information processing, such as quantum teleportation. This approach for analyzing the policy of a learned agent is agent and environment agnostic and can yield interesting insights into any agent's policy

Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories that are in most cases best fit by composite correlated random walks with features that resemble Lévy walks. This composite motion emerges from the collective behavior developed under the specific foraging selection pressures. On the other hand, agents trained to reach nearby resources predominantly exhibit Brownian trajectories.publishe

Honeybee communication during collective defence is shaped by predation

BackgroundSocial insect colonies routinely face large vertebrate predators, against which they need to mount a collective defence. To do so, honeybees use an alarm pheromone that recruits nearby bees into mass stinging of the perceived threat. This alarm pheromone is carried directly on the stinger; hence, its concentration builds up during the course of the attack. We investigate how bees react to different alarm pheromone concentrations and how this evolved response pattern leads to better coordination at the group level.ResultsWe first present a dose-response curve to the alarm pheromone, obtained experimentally. This data reveals two phases in the bees’ response: initially, bees become more likely to sting as the alarm pheromone concentration increases, but aggressiveness drops back when very high concentrations are reached. Second, we apply Projective Simulation to model each bee as an artificial learning agent that relies on the pheromone concentration to decide whether to sting or not. Individuals are rewarded based on the collective performance, thus emulating natural selection in these complex societies. By also modelling predators in a detailed way, we are able to identify the main selection pressures that shaped the response pattern observed experimentally. In particular, the likelihood to sting in the absence of alarm pheromone (starting point of the dose-response curve) is inversely related to the rate of false alarms, such that bees in environments with low predator density are less likely to waste efforts responding to irrelevant stimuli. This is compensated for by a steep increase in aggressiveness when the alarm pheromone concentration starts rising. The later decay in aggressiveness may be explained as a curbing mechanism preventing worker loss.ConclusionsOur work provides a detailed understanding of alarm pheromone responses in honeybees and sheds light on the selection pressures that brought them about. In addition, it establishes our approach as a powerful tool to explore how selection based on a collective outcome shapes individual responses, which remains a challenging issue in the field of evolutionary biology.publishe

Automated gadget discovery in the quantum domain

In recent years, reinforcement learning (RL) has become increasingly successful in its application to the quantum domain and the process of scientific discovery in general. However, while RL algorithms learn to solve increasingly complex problems, interpreting the solutions they provide becomes ever more challenging. In this work, we gain insights into an RL agent’s learned behavior through a post-hoc analysis based on sequence mining and clustering. Specifically, frequent and compact subroutines, used by the agent to solve a given task, are distilled as gadgets and then grouped by various metrics. This process of gadget discovery develops in three stages: First, we use an RL agent to generate data, then, we employ a mining algorithm to extract gadgets and finally, the obtained gadgets are grouped by a density-based clustering algorithm. We demonstrate our method by applying it to two quantum-inspired RL environments. First, we consider simulated quantum optics experiments for the design of high-dimensional multipartite entangled states where the algorithm finds gadgets that correspond to modern interferometer setups. Second, we consider a circuit-based quantum computing environment where the algorithm discovers various gadgets for quantum information processing, such as quantum teleportation. This approach for analyzing the policy of a learned agent is agent and environment agnostic and can yield interesting insights into any agent’s policy