8 research outputs found
All macroscopic quantum states are fragile and hard to prepare
We study the effect of local decoherence on arbitrary quantum states.
Adapting techniques developed in quantum metrology, we show that the action of
generic local noise processes -- though arbitrarily small -- always yields a
state whose Quantum Fisher Information (QFI) with respect to local observables
is linear in system size N, independent of the initial state. This implies that
all macroscopic quantum states, which are characterized by a QFI that is
quadratic in N, are fragile under decoherence, and cannot be maintained if the
system is not perfectly isolated. We also provide analytical bounds on the
effective system size, and show that the effective system size scales as the
inverse of the noise parameter p for small p for all the noise channels
considered, making it increasingly difficult to generate macroscopic or even
mesoscopic quantum states. In turn, we also show that the preparation of a
macroscopic quantum state, with respect to a conserved quantity, requires a
device whose QFI is already at least as large as the one of the desired state.
Given that the preparation device itself is classical and not a perfectly
isolated macroscopic quantum state, the preparation device needs to be
quadratically bigger than the macroscopic target state
Optimal foraging strategies can be learned
The foraging behavior of animals is a paradigm of target search in nature.
Understanding which foraging strategies are optimal and how animals learn them
are central challenges in modeling animal foraging. While the question of
optimality has wide-ranging implications across fields such as economy,
physics, and ecology, the question of learnability is a topic of ongoing debate
in evolutionary biology. Recognizing the interconnected nature of these
challenges, this work addresses them simultaneously by exploring optimal
foraging strategies through a reinforcement learning framework. To this end, we
model foragers as learning agents. We first prove theoretically that maximizing
rewards in our reinforcement learning model is equivalent to optimizing
foraging efficiency. We then show with numerical experiments that, in the
paradigmatic model of non-destructive search, our agents learn foraging
strategies which outperform the efficiency of some of the best known strategies
such as L\'evy walks. These findings highlight the potential of reinforcement
learning as a versatile framework not only for optimizing search strategies but
also to model the learning process, thus shedding light on the role of learning
in natural optimization processes.Comment: 12 pages (6 main text) and 9 figures. Codes and tutorials available
at: https://gorkamunoz.github.io/rl_opts
Automated Gadget Discovery in Science
In recent years, reinforcement learning (RL) has become increasingly
successful in its application to science and the process of scientific
discovery in general. However, while RL algorithms learn to solve increasingly
complex problems, interpreting the solutions they provide becomes ever more
challenging. In this work, we gain insights into an RL agent's learned behavior
through a post-hoc analysis based on sequence mining and clustering.
Specifically, frequent and compact subroutines, used by the agent to solve a
given task, are distilled as gadgets and then grouped by various metrics. This
process of gadget discovery develops in three stages: First, we use an RL agent
to generate data, then, we employ a mining algorithm to extract gadgets and
finally, the obtained gadgets are grouped by a density-based clustering
algorithm. We demonstrate our method by applying it to two quantum-inspired RL
environments. First, we consider simulated quantum optics experiments for the
design of high-dimensional multipartite entangled states where the algorithm
finds gadgets that correspond to modern interferometer setups. Second, we
consider a circuit-based quantum computing environment where the algorithm
discovers various gadgets for quantum information processing, such as quantum
teleportation. This approach for analyzing the policy of a learned agent is
agent and environment agnostic and can yield interesting insights into any
agent's policy
Development of swarm behavior in artificial learning agents that adapt to different foraging environments
Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories that are in most cases best fit by composite correlated random walks with features that resemble Lévy walks. This composite motion emerges from the collective behavior developed under the specific foraging selection pressures. On the other hand, agents trained to reach nearby resources predominantly exhibit Brownian trajectories.publishe
Honeybee communication during collective defence is shaped by predation
BackgroundSocial insect colonies routinely face large vertebrate predators, against which they need to mount a collective defence. To do so, honeybees use an alarm pheromone that recruits nearby bees into mass stinging of the perceived threat. This alarm pheromone is carried directly on the stinger; hence, its concentration builds up during the course of the attack. We investigate how bees react to different alarm pheromone concentrations and how this evolved response pattern leads to better coordination at the group level.ResultsWe first present a dose-response curve to the alarm pheromone, obtained experimentally. This data reveals two phases in the bees’ response: initially, bees become more likely to sting as the alarm pheromone concentration increases, but aggressiveness drops back when very high concentrations are reached. Second, we apply Projective Simulation to model each bee as an artificial learning agent that relies on the pheromone concentration to decide whether to sting or not. Individuals are rewarded based on the collective performance, thus emulating natural selection in these complex societies. By also modelling predators in a detailed way, we are able to identify the main selection pressures that shaped the response pattern observed experimentally. In particular, the likelihood to sting in the absence of alarm pheromone (starting point of the dose-response curve) is inversely related to the rate of false alarms, such that bees in environments with low predator density are less likely to waste efforts responding to irrelevant stimuli. This is compensated for by a steep increase in aggressiveness when the alarm pheromone concentration starts rising. The later decay in aggressiveness may be explained as a curbing mechanism preventing worker loss.ConclusionsOur work provides a detailed understanding of alarm pheromone responses in honeybees and sheds light on the selection pressures that brought them about. In addition, it establishes our approach as a powerful tool to explore how selection based on a collective outcome shapes individual responses, which remains a challenging issue in the field of evolutionary biology.publishe
Automated gadget discovery in the quantum domain
In recent years, reinforcement learning (RL) has become increasingly successful in its application to the quantum domain and the process of scientific discovery in general. However, while RL algorithms learn to solve increasingly complex problems, interpreting the solutions they provide becomes ever more challenging. In this work, we gain insights into an RL agent’s learned behavior through a post-hoc analysis based on sequence mining and clustering. Specifically, frequent and compact subroutines, used by the agent to solve a given task, are distilled as gadgets and then grouped by various metrics. This process of gadget discovery develops in three stages: First, we use an RL agent to generate data, then, we employ a mining algorithm to extract gadgets and finally, the obtained gadgets are grouped by a density-based clustering algorithm. We demonstrate our method by applying it to two quantum-inspired RL environments. First, we consider simulated quantum optics experiments for the design of high-dimensional multipartite entangled states where the algorithm finds gadgets that correspond to modern interferometer setups. Second, we consider a circuit-based quantum computing environment where the algorithm discovers various gadgets for quantum information processing, such as quantum teleportation. This approach for analyzing the policy of a learned agent is agent and environment agnostic and can yield interesting insights into any agent’s policy