5,129 research outputs found
A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems
We introduce a kind of partial observability to the projective simulation
(PS) learning method. It is done by adding a belief projection operator and an
observability parameter to the original framework of the efficiency of the PS
model. I provide theoretical formulations, network representations, and
situated scenarios derived from the invasion toy problem as a starting point
for some multi-agent PS models.Comment: 28 pages, 21 figure
Projective simulation with generalization
The ability to generalize is an important feature of any intelligent agent.
Not only because it may allow the agent to cope with large amounts of data, but
also because in some environments, an agent with no generalization capabilities
cannot learn. In this work we outline several criteria for generalization, and
present a dynamic and autonomous machinery that enables projective simulation
agents to meaningfully generalize. Projective simulation, a novel, physical
approach to artificial intelligence, was recently shown to perform well in
standard reinforcement learning problems, with applications in advanced
robotics as well as quantum experiments. Both the basic projective simulation
model and the presented generalization machinery are based on very simple
principles. This allows us to provide a full analytical analysis of the agent's
performance and to illustrate the benefit the agent gains by generalizing.
Specifically, we show that already in basic (but extreme) environments,
learning without generalization may be impossible, and demonstrate how the
presented generalization machinery enables the projective simulation agent to
learn.Comment: 14 pages, 9 figure
Speeding-up the decision making of a learning agent using an ion trap quantum processor
We report a proof-of-principle experimental demonstration of the quantum
speed-up for learning agents utilizing a small-scale quantum information
processor based on radiofrequency-driven trapped ions. The decision-making
process of a quantum learning agent within the projective simulation paradigm
for machine learning is implemented in a system of two qubits. The latter are
realized using hyperfine states of two frequency-addressed atomic ions exposed
to a static magnetic field gradient. We show that the deliberation time of this
quantum learning agent is quadratically improved with respect to comparable
classical learning agents. The performance of this quantum-enhanced learning
agent highlights the potential of scalable quantum processors taking advantage
of machine learning.Comment: 21 pages, 7 figures, 2 tables. Author names now spelled correctly;
sections rearranged; changes in the wording of the manuscrip
Benchmarking projective simulation in navigation problems
Projective simulation (PS) is a model for intelligent agents with a
deliberation capacity that is based on episodic memory. The model has been
shown to provide a flexible framework for constructing reinforcement-learning
agents, and it allows for quantum mechanical generalization, which leads to a
speed-up in deliberation time. PS agents have been applied successfully in the
context of complex skill learning in robotics, and in the design of
state-of-the-art quantum experiments. In this paper, we study the performance
of projective simulation in two benchmarking problems in navigation, namely the
grid world and the mountain car problem. The performance of PS is compared to
standard tabular reinforcement learning approaches, Q-learning and SARSA. Our
comparison demonstrates that the performance of PS and standard learning
approaches are qualitatively and quantitatively similar, while it is much
easier to choose optimal model parameters in case of projective simulation,
with a reduced computational effort of one to two orders of magnitude. Our
results show that the projective simulation model stands out for its simplicity
in terms of the number of model parameters, which makes it simple to set up the
learning agent in unknown task environments.Comment: 8 pages, 10 figure
DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics
Synthesizing realistic human movements, dynamically responsive to the
environment, is a long-standing objective in character animation, with
applications in computer vision, sports, and healthcare, for motion prediction
and data augmentation. Recent kinematics-based generative motion models offer
impressive scalability in modeling extensive motion data, albeit without an
interface to reason about and interact with physics. While
simulator-in-the-loop learning approaches enable highly physically realistic
behaviors, the challenges in training often affect scalability and adoption. We
introduce DROP, a novel framework for modeling Dynamics Responses of humans
using generative mOtion prior and Projective dynamics. DROP can be viewed as a
highly stable, minimalist physics-based human simulator that interfaces with a
kinematics-based generative motion prior. Utilizing projective dynamics, DROP
allows flexible and simple integration of the learned motion prior as one of
the projective energies, seamlessly incorporating control provided by the
motion prior with Newtonian dynamics. Serving as a model-agnostic plug-in, DROP
enables us to fully leverage recent advances in generative motion models for
physics-based motion synthesis. We conduct extensive evaluations of our model
across different motion tasks and various physical perturbations, demonstrating
the scalability and diversity of responses.Comment: SIGGRAPH Asia 2023, Video https://youtu.be/tF5WW7qNMLI, Website:
https://stanford-tml.github.io/drop
The words of the body: psychophysiological patterns in dissociative narratives
Trauma has severe consequences on both psychological and somatic levels, even affecting the genetic expression and the cell\u2019s DNA repair ability. A key mechanism in the understanding of clinical disorders deriving from trauma is identified in dissociation, as a primitive defense against the fragmentation of the self originated by overwhelming experiences. The dysregulation of the interpersonal patterns due to the traumatic experience and its detrimental effects on the body are supported by influent neuroscientific models such as Damasio\u2019s somatic markers and Porges\u2019 polyvagal theory. On the basis of these premises, and supported by our previous empirical observations on 40 simulated clinical sessions, we will discuss the longitudinal process of a brief psychodynamic psychotherapy (16 sessions, weekly frequency) with a patient who suffered a relational trauma. The research design consists of the collection of self-report and projective tests, pre-post therapy and after each clinical session, in order to assess personality, empathy, clinical alliance and clinical progress, along with the verbatim analysis of the transcripts trough the Psychotherapy Process Q-Set and the Collaborative Interactions Scale. Furthermore, we collected simultaneous psychophysiological measures of the therapeutic dyad: skin conductance and hearth rate. Lastly, we employed a computerized analysis of non-verbal behaviors to assess synchrony in posture and gestures. These automated measures are able to highlight moments of affective concordance and discordance, allowing for a deep understanding of the mutual regulations between the patient and the therapist. Preliminary results showed that psychophysiological changes in dyadic synchrony, observed in body movements, skin conductance and hearth rate, occurred within sessions during the discussion of traumatic experiences, with levels of attunement that changed in both therapist and the patient depending on the quality of the emotional representation of the experience. These results go in the direction of understanding the relational process in trauma therapy, using an integrative language in which both clinical and neurophysiological knowledge may take advantage of each other
- …