244,937 research outputs found
Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes
Humans and animals have a rich and flexible understanding of the physical
world, which enables them to infer the underlying dynamical trajectories of
objects and events, plausible future states, and use that to plan and
anticipate the consequences of actions. However, the neural mechanisms
underlying these computations are unclear. We combine a goal-driven modeling
approach with dense neurophysiological data and high-throughput human
behavioral readouts to directly impinge on this question. Specifically, we
construct and evaluate several classes of sensory-cognitive networks to predict
the future state of rich, ethologically-relevant environments, ranging from
self-supervised end-to-end models with pixel-wise or object-centric objectives,
to models that future predict in the latent space of purely static image-based
or dynamic video-based pretrained foundation models. We find strong
differentiation across these model classes in their ability to predict neural
and behavioral data both within and across diverse environments. In particular,
we find that neural responses are currently best predicted by models trained to
predict the future state of their environment in the latent space of pretrained
foundation models optimized for dynamic scenes in a self-supervised manner.
Notably, models that future predict in the latent space of video foundation
models that are optimized to support a diverse range of sensorimotor tasks,
reasonably match both human behavioral error patterns and neural dynamics
across all environmental scenarios that we were able to test. Overall, these
findings suggest that the neural mechanisms and behaviors of primate mental
simulation are thus far most consistent with being optimized to future predict
on dynamic, reusable visual representations that are useful for embodied AI
more generally.Comment: 17 pages, 6 figure
Walking dynamics are symmetric (enough)
Many biological phenomena such as locomotion, circadian cycles, and breathing
are rhythmic in nature and can be modeled as rhythmic dynamical systems.
Dynamical systems modeling often involves neglecting certain characteristics of
a physical system as a modeling convenience. For example, human locomotion is
frequently treated as symmetric about the sagittal plane. In this work, we test
this assumption by examining human walking dynamics around the steady-state
(limit-cycle). Here we adapt statistical cross validation in order to examine
whether there are statistically significant asymmetries, and even if so, test
the consequences of assuming bilateral symmetry anyway. Indeed, we identify
significant asymmetries in the dynamics of human walking, but nevertheless show
that ignoring these asymmetries results in a more consistent and predictive
model. In general, neglecting evident characteristics of a system can be more
than a modeling convenience---it can produce a better model.Comment: Draft submitted to Journal of the Royal Society Interfac
A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments
This paper reports on a data-driven, interaction-aware motion prediction
approach for pedestrians in environments cluttered with static obstacles. When
navigating in such workspaces shared with humans, robots need accurate motion
predictions of the surrounding pedestrians. Human navigation behavior is mostly
influenced by their surrounding pedestrians and by the static obstacles in
their vicinity. In this paper we introduce a new model based on Long-Short Term
Memory (LSTM) neural networks, which is able to learn human motion behavior
from demonstrated data. To the best of our knowledge, this is the first
approach using LSTMs, that incorporates both static obstacles and surrounding
pedestrians for trajectory forecasting. As part of the model, we introduce a
new way of encoding surrounding pedestrians based on a 1d-grid in polar angle
space. We evaluate the benefit of interaction-aware motion prediction and the
added value of incorporating static obstacles on both simulation and real-world
datasets by comparing with state-of-the-art approaches. The results show, that
our new approach outperforms the other approaches while being very
computationally efficient and that taking into account static obstacles for
motion predictions significantly improves the prediction accuracy, especially
in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International
Conference on Robotics and Automation (ICRA) 201
A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments
This paper reports on a data-driven, interaction-aware motion prediction
approach for pedestrians in environments cluttered with static obstacles. When
navigating in such workspaces shared with humans, robots need accurate motion
predictions of the surrounding pedestrians. Human navigation behavior is mostly
influenced by their surrounding pedestrians and by the static obstacles in
their vicinity. In this paper we introduce a new model based on Long-Short Term
Memory (LSTM) neural networks, which is able to learn human motion behavior
from demonstrated data. To the best of our knowledge, this is the first
approach using LSTMs, that incorporates both static obstacles and surrounding
pedestrians for trajectory forecasting. As part of the model, we introduce a
new way of encoding surrounding pedestrians based on a 1d-grid in polar angle
space. We evaluate the benefit of interaction-aware motion prediction and the
added value of incorporating static obstacles on both simulation and real-world
datasets by comparing with state-of-the-art approaches. The results show, that
our new approach outperforms the other approaches while being very
computationally efficient and that taking into account static obstacles for
motion predictions significantly improves the prediction accuracy, especially
in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International
Conference on Robotics and Automation (ICRA) 201
Reduced order modeling of fluid flows: Machine learning, Kolmogorov barrier, closure modeling, and partitioning
In this paper, we put forth a long short-term memory (LSTM) nudging framework
for the enhancement of reduced order models (ROMs) of fluid flows utilizing
noisy measurements. We build on the fact that in a realistic application, there
are uncertainties in initial conditions, boundary conditions, model parameters,
and/or field measurements. Moreover, conventional nonlinear ROMs based on
Galerkin projection (GROMs) suffer from imperfection and solution instabilities
due to the modal truncation, especially for advection-dominated flows with slow
decay in the Kolmogorov width. In the presented LSTM-Nudge approach, we fuse
forecasts from a combination of imperfect GROM and uncertain state estimates,
with sparse Eulerian sensor measurements to provide more reliable predictions
in a dynamical data assimilation framework. We illustrate the idea with the
viscous Burgers problem, as a benchmark test bed with quadratic nonlinearity
and Laplacian dissipation. We investigate the effects of measurements noise and
state estimate uncertainty on the performance of the LSTM-Nudge behavior. We
also demonstrate that it can sufficiently handle different levels of temporal
and spatial measurement sparsity. This first step in our assessment of the
proposed model shows that the LSTM nudging could represent a viable realtime
predictive tool in emerging digital twin systems
Estimation of muscular forces from SSA smoothed sEMG signals calibrated by inverse dynamics-based physiological static optimization
The estimation of muscular forces is useful in several areas such as biomedical or rehabilitation engineering. As muscular forces cannot be measured in vivo non-invasively they must be estimated by using indirect measurements such as surface electromyography (sEMG) signals or by means of inverse dynamic (ID) analyses. This paper proposes an approach to estimate muscular forces based on both of them. The main idea is to tune a gain matrix so as to compute muscular forces from sEMG signals. To do so, a curve fitting process based on least-squares is carried out. The input is the sEMG signal filtered using singular spectrum analysis technique. The output corresponds to the muscular force estimated by the ID analysis of the recorded task, a dumbbell weightlifting. Once the model parameters are tuned, it is possible to obtain an estimation of muscular forces based on sEMG signal. This procedure might be used to predict muscular forces in vivo outside the space limitations of the gait analysis laboratory.Postprint (published version
- …