255 research outputs found
SGGNet: Speech-Scene Graph Grounding Network for Speech-guided Navigation
The spoken language serves as an accessible and efficient interface, enabling
non-experts and disabled users to interact with complex assistant robots.
However, accurately grounding language utterances gives a significant challenge
due to the acoustic variability in speakers' voices and environmental noise. In
this work, we propose a novel speech-scene graph grounding network (SGGNet)
that robustly grounds spoken utterances by leveraging the acoustic similarity
between correctly recognized and misrecognized words obtained from automatic
speech recognition (ASR) systems. To incorporate the acoustic similarity, we
extend our previous grounding model, the scene-graph-based grounding network
(SGGNet), with the ASR model from NVIDIA NeMo. We accomplish this by feeding
the latent vector of speech pronunciations into the BERT-based grounding
network within SGGNet. We evaluate the effectiveness of using latent vectors of
speech commands in grounding through qualitative and quantitative studies. We
also demonstrate the capability of SGGNet in a speech-based navigation task
using a real quadruped robot, RBQ-3, from Rainbow Robotics.Comment: 7 pages, 6 figures, Paper accepted for the Special Session at the
2023 International Symposium on Robot and Human Interactive Communication
(RO-MAN), [Dohyun Kim, Yeseung Kim, Jaehwi Jang, and Minjae Song] contributed
equally to this wor
Real-time Digital Double Framework to Predict Collapsible Terrains for Legged Robots
Inspired by the digital twinning systems, a novel real-time digital double
framework is developed to enhance robot perception of the terrain conditions.
Based on the very same physical model and motion control, this work exploits
the use of such simulated digital double synchronized with a real robot to
capture and extract discrepancy information between the two systems, which
provides high dimensional cues in multiple physical quantities to represent
differences between the modelled and the real world. Soft, non-rigid terrains
cause common failures in legged locomotion, whereby visual perception solely is
insufficient in estimating such physical properties of terrains. We used
digital double to develop the estimation of the collapsibility, which addressed
this issue through physical interactions during dynamic walking. The
discrepancy in sensory measurements between the real robot and its digital
double are used as input of a learning-based algorithm for terrain
collapsibility analysis. Although trained only in simulation, the learned model
can perform collapsibility estimation successfully in both simulation and real
world. Our evaluation of results showed the generalization to different
scenarios and the advantages of the digital double to reliably detect nuances
in ground conditions.Comment: IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS). Preprint version. Accepted June 202
Genetically evolved dynamic control for quadruped walking
The aim of this dissertation is to show that dynamic control of quadruped locomotion is achievable through the use of genetically evolved central pattern generators. This strategy is tested both in simulation and on a walking robot. The design of the walker has been chosen to be statically unstable, so that during motion less than three supporting feet may be in contact with the ground.
The control strategy adopted is capable of propelling the artificial walker at a forward locomotion speed of ~1.5 Km/h on rugged terrain and provides for stability of motion. The learning of walking, based on simulated genetic evolution, is carried out in simulation to speed up the process and reduce the amount of damage to the hardware of the walking robot. For this reason a general-purpose fast dynamic simulator has been developed, able to efficiently compute the forward dynamics of tree-like robotic mechanisms.
An optimization process to select stable walking patterns is implemented through a purposely designed genetic algorithm, which implements stochastic mutation and cross-over operators. The algorithm has been tailored to address the high cost of evaluation of the optimization function, as well as the characteristics of the parameter space chosen to represent controllers.
Experiments carried out on different conditions give clear indications on the potential of the approach adopted. A proof of concept is achieved, that stable dynamic walking can be obtained through a search process which identifies attractors in the dynamics of the motor-control system of an artificial walker
Contact Models in Robotics: a Comparative Analysis
Physics simulation is ubiquitous in robotics. Whether in model-based
approaches (e.g., trajectory optimization), or model-free algorithms (e.g.,
reinforcement learning), physics simulators are a central component of modern
control pipelines in robotics. Over the past decades, several robotic
simulators have been developed, each with dedicated contact modeling
assumptions and algorithmic solutions. In this article, we survey the main
contact models and the associated numerical methods commonly used in robotics
for simulating advanced robot motions involving contact interactions. In
particular, we recall the physical laws underlying contacts and friction (i.e.,
Signorini condition, Coulomb's law, and the maximum dissipation principle), and
how they are transcribed in current simulators. For each physics engine, we
expose their inherent physical relaxations along with their limitations due to
the numerical techniques employed. Based on our study, we propose theoretically
grounded quantitative criteria on which we build benchmarks assessing both the
physical and computational aspects of simulation. We support our work with an
open-source and efficient C++ implementation of the existing algorithmic
variations. Our results demonstrate that some approximations or algorithms
commonly used in robotics can severely widen the reality gap and impact target
applications. We hope this work will help motivate the development of new
contact models, contact solvers, and robotic simulators in general, at the root
of recent progress in motion generation in robotics
FlightGoggles: A Modular Framework for Photorealistic Camera, Exteroceptive Sensor, and Dynamics Simulation
FlightGoggles is a photorealistic sensor simulator for perception-driven
robotic vehicles. The key contributions of FlightGoggles are twofold. First,
FlightGoggles provides photorealistic exteroceptive sensor simulation using
graphics assets generated with photogrammetry. Second, it provides the ability
to combine (i) synthetic exteroceptive measurements generated in silico in real
time and (ii) vehicle dynamics and proprioceptive measurements generated in
motio by vehicle(s) in a motion-capture facility. FlightGoggles is capable of
simulating a virtual-reality environment around autonomous vehicle(s). While a
vehicle is in flight in the FlightGoggles virtual reality environment,
exteroceptive sensors are rendered synthetically in real time while all complex
extrinsic dynamics are generated organically through the natural interactions
of the vehicle. The FlightGoggles framework allows for researchers to
accelerate development by circumventing the need to estimate complex and
hard-to-model interactions such as aerodynamics, motor mechanics, battery
electrochemistry, and behavior of other agents. The ability to perform
vehicle-in-the-loop experiments with photorealistic exteroceptive sensor
simulation facilitates novel research directions involving, e.g., fast and
agile autonomous flight in obstacle-rich environments, safe human interaction,
and flexible sensor selection. FlightGoggles has been utilized as the main test
for selecting nine teams that will advance in the AlphaPilot autonomous drone
racing challenge. We survey approaches and results from the top AlphaPilot
teams, which may be of independent interest.Comment: Initial version appeared at IROS 2019. Supplementary material can be
found at https://flightgoggles.mit.edu. Revision includes description of new
FlightGoggles features, such as a photogrammetric model of the MIT Stata
Center, new rendering settings, and a Python AP
Microsoft robotics soccer challenge : movement optimization of a quadruped robot
Estágio realizado na Universidade de Aveiro e orientado pelo Prof. Doutor Nuno LauTese de mestrado integrado. Engenharia Electrotécnica e de Computadores. Faculdade de Engenharia. Universidade do Porto. 200
Becoming Human with Humanoid
Nowadays, our expectations of robots have been significantly increases. The robot, which was initially only doing simple jobs, is now expected to be smarter and more dynamic. People want a robot that resembles a human (humanoid) has and has emotional intelligence that can perform action-reaction interactions. This book consists of two sections. The first section focuses on emotional intelligence, while the second section discusses the control of robotics. The contents of the book reveal the outcomes of research conducted by scholars in robotics fields to accommodate needs of society and industry
- …