2,717 research outputs found
Brain-Computer Interface meets ROS: A robotic approach to mentally drive telepresence robots
This paper shows and evaluates a novel approach to integrate a non-invasive
Brain-Computer Interface (BCI) with the Robot Operating System (ROS) to
mentally drive a telepresence robot. Controlling a mobile device by using human
brain signals might improve the quality of life of people suffering from severe
physical disabilities or elderly people who cannot move anymore. Thus, the BCI
user is able to actively interact with relatives and friends located in
different rooms thanks to a video streaming connection to the robot. To
facilitate the control of the robot via BCI, we explore new ROS-based
algorithms for navigation and obstacle avoidance, making the system safer and
more reliable. In this regard, the robot can exploit two maps of the
environment, one for localization and one for navigation, and both can be used
also by the BCI user to watch the position of the robot while it is moving. As
demonstrated by the experimental results, the user's cognitive workload is
reduced, decreasing the number of commands necessary to complete the task and
helping him/her to keep attention for longer periods of time.Comment: Accepted in the Proceedings of the 2018 IEEE International Conference
on Robotics and Automatio
Using Variable Natural Environment Brain-Computer Interface Stimuli for Real-time Humanoid Robot Navigation
This paper addresses the challenge of humanoid robot teleoperation in a
natural indoor environment via a Brain-Computer Interface (BCI). We leverage
deep Convolutional Neural Network (CNN) based image and signal understanding to
facilitate both real-time bject detection and dry-Electroencephalography (EEG)
based human cortical brain bio-signals decoding. We employ recent advances in
dry-EEG technology to stream and collect the cortical waveforms from subjects
while they fixate on variable Steady State Visual Evoked Potential (SSVEP)
stimuli generated directly from the environment the robot is navigating. To
these ends, we propose the use of novel variable BCI stimuli by utilising the
real-time video streamed via the on-board robot camera as visual input for
SSVEP, where the CNN detected natural scene objects are altered and flickered
with differing frequencies (10Hz, 12Hz and 15Hz). These stimuli are not akin to
traditional stimuli - as both the dimensions of the flicker regions and their
on-screen position changes depending on the scene objects detected. On-screen
object selection via such a dry-EEG enabled SSVEP methodology, facilitates the
on-line decoding of human cortical brain signals, via a specialised secondary
CNN, directly into teleoperation robot commands (approach object, move in a
specific direction: right, left or back). This SSVEP decoding model is trained
via a priori offline experimental data in which very similar visual input is
present for all subjects. The resulting classification demonstrates high
performance with mean accuracy of 85% for the real-time robot navigation
experiment across multiple test subjects.Comment: Accepted as a full paper at the 2019 International Conference on
Robotics and Automation (ICRA
On Tackling Fundamental Constraints in Brain-Computer Interface Decoding via Deep Neural Networks
A Brain-Computer Interface (BCI) is a system that provides a communication and control medium between human cortical signals and external devices, with the primary aim to assist or to be used by patients who suffer from a neuromuscular disease. Despite significant recent progress in the area of BCI, there are numerous shortcomings associated with decoding Electroencephalography-based BCI signals in real-world environments. These include, but are not limited to, the cumbersome nature of the equipment, complications in collecting large quantities of real-world data, the rigid experimentation protocol and the challenges of accurate signal decoding, especially in making a system work in real-time. Hence, the core purpose of this work is to investigate improving the applicability and usability of BCI systems, whilst preserving signal decoding accuracy.
Recent advances in Deep Neural Networks (DNN) provide the possibility for signal processing to automatically learn the best representation of a signal, contributing to improved performance even with a noisy input signal. Subsequently, this thesis focuses on the use of novel DNN-based approaches for tackling some of the key underlying constraints within the area of BCI. For example, recent technological improvements in acquisition hardware have made it possible to eliminate the pre-existing rigid experimentation procedure, albeit resulting in noisier signal capture. However, through the use of a DNN-based model, it is possible to preserve the accuracy of the predictions from the decoded signals. Moreover, this research demonstrates that by leveraging DNN-based image and signal understanding, it is feasible to facilitate real-time BCI applications in a natural environment. Additionally, the capability of DNN to generate realistic synthetic data is shown to be a potential solution in reducing the requirement for costly data collection. Work is also performed in addressing the well-known issues regarding subject bias in BCI models by generating data with reduced subject-specific features.
The overall contribution of this thesis is to address the key fundamental limitations of BCI systems. This includes the unyielding traditional experimentation procedure, the mandatory extended calibration stage and sustaining accurate signal decoding in real-time. These limitations lead to a fragile BCI system that is demanding to use and only suited for deployment in a controlled laboratory. Overall contributions of this research aim to improve the robustness of BCI systems and enable new applications for use in the real-world
Audio-visual feedback improves the BCI performance in the navigational control of a humanoid robot
Advancement in brain computer interfaces (BCI) technology allows people to actively interact in the world through surrogates. Controlling real humanoid robots using BCI as intuitively as we control our body represents a challenge for current research in robotics and neuroscience. In order to successfully interact with the environment the brain integrates multiple sensory cues to form a coherent representation of the world. Cognitive neuroscience studies demonstrate that multisensory integration may imply a gain with respect to a single modality and ultimately improve the overall sensorimotor performance. For example, reactivity to simultaneous visual and auditory stimuli may be higher than to the sum of the same stimuli delivered in isolation or in temporal sequence. Yet, knowledge about whether audio-visual integration may improve the control of a surrogate is meager. To explore this issue, we provided human footstep sounds as audio feedback to BCI users while controlling a humanoid robot. Participants were asked to steer their robot surrogate and perform a pick-and-place task through BCI-SSVEPs. We found that audio-visual synchrony between footsteps sound and actual humanoid's walk reduces the time required for steering the robot. Thus, auditory feedback congruent with the humanoid actions may improve motor decisions of the BCI's user and help in the feeling of control over it. Our results shed light on the possibility to increase robot's control through the combination of multisensory feedback to a BCI user. © 2014 Tidoni, Gergondet, Kheddar and Aglioti
Large Language Models for Robotics: A Survey
The human ability to learn, generalize, and control complex manipulation
tasks through multi-modality feedback suggests a unique capability, which we
refer to as dexterity intelligence. Understanding and assessing this
intelligence is a complex task. Amidst the swift progress and extensive
proliferation of large language models (LLMs), their applications in the field
of robotics have garnered increasing attention. LLMs possess the ability to
process and generate natural language, facilitating efficient interaction and
collaboration with robots. Researchers and engineers in the field of robotics
have recognized the immense potential of LLMs in enhancing robot intelligence,
human-robot interaction, and autonomy. Therefore, this comprehensive review
aims to summarize the applications of LLMs in robotics, delving into their
impact and contributions to key areas such as robot control, perception,
decision-making, and path planning. We first provide an overview of the
background and development of LLMs for robotics, followed by a description of
the benefits of LLMs for robotics and recent advancements in robotics models
based on LLMs. We then delve into the various techniques used in the model,
including those employed in perception, decision-making, control, and
interaction. Finally, we explore the applications of LLMs in robotics and some
potential challenges they may face in the near future. Embodied intelligence is
the future of intelligent science, and LLMs-based robotics is one of the
promising but challenging paths to achieve this.Comment: Preprint. 4 figures, 3 table
A Review of Emotion Recognition Using EEG Data and Machine Learning Techniques
Using AI to help humans with handling their emotions and identifying their stress levels in the current stressful lifestyle will greatly help them manage their lifestyle. Using the deep learning techniques, it can be made possible by creating a virtual bot to observe and understand human emotions. In this paper, the researcher try to review the comments from Reddit that are used, preprocessed and trained using Deep Neural Network to learn the emotions of the user. The inference engine module, which is a hybrid network consisting of convolutional neural network and recurrent neural network, is also interfaced. The model provides a high accuracy of response. The selection of frequency bands plays an important role in discerning patterns of brain-related emotions. This document explores a new method for selecting appropriate thematic bands instead of using fixed bands to detect emotions. A common spatial technique and machine machines were used to classify the emotional states. This document describes a number of possible technologies aimed at communication and other applications; however, they represent only a small sample of the extensive future potential of these technologies. We have also focused on relatively anticipated breakthroughs in the discussion of applications in sensory, BCI technologies; but breakthroughs like the new portable sensor technology, which offers ultra-high-resolution spatial and time-based activity in the brain, opens the door to a much broader range of applications. Keywords: Emotions, EEG, Machine Learning, Deep Learning, Systems and Signals DOI: 10.7176/ISDE/11-4-04 Publication date:August 31st 2020
Development of Cognitive Capabilities in Humanoid Robots
Merged with duplicate record 10026.1/645 on 03.04.2017 by CS (TIS)Building intelligent systems with human level of competence is the ultimate
grand challenge for science and technology in general, and especially for the
computational intelligence community. Recent theories in autonomous cognitive
systems have focused on the close integration (grounding) of communication with
perception, categorisation and action. Cognitive systems are essential for
integrated multi-platform systems that are capable of sensing and communicating.
This thesis presents a cognitive system for a humanoid robot that integrates
abilities such as object detection and recognition, which are merged with natural
language understanding and refined motor controls. The work includes three
studies; (1) the use of generic manipulation of objects using the NMFT algorithm,
by successfully testing the extension of the NMFT to control robot behaviour; (2) a
study of the development of a robotic simulator; (3) robotic simulation experiments
showing that a humanoid robot is able to acquire complex behavioural, cognitive,
and linguistic skills through individual and social learning. The robot is able to
learn to handle and manipulate objects autonomously, to cooperate with human
users, and to adapt its abilities to changes in internal and environmental conditions.
The model and the experimental results reported in this thesis, emphasise the
importance of embodied cognition, i.e. the humanoid robot's physical interaction
between its body and the environment
Object Segmentation in Images using EEG Signals
This paper explores the potential of brain-computer interfaces in segmenting
objects from images. Our approach is centered around designing an effective
method for displaying the image parts to the users such that they generate
measurable brain reactions. When an image region, specifically a block of
pixels, is displayed we estimate the probability of the block containing the
object of interest using a score based on EEG activity. After several such
blocks are displayed, the resulting probability map is binarized and combined
with the GrabCut algorithm to segment the image into object and background
regions. This study shows that BCI and simple EEG analysis are useful in
locating object boundaries in images.Comment: This is a preprint version prior to submission for peer-review of the
paper accepted to the 22nd ACM International Conference on Multimedia
(November 3-7, 2014, Orlando, Florida, USA) for the High Risk High Reward
session. 10 page
- …