116 research outputs found
Gaze-based teleprosthetic enables intuitive continuous control of complex robot arm use: Writing & drawing
Eye tracking is a powerful mean for assistive technologies for people with movement disorders, paralysis and amputees. We present a highly intuitive eye tracking-controlled robot arm operating in 3-dimensional space based on the user's gaze target point that enables tele-writing and drawing. The usability and intuitive usage was assessed by a “tele” writing experiment with 8 subjects that learned to operate the system within minutes of first time use. These subjects were naive to the system and the task and had to write three letters on a white board with a white board pen attached to the robot arm's endpoint. The instructions are to imagine they were writing text with the pen and look where the pen would be going, they had to write the letters as fast and as accurate as possible, given a letter size template. Subjects were able to perform the task with facility and accuracy, and movements of the arm did not interfere with subjects ability to control their visual attention so as to enable smooth writing. On the basis of five consecutive trials there was a significant decrease in the total time used and the total number of commands sent to move the robot arm from the first to the second trial but no further improvement thereafter, suggesting that within writing 6 letters subjects had mastered the ability to control the system. Our work demonstrates that eye tracking is a powerful means to control robot arms in closed-loop and real-time, outperforming other invasive and non-invasive approaches to Brain-Machine-Interfaces in terms of calibration time (<;2 minutes), training time (<;10 minutes), interface technology costs. We suggests that gaze-based decoding of action intention may well become one of the most efficient ways to interface with robotic actuators - i.e. Brain-Robot-Interfaces - and become useful beyond paralysed and amputee users also for the general teleoperation of robotic and exoskeleton in human augmentation
Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation
Robotic systems are ever more capable of automation and fulfilment of complex
tasks, particularly with reliance on recent advances in intelligent systems,
deep learning and artificial intelligence. However, as robots and humans come
closer in their interactions, the matter of interpretability, or explainability
of robot decision-making processes for the human grows in importance. A
successful interaction and collaboration will only take place through mutual
understanding of underlying representations of the environment and the task at
hand. This is currently a challenge in deep learning systems. We present a
hierarchical deep reinforcement learning system, consisting of a low-level
agent handling the large actions/states space of a robotic system efficiently,
by following the directives of a high-level agent which is learning the
high-level dynamics of the environment and task. This high-level agent forms a
representation of the world and task at hand that is interpretable for a human
operator. The method, which we call Dot-to-Dot, is tested on a MuJoCo-based
model of the Fetch Robotics Manipulator, as well as a Shadow Hand, to test its
performance. Results show efficient learning of complex actions/states spaces
by the low-level agent, and an interpretable representation of the task and
decision-making process learned by the high-level agent
Physics-informed reinforcement learning via probabilistic co-adjustment functions
Reinforcement learning of real-world tasks is very data inefficient, and
extensive simulation-based modelling has become the dominant approach for
training systems. However, in human-robot interaction and many other real-world
settings, there is no appropriate one-model-for-all due to differences in
individual instances of the system (e.g. different people) or necessary
oversimplifications in the simulation models. This requires two approaches: 1.
either learning the individual system's dynamics approximately from data which
requires data-intensive training or 2. using a complete digital twin of the
instances, which may not be realisable in many cases. We introduce two
approaches: co-kriging adjustments (CKA) and ridge regression adjustment (RRA)
as novel ways to combine the advantages of both approaches. Our adjustment
methods are based on an auto-regressive AR1 co-kriging model that we integrate
with GP priors. This yield a data- and simulation-efficient way of using
simplistic simulation models (e.g., simple two-link model) and rapidly adapting
them to individual instances (e.g., biomechanics of individual people). Using
CKA and RRA, we obtain more accurate uncertainty quantification of the entire
system's dynamics than pure GP-based and AR1 methods. We demonstrate the
efficiency of co-kriging adjustment with an interpretable reinforcement
learning control example, learning to control a biomechanical human arm using
only a two-link arm simulation model (offline part) and CKA derived from a
small amount of interaction data (on-the-fly online). Our method unlocks an
efficient and uncertainty-aware way to implement reinforcement learning methods
in real world complex systems for which only imperfect simulation models exist
Federated deep transfer learning for EEG decoding using multiple BCI tasks
Deep learning has been successful in BCI decoding. However, it is very
data-hungry and requires pooling data from multiple sources. EEG data from
various sources decrease the decoding performance due to negative transfer.
Recently, transfer learning for EEG decoding has been suggested as a remedy and
become subject to recent BCI competitions (e.g. BEETL), but there are two
complications in combining data from many subjects. First, privacy is not
protected as highly personal brain data needs to be shared (and copied across
increasingly tight information governance boundaries). Moreover, BCI data are
collected from different sources and are often based on different BCI tasks,
which has been thought to limit their reusability. Here, we demonstrate a
federated deep transfer learning technique, the Multi-dataset Federated
Separate-Common-Separate Network (MF-SCSN) based on our previous work of SCSN,
which integrates privacy-preserving properties into deep transfer learning to
utilise data sets with different tasks. This framework trains a BCI decoder
using different source data sets obtained from different imagery tasks (e.g.
some data sets with hands and feet, vs others with single hands and tongue,
etc). Therefore, by introducing privacy-preserving transfer learning
techniques, we unlock the reusability and scalability of existing BCI data
sets. We evaluated our federated transfer learning method on the NeurIPS 2021
BEETL competition BCI task. The proposed architecture outperformed the baseline
decoder by 3%. Moreover, compared with the baseline and other transfer learning
algorithms, our method protects the privacy of the brain data from different
data centres.Comment: 4 pages, 3 figure
- …