513 research outputs found
Motion Switching with Sensory and Instruction Signals by designing Dynamical Systems using Deep Neural Network
To ensure that a robot is able to accomplish an extensive range of tasks, it
is necessary to achieve a flexible combination of multiple behaviors. This is
because the design of task motions suited to each situation would become
increasingly difficult as the number of situations and the types of tasks
performed by them increase. To handle the switching and combination of multiple
behaviors, we propose a method to design dynamical systems based on point
attractors that accept (i) "instruction signals" for instruction-driven
switching. We incorporate the (ii) "instruction phase" to form a point
attractor and divide the target task into multiple subtasks. By forming an
instruction phase that consists of point attractors, the model embeds a subtask
in the form of trajectory dynamics that can be manipulated using sensory and
instruction signals. Our model comprises two deep neural networks: a
convolutional autoencoder and a multiple time-scale recurrent neural network.
In this study, we apply the proposed method to manipulate soft materials. To
evaluate our model, we design a cloth-folding task that consists of four
subtasks and three patterns of instruction signals, which indicate the
direction of motion. The results depict that the robot can perform the required
task by combining subtasks based on sensory and instruction signals. And, our
model determined the relations among these signals using its internal dynamics.Comment: 8 pages, 6 figures, accepted for publication in RA-L. An accompanied
video is available at this https://youtu.be/a73KFtOOB5
Sensorimotor input as a language generalisation tool: a neurorobotics model for generation and generalisation of noun-verb combinations with sensorimotor inputs
The paper presents a neurorobotics cognitive model explaining the understanding and generalisation of nouns and verbs combinations when a vocal command consisting of a verb-noun sentence is provided to a humanoid robot. The dataset used for training was obtained from object manipulation tasks with a humanoid robot platform; it includes 9 motor actions and 9 objects placing placed in 6 different locations), which enables the robot to learn to handle real-world objects and actions. Based on the multiple time-scale recurrent neural networks, this study demonstrates its generalisation capability using a large data-set, with which the robot was able to generalise semantic representation of novel combinations of noun-verb sentences, and therefore produce the corresponding motor behaviours. This generalisation process is done via the grounding process: different objects are being interacted, and associated, with different motor behaviours, following a learning approach inspired by developmental language acquisition in infants. Further analyses of the learned network dynamics and representations also demonstrate how the generalisation is possible via the exploitation of this functional hierarchical recurrent network
Learning to walk in confined spaces using 3D representation
Legged robots have the potential to traverse complex terrain and access
confined spaces beyond the reach of traditional platforms thanks to their
ability to carefully select footholds and flexibly adapt their body posture
while walking. However, robust deployment in real-world applications is still
an open challenge. In this paper, we present a method for legged locomotion
control using reinforcement learning and 3D volumetric representations to
enable robust and versatile locomotion in confined and unstructured
environments. By employing a two-layer hierarchical policy structure, we
exploit the capabilities of a highly robust low-level policy to follow 6D
commands and a high-level policy to enable three-dimensional spatial awareness
for navigating under overhanging obstacles. Our study includes the development
of a procedural terrain generator to create diverse training environments. We
present a series of experimental evaluations in both simulation and real-world
settings, demonstrating the effectiveness of our approach in controlling a
quadruped robot in confined, rough terrain. By achieving this, our work extends
the applicability of legged robots to a broader range of scenarios.Comment: Accepted to ICRA 202
VIGOR: A Versatile, Individualized and Generative ORchestrator to Motivate the Movement of the People with Limited Mobility
Physical inactivity is a major national concern, particularly among individuals with chronic conditions and/or disabilities. There is an urgent need to devise practical and innovative fitness methods, designed and grounded in physical, psychological and social considerations that will effectively promote physical fitness participation among individuals of all age groups with chronic health condition(s) and/or disabilities. This research is dedicated to achieving Versatile, Individualized, and Generative ORchestrator (VIGOR) to motivate the movement of the people with limited mobility. Tai-Chi is a traditional mind–body wellness and healing art, and its clinical benefits have been well documented. This work presents a Tai-Chi based VIGOR under development. Through the use of Helping, Pushing and Coaching (HPC) functions by following Tai-Chi kinematics, the VIGOR system is designed to make engagement in physical activity an affordable, individually engaging, and enjoyable experience for individuals who live with mobility due to disease or injury. VIGOR consists of the following major modules: (1) seamless human-machine interaction based on the acquisition, transmission, and reconstruction of 4D data (XYZ plus somatosensory) using affordable I/O instruments such as Kinect, Sensor and Tactile actuator, and active-orthosis/exoskeleton; (2) processing and normalization of kinetic data; (3) Identification and grading of kinetics in real time; (4) adaptive virtual limb generation and its reconstruction on virtual reality (VR) or active-orthosis/exoskeleton; and (5) individualized physical activity choreography (i.e., creative movement design). Aiming at developing a deep-learning-enabled rehab and fitness modality through infusing the domain knowledge (physical therapy, medical anthropology, psychology, electrical engineering, bio-mechanics, and athletic aesthetics) into deep neural network, this work is transformative in that the technology can be applied to the broad research areas of intelligent systems, human-computer interaction, and cyber-physical human systems. The resulting VIGOR has significant potentials as both rehabilitative and fitness modalities and can be adapted to other movement modalities and chronic medical conditions (e.g., yoga and balance exercise; fibromyalgia, multiple sclerosis, Parkinson disease)
Pathway to Future Symbiotic Creativity
This report presents a comprehensive view of our vision on the development
path of the human-machine symbiotic art creation. We propose a classification
of the creative system with a hierarchy of 5 classes, showing the pathway of
creativity evolving from a mimic-human artist (Turing Artists) to a Machine
artist in its own right. We begin with an overview of the limitations of the
Turing Artists then focus on the top two-level systems, Machine Artists,
emphasizing machine-human communication in art creation. In art creation, it is
necessary for machines to understand humans' mental states, including desires,
appreciation, and emotions, humans also need to understand machines' creative
capabilities and limitations. The rapid development of immersive environment
and further evolution into the new concept of metaverse enable symbiotic art
creation through unprecedented flexibility of bi-directional communication
between artists and art manifestation environments. By examining the latest
sensor and XR technologies, we illustrate the novel way for art data collection
to constitute the base of a new form of human-machine bidirectional
communication and understanding in art creation. Based on such communication
and understanding mechanisms, we propose a novel framework for building future
Machine artists, which comes with the philosophy that a human-compatible AI
system should be based on the "human-in-the-loop" principle rather than the
traditional "end-to-end" dogma. By proposing a new form of inverse
reinforcement learning model, we outline the platform design of machine
artists, demonstrate its functions and showcase some examples of technologies
we have developed. We also provide a systematic exposition of the ecosystem for
AI-based symbiotic art form and community with an economic model built on NFT
technology. Ethical issues for the development of machine artists are also
discussed
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
The vast proliferation of sensor devices and Internet of Things enables the
applications of sensor-based activity recognition. However, there exist
substantial challenges that could influence the performance of the recognition
system in practical scenarios. Recently, as deep learning has demonstrated its
effectiveness in many areas, plenty of deep methods have been investigated to
address the challenges in activity recognition. In this study, we present a
survey of the state-of-the-art deep learning methods for sensor-based human
activity recognition. We first introduce the multi-modality of the sensory data
and provide information for public datasets that can be used for evaluation in
different challenge tasks. We then propose a new taxonomy to structure the deep
methods by challenges. Challenges and challenge-related deep methods are
summarized and analyzed to form an overview of the current research progress.
At the end of this work, we discuss the open issues and provide some insights
for future directions
Learning What To Say And What To Do: A Model For Grounding Language And Actions
Automation is becoming increasingly important in nowadays society, with robots performing a lot of repetitive tasks in industry and even entering our households in the form of vacuum cleaners and lawn mowers. When considering regular tasks outside of the controlled environments of industry, robots tend to perform poorly. In particular, in situations where robots have to interact with humans, a problem arises: how can a robot understand what the human means? While a lot of work has been made in the past towards visual perception and classification of objects, but understanding what action a verb translates into has still been an unexplored area. In solving this challenge, we would enable robots to execute commands given in natural language, and also to verbalise what actions they are performing when prompted.
This work studies how a robot can learn the meaning behind the sentences humans use, how it translates into its perception and the real world, but also how to translate its actions into sentences humans understand. To achieve this we propose a novel Bidirectional machine learning model, along with a data collection module that can be used by non-technical users. The main idea behind this model is the ability to generalise to novel concepts, being able to compose new sentences and actions from what it learned previously. Humans show this ability to generalise from a young age, and it is a desirable feature for this model. By using humans natural teaching instincts to teach the robot together with this generalisation ability we hope to obtain a model that allows people everywhere to teach the robot to perform the actions we desire. We validate the model in a number of tasks, using an iCub and Pepper robots physically interacting with objects in order to complete a natural language command. We test different actions, including motor actions and emotional displays, while using both transitive and intransitive verbs in the natural language commands.
The main contribution of this thesis is the development of a Bidirectional Learning Algorithm, applied to a Multiple Timescale Recurrent Neural Network enabling these models to link action and language in a bidirectional way. A second contribution sees the extension of Multiple Timescale architectures to Long Short-Term Memory models, increasing the capabilities of these models. Finally the third contribution is in the form of data collection modules, with the development of an easy-to-use module based on physical interaction and speech to provide the iCub and Pepper robots with the data to be learned
- …