501 research outputs found
Multimodal human machine interactions in industrial environments
This chapter will present a review of Human Machine Interaction techniques for
industrial applications. A set of recent HMI techniques will be provided with
emphasis on multimodal interaction with industrial machines and robots. This list
will include Natural Language Processing techniques and others that make use of
various complementary interfaces: audio, visual, haptic or gestural, to achieve a
more natural human-machine interaction. This chapter will also focus on providing examples and use cases in fields related to multimodal interaction in manufacturing, such as augmented reality. Accordingly, the chapter will present the use of
Artificial Intelligence and Multimodal Human Machine Interaction in the context
of STAR applications
A review on manipulation skill acquisition through teleoperation-based learning from demonstration
Manipulation skill learning and generalization have gained increasing attention due to the wide applications of robot manipulators and the spurt of robot learning techniques. Especially, the learning from demonstration method has been exploited widely and successfully in the robotic community, and it is regarded as a promising direction to realize the manipulation skill learning and generalization. In addition to the learning techniques, the immersive teleoperation enables the human to operate a remote robot with an intuitive interface and achieve the telepresence. Thus, it is a promising way to transfer manipulation skills from humans to robots by combining the learning methods and the teleoperation, and adapting the learned skills to different tasks in new situations. This review, therefore, aims to provide an overview of immersive teleoperation for skill learning and generalization to deal with complex manipulation tasks. To this end, the key technologies, e.g. manipulation skill learning, multimodal interfacing for teleoperation and telerobotic control, are introduced. Then, an overview is given in terms of the most important applications of immersive teleoperation platform for robot skill learning. Finally, this survey discusses the remaining open challenges and promising research topics
RoboChop: Autonomous Framework for Fruit and Vegetable Chopping Leveraging Foundational Models
With the goal of developing fully autonomous cooking robots, developing
robust systems that can chop a wide variety of objects is important. Existing
approaches focus primarily on the low-level dynamics of the cutting action,
which overlooks some of the practical real-world challenges of implementing
autonomous cutting systems. In this work we propose an autonomous framework to
sequence together action primitives for the purpose of chopping fruits and
vegetables on a cluttered cutting board. We present a novel technique to
leverage vision foundational models SAM and YOLO to accurately detect, segment,
and track fruits and vegetables as they visually change through the sequences
of chops, finetuning YOLO on a novel dataset of whole and chopped fruits and
vegetables. In our experiments, we demonstrate that our simple pipeline is able
to reliably chop a variety of fruits and vegetables ranging in size,
appearance, and texture, meeting a variety of chopping specifications,
including fruit type, number of slices, and types of slices
Immersive Technologies in Virtual Companions: A Systematic Literature Review
The emergence of virtual companions is transforming the evolution of
intelligent systems that effortlessly cater to the unique requirements of
users. These advanced systems not only take into account the user present
capabilities, preferences, and needs but also possess the capability to adapt
dynamically to changes in the environment, as well as fluctuations in the users
emotional state or behavior. A virtual companion is an intelligent software or
application that offers support, assistance, and companionship across various
aspects of users lives. Various enabling technologies are involved in building
virtual companion, among these, Augmented Reality (AR), and Virtual Reality
(VR) are emerging as transformative tools. While their potential for use in
virtual companions or digital assistants is promising, their applications in
these domains remain relatively unexplored. To address this gap, a systematic
review was conducted to investigate the applications of VR, AR, and MR
immersive technologies in the development of virtual companions. A
comprehensive search across PubMed, Scopus, and Google Scholar yielded 28
relevant articles out of a pool of 644. The review revealed that immersive
technologies, particularly VR and AR, play a significant role in creating
digital assistants, offering a wide range of applications that brings various
facilities in the individuals life in areas such as addressing social
isolation, enhancing cognitive abilities and dementia care, facilitating
education, and more. Additionally, AR and MR hold potential for enhancing
Quality of life (QoL) within the context of virtual companion technology. The
findings of this review provide a valuable foundation for further research in
this evolving field
- …