11 research outputs found
Design and Quantitative Assessment of Teleoperation-Based Human–Robot Collaboration Method for Robot-Assisted Sonography
Tele-echography has emerged as a promising and effective solution, leveraging the expertise of sonographers and the autonomy of robots to perform ultrasound scanning for patients residing in remote areas, without the need for in-person visits by the sonographer. Designing effective and natural human-robot interfaces for tele-echography remains challenging, with patient safety being a critical concern. In this article, we develop a teleoperation system for robot-assisted sonography with two different interfaces, a haptic device-based interface and a low-cost 3D Mouse-based interface, which can achieve continuous and intuitive telemanipulation by a leader device with a small workspace. To achieve compliant interaction with patients, we design impedance controllers in Cartesian space to track the desired position and orientation for these two teleoperation interfaces. We also propose comprehensive evaluation metrics of robot-assisted sonography, including subjective and objective evaluation, to evaluate tele-echography interfaces and control performance. We evaluate the ergonomic performance based on the estimated muscle fatigue and the acquired ultrasound image quality. We conduct user studies based on the NASA Task Load Index to evaluate the performance of these two human-robot interfaces. The tracking performance and the quantitative comparison of these two teleoperation interfaces are conducted by the Franka Emika Panda robot. The results and findings provide guidance on human-robot collaboration design and implementation for robot-assisted sonography. Note to Practitioners —Robot-assisted sonography has demonstrated efficacy in medical diagnosis during clinical trials. However, deploying fully autonomous robots for ultrasound scanning remains challenging due to various constraints in practice, such as patient safety, dynamic tasks, and environmental uncertainties. Semi-autonomous or teleoperation-based robot sonography represents a promising approach for practical deployment. Previous work has produced various expensive teleoperation interfaces but lacks user studies to guide teleoperation interface selection. In this article, we present two typical teleoperation interfaces and implement a continuous and intuitive teleoperation control system. We also propose a comprehensive evaluation metric for assessing their performance. Our findings show that the haptic device outperforms the 3D Mouse, based on operators’ feedback and acquired image quality. However, the haptic device requires more learning time and effort in the training stage. Furthermore, the developed teleoperation system offers a solution for shared control and human-robot skill transfer. Our results provide valuable guidance for designing and implementing human-robot interfaces for robot-assisted sonography in practice
Coupling Robot-aided assessment and surface electromyography to evaluate wrist and forearm muscles activity, muscle fatigue and its effect on proprioception
Sensorimotor functions and an intact neural control of muscles are essential for the effective
execution of movements during daily living tasks. However, despite the ability of human
sensorimotor system to cope with a great diversity of internal and external demands and
constraints, these mechanisms can be altered as a consequence of neurological disorders,
injuries or just due to excessive effort leading to muscle fatigue.
A precise assessment of both motor and sensory impairment is thus needed in order to provide
useful cues to monitor the progression of the disease in pathological populations or to prevent
injuries in case of workers.
In particular, considering muscle fatigue, an objective assessment of its manifestation may
be crucial when dealing with subjects with neuromuscular disorders for understanding how
specific disease features evolve over time or for testing the efficacy of a potential therapeutic
strategy. Indeed, muscle fatigue accounts for a significant portion of the disease burden in
populations with neuromuscular diseases but, despite its importance, a standardized, reliable
and objective method for fatigue measurement is lacking in clinical practice. The work
presented in this thesis investigates a practical solution through the use of a robotic task and
parameters extracted by surface electromyography signals.
Moreover, a similar approach that combines robot-mediated proprioception test and muscle
fatigue assessment has been developed and used in this thesis to objectively investigate the
influence of muscle fatigue on position sense.
Finally, the effect of posture on muscle activity, from a perspective of injuries prevention,
has been examined. Data on adults and children have been collected and quantitative and
objective information about muscle activity, muscle fatigue and joint sensitivity were obtained
gaining useful insight both in the clinical context and in the prevention of workplace injuries.
A novel method to assess muscle fatigue has been proposed together with the definition of an
easy readable indicator that can help clinicians in the assessment of the patient. As for the
impact of fatigue on the sensorimotor system, results obtained showed a decrease in wrist
proprioceptive acuity which led also to a decline in the performance of a simple tracing task. Regarding the adoption of different muscle strategies depending on postures, results showed
that muscle activity of forearm muscles was overall similar regardless from the postures
Haptics: Science, Technology, Applications
This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility
Haptics: Science, Technology, Applications
This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility
A high speed sensor system for tactile interaction research
Schürmann C. A high speed sensor system for tactile interaction research. Bielefeld: Bielefeld University Library; 2013.In this work we will describe and implement the first tactile sensor system that combines the properties of modularity with a very high sensing speed, a high sensitivity and a high spatial resolution. This unique combination of features enables researchers to develop novel applications and makes it possible to replace
task specific tactile sensors with a single system.
The very high sensing speed of the system allows for slip detection during robot grasping. And as all our sensor cells are sampled with the same high frequency, our system can even enable the slip detection for multiple contact
points at the same time. This high speed was made possible through the development of a highly integrated parallel sensor sampling architecture.
The modularity of the system allows it to be employed in a multitude of applications. Tactile sensitive surfaces of various dimensions can be easily realized through a very simple ’plug and use’ principle without the need for software configuration by the user. This was made possible by developing a new bus system that allows the relative localization of the participants. Our system can be used to create tactile sensitive table surfaces with a large amount of sensor cells and due to its high speed design still provide for real time frame rates.
The flexibility and high performance of the system enabled us to develop a tactile sensitive object that allows the continuous high speed monitoring of human finger forces. For this we solved the problem of integrating the tactile
sensors to allow free movement of the object, while maintaining a constant high rate of data capture and realizing a low latency synchronization to external
devices.
The high sensitivity of the system was made possible through technical innovation in the state of the art of resistive based tactile sensors. We did so by creating an optimized sensor cell shape and investigating the behavior of different sensor materials. The knowledge gained in this process was further used to advance the existing method of sensor normalization into a real time method.
We will present a range of tactile interaction scenarios that have been realized with the tactile sensor system named Myrmex. These scenarios include the investigating of human grasp force control during a pick and place task, a tactile table for integration into an intelligent household and a tactile table for the manipulation of virtual clay as a form of finger training.
In addition we will present a selection of scenarios where the Myrmex system was employed by other researchers, as in using the sensor modules as (large) tactile fingertips on robot arms to implement tactile servoing or slip detection during object grasping. The system has also been used to study human finger forces as well as investigating novel methods for prosthesis control. The positive results from all the scenarios support our conclusion that the developed Myrmex system is a very valuable and reliable tool for the research of tactile interactions
Haptics: Science, Technology, Applications
This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications
Closed-loop prosthetic hand : understanding sensorimotor and multisensory integration under uncertainty.
To make sense of our unpredictable world, humans use sensory information streaming
through billions of peripheral neurons. Uncertainty and ambiguity plague each sensory
stream, yet remarkably our perception of the world is seamless, robust and often
optimal in the sense of minimising perceptual variability. Moreover, humans have
a remarkable capacity for dexterous manipulation. Initiation of precise motor actions
under uncertainty requires awareness of not only the statistics of our environment but
also the reliability of our sensory and motor apparatus.
What happens when our sensory and motor systems are disrupted? Upper-limb amputees
tted with a state-of-the-art prostheses must learn to both control and make
sense of their robotic replacement limb. Tactile feedback is not a standard feature of
these open-loop limbs, fundamentally limiting the degree of rehabilitation. This thesis
introduces a modular closed-loop upper-limb prosthesis, a modified Touch Bionics ilimb
hand with a custom-built linear vibrotactile feedback array. To understand the utility of
the feedback system in the presence of multisensory and sensorimotor influences, three
fundamental open questions were addressed: (i) What are the mechanisms by which
subjects compute sensory uncertainty? (ii) Do subjects integrate an artificial modality
with visual feedback as a function of sensory uncertainty? (iii) What are the influences
of open-loop and closed-loop uncertainty on prosthesis control?
To optimally handle uncertainty in the environment people must acquire estimates of
the mean and uncertainty of sensory cues over time. A novel visual tracking experiment
was developed in order to explore the processes by which people acquire these statistical
estimators. Subjects were required to simultaneously report their evolving estimate of
the mean and uncertainty of visual stimuli over time. This revealed that subjects could
accumulate noisy evidence over the course of a trial to form an optimal continuous estimate
of the mean, hindered only by natural kinematic constraints. Although subjects
had explicit access to a measure of their continuous objective uncertainty, acquired from
sensory information available within a trial, this was limited by a conservative margin
for error.
In the Bayesian framework, sensory evidence (from multiple sensory cues) and prior
beliefs (knowledge of the statistics of sensory cues) are combined to form a posterior
estimate of the state of the world. Multiple studies have revealed that humans behave as
optimal Bayesian observers when making binary decisions in forced-choice tasks. In this
thesis these results were extended to a continuous spatial localisation task. Subjects
could rapidly accumulate evidence presented via vibrotactile feedback (an artificial
modality ), and integrate it with visual feedback. The weight attributed to each sensory
modality was chosen so as to minimise the overall objective uncertainty.
Since subjects were able to combine multiple sources of sensory information with respect
to their sensory uncertainties, it was hypothesised that vibrotactile feedback would benefit prosthesis wearers in the presence of either sensory or motor uncertainty. The
closed-loop prosthesis served as a novel manipulandum to examine the role of feed-forward
and feed-back mechanisms for prosthesis control, known to be required for
successful object manipulation in healthy humans. Subjects formed economical grasps
in idealised (noise-free) conditions and this was maintained even when visual, tactile
and both sources of feedback were removed. However, when uncertainty was introduced
into the hand controller, performance degraded significantly in the absence of visual or
tactile feedback. These results reveal the complementary nature of feed-forward and
feed-back processes in simulated prosthesis wearers, and highlight the importance of
tactile feedback for control of a prosthesis
Enhancing tele-operation - Investigating the effect of sensory feedback on performance
The decline in the number of healthcare service providers in comparison to the growing numbers of service users prompts the development of technologies to improve the efficiency of healthcare services. One such technology which could offer support are assistive robots, remotely tele-operated to provide assistive care and support for older adults with assistive care needs and people living with disabilities. Tele-operation makes it possible to provide human-in-the-loop robotic assistance while also addressing safety concerns in the use of autonomous robots around humans. Unlike many other applications of robot tele-operation, safety is particularly significant as the tele-operated assistive robots will be used in close proximity to vulnerable human users. It is therefore important to provide as much information about the robot (and the robot workspace) as possible to the tele-operators to ensure safety, as well as efficiency. Since robot tele-operation is relatively unexplored in the context of assisted living, this thesis explores different feedback modalities that may be employed to communicate sensor information to tele-operators. The thesis presents research as it transitioned from identifying and evaluating additional feedback modalities that may be used to supplement video feedback, to exploring different strategies for communicating the different feedback modalities. Due to the fact that some of the sensors and feedback needed are not readily available, different design iterations were carried out to develop the necessary hardware and software for the studies carried out. The first human study was carried out to investigate the effect of feedback on tele-operator performance. Performance was measured in terms of task completion time, ease of use of the system, number of robot joint movements, and success or failure of the task. The effect of verbal feedback between the tele-operator and service users was also investigated. Feedback modalities have differing effects on performance metrics and as a result, the choice of optimal feedback may vary from task to task. Results show that participants preferred scenarios with verbal feedback relative to scenarios without verbal feedback, which also reflects in their performance. Gaze metrics from the study also showed that it may be possible to understand how tele-operators interact with the system based on their areas of interest as they carry out tasks. This findings suggest that such studies can be used to improve the design of tele-operation systems.The need for social interaction between the tele-operator and service user suggests that visual and auditory feedback modalities will be engaged as tasks are carried out. This further reduces the number of available sensory modalities through which information can be communicated to tele-operators. A wrist-worn Wi-Fi enabled haptic feedback device was therefore developed and a study was carried out to investigate haptic sensitivities across the wrist. Results suggest that different locations on the wrist have varying sensitivities to haptic stimulation with and without video distraction, duration of haptic stimulation, and varying amplitudes of stimulation. This suggests that dynamic control of haptic feedback can be used to improve haptic perception across the wrist, and it may also be possible to display more than one type of sensor data to tele-operators during a task. The final study carried out was designed to investigate if participants can differentiate between different types of sensor data conveyed through different locations on the wrist via haptic feedback. The effect of increased number of attempts on performance was also investigated. Total task completion time decreased with task repetition. Participants with prior gaming and robot experience had a more significant reduction in total task completion time when compared to participants without prior gaming and robot experience. Reduction in task completion time was noticed for all stages of the task but participants with additional feedback had higher task completion time than participants without supplementary feedback. Reduction in task completion time varied for different stages of the task. Even though gripper trajectory reduced with task repetition, participants with supplementary feedback had longer gripper trajectories than participants without supplementary feedback, while participants with prior gaming experience had shorter gripper trajectories than participants without prior gaming experience. Perceived workload was also found to reduce with task repetition but perceived workload was higher for participants with feedback reported higher perceived workload than participants without feedback. However participants without feedback reported higher frustration than participants without feedback.Results show that the effect of feedback may not be significant where participants can get necessary information from video feedback. However, participants were fully dependent on feedback when video feedback could not provide requisite information needed.The findings presented in this thesis have potential applications in healthcare, and other applications of robot tele-operation and feedback. Findings can be used to improve feedback designs for tele-operation systems to ensure safe and efficient tele-operation. The thesis also provides ways visual feedback can be used with other feedback modalities. The haptic feedback designed in this research may also be used to provide situational awareness for the visually impaired
Proceedings of the 9th international conference on disability, virtual reality and associated technologies (ICDVRAT 2012)
The proceedings of the conferenc