Visual feedback plays a crucial role in the process of amputation patients
completing grasping in the field of prosthesis control. However, for blind and
visually impaired (BVI) amputees, the loss of both visual and grasping
abilities makes the "easy" reach-and-grasp task a feasible challenge. In this
paper, we propose a novel multi-sensory prosthesis system helping BVI amputees
with sensing, navigation and grasp operations. It combines modules of voice
interaction, environmental perception, grasp guidance, collaborative control,
and auditory/tactile feedback. In particular, the voice interaction module
receives user instructions and invokes other functional modules according to
the instructions. The environmental perception and grasp guidance module
obtains environmental information through computer vision, and feedbacks the
information to the user through auditory feedback modules (voice prompts and
spatial sound sources) and tactile feedback modules (vibration stimulation).
The prosthesis collaborative control module obtains the context information of
the grasp guidance process and completes the collaborative control of grasp
gestures and wrist angles of prosthesis in conjunction with the user's control
intention in order to achieve stable grasp of various objects. This paper
details a prototyping design (named viia-hand) and presents its preliminary
experimental verification on healthy subjects completing specific
reach-and-grasp tasks. Our results showed that, with the help of our new
design, the subjects were able to achieve a precise reach and reliable grasp of
the target objects in a relatively cluttered environment. Additionally, the
system is extremely user-friendly, as users can quickly adapt to it with
minimal training