Virtual environments are computer-generated synthetic environments with which a human user can interact to perform a wide variety of perceptual and motor tasks. At present, most of the virtual environment systems engage only the visual and auditory senses, and not the haptic sensorimotor system that conveys the sense of touch and feel of objects in the environment. Computer keyboards, mice, and trackballs constitute relatively simple haptic interfaces. Gloves and exoskeletons that track hand postures have more interaction capabilities and are available in the market. Although desktop and wearable force-reflecting devices have been built and implemented in research laboratories, the current capabilities of such devices are quite limited. To realize the full promise of virtual environments and teleoperation of remote systems, further developments of haptic interfaces are critical. In this paper, the status and research needs in human haptics, technology development and interactions between the two are described. In particular, the excellent performance characteristics of Phantom, a haptic interface recently developed at MIT, are highlighted. Realistic sensations of single point of contact interactions with objects of variable geometry (e.g., smooth, textured, polyhedral) and material properties (e.g., friction, impedance) in the context of a variety of tasks (e.g., needle biopsy, switch panels) achieved through this device are described and the associated issues in haptic rendering are discussed

Srinivasan, Mandayam A.

English

NASA Technical Reports Server

HAPTIC INTERFACES: HARDWARE, SOFTWARE
AND HUMAN PERFORMANCE l
Mandayam A. Srinivasan
Department of Mechanical Engineering
Massachusetts Institute of Technology
Cambridge, MA 02139
ABSTRACT
Virtual environments are computer-generated synthetic environments with which a human user
can interact to perform a wide variety of perceptual and motor tasks. At present, most of the virtual
environment systems engage only the visual and auditory senses, and not the haptic sensorimotor
system that conveys the sense of touch and feel of objects in the environment. Computer keyboards,
mice and trackballs constitute relatively simple haptic interfaces. Gloves and exoskeletons that track
hand postures have more interaction capabilities and are available in the market. Although desktop and
wearable force-reflecting devices have been built and implemented in research laboratories, the current
capabilities of such devices are quite limited. To realize the full promise of virtual environments and
teleoperation of remote systems, further developments of haptic interfaces are critical.
In this paper, the status and research needs in human haptics, technology development and
interactions between the two are described. In particular, the excellent performance characteristics of
Phantom, a haptic interface recently developed at MIT, are highlighted. Realistic sensations of single
point of contact interactions with objects of variable geometry (e.g., smooth, textured, polyhedral) and
material properties (e.g., friction, impedance) in the context of a variety of tasks (e.g., needle biopsy,
switch panels) achieved through this device are described and the associated issues in haptic rendering
are discussed.
IPart of the work reported here was supported by Naval Training Systems Center Contract No. N61339-94-C-0087.
104
https://ntrs.nasa.gov/search.jsp?R=19960026480 2020-06-16T03:56:23+00:00Z
APPLICATIONS OF SYNTHETIC ENVIRONMENTS (SE)
Synthetic environments (SE), which include both virtual environments (VE) and teleoperation,
have generated considerable excitement, owing to the wide variety of applications in which they can
play a significant role (listed below; Ref. 1). At present, most of the VE systems engage only the
visual and auditory senses, and not the haptic sensorimotor system that conveys the sense of touch and
feel of objects in the environment. Manual interactions with SE are important in sensorimotor tasks
such as training of surgeons with VE or conveying the feel of an object to the participants of a
teleconference. They may also play a significant role in cognitive tasks such as memorization and
analysis of multidimensional databases or teaching the implications of the violations of physical laws.
• Design, Manufacturing, and Marketing
• Medicine and Health Care
• Teleoperation for Hazardous Operations
• Training
• Education
• Entertainment
• Information Visualization
• Telecommunications
Manual interaction with SE is important in
particular tasks within each application area.
Figure 1
105
HAPTICS: MANUAL INTERACTIONS WITH THE ENVIRONMENT
- FOR EXPLORATION OR MANIPULATION
The term Haptics refers to manual interactions with real or virtual environments. It includes
both exploration of objects to obtain information about the environment and manipulation of objects to
alter the environment. In contrast to the purely sensory nature of vision and audition, the human haptic
system involves tight integration of both sensory and motor components. Further, the sensory
information can be divided into two classes: (1) tactile information, referring to the sense of contact
with the object, mediated by the responses of low-threshold mechanoreceptors innervating the skin
(say, the finger pad) within and around the contact region; and (2) kinesthetic information, referring to
the sense of position and motion of limbs along with the associated forces, conveyed by the sensory
receptors in the skin around the joints, joint capsules, tendons, and muscles, together with neural
signals derived from motor commands.
Haptics: Manual interactions with the environment
- for exploration or manipulation
(e.g., Hands)
Sensory
Motor
Tactile
Kinesthetic
(proprioceptive)
Figure 2
106
HAPTIC INTERFACES
Haptic interfaces are devices that enable manual interaction with virtual environments or
teleoperated remote systems. They are employed for tasks that are usually performed using hands in
the real world, such as manual exploration and manipulation of objects. In general, they receive motor
action commands from the human user and display appropriate tactual images to the human. The
command and display variables are listed below.
I-htxic Itmfmzs
human operator
COMMAND Haptic
-,, Interface
DISPLAY Hardware
Computer
(nte a ons)
_rn_on _"
flr
(2xl_ f_ce
,_Ccnt_ force
or
=.'_tre/nrtion
Fv_ distrit_ion
(,_lial-ten]x_ml)
Figure 3
107
CLASSIFICATION OF HAPTIC INTERFACES
Haptic interfaces can be classified in several ways. First, classification is based on whether
they are force-reflecting or not, as well as by what types of motions (e.g., how many degrees of
freedom) and contact forces they are capable of simulating. The second type of classification is based
on whether they are simulating the touch, feel, and manipulation of objects directly in contact with the
skin or through a tool. A third set of important distinctions are based on whether the force display
systems are ground-based, such as joysticks and other hand controllers, or body-based, such as gloves
and exoskeletons.
1. Based on motions and/or forces (e.g.,
presence or absence of force reflection,
degrees of freedom, types of forces)
2. Ideal exoskeleton or tool handle approach
3. Ground-based or body-based
Figure 4
108
AVAILABLE HAPTIC INTERFACES
A variety of haptic interfaces are currently available. Computer keyboards, mice and trackballs
constitute relatively simple haptic interfaces. Gloves and exoskeletons that track hand postures have
more interaction capabilities and are available in the market. Although desktop and wearable force-
reflecting devices have been built and implemented in research laboratories, the current capabilities of
such devices are quite limited. There exist a number of examples of tactile stimulators for the finger,
including pneumatic shape changers, electrocutaneous stimulators, and vibrating arrays, but none
provides convincing tactile images and all are awkward to use (Ref. 2).
• Position sensors
• Joysticks
• Point-interaction robotic devices
• Teleoperator masters
• Exoskeletal devices:
- flexible (gloves and suit worn by user)
- rigid links (joined linkages affixed to user)
• Tactile displays:
- shape changers (shape memory actuators,
pneumatic actuators, micro-mechanical
actuators)
- vibrotactile
- electrotactile
Figure 5
109
SOFTWARE FOR HAPTIC INTERACTIONS
Similar to the software needed to generate visual images, the software necessary to generate
tactual images can be classified into three major groups: haptic interaction software, simulation of
object behavior, and software for rendering tactual images. Haptic interaction software mainly consists
of reading the state of the haptic interface device. Simulation of object behavior requires physical
models of virtual objects. This can be accomplished either by a unified model for all the modalities
(e.g., visual, haptic, acoustic) or through separate models for each modality, together with correlation
algorithms for consistency among the displays corresponding to each of the modalities. The software
for rendering the tactual images receives the output of the physical model and generates the commands
needed to drive and control the interface device.
Software for Haptic Interactions
Device State - Reading and interpreting the state of
the haptic devices.
Unified model of all modalities
Simulation j
_ Modality-specific models
Rendering - Control of haptic display
Figure 6
iio
THE PHANTOM
The Phantom is a force-reflecting haptic interface recently developed at MIT (Ref. 3). It is
capable of generating realistic sensations of single point contact interactions. It is essentially a robot
with six degrees of freedom, capable of generating a three-dimensional force vector at its end effector,
which can either be a thimble or a stylus that is manipulated by the human user.
Figure 7
Iii
SLIDER SWITCHES
A variety of touch interactions have been haptically rendered using the Phantom (Ref. 4).
Shown below are three types of slider switches and their respective force-displacement behavior.
addition to the properties of mass, viscosity, stiction, and surface stiffness, each cube has an
underlying characteristic spring function.
In
Slider Switches
0
r
C
e
D_p_L
/
J
I
Free
°
ring
Cen
0
r
e
• •
Dl_lavem_at Displacement
In addition to the properties of mass. viscosity,
suction, and surface stiffness, each cube has an
underlying characteristic spring function.
Figure 8
112
BUTTONS WHICH "CLICK"
The feel of push buttons which "click" has also been displayed through the Phantom to the
human user. Each of the buttons shown below simulate the force-displacement relationship shown
schematically, but feel distinct owing to differences in the values of parameters such as stiffness.
Buttons which "Click"
F
0
r
C
e
Displacement
>
Figure 9
113
RENDER: RENDERS POLYGONAL SURFACES
The haptic display of arbitrarily shaped objects represented as polyhedra has been achieved with
the Phantom. The rendering software uses standard graphics file formats and is capable of simulating
both convex and concave surfaces.
• Allows haptic display of arbitrarily shaped
objects
• Uses standard graphics file formats
• Convex and concave
• Can render arbitrarily thin objects
Figure 10
114
Two Phantoms have been used together to allow a user to perform two-fingered manipulation
of virtual objects as shown below. Such contact interactions are analogous to the use of tools in real
environments.
i k
t t
i
x
,i x
x
ThePhantom
allowsusers
tO 'touch'
virtual
ob/ects.
Figure 11
115
BLOCKS: DYNAMIC SIMULATION OF BLOCKS IN 3D
Using two Phantoms, the dynamic simulation of 3D blocks being manipulated by a user has
been achieved. Both visual and haptic displays were provided to the user.
• Two hand/finger manipulation
• Static friction model
Phantom and walls, Phantom and blocks,
blocks and blocks
• Gravity
Figure 12
116
WHAT ARE ELEMENTS OF HAPTIC INTERACTION?
Listed below is a summary of the elements of haptic interactions.
• Sensed elements:
Motion, force, tactile, temperature,
heat flow, current flow, pain, etc.
• Perceived events and states:
Impact
Sustained contact
Slip
Friction, texture
Freedom/constraint in motion
Compliance
Curvature
• Workless interactions:
Imposing and detecting constraint
• Work interactions:
Force over distance, impulse,
momentum and energy
Figure 13
117
WHAT WE HAVE DONE
The status of the work done at the MIT AI Laboratory is given below:
• Developed class of haptic interface
permitting force vector display - Phantom
• Demonstrated basic interaction elements
impact and constraint forces
object shape, motion
friction, texture
surface and object, impedance
• Combined basic elements to build simple
mechanical worlds:
astroid, blocks,
needle-biopsy, switch panels
• Developed rendering algorithms for
polyhedral objects
• Begun development of rendering algorithms
for visco-elastic materials
Figure 14
118
HUMAN HAPTIC PERFORMANCE
A basic understanding of the biomechanical, sensorimotor, and cognitive abilities of the human
haptic system is essential for further improvements in the design of hardware and software of haptic
interfaces. Summarized below are some of the quantitative data (Refs. 2 and 5) on human haptic
performance which set some of the design specifications of interface devices.
Tactile Sensory System:
Vibrations - detectable up to l khz
thresholds - 0.3 to 30 microns
Spatial resolution - l mm at fingerpad
Kinesthetic Sensory System:
Just noticeable differences (JND) for joint
angles ~ 1 to 3 degrees
Motor System:
Bandwidths: 1 to 10 Hz
Active Touch with All Three Systems
JNDs for two-fingered pinch grasp:
Length ~ 10% or less
Force ~ 7%
Compliance ~ 8%
Viscosity ~ 14%
Mass ~ 21%
Figure 15
119
RESEARCH NEEDS
A comprehensive program to develop a variety of haptic interfaces for virtual environments and
teleoperation needs to include research in the areas shown below. Since progress in the three areas is
interdependent, the desirable course of development for a challenging application is to continually build
improved versions of haptic devices based on experimental data obtained from the previous versions on
the performance of humans, devices and the interaction between the two.
Human Haptics
Biomechanics
Psychophysics
Hardware
Position trackers
Force displays
Tactile displays
Software
Multimodal interactions
Real time simulations
• Matching Human and Device
Comfort
Simulation methods
Evaluation
Performance
Figure 16
120
REFERENCES
.
.
*
.
.
Durlach, N. I. and Mavor, A. S. (eds.), Virtual Reality: Scientific and Technical Challenges,
National Academy Press, 1994.
Srinivasan, M. A., "Haptic Interfaces," in Virtual Reality: Scientific and Technical Challenges,
Durlach, N. I. and Mavor, A. S. (eds.), National Academy Press, 1994.
Massie, T. H. and Salisbury, J. K., "The Phantom Haptic Interface: A Device for Probing
Virtual Objects," in Dynamic Systems and Control, C. J. Radcliffe (ed.), DSC Vol. 55-1,
ASME, NY, 1994, pp. 295-299.
Salisbury, J. K., Brock, D., Massie, T., Swarup, N. and Zilles, C., "Haptic Rendering:
Programming Touch Interaction with Virtual Objects," Proc. ACM Symposium on Interactive
Three-Dimensional Graphics, 1995.
Tan, H. Z., Srinivasan, M. A., Eberman, B. and Cheng, B., "Human Factors for the Design of
Force-Reflecting Haptic Interfaces," in Dynamic Systems and Control, C. J. Radcliffe (ed.),
DSC Vol. 55-1, ASME, NY, 1994, pp. 353-359.
121



Haptic interfaces: Hardware, software and human performance

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server