This paper describes our experience with the design of a remote 
pathologists workstation.  We illustrate how our effort to apply 
direct manipulation principles led us to explore remote direct 
manipulation designs.  The use of computer and communication 
systems to operate devices remotely introduces new challenges for 
users and designers.  In addition to the usual concerns, the activation 
delays, reduced feedback, and increased potential for breakdowns 
mean that designers must be especially careful and creative.  The 
user interface design is closely linked to the total system design.  
(Also cross-referenced as CAR-TR-551

Keil-Slawik, R.

Plaisant, Catherine

Shneiderman, Ben

Digital Repository at the University of Maryland

1Remote direct manipulation:A case study of a telemedicine workstation1R. Keil-Slawik+, C. Plaisant and B. Shneiderman*Human-Computer Interaction Laboratory, Center for Automation ResearchUniversity of Maryland, College Park, MD 20742+  On visit from Technical University of Berlin.*   Dept. of Computer Science and UMIACSKeywords:  user interface, direct manipulation, telemedicine, remote controlAbstractThis paper describes our experience with the design of a remote pathologist‘s workstation.We illustrate how our effort to apply direct manipulation principles led us to explore remotedirect manipulation designs.  The use of computer and communication systems to operatedevices remotely introduces new challenges for users and designers.  In addition to the usualconcerns, the activation delays, reduced feedback, and increased potential for breakdowns meanthat designers must be especially careful and creative.  The user interface design is closelylinked to the total system design.1.  INTRODUCTIONDirect manipulation has been described as a visual representation of the world of actionwith rapid, incremental and reversible actions (Shneiderman 1983).  The objects and actions ofinterest are shown continuously, users generally point, click, or drag rather than type, andfeedback, indicating change, is immediate.  However, when the devices being operated areremote, these goals may not be realizable and designers must spend additional effort to copewith slower response, incomplete feedback, increased likelihood of breakdowns, and errorrecovery.  The problems are strongly connected to the hardware, physical environment,network design, and the task domain.We studied these problems in the context of a remotely controlled microscope system usedby pathologists to make diagnoses based on seeing microscope slides of tissues, blood, or otherspecimens.  Our task was to redesign an existing system (developed by Corabi InternationalTelemetrics, Inc.) to enhance its usability and provide for future extensions.  This paperpresents our solutions to some of the problems and discusses the extension of directmanipulation principles to an environment that includes remote control.  We describe theCorabi Project and show examples of user interface design issues and then outline principles ofremote direct manipulation.2.  THE CORABI TELEPATHOLOGY WORKSTATION1  To appear In: Ed. Bullinger H.-J., Proc. of the 4th Int. Conf. on HCI, Stuttgart (Sept. 91).Amsterdam, Elsevier.2Telemedicine is the practice of medicine over communication links.  The physician beingconsulted and the patient are in two different locations.  Corabi International Telemetricsdeveloped the first telepathology system (Weinstein, Bloom and Rozek, 1987 and 1989) thatallows a pathologist to render a diagnosis by examining tissue samples or body fluids under aremotely located microscope.  The transmitting workstation consists of a high resolutioncamera mounted on a motorized light microscope.  The image from the camera is transmittedvia broadband satellite, microwave or cable.  The consulting pathologist sits at the receivingworkstation where she/he can manipulate the microscope using a keypad and look at the highresolution image of the magnified sample.  Both physicians talk by telephone to coordinatecontrol of the system and to request slides that have to be manually placed under themicroscope.ControlScreenHigh ResScreenKeypadModem connectionSatellite connectionMicroscopeCameraFigure 1:  Simplified diagram of a telepathology system.The system also allows the pathologist to store the results, recall the case at a later time, askfor second advice and manage the patient’s records.  During a work session the pathologistalternates between selecting cases to work on and performing a diagnosis.  To conduct thediagnosis the pathologist goes back and forth between reading the patient record, choosing theslide to be viewed and entering the diagnosis.Practically, the pathologist sees a high resolution screen displaying the analog image fromthe microscope, and a control screen (a PC display).  The control screen only displaysalphanumeric data and is used for database management tasks on the patient records, as well asto establish connections with the remote site and display status data during the connection.  Inthe original system a third monochrome screen was used to display a small scanned globalimage of the whole specimen.  To control the microscope the pathologist uses a keypad (witharrows keys and function keys) as well as a large number of buttons and toggles mounted onthe rack holding the circuitry.  The microscope controls include:3- magnification (three or six objectives),- focus (coarse and fine bidirectional control),- illumination (bidirectional adjustment continuous or by step), and- position (2-dimensional placement of the slide under the microscope objective).Our overall task was to redesign the database access, navigation among the tasks, andremote control of the microscope during the diagnosis.  According to the principles of directmanipulation, our first step was to group related displays and controls that were originallydissociated such that all alphanumeric displays (all displays except the high resolution one) andcontrols are found on the control screen and can be manipulated with a pointing device.  Thecontrol screen becomes the central part of the user interface.3.  TYPICAL PROBLEMS OF REMOTE DIRECT MANIPULATIONThe architecture of a remote environment such as described above introduces severalcomplicating factors that rarely occur in direct manipulation environments:Time delays: The network hardware and software cause delays in sending user actions andreceiving feedback: transmission delays, i.e., the time it takes for the command to reach themicroscope (in our case, transmitting the command through the modem), and  operationdelays, i.e., the microscope itself does not respond right away.  These delays in the systemprevent the operator from knowing the current status of the system.  For example, if apositioning command has been issued it may require several seconds for the slide to startmoving.  As the feedback appears showing the motion, the users may recognize that they aregoing to overshoot their destination, but it will also take a few seconds to have the stoppingcommand take effect.Incomplete feedback: Devices originally designed for direct control may not have adequatesensors or status indicators.  For instance, our microscope can report its current position but itis so slow to provide it that it cannot be used continuously.  Thus, it is not possible to indicateon the control screen the exact current position relative to the start and desired positions.Feedback from multiple sources: Incomplete feedback does not imply that there is no feedbackat all.  The image received on the high resolution screen is the main feedback to evaluate theresult of an action.  In addition, the microscope can occasionally report its exact positionallowing recalibration of the status display.  It is also possible to indicate the estimated stageposition during the execution of a movement.  This estimated feedback can be used as aprogress indicator whose accuracy depends on the variability of the time delays.  To complywith the physical incompatibility between the high resolution feedback (analog image) and therest of the system (digital) the multiple feedbacks are spread over several screens.  Thus, thepathologists are forced to switch back and forth between multiple sources of feedback,increasing their cognitive load.Unanticipated interferences: Since the devices operated are remote, and may be also operated byother persons in this or another remote location, unanticipated interferences are more likely tooccur than in traditional direct manipulation environments.  For instance, the slide under themicroscope may be moved (accidentally) by a local operator.  As a result, the positionsindicated may not be correct.  A breakdown may also occur during the execution of a remoteoperation, without indicating this event properly to the remote site.  Such break-downs requireadditional information and actions that allow for the cancellation of actions to prevent their4completion.Our proposed solution to these problems is to make explicit the network delays as part ofthe system, without compromising the overall system usability.  The user needs to see a modelof the:- starting state of the system,- action that has been issued, and- current state of the system as it carries out the action.In addition we believe that it is preferable to provide spatially parametrized positioning actions(i.e.  move of a distance +x,+y or to a fixed point (x,y) in a two dimensional space), rather thanproviding temporal commands (i.e.  start moving right at a 36° angle from the horizontal).  Inother words, the users specify a destination (rather than a motion) and wait until the action iscompleted before readjusting the destination if necessary.  In general, we try to turn the remoteenvironment as much as possible into a direct manipulation environment by applying the samebasic principles.However there may be obstacles to implementing these principles.  We will highlight theproblems involved by discussing the redesign of the slide position control, because it illustratesthe additional challenges of dealing with remotely operated devices.4.  SLIDE MOVEMENT CONTROL - A DESIGN EXAMPLETo provide a visual representation of the world of action it is certainly helpful to present aglobal view of the slide to the pathologist on the control screen.  Since the specimen itself onlyoccupies a small part of the slide, the global view lets the pathologist know what parts of theslide need to be observed and if the specimen under the slide is made of one or several separateparts.  A red rectangle shown on the slide indicates the position of the microscope objective andtells what portion of the slide (the stage) is being viewed on the high resolution screen.Markers can be also placed on the map to indicate points saved.  The rectangle can be selectedwith a pointing device and used to move the stage to another position on the slide.  Similarly asaved point can be selected to be retrieved.  These actions are handled with direct manipulationprinciples, making them easy to learn and to remember.  Unfortunately,  several problemsblocked the extension of these principles to the control of all movements of the slide.5Figure 2: the control screen, showing the global view of the slideand the stage position mark.The first problem is that the microscope movements require very precise and smoothmoves when using a high magnification.  This makes it  very difficult to use directmanipulation of the rectangle (then very small) on the global slide view.  Of course weenvisioned using zoomed images of the global view of the slide.  Despite the fact that thistechnique would require the user to zoom and pan the global view before actually controllingthe slide movement, we were faced with the problem that there was no practical way to obtain auseful zoomed image.  The microscope was too slow to scan a full slide, it could not guaranteea precise and consistent placement of the slide to have the scanning done in advance by atechnician, and the stored image was space expensive.What we could provide, however, was the fine control of the slide relative to its currentposition (i.e.  specify a direction and distance) rather than in an absolute manner (i.e.  specify aposition).  The global view of the slide is used only to give feedback about the position of thestage.  Thus, it can not be used as an object that can be manipulated (moved) directly withrespect to all levels of magnification.  But the spatial representation provides more control forthe pathologists because it indicates where the part visible on the high resolution screen islocated with respect to the whole specimen.  Since the saved points are displayed as well, this6may provide additional, implicit feedback, for instance, about regions that have been scannedalready.  This information is only implicitly given because it can only be derived by connectingthe displayed data with the specific activities performed by the pathologists, e.g., when theychoose a systematic strategy for scanning a  slide.  The example shows that the device characteristics were of paramount importance in the userinterface design.  This is a very common problem: Remotely controlled devices often fail toprovide any usable feedback at all.  For example, current home automation user interfaces areconstrained by the fact that home devices do not return status information to the central control[Plaisant, Shneiderman and Battaglia, 1990].  As a consequence, the conceptual design of theuser interface can not be done without sufficient and often detailed knowledge about the specificdevices and architecture of the overall system.5.  FROM DIRECT MANIPULATION TO REMOTE DIRECT MANIPULATIONThe concept of remote direct manipulation can be rooted in two different domains which, sofar,  have been treated independently.  Direct manipulation originated in the context of personalcomputers and is often identified with the desktop metaphor and office automation.  The otherroot is in process control where human operators control physical processes in complexenvironments.  Typical tasks are operating power plants, flying airplanes, or steering vehicles.If the physical processes take place in a remote location, we talk about teleoperation or remotecontrol.2  To perform the control task, the human operator may interact with a computer whichmay carry out some of the control tasks without any interference by the human operator.  Thisis captured by the  notion of supervisory control  (Sheridan, 1988).  Although supervisorycontrol and direct manipulation stem from different problem domains and are usually applied todifferent system architectures, there is a strong resemblance.Traditional direct manipulation can also be interpreted as a teleoperation, especially withhigh-speed networking and multi-tasking  environments.  Files that appear on a screen maycome from  a remote PC and the software may be distributed throughout the network.Messages and documents can be sent to or retrieved from remote machines, printers, or fileservers.  Even the letters on a display may be composed of font descriptions stored in a remotelocation from where the keystrokes are issued.  Thus, the essential components of ateleoperation environment such as sensors, displays, controls, remote effectors or tools, andcommunication links are involved.Remote direct manipulation (as well as supervisory control) can not be taken as a designcriterion which is either fulfilled or not.    One interface can be slightly more direct than another.Similarly, the control can be felt to be more or less remote.  Thus, remote direct manipulationdenotes a range of possible solutions rather than a binary variable.  Direct manipulation is stillan imprecise and subjective concept3, although it has proved eminently useful in stimulatingdesigners, revising existing systems, training designers, and in comparing systems.  Theconnection between direct manipulation and supervisory control seems promising.6.  CONCLUSIONSWe believe that there are great opportunities for the remote control of devices if proper remotedirect manipulation interfaces can be constructed.  The notion of user control seems to play a2  These notions are often used as synonyms.  There are, however, more subtle distinctions,for instance,  between remote control and remote manipulation.3  Cf.  the summary in Hutchins, Hollan, and Norman (1986)7key role. It requires designers to provide adequate feedback in sufficient time to permit effectivedecision making and operation.  A thorough task analysis as well as detailed knowledge aboutthe technical environment are indispensable means to come up with creative solutions that putthe user into control.  The designers have to understand the system architecture, its strength andweaknesses and  the users’ needs to achieve a good conceptual design.In domains such as office automation and process control, as well as in many others, thedesign of human-computer interfaces and the development of general models of humancomputer interaction rather than the improvement of devices are regarded as the majorchallenge for researchers.4  However, devices are not yet sufficiently well designed to allow fortheir smooth integration in a remote environment according to the principles of remote directmanipulation:- shorten time delays,- provide extensive feedback of status,- coordinate available feedback, and- reduce possible interferences.The development of these new integrated and remotely controlled environments alsoprovides a stimulus for new applications.  Remote controlled environments in medicine couldenable specialists to provide consultations more rapidly.  Home automation applications arebeing developed to allow more than remote operation of telephone answering machines byincluding security and access systems, energy control, and operation of appliances.  Scientificapplications in space, underwater, or in hostile environments can enable new research projectsto be conducted economically and safely.Acknowledgements:  We appreciate the support of the Maryland Industrial Partnershipsprogram and Corabi International Telemetrics, Inc. in providing partial funding for thisresearch.  We would like to thank Beth Newberger, Tom Throop, and Ann Regan-Jean fromCorabi International Telemetrics, Inc. for their helpful suggestions.REFERENCESHutchins, E.L., Hollan, J.D., Norman, D.A.: Direct Manipulation Interfaces.  In: Norman,D.A., Draper, S.W.  (Eds.): User Centered System Design.  New Perspectives on Human-Computer Interaction.  Hillsdale: Lawrence Earlbaum (1986) 87-124.Plaisant, C., Shneiderman, B., Battaglia, J.: Scheduling home-control devices: A case study ofthe transition from  the research project to a product.  Human-Factors in Practice.  (Dec.1990) 7-12.  Santa-Monica, CA:  Computer Systems Technical Group, Human-FactorSociety.Rasmussen, J., Goodstein, L.P.: Information Technology and Work.  In: Helander, M., (Ed.):Handbook of Human-Computer Interaction.  Amsterdam: North-Holland (1988) 175-201Sheridan, T.B.: Teleoperation, Telepresence, and Telerobotics: Research Needs for Space.  In:Sheridan, T.B., Kruser, D.S., Deutsch, S.  (Eds.): Human Factors in Automated and RoboticSpace Systems.  Proceedings, National Research Council, Washington, DC, (1987)  279-291.Sheridan, T.B.  (1988).  Task Allocation and Supervisory Control.  In: Helander, M., (Ed.):Handbook of Human-Computer Interaction.  Amsterdam: North-Holland (1988) 159-173Shneiderman, B., Direct Manipulation: A Step Beyond Programming Languages, IEEEComputer 16  8 (1983) 57-69.Uttal, W.R.: Teleoperators.  Scientific American, Vol.  261, No.  6 (Dec.  1989) 124-129.4   Cf.  Rasmussen, Goodstein (1988),  Sheridan (1987), Uttal (1989), and Van de Vegte,Milgram, Kwong (1990)8Van de Vegte, J.M.E., Milgram, P., Kwong, R.H.: Teleoperator Control Models: Effects ofTime Delay and Imperfect System Knowledge.  IEEE Transactions on Systems, Man, andCybernetics, 20  6 (Nov./Dec.  1990) 1258-1272.Weinstein, R.,Bloom, K., Rozek, S., Telepathology and the Networking of PathologyDiagnostic Services.  Archive of  Pathology and Laboratory Medicine, 111 (1987) 646-652Weinstein, R., Bloom, K., Rozek, S., Telepathology.  Long Distance Diagnosis.  AmericanJournal of Clinical Pathology, 91 (Suppl 1) (1989) S39-S42 .

Remote direct manipulation:A case study of a 
telemedicine workstation

https://drum.lib.umd.edu/bitstream/1903/369/2/CS-TR-2655.pdf

Remote direct manipulation:A case study of a telemedicine workstation

Abstract

Similar works

Full text

Available Versions

Digital Repository at the University of Maryland