Hornbæk, Kasper

Pedersen, Esben Warming

English

Copenhagen University Research Information System

u n i ve r s i t y  o f  co pe n h ag e n  Københavns UniversitetRoMoPedersen, Esben Warming; Hornbæk, KasperPublished in:Proceedings of the 11th Danish Human-Computer Interaction Research SymposiumPublication date:2011Document versionPeer reviewed versionCitation for published version (APA):Pedersen, E. W., & Hornbæk, K. (2011). RoMo: avoiding conflicts between the physical and digital model intabletop interfaces with robotic tangibles. In M. Bødker, A. Nawaz, & G. S. Petersen (Eds.), Proceedings of the11th Danish Human-Computer Interaction Research Symposium: (DHRS2011) (pp. 18-21). CopenhagenBusiness School Press.Download date: 02. Feb. 2020RoMo: Avoiding Conflicts Between the Physical andDigital Model in Tabletop Interfaces with Robotic TangiblesEsben Warming Pedersen & Kasper HornbækDepartment of Computer Science, University of CopenhagenDK-2300 Copenhagen S, Denmark{esbenwp, kash}@diku.dkABSTRACTIn TUIs, physical/digital conflicts can occur when the digitalmodel does not match the model implied by the spatial lay-out of tangibles. We show how tangible tabletop interfaces(TTI) can be modified to allow robot movement of tangi-bles, thereby avoiding conflicts. We present RoMo, an opensource Java library that allow existing TTI applications toperform robot movement, and demonstrate its functionalitywith three applications.Author KeywordsTangible user interfaces, bidirectional user interfaces, robotictangiblesACM Classification KeywordsH.5.2. User Interfaces. D.2.2. Software libraries.INTRODUCTIONIn a landmark paper in the field of tangible user inter-faces (TUIs), Ishii and Ullmer described the risk of phys-ical/digital conflicts [3]. Physical/digital conflicts emergewhen the digital model does not match the model impliedby the spatial layout of tangibles. Conflicts can occur if thedigital model is dynamically changing (i.e., if it depends onother data-sources than user input) or if multiple users arecollaborating on TUIs in different places sharing the samedigital model. Physical/digital conflicts destroy the percep-tion of input/output unification and may lead to ambiguousinterpretations of the tangibles’ spatial layout.To address the problem of physical/digital conflicts in thefield of tabletop tangible interfaces (TTIs), researchers havedeveloped bidirectional tangible interfaces that are capableof moving tangibles around the tabletop surface. In con-trast to unidirectional tangible interfaces, this capability al-lows bidirectional user interfaces to use tangibles as out-put devices and display changes in the digital model phys-ically. The bidirectional tangible interfaces may be dividedSubmitted to DHRS2011Figure 1. A map application using RoMo to move threerobotic tangibles.into two groups: (a) interfaces with small unpowered pucks,moved by an electromagnetic array situated under the table-top surface [1, 5, 6], and (b) interfaces with battery-poweredrobots that move on the tabletop surface [7–10,12]. Most no-table is PICO [6], an interface that uses magnetic attractionand repulsion as haptic user guidance, thereby allowing thecomputer and the user to collaborate in solving optimizationtasks.Whereas existing bidirectional tangible interfaces in partpreclude physical/digital conflicts, they introduce other prob-lems. The systems that employ electromagnetic arrays usecustom-made hardware to move the pucks, and buildingthem requires advanced engineering skills. The number ofmagnets and control boards required to move the pucks growwith the size of the tabletop, making these interfaces veryexpensive. The systems with battery-powered robots requirefar less hardware and are easier to build. However, existingsystems rely on custom-made tracking and moving softwarewhich makes it impossible for other systems to adopt theirrobot control technologies.The present paper makes two contributions to the field oftangible tabletop interfaces. First, we describe the hardwaresetup (Figure 1) and the steps needed to convert a unidirec-tional TTI to a bidirectional TTI using only off-the-shelfcomponents. Second, we present RoMo, an open sourceJava library that allows existing applications to performrobot movement. We describe the design behind RoMo anddemonstrate the functionality of the library with three appli-cations that use robot movement to correct physical/digitalconflicts. The robots and the RoMo library have previously1Robots (with ducials and XBee modules)projectorcamerastreamreacTIVisionTTIApplicationTUIOvideoIR pass lterIR LEDsXBee ModuleRoMoLibrarycommandsmovementinstructions  cameramovementacknowledgementTUIOFigure 2. Overview of the system components.been employed in [7] to study interaction with robotic tangi-bles.RoMo HARDWARE DESCRIPTIONIn this section we describe the hardware needed to use theRoMo library.TabletopFigure 2 shows an overview of the components used in ourtabletop setup. The setup is similar to that of unidirectionalinterfaces: Tangibles are marked with fiducials, which areilluminated by infrared light and tracked. The tracking in-formation is sent as TUIO messages [4] to the TTI applica-tion where it is interpreted according to the interaction de-sign of the application and where graphical visual feedbackis generated. We use a DSI table [11] with reacTIVision [4]as tracking software, but any computer-vision based TUIOcompatible setup will work.The only hardware change made to the tabletop is the ad-dition of a XBee Series 2.5 module to the hardware setup.The XBee module is mounted in an XBee Explorer USB-to-serial unit and is running as Communicator in API mode.This allows RoMo to communicate wirelessly with therobots at a speed of 115.2 kbps.RobotsWe use 3pi robots from Pololu 1 as tangibles (Figure 3). Witha diameter of 9.5 cm and a weight of 83 g., they can easily bemoved with only one hand. Moreover, the 3pi robots moveand turn very quickly (100 cm/second). They are equippedwith an Atmel ATmega328P micro controller running at 20MHz and are programmable in C, Processing and Arduino.Each robot has a Lilypad XBee with a XBee series 2.5 mod-ule soldered on to the serial communication pins of the robotso as to allow wireless communication. The table in Figure 3shows how the robot and the module are connected. A fidu-cial marker is attached to the bottom of the robot to make itvisible to the tracking software. The fiducial marker must beoriented as shown in Figure 3.1http://www.pololu.com/catalog/product/975Figure 3. A Pololu 3pi robot with a XBee modulemounted in a Lilypad XBee. The table shows how themodule is connected to the robot. A and B indicate howthe fiducial should be oriented on the bottom of the robot.MOTION IN RoMoIn RoMo robots are tracked with computer-vision and com-munication happens wirelessly. This introduces some chal-lenges when controlling robots. In this section we explainhow motion is achieved in RoMo.Control and communicationTracking and movement computations are performed solelyby the control computer; the robots hold no positional in-formation. The computer continously tracks the orientationand speed of each robot, determines the proper robot actionand communicates this action to the robots in a round-robinfashion.The steering of the robots is performed with robot move-ment frames. Each movement frame contains three values:(1) speed value for right motor, (2) speed value for left mo-tor, and (3) movement time in milliseconds. The movementtime helps avoid unwanted autonomous movement in caseof package loss or package delay.RotationRotation is achieved by sending a pair of negated speed val-ues to the robot. The value and the movement time are calcu-lated from the angular distance to the destination angle; thebigger the difference, the higher the values. The maximumrotation speed is 220 deg/s.MovementRobot movement is done by driving in a straight line fromthe starting to the finishing position. In theory, this couldbe achieved by rotating the robot to the correct starting an-gle and driving straight until the finishing position has beenreached. In practice, many challenges have to be addressedto make robot movement work. The two motors on therobots are not perfectly matched and at even motor speedsthe robots pull to one side. Also, because of the kinetic en-ergy stored in the motors, changes in motor speed are notimmediately observable. Moreover, the latency caused bythe tracking software and the wireless communication result2Start FinishCorrectional movement New movementSpeedDeviation thresholdFigure 4. The robot’s speed and movement direction iscontinuously adjusted to keep the robot on the path. Ifthe robot exceeds the deviation threshold a new path iscalculated and a new movement is initiated.in a slight inconsistency between the actual robot positionand the tracked robot position. Hardware challenges like theabove mentioned are not specific to RoMo or bidirectionaltangible interfaces in general; they are commonly discussedin the field of robot research, e.g. [2]. The robot controlalgorithm takes all these factors into account by performingconstant tracking and adjustment of robot’s speed and move-ment direction. In the following we describe our algorithm.The robot controller initially calculates the equation for astraight line between the robot’s starting position and its des-tination. The robot controller then rotates the robot to a po-sition in which it is facing the destination and starts movingthe robot at even left/right speed values. For each new posi-tion sample the robot controller receives, the deviation fromthe path is calculated. The deviation value is determined bycalculating the minimum distance between the robot and theline. Small deviations are corrected by adjusting the motorspeeds without stopping the robot. However, if the robot ex-ceeds the deviation threshold, it is stopped, then a new pathis calculated, and a new movement initiated. This results ina moving pattern similar to the one displayed in Figure 4.The deviation threshold and the maximum robot speed canbe adjusted to achieve faster or more precise movement. Asshown in Figure 4, the speed of the robot is adjusted depend-ing on the distance to the finishing point. To ensure stabletracking and control, the robots move at a maximum speedof 24.5 cm/s.THE RoMo LIBRARYThe RoMo java library consist of a small number of methodsthat TTI applications can use to initiate robot movement. Inthis section we cover the most important ones. RoMo is im-plemented to fit the observer design pattern commonly usedin Java. The TTI application simply instructs RoMo to movea robot to a specific position or angle; when the desired des-tination has been reached, the TTI application is notified.RotationOne advantage of using robots in TTIs, compared to electro-magnetic TTIs, is the ability to rotate tangibles. In RoMorobots can be rotated with the method rotateRobot(intfID, float angleDegree), where fID is the fiducial id ofthe robot to be rotated and angleDegree specifies the an-Figure 5. A user rotates a tangible to adjust the percentvalue. Physical/digital conflicts are avoided by rotatingthe other tangible, thereby preventing the total from ex-ceeding 100%gle (0 ≤ angleDegree < 360) to which the robot shouldbe rotated. With angleDegree set to 0, the robot is alignedparallel to the vertical edges of the screen facing the bottomedge.Figure 5 shows an application that makes use of robot rota-tion to avoid physical/digital conflicts. Here, each robot isassociated with a percent value displayed above the robots.The user adjusts the values by rotating the tangibles, usingthem as knobs. When the user rotates a robot to adjust thevalue, the other robot immediately rotates to the correspond-ing percent value, thereby ensuring a total percent value of100%.MovementRobot can be moved to a new position by using the methodmoveRobot(int fID, float x, float y), where x and yspecify the coordinates to which the robot should be moved.x and y are expressed as percentage of the width and heightof the screen (0 ≤ x, y ≤ 1). Figure 6 shows a simple ap-plication that uses robot movement to introduce constraintson the spatial layout of tangibles. In the application the usermoves tangibles freely around the surface, and when the userstops interacting, the application forms an equilateral trian-gle by moving one tangible. The robot’s movement path isdisplayed as a yellow dotted line.The interaction design of this simple application illustratesan important difference between designing for unidirectionalinterfaces and bidirectional interfaces. In unidirectional in-terfaces the tangibles are only moved by the user and there-fore tangibles can be used for interaction at any time. Thisis not possible when designing for bidirectional interfaces,as RoMo cannot differentiate autonomous robot movementfrom human intervention. If the user grabs and stops a mov-ing robot, RoMo will interpret this as package loss and keepresending robot movement frames until the robot is finally atits finishing position. Consequently, the interaction designof applications for bidirectional interfaces will often need to3Figure 6. RoMo is used to introduce constrains on thespatial layout of tangibles. In this application robots aremoved to ensure that the tangibles form an equilateraltriangle.Figure 7. A map application using RoMo to allow pan-ning and rotation the map. Here, the user rotates themap by rotating a tangible.employ turn taking and provide the user with feedback onwhen tangibles can be manipulated.Figure 7 shows how RoMo may be used to allow interactionsthat would be impossible to employ in unidirectional tangi-ble interfaces. In this application the user places tangibleson the map to mark certain sites (i.e., good places to dine).In unidirectional interfaces this would lock the view of themap, making it impossible to pan or zoom without creatingphysical/digital conflicts. However, in the bidirectional ap-plication shown in Figure 7, the robots move with the map,allowing the user to rotate the map by rotating a tangible orpan the map by moving one tangible. These interactions arenormally only seen in multi-touch applications.CONCLUSIONWe have presented RoMo, an open source Java library thatallows existing TTI applications to perform robot move-ment. We have described how unidirectional TTIs can beconverted to bidirectional TTIs using only off-the-shelf com-ponents. In this paper we have focused on the use of RoMoto avoid physical/digital conflicts in TTIs. While we con-sider this an important quality of RoMo, we look forwardto exploring and evaluating the novel interaction techniquesthat RoMo introduce to TTIs (i.e., undo of actions, recall ofprior spatial layout and imitation of movement).In our future research we plan on addressing the inabilityto detect human intervention by adding proximity sensors tothe robots. Also, we will experiment with omnidirectionalwheels to improve the robot’s freedom of movement.REFERENCES1. Brave, S., Ishii, H., and Dahley, A. 1998. Tangibleinterfaces for remote collaboration and communication.In Proc. CSCW 1998. ACM, New York, 169-178.2. Hogg, R., Rankin, A., Roumeliotis, S., McHenry, M.,Helmick, D., Bergh, C., and Matthies., L. 2002.Algorithms and sensors for small robot path following.In Proc. ICRA 2002. IEEE. Washington, DC3. Ishii, H. and Ullmer, B. 1997. Tangible bits: towardsseamless interfaces between people, bits and atoms. InProc. CHI 2007. ACM, New York, NY, 234-241.4. Kaltenbrunner, M. 2009. reacTIVision and TUIO: atangible tabletop toolkit. In Proc. ITS 2009. ACM, NewYork, 9-16.5. Pangaro, G., Maynes-Aminzade, D., and Ishii, H. 2002.The actuated workbench: computer-controlled actuationin tabletop tangible interfaces. In Proc. UIST 2002.ACM, New York, 181-190.6. Patten, J. and Ishii, H. 2007. Mechanical constraints ascomputational constraints in tabletop tangible interfaces.In Proc. CHI 2007. ACM, New York, 809-818.7. Pedersen, E. and Hornbæk, K., 2011. Tangible Bots:Interaction with Active Tangibles in Tabletop Interfaces.In Proc. CHI 2011. ACM, New York. Anonymized8. Ressler, S., Antonishek, B., Wang, Q., and Godil, A.2001. Integrating active tangible devices with a syntheticenvironment for collaborative engineering. In Proc.Web3D 2001. ACM, New York, 93-100.9. Richter, J., Thomas, B. H., Sugimoto, M., and Inami, M.2007. Remote active tangible interactions. In Proc. TEI2007. ACM, New York, 39-42.10. Rosenfeld, D., Zawadzki, M., Sudol, J., Perlin, K.2004. Physical Objects as Bidirectional User InterfaceElements. In IEEE Comput. Graph. Appl. (Vol. 24, no.1), 44-49.11. Scho¨ning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges,O., Hook, J., Lo¨chtefeld, M., Motamedi, N., Muller, L.,Olivier, P., Roth, T. and Zadow, U. von Scho¨ning. 2008.Multi-touch surfaces: A technical guide. In TechnicalReport TUM-I0833, Technical Reports of the TechnicalUniversity of Munich, (October 2008)12. Sugimoto, M., Kagotani, G., Kojima, M., Nii, H.,Nakamura, A., and Inami, M. 2005. Augmentedcoliseum: display-based computing for augmentedreality inspiration computing robot. In Proc. SIGGRAPH2005. ACM, New York.4

RoMo:avoiding conflicts between the physical and digital model in tabletop interfaces with robotic tangibles

https://curis.ku.dk/ws/files/172392707/B_dker_2011_PROCEEDINGS_DHRS2011.pdf

RoMo:avoiding conflicts between the physical and digital model in tabletop interfaces with robotic tangibles

Abstract

Similar works

Full text

Available Versions

Copenhagen University Research Information System