In the last few years, research in human-robot interaction has moved beyond the issues concerning the design of the interaction between a person and a single robot. Today many researchers have shifted their focus toward the problem of how humans can control a multi-robot team. The rising of multi-touch devices provides a new range of opportunities in this sense. Our research seeks to discover new insights and guidelines for the design of multi-touch interfaces for the control of biologically inspired multi-robot teams. We have developed an iPad touch interface that lets users exert partial control over a set of autonomous robots. The interface also serves as an experimental platform to study how human operators design multi-robot motion in a pursuit-evasion setting

Andolina S.

Forlizzi J.

English

Archivio istituzionale della ricerca - Politecnico di Milano

A multi-touch interface for multi-robot path planning and control

Archivio istituzionale della ricerca - Università di Palermo

A Multi-touch Interface for Multi-Robot PathPlanning and ControlSalvatore Andolina1 and Jodi Forlizzi21 HIIT and Department of Computer Science, University of Helsinkisalvatore.andolina@cs.helsinki.fi2 HCII and School of Design, Carnegie Mellon Universityforlizzi@cs.cmu.eduAbstract. In the last few years, research in human-robot interaction hasmoved beyond the issues concerning the design of the interaction betweena person and a single robot. Today many researchers have shifted theirfocus toward the problem of how humans can control a multi-robot team.The rising of multi-touch devices provides a new range of opportunitiesin this sense. Our research seeks to discover new insights and guidelinesfor the design of multi-touch interfaces for the control of biologicallyinspired multi-robot teams. We have developed an iPad touch interfacethat lets users exert partial control over a set of autonomous robots. Theinterface also serves as an experimental platform to study how humanoperators design multi-robot motion in a pursuit-evasion setting.Keywords: human-robot interaction; multi-agent path planning; bio-logically inspired motion; multi-touch interface1 IntroductionThe field of human-robot interaction has evolved beyond issues concerning thedesign and development of one person controlling one robot. Today, robots andhumans routinely collaborate in multi-robot teams. Robotics has advanced suchthat teams of robots can be fully autonomous, completing multiple tasks in par-allel. In addition to interaction with a group of agents that is fully autonomous,a group of agents could be fully controlled by the user, or some combination thatexists in between.In this paper, we present an experimental platform that helps to understandhow motion and path planning for multi-robot swarms should be designed. Weare especially interested in cases where a human exerts partial control over mul-tiple robots. In [1] researchers explore biologically inspired motion by using anapproach where the input is initiated by an operator, applies to the motion ofa subset of robots, and in turn, affects the motion of other robots. In our work,we developed a multi-touch interface to study how decentralized control over aset of biologically inspired agents can be used to reach high-level pursuit-evasionstrategies. In this paper, we describe the design and development of the interface.2 BackgroundResearch on path-planning has explored a number of algorithms for fully au-tonomous systems. Some of them, such as discrete search methods [5], [13], [12],work for path planning, but do not scale well to multiple robots. Other optimizedmethods [7], while working for multiple robots, do not function well in complexenvironments.Multi-robot motion design research has often leveraged pursuit-evasion tasks.Different algorithms have been designed and tested in simulation to plan the pathof pursuers and evaders, ranging from two pursuers are searching for one evader[16] to multiple pursuers searching for multiple evaders [3], [18].Biological motion has also been a source of inspiration for those researchingmulti-robot path planning. The study of the collective behavior found in naturehas led to attempts to create computational models to characterize and replicatethat behavior. [4] offers a popular bio-inspired model that describes three circularzones around an agent: repulsion, orientation, and attraction. The motion of eachrobot is calculated based on the presence or absence of other individuals on eachof these zones. This globally generates a collective behavior that resembles thebehavior of a real swarms.A number of interfaces and interaction studies can be found exploring how tocontrol a single and multiple robots [2], [6], [8], [10], [11]. A special case is thatof multi-touch interfaces. While surface and direct touch computing can offerchallenges in the forms of less traditional input than the keyboard and mouse,it offers a direct means of controlling elements through multitouch freehand ges-tures [17]. A number of systems have explored multi-touch interfaces to controlphysical robot agents [14], [15]. In our work we try to understand how simpleand direct multi-touch interaction could allow the user to create motion pathsthat affected subsets of robots, which would in turn affect others. We presenta multi-touch interface that gives users the flexibility to use natural gestures,such as those defined [17], to control a swarm of biologically inspired robots.We leverage research on biological motion [4], multi-robot pursuit evasion, anddecentralized control [9].3 ImplementationWe implemented a multi-touch interface on an iPad app that supports up to tenfingers. The app is meant to control a swarm of semi-autonomous biologicallyinspired robots with simple behavior and low connectivity. Each robot can senseother robots within a certain range, and no communication between robots isassumed.In our design three main components contribute to the swarm motion:1. Biological data2. User input3. Local interaction rulesWhen no input is given the swarm enters in a wandering status. While in thisstatus the swarm follows a vector field derived from biological data stored in anxml file, i.e., the velocities of the individuals exactly match velocities measuredfrom real swarms, flocks, etc.When users touch the screen they start controlling the swarm. The fingersthat touch the iPad create a virtual leader behavior at the position of the touch.Raising a finger from the surface corresponds to the removal of that leader fromthe swarm. As the user slides her fingers on the touch surface, the swarm reactsby following the leaders.This behavior is coupled with additional interaction rules that make surerobots will avoid collisions and will try to stay with the group.Following from [4], we define three different zones relative to each robot:– Zone of repulsion (ZOR). Each individual attempts to maintain a minimumdistance from others within a “zone of repulsion”.– Zone of orientation (ZOO). An individual will attempt to align itself withneighbors within the “zone of orientation”.– Zone of attraction (ZOA). An individual will attempt to move toward theposition of leaders within the “zone of attraction”.The app provides a drop-down panel to set parameters and save them forfuture simulations (Figure 1a). It is also possible to explore the effect of a pa-rameter change in real-time, in effect, making the app usable for motion design.We provide two different instances of the interface, one meant to test the eva-sion behavior and the other meant to test the pursuit behavior. In one instance,mobile pursuers try to hit the swarm and the goal is avoiding as much pursuersas possible. In the other instance, the iPad screen is filled with mobile targets,and participants are asked to control the swarm in order to catch as much tar-gets as possible. Individuals in the swarm are represented as orange cirles, whileblue circles represents computer-controlled pursuers/evaders (Figure 1). Belowwe give some examples of possible uses of this interface to design different kindof motions for the swarm.One finger One finger can be used to control the entire swarm (Figure 1b).This strategy can be used to hide the swarm outside the field of view ofpursuers (evasion task)Two fingers By using two fingers a line formation can be obtained (Figure 1c).This formation can be used for pursuing. Another strategy is to split the mainpack of robot in two subgroups (Figure 1d). This strategy can be used bothfor evading and pursuing. When evading it is also possible to use a smallnumber of individuals to draw the attention of the pursuers from the maingroup (Figure 1e).Three fingers A V-formation, typically used for pursuing, can be obtainedwith three fingers (Figure 1f). The same number of fingers can also be usedto shape the swarm as a triangle (Figure 1g).Four fingers More than three fingers are typically necessary when trying toobtain complex configurations. Figure 1h shows how a surround behavior isobtained using four fingers.Playing with parameters A number of different kind of motion can also beobtained by playing with parameters. Figure 1h, for example shows the ef-fects of an increment of the ZOR radius. This increases the repulsion com-ponent among robots of the swarm which causes a spreading behavior.(a) Initial configura-tion and parameterssetting(b) One finger interac-tion: controlling theentire swarm(c) Two fingers inter-action: line formation(d) Two fingers inter-action: split in sub-groups(e) Two fingers inter-action: draw atten-tion from the maingroup(f) Three fingers inter-action: V formation(g) Three fingers inter-action: triangle for-mation(h) Four fingers inter-action: surround(i) Playing with pa-rameters: increasingrepulsionFig. 1: iPad interface.4 Demo Set UpThe setup comprises an iPad mini and a charger. In order to fully take advan-tage of the system, the user should be allowed to interact with the iPad in acomfortable way. A desk and a chair may be necessary to allow users to comfort-ably input commands even with two hands. Alternatively a flexible iPad standto hold the device can be used.5 Running experimentsThe platform described can be used to run different kind of experiments. In [1]it was used to understand how users would take advantage of the multi-fingercontrol in the context of pursuit-evasion.During the experiment,participants were given three to five minutes for eachof the two tasks. Within that time, they performed multiple trials to iterativelyrefine the evading/pursuing strategy. Experiments containing iterative trials areuseful to understand how people interact with swarms of autonomous robots. Forexample in [1], a similar setup was used to gain insights about frequently-usedstrategies for pursuit-evasion and the way they are achieved. Our work seeks todefine new pursuit-evasion strategies and to suggest natural high level gesturesthat can be used to provide users with even more control over the interface whileminimizing attentional demands.References1. Andolina, S., Forlizzi, J.: The design of interfaces for multi-robot path planningand control. In: Advanced Robotics and its Social Impacts (ARSO), 2014 IEEEWorkshop on (September 2014)2. Brooks, D., Yanco, H.: Design of a haptic joystick for shared robot control. In:Human-Robot Interaction (HRI), 2012 7th ACM/IEEE International Conferenceon. pp. 113–114 (March 2012)3. Cai, Z.S., Sun, L.N., Gao, H.B., Zhou, P.C., Piao, S.H., Huang, Q.C.: Multi-robotcooperative pursuit based on task bundle auctions. In: Proceedings of the FirstInternational Conference on Intelligent Robotics and Applications: Part I. pp. 235–244. ICIRA ’08, Springer-Verlag, Berlin, Heidelberg (2008)4. Couzin, I.D., Krause, J., Franks, N.R., Levin, S.A.: Effective leadership anddecision-making in animal groups on the move. Nature 433(7025), 513–516 (2005)5. Desaraju, V.R., How, J.P.: Decentralized path planning for multi-agent teams withcomplex constraints. Auton. Robots 32(4), 385–403 (May 2012)6. Drury, J., Scholtz, J., Yanco, H.: Awareness in human-robot interactions. In: Sys-tems, Man and Cybernetics, 2003. IEEE International Conference on. vol. 1, pp.912–918 vol.1 (Oct 2003)7. Dunbar, W., Murray, R.: Model predictive control of coordinated multi-vehicle for-mations. In: Decision and Control, 2002, Proceedings of the 41st IEEE Conferenceon. vol. 4, pp. 4631–4636 vol.4 (Dec 2002)8. Fong, T., Thorpe, C., Baur, C.: Multi-robot remote driving with collaborativecontrol. Industrial Electronics, IEEE Transactions on 50(4), 699–704 (Aug 2003)9. Goodrich, M.A., Kerman, S., Pendleton, B., Sujit, P.B.: What types of interactionsdo bio-inspired robot swarms and flocks afford a human? In: Robotics: Science andSystems (2012)10. Hayes, S., Hooten, E., Adams, J.: Multi-touch interaction for tasking robots. In:Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conferenceon. pp. 97–98 (March 2010)11. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-sourcemulti-robot simulator. In: Intelligent Robots and Systems, 2004. (IROS 2004).Proceedings. 2004 IEEE/RSJ International Conference on. vol. 3, pp. 2149–2154vol.3 (Sept 2004)12. Likhachev, M., Stentz, A.: R* search. In: Proceedings of the 23rd National Con-ference on Artificial Intelligence - Volume 1. pp. 344–350. AAAI’08, AAAI Press(2008)13. M. Flint, M. Polycarpou, E.F.G.: Cooperative path-planning for autonomous ve-hicles using dynamic programming. In: IFAC World Congress (2002)14. Malik, S., Ranjan, A., Balakrishnan, R.: Interacting with large displays from adistance with vision-tracked multi-finger gestural input. In: Proceedings of the18th Annual ACM Symposium on User Interface Software and Technology. pp.43–52. UIST ’05, ACM, New York, NY, USA (2005)15. Micire, M., Drury, J.L., Keyes, B., Yanco, H.A.: Multi-touch interaction for robotcontrol. In: Proceedings of the 14th International Conference on Intelligent UserInterfaces. pp. 425–428. IUI ’09, ACM, New York, NY, USA (2009)16. Simov, B., Slutzki, G., LaValle, S.: Pursuit-evasion using beam detection. In:Robotics and Automation, 2000. Proceedings. ICRA ’00. IEEE International Con-ference on. vol. 2, pp. 1657–1662 vol.2 (2000)17. Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surfacecomputing. In: Proceedings of the SIGCHI Conference on Human Factors in Com-puting Systems. pp. 1083–1092. CHI ’09, ACM, New York, NY, USA (2009)18. Zhang, Y., Kuhn, L., Fromherz, M.: Improvements on ant routing for sensor net-works. In: Dorigo, M., Birattari, M., Blum, C., Gambardella, L., Mondada, F.,Sttzle, T. (eds.) Ant Colony Optimization and Swarm Intelligence, Lecture Notesin Computer Science, vol. 3172, pp. 154–165. Springer Berlin Heidelberg (2004)

https://iris.unipa.it/bitstream/10447/391282/2/andolina2014multirobot2.pdf

A multi-touch interface for multi-robot path planning and control

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Politecnico di Milano

Archivio istituzionale della ricerca - Università di Palermo