7 research outputs found
Interactive spaces for children: gesture elicitation for controlling ground mini-robots
[EN] Interactive spaces for education are emerging as a mechanism for fostering children's natural ways of learning by means of play and exploration in physical spaces. The advanced interactive modalities and devices for such environments need to be both motivating and intuitive for children. Among the wide variety of interactive mechanisms, robots have been a popular research topic in the context of educational tools due to their attractiveness for children. However, few studies have focused on how children would naturally interact and explore interactive environments with robots. While there is abundant research on full-body interaction and intuitive manipulation of robots by adults, no similar research has been done with children. This paper therefore describes a gesture elicitation study that identified the preferred gestures and body language communication used by children to control ground robots. The results of the elicitation study were used to define a gestural language that covers the different preferences of the gestures by age group and gender, with a good acceptance rate in the 6-12 age range. The study also revealed interactive spaces with robots using body gestures as motivating and promising scenarios for collaborative or remote learning activities.This work is funded by the European Development Regional Fund (EDRF-FEDER) and supported by the Spanish MINECO (TIN2014-60077-R). The work of Patricia Pons is supported by a national grant from the Spanish MECD (FPU13/03831). Special thanks are due to the children and teachers of the Col-legi Public Vicente Gaos for their valuable collaboration and dedication.Pons TomĂĄs, P.; JaĂ©n MartĂnez, FJ. (2020). Interactive spaces for children: gesture elicitation for controlling ground mini-robots. Journal of Ambient Intelligence and Humanized Computing. 11(6):2467-2488. https://doi.org/10.1007/s12652-019-01290-6S24672488116Alborzi H, Hammer J, Kruskal A et al (2000) Designing StoryRooms: interactive storytelling spaces for children. In: Proceedings of the conference on designing interactive systems processes, practices, methods, and techniquesâDISâ00. ACM Press, New York, pp 95â104Antle AN, Corness G, Droumeva M (2009) What the body knows: exploring the benefits of embodied metaphors in hybrid physical digital environments. Interact Comput 21:66â75. https://doi.org/10.1016/j.intcom.2008.10.005Belpaeme T, Baxter PE, Read R et al (2013) Multimodal childârobot interaction: building social bonds. J Human-Robot Interact 1:33â53. https://doi.org/10.5898/JHRI.1.2.BelpaemeBenko H, Wilson AD, Zannier F, Benko H (2014) Dyadic projected spatial augmented reality. In: Proceedings of the 27th annual ACM symposium on user interface software and technologyâUISTâ14, pp 645â655Bobick AF, Intille SS, Davis JW et al (1999) The KidsRoom: a perceptually-based interactive and immersive story environment. Presence Teleoper Virtual Environ 8:367â391. https://doi.org/10.1162/105474699566297Bonarini A, Clasadonte F, Garzotto F, Gelsomini M (2015) Blending robots and full-body interaction with large screens for children with intellectual disability. In: Proceedings of the 14th international conference on interaction design and childrenâIDCâ15. ACM Press, New York, pp 351â354Cauchard JR, E JL, Zhai KY, Landay JA (2015) Drone & me: an exploration into natural humanâdrone interaction. In: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computingâUbiCompâ15. ACM Press, New York, pp 361â365Connell S, Kuo P-Y, Liu L, Piper AM (2013) A Wizard-of-Oz elicitation study examining child-defined gestures with a whole-body interface. In: Proceedings of the 12th international conference on interaction design and childrenâIDCâ13. ACM Press, New York, pp 277â280Derboven J, Van Mechelen M, Slegers K (2015) Multimodal analysis in participatory design with children. In: Proceedings of the 33rd annual ACM conference on human factors in computing systemsâCHIâ15. ACM Press, New York, pp 2825â2828Dong H, Danesh A, Figueroa N, El Saddik A (2015) An elicitation study on gesture preferences and memorability toward a practical hand-gesture vocabulary for smart televisions. IEEE Access 3:543â555. https://doi.org/10.1109/ACCESS.2015.2432679Druin A (1999) Cooperative inquiry: developing new technologies for children with children. In: Proceedings of the SIGCHI conference on human factors computer system CHI is limitâCHIâ99, vol 14, pp 592â599. https://doi.org/10.1145/302979.303166Druin A (2002) The role of children in the design of new technology. Behav Inf Technol 21:1â25. https://doi.org/10.1080/01449290110108659Druin A, Bederson B, Boltman A et al (1999) Children as our technology design partners. In: Druin A (ed) The design of childrenâs technology. Morgan Kaufman, San Francisco, pp 51â72Epps J, Lichman S, Wu M (2006) A study of hand shape use in tabletop gesture interaction. CHIâ06 extended abstracts on human factors in computing systemsâCHI EAâ06. ACM Press, New York, pp 748â753Fender AR, Benko H, Wilson A (2017) MeetAliveâŻ: room-scale omni-directional display system for multi-user content and control sharing. In: Proceedings of the 2017 ACM international conference on interactive surfaces and spaces, pp 106â115Fernandez RAS, Sanchez-Lopez JL, Sampedro C et al (2016) Natural user interfaces for humanâdrone multi-modal interaction. In: 2016 international conference on unmanned aircraft systems (ICUAS). IEEE, New York, pp 1013â1022Garcia-Sanjuan F, Jaen J, Nacher V, Catala A (2015) Design and evaluation of a tangible-mediated robot for kindergarten instruction. In: Proceedings of the 12th international conference on advances in computer entertainment technologyâACEâ15. ACM Press, New York, pp 1â11Garcia-Sanjuan F, Jaen J, Jurdi S (2016) Towards encouraging communication in hospitalized children through multi-tablet activities. In: Proceedings of the XVII international conference on human computer interaction, pp 29.1â29.4Gindling J, Ioannidou A, Loh J et al (1995) LEGOsheets: a rule-based programming, simulation and manipulation environment for the LEGO programmable brick. In: Proceedings of symposium on visual languages. IEEE Computer Society Press, New York, pp 172â179Gonzalez B, Borland J, Geraghty K (2009) Whole body interaction for child-centered multimodal language learning. In: Proceedings of the 2nd workshop on child, computer and interactionâWOCCIâ09. ACM Press, New York, pp 1â5GrĂžnbĂŠk K, Iversen OS, Kortbek KJ et al (2007) Interactive floor support for kinesthetic interaction in children learning environments. In: Humanâcomputer interactionâINTERACT 2007. Lecture notes in computer science, pp 361â375Guha ML, Druin A, Chipman G et al (2005) Working with young children as technology design partners. Commun ACM 48:39â42. https://doi.org/10.1145/1039539.1039567Hansen JP, Alapetite A, MacKenzie IS, MĂžllenbach E (2014) The use of gaze to control drones. In: Proceedings of the symposium on eye tracking research and applicationsâETRAâ14. ACM Press, New York, pp 27â34Henkemans OAB, Bierman BPB, Janssen J et al (2017) Design and evaluation of a personal robot playing a self-management education game with children with diabetes type 1. Int J Hum Comput Stud 106:63â76. https://doi.org/10.1016/j.ijhcs.2017.06.001Horn MS, Crouser RJ, Bers MU (2011) Tangible interaction and learning: the case for a hybrid approach. Pers Ubiquitous Comput 16:379â389. https://doi.org/10.1007/s00779-011-0404-2Hourcade JP (2015) Child computer interaction. CreateSpace Independent Publishing Platform, North CharlestonHöysniemi J, HĂ€mĂ€lĂ€inen P, Turkki L (2004) Wizard of Oz prototyping of computer vision based action games for children. Proceeding of the 2004 conference on interaction design and children building a communityâIDCâ04. ACM Press, New York, pp 27â34Höysniemi J, HĂ€mĂ€lĂ€inen P, Turkki L, Rouvi T (2005) Childrenâs intuitive gestures in vision-based action games. Commun ACM 48:44â50. https://doi.org/10.1145/1039539.1039568Hsiao H-S, Chen J-C (2016) Using a gesture interactive game-based learning approach to improve preschool childrenâs learning performance and motor skills. Comput Educ 95:151â162. https://doi.org/10.1016/j.compedu.2016.01.005Jokela T, Rezaei PP, VÀÀnĂ€nen K (2016) Using elicitation studies to generate collocated interaction methods. In: Proceedings of the 18th international conference on humanâcomputer interaction with mobile devices and services adjunct, pp 1129â1133. https://doi.org/10.1145/2957265.2962654Jones B, Benko H, Ofek E, Wilson AD (2013) IllumiRoom: peripheral projected illusions for interactive experiences. In: Proceedings of the SIGCHI conference on human factors in computing systemsâCHIâ13, pp 869â878Jones B, Shapira L, Sodhi R et al (2014) RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the 27th annual ACM symposium on user interface software and technologyâUISTâ14, pp 637â644Kaminski M, Pellino T, Wish J (2002) Play and pets: the physical and emotional impact of child-life and pet therapy on hospitalized children. Child Heal Care 31:321â335. https://doi.org/10.1207/S15326888CHC3104_5Karam M, Schraefel MC (2005) A taxonomy of gestures in human computer interactions. In: Technical report in electronics and computer science, pp 1â45Kistler F, AndrĂ© E (2013) User-defined body gestures for an interactive storytelling scenario. Lect Notes Comput Sci (including subser Lect Notes Artif Intell Lect Notes Bioinform) 8118:264â281. https://doi.org/10.1007/978-3-642-40480-1_17Konda KR, Königs A, Schulz H, Schulz D (2012) Real time interaction with mobile robots using hand gestures. In: Proceedings of the seventh annual ACM/IEEE international conference on humanârobot interactionâHRIâ12. ACM Press, New York, pp 177â178Kray C, Nesbitt D, Dawson J, Rohs M (2010) User-defined gestures for connecting mobile phones, public displays, and tabletops. In: Proceedings of the 12th international conference on human computer interaction with mobile devices and servicesâMobileHCIâ10. ACM Press, New York, pp 239â248Kurdyukova E, Redlin M, AndrĂ© E (2012) Studying user-defined iPad gestures for interaction in multi-display environment. In: Proceedings of the 2012 ACM international conference on intelligent user interfacesâIUIâ12. ACM Press, New York, pp 93â96Lambert V, Coad J, Hicks P, Glacken M (2014) Social spaces for young children in hospital. Child Care Health Dev 40:195â204. https://doi.org/10.1111/cch.12016Lee S-S, Chae J, Kim H et al (2013) Towards more natural digital content manipulation via user freehand gestural interaction in a living room. In: Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computingâUbiCompâ13. ACM Press, New York, p 617Malinverni L, Mora-Guiard J, Pares N (2016) Towards methods for evaluating and communicating participatory design: a multimodal approach. Int J Hum Comput Stud 94:53â63. https://doi.org/10.1016/j.ijhcs.2016.03.004Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18:50â60. https://doi.org/10.1214/aoms/1177730491Marco J, Cerezo E, Baldassarri S et al (2009) Bringing tabletop technologies to kindergarten children. In: Proceedings of the 23rd British HCI Group annual conference on people and computers: celebrating people and technology, pp 103â111Michaud F, Caron S (2002) Roball, the rolling robot. Auton Robots 12:211â222. https://doi.org/10.1023/A:1014005728519Micire M, Desai M, Courtemanche A et al (2009) Analysis of natural gestures for controlling robot teams on multi-touch tabletop surfaces. In: Proceedings of the ACM international conference on interactive tabletops and surfacesâITSâ09. ACM Press, New York, pp 41â48Mora-Guiard J, Crowell C, Pares N, Heaton P (2016) Lands of fog: helping children with autism in social interaction through a full-body interactive experience. In: Proceedings of the 15th international conference on interaction design and childrenâIDCâ16. ACM Press, New York, pp 262â274Morris MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In: Proceedings of the 2012 ACM international conference on interactive tabletops and surfaces. ACM Press, New York, pp 95â104Morris MR, Wobbrock JO, Wilson AD (2010) Understanding usersâ preferences for surface gestures. Proc Graph Interface 2010:261â268Nacher V, Garcia-Sanjuan F, Jaen J (2016) Evaluating the usability of a tangible-mediated robot for kindergarten children instruction. In: 2016 IEEE 16th international conference on advanced learning technologies (ICALT). IEEE, New York, pp 130â132Nahapetyan VE, Khachumov VM (2015) Gesture recognition in the problem of contactless control of an unmanned aerial vehicle. Optoelectron Instrum Data Process 51:192â197. https://doi.org/10.3103/S8756699015020132Obaid M, HĂ€ring M, Kistler F et al (2012) User-defined body gestures for navigational control of a humanoid robot. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 367â377Obaid M, Kistler F, HĂ€ring M et al (2014) A framework for user-defined body gestures to control a humanoid robot. Int J Soc Robot 6:383â396. https://doi.org/10.1007/s12369-014-0233-3Obaid M, Kistler F, KasparaviÄiĆ«tÄ G, et al (2016) How would you gesture navigate a drone?: a user-centered approach to control a drone. In: Proceedings of the 20th international academic Mindtrek conferenceâAcademicMindtrekâ16. ACM Press, New York, pp 113â121Pares N, Soler M, Sanjurjo Ă et al (2005) Promotion of creative activity in children with severe autism through visuals in an interactive multisensory environment. In: Proceeding of the 2005 conference on interaction design and childrenâIDCâ05. ACM Press, New York, pp 110â116Pfeil K, Koh SL, LaViola J (2013) Exploring 3D gesture metaphors for interaction with unmanned aerial vehicles. In: Proceedings of the 2013 international conference on intelligent user interfacesâIUIâ13, pp 257â266. https://doi.org/10.1145/2449396.2449429Piaget J (1956) The childâs conception of space. Norton, New YorkPiaget J (1973) The child and reality: problems of genetic psychology. Grossman, New YorkPiumsomboon T, Clark A, Billinghurst M, Cockburn A (2013) User-defined gestures for augmented reality. CHIâ13 extended abstracts on human factors in computing systemsâCHI EAâ13. ACM Press, New York, pp 955â960Pons P, CarriĂłn A, Jaen J (2018) Remote interspecies interactions: improving humans and animalsâ wellbeing through mobile playful spaces. Pervasive Mob Comput. https://doi.org/10.1016/j.pmcj.2018.12.003Puranam MB (2005) Towards full-body gesture analysis and recognition. University of Kentucky, LexingtonPyryeskin D, Hancock M, Hoey J (2012) Comparing elicited gestures to designer-created gestures for selection above a multitouch surface. In: Proceedings of the 2012 ACM international conference on interactive tabletops and surfacesâITSâ12. ACM Press, New York, pp 1â10Raffle HS, Parkes AJ, Ishii H (2004) Topobo: a constructive assembly system with kinetic memory. System 6:647â654. https://doi.org/10.1145/985692.985774Read JC, Markopoulos P (2013) Childâcomputer interaction. Int J Child-Comput Interact 1:2â6. https://doi.org/10.1016/j.ijcci.2012.09.001Read JC, Macfarlane S, Casey C (2002) Endurability, engagement and expectations: measuring childrenâs fun. In: Interaction design and children, pp 189â198Read JC, Markopoulos P, ParĂ©s N et al (2008) Child computer interaction. In: Proceeding of the 26th annual CHI conference extended abstracts on human factors in computing systemsâCHIâ08. ACM Press, New York, pp 2419â2422Robins B, Dautenhahn K (2014) Tactile interactions with a humanoid robot: novel play scenario implementations with children with autism. Int J Soc Robot 6:397â415. https://doi.org/10.1007/s12369-014-0228-0Robins B, Dautenhahn K, Te Boekhorst R, Nehaniv CL (2008) Behaviour delay and robot expressiveness in childârobot interactions: a user study on interaction kinesics. In: Proceedings of the 3rd ACMIEEE international conference on human robot interaction, pp 17â24. https://doi.org/10.1145/1349822.1349826Ruiz J, Li Y, Lank E (2011) User-defined motion gestures for mobile interaction. In: Proceedings of the 2011 annual conference on human factors in computing systemsâCHIâ11. ACM Press, New York, p 197Rust K, Malu M, Anthony L, Findlater L (2014) Understanding childdefined gestures and childrenâs mental models for touchscreen tabletop interaction. In: Proceedings of the 2014 conference on interaction design and childrenâIDCâ14. ACM Press, New York, pp 201â204Salter T, Dautenhahn K, Te Boekhorst R (2006) Learning about natural human-robot interaction styles. Robot Auton Syst 54:127â134. https://doi.org/10.1016/j.robot.2005.09.022Sanghvi J, Castellano G, Leite I et al (2011) Automatic analysis of affective postures and body motion to detect engagement with a game companion. In: Proceedings of the 6th international conference on humanârobot interactionâHRIâ11. ACM Press, New York, pp 305â311Sanna A, Lamberti F, Paravati G, Manuri F (2013) A Kinect-based natural interface for quadrotor control. Entertain Comput 4:179â186. https://doi.org/10.1016/j.entcom.2013.01.001Sato E, Yamaguchi T, Harashima F (2007) Natural interface using pointing behavior for humanârobot gestural interaction. IEEE Trans Ind Electron 54:1105â1112. https://doi.org/10.1109/TIE.2007.892728Schaper M-M, Pares N (2016) Making sense of body and space through full-body interaction design. In: Proceedings of the 15th international conference on interaction design and childrenâIDCâ16. ACM Press, New York, pp 613â618Schaper M-M, Malinverni L, Pares N (2015) Sketching through the body: child-generated gestures in full-body interaction design. In: Proceedings of the 14th international conference on interaction design and childrenâIDCâ15. ACM Press, New York, pp 255â258Seyed T, Burns C, Costa Sousa M et al (2012) Eliciting usable gestures for multi-display environments. In: Proceedings of the 2012 ACM international conference on interactive tabletops and surfacesâITSâ12. ACM Press, New York, p 41Shimon SSA, Morrison-Smith S, John N et al (2015) Exploring user-defined back-of-device gestures for mobile devices. In: Proceedings of the 17th international conference on humanâcomputer interaction with mobile devices and servicesâMobileHCIâ15. ACM Press, New York, pp 227â232Sipitakiat A, Nusen N (2012) Robo-blocks: a tangible programming system with debugging for children. In: Proceedings of the 11th international conference on interaction design and childrenâIDCâ12. ACM Press, New York, p 98Soler-Adillon J, Ferrer J, Pares N (2009) A novel approach to interactive playgrounds: the interactive slide project. In: Proceedings of the 8th international conference on interaction design and childrenâIDCâ09. ACM Press, New York, pp 131â139Stiefelhagen R, Fogen C, Gieselmann P et al (2004) Natural humanârobot interaction using speech, head pose and gestures. In: 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE Cat. No. 04CH37566). IEEE, New York, pp 2422â2427Subrahmanyam K, Greenfield PM (1994) Effect of video game practice on spatial skills in girls and boys. J Appl Dev Psychol 15:13â32. https://doi.org/10.1016/0193-3973(94)90004-3Sugiyama J, Tsetserukou D, Miura J (2011) NAVIgoid: robot navigation with haptic vision. In: SIGGRAPH Asia 2011 emerging technologies SAâ11, vol 15, p 4503. https://doi.org/10.1145/2073370.2073378Takahashi T, Morita M, Tanaka F (2012) Evaluation of a tricycle-style teleoperational interface for children: a comparative experiment with a video game controller. In: 2012 IEEE RO-MAN: the 21st IEEE international symposium on robot and human interactive communication. IEEE, New York, pp 334â338Tanaka F, Takahashi T (2012) A tricycle-style teleoperational interface that remotely controls a robot for classroom children. In: Proceedings of the seventh annual ACM/IEEE international conference on humanârobot interactionâHRIâ12. ACM Press, New York, pp 255â256Tjaden L, Tong A, Henning P et al (2012) Childrenâs experiences of dialysis: a systematic review of qualitative studies. Arch Dis Child 97:395â402. https://doi.org/10.1136/archdischild-2011-300639Vatavu R-D (2012) User-defined gestures for free-hand TV control. In: Proceedings of the 10th European conference on interactive TV and videoâEuroiTVâ12. ACM Press, New York, pp 45â48Vatavu R-D (2017) Smart-Pockets: body-deictic gestures for fast access to personal data during ambient interactions. Int J Hum Comput Stud 103:1â21. https://doi.org/10.1016/j.ijhcs.2017.01.005Vatavu R-D, Wobbrock JO (2015) Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In: Proceedings of the 33rd annual ACM conference on human factors in computing systemsâCHIâ15. ACM Press, New York, pp 1325â1334Vatavu R-D, Wobbrock JO (2016) Between-subjects elicitation studies: formalization and tool support. In: Proceedings of the 2016 CHI conference on human factors in computing systemsâCHIâ16. ACM Press, New York, pp 3390â3402Voyer D, Voyer S, Bryden MP (1995) Magnitude of sex differences in spatial abilities: a meta-analysis and consideration of critical variables. Psychol Bull 117:250â270. https://doi.org/10.1037/0033-2909.117.2.250Wainer J, Robins B, Amirabdollahian F, Dautenhahn K (2014) Using the humanoid robot KASPAR to autonomously play triadic games and facilitate collaborative play among children with autism. IEEE Trans Auton Ment Dev 6:183â199. https://doi.org/10.1109/TAMD.2014.2303116Wang Y, Zhang L (2015) A track-based gesture recognition algorithm for Kinect. Appl Mech Mater 738â7399:334â338. https://doi.org/10.4028/www.scientific.net/AMM.738-739.334
Haptic feedback in freehand gesture interaction
In this thesis work, haptic feedback in gesture interaction was studied. More precisely, focus was on vibrotactile feedback and freehand gestural input methods. Vibrotactile feedback methods have been studied extensively in the fields of touch-based interaction, remote control and mid-air gestural input, and mostly positive effects on user performance have been found. An experiment was conducted in order to investigate if vibrotactile feedback has an impact on user performance in a simple data entry task. In the study, two gestural input methods were compared and the effects of visual and vibrotactile feedback added to each method were examined. Statistically significant differences in task performance between input methods were found. Results also showed that less keystrokes per character were required with visual feedback. No other significant differences were found between the types of feedback. However, preference for vibrotactile feedback was observed. The findings indicate that the careful design of an input method primarily has an impact on user performance and the feedback method can enhance this performance in diverse ways
Investigating Selection above a Multitouch Surface
Above-surface interaction is a new and exciting topic in the field of human-computer interaction (HCI). It focuses on the design and evaluation of systems that humans can operate by moving their hands in the space above or in front of interactive displays. While many technologies emerge that make such systems possible, much research is still needed to make this interaction as natural and effortless as possible. First this thesis presents a set of guidelines for designing above-surface interactions, a collection of widgets that were designed based on these guidelines, and a system that can approximate the height of hands above a diffused surface illumination (DSI) device without any additional sensors. Then the thesis focuses on interaction techniques for activating graphical widgets located in this above-surface space. Finally, it presents a pair of studies that were conducted to investigate item selection in the space above a multitouch surface. The first study was conducted to elicit a set of gestures for above-table widget activation from a group of users. Several gestures were proposed by the designers to be compared with the user-generated gestures. The follow-up study was conducted to evaluate and compare these gestures based on their performance. The findings of these studies showed that there was no clear agreement on what gestures should be used to select objects in mid-air, and that performance was better when using gestures that were chosen less frequently, but predicted to be better by the designers, as opposed to those most frequently suggested by participants
SimSense - Gestural Interaction Design for Information Exchange between Large Public Displays and Personal Mobile Devices
Large displays in public and semi-public spaces continuously permeate our everyday lives as the price of display hardware continues to drop. These displays act as sources of information, entertainment and advertisement in public environments such as airports, hotels, universities, retail stores, hospitals, and stadiums, amongst others. The information shown on these displays often varies in form that ranges from simple text to rich interactive content. However, most of this rich information remains in the displays and methods to effectively retrieve them to onesâ mobile devices without the need to explicitly manipulate them remains unexplored.
Sensing technologies were used to implement a use case, wherein a person can simply walk up to a public display, retrieve interesting content onto their personal device without having the need to take it out of their pockets or bags. For this purpose a novel system called SimSense, which is capable of automatically detecting and establishing a connection with mobile phones that come in close proximity with the public display was developed. This thesis presents two alternative mid-air hand gesture interaction techniques: âGrab & Pullâ and âGrab & Dropâ to retrieve content from the public display without explicitly operating the mobile device. The results of a laboratory experiment conducted to evaluate these interaction techniques and gather preliminary impressions on the overall concept, are also presented. The results indicate that participants found âGrab & Pullâ to be slightly easier, more confident, and requires less effort to perform in comparison with âGrab & Dropâ. Participants found the overall concept to be seamless and a useful way to retrieve interesting content
Expanding tangible tabletop interfaces beyond the display
Lâaugment
de
popularitat
de
les
taules
i
superfĂcies
interactives
estĂ
impulsant
la
recerca
i
la
innovaciĂł
en
una
gran
varietat
dâĂ rees,
incloent-Ââhi
maquinari,
programari,
disseny
de
la
interacciĂł
i
noves
tĂšcniques
dâinteracciĂł.
Totes,
amb
lâobjectiu
de
promoure
noves
interfĂcies
dotades
dâun
llenguatge
més
ric,
potent
i
natural.
Entre
totes
aquestes
modalitats,
la
interacciĂł
combinada
a
sobre
i
per
damunt
de
la
superfĂcie
de
la
taula
mitjançant
tangibles
i
gestos
Ă©s
actualment
una
Ă rea
molt
prometedora.
Aquest
document
tracta
dâexpandir
les
taules
interactives
més
enllĂ
de
la
superfĂcie
per
mitjĂ
de
lâexploraciĂł
i
el
desenvolupament
dâun
sistema
o
dispositiu
enfocat
des
de
tres
vessants
diferents:
maquinari,
programari
i
disseny
de
la
interacciĂł.
Durant
lâinici
dâaquest
document
sâestudien
i
es
resumeixen
els
diferents
trets
caracterĂstics
de
les
superfĂcies
interactives
tangibles
convencionals
o
2D
i
es
presenten
els
treballs
previs
desenvolupats
per
lâautor
en
solucions
de
programari
que
acaben
resultant
en
aplicacions
que
suggereixen
lâĂșs
de
la
tercera
dimensiĂł
a
les
superfĂcies
tangibles.
Seguidament,
es
presenta
un
repĂ s
del
maquinari
existent
en
aquest
tipus
dâinterfĂcies
per
tal
de
concebre
un
dispositiu
capaç
de
detectar
gestos
i
generar
visuals
per
sobre
de
la
superfĂcie,
per
introduir
els
canvis
realitzats
a
un
dispositiu
existent,
desenvolupat
i
cedit
per
Microsoft
Reseach
Cambridge.
Per
tal
dâexplotar
tot
el
potencial
dâaquest
nou
dispositiu,
es
desenvolupa
un
nou
sistema
de
visiĂł
per
ordinador
que
estén
el
seguiment
dâobjectes
i
mans
en
una
superfĂcie
2D
a
la
detecciĂł
de
mans,
dits
i
etiquetes
amb
sis
graus
de
llibertat
per
sobre
la
superfĂcie
incloent-Ââhi
la
interacciĂł
tangible
i
tĂ ctil
convencional
a
la
superfĂcie.
Finalment,
es
presenta
una
eina
de
programari
per
a
generar
aplicacions
per
al
nou
sistema
i
es
presenten
un
seguit
dâaplicacions
per
tal
de
provar
tot
el
desenvolupament
generat
al
llarg
de
la
tesi
que
es
conclou
presentant
un
seguit
de
gestos
tant
a
la
superfĂcie
com
per
sobre
dâaquesta
i
situant-Ââlos
en
una
nova
classificaciĂł
que
alhora
recull
la
interacciĂł
convencional
2D
i
la
interacciĂł
estesa
per
damunt
de
la
superfĂcie
desenvolupada.The
rising
popularity
of
interactive
tabletops
and
surfaces
is
spawning
research
and
innovation
in
a
wide
variety
of
areas,
including
hardware
and
software
technologies,
interaction
design
and
novel
interaction
techniques,
all
of
which
seek
to
promote
richer,
more
powerful
and
more
natural
interaction
modalities.
Among
these
modalities,
combined
interaction
on
and
above
the
surface,
both
with
gestures
and
with
tangible
objects,
is
a
very
promising
area.
This
dissertation
is
about
expanding
tangible
and
tabletops
surfaces
beyond
the
display
by
exploring
and
developing
a
system
from
the
three
different
perspectives:
hardware,
software,
and
interaction
design.
This
dissertation,
studies
and
summarizes
the
distinctive
affordances
of
conventional
2D
tabletop
devices,
with
a
vast
literature
review
and
some
additional
use
cases
developed
by
the
author
for
supporting
these
findings,
and
subsequently
explores
the
novel
and
not
yet
unveiled
potential
affordances
of
3D-Ââaugmented
tabletops.
It
overviews
the
existing
hardware
solutions
for
conceiving
such
a
device,
and
applies
the
needed
hardware
modifications
to
an
existing
prototype
developed
and
rendered
to
us
by
Microsoft
Research
Cambridge.
For
accomplishing
the
interaction
purposes,
it
is
developed
a
vision
system
for
3D
interaction
that
extends
conventional
2D
tabletop
tracking
for
the
tracking
of
hand
gestures,
6DoF
markers
and
on-Ââsurface
finger
interaction.
It
finishes
by
conceiving
a
complete
software
framework
solution,
for
the
development
and
implementation
of
such
type
of
applications
that
can
benefit
from
these
novel
3D
interaction
techniques,
and
implements
and
test
several
software
prototypes
as
proof
of
concepts,
using
this
framework.
With
these
findings,
it
concludes
presenting
continuous
tangible
interaction
gestures
and
proposing
a
novel
classification
for
3D
tangible
and
tabletop
gestures
Understanding interaction mechanics in touchless target selection
Indiana University-Purdue University Indianapolis (IUPUI)We use gestures frequently in daily lifeâto interact with people, pets, or objects. But interacting with computers using mid-air gestures continues to challenge the design of touchless systems. Traditional approaches to touchless interaction focus on exploring gesture inputs and evaluating user interfaces. I shift the focus from gesture elicitation and interface evaluation to touchless interaction mechanics. I argue for a novel approach to generate design guidelines for touchless systems: to use fundamental interaction principles, instead of a reactive adaptation to the sensing technology. In five sets of experiments, I explore visual and pseudo-haptic feedback, motor intuitiveness, handedness, and perceptual Gestalt effects. Particularly, I study the interaction mechanics in touchless target selection. To that end, I introduce two novel interaction techniques: touchless circular menus that allow command selection using directional strokes and interface topographies that use pseudo-haptic feedback to guide steeringâtargeting tasks. Results illuminate different facets of touchless interaction mechanics. For example, motor-intuitive touchless interactions explain how our sensorimotor abilities inform touchless interface affordances: we often make a holistic oblique gesture instead of several orthogonal hand gestures while reaching toward a distant display. Following the Gestalt theory of visual perception, we found similarity between user interface (UI) components decreased user accuracy while good continuity made users faster. Other findings include hemispheric asymmetry affecting transfer of training between dominant and nondominant hands and pseudo-haptic feedback improving touchless accuracy. The results of this dissertation contribute design guidelines for future touchless systems. Practical applications of this work include the use of touchless interaction techniques in various domains, such as entertainment, consumer appliances, surgery, patient-centric health settings, smart cities, interactive visualization, and collaboration