Integrating 2D Mouse Emulation with 3D Manipulation for Visualizations on a Multi-Touch Table

We present the Rizzo, a multi-touch virtual mouse that has been designed to provide the fine grained interaction for information visualization on a multi-touch table. Our solution enables touch interaction for existing mouse-based visualizations. Previously, this transition to a multi-touch environment was difficult because the mouse emulation of touch surfaces is often insufficient to provide full information visualization functionality. We present a unified design, combining many Rizzos that have been designed not only to provide mouse capabilities but also to act as zoomable lenses that make precise information access feasible. The Rizzos and the information visualizations all exist within a touch-enabled 3D window management system. Our approach permits touch interaction with both the 3D windowing environment as well as with the contents of the individual windows contained therein. We describe an implementation of our technique that augments the VisLink 3D visualization environment to demonstrate how to enable multi-touch capabilities on all visualizations written with the popular prefuse visualization toolkit.

Usabilidad de una mesa multitáctil para el fomento de habilidades básicas de aprendizaje en estudiantes de primaria

Este trabajo presenta la investigación realizada en torno al uso de una mesa multitáctil como apoyo al desarrollo de habilidades básicas de aprendizaje en estudiantes de tercero de primaria. Para tal propósito se describe el proceso de diseño y construcción de la mesa multitáctil, el software y las pruebas realizadas con los niños, así como los resultados en torno a los objetivos educativos planteados. También se incluye el análisis sobre la interacción de los estudiantes con el dispositivo, así como las ventajas y desventajas encontradas en todo el proceso

Improving Pre-Kindergarten Touch Performance

© ACM, 2014. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (pp. 163-166). http://doi.acm.org/10.1145/2669485.2669498Multi-touch technology provides users with a more intuitive way of interaction. However, pre-kindergarten children, a growing group of potential users, have problems with some basic gestures according to previous studies. This is particularly the case of the double tap and long pressed gestures, which have some issues related to spurious entry events and time-constrained interactions, respectively. In this paper, we empirically test specific strategies to deal with these issues by evaluating off-the-shelf implementations of these gestures against alternative implementations that follow these guidelines. The study shows that the implementation of these design guidelines has a positive effect on success rates of these two gestures, being feasible their inclusion in future multi-touch applications targeted at pre-kindergarten children.This work received financial support from the Spanish Ministry of Education under the National Strategic Program of Research and Projects TIN2010-20488 (CREATEWORLD) and TIN2012-34003 (insPIre). This work is also supported by a postdoctoral fellowship within the VALi+d program from Conselleria d’Educació, Cultura i Esport (Generalitat Valenciana) to A. Catalá (APOSTD/2013/013).Nácher Soler, VE.; Jaén Martínez, FJ.; Catalá Bolós, A.; Navarro, E.; González, P. (2014). Improving Pre-Kindergarten Touch Performance. ACM. https://doi.org/10.1145/2669485.2669498SCouse, L.J. and Chen, D.W. A Tablet Computer for Young Children? Exploring Its Viability for Early Childhood Education. Journal of Research on Technology in Education 43, 1 (2010), 75--98.Harris, A., Rick, J., Bonnett, V., et al. Around the table: are multiple-touch surfaces better than single-touch for children's collaborative interactions? In Proc. CSCL'09, 335--344.Hoggan, E., Nacenta, M., Kristensson, P.O., Williamson, J., Oulasvirta, A., and Lehtiö, A. MultiTouch Pinch Gestures: Performance and Ergonomics. In Proc. ITS'13, 219--222.Hoggan, E., Williamson, J., Oulasvirta, A., Nacenta, M., Kristensson, P.O., and Lehtiö, A. Multi-Touch Rotation Gestures: Performance and Ergonomics. In Proc. CHI'13, 3--6.Nacenta, M.A., Baudisch, P., Benko, H., and Wilson, A. Separability of Spatial Manipulations in Multi-touch Interfaces. In Proc. GI'09, 175--182.Nacher, V., Jaen, J., Navarro, E., Catala, A., and González, P. Multi-touch gestures for pre-kindergarten children. International Journal of Human-Computer Studies. Available online http://dx.doi.org/10.1016/j.ijhcs.2014.08.004Rideout, V. Zero to Eight: Children's Media Use in America. Common Sense Media, 2011.Smith, S.P., Burd, E., and Rick, J. Developing, evaluating and deploying multi-touch systems. International Journal of Human-Computer Studies 70, 10 (2012), 653--656.Terra, D., Brinkman, W.P., and Heynderickx, I. Ease-ofUse and Enjoyment of Traditional vs. Stylus Input for Children in a Brazilian Primary School. LatinDisplay, (2009), 151--155

Enhanced device-based 3D object manipulation technique for handheld mobile augmented reality

3D object manipulation is one of the most important tasks for handheld mobile Augmented Reality (AR) towards its practical potential, especially for realworld assembly support. In this context, techniques used to manipulate 3D object is an important research area. Therefore, this study developed an improved device based interaction technique within handheld mobile AR interfaces to solve the large range 3D object rotation problem as well as issues related to 3D object position and orientation deviations in manipulating 3D object. The research firstly enhanced the existing device-based 3D object rotation technique with an innovative control structure that utilizes the handheld mobile device tilting and skewing amplitudes to determine the rotation axes and directions of the 3D object. Whenever the device is tilted or skewed exceeding the threshold values of the amplitudes, the 3D object rotation will start continuously with a pre-defined angular speed per second to prevent over-rotation of the handheld mobile device. This over-rotation is a common occurrence when using the existing technique to perform large-range 3D object rotations. The problem of over-rotation of the handheld mobile device needs to be solved since it causes a 3D object registration error and a 3D object display issue where the 3D object does not appear consistent within the user’s range of view. Secondly, restructuring the existing device-based 3D object manipulation technique was done by separating the degrees of freedom (DOF) of the 3D object translation and rotation to prevent the 3D object position and orientation deviations caused by the DOF integration that utilizes the same control structure for both tasks. Next, an improved device-based interaction technique, with better performance on task completion time for 3D object rotation unilaterally and 3D object manipulation comprehensively within handheld mobile AR interfaces was developed. A pilot test was carried out before other main tests to determine several pre-defined values designed in the control structure of the proposed 3D object rotation technique. A series of 3D object rotation and manipulation tasks was designed and developed as separate experimental tasks to benchmark both the proposed 3D object rotation and manipulation techniques with existing ones on task completion time (s). Two different groups of participants aged 19-24 years old were selected for both experiments, with each group consisting sixteen participants. Each participant had to complete twelve trials, which came to a total 192 trials per experiment for all the participants. Repeated measure analysis was used to analyze the data. The results obtained have statistically proven that the developed 3D object rotation technique markedly outpaced existing technique with significant shorter task completion times of 2.04s shorter on easy tasks and 3.09s shorter on hard tasks after comparing the mean times upon all successful trials. On the other hand, for the failed trials, the 3D object rotation technique was 4.99% more accurate on easy tasks and 1.78% more accurate on hard tasks in comparison to the existing technique. Similar results were also extended to 3D object manipulation tasks with an overall 9.529s significant shorter task completion time of the proposed manipulation technique as compared to the existing technique. Based on the findings, an improved device-based interaction technique has been successfully developed to address the insufficient functionalities of the current technique

An Exploration of Multi-touch Interaction Techniques

Research in multi-touch interaction has typically been focused on direct spatial manipulation; techniques have been created to result in the most intuitive mapping between the movement of the hand and the resultant change in the virtual object. As we attempt to design for more complex operations, the effectiveness of spatial manipulation as a metaphor becomes weak. We introduce two new platforms for multi-touch computing: a gesture recognition system, and a new interaction technique. I present Multi-Tap Sliders, a new interaction technique for operation in what we call non-spatial parametric spaces. Such spaces do not have an obvious literal spatial representation, (Eg.: exposure, brightness, contrast and saturation for image editing). The multi-tap sliders encourage the user to keep her visual focus on the tar- get, instead of requiring her to look back at the interface. My research emphasizes ergonomics, clear visual design, and fluid transition between modes of operation. Through a series of iterations, I develop a new technique for quickly selecting and adjusting multiple numerical parameters. Evaluations of multi-tap sliders show improvements over traditional sliders. To facilitate further research on multi-touch gestural interaction, I developed mGestr: a training and recognition system using hidden Markov models for designing a multi-touch gesture set. Our evaluation shows successful recognition rates of up to 95%. The recognition framework is packaged into a service for easy integration with existing applications

Kindertivity: Usability and Communicability Strategies for Interactive Surfaces and Pre-Kindergarten Children

Tesis por compendio[ES] La tecnología multi-táctil se ha convertido en una de las más emergentes tras experimentar un enorme crecimiento desde sus pasos iniciales en los años ochenta hasta su amplia aceptación y uso en la actualidad. Por una parte, la tecnología multi-táctil se basa en el estilo de interacción de manipulación directa el cual proporciona a los usuarios la ventaja de ver los objetos y las acciones de interés, sustituir comandos escritos por acciones de señalado y, además, permite la realización de acciones rápidas, reversibles e incrementales evitando el uso de instrucciones complejas. Por otra parte, diversos trabajos han evaluado las virtudes derivadas de utilizar conjuntamente la manipulación directa con el toque directo mostrando que es posible evitar los problemas inherentes a otras técnicas de interacción como el ratón y el teclado. Por lo tanto, aprovechando la interacción natural e intuitiva proporcionada por la tecnología multi-táctil, ésta parece una forma ideal para dar soporte a la creación de escenarios educativos dirigidos a niños en edad preescolar. Sin embargo, a pesar de la existencia de diversos estudios que evalúan la idoneidad de utilizar el estilo de interacción de manipulación directa, existe una falta de trabajos abordando el uso dispositivos basados en superficies táctiles con niños de una temprana edad. Asimismo, en la actualidad existe una creciente tendencia a diseñar aplicaciones educativas y lúdicas dirigidas a niños en edad preescolar utilizando dispositivos multi-táctiles como los teléfonos inteligentes o las tabletas. Además, diversos informes señalan que los niños son usuarios frecuentes de este tipo de dispositivos y los utilizan incluso antes de ser capaces de hablar. Sin embargo, a pesar de este crecimiento en el uso de la tecnología multi-táctil y su aparente idoneidad para ser utilizado en el desarrollo de aplicaciones educativas para niños en edad preescolar, no existen unas interacciones universales y estandarizadas para preescolares a la hora de utilizar dispositivos táctiles ya que habitualmente sólo se utilizan dos gestos básicos (básicamente, el toque con un dedo para seleccionar y el arrastre con un dedo para el movimiento). Por lo tanto, existe una clara necesidad de llevar a cabo estudios empíricos para contribuir y avanzar en el diseño de aplicaciones que den un soporte adecuado y encaje con las habilidades de los niños en su temprano desarrollo. Por tanto, esta tesis propone, diseña y evalúa diversas estrategias de usabilidad y comunicabilidad adaptadas a los niños en edad preescolar para establecer la base para el diseño y desarrollo de futuras aplicaciones basadas en dispositivos táctiles dirigidas a preescolares. Estas estrategias llevarán a la adecuada definición de guías de diseño que permitirán a los niños aprovechar al máximo la tecnología multi-táctil, harán posible el desarrollo de nuevas y atractivas aplicaciones y, eventualmente, también podrán ayudar al desarrollo cognitivo y motor de los niños.[CA] La tecnologia multi-tàctil s'ha convertit en una de les més emergents després d'experimentar un enorme creixement des dels seus passos inicials als anys vuitanta fins l'actualitat on es àmpliament acceptada i utilitzada. D'una banda, la tecnologia multi-tàctil es basa en l'estil d'interacció de manipulació directa, el qual proporciona als usuaris l'avantatge de veure els objectes i les accions d'interès, substituir comandos escrits per accions d'assenyalament i, a més, permet la realització d'accions, ràpides, reversibles i incrementals evitant l'ús d'instruccions complexes. D'altra banda, diversos treballs han avaluat les virtuts derivades d'utilitzar conjuntament la manipulació directa amb el toc directe mostrant que és possible evitar els problemes inherents a altres tècniques d'interacció com el ratolí i el teclat. Per tant, aprofitant la interacció natural i intuïtiva proporcionada per la tecnologia multi-tàctil, aquesta sembla una forma ideal per donar suport a la creació d'escenaris educatius per a xiquets en edat preescolar. No obstant això, malgrat l'existència de diversos estudis que avaluen la idoneïtat d'utilitzar l'estil d'interacció de manipulació directa, existeix una manca de treballs abordant l'ús de dispositius basats en superfícies tàctils amb xiquets d'edat primerenca. Així mateix, en l'actualitat existeix una creixent tendència a dissenyar aplicacions educatives i lúdiques dirigides a xiquets en edat preescolar utilitzant dispositius tàctils com els telèfons intel¿ligents o les tauletes. A més, diversos informes assenyalen que els xiquets són usuaris freqüents d'aquests tipus de dispositius i els utilitzen fins i tot abans de ser capaços de parlar. Malgrat aquest creixement en l'ús de la tecnologia multi-tàctil i la seua aparent idoneïtat per a ser utilitzada en el desenvolupament d'aplicacions educatives per a xiquets en edat preescolar, no existeixen unes interaccions universals i estandarditzades per a preescolars a l'hora d'utilitzar dispositius tàctils ja que habitualment només s'utilitzen dos gestos bàsics (bàsicament, el toc amb un dit per a seleccionar i l'arrossegament amb un dit per al moviment). Per tant, hi ha una clara necessitat de dur a terme estudis empírics per a contribuir i avançar en el disseny d'aplicacions que donen un suport adequat i s'ajusten amb les habilitats dels xiquets en el seu primerenc desenvolupament. Per tant, la tesi proposa, dissenya i avalua diverses estratègies de usabilitat i comunicabilitat adaptades als xiquets en edat preescolar per tal d'establir la base per al disseny i desenvolupament de futures aplicacions basades en dispositius tàctils dirigides a preescolars. Aquestes estratègies portaran a l'adequada definició de guies de disseny que permetran als xiquets aprofitar al màxim la tecnologia multi-tàctil, faran possible el desenvolupament de noves i atractives aplicacions i, eventualment, podran també ajudar al desenvolupament cognitiu i motor dels xiquets.[EN] Multi-touch technology has become one of the most emergent technologies and has had an enormous growth since its initial steps in the eighties to be widespread accepted and used in the present. On the one hand, multi-touch technology relies on the direct manipulation interaction style which gives users the advantage to view the objects and actions of interest, replace typed commands by pointing actions and to perform rapid, reversible and incremental actions avoiding using complex instructions. On the other hand, several works have evaluated the virtues when joining direct manipulation with direct-touching showing that it solves the problems inherent in other interaction devices, such as those involving mouse or keyboard. Hence, taking advantage of the intuitive and natural interaction provided by multi-touch technology it seems an ideal way to support educational scenarios targeted to kindergarten children. Although several works have assessed the suitability of using the direct manipulation style with children, there is a lack of works addressing the use of touchscreen devices by this specific type of users. Moreover, there is a growing trend of designing educational and playful applications targeted to kindergarten children based on touchscreen devices such as smartphones and tablets. In addition, several reports point out that children use touchscreen devices even before they are able to speak and they are frequent users of devices such as smartphones and tablets. However, despite this growth in the use of multi-touch technology by children and its apparent suitability to be used to develop applications targeted to young children, there is a lack of standardized and universally accepted interactions for young children when using touchscreen devices since only two basic gestures are commonly used (basically, consisting of only one-finger touch for selection and one-finger drag for movement). Hence, there is a need of carrying out empirical studies to help and advance in the design of applications that adequately support and fit with children's development and skills. Therefore, this thesis proposes, designs and evaluates several usability and communicability strategies tailored to children in their early development stage to establish the design and development of future applications targeted to kindergarten children. These strategies will lead to define appropriate design strategies that enable infants to take full advantage of multi-touch technology, would make it possible to develop attractive new applications and, eventually, could also aid children's cognitive and motor development.Nácher Soler, VE. (2019). Kindertivity: Usability and Communicability Strategies for Interactive Surfaces and Pre-Kindergarten Children [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/116833TESISCompendi

3D-Modellierung mit interaktiven Oberflächen

3D models are at the core of many important applications from industry, science, and also entertainment. The creation of 3D models is a complex and time consuming process. Current modeling tools are hard to learn and require a deep understanding of the underlying mathematical models. Furthermore, established input devices like the mouse and keyboard do not utilize the full interaction potential -- especially regarding bimanual control -- of the human hand. The growing interest and the commercial breakthrough of multi-touch displays and interactive surfaces raises questions about their potential in the context of 3d modeling, which are thoroughly discussed and evaluated in this work. The presented approach is closely aligned to the whole processing chain for multi-touch applications, starting with the hardware and tracking issues, continuing with fundamental design discussions and operations like selection and 3D manipulation of objects and finishing with complex modeling techniques and metaphors. In regard to hardware and tracking, a robust illumination setup for the diffuse illumination technique is presented along with two extensions of this approach, i.e., hover detection and hand distinction. The design space is organized into specific design dimensions characterized by extremal positions to allow a better overview of design choices and a classification of existing and future systems. Fundamental techniques for selection and integrated 3D manipulation with six degrees of freedom are presented and empirically evaluated. Finally, two established modeling techniques -- implicit surfaces and virtual sculpting -- are extended and evaluated for multi-touch input

Separability of spatial manipulations in multi-touch interfaces

Multi-touch interfaces allow users to translate, rotate, and scale digital objects in a single interaction. However, this freedom represents a problem when users intend to perform only a subset of manipulations. A user trying to scale an object in a print layout program, for example, might find that the object was also slightly translated and rotated, interfering with what was already carefully laid out earlier. We implemented and tested interaction techniques that allow users to select a subset of manipulations. Magnitude Filtering eliminates transformations (e.g., rotation) that are small in magnitude. Gesture Matching attempts to classify the user’s input into a subset of manipulation gestures. Handles adopts a conventional single-touch handles approach for touch input. Our empirical study showed that these techniques significantly reduce errors i