An Immersive Multitouch Workspace

International audienceWe propose a new system that efficiently combines direct multitouch interaction and 3D stereoscopic visualization. In our approach, users interact by way of simple 2D gestures on a monoscopic touchscreen, while visualizing occlusion-free 3D stereoscopic objects floating above the surface at an optically correct distance. By registering the 3D virtual space with the physical space, we produce a rich seamless workspace where both the advantages of direct and indirect interaction are jointly exploited. In addition to standard multitouch gestures and controls (e.g. pan, zoom, and standard 2D widgets) from which we take advantage, we have designed a dedicated multitouch 3D transformation widget. This widget allows the near-direct control of rotations, scaling, and translations of the manipulated objects. To illustrate the power of our setup, we have designed a demo scenario where participants reassemble 3D virtual fragments. This scenario, as many others, takes benefit of our proposal, where the strength of both multitouch interaction and stereoscopic visualization are unified in an innovative and relevant workspace

Toucheo: Multitouch and Stereo Combined in a Seamless Workspace.

International audienceWe propose a new system that efficiently combines direct multitouch interaction with co-located 3D stereoscopic visualization. In our approach, users benefit from well-known 2D metaphors and widgets displayed on a monoscopic touchscreen, while visualizing occlusion-free 3D objects floating above the surface at an optically correct distance. Technically, a horizontal semi-transparent mirror is used to reflect 3D images produced by a stereoscopic screen, while the user's hand as well as a multitouch screen located below this mirror remain visible. By registering the 3D virtual space and the physical space, we produce a rich and unified workspace where users benefit simultaneously from the advantages of both direct and indirect interaction, and from 2D and 3D visualizations. A pilot usability study shows that this combination of technology provides a good user experience

Tabletop Tangible Interfaces for Music Performance: Design and Evaluation

This thesis investigates a new generation of collaborative systems: tabletop tangible interfaces (TTIs) for music performance or musical tabletops. Musical tabletops are designed for professional musical performance, as well as for casual interaction in public settings. These systems support co-located collaboration, offered by a shared interface. However, we still know little about their challenges and opportunities for collaborative musical practice: in particular, how to best support beginners or experts or both. This thesis explores the nature of collaboration on TTIs for music performance between beginners, experts, or both. Empirical work was done in two stages: 1) an exploratory stage; and 2) an experimental stage. In the exploratory stage we studied the Reactable, a commercial musical tabletop designed for beginners and experts. In particular, we explored its use in two environments: a multi-session study with expert musicians in a casual lab setting; and a field study with casual visitors in a science centre. In the experimental stage we conducted a controlled experiment for mixed groups using a bespoke musical tabletop interface, SoundXY4. The design of this study was informed by the previous stage about a need to support better real-time awareness of the group activity (workspace awareness) in early interactions. For the three studies, groups musical improvisation was video-captured unobtrusively with the aim of understanding natural uses during group musical practice. Rich video data was carefully analysed focusing on the nature of social interaction and how workspace awareness was manifested. The findings suggest that musical tabletops can support peer learning during multiple sessions; fluid between-group social interaction in public settings; and a democratic and ecological approach to music performance. The findings also point to how workspace awareness can be enhanced in early interactions with TTIs using auditory feedback with ambisonics spatialisation. The thesis concludes with theoretical, methodological, and practical implications for future research in New Interfaces for Musical Expression (NIME), tabletop studies, and Human-Computer Interaction (HCI)

Modeling On and Above a Stereoscopic Multitouch Display

International audienceWe present a semi-immersive environment for conceptual design where virtual mockups are obtained from gestures we aim to get closer to the way people conceive, create and manipulate three-dimensional shapes. We developed on-and-above-the-surface interaction techniques based on asymmetric bimanual interaction for creating and editing 3D models in a stereoscopic environment. Our approach combines hand and nger tracking in the space on and above a multitouch surface. This combination brings forth an alternative design environment where users can seamlessly switch between interacting on the surface or in the space above it to leverage the bene t of both interaction spaces

Virtual Valcamonica: collaborative exploration of prehistoric petroglyphs and their surrounding environment in multi-user virtual reality

In this paper, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rock-art and allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques

Sensor Augmented Large Interactive Surfaces

Large interactive surfaces enable effective multi-user collaboration, but a majority of the current multi-touch systems are not truly multi-user. In this work we present a novel sensor-based approach for both user identification around a touch table and integration of unique gestures above the table. The work proposes the criteria for a successful and robust user identification system. The Cricket sensor based user identification system is integrated with an open source gesture recognition system Sparsh-UI to enable rapid multi-touch application development. Finally we evaluate the Cricket-based algorithm with contemporary multi-user, multi-touch systems and describe the various interaction affordances provided by the Cricket based user identification system

Virtual Collaborative Design Environment: Information structure and interfaces

The failure to identify design issues in early phases of construction projects has been identified as a significant cause of costly rework, as these issues can impact the building occupants’ abilities to efficiently perform their daily work tasks. Therefore, it is crucial to consider their feedback when design reviewing. To date, efforts have been made to involve building occupants via a variety of user-interfaces that provide different understandings of the project. One such example is Virtual Reality (VR), which increases building occupants’ spatial understanding. Another, is use of design guidelines, intended to support both end-users such as building occupants and also the design team in basing their decision-making on best-practice and ensuring compliance with design requirements. When used together, these different user-interfaces can complement each other by enabling, for instance, visualization of the furniture layout depicted in design guideline documents. However, few studies have identified what is required of a design tool capable of supporting both visualization of design and design-compliance via different user-interfaces. Therefore, the aim of this thesis is to advance the understanding of end-users’ involvement in virtual collaborative environments in the building design process. Accordingly, Design Science Research was applied with a two-fold purpose. First, to identify different stakeholders’ challenges that are faced in the design process and specifically how building occupants’ daily work tasks are considered in the design process. Secondly, the research methods such as workshops, semi-structured interviews and documentation analysis helped identify the requirements of a design tool that would enable this knowledge to be transferred and accessible at a cross-project level. The results show that the information structure and user-interface of design guidelines determine to a large degree how effectively compliance with requirements can be validated. An example is the absence of user-interfaces in design guidelines which prevents building occupants from gaining sufficient spatial understanding. This lack of spatial understanding results in them to being reliant on other project members, such as architects and facility planners, for providing input on the design. Moreover, the results show how cross-project knowledge is difficult to facilitate due to how design guidelines have not been created in relation to today’s digital design process. Therefore, this thesis bridges the concepts of integrating design guidelines and VR in the same design tool

A community environment design hypertool. Application to sustainable cities interaction regulation

International audienceThis paper develops an innovative structure of knowledge collaboration forms that allows effective and creative collaboration among the actors along territorial design process. To contribute to the development of the social dimension of the sustainable development, we investigate a new methodology and computational tools for understanding and facilitating multi-disciplinary, cross-cultural collaboration in sustainable architectural-, urban- land- and ambient-scape design, through evaluating collaboration among project developers, local authorities, architects, engineers, contractors, building inspectors, clients and future users. Therefore, this proposed integrative tool increases the pool of knowledge and skills available to undertake activities that exceed the capacities of any single person or organization on its own, thus making possible the design and construction of complex skills, especially buildings and ambiences in urban or suburban environmen