41 research outputs found

    Phrasing Bimanual Interaction for Visual Design

    Get PDF
    Architects and other visual thinkers create external representations of their ideas to support early-stage design. They compose visual imagery with sketching to form abstract diagrams as representations. When working with digital media, they apply various visual operations to transform representations, often engaging in complex sequences. This research investigates how to build interactive capabilities to support designers in putting together, that is phrasing, sequences of operations using both hands. In particular, we examine how phrasing interactions with pen and multi-touch input can support modal switching among different visual operations that in many commercial design tools require using menus and tool palettes—techniques originally designed for the mouse, not pen and touch. We develop an interactive bimanual pen+touch diagramming environment and study its use in landscape architecture design studio education. We observe interesting forms of interaction that emerge, and how our bimanual interaction techniques support visual design processes. Based on the needs of architects, we develop LayerFish, a new bimanual technique for layering overlapping content. We conduct a controlled experiment to evaluate its efficacy. We explore the use of wearables to identify which user, and distinguish what hand, is touching to support phrasing together direct-touch interactions on large displays. From design and development of the environment and both field and controlled studies, we derive a set methods, based upon human bimanual specialization theory, for phrasing modal operations through bimanual interactions without menus or tool palettes

    Tangible user interfaces : past, present and future directions

    Get PDF
    In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this ïŹeld. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

    Interactions gestuelles multi-point et gĂ©omĂ©trie dĂ©formable pour l’édition 3D sur Ă©cran tactile

    Get PDF
    Despite the advances made in the fields of existing objects capture and of procedural generation, creation of content for virtual worlds can not be perform without human interaction. This thesis suggests to exploit new touch devices ("multi-touch" screens) to obtain an easy, intuitive 2D interaction in order to navigate inside a virtual environment, to manipulate, position and deform 3D objects.First, we study the possibilities and limitations of the hand and finger gestures while interacting on a touch screen in order to discover which gestures are the most adapted to edit 3D scene and environment. In particular, we evaluate the effective number of degrees of freedom of the human hand when constrained on a planar surface. Meanwhile, we develop a new gesture analysis method using phases to identify key motion of the hand and fingers in real time. These results, combined to several specific user-studies, lead to a gestural design pattern which handle not only navigation (camera positioning), but also object positioning, rotation and global scaling. Then, this pattern is extended to complex deformation (such as adding and deleting material, bending or twisting part of objects, using local control). Using these results, we are able to propose and evaluate a 3D world editing interface that handle a naturaltouch interaction, in which mode selection (i.e. navigation, object positioning or object deformation) and task selections is automatically processed by the system, relying on the gesture and the interaction context (without any menu or button). Finally, we extend this interface to integrate more complex deformations, adapting the garment transfer from a character to any other in order to process interactive deformation of the garment while the wearing character is deformed.MalgrĂ© les progrĂšs en capture d’objets rĂ©els et en gĂ©nĂ©ration procĂ©durale, la crĂ©ation de contenus pour les mondes virtuels ne peut se faire sans interaction humaine. Cette thĂšse propose d’exploiter les nouvelles technologies tactiles (Ă©crans "multi-touch") pour offrir une interaction 2D simple et intuitive afin de naviguer dans un environnement virtuel, et d’y manipuler, positionner et dĂ©former des objets 3D.En premier lieu, nous Ă©tudions les possibilitĂ© et les limitations gestuelles de la main et des doigts lors d’une interaction sur Ă©cran tactile afin de dĂ©couvrir quels gestes semblent les plus adaptĂ©s Ă  l’édition des environnements et des objets 3D. En particulier, nous Ă©valuons le nombre de degrĂ© de libertĂ© efficaces d’une main humaine lorsque son geste est contraint Ă  une surface plane. Nous proposons Ă©galement une nouvelle mĂ©thode d’analyse gestuelle par phases permettant d’identifier en temps rĂ©el les mouvements clĂ©s de la main et des doigts. Ces rĂ©sultats, combinĂ©s Ă  plusieurs Ă©tudes utilisateur spĂ©cifiques, dĂ©bouchent sur l’identification d’un patron pour les interactions gestuelles de base incluant non seulement navigation (placement de camĂ©ra), mais aussi placement, rotation et mise Ă  l’échelle des objets. Ce patron est Ă©tendudans un second temps aux dĂ©formations complexes (ajout et suppression de matiĂšre ainsi que courbure ou torsion des objets, avec contrĂŽle de la localitĂ©). Tout ceci nous permet de proposer et d’évaluer une interface d’édition des mondes 3D permettant une interaction tactile naturelle, pour laquelle le choix du mode (navigation, positionnement ou dĂ©formation) et des tĂąches correspondantes est automatiquement gĂ©rĂ© par le systĂšme en fonction du geste et de son contexte (sans menu ni boutons). Enfin, nous Ă©tendons cette interface pour y intĂ©grer des dĂ©formations plus complexe Ă  travers le transfert de vĂȘtements d’un personnage Ă  un autre, qui est Ă©tendu pour permettre la dĂ©formation interactive du vĂȘtement lorsque le personnage qui le porte est dĂ©formĂ© par interaction tactile

    Blending the Material and Digital World for Hybrid Interfaces

    Get PDF
    The development of digital technologies in the 21st century is progressing continuously and new device classes such as tablets, smartphones or smartwatches are finding their way into our everyday lives. However, this development also poses problems, as these prevailing touch and gestural interfaces often lack tangibility, take little account of haptic qualities and therefore require full attention from their users. Compared to traditional tools and analog interfaces, the human skills to experience and manipulate material in its natural environment and context remain unexploited. To combine the best of both, a key question is how it is possible to blend the material world and digital world to design and realize novel hybrid interfaces in a meaningful way. Research on Tangible User Interfaces (TUIs) investigates the coupling between physical objects and virtual data. In contrast, hybrid interfaces, which specifically aim to digitally enrich analog artifacts of everyday work, have not yet been sufficiently researched and systematically discussed. Therefore, this doctoral thesis rethinks how user interfaces can provide useful digital functionality while maintaining their physical properties and familiar patterns of use in the real world. However, the development of such hybrid interfaces raises overarching research questions about the design: Which kind of physical interfaces are worth exploring? What type of digital enhancement will improve existing interfaces? How can hybrid interfaces retain their physical properties while enabling new digital functions? What are suitable methods to explore different design? And how to support technology-enthusiast users in prototyping? For a systematic investigation, the thesis builds on a design-oriented, exploratory and iterative development process using digital fabrication methods and novel materials. As a main contribution, four specific research projects are presented that apply and discuss different visual and interactive augmentation principles along real-world applications. The applications range from digitally-enhanced paper, interactive cords over visual watch strap extensions to novel prototyping tools for smart garments. While almost all of them integrate visual feedback and haptic input, none of them are built on rigid, rectangular pixel screens or use standard input modalities, as they all aim to reveal new design approaches. The dissertation shows how valuable it can be to rethink familiar, analog applications while thoughtfully extending them digitally. Finally, this thesis’ extensive work of engineering versatile research platforms is accompanied by overarching conceptual work, user evaluations and technical experiments, as well as literature reviews.Die Durchdringung digitaler Technologien im 21. Jahrhundert schreitet stetig voran und neue GerĂ€teklassen wie Tablets, Smartphones oder Smartwatches erobern unseren Alltag. Diese Entwicklung birgt aber auch Probleme, denn die vorherrschenden berĂŒhrungsempfindlichen OberflĂ€chen berĂŒcksichtigen kaum haptische QualitĂ€ten und erfordern daher die volle Aufmerksamkeit ihrer Nutzer:innen. Im Vergleich zu traditionellen Werkzeugen und analogen Schnittstellen bleiben die menschlichen FĂ€higkeiten ungenutzt, die Umwelt mit allen Sinnen zu begreifen und wahrzunehmen. Um das Beste aus beiden Welten zu vereinen, stellt sich daher die Frage, wie neuartige hybride Schnittstellen sinnvoll gestaltet und realisiert werden können, um die materielle und die digitale Welt zu verschmelzen. In der Forschung zu Tangible User Interfaces (TUIs) wird die Verbindung zwischen physischen Objekten und virtuellen Daten untersucht. Noch nicht ausreichend erforscht wurden hingegen hybride Schnittstellen, die speziell darauf abzielen, physische GegenstĂ€nde des Alltags digital zu erweitern und anhand geeigneter Designparameter und EntwurfsrĂ€ume systematisch zu untersuchen. In dieser Dissertation wird daher untersucht, wie MaterialitĂ€t und DigitalitĂ€t nahtlos ineinander ĂŒbergehen können. Es soll erforscht werden, wie kĂŒnftige Benutzungsschnittstellen nĂŒtzliche digitale Funktionen bereitstellen können, ohne ihre physischen Eigenschaften und vertrauten Nutzungsmuster in der realen Welt zu verlieren. Die Entwicklung solcher hybriden AnsĂ€tze wirft jedoch ĂŒbergreifende Forschungsfragen zum Design auf: Welche Arten von physischen Schnittstellen sind es wert, betrachtet zu werden? Welche Art von digitaler Erweiterung verbessert das Bestehende? Wie können hybride Konzepte ihre physischen Eigenschaften beibehalten und gleichzeitig neue digitale Funktionen ermöglichen? Was sind geeignete Methoden, um verschiedene Designs zu erforschen? Wie kann man Technologiebegeisterte bei der Erstellung von Prototypen unterstĂŒtzen? FĂŒr eine systematische Untersuchung stĂŒtzt sich die Arbeit auf einen designorientierten, explorativen und iterativen Entwicklungsprozess unter Verwendung digitaler Fabrikationsmethoden und neuartiger Materialien. Im Hauptteil werden vier Forschungsprojekte vorgestellt, die verschiedene visuelle und interaktive Prinzipien entlang realer Anwendungen diskutieren. Die Szenarien reichen von digital angereichertem Papier, interaktiven Kordeln ĂŒber visuelle Erweiterungen von UhrarmbĂ€ndern bis hin zu neuartigen Prototyping-Tools fĂŒr intelligente KleidungsstĂŒcke. Um neue DesignansĂ€tze aufzuzeigen, integrieren nahezu alle visuelles Feedback und haptische Eingaben, um Alternativen zu Standard-EingabemodalitĂ€ten auf starren Pixelbildschirmen zu schaffen. Die Dissertation hat gezeigt, wie wertvoll es sein kann, bekannte, analoge Anwendungen zu ĂŒberdenken und sie dabei gleichzeitig mit Bedacht digital zu erweitern. Dabei umfasst die vorliegende Arbeit sowohl realisierte technische Forschungsplattformen als auch ĂŒbergreifende konzeptionelle Arbeiten, Nutzerstudien und technische Experimente sowie die Analyse existierender Forschungsarbeiten

    Digital Fabrication Approaches for the Design and Development of Shape-Changing Displays

    Get PDF
    Interactive shape-changing displays enable dynamic representations of data and information through physically reconfigurable geometry. The actuated physical deformations of these displays can be utilised in a wide range of new application areas, such as dynamic landscape and topographical modelling, architectural design, physical telepresence and object manipulation. Traditionally, shape-changing displays have a high development cost in mechanical complexity, technical skills and time/finances required for fabrication. There is still a limited number of robust shape-changing displays that go beyond one-off prototypes. Specifically, there is limited focus on low-cost/accessible design and development approaches involving digital fabrication (e.g. 3D printing). To address this challenge, this thesis presents accessible digital fabrication approaches that support the development of shape-changing displays with a range of application examples – such as physical terrain modelling and interior design artefacts. Both laser cutting and 3D printing methods have been explored to ensure generalisability and accessibility for a range of potential users. The first design-led content generation explorations show that novice users, from the general public, can successfully design and present their own application ideas using the physical animation features of the display. By engaging with domain experts in designing shape-changing content to represent data specific to their work domains the thesis was able to demonstrate the utility of shape-changing displays beyond novel systems and describe practical use-case scenarios and applications through rapid prototyping methods. This thesis then demonstrates new ways of designing and building shape-changing displays that goes beyond current implementation examples available (e.g. pin arrays and continuous surface shape-changing displays). To achieve this, the thesis demonstrates how laser cutting and 3D printing can be utilised to rapidly fabricate deformable surfaces for shape-changing displays with embedded electronics. This thesis is concluded with a discussion of research implications and future direction for this work

    Remixing physical objects through tangible tools

    Get PDF
    Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2011.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 147-164).In this document we present new tools for remixing physical objects. These tools allow users to copy, edit and manipulate the properties of one or more objects to create a new physical object. We already have these capabilities using digital media: we can easily mash up videos, music and text. However, it remains difficult to remix physical objects and we cannot access the advantages of digital media, which are nondestructive, scalable and scriptable. We can bridge this gap by both integrating 2D and 3D scanning technology into design tools and employing aordable rapid prototyping technology to materialize these remixed objects. In so doing, we hope to promote copying as a tool for creation. This document presents two tools, CopyCAD and KidCAD, the first designed for makers and crafters, the second for children. CopyCAD is an augmented Computer Numerically Controlled (CNC) milling machine which allows users to copy arbitrary real world object geometry into 2D CAD designs at scale through the use of a camera-projector system. CopyCAD gathers properties from physical objects, sketches and touch interactions directly on a milling machine, allowing novice users to copy parts of real world objects, modify them and create a new physical part. KidCAD is a sculpting interface built on top of a gel-based realtime 2.5D scanner. It allows children to stamp objects into the block of gel, which are scanned in realtime, as if they were stamped into clay. Children can use everyday objects, their hands and tangible tools to design new toys or objects that will be 3D printed. This work enables novice users to easily approach designing physical objects by copying from other objects and sketching new designs. With increased access to such tools we hope that a wide range of people will be empowered to create their own objects, toys, tools and parts.by Sean Follmer.S.M

    The tool space

    Get PDF
    Visions of futuristic desktop computer work spaces have often incorporated large interactive surfaces that either integrate into or replace the prevailing desk setup with displays, keyboard and mouse. Such visions often connote the distinct characteristics of direct touch interaction, e.g. by transforming the desktop into a large touch screen that allows interacting with content using one’s bare hands. However, the role of interactive surfaces for desktop computing may not be restricted to enabling direct interaction. Especially for prolonged interaction times, the separation of visual focus and manual input has proven to be ergonomic and is usually supported by vertical monitors and separate – hence indirect – input devices placed on the horizontal desktop. If we want to maintain this ergonomically matured style of computing with the introduction of interactive desktop displays, the following question arises: How can and should this novel input and output modality affect prevailing interaction techniques. While touch input devices have been used for decades in desktop computing as track pads or graphic tablets, the dynamic rendering of content and increasing physical dimensions of novel interactive surfaces open up new design opportunities for direct, indirect and hybrid touch input techniques. Informed design decisions require a careful consideration of the relationship between input sensing, visual display and applied interaction styles. Previous work in the context of desktop computing has focused on understanding the dual-surface setup as a holistic unit that supports direct touch input and allows the seamless transfer of objects across horizontal and vertical surfaces. In contrast, this thesis assumes separate spaces for input (horizontal input space) and output (vertical display space) and contributes to the understanding of how interactive surfaces can enrich indirect input for complex tasks, such as 3D modeling or audio editing. The contribution of this thesis is threefold: First, we present a set of case studies on user interface design for dual-surface computer workspaces. These case studies cover several application areas such as gaming, music production and analysis or collaborative visual layout and comprise formative evaluations. On the one hand, these case studies highlight the conflict that arises when the direct touch interaction paradigm is applied to dual-surface workspaces. On the other hand, they indicate how the deliberate avoidance of established input devices (i.e. mouse and keyboard) leads to novel design ideas for indirect touch-based input. Second, we introduce our concept of the tool space as an interaction model for dual-surface workspaces, which is derived from a theoretical argument and the previous case studies. The tool space dynamically renders task-specific input areas that enable spatial command activation and increase input bandwidth through leveraging multi-touch and two-handed input. We further present evaluations of two concept implementations in the domains 3D modeling and audio editing which demonstrate the high degrees of control, precision and sense of directness that can be achieved with our tools. Third, we present experimental results that inform the design of the tool space input areas. In particular, we contribute a set of design recommendations regarding the understanding of two-handed indirect multi-touch input and the impact of input area form factors on spatial cognition and navigation performance.Zukunftsvisionen thematisieren zuweilen neuartige, auf großen interaktiven OberflĂ€chen basierende ComputerarbeitsplĂ€tze, wobei etablierte PC-Komponenten entweder ersetzt oder erweitert werden. Oft schwingt bei derartigen Konzepten die Idee von natĂŒrlicher oder direkter Toucheingabe mit, die es beispielsweise erlaubt mit den Fingern direkt auf virtuelle Objekte auf einem großen Touchscreen zuzugreifen. Die Eingabe auf interaktiven OberflĂ€chen muss aber nicht auf direkte Interaktionstechniken beschrĂ€nkt sein. Gerade bei lĂ€ngerer Benutzung ist aus ergonomischer Sicht eine Trennung von visuellem Fokus und manueller Eingabe von Vorteil, wie es zum Beispiel bei der Verwendung von Monitoren und den gĂ€ngigen EingabegerĂ€ten der Fall ist. Soll diese Art der Eingabe auch bei ComputerarbeitsplĂ€tzen unterstĂŒtzt werden, die auf interaktiven OberflĂ€chen basieren, dann stellt sich folgende Frage: Wie wirken sich die neuen Ein- und AusgabemodalitĂ€ten auf vorherrschende Interaktionstechniken aus? Toucheingabe kommt beim klassischen Desktop-Computing schon lange zur Anwendung: Im Gegensatz zu sogenannten Trackpads oder Grafiktabletts eröffnen neue interaktive OberflĂ€chen durch ihre visuellen Darstellungsmöglichkeiten und ihre GrĂ¶ĂŸe neue Möglichkeiten fĂŒr das Design von direkten, indirekten oder hybriden Eingabetechniken. Fundierte Designentscheidungen erfordern jedoch eine sorgfĂ€ltige Auseinandersetzung mit Ein- und Ausgabetechnologien sowie adequaten Interaktionsstilen. Verwandte Forschungsarbeiten haben sich auf eine konzeptuelle Vereinheitlichung von Arbeitsbereichen konzentriert, die es beispielsweise erlaubt, digitale Objekte mit dem Finger zwischen horizontalen und vertikalen Arbeitsbereichen zu verschieben. Im Gegensatz dazu geht die vorliegende Arbeit von logisch und rĂ€umlich getrennten Bereichen aus: Die horizontale interaktive OberflĂ€che dient primĂ€r zur Eingabe, wĂ€hrend die vertikale als Display fungiert. Insbesondere trĂ€gt diese Arbeit zu einem VerstĂ€ndnis bei, wie durch eine derartige Auffassung interaktiver OberflĂ€chen komplexe Aufgaben, wie zum Beispiel 3D-Modellierung oder Audiobearbeitung auf neue und gewinnbringende Art und Weise unterstĂŒtzt werden können. Der wissenschaftliche Beitrag der vorliegenden Arbeit lĂ€sst sich in drei Bereiche gliedern: ZunĂ€chst werden Fallstudien prĂ€sentiert, die anhand konkreter Anwendungen (z.B. Spiele, Musikproduktion, kollaboratives Layout) neuartige Nutzerschnittstellen fĂŒr ComputerarbeitsplĂ€tze explorieren und evaluieren, die horizontale und vertikale interaktive OberflĂ€chen miteinander verbinden. Einerseits verdeutlichen diese Fallstudien verschiedene Konflikte, die bei der Anwendung von direkter Toucheingabe an solchen ComputerarbeitsplĂ€tzen hervorgerufen werden. Andererseits zeigen sie auf, wie der bewusste Verzicht auf etablierte EingabegerĂ€te zu neuen Toucheingabe-Konzepten fĂŒhren kann. In einem zweiten Schritt wird das Toolspace-Konzept als Interaktionsmodell fĂŒr ComputerarbeitsplĂ€tze vorgestellt, die auf einem Verbund aus horizontaler und vertikaler interaktiver OberflĂ€che bestehen. Dieses Modell ergibt sich aus den vorangegangenen Fallstudien und wird zusĂ€tzlich theoretisch motiviert. Der Toolspace stellt anwendungsspezifische und dynamische EingabeflĂ€chen dar, die durch rĂ€umliche Aktivierung und die UnterstĂŒtzung beidhĂ€ndiger Multitouch-Eingabe die Eingabebandbreite erhöhen. Diese Idee wird anhand zweier Fallstudien illustriert und evaluiert, die zeigen, dass dadurch ein hohes Maß an Kontrolle und Genauigkeit erreicht sowie ein GefĂŒhl von Direktheit vermittelt wird. Zuletzt werden Studienergebnisse vorgestellt, die Erkenntnisse zum Entwurf von EingabeflĂ€chen im Tool Space liefern, insbesondere zu den Themen beidhĂ€ndige indirekte Multitouch-Eingabe sowie zum Einfluss von Formfaktoren auf rĂ€umliche Kognition und Navigation
    corecore