Blending the Material and Digital World for Hybrid Interfaces

The development of digital technologies in the 21st century is progressing continuously and new device classes such as tablets, smartphones or smartwatches are finding their way into our everyday lives. However, this development also poses problems, as these prevailing touch and gestural interfaces often lack tangibility, take little account of haptic qualities and therefore require full attention from their users. Compared to traditional tools and analog interfaces, the human skills to experience and manipulate material in its natural environment and context remain unexploited. To combine the best of both, a key question is how it is possible to blend the material world and digital world to design and realize novel hybrid interfaces in a meaningful way. Research on Tangible User Interfaces (TUIs) investigates the coupling between physical objects and virtual data. In contrast, hybrid interfaces, which specifically aim to digitally enrich analog artifacts of everyday work, have not yet been sufficiently researched and systematically discussed. Therefore, this doctoral thesis rethinks how user interfaces can provide useful digital functionality while maintaining their physical properties and familiar patterns of use in the real world. However, the development of such hybrid interfaces raises overarching research questions about the design: Which kind of physical interfaces are worth exploring? What type of digital enhancement will improve existing interfaces? How can hybrid interfaces retain their physical properties while enabling new digital functions? What are suitable methods to explore different design? And how to support technology-enthusiast users in prototyping? For a systematic investigation, the thesis builds on a design-oriented, exploratory and iterative development process using digital fabrication methods and novel materials. As a main contribution, four specific research projects are presented that apply and discuss different visual and interactive augmentation principles along real-world applications. The applications range from digitally-enhanced paper, interactive cords over visual watch strap extensions to novel prototyping tools for smart garments. While almost all of them integrate visual feedback and haptic input, none of them are built on rigid, rectangular pixel screens or use standard input modalities, as they all aim to reveal new design approaches. The dissertation shows how valuable it can be to rethink familiar, analog applications while thoughtfully extending them digitally. Finally, this thesis’ extensive work of engineering versatile research platforms is accompanied by overarching conceptual work, user evaluations and technical experiments, as well as literature reviews.Die Durchdringung digitaler Technologien im 21. Jahrhundert schreitet stetig voran und neue Geräteklassen wie Tablets, Smartphones oder Smartwatches erobern unseren Alltag. Diese Entwicklung birgt aber auch Probleme, denn die vorherrschenden berührungsempfindlichen Oberflächen berücksichtigen kaum haptische Qualitäten und erfordern daher die volle Aufmerksamkeit ihrer Nutzer:innen. Im Vergleich zu traditionellen Werkzeugen und analogen Schnittstellen bleiben die menschlichen Fähigkeiten ungenutzt, die Umwelt mit allen Sinnen zu begreifen und wahrzunehmen. Um das Beste aus beiden Welten zu vereinen, stellt sich daher die Frage, wie neuartige hybride Schnittstellen sinnvoll gestaltet und realisiert werden können, um die materielle und die digitale Welt zu verschmelzen. In der Forschung zu Tangible User Interfaces (TUIs) wird die Verbindung zwischen physischen Objekten und virtuellen Daten untersucht. Noch nicht ausreichend erforscht wurden hingegen hybride Schnittstellen, die speziell darauf abzielen, physische Gegenstände des Alltags digital zu erweitern und anhand geeigneter Designparameter und Entwurfsräume systematisch zu untersuchen. In dieser Dissertation wird daher untersucht, wie Materialität und Digitalität nahtlos ineinander übergehen können. Es soll erforscht werden, wie künftige Benutzungsschnittstellen nützliche digitale Funktionen bereitstellen können, ohne ihre physischen Eigenschaften und vertrauten Nutzungsmuster in der realen Welt zu verlieren. Die Entwicklung solcher hybriden Ansätze wirft jedoch übergreifende Forschungsfragen zum Design auf: Welche Arten von physischen Schnittstellen sind es wert, betrachtet zu werden? Welche Art von digitaler Erweiterung verbessert das Bestehende? Wie können hybride Konzepte ihre physischen Eigenschaften beibehalten und gleichzeitig neue digitale Funktionen ermöglichen? Was sind geeignete Methoden, um verschiedene Designs zu erforschen? Wie kann man Technologiebegeisterte bei der Erstellung von Prototypen unterstützen? Für eine systematische Untersuchung stützt sich die Arbeit auf einen designorientierten, explorativen und iterativen Entwicklungsprozess unter Verwendung digitaler Fabrikationsmethoden und neuartiger Materialien. Im Hauptteil werden vier Forschungsprojekte vorgestellt, die verschiedene visuelle und interaktive Prinzipien entlang realer Anwendungen diskutieren. Die Szenarien reichen von digital angereichertem Papier, interaktiven Kordeln über visuelle Erweiterungen von Uhrarmbändern bis hin zu neuartigen Prototyping-Tools für intelligente Kleidungsstücke. Um neue Designansätze aufzuzeigen, integrieren nahezu alle visuelles Feedback und haptische Eingaben, um Alternativen zu Standard-Eingabemodalitäten auf starren Pixelbildschirmen zu schaffen. Die Dissertation hat gezeigt, wie wertvoll es sein kann, bekannte, analoge Anwendungen zu überdenken und sie dabei gleichzeitig mit Bedacht digital zu erweitern. Dabei umfasst die vorliegende Arbeit sowohl realisierte technische Forschungsplattformen als auch übergreifende konzeptionelle Arbeiten, Nutzerstudien und technische Experimente sowie die Analyse existierender Forschungsarbeiten

WristOrigami: Exploring foldable design for multi-display smartwatch

We present WristOrigami, an origami-inspired design concept and system extending the interaction with smartwatches through a foldable structure with multiple on-wrist displays. The current design provides extra affordances via folding, flipping, and elastic pulling actions on a multidisplay smartwatch. To motivate the design of WristOrigami, we developed a taxonomy that could be useful for analyzing and characterizing the origami-inspired multi-display smartwatch interaction. Through a participatory-design study with a set of prototypes with different levels of fidelity, we investigated users\u27 perception of WristOrigami in a wide range of applications with the presented features, and summarized a list of common shape configurations. We summarized our findings into seven design recommendations, to inform the future design of foldable smartwatch interactions. We further developed a set of application demonstrations as proofs-of-concept

Optimizing urban charging infrastructure for shared electric vehicles

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 115-117).This thesis analyses the opportunities and constraints of deploying charging infrastructure for shared electric vehicles in urban environments. Existing electric vehicle charging infrastructure for privately owned vehicles is examined and critiqued. A prototype of smartCharge, an integrated locking, charging, and ambient information system for shared electric vehicles is presented. Design methodology, fabrication of mechanical and electrical systems, and testing of the smartCharge system is documented. Urban implementation case studies for such a universal charging and locking station illustrate the potential of optimized infrastructure for shared vehicles to transform urban streetscapes and improve mobility. An analysis of leveraging existing building electrical infrastructure for vehicle charging is conducted, including phasing strategies for deploying rapid charging. Technological constraints to rapid charging such as battery chemistry, pack design, and power input are presented and evaluated. A strategy for buffering rapid electric vehicle charging with commercial uninterruptible power supply (UPS) systems is described. Two recent buildings on the MIT campus are used as case studies to demonstrate the overhead transformational capacity that exists in many modem, multi-purpose buildings. Connectivity between electrified transport, the electrical grid, and renewable energy sources is explored. A vision for personal urban mobility enabled by fleets of shared electric vehicles powered by clean, renewable energy and intelligent charging infrastructure is proposed.by Praveen Subramani.S.M

Compact and kinetic projected augmented reality interface

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 143-150).For quite some time, researchers and designers in the field of human computer interaction have strived to better integrate information interfaces into our physical environment. They envisioned a future where computing and interface components would be integrated into the physical environment, creating a seamless experience that uses all our senses. One possible approach to this problem employs projected augmented reality. Such systems project digital information and interfaces onto the physical world and are typically implemented using interactive projector-camera systems. This thesis work is centered on design and implementation of a new form factor for computing, a system we call LuminAR. LuminAR is a compact and kinetic projected augmented reality interface embodied in familiar everyday objects, namely a light bulb and a task light. It allows users to dynamically augment physical surfaces and objects with superimposed digital information using gestural and multi-touch interfaces. This thesis documents LuminAR's design process, hardware and software implementation and interaction techniques. The work is motivated through a set of applications that explore scenarios for interactive and kinetic projected augmented reality interfaces. It also opens the door for further explorations of kinetic interaction and promotes the adoption of projected augmented reality as a commonplace user interface modality. This thesis work was partially supported by a research grant from Intel Corporation.Supported by a research grant from Intel Corporationby Natan Linder.S.M

Tabletop tangible maps and diagrams for visually impaired users

En dépit de leur omniprésence et de leur rôle essentiel dans nos vies professionnelles et personnelles, les représentations graphiques, qu'elles soient numériques ou sur papier, ne sont pas accessibles aux personnes déficientes visuelles car elles ne fournissent pas d'informations tactiles. Par ailleurs, les inégalités d'accès à ces représentations ne cessent de s'accroître ; grâce au développement de représentations graphiques dynamiques et disponibles en ligne, les personnes voyantes peuvent non seulement accéder à de grandes quantités de données, mais aussi interagir avec ces données par le biais de fonctionnalités avancées (changement d'échelle, sélection des données à afficher, etc.). En revanche, pour les personnes déficientes visuelles, les techniques actuellement utilisées pour rendre accessibles les cartes et les diagrammes nécessitent l'intervention de spécialistes et ne permettent pas la création de représentations interactives. Cependant, les récentes avancées dans le domaine de l'adaptation automatique de contenus laissent entrevoir, dans les prochaines années, une augmentation de la quantité de contenus adaptés. Cette augmentation doit aller de pair avec le développement de dispositifs utilisables et abordables en mesure de supporter l'affichage de représentations interactives et rapidement modifiables, tout en étant accessibles aux personnes déficientes visuelles. Certains prototypes de recherche s'appuient sur une représentation numérique seulement : ils peuvent être instantanément modifiés mais ne fournissent que très peu de retour tactile, ce qui rend leur exploration complexe d'un point de vue cognitif et impose de fortes contraintes sur le contenu. D'autres prototypes s'appuient sur une représentation numérique et physique : bien qu'ils puissent être explorés tactilement, ce qui est un réel avantage, ils nécessitent un support tactile qui empêche toute modification rapide. Quant aux dispositifs similaires à des tablettes Braille, mais avec des milliers de picots, leur coût est prohibitif. L'objectif de cette thèse est de pallier les limitations de ces approches en étudiant comment développer des cartes et diagrammes interactifs physiques, modifiables et abordables. Pour cela, nous nous appuyons sur un type d'interface qui a rarement été étudié pour des utilisateurs déficients visuels : les interfaces tangibles, et plus particulièrement les interfaces tangibles sur table. Dans ces interfaces, des objets physiques représentent des informations numériques et peuvent être manipulés par l'utilisateur pour interagir avec le système, ou par le système lui-même pour refléter un changement du modèle numérique - on parle alors d'interfaces tangibles sur tables animées, ou actuated. Grâce à la conception, au développement et à l'évaluation de trois interfaces tangibles sur table (les Tangible Reels, la Tangible Box et BotMap), nous proposons un ensemble de solutions techniques répondant aux spécificités des interfaces tangibles pour des personnes déficientes visuelles, ainsi que de nouvelles techniques d'interaction non-visuelles, notamment pour la reconstruction d'une carte ou d'un diagramme et l'exploration de cartes de type " Pan & Zoom ". D'un point de vue théorique, nous proposons aussi une nouvelle classification pour les dispositifs interactifs accessibles.Despite their omnipresence and essential role in our everyday lives, online and printed graphical representations are inaccessible to visually impaired people because they cannot be explored using the sense of touch. The gap between sighted and visually impaired people's access to graphical representations is constantly growing due to the increasing development and availability of online and dynamic representations that not only give sighted people the opportunity to access large amounts of data, but also to interact with them using advanced functionalities such as panning, zooming and filtering. In contrast, the techniques currently used to make maps and diagrams accessible to visually impaired people require the intervention of tactile graphics specialists and result in non-interactive tactile representations. However, based on recent advances in the automatic production of content, we can expect in the coming years a growth in the availability of adapted content, which must go hand-in-hand with the development of affordable and usable devices. In particular, these devices should make full use of visually impaired users' perceptual capacities and support the display of interactive and updatable representations. A number of research prototypes have already been developed. Some rely on digital representation only, and although they have the great advantage of being instantly updatable, they provide very limited tactile feedback, which makes their exploration cognitively demanding and imposes heavy restrictions on content. On the other hand, most prototypes that rely on digital and physical representations allow for a two-handed exploration that is both natural and efficient at retrieving and encoding spatial information, but they are physically limited by the use of a tactile overlay, making them impossible to update. Other alternatives are either extremely expensive (e.g. braille tablets) or offer a slow and limited way to update the representation (e.g. maps that are 3D-printed based on users' inputs). In this thesis, we propose to bridge the gap between these two approaches by investigating how to develop physical interactive maps and diagrams that support two-handed exploration, while at the same time being updatable and affordable. To do so, we build on previous research on Tangible User Interfaces (TUI) and particularly on (actuated) tabletop TUIs, two fields of research that have surprisingly received very little interest concerning visually impaired users. Based on the design, implementation and evaluation of three tabletop TUIs (the Tangible Reels, the Tangible Box and BotMap), we propose innovative non-visual interaction techniques and technical solutions that will hopefully serve as a basis for the design of future TUIs for visually impaired users, and encourage their development and use. We investigate how tangible maps and diagrams can support various tasks, ranging from the (re)construction of diagrams to the exploration of maps by panning and zooming. From a theoretical perspective we contribute to the research on accessible graphical representations by highlighting how research on maps can feed research on diagrams and vice-versa. We also propose a classification and comparison of existing prototypes to deliver a structured overview of current research

Exploring the design space of programming by demonstration with context aware assistive systems

Due to the recent development in head-mounted displays, projection systems, rising capabilities of Augmented Reality and availability of Kinect for depth images, the advance of assistive systems in industrial context began. We propose a system that not only provides methods to assist industrial workers in their everyday tasks by assisting and providing instructions and giving feedback about the executed tasks. But although present a system that provides the capability to record and therefore create instructions by demonstration. This tackles the problem of current assistive systems and the complexity editors that are used for creating these instructions. We conducted a study that involved the creating of instruction via different conditions that were performed by experts in the field of manual assembly. Furthermore we verified these instruction with 51 industrial workers that completed assembly tasks guided by the instruction. Our results indicated that interactive instructions created through Programming by Demonstration are equal to existing approaches. Additional qualitative feedback showed that instructions through Programming by Demonstration are generally well perceived.Durch die jüngsten Entwicklungen im Feld der Head-Mounted Displays, Projektions Systeme, wachsende Möglichkeiten bezüglich augmentierter Realität und der Verfügbarkeit von Tiefendaten durch die Kinect Kamera, finden Assistenzsysteme immer mehr Anwendung in industriellen Bereichen. In dieser Arbeit wird ein System vorgestellt, welches nicht nur die Möglichkeit bietet, Arbeiter in der Industrie bei ihrer täglichen Arbeit, durch Anzeigen von Anleitungen und Feedback über ausgeführte Arbeitsschritte, zu unterstützen. Des Weiteren wird ein System vorgestellt das ermöglicht, Arbeitsanleitungen durch Demonstration zu erstellen. Hierdurch wird das Problem aktueller Assistenzsysteme beseitigt, Anleitungen mit komplizierten Editoren erstellen zu müssen. Mit einer Studie wurde das Erstellen von Anleitungen über verschiedene Ansätze überprüft. Die Studie wurde mit Experten der manuellen Montage durchgeführt. Weiterhin wurden die erstellten Anleitungen von 51 Arbeitern aus der Industrie getestet, welche die Anleitungen zum erfüllen eines Arbeitsvorganges genutzt haben. Die Ergebnisse weisen darauf hin, dass interaktive Anleitungen, die per Programmierung durch Demonstration erstellt wurden, gleichwertig gegenüber herkömmlichen Anleitungen sind. Die Auswertung des qualitatives Feedbacks der Studie hat zusätzlich gezeigt, dass Anleitungen die per Programmierung durch Demonstration erstellt wurden, generell als positiv wahrgenommen werden

Mobile Augmented Reality Application for Monitoring Industrial Systems

Augmented Reality (AR) applications have constantly evolved during the past few years in the mobile technology field. The potential of Mobile Augmented Reality (MAR) permits to enrich the reality with digital information by integrating the physical context of the user, localization-based data, mobile embedded sensors and Internet connectivity makes out of MAR a promising technology to be used in a wide range of real use cases. Nowadays the AR communities are growing and more companies are investing in Virtual Reality (VR) and MAR technologies. Lots of applications have been developed for medical, military, entertainment, manufacturing and industrial environments. Besides this, with the exponential grow up of the mobile market, this type of applications are becoming closer to final users. This work not only details the fundaments and concepts of AR but also describes the design and implementation of a MAR application within the area of industrial systems, specifically for the domain of discrete manufacturing. The application is designed to be used in mobile devices and special attention was taken, by using the latest technological trends in this area. The developed MAR application is intended to be used by people working with Manufacturing Systems (MS), allowing an intuitive and better visualization of huge amounts of data which are generated in the industrial system. An industrial testbed was used for testing this diploma work. Several tracking patterns were deployed along the production line to create points where the user can experience an AR interaction. The MAR application displays 3D objects in front of the user view by tracking the AR markers using di erent visualization gadgets, mobile devices or smart wearable devices (AR glasses). The developed application is designed based on the requirements and specifications of a project named: Pro-active decision support for data-intensive environments (ASTUTE), supported by the European Union (EU). The final results of the implemented application is included as a part of ASTUTE, in the demonstrator of virtual control room for building and manufacturing process management

A Stationless Bikeshare Proof of Concept for College Campuses

Bikeshares promote healthy lifestyles and sustainability among commuters, casual riders, and tourists. However, the central pillar of modern systems, the bike station, cannot be easily integrated into a compact college campus. Fixed stations lack the flexibility to meet the needs of college students who make quick, short-distance trips. Additionally, the necessary cost of implementing and maintaining each station prohibits increasing the number of stations for user convenience. Therefore, the team developed a stationless bikeshare based on a smartlock permanently attached to bicycles in the system. The smartlock system design incorporates several innovative approaches to provide usability, security, and reliability that overcome the limitations of a station centered design. A focus group discussion allowed the team to receive feedback on the early lock, system, and website designs, identify improvements and craft a pleasant user experience. The team designed a unique, two-step lock system that is intuitive to operate while mitigating user error. To ensure security, user access is limited through near field ii communications (NFC) technology connected to a mechatronic release system. The said system relied on a NFC module and a servo working through an Arduino microcontroller coded in the Arduino IDE. To track rentals and maintain the system, each bike is fitted with an XBee module to communicate with a scalable ZigBee mesh network. The network allows for bidirectional, real-time communication with a Meteor.js web application, which enables user and administrator functions through an intuitive user interface available on mobile and desktop. The development of an independent smartlock to replace bike stations is essential to meet the needs of the modern college student. With the goal of creating a bikeshare that better serves college students, Team BIKES has laid the framework for a system that is affordable, easily adaptable, and implementable on any university expressing an interest in bringing a bikeshare to its campus

NASA Tech Briefs, November 2002

Topics include: a technology focus on engineering materials, electronic components and systems, software, mechanics, machinery/automation, manufacturing, bio-medical, physical sciences, information sciences book and reports, and a special section of Photonics Tech Briefs

Embedded antenna technology in smart polymeric composite structures

One of the fastest-growing uses of sheet moulding compound (SMC) material is in the area of manufacturing of vehicle body components for both structural and non-structural applications. This trend is accelerating, driven by original equipment manufacturers (OEM) and their need for lighter and more fuel-efficient vehicles. In addition, over the last 20 years, the number of entertainment and communication systems in vehicles has also been expanding. The aim of this research is the development of a single wideband antenna that is capable of receiving all of the major services of interest. Taking this approach one step further and embedding such an antenna in a polymeric composite vehicle body panel would combine the benefits of reduced coefficient of drag, lower vehicle weight, reduced assembly complexity and shorter assembly time. These benefits would manifest themselves in the form of lower overall design and manufacturing vehicle cost for the OEMs and lower fuel consumption for customers. This thesis will deal with the development of such an antenna and the challenges faced in embedding it in a polymeric composite vehicle panel to such an extent that it becomes a seamless part of the vehicle body. This application required the development of a detailed understanding of the following three areas. Firstly, understanding of the interactions and effects of SMC material and automotive paint on antenna signal quality and performance through experiments and electromagnetic modelling (EM). Secondly, the development of the manufacturing process and material used to embed the antenna and its impact on the physical properties of the antenna through rheological testing, analytical modelling and experimentation. Lastly, the development of a wideband antenna capable of receiving pre-determined signals, through EM and field testing. The effects of paint application and presence of SMC resulted in a frequency shift of less than 1%. The experiments correlated well with the analytical model developed for compression moulding which incorporates a novel inclusion of the Maxwell’s model to predict the shear forces in the material flow within a confined space. A modular planar inverted conical antenna (PICA) was developed and optimised for the frequency range 700MHz – 9000MHz, which includes the commercial global positioning system (GPS) frequency. This development was then deployed as an embedded prototype in the deck lid of a test vehicle. Comparison against commercial GPS and mobile phone antennas was undertaken. This field test comparison showed that the developed PICA antenna performed better than the commercial antenna by up to 17%, especially in spaces devoid of multi-path signals