TableHop: an actuated fabric display using transparent electrodes

We present TableHop, a tabletop display that provides controlled self-actuated deformation and vibro-tactile feedback to an elastic fabric surface while retaining the ability for high-resolution visual projection. The TableHop surface is made of a highly stretchable pure spandex fabric that is electrostatically actuated using electrodes mounted on its underside. We use transparent indium tin oxide electrodes and high-voltage modulation to create controlled surface deformations. This setup actuates pixels and creates deformations in the fabric up to $\pm$ 5mm. Since the electrodes are transparent, the fabric surface can function as a diffuser for rear-projected visual images, and avoid occlusion by users. Users can touch and interact with the fabric to create expressive interactions as with any fabric based shape-changing interface. By using frequency modulation in the high-voltage circuit, we can also create localised tactile sensations on the user's finger-tip when touching the surface. We provide detailed simulation results of the shape of the surface deformation and the frequency of the haptic vibrations. These results can be used to build prototypes of different sizes and form-factors. We finally create a working prototype of TableHop that has 30$\times$40 cm surface area and uses a grid of 3$\times$3 transparent electrodes. Our prototype uses a maximum of 2.2 mW and can create tactile vibrations of up to 20 $Hz$. TableHop can be scaled to large interactive surfaces and integrated with other objects and devices. TableHop will improve user interaction experience on 2.5D deformable displays

PolySurface:a design approach for rapid prototyping of shape-changing displays using semi-solid surfaces

We present a design approach for rapid fabrication of high fidelity interactive shape-changing displays using bespoke semi-solid surfaces. This is achieved by segmenting virtual representations of the given data and mapping it to a dynamic physical polygonal surface. First, we establish the design and fabrication approach for generating semi-solid reconfigurable surfaces. Secondly, we demonstrate the generalizability of this approach by presenting design sessions using datasets provided by experts from a diverse range of domains. Thirdly, we evaluate user engagement with the prototype hardware systems that are built. We learned that all participants, all of whom had no previous interaction with shape-changing displays, were able to successfully design interactive hardware systems that physically represent data specific to their work. Finally, we reflect on the content generated to understand if our approach is effective at representing intended output based on a set of user defined functionality requirements

Digital Fabrication Approaches for the Design and Development of Shape-Changing Displays

Interactive shape-changing displays enable dynamic representations of data and information through physically reconfigurable geometry. The actuated physical deformations of these displays can be utilised in a wide range of new application areas, such as dynamic landscape and topographical modelling, architectural design, physical telepresence and object manipulation. Traditionally, shape-changing displays have a high development cost in mechanical complexity, technical skills and time/finances required for fabrication. There is still a limited number of robust shape-changing displays that go beyond one-off prototypes. Specifically, there is limited focus on low-cost/accessible design and development approaches involving digital fabrication (e.g. 3D printing). To address this challenge, this thesis presents accessible digital fabrication approaches that support the development of shape-changing displays with a range of application examples – such as physical terrain modelling and interior design artefacts. Both laser cutting and 3D printing methods have been explored to ensure generalisability and accessibility for a range of potential users. The first design-led content generation explorations show that novice users, from the general public, can successfully design and present their own application ideas using the physical animation features of the display. By engaging with domain experts in designing shape-changing content to represent data specific to their work domains the thesis was able to demonstrate the utility of shape-changing displays beyond novel systems and describe practical use-case scenarios and applications through rapid prototyping methods. This thesis then demonstrates new ways of designing and building shape-changing displays that goes beyond current implementation examples available (e.g. pin arrays and continuous surface shape-changing displays). To achieve this, the thesis demonstrates how laser cutting and 3D printing can be utilised to rapidly fabricate deformable surfaces for shape-changing displays with embedded electronics. This thesis is concluded with a discussion of research implications and future direction for this work

PolySurface: a design approach for rapid prototyping of shape-changing displays using semi-solid surfaces

We present a design approach for rapid fabrication of high fidelity interactive shape-changing displays using bespoke semi-solid surfaces. This is achieved by segmenting virtual representations of the given data and mapping it to a dynamic physical polygonal surface. First, we establish the design and fabrication approach for generating semi-solid reconfigurable surfaces. Secondly, we demonstrate the generalizability of this approach by presenting design sessions using datasets provided by experts from a diverse range of domains. Thirdly, we evaluate user engagement with the prototype hardware systems that are built. We learned that all participants, all of whom had no previous interaction with shape-changing displays, were able to successfully design interactive hardware systems that physically represent data specific to their work. Finally, we reflect on the content generated to understand if our approach is effective at representing intended output based on a set of user defined functionality requirements

BendableSound: An Elastic Multisensory Surface Using Touch-based interactions to Assist Children with Severe Autism During Music Therapy

Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as “usable” and “attractive”. Then, we present a deployment study with 24 children with severe autism showing BendableSound is “easy to use” and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work

Real time hand gesture recognition including hand segmentation and tracking

In this paper we present a system that performs automatic gesture recognition. The system consists of two main components: (i) A unified technique for segmentation and tracking of face and hands using a skin detection algorithm along with handling occlusion between skin objects to keep track of the status of the occluded parts. This is realized by combining 3 useful features, namely, color, motion and position. (ii) A static and dynamic gesture recognition system. Static gesture recognition is achieved using a robust hand shape classification, based on PCA subspaces, that is invariant to scale along with small translation and rotation transformations. Combining hand shape classification with position information and using DHMMs allows us to accomplish dynamic gesture recognition

Digital fabrication of custom interactive objects with rich materials

As ubiquitous computing is becoming reality, people interact with an increasing number of computer interfaces embedded in physical objects. Today, interaction with those objects largely relies on integrated touchscreens. In contrast, humans are capable of rich interaction with physical objects and their materials through sensory feedback and dexterous manipulation skills. However, developing physical user interfaces that offer versatile interaction and leverage these capabilities is challenging. It requires novel technologies for prototyping interfaces with custom interactivity that support rich materials of everyday objects. Moreover, such technologies need to be accessible to empower a wide audience of researchers, makers, and users. This thesis investigates digital fabrication as a key technology to address these challenges. It contributes four novel design and fabrication approaches for interactive objects with rich materials. The contributions enable easy, accessible, and versatile design and fabrication of interactive objects with custom stretchability, input and output on complex geometries and diverse materials, tactile output on 3D-object geometries, and capabilities of changing their shape and material properties. Together, the contributions of this thesis advance the fields of digital fabrication, rapid prototyping, and ubiquitous computing towards the bigger goal of exploring interactive objects with rich materials as a new generation of physical interfaces.Computer werden zunehmend in Geräten integriert, mit welchen Menschen im Alltag interagieren. Heutzutage basiert diese Interaktion weitgehend auf Touchscreens. Im Kontrast dazu steht die reichhaltige Interaktion mit physischen Objekten und Materialien durch sensorisches Feedback und geschickte Manipulation. Interfaces zu entwerfen, die diese Fähigkeiten nutzen, ist allerdings problematisch. Hierfür sind Technologien zum Prototyping neuer Interfaces mit benutzerdefinierter Interaktivität und Kompatibilität mit vielfältigen Materialien erforderlich. Zudem sollten solche Technologien zugänglich sein, um ein breites Publikum zu erreichen. Diese Dissertation erforscht die digitale Fabrikation als Schlüsseltechnologie, um diese Probleme zu adressieren. Sie trägt vier neue Design- und Fabrikationsansätze für das Prototyping interaktiver Objekte mit reichhaltigen Materialien bei. Diese ermöglichen einfaches, zugängliches und vielseitiges Design und Fabrikation von interaktiven Objekten mit individueller Dehnbarkeit, Ein- und Ausgabe auf komplexen Geometrien und vielfältigen Materialien, taktiler Ausgabe auf 3D-Objektgeometrien und der Fähigkeit ihre Form und Materialeigenschaften zu ändern. Insgesamt trägt diese Dissertation zum Fortschritt der Bereiche der digitalen Fabrikation, des Rapid Prototyping und des Ubiquitous Computing in Richtung des größeren Ziels, der Exploration interaktiver Objekte mit reichhaltigen Materialien als eine neue Generation von physischen Interfaces, bei

Analysis and Classification of Shape-Changing Interfaces for Design and Application-based Research

Shape-changing interfaces are physically tangible, interactive devices, surfaces, or spaces that allow for rich, organic, and novel experiences with computational devices. Over the last 15 years, research has produced functional prototypes over many use applications; reviews have identified themes and possible future directions but have not yet looked at possible design or application-based research. Here, we gather this information together to provide a reference for designers and researchers wishing to build upon existing prototyping work, using synthesis and discussion of existing shape-changing interface reviews and comprehensive analysis and classification of 84 shape-changing interfaces. Eight categories of prototype are identified alongside recommendations for the field

Interactions of children and young adults using large-scale elastic displays

Elastic displays provide a unique and intuitive interaction and could be deployed at large-scale. As an emerging technology, open questions about the benefits large-scale elastic displays offer over rigid displays and their potential application to our everyday lives. In this paper, we present an overview of a 4-year project. First, we describe the development of a large-scale elastic display, called BendableSound. Second, we explain the results of a laboratory study, showing the elastic display has a better user and sensory experience than a rigid one. Third, we describe the results of two deployment studies showing how BendableSound could support the therapeutic practices of children with autism and the early development of toddlers. We close discussing open challenges to study the untapped potential of elastic displays in pervasive computing