Art and Design Practices as a Driver for Deformable Controls, Textures and Screen Interactions

In this thesis, we demonstrate the innovative uses of deformable interfaces to help de-velop future digital art and design interactions. The great benefits of advancing digital art can often come at a cost of tactile feeling and physical expression, while traditional methods celebrate the diverse sets of physical tools and materials. We identified these sets of tools and materials to inform the development of new art and design interfaces that offer rich physical mediums for digital artist and designers. In order to bring forth these unique inter-actions, we draw on the latest advances in deformable interface technology. Therefore, our research contributes a set of understandings about how deformable interfaces can be har-nessed for art and design interfaces. We identify and discuss the following contributions: insights into tangible and digital practices of artists and designers; prototypes to probe the benefits and possibilities of deformable displays and materials in support of digital-physical art and design, user-centred evaluations of these prototypes to inform future developments, and broader insights into the deformable interface research.Each chapter of this thesis investigates a specific element of art and design, alongside an aspect of deformable interfaces resulting in a new prototype. We begin the thesis by studying the use of physical actuation to simulate artist tools in deformable surfaces. In this chapter, our evaluations highlight the merits of improved user experiences and insights into eyes-free interactions. We then turn to explore deformable textures. Driven by the tactile feeling of mixing paints, we present a gel-based interface that is capable of simulating the feeling of paints on the back of mobile devices. Our evaluations showed how artists endorsed the interactions and held potential for digital oil painting.Our final chapter presents research conducted with digital designers. We explore their colour picking processes and developed a digital version of physical swatches using a mod-ular screen system. This use of tangible proxies in digital-based processes brought a level of playfulness and held potential to support collaborative workflows across disciplines. To conclude, we share how our outcomes from these studies could help shape the broader space of art and design interactions and deformable interface research. We suggest future work and directions based on our findings

Digital Fabrication Approaches for the Design and Development of Shape-Changing Displays

Interactive shape-changing displays enable dynamic representations of data and information through physically reconfigurable geometry. The actuated physical deformations of these displays can be utilised in a wide range of new application areas, such as dynamic landscape and topographical modelling, architectural design, physical telepresence and object manipulation. Traditionally, shape-changing displays have a high development cost in mechanical complexity, technical skills and time/finances required for fabrication. There is still a limited number of robust shape-changing displays that go beyond one-off prototypes. Specifically, there is limited focus on low-cost/accessible design and development approaches involving digital fabrication (e.g. 3D printing). To address this challenge, this thesis presents accessible digital fabrication approaches that support the development of shape-changing displays with a range of application examples – such as physical terrain modelling and interior design artefacts. Both laser cutting and 3D printing methods have been explored to ensure generalisability and accessibility for a range of potential users. The first design-led content generation explorations show that novice users, from the general public, can successfully design and present their own application ideas using the physical animation features of the display. By engaging with domain experts in designing shape-changing content to represent data specific to their work domains the thesis was able to demonstrate the utility of shape-changing displays beyond novel systems and describe practical use-case scenarios and applications through rapid prototyping methods. This thesis then demonstrates new ways of designing and building shape-changing displays that goes beyond current implementation examples available (e.g. pin arrays and continuous surface shape-changing displays). To achieve this, the thesis demonstrates how laser cutting and 3D printing can be utilised to rapidly fabricate deformable surfaces for shape-changing displays with embedded electronics. This thesis is concluded with a discussion of research implications and future direction for this work

TableHop: an actuated fabric display using transparent electrodes

We present TableHop, a tabletop display that provides controlled self-actuated deformation and vibro-tactile feedback to an elastic fabric surface while retaining the ability for high-resolution visual projection. The TableHop surface is made of a highly stretchable pure spandex fabric that is electrostatically actuated using electrodes mounted on its underside. We use transparent indium tin oxide electrodes and high-voltage modulation to create controlled surface deformations. This setup actuates pixels and creates deformations in the fabric up to $\pm$ 5mm. Since the electrodes are transparent, the fabric surface can function as a diffuser for rear-projected visual images, and avoid occlusion by users. Users can touch and interact with the fabric to create expressive interactions as with any fabric based shape-changing interface. By using frequency modulation in the high-voltage circuit, we can also create localised tactile sensations on the user's finger-tip when touching the surface. We provide detailed simulation results of the shape of the surface deformation and the frequency of the haptic vibrations. These results can be used to build prototypes of different sizes and form-factors. We finally create a working prototype of TableHop that has 30$\times$40 cm surface area and uses a grid of 3$\times$3 transparent electrodes. Our prototype uses a maximum of 2.2 mW and can create tactile vibrations of up to 20 $Hz$. TableHop can be scaled to large interactive surfaces and integrated with other objects and devices. TableHop will improve user interaction experience on 2.5D deformable displays

Applications of high and low fidelity prototypes in researching intuitive interaction

This paper addresses some of the issues involved in incorporating use of prototypes into a research program. Definitions, merits and uses of both low and high-fidelity prototypes are discussed and then the applications of prototypes in our research program into intuitive interaction are explored. It has previously been established that intuitive interaction is based on past experience, and can be encouraged by designing interfaces that contain familiar features (Blackler, 2006; Blackler, Popovic, & Mahar, 2007b). Two aspects of the research program which are relevant to prototyping are: researching the issues of how intuitive use happens and how it can be better facilitated; and developing ways to help designers include investigations about users and their existing knowledge into their design processes in order to make interfaces more intuitive. The current and future planned applications of high and low-fidelity prototypes in each of these areas are explored. Then experiences with using high-fidelity touchscreen prototypes for experimental research into intuitive interaction are discussed, including problems with the prototypes, how they were addressed and what we have learned from the process. Next the potential for low-fidelity prototypes to elicit users’ tacit knowledge during the design process is explored. This has exciting possibilities due to the link between intuitive interaction and tacit knowledge. Finally, the challenges of developing prototype-based design tools for use by older people are discussed and future directions for using prototypes in our research program are considered. Keywords: Prototypes; intuitive interaction; experimental methodology; implicit or tacit knowledge</p

GazeDPM: Early Integration of Gaze Information in Deformable Part Models

An increasing number of works explore collaborative human-computer systems in which human gaze is used to enhance computer vision systems. For object detection these efforts were so far restricted to late integration approaches that have inherent limitations, such as increased precision without increase in recall. We propose an early integration approach in a deformable part model, which constitutes a joint formulation over gaze and visual data. We show that our GazeDPM method improves over the state-of-the-art DPM baseline by 4% and a recent method for gaze-supported object detection by 3% on the public POET dataset. Our approach additionally provides introspection of the learnt models, can reveal salient image structures, and allows us to investigate the interplay between gaze attracting and repelling areas, the importance of view-specific models, as well as viewers' personal biases in gaze patterns. We finally study important practical aspects of our approach, such as the impact of using saliency maps instead of real fixations, the impact of the number of fixations, as well as robustness to gaze estimation error

3D Printed Deformable Surfaces for Shape-Changing Displays

We use interlinked 3D printed panels to fabricate deformable surfaces that are specifically designed for shape-changing displays. Our exploration of 3D printed deformable surfaces, as a fabrication technique for shape-changing displays, shows new and diverse forms of shape output, visualizations, and interaction capabilities. This article describes our general design and fabrication approach, the impact of varying surface design parameters, and a demonstration of possible application examples. We conclude by discussing current limitations and future directions for this work

Computational Simulation and 3D Virtual Reality Engineering Tools for Dynamical Modeling and Imaging of Composite Nanomaterials

An adventure at engineering design and modeling is possible with a Virtual Reality Environment (VRE) that uses multiple computer-generated media to let a user experience situations that are temporally and spatially prohibiting. In this paper, an approach to developing some advanced architecture and modeling tools is presented to allow multiple frameworks work together while being shielded from the application program. This architecture is being developed in a framework of workbench interactive tools for next generation nanoparticle-reinforced damping/dynamic systems. Through the use of system, an engineer/programmer can respectively concentrate on tailoring an engineering design concept of novel system and the application software design while using existing databases/software outputs.Comment: Submitted on behalf of TIMA Editions (http://irevues.inist.fr/tima-editions