Tailoring Interaction. Sensing Social Signals with Textiles.

Nonverbal behaviour is an important part of conversation and can reveal much about the nature of an interaction. It includes phenomena ranging from large-scale posture shifts to small scale nods. Capturing these often spontaneous phenomena requires unobtrusive sensing techniques that do not interfere with the interaction. We propose an underexploited sensing modality for sensing nonverbal behaviours: textiles. As a material in close contact with the body, they provide ubiquitous, large surfaces that make them a suitable soft interface. Although the literature on nonverbal communication focuses on upper body movements such as gestures, observations of multi-party, seated conversations suggest that sitting postures, leg and foot movements are also systematically related to patterns of social interaction. This thesis addressees the following questions: Can the textiles surrounding us measure social engagement? Can they tell who is speaking, and who, if anyone, is listening? Furthermore, how should wearable textile sensing systems be designed and what behavioural signals could textiles reveal? To address these questions, we have designed and manufactured bespoke chairs and trousers with integrated textile pressure sensors, that are introduced here. The designs are evaluated in three user studies that produce multi-modal datasets for the exploration of fine-grained interactional signals. Two approaches to using these bespoke textile sensors are explored. First, hand crafted sensor patches in chair covers serve to distinguish speakers and listeners. Second, a pressure sensitive matrix in custom-made smart trousers is developed to detect static sitting postures, dynamic bodily movement, as well as basic conversational states. Statistical analyses, machine learning approaches, and ethnographic methods show that by moni- toring patterns of pressure change alone it is possible to not only classify postures with high accuracy, but also to identify a wide range of behaviours reliably in individuals and groups. These findings es- tablish textiles as a novel, wearable sensing system for applications in social sciences, and contribute towards a better understanding of nonverbal communication, especially the significance of posture shifts when seated. If chairs know who is speaking, if our trousers can capture our social engagement, what role can smart textiles have in the future of human interaction? How can we build new ways to map social ecologies and tailor interactions

Extending the Design Space of E-textile Assistive Smart Environment Applications

The thriving field of Smart Environments has allowed computing devices to gain new capabilities and develop new interfaces, thus becoming more and more part of our lives. In many of these areas it is unthinkable to renounce to the assisting functionality such as e.g. comfort and safety functions during driving, safety functionality while working in an industrial plant, or self-optimization of daily activities with a Smartwatch. Adults spend a lot of time on flexible surfaces such as in the office chair, in bed or in the car seat. These are crucial parts of our environments. Even though environments have become smarter with integrated computing gaining new capabilities and new interfaces, mostly rigid surfaces and objects have become smarter. In this thesis, I build on the advantages flexible and bendable surfaces have to offer and look into the creation process of assistive Smart Environment applications leveraging these surfaces. I have done this with three main contributions. First, since most Smart Environment applications are built-in into rigid surfaces, I extend the body of knowledge by designing new assistive applications integrated in flexible surfaces such as comfortable chairs, beds, or any type of soft, flexible objects. These developed applications offer assistance e.g. through preventive functionality such as decubitus ulcer prevention while lying in bed, back pain prevention while sitting on a chair or emotion detection while detecting movements on a couch. Second, I propose a new framework for the design process of flexible surface prototypes and its challenges of creating hardware prototypes in multiple iterations, using resources such as work time and material costs. I address this research challenge by creating a simulation framework which can be used to design applications with changing surface shape. In a first step I validate the simulation framework by building a real prototype and a simulated prototype and compare the results in terms of sensor amount and sensor placement. Furthermore, I use this developed simulation framework to analyse the influence it has on an application design if the developer is experienced or not. Finally, since sensor capabilities play a major role during the design process, and humans come often in contact with surfaces made of fabric, I combine the integration advantages of fabric and those of capacitive proximity sensing electrodes. By conducting a multitude of capacitive proximity sensing measurements, I determine the performance of electrodes made by varying properties such as material, shape, size, pattern density, stitching type, or supporting fabric. I discuss the results from this performance evaluation and condense them into e-textile capacitive sensing electrode guidelines, applied exemplary on the use case of creating a bed sheet for breathing rate detection