Sensing with Earables: A Systematic Literature Review and Taxonomy of Phenomena

Earables have emerged as a unique platform for ubiquitous computing by augmenting ear-worn devices with state-of-the-art sensing. This new platform has spurred a wealth of new research exploring what can be detected on a wearable, small form factor. As a sensing platform, the ears are less susceptible to motion artifacts and are located in close proximity to a number of important anatomical structures including the brain, blood vessels, and facial muscles which reveal a wealth of information. They can be easily reached by the hands and the ear canal itself is affected by mouth, face, and head movements. We have conducted a systematic literature review of 271 earable publications from the ACM and IEEE libraries. These were synthesized into an open-ended taxonomy of 47 different phenomena that can be sensed in, on, or around the ear. Through analysis, we identify 13 fundamental phenomena from which all other phenomena can be derived, and discuss the different sensors and sensing principles used to detect them. We comprehensively review the phenomena in four main areas of (i) physiological monitoring and health, (ii) movement and activity, (iii) interaction, and (iv) authentication and identification. This breadth highlights the potential that earables have to offer as a ubiquitous, general-purpose platform

Sensing with Earables: A Systematic Literature Review and Taxonomy of Phenomena

Earables have emerged as a unique platform for ubiquitous computing by augmenting ear-worn devices with state-of-the-art sensing. This new platform has spurred a wealth of new research exploring what can be detected on a wearable, small form factor. As a sensing platform, the ears are less susceptible to motion artifacts and are located in close proximity to a number of important anatomical structures including the brain, blood vessels, and facial muscles which reveal a wealth of information. They can be easily reached by the hands and the ear canal itself is affected by mouth, face, and head movements. We have conducted a systematic literature review of 271 earable publications from the ACM and IEEE libraries. These were synthesized into an open-ended taxonomy of 47 different phenomena that can be sensed in, on, or around the ear. Through analysis, we identify 13 fundamental phenomena from which all other phenomena can be derived, and discuss the different sensors and sensing principles used to detect them. We comprehensively review the phenomena in four main areas of (i) physiological monitoring and health, (ii) movement and activity, (iii) interaction, and (iv) authentication and identification. This breadth highlights the potential that earables have to offer as a ubiquitous, general-purpose platform

Obstacles to wearable computing

In the year 2021, wearable technology could look beautiful and feel magical, but instead is exemplified by a plain wristband that looks suspiciously like a prison monitor. How can we make wearable technology that respects our privacy, enhances our daily lives, integrates with our other connected devices without leashing us to a smartphone, and visually expresses who we are? This study uses a novel method of participatory design fiction (PDFi) to understand potential users of everyday wearable technology through storytelling. I recruited participants from the general public and gave them a five-point prompt to create a design fiction (DF), which inspired the user-centred design of an everyday connected wearable device. The participants each received a technology probe to wear in the wild for a year. They then updated their DFs as a way to reflect on the implications of the technology. For the purposes of privacy, augmenting device functionality through interoperability, and integration into an Internet of Things (IoT) ecosystem, I used the Hub-of-All-Things personal data store to provide the software infrastructure. By listening to their stories, we can elicit design concepts directly from the users, to help us create wearable IoT devices that put the wearer at the centre of the design process, and are satisfying both functionally and emotionally.The Alan Turing Institute Doctoral Scheme, University of Cambridge Department of Computer Science and Technology, The Kenneth Hayter Memorial Fun

From wearable towards epidermal computing : soft wearable devices for rich interaction on the skin

Human skin provides a large, always available, and easy to access real-estate for interaction. Recent advances in new materials, electronics, and human-computer interaction have led to the emergence of electronic devices that reside directly on the user's skin. These conformal devices, referred to as Epidermal Devices, have mechanical properties compatible with human skin: they are very thin, often thinner than human hair; they elastically deform when the body is moving, and stretch with the user's skin. Firstly, this thesis provides a conceptual understanding of Epidermal Devices in the HCI literature. We compare and contrast them with other technical approaches that enable novel on-skin interactions. Then, through a multi-disciplinary analysis of Epidermal Devices, we identify the design goals and challenges that need to be addressed for advancing this emerging research area in HCI. Following this, our fundamental empirical research investigated how epidermal devices of different rigidity levels affect passive and active tactile perception. Generally, a correlation was found between the device rigidity and tactile sensitivity thresholds as well as roughness discrimination ability. Based on these findings, we derive design recommendations for realizing epidermal devices. Secondly, this thesis contributes novel Epidermal Devices that enable rich on-body interaction. SkinMarks contributes to the fabrication and design of novel Epidermal Devices that are highly skin-conformal and enable touch, squeeze, and bend sensing with co-located visual output. These devices can be deployed on highly challenging body locations, enabling novel interaction techniques and expanding the design space of on-body interaction. Multi-Touch Skin enables high-resolution multi-touch input on the body. We present the first non-rectangular and high-resolution multi-touch sensor overlays for use on skin and introduce a design tool that generates such sensors in custom shapes and sizes. Empirical results from two technical evaluations confirm that the sensor achieves a high signal-to-noise ratio on the body under various grounding conditions and has a high spatial accuracy even when subjected to strong deformations. Thirdly, Epidermal Devices are in contact with the skin, they offer opportunities for sensing rich physiological signals from the body. To leverage this unique property, this thesis presents rapid fabrication and computational design techniques for realizing Multi-Modal Epidermal Devices that can measure multiple physiological signals from the human body. Devices fabricated through these techniques can measure ECG (Electrocardiogram), EMG (Electromyogram), and EDA (Electro-Dermal Activity). We also contribute a computational design and optimization method based on underlying human anatomical models to create optimized device designs that provide an optimal trade-off between physiological signal acquisition capability and device size. The graphical tool allows for easily specifying design preferences and to visually analyze the generated designs in real-time, enabling designer-in-the-loop optimization. Experimental results show high quantitative agreement between the prediction of the optimizer and experimentally collected physiological data. Finally, taking a multi-disciplinary perspective, we outline the roadmap for future research in this area by highlighting the next important steps, opportunities, and challenges. Taken together, this thesis contributes towards a holistic understanding of Epidermal Devices}: it provides an empirical and conceptual understanding as well as technical insights through contributions in DIY (Do-It-Yourself), rapid fabrication, and computational design techniques.Die menschliche Haut bietet eine große, stets verfügbare und leicht zugängliche Fläche für Interaktion. Jüngste Fortschritte in den Bereichen Materialwissenschaft, Elektronik und Mensch-Computer-Interaktion (Human-Computer-Interaction, HCI) [so that you can later use the Englisch abbreviation] haben zur Entwicklung elektronischer Geräte geführt, die sich direkt auf der Haut des Benutzers befinden. Diese sogenannten Epidermisgeräte haben mechanische Eigenschaften, die mit der menschlichen Haut kompatibel sind: Sie sind sehr dünn, oft dünner als ein menschliches Haar; sie verformen sich elastisch, wenn sich der Körper bewegt, und dehnen sich mit der Haut des Benutzers. Diese Thesis bietet, erstens, ein konzeptionelles Verständnis von Epidermisgeräten in der HCI-Literatur. Wir vergleichen sie mit anderen technischen Ansätzen, die neuartige Interaktionen auf der Haut ermöglichen. Dann identifizieren wir durch eine multidisziplinäre Analyse von Epidermisgeräten die Designziele und Herausforderungen, die angegangen werden müssen, um diesen aufstrebenden Forschungsbereich voranzubringen. Im Anschluss daran untersuchten wir in unserer empirischen Grundlagenforschung, wie epidermale Geräte unterschiedlicher Steifigkeit die passive und aktive taktile Wahrnehmung beeinflussen. Im Allgemeinen wurde eine Korrelation zwischen der Steifigkeit des Geräts und den taktilen Empfindlichkeitsschwellen sowie der Fähigkeit zur Rauheitsunterscheidung festgestellt. Basierend auf diesen Ergebnissen leiten wir Designempfehlungen für die Realisierung epidermaler Geräte ab. Zweitens trägt diese Thesis zu neuartigen Epidermisgeräten bei, die eine reichhaltige Interaktion am Körper ermöglichen. SkinMarks trägt zur Herstellung und zum Design neuartiger Epidermisgeräte bei, die hochgradig an die Haut angepasst sind und Berührungs-, Quetsch- und Biegesensoren mit gleichzeitiger visueller Ausgabe ermöglichen. Diese Geräte können an sehr schwierigen Körperstellen eingesetzt werden, ermöglichen neuartige Interaktionstechniken und erweitern den Designraum für die Interaktion am Körper. Multi-Touch Skin ermöglicht hochauflösende Multi-Touch-Eingaben am Körper. Wir präsentieren die ersten nicht-rechteckigen und hochauflösenden Multi-Touch-Sensor-Overlays zur Verwendung auf der Haut und stellen ein Design-Tool vor, das solche Sensoren in benutzerdefinierten Formen und Größen erzeugt. Empirische Ergebnisse aus zwei technischen Evaluierungen bestätigen, dass der Sensor auf dem Körper unter verschiedenen Bedingungen ein hohes Signal-Rausch-Verhältnis erreicht und eine hohe räumliche Auflösung aufweist, selbst wenn er starken Verformungen ausgesetzt ist. Drittens, da Epidermisgeräte in Kontakt mit der Haut stehen, bieten sie die Möglichkeit, reichhaltige physiologische Signale des Körpers zu erfassen. Um diese einzigartige Eigenschaft zu nutzen, werden in dieser Arbeit Techniken zur schnellen Herstellung und zum computergestützten Design von multimodalen Epidermisgeräten vorgestellt, die mehrere physiologische Signale des menschlichen Körpers messen können. Die mit diesen Techniken hergestellten Geräte können EKG (Elektrokardiogramm), EMG (Elektromyogramm) und EDA (elektrodermale Aktivität) messen. Darüber hinaus stellen wir eine computergestützte Design- und Optimierungsmethode vor, die auf den zugrunde liegenden anatomischen Modellen des Menschen basiert, um optimierte Gerätedesigns zu erstellen. Diese Designs bieten einen optimalen Kompromiss zwischen der Fähigkeit zur Erfassung physiologischer Signale und der Größe des Geräts. Das grafische Tool ermöglicht die einfache Festlegung von Designpräferenzen und die visuelle Analyse der generierten Designs in Echtzeit, was eine Optimierung durch den Designer im laufenden Betrieb ermöglicht. Experimentelle Ergebnisse zeigen eine hohe quantitative Übereinstimmung zwischen den Vorhersagen des Optimierers und den experimentell erfassten physiologischen Daten. Schließlich skizzieren wir aus einer multidisziplinären Perspektive einen Fahrplan für zukünftige Forschung in diesem Bereich, indem wir die nächsten wichtigen Schritte, Möglichkeiten und Herausforderungen hervorheben. Insgesamt trägt diese Arbeit zu einem ganzheitlichen Verständnis von Epidermisgeräten bei: Sie liefert ein empirisches und konzeptionelles Verständnis sowie technische Einblicke durch Beiträge zu DIY (Do-It-Yourself), schneller Fertigung und computergestützten Entwurfstechniken

Wearable Technology Devices: Heart Rate and Step Count Analysis

The overarching purpose of this dissertation was to evaluate and analyze heart rate and/or step count measurements for six popular wearable technology devices: the Samsung Gear 2, FitBit Surge, Polar A360, Garmin Vivosmart HR+, Leaf Health Tracker, and the Scosche Rhythm+ in four separate conditions: free motion walking, free motion jogging, treadmill walking, and treadmill jogging. The four studies presented here utilized one test design and data collection protocol in which many measurements could be addressed simultaneously. Currently, there is no accepted standardized protocol to evaluate wearable technology devices. The test design utilized for this research series was introduced as a potential foundation for the establishment of a common procedure. There were three purposes for the first study in this series of four research projects. First, this study looked at whether the tested devices that recorded heart rate were reliable and valid in each of the four stated conditions. Only the Garmin Vivosmart HR+ and the Scosche Rhythm+ were significantly acceptable for all four conditions. Secondly, while all the tested devices used photoplethysmography to record heart rate, this technique has not been thoroughly validated for this purpose. Limited research indicates that devices that use this method as a measurement technique and are worn on the forearm are more accurate than those worn elsewhere on the body. Results from our study supported this conclusion. The Scosche Rhythm+, being a fore arm worn device, did produce more significantly acceptable results than the wrist worn Garmin Vivosmart HR+. Third, a standardized heart rate testing protocol has been introduced by the Consumer Technology Association. However, their recommended measurement criteria (a measurement every 1-5 seconds which would require special software to record) can be viewed as financially prohibitive, restrictive, and over compensating. The protocol used in our research presented evidence that ours, which used an average of several minutes of heart rate values, was easier to implement and did not required a financial investment to perform. The second study had two purposes. First, this study looked at whether the tested devices that recorded step count were reliable and valid in each of the four conditions. Only the FitBit Surge, Garmin Vivosmart HR+ and the Leaf Health Tracker were significantly acceptable for all four conditions. Secondly, the Consumer Technology Association has recommended a standardized step count protocol which would require the videotaping of an activity with separate tape reviews by two persons at a future time. This protocol is not feasible in certain conditions such as outside testing. Additionally, both reviewers would need to produce the exact same step count. Our testing used two manual counters where the mean of the two were used as the criterion measure. We provided strong evidence that this is an acceptable criterion measure for step counting that does not require additional time or resources. The third study compared heart rate and step count values measured by the tested devices between the different conditions. Measurements taken during free motion walking were compared to treadmill walking and those taken during free motion jogging were compared to treadmill jogging. It is generally believed that most wearable technology device companies perform device testing on a treadmill in a laboratory. Our conclusion was that there was no significant interaction or main effects for walking heart rate value comparisons. Jogging heart rate values saw significant main effects from both the environment and between the devices. Walking step count values had a significant interaction between the devices and the environment. Jogging step count values had a significant main effect between the devices. When utilizing wearable technology devices for the measurement of heart rate during walking or jogging, the Garmin Vivosmart HR+ and Rhythm Scosche Rhythm+ provided acceptable measures both in the laboratory as well as in a free motion environment. The FitBit Surge, Garmin Vivo Smart HR+, and the Leaf Health Tracker produced similar results for step count. The fourth study evaluated whether there was a correlation between both body composition percentages and body mass index values and the percent error calculated between a manual step count and that recorded by the wearable technology devices. Our results gave evidence that there are no significant correlations between body mass index and the calculated percent error. For body composition, only two conditions for the wrist worn devices had a positive significant correlation; the Samsung Gear 2 when free motion walking and the Garmin Vivosmart HR+ when free motion walking. The waist worn Leaf Activity Tracker had positive significant correlations for both treadmill walking and treadmill jogging. Even though our study produced four conditions with significant correlations, all were low to moderate in value

Exploring young children???s ideas about wearable technology: a case study

This case study, which forms a part of the Kids, Creative Storyworlds and Wearables project, explores children???s perspectives on wearable technology through their stories and other creative ideas inspired by wearable technology. Five children between the ages of four and seven were each given a smartwatch and were interviewed three times over the span of four weeks. Using a multi-method approach, inspired by the Mosaic approach to ethnography (Clark & Moss, 2011; Clark, 2005) and social semiotics (Kress and van Leeuwen, 2006; Kress, 1997), children were invited to share their ideas in a variety of ways (face-to-face discussion, oral storytelling, written text, drawings). This research viewed children as meaning-makers and sign-makers. Results supported and extended elements of Papert???s constructionist learning theory and Sutton Smith???s ???play as a viability variable??? theory (2008) and provided novel insights relevant to formal education practices. Empowerment is a key theme that emerged from this case study

ICS Materials

This present book covers a series of outstanding reputation researchers’ contributions on the topic of ICS Materials: a new class of emerging materials with properties and qualities concerning interactivity, connectivity and intelligence. In the general framework of ICS Materials’ domain, each chapter deals with a specific aspect following the characteristic perspective of each researcher. As result, methods, tools, guidelines emerged that are relevant and applicable to several contexts such as product, interaction design, materials science and many more

Design and Application of Wireless Body Sensors

Hörmann T. Design and Application of Wireless Body Sensors. Bielefeld: Universität Bielefeld; 2019