A Wearable Electrostatic Sensor for Human Activity Monitoring

Human activity monitoring based on wearable sensors is important in a wide range of biomedical and healthcare applications. Existing wearable sensors using a single unit cannot capture the movements of all body segments. This paper presents a novel wearable electrostatic sensor that can detect limb and torso movements during routine daily activities from any location on the body. Because the electric potential of the human body varies during movements, the sensor measures the potential difference between the body and the electrode for motion sensing. A charge amplifier converts the induced charge on the electrode into a voltage signal, which is further amplified, filtered, digitalized and transmitted via ZigBee. Experimental assessment was carried out by collecting sensor signals from three locations simultaneously while the subject performing different movements. The capability of the sensor to capture limb and torso movements from any location is validated. The characteristics of the sensor are quantified by correlating the sensor signal to simple and cyclic movements. It is found that the sensor signal depends on the sensor’s mounting location on the body, the type of activity and various factors

Intelligent textiles: Designing a gesture-controlled illuminated textile based on computer vision

Artificial intelligence (AI) offers the potential for the development of e-textiles that give wearers a smart and intuitive experience. An emerging challenge in intelligent materials design is hand gesture recognition textiles. Most current research focuses on number gesture recognition via smart gloves, so there is a gap in research that studies contact-less number gesture recognition textiles via computer vision. Meanwhile, there is lack of exploration on the integration of illuminating function and number gesture recognition textiles to improve interactivity by real-time visualizing detection results. In this research, a novel interactive illuminating textile with a touch-less number gesture recognition function has been designed and fabricated by using an open-source AI model. It is used in sync with a polymeric optical fiber textile with illuminative features. The textile is color-changing, controlled by the system's mid-air interactive number gesture recognition capability and has a woven stripe pattern and a double-layer weave structure with open pockets to facilitate integration of the system's components. Also described here is a novel design process that permits textile design and intelligent technology to integrate seamlessly and in synchronization, so that design in effect mediates continuously between the physical textile and the intangible technology. Moreover, this design method serves as a reference for the integration of open-source intelligent hardware and software into e-textiles for enhancement of the intuitive function and value via economy of labor

Robotic Caregivers -- Simulation and Capacitive Servoing for Physical Human-Robot Interaction

Physical human-robot interaction and robotic assistance presents an opportunity to benefit the lives of many people, including the millions of older adults and people with physical disabilities, who have difficulty performing activities of daily living (ADLs) on their own. Robotic caregiving for activities of daily living could increase the independence of people with disabilities, improve quality of life, and help address global societal issues, such as aging populations, high healthcare costs, and shortages of healthcare workers. Yet, robotic assistance presents several challenges, including risks associated with physical human-robot interaction, difficulty sensing the human body, and complexities of modeling deformable materials (e.g. clothes). We address these challenges through techniques that span the intersection of machine learning, physics simulation, sensing, and physical human-robot interaction. Haptic Perspective-taking: We first demonstrate that by enabling a robot to predict how its future actions will physically affect a person (haptic perspective-taking), robots can provide safer assistance, especially within the context of robot-assisted dressing and manipulating deformable clothes. We train a recurrent model consisting of both a temporal estimator and predictor that allows a robot to predict the forces a garment is applying onto a person using haptic measurements from the robot's end effector. By combining this predictor with model predictive control (MPC), we observe emergent behaviors that result in the robot navigating a garment up a person's entire arm. Capacitive Sensing for Tracking Human Pose: Towards the goal of robots performing robust and intelligent physical interactions with people, it is crucial that robots are able to accurately sense the human body, follow trajectories around the body, and track human motion. We have introduced a capacitive servoing control scheme that allows a robot to sense and navigate around human limbs during close physical interactions. Capacitive servoing leverages temporal measurements from a capacitive sensor mounted on a robot's end effector to estimate the relative pose of a nearby human limb. Capacitive servoing then uses these human pose estimates within a feedback control loop in order to maneuver the robot's end effector around the surface of a human limb. Through studies with human participants, we have demonstrated that these sensors can enable a robot to track human motion in real time while providing assistance with dressing and bathing. We have also shown how these sensors can benefit a robot providing dressing assistance to real people with physical disabilities. Physics Simulation for Assistive Robotics: While robotic caregivers may present an opportunity to improve the quality of life for people who require daily assistance, conducting this type of research presents several challenges, including high costs, slow data collection, and risks of physical interaction between people and robots. We have recently introduced Assistive Gym, the first open source physics-based simulation framework for modeling physical human-robot interaction and robotic assistance. We demonstrate how physics simulation can open up entirely new research directions and opportunities within physical human-robot interaction. This includes training versatile assistive robots, developing control algorithms towards common sense reasoning, constructing baselines and benchmarks for robotic caregiving, and investigating generalization of physical human-robot interaction from a number of angles, including human motion, preferences, and variation in human body shape and impairments. Finally, we show how virtual reality (VR) can help bridge the reality gap by bringing real people into physics simulation to interact with and receive assistance from virtual robotic caregivers.Ph.D

Sensing Systems for Respiration Monitoring: A Technical Systematic Review

Respiratory monitoring is essential in sleep studies, sport training, patient monitoring, or health at work, among other applications. This paper presents a comprehensive systematic review of respiration sensing systems. After several systematic searches in scientific repositories, the 198 most relevant papers in this field were analyzed in detail. Different items were examined: sensing technique and sensor, respiration parameter, sensor location and size, general system setup, communication protocol, processing station, energy autonomy and power consumption, sensor validation, processing algorithm, performance evaluation, and analysis software. As a result, several trends and the remaining research challenges of respiration sensors were identified. Long-term evaluations and usability tests should be performed. Researchers designed custom experiments to validate the sensing systems, making it difficult to compare results. Therefore, another challenge is to have a common validation framework to fairly compare sensor performance. The implementation of energy-saving strategies, the incorporation of energy harvesting techniques, the calculation of volume parameters of breathing, or the effective integration of respiration sensors into clothing are other remaining research efforts. Addressing these and other challenges outlined in the paper is a required step to obtain a feasible, robust, affordable, and unobtrusive respiration sensing system

A Biomechanical and Physiological Signal Monitoring System for Four Degrees of Upper Limb Movement

A lack of adherence to prescribed physical therapy regimens in improper healing results in poor outcomes for those affected by musculoskeletal disorders (MSDs) of the upper limb. Societal and psychological barriers to proper adherence can be addressed through the system presented in this work consisting of the following components: an ambulatory biosignal acquisition sleeve, an electromyography (EMG) based motion repetition detection algorithm, and the design of a compatible capacitive EMG acquisition module. The biosignal acquisition sleeve was untethered, unobtrusive to motion, contained only modular components, and collected biomechanical and physiological sensor data to form full motion profiles of the following four degrees of freedom: elbow flexion—extension, forearm pronation—supination, wrist flexion—extension, and ulnar--radial deviation. The piloted sleeve simultaneously collected data from four inertial sensors, two electromyography (EMG) sensors and a flex-bend sensor. A visualization application was developed to present the information in a manner meaningful to the user. As well, an EMG based motion repetition detector was developed for use within the system. It was validated using an existing database of 23 subjects with varying musculoskeletal health, achieving a success rate of 95.43%. This algorithm was modified for use with the sleeve, resulting in a 95% success rate. An electrode and analog front end module was proposed, relying on unique material structures and low-noise, precision sensing techniques. The system prototype presented a resource-conscious tool for multi-modality tracking of elbow, forearm, and wrist motion, which could eventually be integrated into upper limb MSD rehabilitation

From wearable towards epidermal computing : soft wearable devices for rich interaction on the skin

Human skin provides a large, always available, and easy to access real-estate for interaction. Recent advances in new materials, electronics, and human-computer interaction have led to the emergence of electronic devices that reside directly on the user's skin. These conformal devices, referred to as Epidermal Devices, have mechanical properties compatible with human skin: they are very thin, often thinner than human hair; they elastically deform when the body is moving, and stretch with the user's skin. Firstly, this thesis provides a conceptual understanding of Epidermal Devices in the HCI literature. We compare and contrast them with other technical approaches that enable novel on-skin interactions. Then, through a multi-disciplinary analysis of Epidermal Devices, we identify the design goals and challenges that need to be addressed for advancing this emerging research area in HCI. Following this, our fundamental empirical research investigated how epidermal devices of different rigidity levels affect passive and active tactile perception. Generally, a correlation was found between the device rigidity and tactile sensitivity thresholds as well as roughness discrimination ability. Based on these findings, we derive design recommendations for realizing epidermal devices. Secondly, this thesis contributes novel Epidermal Devices that enable rich on-body interaction. SkinMarks contributes to the fabrication and design of novel Epidermal Devices that are highly skin-conformal and enable touch, squeeze, and bend sensing with co-located visual output. These devices can be deployed on highly challenging body locations, enabling novel interaction techniques and expanding the design space of on-body interaction. Multi-Touch Skin enables high-resolution multi-touch input on the body. We present the first non-rectangular and high-resolution multi-touch sensor overlays for use on skin and introduce a design tool that generates such sensors in custom shapes and sizes. Empirical results from two technical evaluations confirm that the sensor achieves a high signal-to-noise ratio on the body under various grounding conditions and has a high spatial accuracy even when subjected to strong deformations. Thirdly, Epidermal Devices are in contact with the skin, they offer opportunities for sensing rich physiological signals from the body. To leverage this unique property, this thesis presents rapid fabrication and computational design techniques for realizing Multi-Modal Epidermal Devices that can measure multiple physiological signals from the human body. Devices fabricated through these techniques can measure ECG (Electrocardiogram), EMG (Electromyogram), and EDA (Electro-Dermal Activity). We also contribute a computational design and optimization method based on underlying human anatomical models to create optimized device designs that provide an optimal trade-off between physiological signal acquisition capability and device size. The graphical tool allows for easily specifying design preferences and to visually analyze the generated designs in real-time, enabling designer-in-the-loop optimization. Experimental results show high quantitative agreement between the prediction of the optimizer and experimentally collected physiological data. Finally, taking a multi-disciplinary perspective, we outline the roadmap for future research in this area by highlighting the next important steps, opportunities, and challenges. Taken together, this thesis contributes towards a holistic understanding of Epidermal Devices}: it provides an empirical and conceptual understanding as well as technical insights through contributions in DIY (Do-It-Yourself), rapid fabrication, and computational design techniques.Die menschliche Haut bietet eine große, stets verfügbare und leicht zugängliche Fläche für Interaktion. Jüngste Fortschritte in den Bereichen Materialwissenschaft, Elektronik und Mensch-Computer-Interaktion (Human-Computer-Interaction, HCI) [so that you can later use the Englisch abbreviation] haben zur Entwicklung elektronischer Geräte geführt, die sich direkt auf der Haut des Benutzers befinden. Diese sogenannten Epidermisgeräte haben mechanische Eigenschaften, die mit der menschlichen Haut kompatibel sind: Sie sind sehr dünn, oft dünner als ein menschliches Haar; sie verformen sich elastisch, wenn sich der Körper bewegt, und dehnen sich mit der Haut des Benutzers. Diese Thesis bietet, erstens, ein konzeptionelles Verständnis von Epidermisgeräten in der HCI-Literatur. Wir vergleichen sie mit anderen technischen Ansätzen, die neuartige Interaktionen auf der Haut ermöglichen. Dann identifizieren wir durch eine multidisziplinäre Analyse von Epidermisgeräten die Designziele und Herausforderungen, die angegangen werden müssen, um diesen aufstrebenden Forschungsbereich voranzubringen. Im Anschluss daran untersuchten wir in unserer empirischen Grundlagenforschung, wie epidermale Geräte unterschiedlicher Steifigkeit die passive und aktive taktile Wahrnehmung beeinflussen. Im Allgemeinen wurde eine Korrelation zwischen der Steifigkeit des Geräts und den taktilen Empfindlichkeitsschwellen sowie der Fähigkeit zur Rauheitsunterscheidung festgestellt. Basierend auf diesen Ergebnissen leiten wir Designempfehlungen für die Realisierung epidermaler Geräte ab. Zweitens trägt diese Thesis zu neuartigen Epidermisgeräten bei, die eine reichhaltige Interaktion am Körper ermöglichen. SkinMarks trägt zur Herstellung und zum Design neuartiger Epidermisgeräte bei, die hochgradig an die Haut angepasst sind und Berührungs-, Quetsch- und Biegesensoren mit gleichzeitiger visueller Ausgabe ermöglichen. Diese Geräte können an sehr schwierigen Körperstellen eingesetzt werden, ermöglichen neuartige Interaktionstechniken und erweitern den Designraum für die Interaktion am Körper. Multi-Touch Skin ermöglicht hochauflösende Multi-Touch-Eingaben am Körper. Wir präsentieren die ersten nicht-rechteckigen und hochauflösenden Multi-Touch-Sensor-Overlays zur Verwendung auf der Haut und stellen ein Design-Tool vor, das solche Sensoren in benutzerdefinierten Formen und Größen erzeugt. Empirische Ergebnisse aus zwei technischen Evaluierungen bestätigen, dass der Sensor auf dem Körper unter verschiedenen Bedingungen ein hohes Signal-Rausch-Verhältnis erreicht und eine hohe räumliche Auflösung aufweist, selbst wenn er starken Verformungen ausgesetzt ist. Drittens, da Epidermisgeräte in Kontakt mit der Haut stehen, bieten sie die Möglichkeit, reichhaltige physiologische Signale des Körpers zu erfassen. Um diese einzigartige Eigenschaft zu nutzen, werden in dieser Arbeit Techniken zur schnellen Herstellung und zum computergestützten Design von multimodalen Epidermisgeräten vorgestellt, die mehrere physiologische Signale des menschlichen Körpers messen können. Die mit diesen Techniken hergestellten Geräte können EKG (Elektrokardiogramm), EMG (Elektromyogramm) und EDA (elektrodermale Aktivität) messen. Darüber hinaus stellen wir eine computergestützte Design- und Optimierungsmethode vor, die auf den zugrunde liegenden anatomischen Modellen des Menschen basiert, um optimierte Gerätedesigns zu erstellen. Diese Designs bieten einen optimalen Kompromiss zwischen der Fähigkeit zur Erfassung physiologischer Signale und der Größe des Geräts. Das grafische Tool ermöglicht die einfache Festlegung von Designpräferenzen und die visuelle Analyse der generierten Designs in Echtzeit, was eine Optimierung durch den Designer im laufenden Betrieb ermöglicht. Experimentelle Ergebnisse zeigen eine hohe quantitative Übereinstimmung zwischen den Vorhersagen des Optimierers und den experimentell erfassten physiologischen Daten. Schließlich skizzieren wir aus einer multidisziplinären Perspektive einen Fahrplan für zukünftige Forschung in diesem Bereich, indem wir die nächsten wichtigen Schritte, Möglichkeiten und Herausforderungen hervorheben. Insgesamt trägt diese Arbeit zu einem ganzheitlichen Verständnis von Epidermisgeräten bei: Sie liefert ein empirisches und konzeptionelles Verständnis sowie technische Einblicke durch Beiträge zu DIY (Do-It-Yourself), schneller Fertigung und computergestützten Entwurfstechniken

Sensor Developments for Electrophysiological Monitoring in Healthcare

Recent years have seen a renewal of interest in the development of sensor systems which can be used to monitor electrophysiological signals in a number of different settings. These include clinical, outside of the clinical setting with the subject ambulatory and going about their daily lives, and over long periods. The primary impetus for this is the challenge of providing healthcare for the ageing population based on home health monitoring, telehealth and telemedicine. Another stimulus is the demand for life sign monitoring of critical personnel such as fire fighters and military combatants. A related area of interest which, whilst not in the category of healthcare, utilises many of the same approaches, is that of sports physiology for both professional athletes and for recreation. Clinical diagnosis of conditions in, for example, cardiology and neurology remain based on conventional sensors, using established electrodes and well understood electrode placements. However, the demands of long term health monitoring, rehabilitation support and assistive technology for the disabled and elderly are leading research groups such as ours towards novel sensors, wearable and wireless enabled systems and flexible sensor arrays