Master of Science

thesisHaptic feedback in modern game controllers is limited to vibrotactile feedback. The addition of skin-stretch feedback would significantly improve the type and quality of haptic feedback provided by game controllers. Skin-stretch feedback requires small forces (around a few newtons) and translations (as small as 0.5 mm) to provide identifiable direction cues. Prior work has developed skin-stretch mechanisms in two form factors: a flat form factor and a tall but compact (cubic) form factor. These mechanisms have been shown to be effective actuators for skin-stretch feedback, and are small enough to fit inside of a game controller. Additional prior work has shown that the cubic skin-stretch mechanism can be integrated into a thumb joystick for use with game controllers. This thesis presents the design, characterization, and testing of two skin-stretch game controllers. The first game controller provides skin stretch via a 2-axis mechanism integrated into its thumb joysticks. This controller uses the cubic skin-stretch mechanism to drive the skin stretch. Concerns that users' motions of the joystick could negatively impact the saliency of skin stretch rendered from the joystick prompted the design of a controller that provides 2-axis skin stretch to users' middle fingers on the back side of the controller. Two experiments were conducted with the two controllers. One experiment had participants identify the direction of skin stretch from a selection of 8 possible directions. This test compared users' accuracies with both controllers, and with five different finger restraints on the back-tactor controller. Results show that users' identification accuracy was similar across feedback conditions. A second experiment used skin stretch to rotationally guide participants to a randomized target angle. Three different feedback strategies were tested. Results showed that a strategy called sinusoidal feedback, which provided feedback that varied in frequency and amplitude as a function of the user's relative position to the tactor, performed significantly better on all performance metrics than the other feedback strategies. It is important to note that the sinusoidal feedback only requires two 1-axis skin-stretch actuators, which are spatially separated, in order to provide feedback. The other lower performing feedback strategies used two 2-axis skin-stretch actuators

Doctor of Philosophy

dissertationWhen interacting with objects, humans utilize their sense of touch to provide information about the object and surroundings. However, in video games, virtual reality, and training exercises, humans do not always have information available through the sense of touch. Several types of haptic feedback devices have been created to provide touch information in these scenarios. This dissertation describes the use of tactile skin stretch feedback to provide cues that convey direction information to a user. The direction cues can be used to guide a user or provide information about the environment. The tactile skin stretch feedback devices described herein provide feedback directly to the hands, just as in many real life interactions involving the sense of touch. The devices utilize a moving tactor (actuated skin contact surface, also called a contactor) and surrounding material to give the user a sense of the relative motion. Several game controller prototypes with skin stretch feedback embedded into the device to interface with the fingers were constructed. Experiments were conducted to evaluate user performance in moving the joysticks to match the direction of the stimulus. These experiments investigated stimulus masking effects with both skin stretch feedback and vibrotactile feedback. A controller with feedback on the thumb joysticks was found to have higher user accuracy. Next, precision grip and power grip skin stretch feedback devices were created to investigate cues to convey motion in a three-dimensional space. Experiments were conducted to compare the two devices and to explore user accuracy in identifying different direction cue types. The precision grip device was found to be superior in communicating direction cues to users in four degrees of freedom. Finally, closed-loop control was implemented to guide users to a specific location and orientation within a three-dimensional space. Experiments were conducted to improve controller feedback which in turn improved user performance. Experiments were also conducted to investigate the feasibility of providing multiple cues in succession, in order to guide a user with multiple motions of the hand. It was found that users can successfully reach multiple target locations and orientations in succession

Master of Science

thesisHaptic interactions with smartphones are generally restricted to vibrotactile feedback that offers limited distinction between delivered tactile cues. The lateral movement of a small, high-friction contactor at the fingerpad can be used to induce skin stretch tangent to the skin's surface. This method has been demonstrated to reliably communicate four cardinal directions with 1 mm translations of the device's contactor, when finger motion is properly restrained. While earlier research has used a thimble to restrain the finger, this interface has been made portable by incorporating a simple conical hole as a finger restraint. An initial portable device design used RC hobby servos and the conical hole finger restraint, but the shape and size of this portable device wasn't compatible with smartphone form factors. This design also had significant compliance and backlash that must be compensated for with additional control schemes. In contrast, this thesis presents the design, fabrication, and testing of a low-profile skin-stretch display (LPSSD) with a novel actuation design for delivering complex tactile cues with minimal backlash or hysteresis of the skin contactor or "tactor." This flatter mechanism features embedded sensors for fingertip cursor control and selection. This device's nonlinear tactor motions are compensated for using table look-up and high-frequency open-loop control to create direction cues with 1.8 mm radial tactor displacements in 16 directions (distributed evenly every 22.5°) before returning to center. Two LPSSDs are incorporated into a smartphone peripheral and used in single-handed and bimanual tests to identify 16 directions. Users also participated in "relative" identification tests where they were first provided a reference direction cue in the forward/north direction followed by the cue direction that they were to identify. Tests were performed with the user's thumbs oriented in the forward direction and with thumbs angled inward slightly, similar to the angledthumb orientation console game controllers. Users are found to have increased performance with an angled-thumb orientation. They performed similarly when stimuli were delivered to their right or left thumbs, and had significantly better performance judging direction cues with both thumbs simultaneously. Participants also performed slightly better in identifying the relative direction cues than the absolute

Wearable haptic systems for the fingertip and the hand: taxonomy, review and perspectives

In the last decade, we have witnessed a drastic change in the form factor of audio and vision technologies, from heavy and grounded machines to lightweight devices that naturally fit our bodies. However, only recently, haptic systems have started to be designed with wearability in mind. The wearability of haptic systems enables novel forms of communication, cooperation, and integration between humans and machines. Wearable haptic interfaces are capable of communicating with the human wearers during their interaction with the environment they share, in a natural and yet private way. This paper presents a taxonomy and review of wearable haptic systems for the fingertip and the hand, focusing on those systems directly addressing wearability challenges. The paper also discusses the main technological and design challenges for the development of wearable haptic interfaces, and it reports on the future perspectives of the field. Finally, the paper includes two tables summarizing the characteristics and features of the most representative wearable haptic systems for the fingertip and the hand

From wearable towards epidermal computing : soft wearable devices for rich interaction on the skin

Human skin provides a large, always available, and easy to access real-estate for interaction. Recent advances in new materials, electronics, and human-computer interaction have led to the emergence of electronic devices that reside directly on the user's skin. These conformal devices, referred to as Epidermal Devices, have mechanical properties compatible with human skin: they are very thin, often thinner than human hair; they elastically deform when the body is moving, and stretch with the user's skin. Firstly, this thesis provides a conceptual understanding of Epidermal Devices in the HCI literature. We compare and contrast them with other technical approaches that enable novel on-skin interactions. Then, through a multi-disciplinary analysis of Epidermal Devices, we identify the design goals and challenges that need to be addressed for advancing this emerging research area in HCI. Following this, our fundamental empirical research investigated how epidermal devices of different rigidity levels affect passive and active tactile perception. Generally, a correlation was found between the device rigidity and tactile sensitivity thresholds as well as roughness discrimination ability. Based on these findings, we derive design recommendations for realizing epidermal devices. Secondly, this thesis contributes novel Epidermal Devices that enable rich on-body interaction. SkinMarks contributes to the fabrication and design of novel Epidermal Devices that are highly skin-conformal and enable touch, squeeze, and bend sensing with co-located visual output. These devices can be deployed on highly challenging body locations, enabling novel interaction techniques and expanding the design space of on-body interaction. Multi-Touch Skin enables high-resolution multi-touch input on the body. We present the first non-rectangular and high-resolution multi-touch sensor overlays for use on skin and introduce a design tool that generates such sensors in custom shapes and sizes. Empirical results from two technical evaluations confirm that the sensor achieves a high signal-to-noise ratio on the body under various grounding conditions and has a high spatial accuracy even when subjected to strong deformations. Thirdly, Epidermal Devices are in contact with the skin, they offer opportunities for sensing rich physiological signals from the body. To leverage this unique property, this thesis presents rapid fabrication and computational design techniques for realizing Multi-Modal Epidermal Devices that can measure multiple physiological signals from the human body. Devices fabricated through these techniques can measure ECG (Electrocardiogram), EMG (Electromyogram), and EDA (Electro-Dermal Activity). We also contribute a computational design and optimization method based on underlying human anatomical models to create optimized device designs that provide an optimal trade-off between physiological signal acquisition capability and device size. The graphical tool allows for easily specifying design preferences and to visually analyze the generated designs in real-time, enabling designer-in-the-loop optimization. Experimental results show high quantitative agreement between the prediction of the optimizer and experimentally collected physiological data. Finally, taking a multi-disciplinary perspective, we outline the roadmap for future research in this area by highlighting the next important steps, opportunities, and challenges. Taken together, this thesis contributes towards a holistic understanding of Epidermal Devices}: it provides an empirical and conceptual understanding as well as technical insights through contributions in DIY (Do-It-Yourself), rapid fabrication, and computational design techniques.Die menschliche Haut bietet eine große, stets verfügbare und leicht zugängliche Fläche für Interaktion. Jüngste Fortschritte in den Bereichen Materialwissenschaft, Elektronik und Mensch-Computer-Interaktion (Human-Computer-Interaction, HCI) [so that you can later use the Englisch abbreviation] haben zur Entwicklung elektronischer Geräte geführt, die sich direkt auf der Haut des Benutzers befinden. Diese sogenannten Epidermisgeräte haben mechanische Eigenschaften, die mit der menschlichen Haut kompatibel sind: Sie sind sehr dünn, oft dünner als ein menschliches Haar; sie verformen sich elastisch, wenn sich der Körper bewegt, und dehnen sich mit der Haut des Benutzers. Diese Thesis bietet, erstens, ein konzeptionelles Verständnis von Epidermisgeräten in der HCI-Literatur. Wir vergleichen sie mit anderen technischen Ansätzen, die neuartige Interaktionen auf der Haut ermöglichen. Dann identifizieren wir durch eine multidisziplinäre Analyse von Epidermisgeräten die Designziele und Herausforderungen, die angegangen werden müssen, um diesen aufstrebenden Forschungsbereich voranzubringen. Im Anschluss daran untersuchten wir in unserer empirischen Grundlagenforschung, wie epidermale Geräte unterschiedlicher Steifigkeit die passive und aktive taktile Wahrnehmung beeinflussen. Im Allgemeinen wurde eine Korrelation zwischen der Steifigkeit des Geräts und den taktilen Empfindlichkeitsschwellen sowie der Fähigkeit zur Rauheitsunterscheidung festgestellt. Basierend auf diesen Ergebnissen leiten wir Designempfehlungen für die Realisierung epidermaler Geräte ab. Zweitens trägt diese Thesis zu neuartigen Epidermisgeräten bei, die eine reichhaltige Interaktion am Körper ermöglichen. SkinMarks trägt zur Herstellung und zum Design neuartiger Epidermisgeräte bei, die hochgradig an die Haut angepasst sind und Berührungs-, Quetsch- und Biegesensoren mit gleichzeitiger visueller Ausgabe ermöglichen. Diese Geräte können an sehr schwierigen Körperstellen eingesetzt werden, ermöglichen neuartige Interaktionstechniken und erweitern den Designraum für die Interaktion am Körper. Multi-Touch Skin ermöglicht hochauflösende Multi-Touch-Eingaben am Körper. Wir präsentieren die ersten nicht-rechteckigen und hochauflösenden Multi-Touch-Sensor-Overlays zur Verwendung auf der Haut und stellen ein Design-Tool vor, das solche Sensoren in benutzerdefinierten Formen und Größen erzeugt. Empirische Ergebnisse aus zwei technischen Evaluierungen bestätigen, dass der Sensor auf dem Körper unter verschiedenen Bedingungen ein hohes Signal-Rausch-Verhältnis erreicht und eine hohe räumliche Auflösung aufweist, selbst wenn er starken Verformungen ausgesetzt ist. Drittens, da Epidermisgeräte in Kontakt mit der Haut stehen, bieten sie die Möglichkeit, reichhaltige physiologische Signale des Körpers zu erfassen. Um diese einzigartige Eigenschaft zu nutzen, werden in dieser Arbeit Techniken zur schnellen Herstellung und zum computergestützten Design von multimodalen Epidermisgeräten vorgestellt, die mehrere physiologische Signale des menschlichen Körpers messen können. Die mit diesen Techniken hergestellten Geräte können EKG (Elektrokardiogramm), EMG (Elektromyogramm) und EDA (elektrodermale Aktivität) messen. Darüber hinaus stellen wir eine computergestützte Design- und Optimierungsmethode vor, die auf den zugrunde liegenden anatomischen Modellen des Menschen basiert, um optimierte Gerätedesigns zu erstellen. Diese Designs bieten einen optimalen Kompromiss zwischen der Fähigkeit zur Erfassung physiologischer Signale und der Größe des Geräts. Das grafische Tool ermöglicht die einfache Festlegung von Designpräferenzen und die visuelle Analyse der generierten Designs in Echtzeit, was eine Optimierung durch den Designer im laufenden Betrieb ermöglicht. Experimentelle Ergebnisse zeigen eine hohe quantitative Übereinstimmung zwischen den Vorhersagen des Optimierers und den experimentell erfassten physiologischen Daten. Schließlich skizzieren wir aus einer multidisziplinären Perspektive einen Fahrplan für zukünftige Forschung in diesem Bereich, indem wir die nächsten wichtigen Schritte, Möglichkeiten und Herausforderungen hervorheben. Insgesamt trägt diese Arbeit zu einem ganzheitlichen Verständnis von Epidermisgeräten bei: Sie liefert ein empirisches und konzeptionelles Verständnis sowie technische Einblicke durch Beiträge zu DIY (Do-It-Yourself), schneller Fertigung und computergestützten Entwurfstechniken

Investigation of the feasibility of using focal vibratory stimulation with robotic aided therapy for spasticity rehabilitation in spinal cord injury

The occurrence of a traumatic spinal cord injury is in hundreds of thousands of people every year. Survivors are left with loss of many bodily functions, loss of sensation below the point of injury and many more painful and uncomfortable repercussions which interfere with activities of daily living. Over 70% of people with SCI develop spasticity: abnormally increased muscle tone and connected joint stiffness that interfere with residual volitional control of the limbs. Treatments for spasticity include many pharmacological and non-pharmacological techniques, however many of them have severe sideeffects. Evidence suggest the use of vibratory stimulation to relieve repercussions of spasticity, despite not agreeing on the most advantageous protocol. This thesis evaluated effects that focal vibratory stimulation have on the muscle performance. Within two studies, focal muscle vibration is compared against different application conditions such as timing and location. The results suggests that if focal vibrations are applied to the relaxed muscle, the increase in muscle's force is observed. Analysis of the cortical waves indicates minimal cortical involvement in vibratory stimulation modulation. On the other hand, FV applied of the connected tendon/bone imposed to a contraction seems to have a potential to increase muscle's activation. There is evidence that motor cortex is responding to this stimulation to stabilise the muscle in order to perform the contraction. Within clinical trial, focal muscle vibratory stimulation is employed in total of 6 interventional sessions while a joint's spastic exor and extensor muscles were relaxed. Spasticity appears to be reduced as a consequence of the stimulation. Moreover, engaging the joint into robotic-aided therapy increase volitional control of the wrist, according to the analysis of the active range of motion, joint stiffness and kinematic parameters associated to the movement. The measurement and movement facilitation device used in the clinical trial was designed and developed in accordance to the spasticity and spinal cord injury repercussions consideration. The studies conducted for this thesis demonstrated feasibility and potential for the use of focal muscle vibratory stimulation to enhance muscle power in healthy muscles but also relieve consequences of spasticity. Vibrations combined with movement robotic-aided therapy have a prospects to enhance motor control

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Ubiquitous haptic feedback in human-computer interaction through electrical muscle stimulation

[no abstract