Gamma Band Oscillation Response to Somatosensory Feedback Stimulation Schemes Constructed on Basis of Biphasic Neural Touch Representation

abstract: Prosthetic users abandon devices due to difficulties performing tasks without proper graded or interpretable feedback. The inability to adequately detect and correct error of the device leads to failure and frustration. In advanced prostheses, peripheral nerve stimulation can be used to deliver sensations, but standard schemes used in sensorized prosthetic systems induce percepts inconsistent with natural sensations, providing limited benefit. Recent uses of time varying stimulation strategies appear to produce more practical sensations, but without a clear path to pursue improvements. This dissertation examines the use of physiologically based stimulation strategies to elicit sensations that are more readily interpretable. A psychophysical experiment designed to investigate sensitivities to the discrimination of perturbation direction within precision grip suggests that perception is biomechanically referenced: increased sensitivities along the ulnar-radial axis align with potential anisotropic deformation of the finger pad, indicating somatosensation uses internal information rather than environmental. Contact-site and direction dependent deformation of the finger pad activates complimentary fast adapting and slow adapting mechanoreceptors, exhibiting parallel activity of the two associate temporal patterns: static and dynamic. The spectrum of temporal activity seen in somatosensory cortex can be explained by a combined representation of these distinct response dynamics, a phenomenon referred in this dissertation to “biphasic representation.” In a reach-to-precision-grasp task, neurons in somatosensory cortex were found to possess biphasic firing patterns in their responses to texture, orientation, and movement. Sensitivities seem to align with variable deformation and mechanoreceptor activity: movement and smooth texture responses align with potential fast adapting activation, non-movement and coarse texture responses align with potential increased slow adapting activation, and responses to orientation are conceptually consistent with coding of tangential load. Using evidence of biphasic representations’ association with perceptual priorities, gamma band phase locking is used to compare responses to peripheral nerve stimulation patterns and mechanical stimulation. Vibrotactile and punctate mechanical stimuli are used to represent the practical and impractical percepts commonly observed in peripheral nerve stimulation feedback. Standard patterns of constant parameters closely mimic impractical vibrotactile stimulation while biphasic patterns better mimic punctate stimulation and provide a platform to investigate intragrip dynamics representing contextual activation.Dissertation/ThesisDoctoral Dissertation Biomedical Engineering 201

Cortical Orchestra Conducted by Purpose and Function

학위논문(박사)--서울대학교 대학원 :자연과학대학 협동과정 뇌과학전공,2020. 2. 정천기.촉각과 자기수용감각은 우리의 생존 및 일상생활에 절대적인 영향을 미치는 중요한 감각 기능이다. 말초신경계에서 이 두 가지 기능들에 필요한 정보를 수집하고 전달하는 기계적 수용기 및 그 구심성 신경들에 대한 신호 전달 메커니즘 및 그 특징들은 상대적으로 잘 알려져 있는 편이다. 그러나, 촉각과 자기수용감각을 형성하기 위한 인간 뇌의 피질에서의 정보 처리 메커니즘에 대하여 우리가 현재 알고 있는 바는 극히 일부분이다. 이 논문에서 제시하는 일련의 연구들은 인간 뇌 피질 단계에서 촉각과 자기수용감각의 지각적 처리과정에 대한 거시적 신경계 정보처리 메커니즘을 다룬다. 첫 번째 연구에서는 뇌피질뇌파를 이용하여 인간 일차 및 이차 체성감각 피질에서 인공적인 자극과 일상생활에서 접할 수 있는 자극을 포함하는 다양한 진동촉감각 및 질감 자극에 대한 거시적 신경계 정보처리 특성을 밝혔다. 이 연구에서는 일차 및 이차 체성감각 피질의 촉감각 주파수 특이적인 하이-감마 영역 신경활동이 자극 주파수에 따라 각각 상이한 시간적 다이나믹스를 가지고 변화하는 것을 확인하였다. 또한, 이러한 하이-감마 활동은 성긴 질감과 미세한 입자감을 가진 자연스러운 질감 자극에 대해서도 진동촉감각의 경우와 유사한 패턴을 보였다. 이러한 결과들은 인간의 진동촉감각이 매우 단순한 형태에 자극일지라도 대뇌 체성감각 시스템에 있어 거시적인 다중 영역에서의 계층적 정보처리를 동반한다는 점을 시사한다. 두 번째 연구에서는 인간의 움직임과 관련된 두정엽 영역에서의 하이-감마 뇌활성이 자기수용감각과 같은 말초신경계로부터의 체성감각 피드백을 주로 반영하는지, 아니면 움직임 준비 및 제어를 위한 피질 간 신경 프로세스에 대한 활동을 반영하는지를 조사하였다. 연구 결과, 자발적 운동 중 대뇌 운동감각령에서의 하이-감마 활동은 일차 체성감각피질이 일차 운동피질보다 더 지배적인 것으로 나타났다. 또한 이 연구에서는, 움직임과 관련된 일차 체성감각피질에서의 하이-감마 뇌활동은 말초신경계로부터의 자기수용감각과 촉각에 대한 신경계 정보처리를 주로 반영하는 것을 밝혔다. 이러한 연구들을 바탕으로, 마지막 연구에서는 인간 대뇌에서의 체성감각 지각 프로세스에 대한 거시적 피질 간 네트워크를 규명하고자 하였다. 이를 위해, 51명의 뇌전증 환자에게서 체성감각을 유발했던 뇌피질전기자극 데이터와 46명의 환자에게서 촉감각 자극 및 운동 수행 중에 측정한 뇌피질뇌파 하이-감마 매핑 데이터를 종합적으로 분석하였다. 그 결과, 체성감각 지각 프로세스는 대뇌에서 넓은 영역에 걸쳐 분포하는 체성감각 관련 네트워크의 신경 활성을 수반한다는 것을 알아냈다. 또한, 뇌피질전기자극을 통한 대뇌 지도와 하이-감마 매핑을 통한 대뇌 지도는 서로 상당한 유사성을 보였다. 흥미롭게도, 뇌피질전기자극과 하이-감마 활동을 종합한 뇌지도들로부터 체성감각 관련 뇌 영역의 공간적 분포가 체성감각 기능에 따라 서로 달랐고, 그에 해당하는 각 영역들은 서로 뚜렷하게 다른 시간적 다이나믹스를 가지고 순차적으로 활성화되었다. 이러한 결과들은 체성감각에 대한 거시적 신경계 프로세스가 그 지각적 기능에 따라 뚜렷이 다른 계층적 네트워크를 가진다는 점을 시사한다. 더 나아가, 본 연구에서의 결과들은 체성감각 시스템의 지각-행동 관련 신경활동 흐름에 관한 이론적인 가설에 대하여 설득력 있는 증거를 제시하고 있다.Tactile and proprioceptive perceptions are crucial for our daily life as well as survival. At the peripheral level, the transduction mechanisms and characteristics of mechanoreceptive afferents containing information required for these functions, have been well identified. However, our knowledge about the cortical processing mechanism for them in human is limited. The present series of studies addressed the macroscopic neural mechanism for perceptual processing of tactile and proprioceptive perception in human cortex. In the first study, I investigated the macroscopic neural characteristics for various vibrotactile and texture stimuli including artificial and naturalistic ones in human primary and secondary somatosensory cortices (S1 and S2, respectively) using electrocorticography (ECoG). I found robust tactile frequency-specific high-gamma (HG, 50–140 Hz) activities in both S1 and S2 with different temporal dynamics depending on the stimulus frequency. Furthermore, similar HG patterns of S1 and S2 were found in naturalistic stimulus conditions such as coarse/fine textures. These results suggest that human vibrotactile sensation involves macroscopic multi-regional hierarchical processing in the somatosensory system, even during the simplified stimulation. In the second study, I tested whether the movement-related HG activities in parietal region mainly represent somatosensory feedback such as proprioception from periphery or primarily indicate cortico-cortical neural processing for movement preparation and control. I found that sensorimotor HG activities are more dominant in S1 than in M1 during voluntary movement. Furthermore, the results showed that movement-related HG activities in S1 mainly represent proprioceptive and tactile feedback from periphery. Given the results of previous two studies, the final study aimed to identify the large-scale cortical networks for perceptual processing in human. To do this, I combined direct cortical stimulation (DCS) data for eliciting somatosensation and ECoG HG band (50 to 150 Hz) mapping data during tactile stimulation and movement tasks, from 51 (for DCS mapping) and 46 patients (for HG mapping) with intractable epilepsy. The results showed that somatosensory perceptual processing involves neural activation of widespread somatosensory-related network in the cortex. In addition, the spatial distributions of DCS and HG functional maps showed considerable similarity in spatial distribution between high-gamma and DCS functional maps. Interestingly, the DCS-HG combined maps showed distinct spatial distributions depending on the somatosensory functions, and each area was sequentially activated with distinct temporal dynamics. These results suggest that macroscopic neural processing for somatosensation has distinct hierarchical networks depending on the perceptual functions. In addition, the results of the present study provide evidence for the perception and action related neural streams of somatosensory system. Throughout this series of studies, I suggest that macroscopic somatosensory network and structures of our brain are intrinsically organized by perceptual function and its purpose, not by somatosensory modality or submodality itself. Just as there is a purpose for human behavior, so is our brain.PART I. INTRODUCTION 1 CHAPTER 1: Somatosensory System 1 1.1. Mechanoreceptors in the Periphery 2 1.2. Somatosensory Afferent Pathways 4 1.3. Cortico-cortical Connections among Somatosensory-related Areas 7 1.4. Somatosensory-related Cortical Regions 8 CHAPTER 2: Electrocorticography 14 2.1. Intracranial Electroencephalography 14 2.2. High-Gamma Band Activity 18 CHAPTER 3: Purpose of This Study 24 PART II. EXPERIMENTAL STUDY 26 CHAPTER 4: Apparatus Design 26 4.1. Piezoelectric Vibrotactile Stimulator 26 4.2. Magnetic Vibrotactile Stimulator 29 4.3. Disc-type Texture Stimulator 33 4.4. Drum-type Texture Stimulator 36 CHAPTER 5: Vibrotactile and Texture Study 41 5.1. Introduction 42 5.2. Materials and Methods 46 5.2.1. Patients 46 5.2.2. Apparatus 47 5.2.3. Experimental Design 49 5.2.4. Data Acquisition and Preprocessing 50 5.2.5. Analysis 51 5.3. Results 54 5.3.1. Frequency-specific S1/S2 HG Activities 54 5.3.2. S1 HG Attenuation during Flutter and Vibration 62 5.3.3. Single-trial Vibration Frequency Classification 64 5.3.4. S1/S2 HG Activities during Texture Stimuli 65 5.4. Discussion 69 5.4.1. Comparison with Previous Findings 69 5.4.2. Tactile Frequency-dependent Neural Adaptation 70 5.4.3. Serial vs. Parallel Processing between S1 and S2 72 5.4.4. Conclusion of Chapter 5 73 CHAPTER 6: Somatosensory Feedback during Movement 74 6.1. Introduction 75 6.2. Materials and Methods 79 6.2.1. Subjects 79 6.2.2. Tasks 80 6.2.3. Data Acquisition and Preprocessing 82 6.2.4. S1-M1 HG Power Difference 85 6.2.5. Classification 86 6.2.6. Timing of S1 HG Activity 86 6.2.7. Correlation between HG and EMG signals 87 6.3. Results 89 6.3.1. HG Activities Are More Dominant in S1 than in M1 89 6.3.2. HG Activities in S1 Mainly Represent Somatosensory Feedback 94 6.4. Discussion 100 6.4.1. S1 HG Activity Mainly Represents Somatosensory Feedback 100 6.4.2. Further Discussion and Future Direction in BMI 102 6.4.3. Conclusion of Chapter 6 103 CHAPTER 7: Cortical Maps of Somatosensory Function 104 7.1. Introduction 106 7.2. Materials and Methods 110 7.2.1. Participants 110 7.2.2. Direct Cortical Stimulation 114 7.2.3. Classification of Verbal Feedbacks 115 7.2.4. Localization of Electrodes 115 7.2.5. Apparatus 116 7.2.6. Tasks 117 7.2.7. Data Recording and Processing 119 7.2.8. Mapping on the Brain 120 7.2.9. ROI-based Analysis 122 7.3. Results 123 7.3.1. DCS Mapping 123 7.3.2. Three and Four-dimensional HG Mapping 131 7.3.3. Neural Characteristics among Somatosensory-related Areas 144 7.4. Discussion 146 7.4.1. DCS on the Non-Primary Areas 146 7.4.2. Two Streams of Somatosensory System 148 7.4.3. Functional Role of ventral PM 151 7.4.4. Limitation and Perspective 152 7.4.5. Conclusion of Chapter 7 155 PART III. CONCLUSION 156 CHAPTER 8: Conclusion and Perspective 156 8.1. Perspective and Future Work 157 References 160 Abstract in Korean 173Docto

A Review of Smart Materials in Tactile Actuators for Information Delivery

As the largest organ in the human body, the skin provides the important sensory channel for humans to receive external stimulations based on touch. By the information perceived through touch, people can feel and guess the properties of objects, like weight, temperature, textures, and motion, etc. In fact, those properties are nerve stimuli to our brain received by different kinds of receptors in the skin. Mechanical, electrical, and thermal stimuli can stimulate these receptors and cause different information to be conveyed through the nerves. Technologies for actuators to provide mechanical, electrical or thermal stimuli have been developed. These include static or vibrational actuation, electrostatic stimulation, focused ultrasound, and more. Smart materials, such as piezoelectric materials, carbon nanotubes, and shape memory alloys, play important roles in providing actuation for tactile sensation. This paper aims to review the background biological knowledge of human tactile sensing, to give an understanding of how we sense and interact with the world through the sense of touch, as well as the conventional and state-of-the-art technologies of tactile actuators for tactile feedback delivery

Tactons: structured tactile messages for non-visual information display

Tactile displays are now becoming available in a form that can be easily used in a user interface. This paper describes a new form of tactile output. Tactons, or tactile icons, are structured, abstract messages that can be used to communicate messages non-visually. A range of different parameters can be used for Tacton construction including: frequency, amplitude and duration of a tactile pulse, plus other parameters such as rhythm and location. Tactons have the potential to improve interaction in a range of different areas, particularly where the visual display is overloaded, limited in size or not available, such as interfaces for blind people or in mobile and wearable devices. This paper describes Tactons, the parameters used to construct them and some possible ways to design them. Examples of where Tactons might prove useful in user interfaces are given

Sensing with the Motor Cortex

The primary motor cortex is a critical node in the network of brain regions responsible for voluntary motor behavior. It has been less appreciated, however, that the motor cortex exhibits sensory responses in a variety of modalities including vision and somatosensation. We review current work that emphasizes the heterogeneity in sensorimotor responses in the motor cortex and focus on its implications for cortical control of movement as well as for brain-machine interface development

VIBES: Vibro-Inertial Bionic Enhancement System in a Prosthetic Socket

The use of vibrotactile feedback is of growing interest in the field of prosthetics, but few devices fully integrate this technology in the prosthesis to transmit high-frequency contact information (such as surface roughness and first contact) arising from the interaction of the prosthetic device with external items. This study describes a wearable vibrotactile system for high-frequency tactile information embedded in the prosthetic socket. The device consists of two compact planar vibrotactile actuators in direct contact with the user's skin to transmit tactile cues. These stimuli are directly related to the acceleration profiles recorded with two IMUS placed on the distal phalanx of a soft under-actuated robotic prosthesis (Soft-Hand Pro). We characterized the system from a psychophysical point of view with fifteen able-bodied participants by computing participants' Just Noticeable Difference (JND) related to the discrimination of vibrotactile cues delivered on the index finger, which are associated with the exploration of different sandpapers. Moreover, we performed a pilot experiment with one SoftHand Pro prosthesis user by designing a task, i.e. Active Texture Identification, to investigate if our feedback could enhance users' roughness discrimination. Results indicate that the device can effectively convey contact and texture cues, which users can readily detect and distinguish

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Touching on elements for a non-invasive sensory feedback system for use in a prosthetic hand

Hand amputation results in the loss of motor and sensory functions, impacting activities of daily life and quality of life. Commercially available prosthetic hands restore the motor function but lack sensory feedback, which is crucial to receive information about the prosthesis state in real-time when interacting with the external environment. As a supplement to the missing sensory feedback, the amputee needs to rely on visual and audio cues to operate the prosthetic hand, which can be mentally demanding. This thesis revolves around finding potential solutions to contribute to an intuitive non-invasive sensory feedback system that could be cognitively less burdensome and enhance the sense of embodiment (the feeling that an artificial limb belongs to one’s own body), increasing acceptance of wearing a prosthesis.A sensory feedback system contains sensors to detect signals applied to the prosthetics. The signals are encoded via signal processing to resemble the detected sensation delivered by actuators on the skin. There is a challenge in implementing commercial sensors in a prosthetic finger. Due to the prosthetic finger’s curvature and the fact that some prosthetic hands use a covering rubber glove, the sensor response would be inaccurate. This thesis shows that a pneumatic touch sensor integrated into a rubber glove eliminates these errors. This sensor provides a consistent reading independent of the incident angle of stimulus, has a sensitivity of 0.82 kPa/N, a hysteresis error of 2.39±0.17%, and a linearity error of 2.95±0.40%.For intuitive tactile stimulation, it has been suggested that the feedback stimulus should be modality-matched with the intention to provide a sensation that can be easily associated with the real touch on the prosthetic hand, e.g., pressure on the prosthetic finger should provide pressure on the residual limb. A stimulus should also be spatially matched (e.g., position, size, and shape). Electrotactile stimulation has the ability to provide various sensations due to it having several adjustable parameters. Therefore, this type of stimulus is a good candidate for discrimination of textures. A microphone can detect texture-elicited vibrations to be processed, and by varying, e.g., the median frequency of the electrical stimulation, the signal can be presented on the skin. Participants in a study using electrotactile feedback showed a median accuracy of 85% in differentiating between four textures.During active exploration, electrotactile and vibrotactile feedback provide spatially matched modality stimulations, providing continuous feedback and providing a displaced sensation or a sensation dispatched on a larger area. Evaluating commonly used stimulation modalities using the Rubber Hand Illusion, modalities which resemble the intended sensation provide a more vivid illusion of ownership for the rubber hand.For a potentially more intuitive sensory feedback, the stimulation can be somatotopically matched, where the stimulus is experienced as being applied on a site corresponding to their missing hand. This is possible for amputees who experience referred sensation on their residual stump. However, not all amputees experience referred sensations. Nonetheless, after a structured training period, it is possible to learn to associate touch with specific fingers, and the effect persisted after two weeks. This effect was evaluated on participants with intact limbs, so it remains to evaluate this effect for amputees.In conclusion, this thesis proposes suggestions on sensory feedback systems that could be helpful in future prosthetic hands to (1) reduce their complexity and (2) enhance the sense of body ownership to enhance the overall sense of embodiment as an addition to an intuitive control system