Sensory and cognitive factors in multi-digit touch, and its integration with vision

Every tactile sensation – an itch, a kiss, a hug, a pen gripped between fingers, a soft fabric brushing against the skin – is experienced in relation to the body. Normally, they occur somewhere on the body’s surface – they have spatiality. This sense of spatiality is what allows us to perceive a partner’s caress in terms of its changing location on the skin, its movement direction, speed, and extent. How this spatiality arises and how it is experienced is a thriving research topic, compelled by growing interest in the nature of tactile experiences from product design to brain-machine interfaces. The present thesis adds to this flourishing area of research by examining the unified spatial quality of touch. How does distinct spatial information converge from separate areas of the body surface to give rise to our normal unified experience of touch? After explaining the importance of this question in Chapter 1, a novel paradigm to tackle this problem will be presented, whereby participants are asked to estimate the average direction of two stimuli that are simultaneously moved across two different fingerpads. This paradigm is a laboratory analogue of the more ecological task of representing the overall movement of an object held between multiple fingers. An EEG study in Chapter 2 will reveal a brain mechanism that could facilitate such aggregated perception. Next, by characterising participants’ performance not just in terms of error rates, but by considering perceptual sensitivity, bias, precision, and signal weighting, a series of psychophysical experiments will show that this aggregation ability differs for within- and between-hand perception (Chapter 3), is independent from somatotopically-defined circuitry (Chapter 4) and arises after proprioceptive input about hand posture is accounted for (Chapter 5). Finally, inspired by the demand for integrated tactile and visual experience in virtual reality and the potential of tactile interface to aid navigation, Chapter 6 will examine the contribution of tactile spatiality on visual spatial experience. Ultimately, the present thesis will reveal sensory factors that limit precise representation of concurrently occurring dynamic tactile events. It will point to cognitive strategies the brain may employ to overcome those limitations to tactually perceive coherent objects. As such, this thesis advances somatosensory research beyond merely examining the selectivity to and discrimination between experienced tactile inputs, to considering the unified experience of touch despite distinct stimulus elements. The findings also have practical implications for the design of functional tactile interfaces

Multi-digit tactile perception I: motion integration benefits for tactile trajectories presented bimanually

Interactions with objects involve simultaneous contact with multiple, not necessarily adjacent, skin regions. While advances have been made in understanding the capacity to selectively attend to a single tactile element among distracting stimulations, here, we examine how multiple stimulus elements are explicitly integrated into an overall tactile percept. Across four experiments, participants averaged the direction of two simultaneous tactile motion trajectories of varying discrepancy delivered to different fingerpads. Averaging performance differed between within- and between-hands conditions in terms of sensitivity and precision but was unaffected by somatotopic proximity between stimulated fingers. First, precision was greater in between-hand compared to within-hand conditions, demonstrating a bimanual perceptual advantage in multi-touch integration. Second, sensitivity to the average direction was influenced by the discrepancy between individual motion signals, but only for within-hand conditions. Overall, our experiments identify key factors that influence perception of simultaneous tactile events. In particular, we show that multi-touch integration is constrained by hand-specific rather than digit-specific mechanisms

Somatosensory evoked potentials that index lateral inhibition are modulated according to the mode of perceptual processing: comparing or combining multi-digit tactile motion

Many perceptual studies focus on the brain’s capacity to discriminate between stimuli. However, our normal experience of the world also involves integrating multiple stimuli into a single perceptual event. Neural mechanisms such as lateral inhibition are believed to enhance local differences between sensory inputs from nearby regions of the receptor surface. However, this mechanism would seem dysfunctional when sensory inputs need to be combined rather than contrasted. Here, we investigated whether the brain can strategically regulate the strength of suppressive interactions that underlie lateral inhibition between finger representations in human somatosensory processing. To do this, we compared sensory processing between conditions that required either comparing or combining information. We delivered two simultaneous tactile motion trajectories to index and middle fingertips of the right hand. Participants had to either compare the directions of the two stimuli, or to combine them to form their average direction. To reveal preparatory tuning of somatosensory cortex, we used an established event-related potential design to measure the interaction between cortical representations evoked by digital nerve shocks immediately before each tactile stimulus. Consistent with previous studies, we found a clear suppression between cortical activations when participants were instructed to compare the tactile motion directions. Importantly, this suppression was significantly reduced when participants had to combine the same stimuli. These findings suggest that the brain can strategically switch between a comparative and a combinative mode of somatosensory processing, according to the perceptual goal, by preparatorily adjusting the strength of a process akin to lateral inhibition

The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer

Large multilingual language models such as mBERT or XLM-R enable zero-shot cross-lingual transfer in various IR and NLP tasks. Cao et al. (2020) proposed a data- and compute-efficient method for cross-lingual adjustment of mBERT that uses a small parallel corpus to make embeddings of related words across languages similar to each other. They showed it to be effective in NLI for five European languages. In contrast we experiment with a typologically diverse set of languages (Spanish, Russian, Vietnamese, and Hindi) and extend their original implementations to new tasks (XSR, NER, and QA) and an additional training regime (continual learning). Our study reproduced gains in NLI for four languages, showed improved NER, XSR, and cross-lingual QA results in three languages (though some cross-lingual QA gains were not statistically significant), while mono-lingual QA performance never improved and sometimes degraded. Analysis of distances between contextualized embeddings of related and unrelated words (across languages) showed that fine-tuning leads to "forgetting" some of the cross-lingual alignment information. Based on this observation, we further improved NLI performance using continual learning.Comment: Presented at ECIR 202

Students' professional competence formation in the process of foreign language learning in high school as a modern challenge

The actuality of the investigated problem is caused by changes in society and in the modern production. These changes make it necessary to improve the training of students at universities. This involves improving the efficiency of teaching of all subjects, including foreign languages. The purpose of the article is to identify the need to disclose the nature and show the ways of students' professional competence formation in the process of language training. The leading approaches to the study of this problem are competence, student-centered and communicative approaches. The main results of the study prove to be systematization of approaches to the interpretation of the concepts of competence and expertise in scientific literature. The importance and the possibility of students' professional competence formation by means of a foreign language are established

Seeing Through Each Other's Hearts: Inferring Others' Heart Rate as a Function of Own Heart Rate Perception and Perceived Social Intelligence

Successful social interactions require a good understanding of the emotional states of other people. This information is often not directly communicated but must be inferred. As all emotional experiences are also imbedded in the visceral or interoceptive state of the body (i.e., accelerating heart rate during arousal), successfully inferring the interoceptive states of others may open a window into their emotional state. But how well can people do that? Here, we replicate recent results showing that people can discriminate between the cardiac states (i.e., the resting heartrate) of other people by simply looking at them. We further tested whether the ability to infer the interoceptive states of others depends on one’s own interoceptive abilities. We measured people’s performance in a cardioception task and their self-reported interoceptive accuracy. Whilst neither was directly associated to their ability to infer the heartrate of another person, we found a significant interaction. Specifically, overestimating one’s own interoceptive capacities was associated with a worse performance at inferring the heartrate of others. In contrast, underestimating one’s own interoceptive capacities did not have such influence. This pattern suggests that deficient beliefs about own interoceptive capacities can have detrimental effects on inferring the interoceptive states of other people

Perceived time expands and contracts within each heartbeat

Perception of passing time can be distorted. Emotional experiences, particularly arousal, can contract or expand experienced duration via their interactions with attentional and sensory processing mechanisms. Current models suggest that perceived duration can be encoded from accumulation processes and from temporally evolving neural dynamics. Yet all neural dynamics and information processing ensue at the backdrop of continuous interoceptive signals originating from within the body. Indeed, phasic fluctuations within the cardiac cycle impact neural and information processing. Here, we show that these momentary cardiac fluctuations distort experienced time and that their effect interacts with subjectively experienced arousal. In a temporal bisection task, durations (200-400 ms) of an emotionally neutral visual shape or auditory tone (experiment 1) or of an image displaying happy or fearful facial expressions (experiment 2) were categorized as short or long. Across both experiments, stimulus presentation was time-locked to systole, when the heart contracts and baroreceptors fire signals to the brain, and to diastole, when the heart relaxes, and baroreceptors are quiescent. When participants judged the duration of emotionally neural stimuli (experiment 1), systole led to temporal contraction, whereas diastole led to temporal expansion. Such cardiac-led distortions were further modulated by the arousal ratings of the perceived facial expressions (experiment 2). At low arousal, systole contracted while diastole expanded time, but as arousal increased, this cardiac-led time distortion disappeared, shifting duration perception toward contraction. Thus, experienced time contracts and expands within each heartbeat-a balance that is disrupted under heightened arousal. [Abstract copyright: Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.