Cross-modal visuo-haptic mental rotation: comparing objects between senses

The simple experience of a coherent percept while looking and touching an object conceals an intriguing issue: different senses encode and compare information in different modality-specific reference frames. We addressed this problem in a cross-modal visuo-haptic mental rotation task. Two objects in various orientations were presented at the same spatial location, one visually and one haptically. Participants had to identify the objects as same or different. The relative angle between viewing direction and hand orientation was manipulated (Aligned versus Orthogonal). In an additional condition (Delay), a temporal delay was introduced between haptic and visual explorations while the viewing direction and the hand orientation were orthogonal to each other. Whereas the phase shift of the response time function was close to 0° in the Aligned condition, we observed a consistent phase shift in the hand’s direction in the Orthogonal condition. A phase shift, although reduced, was also found in the Delay condition. Counterintuitively, these results mean that seen and touched objects do not need to be physically aligned for optimal performance to occur. The present results suggest that the information about an object is acquired in separate visual and hand-centered reference frames, which directly influence each other and which combine in a time-dependent manner

Spike residue 403 affects binding of coronavirus spikes to human ACE2

The bat sarbecovirus RaTG13 is a close relative of SARS-CoV-2, the cause of the COVID-19 pandemic. However, this bat virus was most likely unable to directly infect humans since its Spike (S) protein does not interact efficiently with the human ACE2 receptor. Here, we show that a single T403R mutation increases binding of RaTG13 S to human ACE2 and allows VSV pseudoparticle infection of human lung cells and intestinal organoids. Conversely, mutation of R403T in the SARS-CoV-2 S reduces pseudoparticle infection and viral replication. The T403R RaTG13 S is neutralized by sera from individuals vaccinated against COVID-19 indicating that vaccination might protect against future zoonoses. Our data suggest that a positively charged amino acid at position 403 in the S protein is critical for efficient utilization of human ACE2 by S proteins of bat coronaviruses. This finding could help to better predict the zoonotic potential of animal coronaviruses

Comparison of the haptic and visual deviations in a parallelity task

Deviations in both haptic and visual spatial experiments are thought to be caused by a biasing influence of an egocentric reference frame. The strength of this influence is strongly participant-dependent. By using a parallelity test, it is studied whether this strength is modality-independent. In both haptic and visual conditions, large, systematic and participant-dependent deviations were found. However, although the correlation between the haptic and visual deviations was significant, the explained variance due to a common factor was only 20%. Therefore, the degree to which a participant is “egocentric” depends on modality and possibly even more generally, on experimental condition

Keep an eye on your hands: on the role of visual mechanisms in processing of haptic space

The present paper reviews research on a haptic orientation processing. Central is a task in which a test bar has to be set parallel to a reference bar at another location. Introducing a delay between inspecting the reference bar and setting the test bar leads to a surprising improvement. Moreover, offering visual background information also elevates performance. Interestingly, (congenitally) blind individuals do not or to a weaker extent show the improvement with time, while in parallel to this, they appear to benefit less from spatial imagery processing. Together this strongly points to an important role for visual processing mechanisms in the perception of haptic inputs

Gaze-grasp coordination in obstacle avoidance: differences between binocular and monocular viewing

Most adults can skillfully avoid potential obstacles when acting in everyday cluttered scenes. We examined how gaze and hand movements are normally coordinated for obstacle avoidance and whether these are altered when binocular depth information is unavailable. Visual fixations and hand movement kinematics were simultaneously recorded, while 13 right-handed subjects reached-to-precision grasp a cylindrical household object presented alone or with a potential obstacle (wine glass) located to its left (thumb's grasp side), right or just behind it (both closer to the finger's grasp side) using binocular or monocular vision. Gaze and hand movement strategies differed significantly by view and obstacle location. With binocular vision, initial fixations were near the target's centre of mass (COM) around the time of hand movement onset, but usually shifted to end just above the thumb's grasp site at initial object contact, this mainly being made by the thumb, consistent with selecting this digit for guiding the grasp. This strategy was associated with faster binocular hand movements and improved end-point grip precision across all trials than with monocular viewing, during which subjects usually continued to fixate the target closer to its COM despite a similar prevalence of thumb-first contacts. While subjects looked directly at the obstacle at each location on a minority of trials and their overall fixations on the target were somewhat biased towards the grasp side nearest to it, these gaze behaviours were particularly marked on monocular vision-obstacle behind trials which also commonly ended in finger-first contact. Subjects avoided colliding with the wine glass under both views when on the right (finger side) of the workspace by producing slower and straighter reaches, with this and the behind obstacle location also resulting in 'safer' (i.e. narrower) peak grip apertures and longer deceleration times than when the goal object was alone or the obstacle was on its thumb side. But monocular reach paths were more variable and deceleration times were selectively prolonged on finger-side and behind obstacle trials, with this latter condition further resulting in selectively increased grip closure times and corrections. Binocular vision thus provided added advantages for collision avoidance, known to require intact dorsal cortical stream processing mechanisms, particularly when the target of the grasp and potential obstacle to it were fairly closely separated in depth. Different accounts of the altered monocular gaze behaviour converged on the conclusion that additional perceptual and/or attentional resources are likely engaged compared to when continuous binocular depth information is available. Implications for people lacking binocular stereopsis are briefly considered

Multiple reference frames in haptic spatial processing

The present thesis focused on haptic spatial processing. In particular, our interest was directed to the perception of spatial relations with the main focus on the perception of orientation. To this end, we studied haptic perception in different tasks, either in isolation or in combination with vision. The parallelity task, where participants have to match the orientations of two spatially separated bars, was used in its two-dimensional and three-dimensional versions in Chapter 2 and Chapter 3, respectively. The influence of non-informative vision and visual interference on performance in the parallelity task was studied in Chapter 4. A different task, the mental rotation task, was introduced in a purely haptic study in Chapter 5 and in a visuo-haptic cross-modal study in Chapter 6. The interaction of multiple reference frames and their influence on haptic spatial processing were the common denominators of these studies. In this thesis we approached the problems of which reference frames play the major role in haptic spatial processing and how the relative roles of distinct reference frames change depending on the available information and the constraints imposed by different tasks. We found that the influence of a reference frame centered on the hand was the major cause of the deviations from veridicality observed in both the two-dimensional and three-dimensional studies. The results were described by a weighted average model, in which the hand-centered egocentric reference frame is supposed to have a biasing influence on the allocentric reference frame. Performance in haptic spatial processing has been shown to depend also on sources of information or processing that are not strictly connected to the task at hand. When non-informative vision was provided, a beneficial effect was observed in the haptic performance. This improvement was interpreted as a shift from the egocentric to the allocentric reference frame. Moreover, interfering visual information presented in the vicinity of the haptic stimuli parametrically modulated the magnitude of the deviations. The influence of the hand-centered reference frame was shown also in the haptic mental rotation task where participants were quicker in judging the parity of objects when these were aligned with respect to the hands than when they were physically aligned. Similarly, in the visuo-haptic cross-modal mental rotation task the parity judgments were influenced by the orientation of the exploring hand with respect to the viewing direction. This effect was shown to be modulated also by an intervening temporal delay that supposedly counteracts the influence of the hand-centered reference frame. We suggest that the hand-centered reference frame is embedded in a hierarchical structure of reference frames where some of these emerge depending on the demands and the circumstances of the surrounding environment and the needs of an active perceiver

Allocentric and egocentric reference frames in the processing of three-dimensional haptic space

The main goal of our study is to gain insight into the reference frames involved in three-dimensional haptic spatial processing. Previous research has shown that two-dimensional haptic spatial processing is prone to large systematic deviations. A weighted average model that identifies the origin of the systematic error patterns in the biasing influence of an egocentric reference frame on the allocentric reference frame was proposed as an explanation of the results. The basis of the egocentric reference frame was linked either to the hand or to the body. In the present study participants had to construct a field of parallel bars that could be oriented in three dimensions. First, systematic error patterns were found also in this three-dimensional haptic parallelity task. Second, among the different models tested for their accuracy in explaining the error patterns, the Hand-centered weighted average model proved to most closely resemble the data. A participant-specific weighting factor determined the biasing influence of the hand-centered egocentric reference frame. A shift from the allocentric towards the egocentric frame of reference of approximately 20% was observed. These results support the hypothesis that haptic spatial processing is a product of the interplay of diverse, but synergistically operating frames of reference

Haptic mental rotation revisited: multiple reference frame dependence

The nature of reference frames involved in haptic spatial processing was addressed by means of a haptic mental rotation task. Participants assessed the parity of two objects located in various spatial locations by exploring them with different hand orientations. The resulting response times were fitted with a triangle wave function. Phase shifts were found to depend on the relation between the hands and the objects, and between the objects and the body. We rejected the possibility that a single reference frame drives spatial processing. Instead, we found evidence of multiple interacting reference frames with the hand-centered reference frame playing the dominant role. We propose that a weighted average of the allocentric, the hand-centered and the body-centered reference frames influences the haptic encoding of spatial information. In addition, we showed that previous results can be reinterpreted within the framework of multiple reference frames. This mechanism has proved to be ubiquitously present in haptic spatial processing