Monkeys and Humans Share a Common Computation for Face/Voice Integration

Speech production involves the movement of the mouth and other regions of the face resulting in visual motion cues. These visual cues enhance intelligibility and detection of auditory speech. As such, face-to-face speech is fundamentally a multisensory phenomenon. If speech is fundamentally multisensory, it should be reflected in the evolution of vocal communication: similar behavioral effects should be observed in other primates. Old World monkeys share with humans vocal production biomechanics and communicate face-to-face with vocalizations. It is unknown, however, if they, too, combine faces and voices to enhance their perception of vocalizations. We show that they do: monkeys combine faces and voices in noisy environments to enhance their detection of vocalizations. Their behavior parallels that of humans performing an identical task. We explored what common computational mechanism(s) could explain the pattern of results we observed across species. Standard explanations or models such as the principle of inverse effectiveness and a “race” model failed to account for their behavior patterns. Conversely, a “superposition model”, positing the linear summation of activity patterns in response to visual and auditory components of vocalizations, served as a straightforward but powerful explanatory mechanism for the observed behaviors in both species. As such, it represents a putative homologous mechanism for integrating faces and voices across primates

The science behind virtual reality displays

Virtual reality (VR) is becoming an increasingly important way to investigate sensory processing. The converse is also true: in order to build good VR technologies, one needs an intimate understanding of how our brain processes sensory information. One of the key advantages of studying perception with VR is that it allows an experimenter to probe perceptual processing in a more naturalistic way than has been possible previously. In VR, one is able to actively explore and interact with the environment, just as one would do in real life. In this article, we review the history of VR displays, including the philosophical origins of VR, before discussing some key challenges involved in generating good VR and how a sense of presence in a virtual environment can be measured. We discuss the importance of multisensory VR and evaluate the experimental tension that exists between artifice and realism when investigating sensory processing

Impact of the spatial congruence of redundant targets on within-modal and cross-modal integration

Although the topic of sensory integration has raised increasing interest, the differing behavioral outcome of combining unisensory versus multisensory inputs has surprisingly only been scarcely investigated. In the present experiment, observers were required to respond as fast as possible to (1) lateralized visual or tactile targets presented alone, (2) double stimulation within the same modality or (3) double stimulation across modalities. Each combination was either delivered within the same hemispace (spatially aligned) or in different hemispaces (spatially misaligned). Results show that the redundancy gains (RG) obtained from the cross-modal conditions were far greater than those obtained from combinations of two visual or two tactile targets. Consistently, we observed that the reaction time distributions of cross-modal targets, but not those of within-modal targets, surpass the predicted reaction time distribution based on the summed probability distributions of each constituent stimulus presented alone. Moreover, we found that the spatial alignment of the targets did not influence the RG obtained in cross-modal conditions, whereas within-modal stimuli produced a greater RG when the targets where delivered in separate hemispaces. These results suggest that within-modal and cross-modal integration are not only distinguishable by the amount of facilitation they produce, but also by the spatial configuration under which this facilitation occurs. Our study strongly supports the notion that estimates of the same event that are more independent produce enhanced integrative gains