What drives sound symbolism? Different acoustic cues underlie sound-size and sound-shape mappings

Sound symbolism refers to the non-arbitrary mappings that exist between phonetic properties of speech sounds and their meaning. Despite there being an extensive literature on the topic, the acoustic features and psychological mechanisms that give rise to sound symbolism are not, as yet, altogether clear. The present study was designed to investigate whether different sets of acoustic cues predict size and shape symbolism, respectively. In two experiments, participants judged whether a given consonant-vowel speech sound was large or small, round or angular, using a size or shape scale. Visual size judgments were predicted by vowel formant F1 in combination with F2, and by vowel duration. Visual shape judgments were, however, predicted by formants F2 and F3. Size and shape symbolism were thus not induced by a common mechanism, but rather were distinctly affected by acoustic properties of speech sounds. These findings portray sound symbolism as a process that is not based merely on broad categorical contrasts, such as round/unround and front/back vowels. Rather, individuals seem to base their sound-symbolic judgments on specific sets of acoustic cues, extracted from speech sounds, which vary across judgment dimensions

Grounding Word Learning in Space

Humans and objects, and thus social interactions about objects, exist within space. Words direct listeners' attention to specific regions of space. Thus, a strong correspondence exists between where one looks, one's bodily orientation, and what one sees. This leads to further correspondence with what one remembers. Here, we present data suggesting that children use associations between space and objects and space and words to link words and objects—space binds labels to their referents. We tested this claim in four experiments, showing that the spatial consistency of where objects are presented affects children's word learning. Next, we demonstrate that a process model that grounds word learning in the known neural dynamics of spatial attention, spatial memory, and associative learning can capture the suite of results reported here. This model also predicts that space is special, a prediction supported in a fifth experiment that shows children do not use color as a cue to bind words and objects. In a final experiment, we ask whether spatial consistency affects word learning in naturalistic word learning contexts. Children of parents who spontaneously keep objects in a consistent spatial location during naming interactions learn words more effectively. Together, the model and data show that space is a powerful tool that can effectively ground word learning in social contexts

Integrating Mechanisms of Visual Guidance in Naturalistic Language Production

Situated language production requires the integration of visual attention and lin-guistic processing. Previous work has not conclusively disentangled the role of perceptual scene information and structural sentence information in guiding visual attention. In this paper, we present an eye-tracking study that demonstrates that three types of guidance, perceptual, conceptual, and structural, interact to control visual attention. In a cued language production experiment, we manipulate percep-tual (scene clutter) and conceptual guidance (cue animacy), and measure structural guidance (syntactic complexity of the utterance). Analysis of the time course of lan-guage production, before and during speech, reveals that all three forms of guidance affect the complexity of visual responses, quantified in terms of the entropy of atten-tional landscapes and the turbulence of scan patterns, especially during speech. We find that perceptual and conceptual guidance mediate the distribution of attention in the scene, whereas structural guidance closely relates to scan-pattern complexity. Furthermore, the eye-voice span of the cued object and its perceptual competitor are similar; its latency mediated by both perceptual and structural guidance. These results rule out a strict interpretation of structural guidance as the single dominant form of visual guidance in situated language production. Rather, the phase of the task and the associated demands of cross-modal cognitive processing determine the mechanisms that guide attention

The interaction of visual and linguistic saliency during syntactic ambiguity resolution

Psycholinguistic research using the visual world paradigm has shown that the pro-cessing of sentences is constrained by the visual context in which they occur. Re-cently, there has been growing interest on the interactions observed when both lan-guage and vision provide relevant information during sentence processing. In three visual world experiments on syntactic ambiguity resolution, we investigate how vi-sual and linguistic information influence the interpretation of ambiguous sentences. We hypothesize that (1) visual and linguistic information both constrain which in-terpretation is pursued by the sentence processor, and (2) the two types of informa-tion act upon the interpretation of the sentence at different points during processing. In Experiment 1, we show that visual saliency is utilized to anticipate the upcoming arguments of a verb. In Experiment 2, we operationalize linguistic saliency using intonational breaks and demonstrate that these give prominence to linguistic refer-ents. These results confirm prediction (1). In Experiment 3, we manipulate visual and linguistic saliency together and find that both types of information are used, but at different points in the sentence, to incrementally update its current interpre-tation. This finding is consistent with prediction (2). Overall, our results suggest an adaptive processing architecture in which different types of information are used when they become available, optimizing different aspects of situated language pro-cessing

Wine and music (III): so what if music influences the taste of the wine?

A growing body of evidence, both anecdotal and scientifically rigorous, now points to the fact that what people taste when evaluating a wine, not to mention how much they enjoy the experience, can be influenced by the specifics of any music that happens to be playing at the same time. The question that we wish to address here is ‘So what?’ Why should anyone care that music (or, for that matter, specially composed soundscapes) exert(s) a crossmodal influence over the wine-tasting experience? ‘Why not just drink great wine and forget about the music?’ a sceptic might ask. Here, we outline a number of the uses that such research findings have been put to in the marketplace, in experiential events, in artistic performances, and in terms of furthering our theoretical understanding of those factors that influence the tasting experience. We also highlight how the latest in technology (think sensory apps and hyperdirectional loudspeakers, not to mention digitally augmented glassware) augurs well for those wanting to deliver the most stimulating, the most memorable, and certainly the most multisensory of tasting experiences in the years to come. Demonstrations of sound’s influence on wine perception will most likely be applicable to a variety of other drinks and foods too. Ultimately, the argument is forwarded that there are many reasons, both theoretical and applied, as to why we should all care about the fact that what we listen to can change the sensory-discriminative, the descriptive, and the hedonic attributes of what we taste

Music to make your mouth water? Assessing the potential influence of sour music on salivation

People robustly associate various sound attributes with specific smells/tastes, and soundtracks that are associated with specific tastes can influence people’s evaluation of the taste of food and drink. However, it is currently unknown whether such soundtracks directly impact the eating experience via physiological changes (an embodiment account), or whether they act at a higher cognitive level, or both. The present research assessed a version of the embodiment account, where a soundtrack associated with sourness is hypothesized to induce a physiological response in the listener by increasing salivary flow. Salivation was measured while participants were exposed to three different experimental conditions – a sour soundtrack, a muted lemon video showing a man eating a lemon, and a silent baseline condition. The results revealed that salivation during the lemon video condition was significantly greater than in the sour soundtrack and baseline conditions. However, contrary to our hypothesis, there was no significant difference between salivation levels in the sour soundtrack compared to the baseline condition. These results are discussed in terms of potential mechanisms underlying the auditory modulation of taste perception/evaluation

A large sample study on the influence of the multisensory environment on the wine drinking experience

Background We report on what may well be the world’s largest multisensory tasting experiment. Over a period of 4 days in May 2014, almost 3,000 people sampled a glass of red wine in a room in which the colour of the lighting and/or the music was changed repeatedly. The participants rated the wine, presented in a black tasting glass, on taste, intensity and liking scales while standing in each of four different environments over a period of 7 to 8 minutes. During the first 2 days (Experiment 1), the participants rated the wine while exposed to white lighting, red lighting, green lighting with music designed to enhance sourness and finally under red lighting paired with music associated with sweetness. During the latter 2 days of the event (Experiment 2), the same wine was rated under white lighting, green lighting, red lighting with sweet music and finally green lighting with sour music. Results In Experiment 1, the wine was perceived as fresher and less intense under green lighting and sour music, as compared to any of the other three environments. On average, the participants liked the wine most under red lighting while listening to sweet music. A similar pattern of results was reported in Experiment 2. Conclusions These results demonstrate that the environment can exert a significant influence on the perception of wine (at least in a random sample of social drinkers). We outline a number of possible explanations for how the sensory properties of the environment might influence the perception of wine. Finally, we consider some of the implications of these results for the wine drinking experience

A large sample study on the influence of the multisensory environment on the wine drinking experience

Background We report on what may well be the world’s largest multisensory tasting experiment. Over a period of 4 days in May 2014, almost 3,000 people sampled a glass of red wine in a room in which the colour of the lighting and/or the music was changed repeatedly. The participants rated the wine, presented in a black tasting glass, on taste, intensity and liking scales while standing in each of four different environments over a period of 7 to 8 minutes. During the first 2 days (Experiment 1), the participants rated the wine while exposed to white lighting, red lighting, green lighting with music designed to enhance sourness and finally under red lighting paired with music associated with sweetness. During the latter 2 days of the event (Experiment 2), the same wine was rated under white lighting, green lighting, red lighting with sweet music and finally green lighting with sour music. Results In Experiment 1, the wine was perceived as fresher and less intense under green lighting and sour music, as compared to any of the other three environments. On average, the participants liked the wine most under red lighting while listening to sweet music. A similar pattern of results was reported in Experiment 2. Conclusions These results demonstrate that the environment can exert a significant influence on the perception of wine (at least in a random sample of social drinkers). We outline a number of possible explanations for how the sensory properties of the environment might influence the perception of wine. Finally, we consider some of the implications of these results for the wine drinking experience