Embodiment, sound and visualization : a multimodal perspective in music education

Recently, many studies have emphasized the role of body movements in processing, sharing and giving meaning to music. At the same time, neuroscience studies, suggest that different parts of the brain are integrated and activated by the same stimuli: sounds, for example, can be perceived by touch and can evoke imagery, energy, fluency and periodicity. This interaction of auditory, visual and motor senses can be found in the verbal descriptions of music and among children during their spontaneous games. The question to be asked is, if a more multisensory and embodied approach could redefine some of our assumptions regarding musical education. Recent research on embodiment and multimodal perception in instrumental teaching could suggest new directions in musical education. Can we consider the integration between the activities of body movement, listening, metaphor visualization, and singing, as more effective than a disembodied and fragmented approach for the process of musical understanding

On Picturing a Candle: The Prehistory of Imagery Science

The past 25 years have seen a rapid growth of knowledge about brain mechanisms involved in visual mental imagery. These advances have largely been made independently of the long history of philosophical – and even psychological – reckoning with imagery and its parent concept ‘imagination’. We suggest that the view from these empirical findings can be widened by an appreciation of imagination’s intellectual history, and we seek to show how that history both created the conditions for – and presents challenges to – the scientific endeavor. We focus on the neuroscientific literature’s most commonly used task – imagining a concrete object – and, after sketching what is known of the neurobiological mechanisms involved, we examine the same basic act of imagining from the perspective of several key positions in the history of philosophy and psychology. We present positions that, firstly, contextualize and inform the neuroscientific account, and secondly, pose conceptual and methodological challenges to the scientific analysis of imagery. We conclude by reflecting on the intellectual history of visualization in the light of contemporary science, and the extent to which such science may resolve long-standing theoretical debates

Multimodal Grounding for Language Processing

This survey discusses how recent developments in multimodal processing facilitate conceptual grounding of language. We categorize the information flow in multimodal processing with respect to cognitive models of human information processing and analyze different methods for combining multimodal representations. Based on this methodological inventory, we discuss the benefit of multimodal grounding for a variety of language processing tasks and the challenges that arise. We particularly focus on multimodal grounding of verbs which play a crucial role for the compositional power of language.Comment: The paper has been published in the Proceedings of the 27 Conference of Computational Linguistics. Please refer to this version for citations: https://www.aclweb.org/anthology/papers/C/C18/C18-1197

Concerns about losing face moderate the effect of visual perspective on health-related intentions and behaviors

Visualizing oneself engaging in future actions has been shown to increase the likelihood of actually engaging in the visualized action. In three studies, we examined the effect of perspective taken to visualize a future action (first-person vs. third-person) as a function of the degree to which individuals worry about others’ evaluation of themselves (face) and whether the visualized behavior is public or private. Across all studies, the effect of visual perspective was present only for participants with high level of face. In this group, the third-person visualization induced stronger intentions to engage in the behavior when the imagined behavior was public (Study 1), whereas the first-person visualization induced stronger intentions and greater likelihood to engage in that behavior when it was private (Study 2). The influence of the first-person perspective on flossing behavior was eliminated when people with high levels of face were encouraged to consider inter-personal consequences of the action (Study 3). Results are discussed in the light of recent theorizing on the cognitive consequences of taking a third-person versus a first-person perspective in visual imagery

Shifting the Focus: The Role of Presence in Reconceptualising the Design Process

In this paper the relationship between presence and imaging is examined with the view to establish how our understanding of imaging, and subsequently the design process, may be reconceptualised to give greater focus to its experiential potential. First, the paper outlines the research project contributing to the discussion. Then, it provides brief overviews of research on both imaging and presence in the process highlighting the narrow conceptions of imaging (and the recognition of the need for further research) compared to the more holistic and experiential understandings of presence. The paper concludes with an argument and proposed study for exploring the role of digital technology and presence in extending the potential of imaging and its role in the design process. As indicated in the DRS Conference Theme, this paper focuses “…on what people experience and the systems and actions that create those experiences.” Interface designers, information architects and interactive media artists understand the powerful influence of experience in design. ‘Experience design’ is a community of practice driven by individuals within digital based disciplines where the belief is that understanding people is essential to any successful design in any medium and that “…experience is the personal connection with the moment and… every aspect of living is an experience, whether we are the creators or simply chance participants” (Shedroff, 2001, p. 5). Keywords: Design, Design Process, Presence, Imaging, Grounded Theory</p

Is Vivaldi smooth and takete? Non-verbal sensory scales for describing music qualities

Studies on the perception of music qualities (such as induced or perceived emotions, performance styles, or timbre nuances) make a large use of verbal descriptors. Although many authors noted that particular music qualities can hardly be described by means of verbal labels, few studies have tried alternatives. This paper aims at exploring the use of non-verbal sensory scales, in order to represent different perceived qualities in Western classical music. Musically trained and untrained listeners were required to listen to six musical excerpts in major key and to evaluate them from a sensorial and semantic point of view (Experiment 1). The same design (Experiment 2) was conducted using musically trained and untrained listeners who were required to listen to six musical excerpts in minor key. The overall findings indicate that subjects\u2019 ratings on non-verbal sensory scales are consistent throughout and the results support the hypothesis that sensory scales can convey some specific sensations that cannot be described verbally, offering interesting insights to deepen our knowledge on the relationship between music and other sensorial experiences. Such research can foster interesting applications in the field of music information retrieval and timbre spaces explorations together with experiments applied to different musical cultures and contexts

Perceptual content and the content of mental imagery

The aim of this paper is to argue that the phenomenal similarity between perceiving and visualizing can be explained by the similarity between the structure of the content of these two different mental states. And this puts important constraints on how we should think about perceptual content and the content of mental imager

Macaques preferentially attend to visual patterns with higher fractal dimension contours.

Animals' sensory systems evolved to efficiently process information from their environmental niches. Niches often include irregular shapes and rough textures (e.g., jagged terrain, canopy outlines) that must be navigated to find food, escape predators, and master other fitness-related challenges. For most primates, vision is the dominant sensory modality and thus, primates have evolved systems for processing complicated visual stimuli. One way to quantify information present in visual stimuli in natural scenes is evaluating their fractal dimension. We hypothesized that sensitivity to complicated geometric forms, indexed by fractal dimension, is an evolutionarily conserved capacity, and tested this capacity in rhesus macaques (Macaca mulatta). Monkeys viewed paired black and white images of simulated self-similar contours that systematically varied in fractal dimension while their attention to the stimuli was measured using noninvasive infrared eye tracking. They fixated more frequently on, dwelled for longer durations on, and had attentional biases towards images that contain boundary contours with higher fractal dimensions. This indicates that, like humans, they discriminate between visual stimuli on the basis of fractal dimension and may prefer viewing informationally rich visual stimuli. Our findings suggest that sensitivity to fractal dimension may be a wider ability of the vertebrate vision system