A biomimetic basis for auditory processing and the perception of natural sounds

Biomimicry is a powerful science that aims to take advantage of nature's remarkable ability to devise innovative solutions to challenging problems. In the setting of hearing, mimicking how humans hear is the foremost strategy in designing effective artificial hearing approaches. In this work, we explore the mathematical foundations for the exchange of design inspiration and features between biological hearing systems, artificial sound-filtering devices, and signal processing algorithms. Our starting point is a concise asymptotic analysis of subwavelength acoustic metamaterials. We are able to fine tune this structure to mimic the biomechanical properties of the cochlea, at the same scale. We then turn our attention to developing a biomimetic signal processing algorithm. We use the response of the cochlea-like structure as an initial filtering layer and then add additional biomimetic processing stages, designed to mimic the human auditory system's ability to recognise the global properties of natural sounds

Reenacting sensorimotor features of drawing movements from friction sounds

International audienceEven though we generally don't pay attention to the friction sounds produced when we are writing or drawing, these sounds are recordable, and can even evoke the underlying gesture. In this paper, auditory perception of such sounds, and the internal representations they evoke when we listen to them, is considered from the sensorimotor learning point of view. The use of synthesis processes of friction sounds makes it possible to investigate the perceptual influence of each gestures parameter separately. Here, the influence of the velocity profile on the mental representation of the gesture induced by a friction sound was investigated through 3 experiments. The results reveal the perceptual relevance of this parameter, and particularly a specific morphology corresponding to biological movements, the so-called 1/3-power law. The experiments are discussed according to the sensorimotor theory and the invariant taxonomy of the ecological approach

Timbre from Sound Synthesis and High-level Control Perspectives

International audienceExploring the many surprising facets of timbre through sound manipulations has been a common practice among composers and instrument makers of all times. The digital era radically changed the approach to sounds thanks to the unlimited possibilities offered by computers that made it possible to investigate sounds without physical constraints. In this chapter we describe investigations on timbre based on the analysis by synthesis approach that consists in using digital synthesis algorithms to reproduce sounds and further modify the parameters of the algorithms to investigate their perceptual relevance. In the first part of the chapter timbre is investigated in a musical context. An examination of the sound quality of different wood species for xylophone making is first presented. Then the influence of instrumental control on timbre is described in the case of clarinet and cello performances. In the second part of the chapter, we mainly focus on the identification of sound morphologies, so called invariant sound structures responsible for the evocations induced by environmental sounds by relating basic signal descriptors and timbre descriptors to evocations in the case of car door noises, motor noises, solid objects, and their interactions

Relating Objective and Subjective Performance Measures for AAM-based Visual Speech Synthesizers

We compare two approaches for synthesizing visual speech using Active Appearance Models (AAMs): one that utilizes acoustic features as input, and one that utilizes a phonetic transcription as input. Both synthesizers are trained using the same data and the performance is measured using both objective and subjective testing. We investigate the impact of likely sources of error in the synthesized visual speech by introducing typical errors into real visual speech sequences and subjectively measuring the perceived degradation. When only a small region (e.g. a single syllable) of ground-truth visual speech is incorrect we find that the subjective score for the entire sequence is subjectively lower than sequences generated by our synthesizers. This observation motivates further consideration of an often ignored issue, which is to what extent are subjective measures correlated with objective measures of performance? Significantly, we find that the most commonly used objective measures of performance are not necessarily the best indicator of viewer perception of quality. We empirically evaluate alternatives and show that the cost of a dynamic time warp of synthesized visual speech parameters to the respective ground-truth parameters is a better indicator of subjective quality

Basic gestures as spatiotemporal reference frames for repetitive dance/music patterns in samba and charleston

THE GOAL OF THE PRESENT STUDY IS TO GAIN BETTER insight into how dancers establish, through dancing, a spatiotemporal reference frame in synchrony with musical cues. With the aim of achieving this, repetitive dance patterns of samba and Charleston were recorded using a three-dimensional motion capture system. Geometric patterns then were extracted from each joint of the dancer's body. The method uses a body-centered reference frame and decomposes the movement into non-orthogonal periodicities that match periods of the musical meter. Musical cues (such as meter and loudness) as well as action-based cues (such as velocity) can be projected onto the patterns, thus providing spatiotemporal reference frames, or 'basic gestures,' for action-perception couplings. Conceptually speaking, the spatiotemporal reference frames control minimum effort points in action-perception couplings. They reside as memory patterns in the mental and/or motor domains, ready to be dynamically transformed in dance movements. The present study raises a number of hypotheses related to spatial cognition that may serve as guiding principles for future dance/music studies

Reflexive Monism

Reflexive monism is, in essence, an ancient view of how consciousness relates to the material world that has, in recent decades, been resurrected in modern form. In this paper I discuss how some of its basic features differ from both dualism and variants of physicalist and functionalist reductionism, focusing on those aspects of the theory that challenge deeply rooted presuppositions in current Western thought. I pay particular attention to the ontological status and seeming “out-thereness” of the phenomenal world and to how the “phenomenal world” relates to the “physical world”, the “world itself”, and processing in the brain. In order to place the theory within the context of current thought and debate, I address questions that have been raised about reflexive monism in recent commentaries and also evaluate competing accounts of the same issues offered by “transparency theory” and by “biological naturalism”. I argue that, of the competing views on offer, reflexive monism most closely follows the contours of ordinary experience, the findings of science, and common sense