The Psychophysics of Harmony Perception: Harmony is a Three-Tone Phenomenon

In line with musical “common sense” (but contrary to the century-old tradition of musical psychophysics), we show that harmony is an inherently three-tone phenomenon. Previous attempts at explaining the affective response to major/minor chords and resolved/unresolved chords on the basis of the summation of interval dissonance have been notably unsuccessful, but consideration of the relative size of the intervals contained in triads leads directly to solutions to these historical problems. At the heart of our model is Leonard Meyer’s idea from 1956 concerning “intervallic equidistance” – i.e., the perception of “tension” inherent to any three-tone combination that has two intervals of equivalent size (e.g., the augmented chord). By including the effects of the upper partials, a psychophysical explanation of the perceived sonority of the triads (major>minor>diminished>augmented) and the affective valence of major and minor chords is easily achieved. We conclude that the perceptual regularities of traditional diatonic harmony are neither due to the summation of interval effects nor simply arbitrary, learned cultural artifacts, but rather that harmony has a psychophysical basis dependent on three-tone combinations

Why Not Study Polytonal Psychophysics?

The relative consonance/dissonance of 2-tone intervals is well understood both experimentally and theoretically and provides a strong foundation for explaining why diatonic scales or their subsets are used in most musical cultures. Frequent textbook assertions notwithstanding, however, the consonance of intervals fails to account for the basic facts of harmony (3 or more tone combinations). We have recently shown (Cook & Fujisawa, 2006) how consideration of 3-tone psychophysics can explain the fundamental regularities of diatonic harmony. Distinct from the dissonance of 2-tone intervals, 3-tone combinations introduce an effect described by Leonard Meyer (1956) as harmonic “tension”: when a third tone is located midway between an upper and a lower tone, the chord takes on an unresolved, unstable, tense character – a psychoacoustical property inherent to the diminished and augmented chords. If the effects of the upper partials are included in a formal model that includes both 2-tone and 3-tone effects, the perceived sonority of the triads (major>minor> diminished>augmented) is easily explained

Evaluation of the affect of speech intonation using a model of the perception of interval dissonance and harmonic tension

Abstract We report the application of a psychophysical model of pitch perception to the analysis of speech intonation. The model was designed to reproduce the empirical findings on the perception of musical phenomena (the dissonance/consonance of intervals and the tension/sonority of chords), but does not depend on specific musical scales or tuning systems. Application to intonation allows us to calculate the total dissonance and tension among the pitches in the speech utterance. In an experiment using the 144 utterances of 18 male and female subjects, we found greater dissonance and harmonic tension in sentences with negative affect, in comparison with sentences with positive affect

The Triadic Roots of Human Cognition: “Mind” Is the Ability to go Beyond Dyadic Associations

Empirical evidence is reviewed indicating that the extraordinary aspects of the human mind are due to our species’ ability to go beyond simple “dyadic associations” and to process the relations among three items of information simultaneously. Classic explanations of the “triadic” nature of human skills have been advocated by various scholars in the context of the evolution of human cognition. Here I summarize the core processes as found in (i) the syntax of language, (ii) tool-usage, and (iii) joint attention. I then review the triadic foundations of two perceptual phenomena of great importance in human aesthetics: (iv) harmony perception and (v) pictorial depth perception. In all five subfields of human psychology, most previous work has emphasized the recursive, hierarchical complexity of such “higher cognition,” but a strongly reductionist approach indicates that the core mechanisms are triadic. It is concluded that the cognitive skills traditionally considered to be “uniquely” human require three-way associational processing that most non-Primate animal species find difficult or impossible, but all members of Homo sapiens – regardless of small cultural differences – find easy and inherently intriguing

Mapping Nonverbal Communication into Social Status: Automatic Recognition of Journalists and Non-journalists in Radio News

This work shows how features accounting for nonverbal speaking characteristics can be used to map people into predefined categories. In particular, the results of this paper show that the speakers participating in radio broadcast news can be classified into journalists and non-journalists with an accuracy higher than 80 percent. The results of the approach proposed for this task is compared with the effectiveness of 16 human assessors performing the same task. The assessors do not understand the language of the data and are thus forced to use mostly nonverbal features. The results of the comparison suggest that the assessors and the automatic system have a similar performance

The SEMAINE API : a component integration framework for a naturally interacting and emotionally competent embodied conversational agent

The present thesis addresses the topic area of Embodied Conversational Agents (ECAs) with capabilities for natural interaction with a human user and emotional competence with respect to the perception and generation of emotional expressivity. The focus is on the technological underpinnings that facilitate the implementation of a real-time system with these capabilities, built from re-usable components. The thesis comprises three main contributions. First, it describes a new component integration framework, the SEMAINE API, which makes it easy to build emotion-oriented systems from components which interact with one another using standard and pre-standard XML representations. Second, it presents a prepare-and-trigger system architecture which substantially speeds up the time to animation for system utterances that can be pre-planned. Third, it reports on the W3C Emotion Markup Language, an upcoming web standard for representing emotions in technological systems. We assess critical aspects of system performance, showing that the framework provides a good basis for implementing real-time interactive ECA systems, and illustrate by means of three examples that the SEMAINE API makes it is easy to build new emotion-oriented systems from new and existing components.Die vorliegende Dissertation behandelt das Thema der virtuellen Agenten mit Fähigkeiten zur natürlichen Benutzer-Interaktion sowie emotionaler Kompetenz bzgl. der Wahrnehmung und Generierung emotionalen Ausdrucks. Der Schwerpunkt der Arbeit liegt auf den technologischen Grundlagen für die Implementierung eines echtzeitfähigen Systems mit diesen Fähigkeiten, das aus wiederverwendbaren Komponenten erstellt werden kann. Die Arbeit umfasst drei Kernaspekte. Zum Einen beschreibt sie ein neues Framework zur Komponenten-Integration, die SEMAINE API: Diese erleichtert die Erstellung von Emotions-orientierten Systemen aus Komponenten, die untereinander mittels Standard- oder Prä-Standard-Repräsentationen kommunizieren. Zweitens wird eine Systemarchitektur vorgestellt, welche Vorbereitung und Auslösung von Systemverhalten entkoppelt und so zu einer substanziellen Beschleunigung der Generierungszeit führt, wenn Systemäußerungen im Voraus geplant werden können. Drittens beschreibt die Arbeit die W3C Emotion Markup Language, einen werdenden Web-Standard zur Repräsentation von Emotionen in technologischen Systemen. Es werden kritische Aspekte der Systemperformanz untersucht, wodurch gezeigt wird, dass das Framework eine gute Basis für die Implementierung echtzeitfähiger interaktiver Agentensysteme darstellt. Anhand von drei Beispielen wird illustriert, dass mit der SEMAINE API leicht neue Emotions-orientierte Systeme aus neuen und existierenden Komponenten erstellt werden können

Adaptive music: Automated music composition and distribution

Creativity, or the ability to produce new useful ideas, is commonly associated to the human being; but there are many other examples in nature where this phenomenon can be observed. Inspired by this fact, in engineering, and particularly in computational sciences, many different models have been developed to tackle a number of problems. Music, a form of art broadly present along the human history, is the main field addressed in this thesis, taking advantage of the kind of ideas that bring diversity and creativity to nature and computation. We present Melomics, an algorithmic composition method based on evolutionary search, with a genetic encoding of the solutions, which are interpreted in a complex developmental process that leads to music in the standard formats. This bioinspired compositional system has exhibited a high creative power and versatility to produce music of different type, which in many occasions has proven to be indistinguishable from the music made by human composers. The system also has enabled the emergence of a set of completely novel applications: from effective tools to help anyone to easily obtain the precise music they need, to radically new uses like adaptive music for therapy, amusement or many other purposes. It is clear to us that there is much research work yet to do in this field; and that countless and new unimaginable uses will derive from it

Brain-Inspired Computing

This open access book constitutes revised selected papers from the 4th International Workshop on Brain-Inspired Computing, BrainComp 2019, held in Cetraro, Italy, in July 2019. The 11 papers presented in this volume were carefully reviewed and selected for inclusion in this book. They deal with research on brain atlasing, multi-scale models and simulation, HPC and data infra-structures for neuroscience as well as artificial and natural neural architectures

Proceedings of the VIIth GSCP International Conference

The 7th International Conference of the Gruppo di Studi sulla Comunicazione Parlata, dedicated to the memory of Claire Blanche-Benveniste, chose as its main theme Speech and Corpora. The wide international origin of the 235 authors from 21 countries and 95 institutions led to papers on many different languages. The 89 papers of this volume reflect the themes of the conference: spoken corpora compilation and annotation, with the technological connected fields; the relation between prosody and pragmatics; speech pathologies; and different papers on phonetics, speech and linguistic analysis, pragmatics and sociolinguistics. Many papers are also dedicated to speech and second language studies. The online publication with FUP allows direct access to sound and video linked to papers (when downloaded)