How Do Gestures Influence Thinking and Speaking? The Gesture-for-Conceptualization Hypothesis.

Peer reviewedPostprin

The spatiotemporal representation of dance and music gestures using topological gesture analysis (TGA)

SPATIOTEMPORAL GESTURES IN MUSIC AND DANCE HAVE been approached using both qualitative and quantitative research methods. Applying quantitative methods has offered new perspectives but imposed several constraints such as artificial metric systems, weak links with qualitative information, and incomplete accounts of variability. In this study, we tackle these problems using concepts from topology to analyze gestural relationships in space. The Topological Gesture Analysis (TGA) relies on the projection of musical cues onto gesture trajectories, which generates point clouds in a three-dimensional space. Point clouds can be interpreted as topologies equipped with musical qualities, which gives us an idea about the relationships between gesture, space, and music. Using this method, we investigate the relationships between musical meter, dance style, and expertise in two popular dances (samba and Charleston). The results show how musical meter is encoded in the dancer's space and how relevant information about styles and expertise can be revealed by means of simple topological relationships

Synthesis of variable dancing styles based on a compact spatiotemporal representation of dance

Dance as a complex expressive form of motion is able to convey emotion, meaning and social idiosyncrasies that opens channels for non-verbal communication, and promotes rich cross-modal interactions with music and the environment. As such, realistic dancing characters may incorporate crossmodal information and variability of the dance forms through compact representations that may describe the movement structure in terms of its spatial and temporal organization. In this paper, we propose a novel method for synthesizing beatsynchronous dancing motions based on a compact topological model of dance styles, previously captured with a motion capture system. The model was based on the Topological Gesture Analysis (TGA) which conveys a discrete three-dimensional point-cloud representation of the dance, by describing the spatiotemporal variability of its gestural trajectories into uniform spherical distributions, according to classes of the musical meter. The methodology for synthesizing the modeled dance traces back the topological representations, constrained with definable metrical and spatial parameters, into complete dance instances whose variability is controlled by stochastic processes that considers both TGA distributions and the kinematic constraints of the body morphology. In order to assess the relevance and flexibility of each parameter into feasibly reproducing the style of the captured dance, we correlated both captured and synthesized trajectories of samba dancing sequences in relation to the level of compression of the used model, and report on a subjective evaluation over a set of six tests. The achieved results validated our approach, suggesting that a periodic dancing style, and its musical synchrony, can be feasibly reproduced from a suitably parametrized discrete spatiotemporal representation of the gestural motion trajectories, with a notable degree of compression

Sketching sonic interactions by imitation-driven sound synthesis

Sketching is at the core of every design activity. In visual design, pencil and paper are the preferred tools to produce sketches for their simplicity and immediacy. Analogue tools for sonic sketching do not exist yet, although voice and gesture are embodied abilities commonly exploited to communicate sound concepts. The EU project SkAT-VG aims to support vocal sketching with computeraided technologies that can be easily accessed, understood and controlled through vocal and gestural imitations. This imitation-driven sound synthesis approach is meant to overcome the ephemerality and timbral limitations of human voice and gesture, allowing to produce more refined sonic sketches and to think about sound in a more designerly way. This paper presents two main outcomes of the project: The Sound Design Toolkit, a palette of basic sound synthesis models grounded on ecological perception and physical description of sound-producing phenomena, and SkAT-Studio, a visual framework based on sound design workflows organized in stages of input, analysis, mapping, synthesis, and output. The integration of these two software packages provides an environment in which sound designers can go from concepts, through exploration and mocking-up, to prototyping in sonic interaction design, taking advantage of all the possibilities of- fered by vocal and gestural imitations in every step of the process

The nature of gestures’ beneficial role in spatial problem solving

Peer reviewedPostprin

On topics today

This article surveys the state of so-called topic theory today. It charts its development through two generations of topic theorists. The first is constructed around three influential texts: Leonard Ratners seminal book that established the discipline in its own right, Classic music: expression, form and style (1980); Wye Allanbrooks. Rhythmic gesture in Mozart: Le nozze di Figaro and Don Giovanni (1983); and Kofi Agawus. Playing with signs: a semiotic interpretation of classical music (1991). The second comprises significant advances in topic theory essayed through two further pairs of texts: Robert Hattens Musical meaning in Beethoven: markedness, correlation, and interpretation (1994) and Interpreting musical gestures, topics, and tropes: Mozart, Beethoven, Schubert (2004); and Raymond Monelles Linguistics and semiotics in music (1992) and The sense of music: semiotic essays (2000). Topic Theory's role as the soft hermeneutic sub-field of music semiotics (relative to the harder, formalist practices of Nattiezs neutral level analysis) is portrayed here as navigating a number of treacherous polemical paths. These wend their way between referential style (expression) and structural syntax (form); historical reconstruction and hermeneutic construction; and heightened sensitivity to social meanings and imposed acts of creative interpretation. This existence of topic theory in a continuous dialogue between structural formalism and the semantics of expressive discourse is held responsible for its marginal position both to the dominant strains of contemporary postmodern musicology and to the dying embers of formalist analysis. The failure of topic theory to strike a fashionable text-context balance thus highlights why musicology continues to view semiotics with scepticism. Ratner presents his thesaurus of style labelssomewhat dubiouslyas the historically authentic ready-to-hand materials (types and styles) of eighteenth-century expressive musical rhetoric. But it is Agawus combination of this universe of topics with a Schenker-influenced beginning-middle-end paradigm that establishes the hallmark of first generation topic theory on which the first half of this paper focuses. Agawus delicate equation between extroversive and introversive semiosis is essayed as a pivotal turning point in topic theorys ability to transcend the mere passive ascription of rhetorical labels. Out of this equation, expressive meanings can ariseas much from the non-congruence, as the congruence, of signs and structure. Hatten's critique of Agawu for neglecting the full interpretative consequences of his signifieds is the springboard for his more hermeneutically replete brand of topic theory and the emergence of the second generation topic theorists. Hattens use of troping (a kind of musical metaphor), is one of many interpretative tools that are responsible for broadening the arena of topic theorysome of his others being: expressive genres, emergent meanings and markedness theory. These are deployed across a variety of musical parameters as Hattens attention increasingly turns to the prototypicality of topics in their euphoric and dysphoric states. Hattens interpretative work is shown to transcend historical reconstruction to comprise creative interpretation built on a much broader definition of expressive gestures, of which topics are only a constituent part. The article concludes with Monelles expos of the dubious historical underpinnings of Ratners topic theory foundations. This does not render this vibrant branch of semiotics redundant but, on the contrary, charts its future direction as one calling out for far deeper historical investigation and cultural criticism. Monelles enlightening forays into the more replete expressive meanings of such topics as the horse and pianto make this point abundantly clear. The future of topics today, if not musicology in general, is one of cultural criticism