Multimodal music information processing and retrieval: survey and future challenges

Towards improving the performance in various music information processing tasks, recent studies exploit different modalities able to capture diverse aspects of music. Such modalities include audio recordings, symbolic music scores, mid-level representations, motion, and gestural data, video recordings, editorial or cultural tags, lyrics and album cover arts. This paper critically reviews the various approaches adopted in Music Information Processing and Retrieval and highlights how multimodal algorithms can help Music Computing applications. First, we categorize the related literature based on the application they address. Subsequently, we analyze existing information fusion approaches, and we conclude with the set of challenges that Music Information Retrieval and Sound and Music Computing research communities should focus in the next years

Regulation of synaptic connectivity: levels of fasciclin II influence synaptic growth in the Drosophila CNS

Much of our understanding of synaptogenesis comes from studies that deal with the development of the neuromuscular junction (NMJ). Although well studied, it is not clear how far the NMJ represents an adequate model for the formation of synapses within the CNS. Here we investigate the role of Fasciclin II (Fas II) in the development of synapses between identified motor neurons and cholinergic interneurons in the CNS of Drosophila. Fas II is a neural cell adhesion molecule homolog that is involved in both target selection and synaptic plasticity at the NMJ in Drosophila. In this study, we show that levels of Fas II are critical determinants of synapse formation and growth in the CNS. The initial establishment of synaptic contacts between these identified neurons is seemingly independent of Fas II. The subsequent proliferation of these synaptic connections that occurs postembryonically is, in contrast, significantly retarded by the absence of Fas II. Although the initial formation of synaptic connectivity between these neurons is seemingly independent of Fas II, we show that their formation is, nevertheless, significantly affected by manipulations that alter the relative balance of Fas II in the presynaptic and postsynaptic neurons. Increasing expression of Fas II in either the presynaptic or postsynaptic neurons, during embryogenesis, is sufficient to disrupt the normal level of synaptic connectivity that occurs between these neurons. This effect of Fas II is isoform specific and, moreover, phenocopies the disruption to synaptic connectivity observed previously after tetanus toxin light chain-dependent blockade of evoked synaptic vesicle release in these neurons

Breaking the vicious circle of rhythm–tempo definitions

In music literature, rhythm is defined relative to a certain tempo, and tempo is defined relative to a certain rhythm. This vicious circle implies that any sequence of time durations can be regarded as either (a) a sequence of these durations at a constant tempo or (b) a sequence of equal durations at a varying tempo or (c) a sequence of unequal durations at a varying tempo in numerous ways. Most listeners, however, perceive rhythm and tempo in the same way, which we explain as the result of a close interaction of the grouping and simplicity laws of Gestalt psychology. Operationally, the complexity of a data representation is defined as the amount of memory that is required for the algorithm of the data generation. Each rhythm-tempo representation includes rhythmic patterns and the tempo curve that ‘generates’ their augmentations and diminutions in time. The complexity of such a representation is split between the rhythmic patterns and the tempo curve, and the representation with the least total complexity is selected. Rhythm and tempo are thus complementary structures that mutually adapt according to the criterion of simplicity, which leads to an optimal rhythm-tempo perception. In addition to general provisions, we consider a few rules for grouping time events into patterns, a directed search for optimal representations of time events, and the influence of the musical context on the perception of rhythm and tempo

A Transformational Approach for Musical Variation

This article is associated with a broad research addressing musical variation, whose main objective is the systematization of the analysis through the elaboration of an original analytical model. A new version of this model proposes a formal approach based on principles of the transformational theory. The present article is focused in the notion of variation isolated from a contextual framework (that is, out of temporal perspectives), setting the basis for further explorations. .Some original concepts, like derivative work, derivative space, attributes, among others, provide means for measurement of similarity relations between referential and derived musical ideas, as well as graphic representation for these relationships.This article is associated with a broad research addressing musical variation, whose main objective is the systematization of the analysis through the elaboration of an original analytical model. A new version of this model proposes a formal approach based on principles of the transformational theory. The present article is focused on the notion of variation isolated from a contextual framework (that is, out of temporal perspectives), setting the basis for further exploration. Some original concepts, like derivative work, derivative space, attributes, among others, provide means for measurement of similarity relations between referential and derived musical ideas, as well as graphic representation for these relationships

Brain Networks for Integrative Rhythm Formation

Performance of externally paced rhythmic movements requires brain and behavioral integration of sensory stimuli with motor commands. The underlying brain mechanisms to elaborate beat-synchronized rhythm and polyrhythms that musicians readily perform may differ. Given known roles in perceiving time and repetitive movements, we hypothesized that basal ganglia and cerebellar structures would have greater activation for polyrhythms than for on-the-beat rhythms.Using functional MRI methods, we investigated brain networks for performing rhythmic movements paced by auditory cues. Musically trained participants performed rhythmic movements at 2 and 3 Hz either at a 1:1 on-the-beat or with a 3:2 or a 2:3 stimulus-movement structure. Due to their prior musical experience, participants performed the 3:2 or 2:3 rhythmic movements automatically. Both the isorhythmic 1:1 and the polyrhythmic 3:2 or 2:3 movements yielded the expected activation in contralateral primary motor cortex and related motor areas and ipsilateral cerebellum. Direct comparison of functional MRI signals obtained during 3:2 or 2:3 and on-the-beat rhythms indicated activation differences bilaterally in the supplementary motor area, ipsilaterally in the supramarginal gyrus and caudate-putamen and contralaterally in the cerebellum.The activated brain areas suggest the existence of an interconnected brain network specific for complex sensory-motor rhythmic integration that might have specificity for elaboration of musical abilities

Revealing Originality of Song Works: an Analysis to the Copyright Law

The topic of this paper is to describe the defining criteria of originality of song works. The aspect of originality is important to make such work be protected by Copyright Law. In this research, the criteria to define originality are based on certain doctrines and/or theories of originality that may vary case by case. The use of such doctrines and/or theories are necessary, since the stipulations regarding originality in the Indonesian Copyright Act has not been considered suffice. With regard to the song works, the criteria of originality may be different from other works. Therefore, a comprehensive research on the characteristics of song as a work is also important. This research is a qualitative research with prescriptive design. The research depicts the use of certain doctrines and/or theories as supplementary provisions to the Copyright Law in defining the originality of songs, which have specific characteristics resulted from their author's creations and intellectuals

A Comprehensive Trainable Error Model for Sung Music Queries

We propose a model for errors in sung queries, a variant of the hidden Markov model (HMM). This is a solution to the problem of identifying the degree of similarity between a (typically error-laden) sung query and a potential target in a database of musical works, an important problem in the field of music information retrieval. Similarity metrics are a critical component of query-by-humming (QBH) applications which search audio and multimedia databases for strong matches to oral queries. Our model comprehensively expresses the types of error or variation between target and query: cumulative and non-cumulative local errors, transposition, tempo and tempo changes, insertions, deletions and modulation. The model is not only expressive, but automatically trainable, or able to learn and generalize from query examples. We present results of simulations, designed to assess the discriminatory potential of the model, and tests with real sung queries, to demonstrate relevance to real-world applications

Higher-level relationships in Dhomont's Novars

A sense of structure can often be traced to the perception of recurrent events in a work. Returning sound materials, or those that remind of earlier instances, can become significant features that provide a temporal perspective, creating musical landmarks against which other material might be considered, compared and appraised. The significance of these landmarks may change as the work unfolds and as other families of recurrent sounds become established, potentially resulting in webs of correspondence that influence overall impressions of form. Investigating recurrence within a work involves assessing aspects of similarity or difference among the various constituent sound identities, considering how they function when they recur, and reflecting on why they are significant. Many kinds of music feature recurrent identities founded on melodic, harmonic and rhythmic patterns and formations, which may be subject to different sorts of repetition and variation. However, the kinds of sound material and the possibilities of sound transformation available to acousmatic composers are sufficiently different and varied that concepts of recurrence within this context require specific attention in order to understand more fully what ‘a recurrence’ can be, how recurrent phenomena operate over different timescales, and how they contribute to impressions of structure and form. With these ideas in mind, a listening strategy for Francis Dhomont’s Novars will be presented focusing on the recurrence of sound material at higher, or more global, levels of structure. The analytical strategy adopted here is primarily based on listening, and so all observations are those of a focused listener rather than reflecting the intentions of the composer

Massimiliano Balduzzi: Research in Physical Training for Performers

This essay begins the process of contextualizing and analyzing Massimiliano Balduzzi’s solo physical training practice by introducing six newly created video documents. It locates Balduzzi’s work in a wider historical and artistic context – touching upon the work of Konstantin Stanislavski, Jerzy Grotowski, and Eugenio Barba, as well as acrobatics, martial arts, and Balinese dance – while arguing that the documented physical training constitutes an original research contribution to the field of embodied technique. The essay has three main purposes: First, to give verbal articulation to some important aspects of Balduzzi’s practice, as he begins to teach more widely in New York City and beyond. Second, to test and develop a theoretical framework that conceives of embodied technique as a field of knowledge in which rigorously framed research can and does give rise to new knowledge in the form of new technique. Third, to explore the epistemological status of multimedia documentation through a focused case study. Each of these goals has the potential to expand and clarify current discussions of actor and performer training, movement analysis and documentation, and practice-as-research