Synthesis from Probabilistic Components

Synthesis is the automatic construction of a system from its specification. In classical synthesis algorithms, it is always assumed that the system is "constructed from scratch" rather than composed from reusable components. This, of course, rarely happens in real life, where almost every non-trivial commercial software system relies heavily on using libraries of reusable components. Furthermore, other contexts, such as web-service orchestration, can be modeled as synthesis of a system from a library of components. Recently, Lustig and Vardi introduced dataflow and control-flow synthesis from libraries of reusable components. They proved that dataflow synthesis is undecidable, while control-flow synthesis is decidable. In this work, we consider the problem of control-flow synthesis from libraries of probabilistic components. We show that this more general problem is also decidable

Using conditional restricted Boltzmann machines to generate timbral music composition systems

Machine-learning models have been successfully applied to musical composition in a variety of forms, including audio classification, recognition, and synthesis. The capability of algorithms to learn complex musical elements allows composers to more deeply investigate the development of their aesthetic. Coupled with the history of interdisciplinary solutions found in computer music and system aesthetics, this capability has led to an exploration of the integration of machine learning and music composition. Composition systems that take advantage of this integration have the opportunity to be connected with algorithms in theory, application, and art. In my systems, conditional restricted Boltzmann machines (CRBM) synthesize musical timbre by learning autoregressive connections between the current output, an abstracted non-linear hidden feature layer, and past out- puts. This provides a creative space where composers can synthesize audio spectra in collaboration with machines, defining novel creative systems that explore compositional material in an abstract, non-linear paradigm. By implementing CRBMs in timbral-synthesis composition systems, I provide concrete support that such an integration advances art through the exploration of machine learning. I demonstrate this in a variety of audio synthesis experiments validating the capabilities of two algorithmic structures to synthesize and control timbre: a single layer conditional restricted Boltzmann machine (CRBM) and a single layer factored conditional restricted Boltzmann machine (FCRBM). I start by accurately synthesizing specific instrumental timbres and different musical pitches, demonstrating the aural capabilities of directly using the algorithms. I then build from these experiments, creating a set of compositional utilities that provide the composer with a rich pallet to provoke aesthetic introspection. These compositional utilities are then implemented in two music composition systems that synthesize and control timbre in application, where the algorithms themselves are designed and manipulated as a means to realize artwork. Through the creation of music composition systems that are able to accurately synthesize and control musical timbre, I demonstrate these models have the capability of provoking the aesthetic introspection of composers. The resulting systems show the power and potential of integrating music composition and machine learning, endorsing an interdisciplinary approach to the development of art and technology

A Mathematical, Graphical and Visual Approach to Granular Synthesis Composition

We show a method for Granular Synthesis Composition based on a mathematical modeling for the musical gesture. Each gesture is drawn as a curve generated from a particular mathematical model (or function) and coded as a MATLAB script. The gestures can be deterministic through defining mathematical time functions, hand free drawn, or even randomly generated. This parametric information of gestures is interpreted through OSC messages by a granular synthesizer (Granular Streamer). The musical composition is then realized with the models (scripts) written in MATLAB and exported to a graphical score (Granular Score). The method is amenable to allow statistical analysis of the granular sound streams and the final music composition. We also offer a way to create granular streams based on correlated pair of grains parameters

Attentive Aspect Modeling for Review-aware Recommendation

In recent years, many studies extract aspects from user reviews and integrate them with ratings for improving the recommendation performance. The common aspects mentioned in a user's reviews and a product's reviews indicate indirect connections between the user and product. However, these aspect-based methods suffer from two problems. First, the common aspects are usually very sparse, which is caused by the sparsity of user-product interactions and the diversity of individual users' vocabularies. Second, a user's interests on aspects could be different with respect to different products, which are usually assumed to be static in existing methods. In this paper, we propose an Attentive Aspect-based Recommendation Model (AARM) to tackle these challenges. For the first problem, to enrich the aspect connections between user and product, besides common aspects, AARM also models the interactions between synonymous and similar aspects. For the second problem, a neural attention network which simultaneously considers user, product and aspect information is constructed to capture a user's attention towards aspects when examining different products. Extensive quantitative and qualitative experiments show that AARM can effectively alleviate the two aforementioned problems and significantly outperforms several state-of-the-art recommendation methods on top-N recommendation task.Comment: Camera-ready manuscript for TOI

Proceedings of the 4th International Workshop on Reading Music Systems

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 4th International Workshop on Reading Music Systems, held online on Nov. 18th 2022.Comment: Proceedings edited by Jorge Calvo-Zaragoza, Alexander Pacha and Elona Shatr