Real-time detection of overlapping sound events with non-negative matrix factorization

International audienceIn this paper, we investigate the problem of real-time detection of overlapping sound events by employing non-negative matrix factorization techniques. We consider a setup where audio streams arrive in real-time to the system and are decomposed onto a dictionary of event templates learned off-line prior to the decomposition. An important drawback of existing approaches in this context is the lack of controls on the decomposition. We propose and compare two provably convergent algorithms that address this issue, by controlling respectively the sparsity of the decomposition and the trade-off of the decomposition between the different frequency components. Sparsity regularization is considered in the framework of convex quadratic programming, while frequency compromise is introduced by employing the beta-divergence as a cost function. The two algorithms are evaluated on the multi-source detection tasks of polyphonic music transcription, drum transcription and environmental sound recognition. The obtained results show how the proposed approaches can improve detection in such applications, while maintaining low computational costs that are suitable for real-time

Computational Stylistics in Poetry, Prose, and Drama

The contributions in this edited volume approach poetry, narrative, and drama from the perspective of Computational Stylistics. They exemplify methods of computational textual analysis and explore the possibility of computational generation of literary texts. The volume presents a range of computational and Natural Language Processing applications to literary studies, such as motif detection, network analysis, machine learning, and deep learning

Computational Stylistics in Poetry, Prose, and Drama

The contributions in this edited volume approach poetry, narrative, and drama from the perspective of Computational Stylistics. They exemplify methods of computational textual analysis and explore the possibility of computational generation of literary texts. The volume presents a range of computational and Natural Language Processing applications to literary studies, such as motif detection, network analysis, machine learning, and deep learning

A User-assisted Approach to Multiple Instrument Music Transcription

PhDThe task of automatic music transcription has been studied for several decades and is regarded as an enabling technology for a multitude of applications such as music retrieval and discovery, intelligent music processing and large-scale musicological analyses. It refers to the process of identifying the musical content of a performance and representing it in a symbolic format. Despite its long research history, fully automatic music transcription systems are still error prone and often fail when more complex polyphonic music is analysed. This gives rise to the question in what ways human knowledge can be incorporated in the transcription process. This thesis investigates ways to involve a human user in the transcription process. More specifically, it is investigated how user input can be employed to derive timbre models for the instruments in a music recording, which are employed to obtain instrument-specific (parts-based) transcriptions. A first investigation studies different types of user input in order to derive instrument models by means of a non-negative matrix factorisation framework. The transcription accuracy of the different models is evaluated and a method is proposed that refines the models by allowing each pitch of each instrument to be represented by multiple basis functions. A second study aims at limiting the amount of user input to make the method more applicable in practice. Different methods are considered to estimate missing non-negative basis functions when only a subset of basis functions can be extracted based on the user information. A method is proposed to track the pitches of individual instruments over time by means of a Viterbi framework in which the states at each time frame contain several candidate instrument-pitch combinations. A transition probability is employed that combines three different criteria: the frame-wise reconstruction error of each combination, a pitch continuity measure that favours similar pitches in consecutive frames, and an explicit activity model for each instrument. The method is shown to outperform other state-of-the-art multi-instrument tracking methods. Finally, the extraction of instrument models that include phase information is investigated as a step towards complex matrix decomposition. The phase relations between the partials of harmonic sounds are explored as a time-invariant property that can be employed to form complex-valued basis functions. The application of the model for a user-assisted transcription task is illustrated with a saxophone example.QMU

Measuring and understanding light in real life scenarios

Lighting design and modelling (the efficient and aesthetic placement of luminaires in a virtual or real scene) or industrial applications like luminaire planning and commissioning (the luminaire's installation and evaluation process along to the scene's geometry and structure) rely heavily on high realism and physically correct simulations. The current typical approaches are based only on CAD modeling simulations and offline rendering, with long processing times and therefore inflexible workflows. In this thesis we examine whether different camera-aided light modeling and numerical optimization approaches could be used to accurately understand, model and measure the light distribution in real life scenarios within real world environments. We show that factorization techniques could play a semantic role for light decomposition and light source identification, while we contribute a novel benchmark dataset and metrics for it. Thereafter we adapt a well known global illumination model (i.e. radiosity) and we extend it so that to overcome some of its basic limitations related to the assumption of point based only light sources or the adaption of only isotropic light perception sensors. We show that this extended radiosity numerical model can challenge the state-of-the-art in obtaining accurate dense spatial light measurements over time and in different scenarios. Finally we combine the latter model with human-centric sensing information and present how this could be beneficial for smart lighting applications related to quality lighting and power efficiency. Thus, with this work we contribute by setting the baselines for using an RGBD camera input as the only requirement to light modeling methods for light estimation in real life scenarios, and open a new applicability where the illumination modeling can be turned into an interactive process, allowing for real-time modifications and immediate feedback on the spatial illumination of a scene over time towards quality lighting and energy efficient solutions

A Spiral Workbook for Discrete Mathematics

This is a text that covers the standard topics in a sophomore-level course in discrete mathematics: logic, sets, proof techniques, basic number theory, functions, relations, and elementary combinatorics, with an emphasis on motivation. It explains and clarifies the unwritten conventions in mathematics, and guides the students through a detailed discussion on how a proof is revised from its draft to a final polished form. Hands-on exercises help students understand a concept soon after learning it. The text adopts a spiral approach: many topics are revisited multiple times, sometimes from a different perspective or at a higher level of complexity. The goal is to slowly develop students’ problem-solving and writing skills.https://knightscholar.geneseo.edu/oer-ost/1010/thumbnail.jp