Query-based Deep Improvisation

In this paper we explore techniques for generating new music using a Variational Autoencoder (VAE) neural network that was trained on a corpus of specific style. Instead of randomly sampling the latent states of the network to produce free improvisation, we generate new music by querying the network with musical input in a style different from the training corpus. This allows us to produce new musical output with longer-term structure that blends aspects of the query to the style of the network. In order to control the level of this blending we add a noisy channel between the VAE encoder and decoder using bit-allocation algorithm from communication rate-distortion theory. Our experiments provide new insight into relations between the representational and structural information of latent states and the query signal, suggesting their possible use for composition purposes

The Variable Markov Oracle: Algorithms for Human Gesture Applications

This article introduces the Variable Markov Oracle (VMO) data structure for multivariate time series indexing. VMO can identify repetitive fragments and find sequential similarities between observations. VMO can also be viewed as a combination of online clustering algorithms with variable-order Markov constraints. The authors use VMO for gesture query-by-content and gesture following. A probabilistic interpretation of the VMO query-matching algorithm is proposed to find an analogy to the inference problem in a hidden Markov model (HMM). This probabilistic interpretation extends VMO to be not only a data structure but also a model for time series. Query-by-content experiments were conducted on a gesture database that was recorded using a Kinect 3D camera, showing state-of-the-art performance. The query-by-content experiments' results are compared to previous works using HMM and dynamic time warping. Gesture following is described in the context of an interactive dance environment that aims to integrate human movements with computer-generated graphics to create an augmented reality performance

Rethinking Recurrent Latent Variable Model for Music Composition

We present a model for capturing musical features and creating novel sequences of music, called the Convolutional Variational Recurrent Neural Network. To generate sequential data, the model uses an encoder-decoder architecture with latent probabilistic connections to capture the hidden structure of music. Using the sequence-to-sequence model, our generative model can exploit samples from a prior distribution and generate a longer sequence of music. We compare the performance of our proposed model with other types of Neural Networks using the criteria of Information Rate that is implemented by Variable Markov Oracle, a method that allows statistical characterization of musical information dynamics and detection of motifs in a song. Our results suggest that the proposed model has a better statistical resemblance to the musical structure of the training data, which improves the creation of new sequences of music in the style of the originals.Comment: Published as a conference paper at IEEE MMSP 201

Adversarial Reprogramming of Text Classification Neural Networks

Adversarial Reprogramming has demonstrated success in utilizing pre-trained neural network classifiers for alternative classification tasks without modification to the original network. An adversary in such an attack scenario trains an additive contribution to the inputs to repurpose the neural network for the new classification task. While this reprogramming approach works for neural networks with a continuous input space such as that of images, it is not directly applicable to neural networks trained for tasks such as text classification, where the input space is discrete. Repurposing such classification networks would require the attacker to learn an adversarial program that maps inputs from one discrete space to the other. In this work, we introduce a context-based vocabulary remapping model to reprogram neural networks trained on a specific sequence classification task, for a new sequence classification task desired by the adversary. We propose training procedures for this adversarial program in both white-box and black-box settings. We demonstrate the application of our model by adversarially repurposing various text-classification models including LSTM, bi-directional LSTM and CNN for alternate classification tasks

On Ecological Fallacy and Assessment Errors Stemming From Misguided Variable Selection: Investigating the Effect of Data Aggregation on the Outcome of Epidemiological Study

In behavioral studies, ecological fallacy is a wrong assumption about an individual based on aggregate data for a group. In the present study, the validity of this assumption was tested using both individual estimates of exposure to air pollution and aggregate air pollution data estimated for 1,492 schoolchildren living in the in vicinity of a major coal-fired power station in the Hadera sub-district of Israel. In 1996 and 1999, the children underwent subsequent pulmonary function (PF) tests, and their parents completed a detailed questionnaire on their health status, and housing conditions. The association between children’s PF development and their long-term exposure to air pollution was then investigated in two phases. During the first phase, the average rates of PF change observed in small statistical areas in which the children reside were compared with average levels of air pollution detected in these areas. During the second phase of the analysis, an individual pollution estimate was calculated for each child covered by the survey, using a "spatial join" tool in ArcGIS. While the analysis of aggregate data showed no significant differences in the PF development among the schoolchildren surveyed, the comparison of individual pollution estimates with the results of PF tests detected a significant negative association between changes in PF results and the estimated level of air pollution. As argued, these differences are attributed to the fact that average exposure levels are likely to cause a misclassification bias of individual exposure, as further demonstrated in the study using pattern detection techniques of spatial analysis (local Moran's I and Gettis-Ord statistic). The implications of the results of the analysis for geographical and epidemiological studies are discussed, and recommendations for public health policy are formulated.