The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation

With recent breakthroughs in artificial neural networks, deep generative models have become one of the leading techniques for computational creativity. Despite very promising progress on image and short sequence generation, symbolic music generation remains a challenging problem since the structure of compositions are usually complicated. In this study, we attempt to solve the melody generation problem constrained by the given chord progression. This music meta-creation problem can also be incorporated into a plan recognition system with user inputs and predictive structural outputs. In particular, we explore the effect of explicit architectural encoding of musical structure via comparing two sequential generative models: LSTM (a type of RNN) and WaveNet (dilated temporal-CNN). As far as we know, this is the first study of applying WaveNet to symbolic music generation, as well as the first systematic comparison between temporal-CNN and RNN for music generation. We conduct a survey for evaluation in our generations and implemented Variable Markov Oracle in music pattern discovery. Experimental results show that to encode structure more explicitly using a stack of dilated convolution layers improved the performance significantly, and a global encoding of underlying chord progression into the generation procedure gains even more.Comment: 8 pages, 13 figure

Rank and factor loadings estimation in time series tensor factor model by pre-averaging

The idiosyncratic components of a tensor time series factor model can exhibit serial correlations, (e.g., finance or economic data), ruling out many state-of-the-art methods that assume white/independent idiosyncratic components. While the traditional higher order orthogonal iteration (HOOI) is proved to be convergent to a set of factor loading matrices, the closeness of them to the true underlying factor loading matrices are in general not established, or only under i.i.d. Gaussian noises. Under the presence of serial and cross-correlations in the idiosyncratic components and time series variables with only bounded fourth-order moments, for tensor time series data with tensor order two or above, we propose a pre-averaging procedure that can be considered a random projection method. The estimated directions corresponding to the strongest factors are then used for projecting the data for a potentially improved re-estimation of the factor loading spaces themselves, with theoretical guarantees and rate of convergence spelt out when not all factors are pervasive. We also propose a new rank estimation method, which utilizes correlation information from the projected data. Extensive simulations are performed and compared to other state-of-the-art or traditional alternatives. A set of tensor-valued NYC taxi data is also analyzed

Reading Scene Text in Deep Convolutional Sequences

We develop a Deep-Text Recurrent Network (DTRN) that regards scene text reading as a sequence labelling problem. We leverage recent advances of deep convolutional neural networks to generate an ordered high-level sequence from a whole word image, avoiding the difficult character segmentation problem. Then a deep recurrent model, building on long short-term memory (LSTM), is developed to robustly recognize the generated CNN sequences, departing from most existing approaches recognising each character independently. Our model has a number of appealing properties in comparison to existing scene text recognition methods: (i) It can recognise highly ambiguous words by leveraging meaningful context information, allowing it to work reliably without either pre- or post-processing; (ii) the deep CNN feature is robust to various image distortions; (iii) it retains the explicit order information in word image, which is essential to discriminate word strings; (iv) the model does not depend on pre-defined dictionary, and it can process unknown words and arbitrary strings. Codes for the DTRN will be available.Comment: To appear in the 13th AAAI Conference on Artificial Intelligence (AAAI-16), 201

Factor modelling for tensor time series

High dimensional tensor time series data is increasingly prevalent across various fields. In the analysis of such data, factor modelling plays a crucial role as a dimension reduction tool. While traditional factor models primarily handle vector time series, the exploration of matrix or tensor factor models under various assumptions is still in its early stages and has attracted increasing interest in recent years. In this thesis, we develop a tensor factor model under the presence of both serial and cross-correlations in the idiosyncratic components, assuming only bounded fourth order moments for the time series variables. Moreover, we incorporate a spectrum of different factor strengths into the model, in contrast to the prevalent assumption in many literature that considers only pervasive factors. The inclusion of serial dependence noise and weak factors makes our model more compatible with real data, especially in economics and finance. With the relaxed assumptions in our model, we propose a pre-averaging procedure to initially estimate the factor loading spaces, which achieves signal accumulation through the random projection of tensor fibres. Furthermore, we develop an iterative projection algorithm to improve the re-estimation of factor loadings by projecting the data onto the strongest estimated factor directions. To estimate the number of factors, we introduce a new core tensor rank estimation method through correlation analysis on the projected data. Theoretical guarantees are provided for all estimators, and extensive simulations, as well as analyses of real datasets, are conducted to compare our methods with other state-of-the-art or traditional alternatives. Finally, we present a new method for estimating factor strengths with empirical results provided and introduce a novel matrix convergence criterion for specific covariance matrix estimators, offering valuable insights into directions for future research

STATE-OF-ART Algorithms for Injectivity and Bounded Surjectivity of One-dimensional Cellular Automata

Surjectivity and injectivity are the most fundamental problems in cellular automata (CA). We simplify and modify Amoroso's algorithm into optimum and make it compatible with fixed, periodic and reflective boundaries. A new algorithm (injectivity tree algorithm) for injectivity is also proposed. After our theoretic analysis and experiments, our algorithm for injectivity can save much space and 90\% or even more time compared with Amoroso's algorithm for injectivity so that it can support the decision of CA with larger neighborhood sizes. At last, we prove that the reversibility with the periodic boundary and global injectivity of one-dimensional CA is equivalent

On the Role of Entropy-based Loss for Learning Causal Structures with Continuous Optimization

Causal discovery from observational data is an important but challenging task in many scientific fields. Recently, NOTEARS [Zheng et al., 2018] formulates the causal structure learning problem as a continuous optimization problem using least-square loss with an acyclicity constraint. Though the least-square loss function is well justified under the standard Gaussian noise assumption, it is limited if the assumption does not hold. In this work, we theoretically show that the violation of the Gaussian noise assumption will hinder the causal direction identification, making the causal orientation fully determined by the causal strength as well as the variances of noises in the linear case and the noises of strong non-Gaussianity in the nonlinear case. Consequently, we propose a more general entropy-based loss that is theoretically consistent with the likelihood score under any noise distribution. We run extensive empirical evaluations on both synthetic data and real-world data to validate the effectiveness of the proposed method and show that our method achieves the best in Structure Hamming Distance, False Discovery Rate, and True Positive Rate matrices

Use of definite clause grammars

Call number: LD2668 .R4 CMSC 1987 C53Master of ScienceComputing and Information Science

Generalization bound for estimating causal effects from observational network data

Estimating causal effects from observational network data is a significant but challenging problem. Existing works in causal inference for observational network data lack an analysis of the generalization bound, which can theoretically provide support for alleviating the complex confounding bias and practically guide the design of learning objectives in a principled manner. To fill this gap, we derive a generalization bound for causal effect estimation in network scenarios by exploiting 1) the reweighting schema based on joint propensity score and 2) the representation learning schema based on Integral Probability Metric (IPM). We provide two perspectives on the generalization bound in terms of reweighting and representation learning, respectively. Motivated by the analysis of the bound, we propose a weighting regression method based on the joint propensity score augmented with representation learning. Extensive experimental studies on two real-world networks with semi-synthetic data demonstrate the effectiveness of our algorithm