Deep Learning Techniques for Music Generation -- A Survey

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.Comment: 209 pages. This paper is a simplified version of the book: J.-P. Briot, G. Hadjeres and F.-D. Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 201

Gated recurrent neural networks discover attention

Recent architectural developments have enabled recurrent neural networks (RNNs) to reach and even surpass the performance of Transformers on certain sequence modeling tasks. These modern RNNs feature a prominent design pattern: linear recurrent layers interconnected by feedforward paths with multiplicative gating. Here, we show how RNNs equipped with these two design elements can exactly implement (linear) self-attention, the main building block of Transformers. By reverse-engineering a set of trained RNNs, we find that gradient descent in practice discovers our construction. In particular, we examine RNNs trained to solve simple in-context learning tasks on which Transformers are known to excel and find that gradient descent instills in our RNNs the same attention-based in-context learning algorithm used by Transformers. Our findings highlight the importance of multiplicative interactions in neural networks and suggest that certain RNNs might be unexpectedly implementing attention under the hood

Deep neural networks for video classification in ecology

Analyzing large volumes of video data is a challenging and time-consuming task. Automating this process would very valuable, especially in ecological research where massive amounts of video can be used to unlock new avenues of ecological research into the behaviour of animals in their environments. Deep Neural Networks, particularly Deep Convolutional Neural Networks, are a powerful class of models for computer vision. When combined with Recurrent Neural Networks, Deep Convolutional models can be applied to video for frame level video classification. This research studies two datasets: penguins and seals. The purpose of the research is to compare the performance of image-only CNNs, which treat each frame of a video independently, against a combined CNN-RNN approach; and to assess whether incorporating the motion information in the temporal aspect of video improves the accuracy of classifications in these two datasets. Video and image-only models offer similar out-of-sample performance on the simpler seals dataset but the video model led to moderate performance improvements on the more complex penguin action recognition dataset

Steering a Historical Disease Forecasting Model Under a Pandemic: Case of Flu and COVID-19

Forecasting influenza in a timely manner aids health organizations and policymakers in adequate preparation and decision making. However, effective influenza forecasting still remains a challenge despite increasing research interest. It is even more challenging amidst the COVID pandemic, when the influenza-like illness (ILI) counts are affected by various factors such as symptomatic similarities with COVID-19 and shift in healthcare seeking patterns of the general population. Under the current pandemic, historical influenza models carry valuable expertise about the disease dynamics but face difficulties adapting. Therefore, we propose CALI-Net, a neural transfer learning architecture which allows us to 'steer' a historical disease forecasting model to new scenarios where flu and COVID co-exist. Our framework enables this adaptation by automatically learning when it should emphasize learning from COVID-related signals and when it should learn from the historical model. Thus, we exploit representations learned from historical ILI data as well as the limited COVID-related signals. Our experiments demonstrate that our approach is successful in adapting a historical forecasting model to the current pandemic. In addition, we show that success in our primary goal, adaptation, does not sacrifice overall performance as compared with state-of-the-art influenza forecasting approaches.Comment: Appears in AAAI-2

Applied Machine Learning for Classification of Musculoskeletal Inference using Neural Networks and Component Analysis

Artificial Intelligence (AI) is acquiring more recognition than ever by researchers and machine learning practitioners. AI has found significance in many applications like biomedical research for cancer diagnosis using image analysis, pharmaceutical research, and, diagnosis and prognosis of diseases based on knowledge about patients\u27 previous conditions. Due to the increased computational power of modern computers implementing AI, there has been an increase in the feasibility of performing more complex research. Within the field of orthopedic biomechanics, this research considers complex time-series dataset of the sit-to-stand motion of 48 Total Hip Arthroplasty (THA) patients that was collected by the Human Dynamics Laboratory at the University of Denver. The research focuses on predicting the motion quality of the THA patients by analyzing the loads acting on muscles and joints during one motion cycle. We have classified the motion quality into two classes: Fair and Poor , based on muscle forces, and have predicted the motion quality using joint angles. We address different types of Machine Learning techniques: Artificial Neural Networks (LSTM - long short-term memory, CNN - convolutional neural network, and merged CNN-LSTM) and data science approach (principal component analysis and parallel factor analysis), that utilize remodeled datasets: heatmaps and 3-dimensional vectors. These techniques have been demonstrated efficient for the classification and prediction of the motion quality. The research proposes time-based optimization by predicting the motion quality at an initial stage of musculoskeletal model simulation, thereby, saving time and efforts required to perform multiple model simulations to generate a complete musculoskeletal modeling dataset. The research has provided efficient techniques for modeling neural networks and predicting post-operative musculoskeletal inference. We observed the accuracy of 83.33% for the prediction of the motion quality under the merged LSTM and CNN network, and autoencoder followed by feedforward neural network. The research work not only helps in realizing AI as an important tool for biomedical research but also introduces various techniques that can be utilized and incorporated by engineers and AI practitioners while working on a multi-variate time-series wide shaped data set with high variance

The hardware implementation of an artificial neural network using stochastic pulse rate encoding principles

In this thesis the development of a hardware artificial neuron device and artificial neural network using stochastic pulse rate encoding principles is considered. After a review of neural network architectures and algorithmic approaches suitable for hardware implementation, a critical review of hardware techniques which have been considered in analogue and digital systems is presented. New results are presented demonstrating the potential of two learning schemes which adapt by the use of a single reinforcement signal. The techniques for computation using stochastic pulse rate encoding are presented and extended with new novel circuits relevant to the hardware implementation of an artificial neural network. The generation of random numbers is the key to the encoding of data into the stochastic pulse rate domain. The formation of random numbers and multiple random bit sequences from a single PRBS generator have been investigated. Two techniques, Simulated Annealing and Genetic Algorithms, have been applied successfully to the problem of optimising the configuration of a PRBS random number generator for the formation of multiple random bit sequences and hence random numbers. A complete hardware design for an artificial neuron using stochastic pulse rate encoded signals has been described, designed, simulated, fabricated and tested before configuration of the device into a network to perform simple test problems. The implementation has shown that the processing elements of the artificial neuron are small and simple, but that there can be a significant overhead for the encoding of information into the stochastic pulse rate domain. The stochastic artificial neuron has the capability of on-line weight adaption. The implementation of reinforcement schemes using the stochastic neuron as a basic element are discussed

Training Spiking Neural Networks Using Lessons From Deep Learning

The brain is the perfect place to look for inspiration to develop more efficient neural networks. The inner workings of our synapses and neurons provide a glimpse at what the future of deep learning might look like. This paper serves as a tutorial and perspective showing how to apply the lessons learnt from several decades of research in deep learning, gradient descent, backpropagation and neuroscience to biologically plausible spiking neural neural networks. We also explore the delicate interplay between encoding data as spikes and the learning process; the challenges and solutions of applying gradient-based learning to spiking neural networks; the subtle link between temporal backpropagation and spike timing dependent plasticity, and how deep learning might move towards biologically plausible online learning. Some ideas are well accepted and commonly used amongst the neuromorphic engineering community, while others are presented or justified for the first time here. A series of companion interactive tutorials complementary to this paper using our Python package, snnTorch, are also made available: https://snntorch.readthedocs.io/en/latest/tutorials/index.htm

Combining Network Modeling and Experimental Approaches to Predict Drug Combination Responses

Cancer is a lethal disease and complex at multiple levels of cell biology. Despite many advances in treatments, many patients do not respond to therapy. This is owing to the complexity of cancer-genetic variability due to mutations, the multi-variate biochemical networks within which drug targets reside and existence and plasticity of multiple cell states. It is generally understood that a combination of drugs is a way to address the multi-faceted drivers of cancer and drug resistance. However, the sheer number of testable combinations and challenges in matching patients to appropriate combination treatments are major issues. Here, we first present a general method of network inference which can be applied to infer biological networks. We apply this method to infer different kinds of networks in biological levels where cancer complexity resides-a biochemical network, gene expression and cell state transitions. Next, we focus our attention on glioblastoma and with pharmacological and biological considerations, obtain a ranked list of important drug targets in glioblastoms. We perform drug dose response experiments for 22 blood brain barrier penetrant drugs against 3 glioblastoma cell lines. These methods and experimental results inform a construction of a temporal cell state model to predict and experimentally validate combination treatments for certain drugs. We improve an experimental method to perform high throughput western blots and apply the method to discover biochemical interactions among some important proteins involved in temporal cell state transitions. Lastly, we illustrate a method to investigate potential resistance mechanisms in genome scale proteomic data. We hope that methods and results presented here can be adapted and improved upon to help in the discovery of biochemical interactions, capturing cell state transitions and ultimately help predict effective combination therapies for cancer