Learning Deep Belief Networks from Non-Stationary Streams

Deep learning has proven to be beneficial for complex tasks such as classifying images. However, this approach has been mostly applied to static datasets. The analysis of non-stationary (e.g., concept drift) streams of data involves specific issues connected with the temporal and changing nature of the data. In this paper, we propose a proof-of-concept method, called Adaptive Deep Belief Networks, of how deep learning can be generalized to learn online from changing streams of data. We do so by exploiting the generative properties of the model to incrementally re-train the Deep Belief Network whenever new data are collected. This approach eliminates the need to store past observations and, therefore, requires only constant memory consumption. Hence, our approach can be valuable for life-long learning from non-stationary data streams. © 2012 Springer-Verlag

Neurogenesis Deep Learning

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.Comment: 8 pages, 8 figures, Accepted to 2017 International Joint Conference on Neural Networks (IJCNN 2017

Deep Architectures on Drifting Concepts: A Simple Approach

Many real-world problems may vary over time. These non stationary problems have been widely studied in the literature, often called drifting concepts problems. Recently, deep architectures have drawn a growing attention, given that they can easily model functions that are hard to approximate with shallow ones and an effective way of training them have been discovered. In this work we adapt a deep architecture to problems that present concept drift. To this end we show a way of combining them with a widely known drifting concept technique, the Streaming Ensemble Algorithm. We evaluate the new method using appropriate drifting problems and compare its performance with a more traditional approach. The results obtained are promising and show that the proposed variation is effective at combining the expressive power of a deep architecture with the adaptability of SEA.Sociedad Argentina de Informática e Investigación Operativ

Deep Architectures on Drifting Concepts: A Simple Approach

Many real-world problems may vary over time. These non stationary problems have been widely studied in the literature, often called drifting concepts problems. Recently, deep architectures have drawn a growing attention, given that they can easily model functions that are hard to approximate with shallow ones and an effective way of training them have been discovered. In this work we adapt a deep architecture to problems that present concept drift. To this end we show a way of combining them with a widely known drifting concept technique, the Streaming Ensemble Algorithm. We evaluate the new method using appropriate drifting problems and compare its performance with a more traditional approach. The results obtained are promising and show that the proposed variation is effective at combining the expressive power of a deep architecture with the adaptability of SEA.Sociedad Argentina de Informática e Investigación Operativ

Deep Architectures on Drifting Concepts: A Simple Approach

Many real-world problems may vary over time. These non stationary problems have been widely studied in the literature, often called drifting concepts problems. Recently, deep architectures have drawn a growing attention, given that they can easily model functions that are hard to approximate with shallow ones and an effective way of training them have been discovered. In this work we adapt a deep architecture to problems that present concept drift. To this end we show a way of combining them with a widely known drifting concept technique, the Streaming Ensemble Algorithm. We evaluate the new method using appropriate drifting problems and compare its performance with a more traditional approach. The results obtained are promising and show that the proposed variation is effective at combining the expressive power of a deep architecture with the adaptability of SEA.Sociedad Argentina de Informática e Investigación Operativ

Comparing deep belief networks with support vector machines for classifying gene expression data from complex disorders

Genomics data provide great opportunities for translational research and the clinical practice, for example, for predicting disease stages. However, the classification of such data is a challenging task due to their high dimensionality, noise, and heterogeneity. In recent years, deep learning classifiers generated much interest, but due to their complexity, so far, little is known about the utility of this method for genomics. In this paper, we address this problem by studying a computational diagnostics task by classification of breast cancer and inflammatory bowel disease patients based on high-dimensional gene expression data. We provide a comprehensive analysis of the classification performance of deep belief networks (DBNs) in dependence on its multiple model parameters and in comparison with support vector machines (SVMs). Furthermore, we investigate combined classifiers that integrate DBNs with SVMs. Such a classifier utilizes a DBN as representation learner forming the input for a SVM. Overall, our results provide guidelines for the complex usage of DBN for classifying gene expression data from complex diseases

Optimizing Clinical Assessments in Parkinson's Disease Through the Use of Wearable Sensors and Data Driven Modeling

The emergence of motion sensors as a tool that provides objective motor performance data on individuals afflicted with Parkinson's disease offers an opportunity to expand the horizon of clinical care for this neurodegenerative condition. Subjective clinical scales and patient based motor diaries have limited clinometric properties and produce a glimpse rather than continuous real time perspective into motor disability. Furthermore, the expansion of machine learn algorithms is yielding novel classification and probabilistic clinical models that stand to change existing treatment paradigms, refine the application of advance therapeutics, and may facilitate the development and testing of disease modifying agents for this disease. We review the use of inertial sensors and machine learning algorithms in Parkinson's disease

Machine Learning in Nuclear Physics

Advances in machine learning methods provide tools that have broad applicability in scientific research. These techniques are being applied across the diversity of nuclear physics research topics, leading to advances that will facilitate scientific discoveries and societal applications. This Review gives a snapshot of nuclear physics research which has been transformed by machine learning techniques.Comment: Comments are welcom

A self-learning framework for validation of runtime adaptation in service-oriented systems

Ensuring that service-oriented systems can adapt quickly and effectively to changes in service quality, business needs and their runtime environment is an increasingly important research problem. However, while considerable research has focused on developing runtime adaptation frameworks for service-oriented systems, there has been little work on assessing how effective the adaptations are. Effective adaptation ensures the system remains relevant in a changing environment. One way to address the problem is through validation. Validation allows us to assess how well a recommended adaptation addresses the concerns for which the system is reconfigured and provides us with insights into the nature of problems for which different adaptations are suited. However, the dynamic nature of runtime adaptation and the changeable contexts in which service-oriented systems operate make it difficult to specify appropriate validation mechanisms in advance. This thesis describes a novel consumer-centred approach that uses machine learning to continuously validate and refine runtime adaptation in service-oriented systems, through model-based clustering and deep learning. To evaluate the efficacy of the approach a medium sized health care case study was devised and implemented. The results obtained show that self-validation significantly improves the dynamic adaptation process by autonomously addressing changing user requirements at runtime. Further work in this area can improve the framework by integrating other learning algorithms as well as testing the framework on a larger case study