A survey on Bayesian nonparametric learning

© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. Bayesian (machine) learning has been playing a significant role in machine learning for a long time due to its particular ability to embrace uncertainty, encode prior knowledge, and endow interpretability. On the back of Bayesian learning's great success, Bayesian nonparametric learning (BNL) has emerged as a force for further advances in this field due to its greater modelling flexibility and representation power. Instead of playing with the fixed-dimensional probabilistic distributions of Bayesian learning, BNL creates a new “game” with infinite-dimensional stochastic processes. BNL has long been recognised as a research subject in statistics, and, to date, several state-of-the-art pilot studies have demonstrated that BNL has a great deal of potential to solve real-world machine-learning tasks. However, despite these promising results, BNL has not created a huge wave in the machine-learning community. Esotericism may account for this. The books and surveys on BNL written by statisticians are overcomplicated and filled with tedious theories and proofs. Each is certainly meaningful but may scare away new researchers, especially those with computer science backgrounds. Hence, the aim of this article is to provide a plain-spoken, yet comprehensive, theoretical survey of BNL in terms that researchers in the machine-learning community can understand. It is hoped this survey will serve as a starting point for understanding and exploiting the benefits of BNL in our current scholarly endeavours. To achieve this goal, we have collated the extant studies in this field and aligned them with the steps of a standard BNL procedure-from selecting the appropriate stochastic processes through manipulation to executing the model inference algorithms. At each step, past efforts have been thoroughly summarised and discussed. In addition, we have reviewed the common methods for implementing BNL in various machine-learning tasks along with its diverse applications in the real world as examples to motivate future studies

Assessment of Visual Literacy – Contributions of Eye Tracking

Visual Literacy (VL) is defined as a set of competencies to understand and express oneself through visual imagery. An expansive model, the Common European Framework of Reference for Visual Literacy (CEFR-VL) (Wagner & Schönau, 2016), comprises 16 sub-competencies, including abilities such as analyzing, judging, experimenting with or aesthetically experiencing images. To empirically assess VL sub-competencies different visual tasks were presented to VL experts and novices. Problem-solving behavior and cognitive strategies involved in visual logical reasoning (Paper 1), Visual Search (Paper 2), and judgments of visual abstraction (Paper 3) were investigated. Eye tracking in combination with innovative statistical methods were used to uncover latent variables during task performance and to assess the possible effects of differences in expertise level. Furthermore, the relationship between students' self-reported visual abilities and their performance on VL assessment tasks is systematically explored. Results show how effects of perceptual skills of VL experts are less pronounced and more nuanced than implied by VL models. The comprehension of visual logical models does not seem to depend much on VL, as experts and novices did not differ in their solution strategies and eye movement indicators (Paper 1). In contrast, the visual search task on artworks revealed how experts were able to detect target regions with higher efficiency than novices revealed by higher precision of fixations on target regions. Furthermore, latent image features were detected by experts with more certainty (Paper 2). The assessment of perceived level of visual abstraction revealed how, contrary to our expectations, experts did not outperform novices but despite that were able to detect nuanced level of abstraction compared to student groups. Distribution of fixations indicate how attention is directed towards more ambiguous images (Paper 3). Students can be classified based on different levels of visual logical comprehension (Paper 1), on self-reported visual skills, and the time spent on the tasks (Paper 2, Paper 3). Self-reported visual art abilities of students (e.g., imagination) influences the visual search and the judgment of visual abstraction. Taken together the results show how VL skills are not determined solely by the number of correct responses, but rather by how visual tasks are solved and deconstructed; for example, experts are able to focus on less salient image regions during visual search and demonstrate a more nuanced interpretation of visual abstraction. Low-level perceptual abilities of experts and novices differ marginally, which is consistent with research on art expertise. Assessment of VL remains challenging, but new empirical methods are proposed to uncover the underlying components of VL

Music and time: tempomorphism: nested temporalities in perceived experience of music.

This thesis represents the results of a theoretical and practical investigation of acoustic and electro-acoustic elements of Western music at the start of the twentyfirst century, with specific attention to soundscapes. A commentary on the development of soundscapes is drawn from a multidisciplinary overview of concepts of time, followed by an examination of concepts of time in music. As a response to Jonathan Kramer's concept of `vertical' music (a characteristic aesthetic of which is an absence of conventional harmonic teleology), particular attention is paid to those theories of multiple nested temporalities which have been referred to by Kramer in support of non-teleological musical structures. The survey suggests that new musical concepts, such as vertical music, have emerged from sensibilities resulting from the musical and associated styles of minimalism, and represent an ontological development of aesthetics characteristic of the twentieth century. An original contention of the debate is that innovations in the practice of music as the result of technological developments have led to the possibility of defining a methodology of process in addition to auditive strategies, resulting in a duality defined as 'tempomorphic'. Further observations are supplied, using findings derived from original creative practical research, to define tempomorphic performance, which complete the contribution to knowledge offered by the investigation. Tempomorphism, therefore, is defined as a duality of process and audition: as auditive tool, tempomorphic analysis provides a listening strategy suited to harmonically static music; as a procedural tool, it affords a methodology based primarily on duration

Composing Deep Learning and Bayesian Nonparametric Methods

Recent progress in Bayesian methods largely focus on non-conjugate models featured with extensive use of black-box functions: continuous functions implemented with neural networks. Using deep neural networks, Bayesian models can reasonably fit big data while at the same time capturing model uncertainty. This thesis targets at a more challenging problem: how do we model general random objects, including discrete ones, using random functions? Our conclusion is: many (discrete) random objects are in nature a composition of Poisson processes and random functions}. Thus, all discreteness is handled through the Poisson process while random functions captures the rest complexities of the object. Thus the title: composing deep learning and Bayesian nonparametric methods. This conclusion is not a conjecture. In spacial cases such as latent feature models , we can prove this claim by working on infinite dimensional spaces, and that is how Bayesian nonparametric kicks in. Moreover, we will assume some regularity assumptions on random objects such as exchangeability. Then the representations will show up magically using representation theorems. We will see this two times throughout this thesis. One may ask: when a random object is too simple, such as a non-negative random vector in the case of latent feature models, how can we exploit exchangeability? The answer is to aggregate infinite random objects and map them altogether onto an infinite dimensional space. And then assume exchangeability on the infinite dimensional space. We demonstrate two examples of latent feature models by (1) concatenating them as an infinite sequence (Section 2,3) and (2) stacking them as a 2d array (Section 4). Besides, we will see that Bayesian nonparametric methods are useful to model discrete patterns in time series data. We will showcase two examples: (1) using variance Gamma processes to model change points (Section 5), and (2) using Chinese restaurant processes to model speech with switching speakers (Section 6). We also aware that the inference problem can be non-trivial in popular Bayesian nonparametric models. In Section 7, we find a novel solution of online inference for the popular HDP-HMM model

A Survey on Explainable Anomaly Detection

In the past two decades, most research on anomaly detection has focused on improving the accuracy of the detection, while largely ignoring the explainability of the corresponding methods and thus leaving the explanation of outcomes to practitioners. As anomaly detection algorithms are increasingly used in safety-critical domains, providing explanations for the high-stakes decisions made in those domains has become an ethical and regulatory requirement. Therefore, this work provides a comprehensive and structured survey on state-of-the-art explainable anomaly detection techniques. We propose a taxonomy based on the main aspects that characterize each explainable anomaly detection technique, aiming to help practitioners and researchers find the explainable anomaly detection method that best suits their needs.Comment: Paper accepted by the ACM Transactions on Knowledge Discovery from Data (TKDD) for publication (preprint version

Interconnect technologies for very large spiking neural networks

In the scope of this thesis, a neural event communication architecture has been developed for use in an accelerated neuromorphic computing system and with a packet-based high performance interconnection network. Existing neuromorphic computing systems mostly use highly customised interconnection networks, directly routing single spike events to their destination. In contrast, the approach of this thesis uses a general purpose packet-based interconnection network and accumulates multiple spike events at the source node into larger network packets destined to common destinations. This is required to optimise the payload efficiency, given relatively large packet headers as compared to the size of neural spike events. Theoretical considerations are made about the efficiency of different event aggregation strategies. Thereby, important factors are the number of occurring event network-destinations and their relative frequency, as well as the number of available accumulation buffers. Based on the concept of Markov Chains, an analytical method is developed and used to evaluate these aggregation strategies. Additionally, some of these strategies are stochastically simulated in order to verify the analytical method and evaluate them beyond its applicability. Based on the results of this analysis, an optimisation strategy is proposed for the mapping of neural populations onto interconnected neuromorphic chips, as well as the joint assignment of event network-destinations to a set of accumulation buffers. During this thesis, such an event communication architecture has been implemented on the communication FPGAs in the BrainScaleS-2 accelerated neuromorphic computing system. Thereby, its usability can be scaled beyond single chip setups. For this, the EXTOLL network technology is used to transport and route the aggregated neural event packets with high bandwidth and low latency. At the FPGA, a network bandwidth of up to 12 Gbit/s is usable at a maximum payload efficiency of 94 %. The latency has been measured in the scope of this thesis to a range between 1.6 μs and 2.3 μs across the network between two neuron circuits on separate chips. This latency is thereby mostly dominated by the path from the neuromorphic chip across the communication FPGA into the network and back on the receiving side. As the EXTOLL network hardware itself is clocked at a much higher frequency than the FPGAs, the latency is expected to scale in the order of only approximately 75 ns for each additional hop through the network. For being able to globally interpret the arrival timestamps that are transmitted with every spike event, the system time counters on the FPGAs are synchronised across the network. For this, the global interrupt mechanism implemented in the EXTOLL hardware is characterised and used within this thesis. With this, a synchronisation accuracy of ±40ns could be measured. At the end of this thesis, the successful emulation of a neural signal propagation model, distributed across two BrainScaleS-2 chips and FPGAs is demonstrated using the implemented event communication architecture and the described synchronisation mechanism

Music and time : tempomorphism : nested temporalities in perceived experience of music

This thesis represents the results of a theoretical and practical investigation of acoustic and electro-acoustic elements of Western music at the start of the twentyfirst century, with specific attention to soundscapes. A commentary on the development of soundscapes is drawn from a multidisciplinary overview of concepts of time, followed by an examination of concepts of time in music. As a response to Jonathan Kramer's concept of 'vertical' music (a characteristic aesthetic of which is an absence of conventional harmonic teleology), particular attention is paid to those theories of multiple nested temporalities which have been referred to by Kramer in support of non-teleological musical structures. The survey suggests that new musical concepts, such as vertical music, have emerged from sensibilities resulting from the musical and associated styles of minimalism, and represent an ontological development of aesthetics characteristic of the twentieth century. An original contention of the debate is that innovations in the practice of music as the result of technological developments have led to the possibility of defining a methodology of process in addition to auditive strategies, resulting in a duality defined as 'tempomorphic'. Further observations are supplied, using findings derived from original creative practical research, to define tempomorphic performance, which complete the contribution to knowledge offered by the investigation. Tempomorphism, therefore, is defined as a duality of process and audition: as auditive tool, tempomorphic analysis provides a listening strategy suited to harmonically static music; as a procedural tool, it affords a methodology based primarily on duration.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Wavelet-Based Enhancement Technique for Visibility Improvement of Digital Images

Image enhancement techniques for visibility improvement of color digital images based on wavelet transform domain are investigated in this dissertation research. In this research, a novel, fast and robust wavelet-based dynamic range compression and local contrast enhancement (WDRC) algorithm to improve the visibility of digital images captured under non-uniform lighting conditions has been developed. A wavelet transform is mainly used for dimensionality reduction such that a dynamic range compression with local contrast enhancement algorithm is applied only to the approximation coefficients which are obtained by low-pass filtering and down-sampling the original intensity image. The normalized approximation coefficients are transformed using a hyperbolic sine curve and the contrast enhancement is realized by tuning the magnitude of the each coefficient with respect to surrounding coefficients. The transformed coefficients are then de-normalized to their original range. The detail coefficients are also modified to prevent edge deformation. The inverse wavelet transform is carried out resulting in a lower dynamic range and contrast enhanced intensity image. A color restoration process based on the relationship between spectral bands and the luminance of the original image is applied to convert the enhanced intensity image back to a color image. Although the colors of the enhanced images produced by the proposed algorithm are consistent with the colors of the original image, the proposed algorithm fails to produce color constant results for some pathological scenes that have very strong spectral characteristics in a single band. The linear color restoration process is the main reason for this drawback. Hence, a different approach is required for tackling the color constancy problem. The illuminant is modeled having an effect on the image histogram as a linear shift and adjust the image histogram to discount the illuminant. The WDRC algorithm is then applied with a slight modification, i.e. instead of using a linear color restoration, a non-linear color restoration process employing the spectral context relationships of the original image is applied. The proposed technique solves the color constancy issue and the overall enhancement algorithm provides attractive results improving visibility even for scenes with near-zero visibility conditions. In this research, a new wavelet-based image interpolation technique that can be used for improving the visibility of tiny features in an image is presented. In wavelet domain interpolation techniques, the input image is usually treated as the low-pass filtered subbands of an unknown wavelet-transformed high-resolution (HR) image, and then the unknown high-resolution image is produced by estimating the wavelet coefficients of the high-pass filtered subbands. The same approach is used to obtain an initial estimate of the high-resolution image by zero filling the high-pass filtered subbands. Detail coefficients are estimated via feeding this initial estimate to an undecimated wavelet transform (UWT). Taking an inverse transform after replacing the approximation coefficients of the UWT with initially estimated HR image, results in the final interpolated image. Experimental results of the proposed algorithms proved their superiority over the state-of-the-art enhancement and interpolation techniques