Flexibler Birth-Death MCMC Sampler für Changepoint-Modelle

Diese Arbeit beschreibt eine flexible Architektur eines Markov Chain Monte Carlo Samplers, der Bayessche Inferenz für eine Vielzahl von Changepoint-Modellen erlaubt. Die Struktur dieser Klasse von Modellen besteht aus zwei stochastischen Prozessen. Der erste Prozess wird entweder direkt beobachtet oder indirekt durch, möglicherweise verrauschte, Beobachtungen. Der zweite Prozess ist unbeobachtet und bestimmt die Parameter des beobachteten Prozesses. Die Hauptannahme unserer Modellklasse ist, dass der versteckte Prozess stückweise konstant ist, d.h. er springt zwischen diskreten Zuständen. Als beobachteter Prozess diskutieren wir hauptsächlich den Ornstein-Uhlenbeck und Poisson Prozess. Der versteckte Prozess kann eine feste Anzahl von Zuständen haben oder eine unbekannte Anzahl. Im zweiten Fall basiert das Modell auf einem versteckten Chinese Restaurant Prozess und ermöglicht so Bayessche Inferenz über die Anzahl der Zustände des versteckten Parameterprozesses. Der Sampler wendet einen Metropolis-Hastings Random Walk auf den versteckten Prozess an indem Birth-Death Schritte vorgeschlagen werden. Die Arbeit präsentiert unterschiedliche Modifikationen des Pfades des versteckten Prozesses. Die Struktur des Samplers ist sehr flexibel und lässt sich, im Vergleich zu anderen Algorithmen, die für ein spezifisches Modell maßgeschneidert sind, einfach an verschiedene Kombinationen von beobachteten und versteckten Prozessen anpassen. Angewandt auf Genexpressionsdaten ermöglicht der Sampler Bayessche Inferenz für komplexere Modelle als vorherige Methoden. Der berechnete Bayes Faktor deutet an, dass unser Modell, welches es erlaubt die Stärke des intrinsischen Rauschens zu variieren, die Daten besser erklärt als das vorherige Modell. Der Sampler wird für Genexpressionsdaten von Hefezellen benutzt und die Ergebnisse mit denen einer variationellen Näherung verglichen. Der Posterior scheint genauer in der Vorhersage der Aktivierungszeitpunkte der Transkriptionsfaktoren zu sein als es die Näherung zeigt. Die Ergebnisse des Chinese Restaurant Prozess Samplers auf den gleichen Messungen von Hefezellen unterstützt die vorherige Annahme über die Anzahl der Transkriptionsfaktoren, die in die Kontrolle der untersuchten Gene involviert sind. Die Anpassung des Samplers an Markov modulierte Poisson Prozesse beschleunigt die Inferenz und dies wird gezeigt, indem die Zeit zur Berechnung eines unkorrelierten Samples mit einem exakten Gibbs Sampler verglichen wird. Ein Modell, welches einen beobachteten Poisson Prozess mit dem Chinese Restaurant Prozess verbindet wird anschließend benutzt um versteckte Zustände in der Rate von neuronalen Spike-Daten zu finden und sie mit dem Stimulus zu verbinden. Die Vorteile des Modells beim finden und bestimmen von neuronalen Bursts wird diskutiert und mit Modellen verglichen, die eine kontinuierliche Poisson Rate annehmen.This thesis describes a flexible architecture for a Markov chain Monte Carlo sampler which allows Bayesian posterior inference for a variety of changepoint models. The structure of this class of models consists of two stochastic processes. The first process is either observed directly or indirectly through, possibly noisy, observations. The second process is not observed and governs the parameters of the observed process. The main assumption for our class of models is that the hidden process is piecewise constant, i.e. it jumps between discrete states. As the observed process, we discuss mainly the Ornstein-Uhlenbeck and Poisson process. The hidden process can have a fixed number of states, or an unknown number of states. The latter model is based on a hidden Chinese restaurant process and allows Bayesian inference over the number of states of the hidden parameters. The sampler applies a Metropolis-Hastings random walk on the hidden jump process through proposed birth-death moves. Different kinds of proposal moves on the path of the hidden process are presented. The structure of the sampler makes it very flexible and easy to modify to other combinations of observed and hidden processes compared to other inference methods which are tailor-made for a specific model. Applied to gene expression data the sampler allows Bayesian posterior inference on a more complex model than in previous work. We compute the Bayes factor which indicates that our model, which allows the strength of the system noise to switch, is better in explaining the data. The sampler is used on gene expression data from yeast cells and the results are compared to a variational approximation. The posterior is more confident about the times of transcriptional activity than the approximation suggests. The results from the Chinese restaurant process sampler on the same yeast dataset support the initial assumption about the number of transcription factors involved in the control of the examined genes. When the sampler is used on financial data, changepoints are revealed which can be connected to historic events. This is shown both for the Ornstein-Uhlenbeck model as well as a Cox-Ingersoll-Ross model used in a different thesis. Modifying the sampler to work on Markov modulated Poisson processes allows for very fast posterior inference and this is shown when the time to get an uncorrelated sample is compared to an exact Gibbs sampler for the model. A model combining an observed Poisson process with the Chinese restaurant process is then utilized to find hidden states in the rate of neuronal spike trains and linked to the stimulus. The model's advantages in finding and estimating bursting of neurons is discussed and compared to a model which assumes a continuous Poisson rate

Wavenet based low rate speech coding

Traditional parametric coding of speech facilitates low rate but provides poor reconstruction quality because of the inadequacy of the model used. We describe how a WaveNet generative speech model can be used to generate high quality speech from the bit stream of a standard parametric coder operating at 2.4 kb/s. We compare this parametric coder with a waveform coder based on the same generative model and show that approximating the signal waveform incurs a large rate penalty. Our experiments confirm the high performance of the WaveNet based coder and show that the speech produced by the system is able to additionally perform implicit bandwidth extension and does not significantly impair recognition of the original speaker for the human listener, even when that speaker has not been used during the training of the generative model.Comment: 5 pages, 2 figure

Benchmarking Robustness to Adversarial Image Obfuscations

Automated content filtering and moderation is an important tool that allows online platforms to build striving user communities that facilitate cooperation and prevent abuse. Unfortunately, resourceful actors try to bypass automated filters in a bid to post content that violate platform policies and codes of conduct. To reach this goal, these malicious actors may obfuscate policy violating images (e.g. overlay harmful images by carefully selected benign images or visual patterns) to prevent machine learning models from reaching the correct decision. In this paper, we invite researchers to tackle this specific issue and present a new image benchmark. This benchmark, based on ImageNet, simulates the type of obfuscations created by malicious actors. It goes beyond ImageNet-$\textrm{C}$ and ImageNet-$\bar{\textrm{C}}$ by proposing general, drastic, adversarial modifications that preserve the original content intent. It aims to tackle a more common adversarial threat than the one considered by $\ell_p$-norm bounded adversaries. We evaluate 33 pretrained models on the benchmark and train models with different augmentations, architectures and training methods on subsets of the obfuscations to measure generalization. We hope this benchmark will encourage researchers to test their models and methods and try to find new approaches that are more robust to these obfuscations

SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks With at Most One Spike per Neuron

International audienceApplication of deep convolutional spiking neural networks (SNNs) to artificial intelligence (AI) tasks has recently gained a lot of interest since SNNs are hardware-friendly and energy-efficient. Unlike the non-spiking counterparts, most of the existing SNN simulation frameworks are not practically efficient enough for large-scale AI tasks. In this paper, we introduce SpykeTorch, an open-source high-speed simulation framework based on PyTorch. This framework simulates convolutional SNNs with at most one spike per neuron and the rank-order encoding scheme. In terms of learning rules, both spike-timing-dependent plasticity (STDP) and reward-modulated STDP (R-STDP) are implemented, but other rules could be implemented easily. Apart from the aforementioned properties, SpykeTorch is highly generic and capable of reproducing the results of various studies. Computations in the proposed framework are tensor-based and totally done by PyTorch functions, which in turn brings the ability of just-in-time optimization for running on CPUs, GPUs, or Multi-GPU platforms