Small-variance asymptotics for Bayesian neural networks

Bayesian neural networks (BNNs) are a rich and flexible class of models that have several advantages over standard feedforward networks, but are typically expensive to train on large-scale data. In this thesis, we explore the use of small-variance asymptotics-an approach to yielding fast algorithms from probabilistic models-on various Bayesian neural network models. We first demonstrate how small-variance asymptotics shows precise connections between standard neural networks and BNNs; for example, particular sampling algorithms for BNNs reduce to standard backpropagation in the small-variance limit. We then explore a more complex BNN where the number of hidden units is additionally treated as a random variable in the model. While standard sampling schemes would be too slow to be practical, our asymptotic approach yields a simple method for extending standard backpropagation to the case where the number of hidden units is not fixed. We show on several data sets that the resulting algorithm has benefits over backpropagation on networks with a fixed architecture.2019-01-02T00:00:00

The Political Economy of Corruption & the Role of Financial Institutions

In many developing and transition countries, we observe rather high levels of corruption. This is surprising from a political economy perspective, as the majority of people in a corrupt country suffer from high corruption levels. Our model is based on the fact that corrupt offcials have to pay entry fees to get lucrative positions. In a probabilistic voting model, we show that a lack of financial institutions can lead to more corruption as more voters are part of the corrupt system and, more importantly, as the rents from corruption are distributed differently. Thus, the economic system has an effect on political outcomes. Well-functioning financial institutions, in turn, increase the political support for anti-corruption measures.http://deepblue.lib.umich.edu/bitstream/2027.42/64363/1/wp892.pd

Deep Learning for User Comment Moderation

Experimenting with a new dataset of 1.6M user comments from a Greek news portal and existing datasets of English Wikipedia comments, we show that an RNN outperforms the previous state of the art in moderation. A deep, classification-specific attention mechanism improves further the overall performance of the RNN. We also compare against a CNN and a word-list baseline, considering both fully automatic and semi-automatic moderation