Convolutional and Residual Networks Provably Contain Lottery Tickets

The Lottery Ticket Hypothesis continues to have a profound practical impact on the quest for small scale deep neural networks that solve modern deep learning tasks at competitive performance. These lottery tickets are identified by pruning large randomly initialized neural networks with architectures that are as diverse as their applications. Yet, theoretical insights that attest their existence have been mostly focused on deep fully-connected feed forward networks with ReLU activation functions. We prove that also modern architectures consisting of convolutional and residual layers that can be equipped with almost arbitrary activation functions can contain lottery tickets with high probability

The dummies' guide to lottery design

This paper outlines the issues relevant to the operation of lottery games. We consider how such games should be designed, what a portfolio of games might look like, how the operator should be regulated, how spending on lottery games should be taxed, and what considerations are relevant to the use of the revenue from such games. Our research suggests that the lottery tickets sales depend positively on the proportion of revenue returned as prizes (i.e the mean of the prize distribution), the skewness in the prize distribution (e.g how much of the prize money goes to the jackpot), and negatively on the variance in the prize distribution. Thus good causes revenue might be higher if the game were meaner (less of the stakes used as prize money), or if more of the prize money was used for the jackpot, or if the variance in the expected prizes were reduced. In practice, it is difficult to change one aspect of the design of the prize distribution without having a counterveiling effect on another aspect. Thus, it is difficult to make judgements about the merits of alternative game designs without looking at all of the parameters being proposed. We find no empirical evidence to suggest that there is any merit in having much of the take-out (the revenue that is not returned as prizes) dedicated to good-causes, and no evidence that the nature of the operator might make any difference. The current “beauty contest” process of choosing an operator is fraught with risk and we suggest that, subject to a probity check, the license should be auctioned