Local list decoding of homomorphisms

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (leaves 47-49).We investigate the local-list decodability of codes whose codewords are group homomorphisms. The study of such codes was intiated by Goldreich and Levin with the seminal work on decoding the Hadamard code. Many of the recent abstractions of their initial algorithm focus on Locally Decodable Codes (LDC's) over finite fields. We derive our algorithmic approach from the list decoding of the Reed-Muller code over finite fields proposed by Sudan, Trevisan and Vadhan. Given an abelian group G and a fixed abelian group H, we give combinatorial bounds on the number of homomorphisms that have agreement 6 with an oracle-access function f : G --> H. Our bounds are polynomial in , where the degree of the polynomial depends on H. Also, depends on the distance parameter of the code, namely we consider to be slightly greater than 1-minimum distance. Furthermore, we give a local-list decoding algorithm for the homomorphisms that agree on a 3 fraction of the domain with a function f, the running time of which is poly(1/e, log G).by Elena Grigorescu.S.M

Sampling Correctors

In many situations, sample data is obtained from a noisy or imperfect source. In order to address such corruptions, this paper introduces the concept of a sampling corrector. Such algorithms use structure that the distribution is purported to have, in order to allow one to make "on-the-fly" corrections to samples drawn from probability distributions. These algorithms then act as filters between the noisy data and the end user. We show connections between sampling correctors, distribution learning algorithms, and distribution property testing algorithms. We show that these connections can be utilized to expand the applicability of known distribution learning and property testing algorithms as well as to achieve improved algorithms for those tasks. As a first step, we show how to design sampling correctors using proper learning algorithms. We then focus on the question of whether algorithms for sampling correctors can be more efficient in terms of sample complexity than learning algorithms for the analogous families of distributions. When correcting monotonicity, we show that this is indeed the case when also granted query access to the cumulative distribution function. We also obtain sampling correctors for monotonicity without this stronger type of access, provided that the distribution be originally very close to monotone (namely, at a distance $O(1/\log^2 n)$). In addition to that, we consider a restricted error model that aims at capturing "missing data" corruptions. In this model, we show that distributions that are close to monotone have sampling correctors that are significantly more efficient than achievable by the learning approach. We also consider the question of whether an additional source of independent random bits is required by sampling correctors to implement the correction process

Topological Data Analysis of Convolutional Neural Networks Using Depthwise Separable Convolutions

In this dissertation, we present our contribution to a growing body of work combining the fields of Topological Data Analysis (TDA) and machine learning. The object of our analysis is the Convolutional Neural Network, or CNN, a predictive model with a large number of parameters organized using a grid-like geometry. This geometry is engineered to resemble patches of pixels in an image, and thus CNNs are a conventional choice for an image-classifying model. CNNs belong to a larger class of neural network models, which, starting at a random initialization state, undergo a gradual fitting (or training) process, often a variation of gradient descent. The goal of this descent process is to decrease the value of a loss function measuring the error associated with the model’s predictions, and ideally converge to a model minimizing the loss. While such neural networks are known for generating accurate predictions in a wide variety of contexts and having a significant scalability advantage over many other predictive models, they are notoriously difficult to analyze and understand holistically. TDA techniques, such as the persistent homology and Mapper algorithms, have been proposed as methods for exploring the CNN training process. As we will detail, Gunnar Carlsson and Richard Gabrielsson’s TDA work with CNNs has suggested fundamental similarities in the distributions underlying both 3-by-3 image patches in natural images and CNNs trained on these images, i.e., CNNs can learn topological information about their training data. Our work aims to expand this use of TDA to an alternate CNN construction, using the technique of depthwise separable convolution. This construction requires fewer parameters to be trained and requires fewer multiplcation/addition operations than the CNNs discussed in the work of Carlsson and Gabrielsson, which are characterized by what we call the standard convolution. However, we observe that, when trained on a dataset of chest X-Rays, these models are able not only to perform well, but also learn similar topological information to their standard-convolution counterparts