StitchNet: Composing Neural Networks from Pre-Trained Fragments

We propose StitchNet, a novel neural network creation paradigm that stitches together fragments (one or more consecutive network layers) from multiple pre-trained neural networks. StitchNet allows the creation of high-performing neural networks without the large compute and data requirements needed under traditional model creation processes via backpropagation training. We leverage Centered Kernel Alignment (CKA) as a compatibility measure to efficiently guide the selection of these fragments in composing a network for a given task tailored to specific accuracy needs and computing resource constraints. We then show that these fragments can be stitched together to create neural networks with accuracy comparable to that of traditionally trained networks at a fraction of computing resource and data requirements. Finally, we explore a novel on-the-fly personalized model creation and inference application enabled by this new paradigm. The code is available at https://github.com/steerapi/stitchnet

Taming Wireless Fluctuations by Predictive Queuing Using a Sparse-Coding Link-State Model

We introduce State-Informed Link-Layer Queuing (SILQ), a system that models, predicts, and avoids packet delivery failures due to temporary wireless outages in everyday scenarios. By stabilizing connections in adverse link conditions, SILQ boosts throughput and reduces performance variation for network applications, for example by preventing unnecessary TCP timeouts caused by dead zones, elevators, and subway tunnels. SILQ makes predictions in real-time by actively probing links, matching measurements to an overcomplete dictionary of patterns learned offline, and classifying the resulting sparse feature vectors to identify those that precede outages. We use a clustering method called sparse coding to build our data-driven link model, and show that it produces more variation-tolerant predictions than traditional loss-rate, location-based, or Markov chain techniques. We present extensive data collection and field-validation of SILQ in airborne, indoor, and urban scenarios of practical interest. We show how offline unsupervised learning discovers link-state patterns that are stable across diverse networks and signal-propagation environments. Using these canonical primitives, we train outage predictors for 802.11 (Wi-Fi) and 3G cellular networks to demonstrate TCP throughput gains of 4x with off-the-shelf mobile devices. SILQ addresses delivery failures solely at the link layer, requires no new hardware, and upholds the end-to-end design principle, enabling easy integration across applications, devices, and networks.Engineering and Applied Science

A Future of Abundant Sparsity: Novel Use and Analysis of Sparse Coding in Machine Learning Applications

We present novel applications and analysis of the use of sparse coding within the con- text of machine learning. We first present Sparse Coding Trees (SC-trees), a sparse coding-based framework for resolving classification conflicts, which occur when different classes are mapped to similar feature representations. More specifically, SC-trees are novel supervised hierarchical clustering trees that use node specific dictionary and classifier training to direct input images based on classification results in the feature space at each node. We validate SC-trees on image-based emotion classification, combining it with Mirrored Nonnegative Sparse Coding (MNNSC), a novel sparse coding algorithm leveraging a nonnegativity constraint and the inherent symmetry of the domain, to achieve results exceeding or competitive with the state-of-the-art. We next present SILQ, a sparse coding-based link state model that can predictively buffer packets during wireless link outages to avoid disruption to higher layer protocols such as TCP. We demonstrate empirically that SILQ increases TCP throughput by a factor of 2-4x in varied scenarios.Computer Scienc

Lambda means clustering: Automatic parameter search and distributed computing implementation

Recent advances in clustering have shown that ensuring a minimum separation between cluster centroids leads to higher quality clusters compared to those found by methods that explicitly set the number of clusters to be found, such as k-means. One such algorithm is DP-means, which sets a distance parameter λ for the minimum separation. However, without knowing either the true number of clusters or the underlying true distribution, setting λ itself can be difficult, and poor choices in setting λ will negatively impact cluster quality. As a general solution for finding λ, in this paper we present λ-means, a clustering algorithm capable of deriving an optimal value for λ automatically. We contribute both a theoretically-motivated cluster-based version of λ-means, as well as a faster conflict-based version of λ-means. We demonstrate that λ-means discovers the true underlying value of λ asymptotically when run on datasets generated by a Dirichlet Process, and achieves competitive performance on a real world test dataset. Further, we demonstrate that when run on both parallel multicore computers and distributed cluster computers in the cloud, cluster-based λ-means achieves near perfect speedup, and while being a more efficient algorithm, conflict-based λmeans achieves speedups only a factor of two away from the maximum-possible.Engineering and Applied Science