System support for keyword-based search in structured Peer-to-Peer systems

In this dissertation, we present protocols for building a distributed search infrastructure over structured Peer-to-Peer systems. Unlike existing search engines which consist of large server farms managed by a centralized authority, our approach makes use of a distributed set of end-hosts built out of commodity hardware. These end-hosts cooperatively construct and maintain the search infrastructure. The main challenges with distributing such a system include node failures, churn, and data migration. Localities inherent in query patterns also cause load imbalances and hot spots that severely impair performance. Users of search systems want their results returned quickly, and in ranked order. Our main contribution is to show that a scalable, robust, and distributed search infrastructure can be built over existing Peer-to-Peer systems through the use of techniques that address these problems. We present a decentralized scheme for ranking search results without prohibitive network or storage overhead. We show that caching allows for efficient query evaluation and present a distributed data structure, called the View Tree, that enables efficient storage, and retrieval of cached results. We also present a lightweight adaptive replication protocol, called LAR that can adapt to different kinds of query streams and is extremely effective at eliminating hotspots. Finally, we present techniques for storing indexes reliably. Our approach is to use an adaptive partitioning protocol to store large indexes and employ efficient redundancy techniques to handle failures. Through detailed analysis and experiments we show that our techniques are efficient and scalable, and that they make distributed search feasible

Brain Inspired Enhanced Learning Mechanism Based on Spike Timing Dependent Plasticity (STDP) for Efficient Pattern Recognition in Spiking Neural Networks

Artificial neural networks, that try to mimic the brain, are a very active area of research today. Such networks can potentially solve difficult problems such as image recognition, video analytics, lot more energy efficiently than when implemented in standard von-Neumann computing machines. New algorithms for neural computing with high bio-fidelity are being developed today to solve hard machine learning problems. In this work, we used a spiking network model, and implemented a self-learning technique using a Spike Timing Dependent Plasticity (STDP) algorithm, that closely mimics the neural activity of the brain. The basic STDP algorithm modulates the synaptic weights interconnecting the neurons based on pairs of pre- and post-synaptic spikes. This ignores the timing information embedded in the frequency of the post-synaptic spikes. We calculated the average of the membrane potential of each column of neurons to give an idea of how it behaved and spiked for the particular output neuron for a particular image in the past .The update of the weights or the synapses are done on the basis of the frequency obtained. The resultant synaptic updates are less frequent and made wisely making the learning process better. With the present algorithm, we are able to achieve an accuracy of 79% for classifying images from the MNIST data set for a network of 400 output neurons. So the model was able to identify 79% of the total images correctly which is greater than the original STDP signifying that slow and sensible updates are definitely having a better impact on the learning process

Efficient Peer-to-Peer Namespace Searches

In this paper we describe new methods for efficient and exact search (keyword and full-text) in distributed namespaces. Our methods can be used in conjunction with existing distributed lookup schemes, such as Distributed Hash Tables, and distributed directories. We describe how indexes for implementing distributed searches can be efficiently created, located, and stored. We describe techniques for creating approximate indexes that can be used to bound the space requirement at individual hosts; such techniques are particularly useful for full-text searches that may require a very large number of individual indexes to be created and maintained. Our methods use a new distributed data structure called the view tree. View trees can be used to efficiently cache and locate results from prior queries. We describe how view trees are created, and maintained. We present experimental results, using large namespaces and realistic data, showing that the techniques introduced in this paper can reduce search overheads (both network and processing costs) by more than an order of magnitude. (UMIACS-TR-2004-13

Back to the future: Throughput prediction for cellular networks using radio KPIs

The availability of reliable predictions for cellular throughput would offer a fundamental change in the way applications are designed and operated. Numerous cellular applications, including video streaming and VoIP, embed logic that attempts to estimate achievable throughput and adapt their behaviour accordingly. We believe that providing applications with reliable predictions several seconds into the future would enable profoundly better adaptation decisions and dramatically benefit demanding applications like mobile virtual and augmented reality. The question we pose and seek to address is whether such reliable predictions are possible. We conduct a preliminary study of throughput prediction in a cellular environment using statistical machine learning techniques. An accurate prediction can be very challenging in large scale cellular environments because they are characterized by highly fluctuating channel conditions. Using simulations and real-world experiments, we study how prediction error varies as a function of prediction horizon, and granularity of available data. In particular, our simulation experiments show that the prediction error for mobile devices can be reduced significantly by combining measurements from the network with measurements from the end device. Our results indicate that it is possible to accurately predict achievable throughput up to 8 sec in the future where 50th percentile of all errors are less than 15% for mobile and 2% for static devices