Visual Search at eBay

In this paper, we propose a novel end-to-end approach for scalable visual search infrastructure. We discuss the challenges we faced for a massive volatile inventory like at eBay and present our solution to overcome those. We harness the availability of large image collection of eBay listings and state-of-the-art deep learning techniques to perform visual search at scale. Supervised approach for optimized search limited to top predicted categories and also for compact binary signature are key to scale up without compromising accuracy and precision. Both use a common deep neural network requiring only a single forward inference. The system architecture is presented with in-depth discussions of its basic components and optimizations for a trade-off between search relevance and latency. This solution is currently deployed in a distributed cloud infrastructure and fuels visual search in eBay ShopBot and Close5. We show benchmark on ImageNet dataset on which our approach is faster and more accurate than several unsupervised baselines. We share our learnings with the hope that visual search becomes a first class citizen for all large scale search engines rather than an afterthought.Comment: To appear in 23rd SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017. A demonstration video can be found at https://youtu.be/iYtjs32vh4

Distributed storage manager system for synchronized and scalable AV services across networks

This article has been made available through the Brunel Open Access Publishing Fund - Copyright @ 2011 Hindawi Publishing CorporationThis paper provides an innovative solution, namely, the distributed storage manager that opens a new path for highly interactive and personalized services. The distributed storage manager provides an enhancement to the MHP storage management functionality acting as a value added middleware distributed across the network. The distributed storage manager system provides multiple protocol support for initializing and downloading both streamed and file-based content and provides optimum control mechanisms to organize the storing and retrieval of content that are remained accessible to other multiple heterogeneous devices

Dynamic virtual private network provisioning from multiple cloud infrastructure service providers

The Cloud infrastructure service providers currently provision basic virtualized computing resources as on demand and dynamic services but there is no common framework in existence that allows the seamless provisioning of even these basic services across multiple cloud service providers, although this is not due to any inherent incompatibility or proprietary nature of the foundation technologies on which these cloud platforms are built. We present a solution idea which aims to provide a dynamic and service oriented provisioning of secure virtual private networks on top of multiple cloud infrastructure service providers. This solution leverages the benefits of peer to peer overlay networks, i.e., the flexibility and scalability to handle the churn of nodes joining and leaving the VPNs and can adapt the topology of the VPN as per the requirements of the applications utilizing its intercloud secure communication framework

Parallelizing support vector machines for scalable image annotation

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them Support Vector Machines (SVMs) are used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. In this thesis distributed computing paradigms have been investigated to speed up SVM training, by partitioning a large training dataset into small data chunks and process each chunk in parallel utilizing the resources of a cluster of computers. A resource aware parallel SVM algorithm is introduced for large scale image annotation in parallel using a cluster of computers. A genetic algorithm based load balancing scheme is designed to optimize the performance of the algorithm in heterogeneous computing environments. SVM was initially designed for binary classifications. However, most classification problems arising in domains such as image annotation usually involve more than two classes. A resource aware parallel multiclass SVM algorithm for large scale image annotation in parallel using a cluster of computers is introduced. The combination of classifiers leads to substantial reduction of classification error in a wide range of applications. Among them SVM ensembles with bagging is shown to outperform a single SVM in terms of classification accuracy. However, SVM ensembles training are notably a computationally intensive process especially when the number replicated samples based on bootstrapping is large. A distributed SVM ensemble algorithm for image annotation is introduced which re-samples the training data based on bootstrapping and training SVM on each sample in parallel using a cluster of computers. The above algorithms are evaluated in both experimental and simulation environments showing that the distributed SVM algorithm, distributed multiclass SVM algorithm, and distributed SVM ensemble algorithm, reduces the training time significantly while maintaining a high level of accuracy in classifications