1,115 research outputs found
Learning To Rank Diversely At Airbnb
Airbnb is a two-sided marketplace, bringing together hosts who own listings
for rent, with prospective guests from around the globe. Applying neural
network-based learning to rank techniques has led to significant improvements
in matching guests with hosts. These improvements in ranking were driven by a
core strategy: order the listings by their estimated booking probabilities,
then iterate on techniques to make these booking probability estimates more and
more accurate. Embedded implicitly in this strategy was an assumption that the
booking probability of a listing could be determined independently of other
listings in search results. In this paper we discuss how this assumption,
pervasive throughout the commonly-used learning to rank frameworks, is false.
We provide a theoretical foundation correcting this assumption, followed by
efficient neural network architectures based on the theory. Explicitly
accounting for possible similarities between listings, and reducing them to
diversify the search results generated strong positive impact. We discuss these
metric wins as part of the online A/B tests of the theory. Our method provides
a practical way to diversify search results for large-scale production ranking
systems.Comment: Search ranking, Diversity, e-commerc
Towards Lightweight and Automated Representation Learning System for Networks
We propose LIGHTNE 2.0, a cost-effective, scalable, automated, and
high-quality network embedding system that scales to graphs with hundreds of
billions of edges on a single machine. In contrast to the mainstream belief
that distributed architecture and GPUs are needed for large-scale network
embedding with good quality, we prove that we can achieve higher quality,
better scalability, lower cost, and faster runtime with shared-memory, CPU-only
architecture. LIGHTNE 2.0 combines two theoretically grounded embedding methods
NetSMF and ProNE. We introduce the following techniques to network embedding
for the first time: (1) a newly proposed downsampling method to reduce the
sample complexity of NetSMF while preserving its theoretical advantages; (2) a
high-performance parallel graph processing stack GBBS to achieve high memory
efficiency and scalability; (3) sparse parallel hash table to aggregate and
maintain the matrix sparsifier in memory; (4) a fast randomized singular value
decomposition (SVD) enhanced by power iteration and fast orthonormalization to
improve vanilla randomized SVD in terms of both efficiency and effectiveness;
(5) Intel MKL for proposed fast randomized SVD and spectral propagation; and
(6) a fast and lightweight AutoML library FLAML for automated hyperparameter
tuning. Experimental results show that LIGHTNE 2.0 can be up to 84X faster than
GraphVite, 30X faster than PBG and 9X faster than NetSMF while delivering
better performance. LIGHTNE 2.0 can embed very large graph with 1.7 billion
nodes and 124 billion edges in half an hour on a CPU server, while other
baselines cannot handle very large graphs of this scale
Fusion of Model-free Reinforcement Learning with Microgrid Control: Review and Vision
Challenges and opportunities coexist in microgrids as a result of emerging
large-scale distributed energy resources (DERs) and advanced control
techniques. In this paper, a comprehensive review of microgrid control is
presented with its fusion of model-free reinforcement learning (MFRL). A
high-level research map of microgrid control is developed from six distinct
perspectives, followed by bottom-level modularized control blocks illustrating
the configurations of grid-following (GFL) and grid-forming (GFM) inverters.
Then, mainstream MFRL algorithms are introduced with an explanation of how MFRL
can be integrated into the existing control framework. Next, the application
guideline of MFRL is summarized with a discussion of three fusing approaches,
i.e., model identification and parameter tuning, supplementary signal
generation, and controller substitution, with the existing control framework.
Finally, the fundamental challenges associated with adopting MFRL in microgrid
control and corresponding insights for addressing these concerns are fully
discussed.Comment: 14 pages, 4 figures, published on IEEE Transaction on Smart Grid 2022
Nov 15. See:
https://ieeexplore-ieee-org.utk.idm.oclc.org/stamp/stamp.jsp?arnumber=995140
Exploiting the power of multiplicity: a holistic survey of network-layer multipath
The Internet is inherently a multipath network: For an underlying network with only a single path, connecting various nodes would have been debilitatingly fragile. Unfortunately, traditional Internet technologies have been designed around the restrictive assumption of a single working path between a source and a destination. The lack of native multipath support constrains network performance even as the underlying network is richly connected and has redundant multiple paths. Computer networks can exploit the power of multiplicity, through which a diverse collection of paths is resource pooled as a single resource, to unlock the inherent redundancy of the Internet. This opens up a new vista of opportunities, promising increased throughput (through concurrent usage of multiple paths) and increased reliability and fault tolerance (through the use of multiple paths in backup/redundant arrangements). There are many emerging trends in networking that signify that the Internet's future will be multipath, including the use of multipath technology in data center computing; the ready availability of multiple heterogeneous radio interfaces in wireless (such as Wi-Fi and cellular) in wireless devices; ubiquity of mobile devices that are multihomed with heterogeneous access networks; and the development and standardization of multipath transport protocols such as multipath TCP. The aim of this paper is to provide a comprehensive survey of the literature on network-layer multipath solutions. We will present a detailed investigation of two important design issues, namely, the control plane problem of how to compute and select the routes and the data plane problem of how to split the flow on the computed paths. The main contribution of this paper is a systematic articulation of the main design issues in network-layer multipath routing along with a broad-ranging survey of the vast literature on network-layer multipathing. We also highlight open issues and identify directions for future work
Chi: a scalable and programmable control plane for distributed stream processing systems
Stream-processing workloads and modern shared cluster environments exhibit high variability and unpredictability. Combined with the large parameter space and the diverse set of user SLOs, this makes modern streaming systems very challenging to statically configure and tune. To address these issues, in this paper we investigate a novel control-plane design, Chi, which supports continuous monitoring and feedback, and enables dynamic re-configuration. Chi leverages the key insight of embedding control-plane messages in the data-plane channels to achieve a low-latency and flexible control plane for stream-processing systems. Chi introduces a new reactive programming model and design mechanisms to asynchronously execute control policies, thus avoiding global synchronization. We show how this allows us to easily implement a wide spectrum of control policies targeting different use cases observed in production. Large-scale experiments using production workloads from a popular cloud provider demonstrate the flexibility and efficiency of our approach
Resource Allocation, and Survivability in Network Virtualization Environments
Network virtualization can offer more flexibility and better manageability for the future Internet by allowing multiple heterogeneous virtual networks (VN) to coexist on a shared infrastructure provider (InP) network. A major challenge in this respect is the VN embedding problem that deals with the efficient mapping of virtual resources on InP network resources. Previous research focused on heuristic algorithms for the VN embedding problem assuming that the InP network remains operational at all times. In this thesis, we remove that assumption by formulating the survivable virtual network embedding (SVNE) problem and developing baseline policy heuristics and an efficient hybrid policy heuristic to solve it. The hybrid policy is based on a fast re-routing strategy and utilizes a pre-reserved quota for backup on each physical link. Our evaluation results show that our proposed heuristic for SVNE outperforms baseline heuristics in terms of long term business profit for the InP, acceptance ratio, bandwidth efficiency, and response time
- …