One-to-many file transfer using multipath multicast with gossiping

With the recent progress of cloud and distributed computing technologies, data migration and replication among distributed data centers grows rapidly. To manage a simplified scenario that a single sender sends a large-sized file to multiple recipients, i.e., one-to-many file transfer, on a network with full-duplex links, we are developing the Multipath-Multicast (MPMC) file transfer. A file is appropriately divided into equally-sized blocks; different blocks are concurrently transmitted to the same recipient on multiple paths; while the same block is concurrently transmitted to multiple recipients by multicast, aiming at shorter reception completion times of all recipients. However, on large-scale complex network topologies, it is not easy to find a good block transfer schedule, i.e., that realizes the reception completion times of most recipients close to their lower-bounds in MPMC. In this report, therefore, a gossiping approach to allow block transfer among recipients is introduced into MPMC and evaluated through simulation on two real backbone topologies. Since unused capacities of links in the original basic MPMC can be utilized in the MPMC with gossiping, a good schedule can be found more easily compared with the basic MPMC even with the same simple greedy block allocation.The 4th IEEE International Conference on Network Softwarization (NetSoft 2018) , June 25-29, 2018, Montreal, Canada

QuickCast: Fast and Efficient Inter-Datacenter Transfers using Forwarding Tree Cohorts

Large inter-datacenter transfers are crucial for cloud service efficiency and are increasingly used by organizations that have dedicated wide area networks between datacenters. A recent work uses multicast forwarding trees to reduce the bandwidth needs and improve completion times of point-to-multipoint transfers. Using a single forwarding tree per transfer, however, leads to poor performance because the slowest receiver dictates the completion time for all receivers. Using multiple forwarding trees per transfer alleviates this concern--the average receiver could finish early; however, if done naively, bandwidth usage would also increase and it is apriori unclear how best to partition receivers, how to construct the multiple trees and how to determine the rate and schedule of flows on these trees. This paper presents QuickCast, a first solution to these problems. Using simulations on real-world network topologies, we see that QuickCast can speed up the average receiver's completion time by as much as $10\times$ while only using $1.04\times$ more bandwidth; further, the completion time for all receivers also improves by as much as $1.6\times$ faster at high loads.Comment: [Extended Version] Accepted for presentation in IEEE INFOCOM 2018, Honolulu, H

Minimization of reception completion times by one-to-many file transfer using MultiPath-MultiCast with Reed-Solomon coding

分散配置されたデータセンタやサーバ間での大規模なデータやソフトウェアの共有,複製,または移動によるトラヒック量の急激な増加が問題となっている.そこで,単一の送信者から各受信者へのmax-flowを達成する複数経路（max-flow経路）を用いて,各受信者が自身の最短時間で受信を完了する最適スケジュールを実現するために,複数経路マルチキャストによる一対多ファイル転送（MPMC）が検討されてきた.MPMCでは，転送ファイルを均等長のブロックに分割し,同一受信者へ複数のブロックをmax-flow経路で同時転送すると共に,複数の受信者へ同一ブロックをマルチキャスト転送する．本報告の提案手法は,リード・ソロモン符号化を用いて必要な数の符号化ブロックを生成し,フェーズ（ある群が受信完了した後、次の群が受信完了するまでの期間）毎に異なる符号化ブロックを転送することで,受信者間の前フェーズまでの受信ブロックの違いに影響されずにブロック割当を最適化できる.さらに,ヒューリスティックなブロック割当順序管理方法を導入し,大規模な実世界のネットワークトポロジに対しても最適スケジュールが容易に生成できることをシミュレーションで検証した.また,実装した提案手法の動作確認をOpenFlowエミュレータ上で行った. / A rapid increase in network traffic has caused a problem along with the penetration of sharing, duplicating, or migrating a large-sized data and software among distributed servers or sites. We previously proposed the one-to-many file transfer using MultiPath-MultiCast (MPMC) on OpenFlow to realize an optimal schedule in which each recipient can complete the file reception in its minimal time using the max-flow paths from a single sender. In MPMC, a file is divided into equally-sized blocks; different blocks are concurrently transmitted to the same recipient on multiple paths; while the same block is concurrently transmitted to multiple recipients by multicast. This report newly proposes the coded-MPMC in which a sender proactively generates a necessary number of coded blocks using Reed-Solomon coding and transmits different coded blocks in each phase (a period between when a set of recipients completed and when the next set of recipients complete), allowing an optimal block allocation regardless of a difference among recipients’ already-received blocks in the previous phases. A few heuristics in the block allocation order are developed in coded-MPMC and shown to efficiently find optimal schedules on large-scale real-world network topologies through simulation. A preliminary implementation of coded-MPMC is verified on an OpenFlow emulator.電子情報通信学会 コミュニケーションクオリティ研究会（CQ）, 2019年3月14日-15日, 鹿児島市, 日

Coded-MPMC: One-to-Many Transfer Using Multipath Multicast With Sender Coding

One-to-many transfers in a fast and efficient manner are essential to meet the growing need for duplicating, migrating, or sharing bulk data among servers in a datacenter and across geographically distributed datacenters. Some existing works utilize multiple multicast trees for a one-to-many transfer request to increase network link utilization and its transfer throughput. However, since those schemes do not fully utilize the max-flow value of transmission from a single sender to each recipient, there is room for each recipient to retrieve data more quickly. Therefore, assuming fully-controlled networks with full-duplex links, we pose a problem to find a set of multicast flows with an allocation of block-wise transmissions by which each of multiple recipients with diverse max-flow values from the sender can utilize its own max-flow value. Based on that, assuming a sender-side coding capability on file blocks, we design a schedule of block transmissions over multiple phases by which each recipient can achieve a lower-bound of its file retrieval completion time, i.e., the file size divided by its own max-flow value. This paper presents the coded Multipath Multicast (Coded-MPMC) for one-to-many transfers with heuristic procedures to find a desired set of multicast flows on which block transmissions are scheduled. Through extensive simulations on large-scale real-world network topologies and different types of randomly-generated synthetic topologies, the proposed method is shown to design a desired schedule efficiently. A preliminary implementation on OpenFlow is also reported to show the fundamental feasibility of Coded-MPMC

Способ многопутевой маршрутизации в компьютерных сетях большой размерности

Багатошляхова маршрутизація характеризується великою часовою складністю пошуку множини шляхів, що не перетинаються. Часова складність знаходження найкоротшого шляху по алгоритму Дейкстри представляє собою величину O(kN2). При знаходженні k-шляхів часова складність збільшується відповідно в k раз. В зв’язку з цим, для пошуку множини шляхів, що не перетинаються, в рамках цієї роботи був запропонований модифікований метод «гілок та границь». Це досягається за рахунок виключення операцій перебору варіантів формування кожного шляху. В процесі роботи алгоритму у відповідності з методом «гілок та границь» будується дерево рішень, коренем якого є початкова вершина, а листями є вершини, суміжні з кінцевою вершиною