Many-Task Computing and Blue Waters

This report discusses many-task computing (MTC) generically and in the context of the proposed Blue Waters systems, which is planned to be the largest NSF-funded supercomputer when it begins production use in 2012. The aim of this report is to inform the BW project about MTC, including understanding aspects of MTC applications that can be used to characterize the domain and understanding the implications of these aspects to middleware and policies. Many MTC applications do not neatly fit the stereotypes of high-performance computing (HPC) or high-throughput computing (HTC) applications. Like HTC applications, by definition MTC applications are structured as graphs of discrete tasks, with explicit input and output dependencies forming the graph edges. However, MTC applications have significant features that distinguish them from typical HTC applications. In particular, different engineering constraints for hardware and software must be met in order to support these applications. HTC applications have traditionally run on platforms such as grids and clusters, through either workflow systems or parallel programming systems. MTC applications, in contrast, will often demand a short time to solution, may be communication intensive or data intensive, and may comprise very short tasks. Therefore, hardware and software for MTC must be engineered to support the additional communication and I/O and must minimize task dispatch overheads. The hardware of large-scale HPC systems, with its high degree of parallelism and support for intensive communication, is well suited for MTC applications. However, HPC systems often lack a dynamic resource-provisioning feature, are not ideal for task communication via the file system, and have an I/O system that is not optimized for MTC-style applications. Hence, additional software support is likely to be required to gain full benefit from the HPC hardware

FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation

One of the most popular approaches to multi-target tracking is tracking-by-detection. Current min-cost flow algorithms which solve the data association problem optimally have three main drawbacks: they are computationally expensive, they assume that the whole video is given as a batch, and they scale badly in memory and computation with the length of the video sequence. In this paper, we address each of these issues, resulting in a computationally and memory-bounded solution. First, we introduce a dynamic version of the successive shortest-path algorithm which solves the data association problem optimally while reusing computation, resulting in significantly faster inference than standard solvers. Second, we address the optimal solution to the data association problem when dealing with an incoming stream of data (i.e., online setting). Finally, we present our main contribution which is an approximate online solution with bounded memory and computation which is capable of handling videos of arbitrarily length while performing tracking in real time. We demonstrate the effectiveness of our algorithms on the KITTI and PETS2009 benchmarks and show state-of-the-art performance, while being significantly faster than existing solvers

Raptorq-Based Multihop File Broadcast Protocol

The objective of this thesis is to describe and implement a RaptorQ broadcast protocol application layer designed for use in a wireless multihop network. The RaptorQ broadcast protocol is a novel application layer broadcast protocol based on RaptorQ forward error correction. This protocol can deliver a file reliably to a large number of nodes in a wireless multihop network even if the links have high loss rates. We use mixed integer programming with power balance constraints to construct broadcast trees that are suitable for implementing the RaptorQ-based broadcast protocol. The resulting broadcast tree facilitates deployment of mechanisms for verifying successful delivery. We use the Qualcomm proprietary RaptorQ software development kit library as well as a Ruby interface to implement the protocol. During execution, each node operates in one of main modes: source, transmitter, or leaf. Each mode has five different phases: STARTUP, FINISHING (Poll), FINISHING (Wait), FINISHING (Extra), and COMPLETED. Three threads are utilized to implement the RaptorQ-based broadcast protocol features. Thread 1 receives messages and passes them to the receive buffer. Thread 2 evaluates the received message, which can be NORM, POLL, MORE, and DONE, and passes the response message to the send buffer. Thread 3 multicasts the content of the send buffer. Results obtained by testing the implementation of the RaptorQ-based broadcast protocol demonstrate that efficient and reliable distribution of files over multihop wireless networks with a high link loss rates is feasible

Energy Harvesting for Secure OFDMA Systems

Energy harvesting and physical-layer security in wireless networks are of great significance. In this paper, we study the simultaneous wireless information and power transfer (SWIPT) in downlink orthogonal frequency-division multiple access (OFDMA) systems, where each user applies power splitting to coordinate the energy harvesting and information decoding processes while secrecy information requirement is guaranteed. The problem is formulated to maximize the aggregate harvested power at the users while satisfying secrecy rate requirements of all users by subcarrier allocation and the optimal power splitting ratio selection. Due to the NP-hardness of the problem, we propose an efficient iterative algorithm. The numerical results show that the proposed method outperforms conventional methods.Comment: Accepted by WCSP 201