Many-Task Computing and Blue Waters

This report discusses many-task computing (MTC) generically and in the context of the proposed Blue Waters systems, which is planned to be the largest NSF-funded supercomputer when it begins production use in 2012. The aim of this report is to inform the BW project about MTC, including understanding aspects of MTC applications that can be used to characterize the domain and understanding the implications of these aspects to middleware and policies. Many MTC applications do not neatly fit the stereotypes of high-performance computing (HPC) or high-throughput computing (HTC) applications. Like HTC applications, by definition MTC applications are structured as graphs of discrete tasks, with explicit input and output dependencies forming the graph edges. However, MTC applications have significant features that distinguish them from typical HTC applications. In particular, different engineering constraints for hardware and software must be met in order to support these applications. HTC applications have traditionally run on platforms such as grids and clusters, through either workflow systems or parallel programming systems. MTC applications, in contrast, will often demand a short time to solution, may be communication intensive or data intensive, and may comprise very short tasks. Therefore, hardware and software for MTC must be engineered to support the additional communication and I/O and must minimize task dispatch overheads. The hardware of large-scale HPC systems, with its high degree of parallelism and support for intensive communication, is well suited for MTC applications. However, HPC systems often lack a dynamic resource-provisioning feature, are not ideal for task communication via the file system, and have an I/O system that is not optimized for MTC-style applications. Hence, additional software support is likely to be required to gain full benefit from the HPC hardware

Flexible schedule optimization for human-robot collaboration

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 99-101).Robots are increasingly entering domains typically thought of as human-only. This convergence of human and robotic agents leads to a need for new technology to enable safe and efficient collaboration. The goal of this thesis is to develop a task allocation and scheduling algorithm for teams of robots working with or around teams of humans in intense domains where tight, fluid choreography of robotic schedules is required to guarantee the safety of all involved while maintaining high levels of productivity. Three algorithms are presented in this work: the Adaptive Preferences Algorithm, the Multi-Agent Optimization Algorithm, and Tercio. Tercio, the culminatory algorithm, is capable of assigning robots to tasks and producing near-optimal schedules for ten agents and hundreds of tasks in seconds while making guarantees about process specifications such as worker safety and deadline satisfaction. This work extends dynamic scheduling methods to incorporate flexible windows with an optimization framework featuring a mixed integer program and a satisficing hueristic scheduler. By making use of Tercio, a manufacturing facility or other high-intensity domain may fluidly command a team of robots to complete tasks in a quick, efficient manner while maintaining an ability to respond seamlessly to disturbances at execution. This greatly increases both productivity, by decreasing the time spent recompiling solutions, and responsiveness to humans in the area. These improvements in performance are displayed with multiple live demonstrations and simulations of teams of robots responding to disturbances. Tercio acts as an enabling step towards the ultimate goal of fully coordinated factories of dozens to hundreds of robots accomplishing many thousands of tasks in a safe, predictable, efficient manner.by Ronald J. Wilcox.S.M

Collaborative Video Analytics on Distributed Edges with Multiagent Deep Reinforcement Learning

Deep Neural Network (DNN) based video analytics empowers many computer vision-based applications to achieve high recognition accuracy. To reduce inference delay and bandwidth cost for video analytics, the DNN models can be deployed on the edge nodes, which are proximal to end users. However, the processing capacity of an edge node is limited, potentially incurring substantial delay if the inference requests on an edge node is overloaded. While efforts have been made to enhance video analytics by optimizing the configurations on a single edge node, we observe that multiple edge nodes can work collaboratively by utilizing the idle resources on each other to improve the overall processing capacity and resource utilization. To this end, we propose a Multiagent Reinforcement Learning (MARL) based approach, named as EdgeVision, for collaborative video analytics on distributed edges. The edge nodes can jointly learn the optimal policies for video preprocessing, model selection, and request dispatching by collaborating with each other to minimize the overall cost. We design an actor-critic-based MARL algorithm with an attention mechanism to learn the optimal policies. We build a multi-edge-node testbed and conduct experiments with real-world datasets to evaluate the performance of our method. The experimental results show our method can improve the overall rewards by 33.6%-86.4% compared with the most competitive baseline methods

QoS-aware predictive workflow scheduling

This research places the basis of QoS-aware predictive workflow scheduling. This research novel contributions will open up prospects for future research in handling complex big workflow applications with high uncertainty and dynamism. The results from the proposed workflow scheduling algorithm shows significant improvement in terms of the performance and reliability of the workflow applications

Algorithms for Scheduling Problems

This edited book presents new results in the area of algorithm development for different types of scheduling problems. In eleven chapters, algorithms for single machine problems, flow-shop and job-shop scheduling problems (including their hybrid (flexible) variants), the resource-constrained project scheduling problem, scheduling problems in complex manufacturing systems and supply chains, and workflow scheduling problems are given. The chapters address such subjects as insertion heuristics for energy-efficient scheduling, the re-scheduling of train traffic in real time, control algorithms for short-term scheduling in manufacturing systems, bi-objective optimization of tortilla production, scheduling problems with uncertain (interval) processing times, workflow scheduling for digital signal processor (DSP) clusters, and many more