Abstract-Managing workload for large scale web applications is a fundamental task for satisfactory quality of service, low management and operation cost. In this paper, we present SCOPS, a system of distributed workload management to achieve service differentiation and overload protection in such large scale deployment. Our system splits the workload management logic into distributed components on each back-end server and frontend proxy. The control solution is designed to protect the backend server from overloading and to achieve both efficient usage of system resource and service differentiation by employing a unique optimization target. The control components are automatically organized based on the flow of workloads, such that management overhead is minimized. SCOPS is extremely flexible because it requires no source code changes to host OS, application servers, or web applications. Additionally, the distributed design makes it scalable and robust for cloud scale server deployment. Experiments with our implementation confirm SCOPS's performance with dynamic heavy workload, incurring neglectable runtime overhead. More importantly, SCOPS also ensures fault-tolerance and fast convergence to system failures
