Probabilistic Application-Level Connection Scheduling in Web Servers

Abstract

A Web server running a popular Web site might simultaneously service hundreds or even thousands of requests. Many traditional Web servers let the operating system decide the connection scheduling—that is, the order in which requests are serviced—and, in fact, most simply do so on a first-come, first-served basis. In many cases, however, it can be extremely useful, if not downright necessary, to have application-level control over scheduling policies. Application-level connection scheduling lets server administrators specify the scheduling rules they want to impose, rather than relying on programmers to do it. Two obvious cases where this might be useful are: • When different users have different priorities. To determine request priority for different user groups, you can use business rules to determine processing order. This lets you provide differentiated quality of service levels to different clients. • When Web server traffic is bursty. For example, the average peak-hour traffic is typically several times the average daily traffic. Rather than overprovision the server, you can turn to differentiated QoS as a more judicious approach to resource use under high-load conditions. Application-level connection scheduling also offers another benefit that is rathe