On the time scales in video traffic characterization for queueing behavior

To guarantee quality of service (QoS) in future integrated service networks, traffic sources must be characterized to capture the traffic characteristics relevant to network performance. Recent studies reveal that multimedia traffic shows burstiness over multiple time scales and long range dependence (LRD). While researchers agree on the importance of traffic correlation there is no agreement on how much correlation should be incorporated into a traffic model for performance estimation and dimensioning of networks. In this article, we present an approach for defining a relevant time scale for the characterization of VER video traffic in the sense of queueing delay. We first consider the Reich formula and characterize traffic by the Piecewise Linear Arrival Envelope Function (PLAEF). We then define the cutoff interval above which the correlation does not affect the queue buildup. The cutoff interval is the upper bound of the time scale which is required for the estimation of queue size and thus the characterization of VER video traffic. We also give a procedure to approximate the empirical PLAEF with a concave function; this significantly simplifies the calculation in the estimation of the cutoff interval and delay bound with little estimation loss. We quantify the relationship between the time scale in the correlation of video traffic and the queue buildup using a set of experiments with traces of MPEG/JPEG-compressed video. We show that the critical interval i.e. the range for the correlation relevant to the queueing delay, depends on the traffic load: as the traffic load increases, the range of the time scale required for estimation for queueing delay also increases. These results offer further insights into the implication of LRD in VER video traffic. (C) 1999 Elsevier Science B.V. Ail rights reserved

GPS queues with heterogeneous traffic classes

We consider a queue fed by a mixture of light-tailed and heavy-tailed traffic. The two traffic classes are served in accordance with the generalized processor sharing (GPS) discipline. GPS-based scheduling algorithms, such as weighted fair queueing (WFQ), have emerged as an important mechanism for achieving service differentiation in integrated networks. We derive the asymptotic workload behavior of the light-tailed class for the situation where its GPS weight is larger than its traffic intensity. The GPS mechanism ensures that the workload is bounded above by that in an isolated system with the light-tailed class served in isolation at a constant rate equal to its GPS weight. We show that the workload distribution is in fact asymptotically equivalent to that in the isolated system, multiplied with a certain pre-factor, which accounts for the interaction with the heavy-tailed class. Specifically, the pre-factor represents the probability that the heavy-tailed class is backlogged long enough for the light-tailed class to reach overflow. The results provide crucial qualitative insight in the typical overflow scenario

Comprehensive characterization of an open source document search engine

This work performs a thorough characterization and analysis of the open source Lucene search library. The article describes in detail the architecture, functionality, and micro-architectural behavior of the search engine, and investigates prominent online document search research issues. In particular, we study how intra-server index partitioning affects the response time and throughput, explore the potential use of low power servers for document search, and examine the sources of performance degradation ands the causes of tail latencies. Some of our main conclusions are the following: (a) intra-server index partitioning can reduce tail latencies but with diminishing benefits as incoming query traffic increases, (b) low power servers given enough partitioning can provide same average and tail response times as conventional high performance servers, (c) index search is a CPU-intensive cache-friendly application, and (d) C-states are the main culprits for performance degradation in document search.Web of Science162art. no. 1

Towards a System Theoretic Approach to Wireless Network Capacity in Finite Time and Space

In asymptotic regimes, both in time and space (network size), the derivation of network capacity results is grossly simplified by brushing aside queueing behavior in non-Jackson networks. This simplifying double-limit model, however, lends itself to conservative numerical results in finite regimes. To properly account for queueing behavior beyond a simple calculus based on average rates, we advocate a system theoretic methodology for the capacity problem in finite time and space regimes. This methodology also accounts for spatial correlations arising in networks with CSMA/CA scheduling and it delivers rigorous closed-form capacity results in terms of probability distributions. Unlike numerous existing asymptotic results, subject to anecdotal practical concerns, our transient one can be used in practical settings: for example, to compute the time scales at which multi-hop routing is more advantageous than single-hop routing

On the nature and impact of self-similarity in real-time systems

In real-time systems with highly variable task execution times simplistic task models are insufficient to accurately model and to analyze the system. Variability can be tackled using distributions rather than a single value, but the proper charac- terization depends on the degree of variability. Self-similarity is one of the deep- est kinds of variability. It characterizes the fact that a workload is not only highly variable, but it is also bursty on many time-scales. This paper identifies in which situations this source of indeterminism can appear in a real-time system: the com- bination of variability in task inter-arrival times and execution times. Although self- similarity is not a claim for all systems with variable execution times, it is not unusual in some applications with real-time requirements, like video processing, networking and gaming. The paper shows how to properly model and to analyze self-similar task sets and how improper modeling can mask deadline misses. The paper derives an analyti- cal expression for the dependence of the deadline miss ratio on the degree of self- similarity and proofs its negative impact on real-time systems performance through system¿s modeling and simulation. This study about the nature and impact of self- similarity on soft real-time systems can help to reduce its effects, to choose the proper scheduling policies, and to avoid its causes at system design time.This work was developed under a grant from the European Union (FRESCOR-FP6/2005/IST/5-03402).Enrique Hernández-Orallo; Vila Carbó, JA. (2012). On the nature and impact of self-similarity in real-time systems. Real-Time Systems. 48(3):294-319. doi:10.1007/s11241-012-9146-0S294319483Abdelzaher TF, Sharma V, Lu C (2004) A utilization bound for aperiodic tasks and priority driven scheduling. IEEE Trans Comput 53(3):334–350Abeni L, Buttazzo G (1999) QoS guarantee using probabilistic deadlines. In: Proc of the Euromicro confererence on real-time systemsAbeni L, Buttazzo G (2004) Resource reservation in dynamic real-time systems. Real-Time Syst 37(2):123–167Anantharam V (1999) Scheduling strategies and long-range dependence. Queueing Syst 33(1–3):73–89Beran J (1994) Statistics for long-memory processes. Chapman and Hall, LondonBeran J, Sherman R, Taqqu M, Willinger W (1995) Long-range dependence in variable-bit-rate video traffic. IEEE Trans Commun 43(2):1566–1579Boxma O, Zwart B (2007) Tails in scheduling. SIGMETRICS Perform Eval Rev 34(4):13–20Brichet F, Roberts J, Simonian A, Veitch D (1996) Heavy traffic analysis of a storage model with long range dependent on/off sources. Queueing Syst 23(1):197–215Crovella M, Bestavros A (1997) Self-similarity in world wide web traffic: evidence and possible causes. IEEE/ACM Trans Netw 5(6):835–846Dìaz J, Garcìa D, Kim K, Lee C, Bello LL, López J, Min LS, Mirabella O (2002) Stochastic analysis of periodic real-time systems. In: Proc of the 23rd IEEE real-time systems symposium, pp 289–300Erramilli A, Narayan O, Willinger W (1996) Experimental queueing analysis with long-range dependent packet traffic. IEEE/ACM Trans Netw 4(2):209–223Erramilli A, Roughan M, Veitch D, Willinger W (2002) Self-similar traffic and network dynamics. Proc IEEE 90(5):800–819Gardner M (1999) Probabilistic analysis and scheduling of critical soft real-time systems. Phd thesis, University of Illinois, Urbana-ChampaignGarrett MW, Willinger W (1994) Analysis, modeling and generation of self-similar vbr video traffic. In: ACM SIGCOMMHarchol-Balter M (2002) Task assignment with unknown duration. J ACM 49(2):260–288Harchol-Balter M (2007) Foreword: Special issue on new perspective in scheduling. SIGMETRICS Perform Eval Rev 34(4):2–3Harchol-Balter M, Downey AB (1997) Exploiting process lifetime distributions for dynamic load balancing. ACM Trans Comput Syst 15(3):253–285Hernandez-Orallo E, Vila-Carbo J (2007) Network performance analysis based on histogram workload models. In: Proceedings of the 15th international symposium on modeling, analysis, and simulation of computer and telecommunication systems (MASCOTS), pp 331–336Hernandez-Orallo E, Vila-Carbo J (2010) Analysis of self-similar workload on real-time systems. In: IEEE real-time and embedded technology and applications symposium (RTAS). IEEE Computer Society, Washington, pp 343–352Hernández-Orallo E, Vila-Carbó J (2010) Network queue and loss analysis using histogram-based traffic models. Comput Commun 33(2):190–201Hughes CJ, Kaul P, Adve SV, Jain R, Park C, Srinivasan J (2001) Variability in the execution of multimedia applications and implications for architecture. SIGARCH Comput Archit News 29(2):254–265Leland W, Ott TJ (1986) Load-balancing heuristics and process behavior. SIGMETRICS Perform Eval Rev 14(1):54–69Leland WE, Taqqu MS, Willinger W, Wilson DV (1994) On the self-similar nature of ethernet traffic (extended version). IEEE/ACM Trans Netw 2(1):1–15Liu CL, Layland JW (1973) Scheduling algorithms for multiprogramming in a hard-real-time environment. J ACM 20(1):46–61Mandelbrot B (1965) Self-similar error clusters in communication systems and the concept of conditional stationarity. IEEE Trans Commun 13(1):71–90Mandelbrot BB (1969) Long run linearity, locally Gaussian processes, h-spectra and infinite variances. Int Econ Rev 10:82–113Norros I (1994) A storage model with self-similar input. Queueing Syst 16(3):387–396Norros I (2000) Queueing behavior under fractional Brownian traffic. In: Park K, Willinger W (eds) Self-similar network traffic and performance evaluation. Willey, New York, Chap 4Park K, Willinger W (2000) Self-similar network traffic: An overview. In: Park K, Willinger W (eds) Self-similar network traffic and performance evaluation. Willey, New York, Chap 1Paxson V, Floyd S (1995) Wide area traffic: the failure of Poisson modeling. IEEE/ACM Trans Netw 3(3):226–244Rolls DA, Michailidis G, Hernández-Campos F (2005) Queueing analysis of network traffic: methodology and visualization tools. Comput Netw 48(3):447–473Rose O (1995) Statistical properties of mpeg video traffic and their impact on traffic modeling in atm systems. In: Conference on local computer networksRoy N, Hamm N, Madhukar M, Schmidt DC, Dowdy L (2009) The impact of variability on soft real-time system scheduling. In: RTCSA ’09: Proceedings of the 2009 15th IEEE international conference on embedded and real-time computing systems and applications. IEEE Computer Society, Washington, pp 527–532Sha L, Abdelzaher T, Årzén KE, Cervin A, Baker T, Burns A, Buttazzo G, Caccamo M, Lehoczky J, Mok AK (2004) Real time scheduling theory: A historical perspective. Real-Time Syst 28(2):101–155Taqqu MS, Willinger W, Sherman R (1997) Proof of a fundamental result in self-similar traffic modeling. SIGCOMM Comput Commun Rev 27(2):5–23Tia T, Deng Z, Shankar M, Storch M, Sun J, Wu L, Liu J (1995) Probabilistic performance guarantee for real-time tasks with varying computation times. In: Proc of the real-time technology and applications symposium, pp 164–173Vila-Carbó J, Hernández-Orallo E (2008) An analysis method for variable execution time tasks based on histograms. Real-Time Syst 38(1):1–37Willinger W, Taqqu M, Erramilli A (1996) A bibliographical guide to self-similar traffic and performance modeling for modern high-speed networks. In: Stochastic networks: Theory and applications, pp 339–366Willinger W, Taqqu MS, Sherman R, Wilson DV (1997) Self-similarity through high-variability: statistical analysis of ethernet lan traffic at the source level. IEEE/ACM Trans Netw 5(1):71–8

Exact asymptotics for fluid queues fed by multiple heavy-tailed on-off flows

We consider a fluid queue fed by multiple On-Off flows with heavy-tailed (regularly varying) On periods. Under fairly mild assumptions, we prove that the workload distribution is asymptotically equivalent to that in a reduced system. The reduced system consists of a ``dominant'' subset of the flows, with the original service rate subtracted by the mean rate of the other flows. We describe how a dominant set may be determined from a simple knapsack formulation. The dominant set consists of a ``minimally critical'' set of On-Off flows with regularly varying On periods. In case the dominant set contains just a single On-Off flow, the exact asymptotics for the reduced system follow from known results. For the case of several On-Off flows, we exploit a powerful intuitive argument to obtain the exact asymptotics. Combined with the reduced-load equivalence, the results for the reduced system provide a characterization of the tail of the workload distribution for a wide range of traffic scenarios

Performance enhancement of large scale networks with heterogeneous traffic.

Finally, these findings are applied towards improving the performance of the Differentiated Services architecture by developing a new Refined Assured Forwarding framework where heterogeneous traffic flows share the same aggregate class. The new framework requires minimal modification to the existing Diffserv routers. The efficiency of the new architecture in enhancing the performance of Diffserv is demonstrated by simulation results under different traffic scenarios.This dissertation builds on the notion that segregating traffic with disparate characteristics into separate channels generally results in a better performance. Through a quantitative analysis, it precisely defines the number of classes and the allocation of traffic into these classes that will lead to optimal performance from a latency standpoint. Additionally, it weakens the most generally used assumption of exponential or geometric distribution of traffic service time in the integration versus segregation studies to date by including self-similarity in network traffic.The dissertation also develops a pricing model based on resource usage in a system with segregated channels. Based on analytical results, this dissertation proposes a scheme whereby a service provider can develop compensatory and fair prices for customers with varying QoS requirements under a wide variety of ambient traffic scenarios.This dissertation provides novel techniques for improving the Quality of Service by enhancing the performance of queue management in large scale packet switched networks with a high volume of traffic. Networks combine traffic from multiple sources which have disparate characteristics. Multiplexing such heterogeneous traffic usually results in adverse effects on the overall performance of the network