Abstract. Passage-end calculations are a new style of passage measurement for eXtended Stochastic Probes (XSP) [1] which add the ability to split the analysis into several cases depending on conditions which hold at the end of a passage. This makes it possible to separate successful responses to a request from negative responses, timeouts or other failures. This allows the expression of service level agreements such as: “At least 90 percent of all requests receive a response within 10 seconds and at least 60 percent of all such requests are successful.” 

Clark, Allan

Duguid, Adam

Gilmore, Stephen

Passage-end calculations are a new style of passage measurement for eXtended Stochastic Probes (XSP) [1] which add the ability to split the analysis into several cases depending on conditions which hold at the end of a passage. This makes it possible to separate successful responses to a request from negative responses, timeouts or other failures. This allows the expression of service level agreements such as: "At least 90 percent of all requests receive a response within 10 seconds and at least 60 percent of all such requests are successful.

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Passage-End AnalysisCitation for published version:Clark, A, Duguid, A & Gilmore, S 2009, Passage-End Analysis. in JT Bradley (ed.), Computer PerformanceEngineering: 6th European Performance Engineering Workshop, EPEW 2009 London, UK, July 9-10, 2009Proceedings. Lecture Notes in Computer Science, vol. 5652, Springer-Verlag GmbH, pp. 110-115. DOI:10.1007/978-3-642-02924-0_9Digital Object Identifier (DOI):10.1007/978-3-642-02924-0_9Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Computer Performance EngineeringGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Passage-end AnalysisAllan Clark, Adam Duguid and Stephen GilmoreThe University of Edinburgh, ScotlandAbstract. Passage-end calculations are a new style of passage measure-ment for eXtended Stochastic Probes (XSP) [1] which add the ability tosplit the analysis into several cases depending on conditions which holdat the end of a passage. This makes it possible to separate successful re-sponses to a request from negative responses, timeouts or other failures.This allows the expression of service level agreements such as: “At least90 percent of all requests receive a response within 10 seconds and atleast 60 percent of all such requests are successful.”1 IntroductionMany systems which are analysed for response-time profiles have more thanone way in which the passage in question may terminate. Commonly a requestfrom a client may end in a successful completion such that the client receivedthe desired service but may also end in failure due to a timeout or rejection.There may also be two or more ways in which the request can be satisfied, forexample a data retrieval request may be serviced by accessing the cache or thedisk. In these situations we wish to analyse the passage-time profiles for theseparate cases of success or failure, and cache or disk retrieval. It may be thatthe general response-time analysis indicates acceptable performance but thatsuccessful requests are serviced too slowly.A stochastic probe [2, 3] is a component added to a model in order to makereasoning about the model more convenient. In the context of PEPA [4] astochastic probe is a single sequential component which observes the activitiesof a PEPA model.For example we may analyse the response-time of a service as observed by asingle client. Once this is known, in order to provide more information on howthis may be improved, we then analyse the response-time of all requests whichare made when the service is definitely not broken. We would expect this to bebetter than the more general case of all requests. Conversely we may also analysethe response-time for all requests made when the service certainly is broken andexpect this to be worse than for the general case. From this we may determinewhether, in order to improve response-time, it is better to repair the server morequickly or make the server more reliable such that it breaks less often.Although with eXtended Stochastic Probes splitting the measurement of apassage with respect to the starting conditions is convenient it is not clear howone may split a passage based on how the passage completes — or based onsome event during the passage. The novelty in this paper is the use of severalabsorbing states, one for each target event. For a passage-end calculation theuser must specify a list of target actions. These may be actions performed bythe model itself or communication signals sent by user defined probes. Duringtransient analysis each target state is modified by adding a transition to a distinctabsorbing state based on the causal event.As an example, consider analysing the response-time of a service which be-gins with a request but may be concluded with either a cached response or anetworked response. It is not possible to simply measure these two passages withseparate runs using a probe such as: request :start,cached :stop. The reason is thatthis will compute the probability of completing the passage at (or within) timet via a cached response plus the probability of completing the passage via a net-worked response and then restarting and completing the passage via the cachedresponse all at (or within) time t. Indeed you may complete the passage twice,three times or any number of times via the networked response before finallycompleting via the cached response.Using the same algorithm we can calculate the raw pdf and cdf of the passagefrom the request to the cached response. In this case the cdf will not tend toone but to the fraction of requests which are ultimately serviced by the cache.Similarly for the area under the pdf and for both functions of the request tonetworked passage. We can normalise these graphs based on the probability ofcompleting at or within the given time t via any target event.We can instead normalise the raw pdf and cdf by dividing through the prob-ability of completing at (or within) time t by the percentage of all requestswhich are ultimately serviced by the target event in question. We can know thispercentage by calculating enough hops such that sufficiently close to all of theprobability mass at piN is in one of the absorbing states. We may then take theprobability of being in the appropriate absorbing state at piN .Hence using a passage-end calculation it is possible to calculate:– The probability of completing a passage by a cached response at or withina given time.– The probability that, assuming the passage completes at or within a giventime in some way, that it does so via the cached response. This answers suchquestions as: “What percentage of responses received within 5 seconds arereceived via the cache/network?”– The cdf and pdf profiles for all requests which are serviced by the cache.In the above “cached” may be replaced by “networked” or any other targetevent of the given passage, including probe communication signals. The secondkind of question is helpful in evaluating some SLAs (Service-Level Agreements),particularly for services which may end in success, failure or cancellation. Forexample the SLA may say that ninety percent of requests receive a responsewithin 10 seconds. We may analyse the model and find that this is indeed thecase but a passage-end analysis reveals that eighty-nine percent of such requestsare rejection/failure responses. Hence we may wish to modify our SLA to statethat ninety percent of all successful requests receive a response within 10 seconds.2 Example of Passage-end AnalysisWe consider an example of passage-end analysis applied to emergency responseservice quality [5]. The scenario centres on the function of an automatic crashresponse subscription service such as OnStar [6] from General Motors. Theseutilise multiple built-in sensors which capture critical details in the event ofa car crash. The built-in communications module automatically contacts theOnStar service and relays the information obtained from the sensors, includinglocation information from the on-board GPS. The service attempts to contactthe driver to ask if they need help. If the service cannot contact the driver thenthey send medical assistance to the car’s location.The PEPA model in Figure 1 represents the scenario where the protocolfor attempting to call the driver is to try three times. After three unsuccessfulattempts to contact the driver the system assumes that the driver is unavailableand at this point must decide on the basis of the car telemetry whether todispatch an ambulance. We are interested then in the time between the airbagdeploying and either a successful or failed attempt to contact the driver. Thestart event of our passage is then the airbag activity and the two ways in whicha passage may terminate is with an answer or a timeout3 activity. Note that thetimeout activities are distinguished such that only the third timeout will endthe passage.Car1def= (airbag , r1).Car2Car2def= (reportToService, r2).Car3Car3def= (processReport , r3).Car4Car4def= (callDriver , r4).Car5Car5def= (timeout1, p ∗ r5).Car6+ (answer , (1− p) ∗ r5).OKCar6def= (timeout2, p ∗ r5).Car7+ (answer , (1− p) ∗ r5).OKCar7def= (timeout3, p ∗ r5).Rescue+ (answer , (1− p) ∗ r5).OKOKdef= (finish, r1).Car1Rescuedef= (rescue, r6).Car1 0 0.2 0.4 0.6 0.8 1 0  2  4  6  8  10  12  14 0 0.2 0.4 0.6 0.8 1ProbabilityTimeanswertimeoutanswer-normalisedtimeout-normalisedanswer-cdftimeout-cdfFig. 1. The model of the driver contact protocol within the airbag crash assistancecase scenario. The initial state of this process is Car1. The graph shows results ofpassage-end analysis of driver contact retries.The results from our passage-end analysis of the model are shown in thegraph in Figure 1. The first two lines to consider are the ‘answer’ and ‘timeout’lines, these plot the unmodified cumulative distribution functions of completingthe passage via the driver answering the phone or a third timeout occurringrespectively. Note that neither of these two lines tend to one as is usual for acdf of a passage. This is because the passage may never end in the prescribedways. By looking at the long-term probabilities, or the limits of these two lines(which sum together to one) we can say what percentage of airbag initiations endin the driver answering the phone (and conversely a third timeout occurring).From these two lines we can also answer the question: “What is the probabilitythat the driver answers the phone a given time after the airbag was deployed,regardless of whether the driver is capable of answering at any time”.The next two lines to consider are the above cdf functions normalised bydividing the probability (in each case) at each time by the probability of com-pleting the passage within the given time in any way. These lines tend to thesame value in the limit as before because eventually there is a probability of veryclose to one that we will have completed the passage in some way within thattime. These lines allow us to answer the question: “If the passage is completedwithin a given time t, what is the probability that the passage was completed viathe driver answering the phone” (or conversely by a third timeout occurring).The final two lines to consider are the ‘answer-cdf’ and ‘timeout-cdf’. Herewe have normalised the original two lines by dividing the value at each time bythe percentage of all requests which are eventually serviced in the given way.By doing this we achieve the cumulative distribution function of all requestsinitiated by an airbag deployment which then result in the driver answering thephone or the call timing out for the third time respectively. This can then answerour question: “What percentage of airbag deployments whose driver cannot becontacted are serviced within a given amount of time”. This figure is often moreinteresting than just the question “What percentage of airbag deployments areresolved within a given time in some way” since if the driver is unhurt theresponse-time is of less importance.3 ImplementationPassage-end analysis for XSP is fully implemented in the International PEPACompiler (IPC), a formal analysis tool for steady-state and transient evaluationof PEPA models. The IPC compiler is part of the ipclib [7] suite, a collection oftools for the specification and evaluation of complex performance measures overMarkovian process algebra models. The ipclib suite is an extension of the IPCtool previously used for computing response-time quantiles from PEPA mod-els [8, 9]. The IPC tool has been applied to a number of modelling problems suchas performance of personal-area networks [10] and compiler optimisations [11].Although we have concentrated here mostly on passage-time computation, IPCalso supports the computation of steady-state, transient and counting measuresas described in [12].4 ConclusionsWe have modified passage-time analysis to allow for distinct passage results. Wehave done so within the framework of eXtended Stochastic Probes to ensurethat our passage-end queries remain robust over model modifications and do notrequire that the modeller modify their original model. The set of queries whichcan be specified with XSP is therefore extended.Another bonus which we obtain almost for free is the ability to analyse pas-sages which may never complete at all. This only works for passages in whichthere is only one source state, because if the model deadlocks we cannot anal-yse the embedded Markov chain to obtain the distribution of probability to thesource states at the beginning of the passage. This allows the modeller to providea concrete answer to the question: “How long should I wait for my response?”because we can now say that a given percentage of all requests which ultimatelyare successfully serviced are serviced within 10 seconds, hence if you have waitedlonger than 10 seconds it is likely you will never receive a response and hencecan cancel the request yourself.AcknowledgementsThe authors are supported by the EU FET-IST Global Computing 2 projectSENSORIA (“Software Engineering for Service-Oriented Overlay Computers”(IST-3-016004-IP-09)).References1. Clark, A., Gilmore, S.: State-aware performance analysis with eXtended StochasticProbes. In Thomas, N., Juiz, C., eds.: Proceedings of the 5th European Perfor-mance Engineering Workshop (EPEW 2008). Volume 5261 of LNCS., Palma deMallorca, Spain, Springer (September 2008) 125–1402. Argent-Katwala, A., Bradley, J., Dingle, N.: Expressing performance requirementsusing regular expressions to specify stochastic probes over process algebra models.In: Proceedings of the Fourth International Workshop on Software and Perfor-mance, Redwood Shores, California, USA, ACM Press (January 2004) 49–583. Argent-Katwala, A., Bradley, J., Clark, A., Gilmore, S.: Location-aware qualityof service measurements for service-level agreements. In Barthe, G., Fournet, C.,eds.: Proceedings of the Third International Conference on Trustworthy GlobalComputing (TGC’07). Volume 4912 of LNCS., Springer-Verlag (2008) 222–2394. Hillston, J.: A Compositional Approach to Performance Modelling. CambridgeUniversity Press (1996)5. Clark, A., Gilmore, S.: Evaluating quality of service for service level agreements.In Brim, L., Leucker, M., eds.: Proceedings of the 11th International Workshop onFormal Methods for Industrial Critical Systems, Bonn, Germany (August 2006)172–1856. OnStar: OnStar by General Motors, car safety device and vehicle security system(April 2009) http://www.onstar.com/.7. Clark, A.: The ipclib PEPA Library. In Harchol-Balter, M., Kwiatkowska, M.,Telek, M., eds.: Proceedings of the 4th International Conference on the Quantita-tive Evaluation of SysTems (QEST), IEEE (September 2007) 55–568. Bradley, J., Dingle, N., Gilmore, S., Knottenbelt, W.: Extracting passage timesfrom PEPA models with the HYDRA tool: A case study. In Jarvis, S., ed.: Proceed-ings of the Nineteenth annual UK Performance Engineering Workshop, Universityof Warwick (July 2003) 79–909. Bradley, J., Dingle, N., Gilmore, S., Knottenbelt, W.: Derivation of passage-timedensities in PEPA models using IPC: The Imperial PEPA Compiler. In Kotsis, G.,ed.: Proceedings of the 11th IEEE/ACM International Symposium on Modeling,Analysis and Simulation of Computer and Telecommunications Systems, Universityof Central Florida, IEEE Computer Society Press (October 2003) 344–35110. Ding, J., Hillston, J., Laurenson, D.: Performance modelling of content adapta-tion for a personal distributed environment. Personal Wireless Communication(November 2007) doi 10.1007/s11277-007-9427-4.11. Djoudi, L., Kloul, L.: Assembly code analysis using stochastic process algebra.In Thomas, N., Juiz, C., eds.: Proceedings of the 5th European Performance En-gineering Workshop (EPEW 2008). Volume 5261 of LNCS., Palma de Mallorca,Spain, Springer (September 2008) 95–10912. Argent-Katwala, A., Bradley, J.T.: Functional performance specification withstochastic probes. In Horva´th, A., Telek, M., eds.: Formal Methods and Stochas-tic Models for Performance Evaluation: Third European Performance EngineeringWorkshop (EPEW 2006). Number 4054 in LNCS, Springer-Verlag (June 2006)31–46

Passage-End Analysis

https://www.pure.ed.ac.uk/ws/files/13346372/passageend.pdf

Passage-End Analysis

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer