Li, Dahui

Li, Zhangxi

Stahl, Dale

Whinston, Andrew

English

AIS Electronic Library (AISeL)

Association for Information SystemsAIS Electronic Library (AISeL)AMCIS 2001 Proceedings Americas Conference on Information Systems(AMCIS)December 2001Bridging Agent-based Simulations and DirectExperiments: An Experimental System for InternetTraffic PricingDahui LiTexas Tech UniversityZhangxi LiTexas Tech UniversityDale StahlThe University of Texas at AustinAndrew WhinstonThe University of Texas at AustinFollow this and additional works at: http://aisel.aisnet.org/amcis2001This material is brought to you by the Americas Conference on Information Systems (AMCIS) at AIS Electronic Library (AISeL). It has been acceptedfor inclusion in AMCIS 2001 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contactelibrary@aisnet.org.Recommended CitationLi, Dahui; Li, Zhangxi; Stahl, Dale; and Whinston, Andrew, "Bridging Agent-based Simulations and Direct Experiments: AnExperimental System for Internet Traffic Pricing" (2001). AMCIS 2001 Proceedings. 139.http://aisel.aisnet.org/amcis2001/1391This research has been sponsored by Intel Corp., Lucent Technologies Inc., and Mannesman Ag.720 2001  Seventh Americas Conference on Information SystemsBRIDGING AGENT-BASED SIMULATIONS AND DIRECTEXPERIMENTS: AN EXPERIMENTAL SYSTEM FORINTERNET TRAFFIC PRICING1Dahui LiTexas Tech Universitydali@ttacs.ttu.eduZhangxi LinTexas Tech Universityzlin@ba.ttu.eduDale O. StahlThe University of Texas at Austinstahl@eco.utexas.eduAndrew B. WhinstonThe University of Texas at Austinabw@uts.cc.utexas.eduAbstractA network-based experimental system for Internet traffic pricing is introduced. The system can be applied tothe simulations using computer agents as well as to the direct experiments  that employ human subjects. Thefeature of replaying the exogenous traffic patterns obtained from direct experiments allows a close comparisonof the performance between the two types of experiments, leading to the new findings reported briefly in thispaper.IntroductionEvolved from traditional computer-based simulation, agent-based economic experiments are getting popular in recent years(Tesfatsion, 2000). However, the absence of behavioral and psychological factors from humans in an artificial world is still themain weakness in the creditability of the data obtained from simulations using perfect and rational agents. Evidence hasdemonstrated “that human behavior deviates in systematic ways from the idealized behavior attributed to expected utilitymaximizers in particular, and to ‘rational economic man’ in general” (Roth, 1996). A direct experiment using human subjects iscloser to the real world, but may introduce errors caused by subjects’ misbehaviors. So, filtering out noises resulted from theseirrational behaviors remains the key issue in obtaining satisfactory results. To fill up the gap between simulation and directexperiment Sterman (1987) conducted a direct experiment to justify the decision rules used in game simulations. As a result ofthe high availability of Internet technology since 1990s, the marriage of agent-based approach and human subject based approachhas imposed more challenges to experimental economists. This paper is to report a recent effort  in network traffic pricing research relevant to the above experimental methodology. Onestrand of the research in network traffic-pricing since 1990s is GSW network traffic pricing model, which has been tested witha simulation system (Gupta et al. 1997). Lin et al. (1999) reported a prototype network traffic-pricing system on a networkplatform to carry out experiments with real-time data traffic generated by software agents on client computers. Even though thisagent-based experimental system produced the same outcomes as those from GSW’s simulation program, the traffic-pricingscheme is still questionable because the feasibility in implementing a traffic-pricing system for real users is not proved. This paperintroduces an experimental system that can be used to conduct either human subject based experiments or agent-based simulations,or a mixed type – human subject working with computer agents. Specifically, the system is capable of replaying the exogenoustraffic patterns generated by direct experiments in the agent-based mode. This allows a comparative study of user performancebeyond previous focus solely on traffic-pricing efficiency.Li et al./An Experimental System for Internet Traffic Pricing2001  Seventh Americas Conference on Information Systems 721Web ServicesVirtual BandwidthServerTraffic Load GeneratorSystemStatus LogsTraffic Pattern logsPriceTraffic StatusData flowsData flowsTraffic patternsData flowsNetwork UsersBandwidthServicesPrice Traffic StatusFigure 1. The Structure of the Experimental SystemExperimental System DesignThere are four modules in the experimental system (Figure1): 1) a virtual bandwidth server allocating bandwidth witha round-robin scheduling method for service requests; 2) aset of user-oriented applications operating on the webbrowser to generate service requests and display networkstatus for request submission decisions; 3) a web-basedapplication module as an intermediary between users andthe virtual bandwidth server; and 4) a traffic load generatorwhich is an agent for generating or regenerating networkdata flows.The experimental system is a distributed system running ona network platform. All data traffic and bandwidth services for experiments are real-time. The interaction between users and thebandwidth pricing system is a recursive three-step process. First, the virtual bandwidth server decides a bandwidth price accordingto current network traffic status and the average cost of service delay to users. Second, a user estimates his/her request’s value,the service amount and the effect of service quality on the request. Third, the user uses the expected service delay and priceinformation disseminated by the virtual bandwidth server to calculate the net value of the request, i.e. his/her expected utility ofthe service, and submit the request if the expected utility is positive.There are two kinds of service request rates in the experiment: 1) request generation rate, the exogenous rate representing thepotential maximum service demand; 2) request arrival rate, the rate  that reflects the load onto the virtual bandwidth server. Indirect experiments, request generation patterns are controlled by a JavaScript program at the client side. It displays a list ofpotential service requests on the screen periodically in a fixed length of time, e.g. every 8 seconds. The number of requestsdisplayed each time is randomly changed in accordance with the exponential distribution. Each requests is marked with a volume,a gross value, a delay cost, and a net value range. In addition, general network status information, such as the expected unit servicetime and the service price, is displayed. Subjects normally check the net value range to make submission decisions. At present, both human subject based and agent-based experiments with or without network traffic pricing have been conductedusing the same bandwidth capacity preset as a T1 line (1.544 Mbps). The pricing formula derived by Lin (1999) with round-robinbandwidth scheduling is adopted.Implications of Experiment OutcomesRequest Arrival Pattern in Direct ExperimentsThe critical issue in experiments is the stochastic properties of service request arrivals, which have been assumed to be a Poissonprocess. In agent-based experiments an algorithm has been implemented to produce service request arrivals in an approximatePoisson process (Lin 1999). The major concern in the direct experiment is whether the traffic patterns generated by humansubjects also comply with Poisson process. The histogram of request arrival intervals from a typical sample data set recorded ina direct experiment shows that request arrival intervals generated by the experimental system are approximately exponentiallydistributed (Figure 2). This has been further confirmed by an OLS regression. The low value of R2 indicates the insignificanceof correlation between  consecutive intervals, which are evidently “memoryless”. Pricing EffectivenessGupta (1997) and Lin et al. (1999) have proved that traffic pricing can effectively improve total welfare of network bandwidthservice using either first-in-first-out or round-robin bandwidth scheduling method. Current experimental system also has provedthis. Figure 3 shows the outcomes from different setups of experiments. Curves Agent_Pricing, and Agent_Non_P are the welfarerates - total service benefits per second - from experiments using the agent-based traffic generation. The curves indicate thatdynamic pricing improves bandwidth service welfare. This is the same result as the one from previous experiments. Similarly,welfare rates, in regard with different levels of traffic arrival rates, obtained from direct experiments using pricing (curveHuman_Pricing) are relatively higher than the welfare rates from direct experiments without pricing (curve Human_Non_P). Thisstrongly suggests the consistency between direct experiments and agent-based experiments. It is noticeable that direct experimentsshow relatively lower welfare rates than the welfare rates from the same schemes as conducted in agent-based experiment.Electronic Commerce722 2001  Seventh Americas Conference on Information Systems020406080100120140160Request Arrival Interval Ranges0123456Request Arrival Interval Ranges0200400600800100012000 2000 4000 6000 8000 10000 12000Exogenous Traffic Rate (Kbpd)Agent_PricingAgent_Non_pHuman_PricingHuman_Non_PFigure 3.  Welfare Rates(Bandwidth: 1544 Kbps)Nevertheless, curve Human_Pricing is almost parallel to Agent_Pricing, and Human_Non_P is also almost parallel toAgent_Non_P, indicating the compatibility of the outcomes from two types of experiments. Figure 2. The Histogram of Request Submission Intervals Generated by Human SubjectsSubject Performance ComparisonWe have mentioned above that the welfare rate observed from directexperiments is lower than that from agent-based experiments. Twopossible reasons may cause this: 1) the human errors and occasionalirrationality, and 2) the differences of setups between two types ofexperiments. To investigate the issue, a group of simulations aredesigned to replay the exogenous traffic patterns recorded from directexperiments. The simulations reproduce the same exogenous trafficgeneration processes as previously generated from direct experiments.The difference is in that the experimental system with the replayingsetup makes optimum submission decisions to maximize the expecteduser utility for every request. The service welfare rates from theseexperiments are as good as those from experiments using agent-generated network traffic, and are better than those from directexperiments producing these exogenous traffic patterns (see Table 1).The last column of the table shows that the welfare rate ratio between a direct experiment and the agent-based experimentreplaying its traffic pattern, both using traffic pricing scheme, is about 86%. The same ratio for the experiments without usingpricing is higher, being 93.3% in a sample case.Table 1.  Welfare Rate ComparisonSchemesExog. TrafficRate (kbps)Welfare Rate($/sec)(Direct)Welfare Rate($/sec) (Replay)Welfare Rate Ratio(Direct / Replay)Dataset 1 (Pricing) 10601 948 1098 86.4%Dataset 2 (Pricing) 7750 763 893 85.4%Dataset 3 (Non_pricing) 4695 361 387 93.3%In checking the size distribution of data flows that are incurred by submitted requests it is impressive that human subjectssometimes tend to make “regrettable” request submission decisions:  submitting a request having a positive expected net valuebut realizing a negative outcome, or dropping a request because of its negative expected net value which would be actuallypositive if the request were serviced. Each individual subject’s performance, in sense of the surplus ratio in two types ofexperiments, varies from as high as 91.4% to as low as 71.3%, indicating the existence of a subjective factor affecting subjects’performance. In another aspect, as a consequence of the imprecise information available to human subjects and more preciseinformation for computer agents to make decisions, human subjects have a higher average surplus forecasting error with a muchLi et al./An Experimental System for Internet Traffic Pricing2001  Seventh Americas Conference on Information Systems 723higher standard deviation of the error rate, where error rate is the ratio between the error of the estimated user surplus and theaverage user surplus.Remarks and Further Research WorkImplementation of the experimental system that allows comparison of performance between human subjects and computer agentsprovides a good means for us to move the research closer to a real world. It becomes possible to bridge the gap between theeconomic approach and behavioral approach particularly in the network traffic-pricing area. More experiments are to be conductedto confirm current findings. Issues such as the effect of user interface design on the implementation of network pricing systemwill be explored. The idea in such an experimental system is now being applied to other research projects.ReferencesGupta, A., Stahl, D. O., and Whinston, A. B. “A Stochastic Equilibrium Model of Internet Pricing,” Journal of EconomicDynamics and Control (21), 1997, pp.697-722. Lin, Z. “Virtual Private Network Traffic Pricing – An Exploratory Study,” 1999, PhD Dissertation.Lin, Z., Ow, P. S., Stahl, D. O., and Whinston, A. B. “Exploring Traffic Pricing for the Virtual Private Network,” Proceedingsof WITS 1999.Paxson, V., and Floyd, S. “Wide-Area Traffic: The Failure of Poisson Modeling,” IEEE/ACM Transactions on Networking (3:3),1995.Roth, A.E. “Comments on Tversky’s ‘Rational Theory and Constructive Choice’,” The Rational Foundations of EconomicBehavior, K. Arrow, E. Colombatto, M Perlman, and C. Schmidt, editors, Macmillan, 1996, pp.198-202. Sterman, J. “Testing Behavioral Simulation Models by Direct Experiment,” Management Science (33:12), 1987, pp.1572-1592.Tesfatsion, L. “Agent-Based Computational Economics: A Brief Guide to the Literature,” January, 2000.http://www.econ.iastate.edu/tesfatsi/aceintro.pdf.

Bridging Agent-based Simulations and Direct Experiments: An Experimental System for Internet Traffic Pricing

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1578&amp;context=amcis2001

Bridging Agent-based Simulations and Direct Experiments: An Experimental System for Internet Traffic Pricing

Abstract

Similar works

Full text

Available Versions

AIS Electronic Library (AISeL)