2003-2004 > Academic research: refereed > Publication in refereed journalVersion of RecordPublishe

Chan, K

Leung, HKN

Li, MS

Liang, S

Journal of Computer Research and Development

PolyU Institutional Repository

? 40 ? ? 7 ?2003? 7?? ? ? ? ? ? ? ?JOURNAL OF COMPUTER RESEARCH AND DEVELOPMEN TVol.40 , No.7July 2003　????:2002-02-04;????:2003-01-17　????:????????(60273026);??“???”???????????(2001AA113191)　???????Web??????????梁　晟1 　李明树1 　梁金能2 　陈振冲21(??????????　??　100080)2(????????????　????)(ls@intec.iscas.ac.cn)?　?　???Web ??????????? , ?????????????????.??? Internet? Web ??????? , Web ????????????? Client/Server???.??? 3 ???? Web ??????;?????????? 、?????———?????????????.??????:???????????????????????????? 、??????;???????????????;???????????????????;???????????.?????????? ,???????????.???　Web ????;????;????;??;????;??????????　TP302A Simulation-Driven Performance Testing Method for Web ApplicationsLIANG Sheng1 , LI Ming-Shu1 , LIANG Kam-Nang2 , and CHEN Chun-Chung21(I nstitute of Sof tware , Chinese Academy of Sciences , Beijing 100080)2(Department of Computing , Hong Kong Polytechnic University , Kow loon , Hong Kong)Abstract　Performance is a crucial factor to the success of a Web application , and performance testing is animportant w ay of ensuring this quality.However , due to many uncertainties of Internet and Web users ,performance testing of Web applications is mo re dif ficult than traditional client/server testing.Three per-formance testing methods fo r Web are compared.A practical , simulation-driven automated load testingmethod is presented w ith an example.The key steps are:① identify the test ing workload and design testcases according to sy stem usage pat terns and dif ferent client characteristics , ②use sui table tools to developtest scripts , ③ test the application wi th predefined wo rkload to simulate the behaviors of real users , and ④collect performance data.A template of the performance testing plan is also provided.Key words　Web application;performance testing;load test ing;workload;test script;test scenario1　?　　????Web????????????? ,?????? ,??????????.??? Zona??????[ 1]?????:?????????1s ,??????? 30%??? 6%～ 8%;?????? ,?? 34%????????????????? ,???? 21%????????????.??? , ????????????????43.5??? ,????? 15%[ 2].?? ,??? Web ??????????? ,?????????????.???????????????????[ 3].????????????????? ,?????? ,?????? ,??????????????????????.?? ,?? Internet ? Web ??????? ,Web???????????:(1)???????????????????????????[ 4].?????? ,??????????? ,????????????????.?? Client/Server????? LAN/WAN ?? ,???????? 、???? ,????????????????????.?Web???????? Internet?? ,?????????? 、???? 、????? ,???????????????????.?? ,??Web??????????????????.(2)???????????????????????? ,????????????.???????????????????.?? ,????? Web???????? ,????????????????????? ,????????????????? ,????????????;?????????????????? ,???????????? ,????????? ,????????????? ,?????????[ 5].???????????????????????.(3)????????????Web??????????? Internet ,???????? LAN?? , Internet???????????? ,????????????????.?????? ,????????????????.???????????????????? ,?? ,???????????????????? 、???????????????? IP ????.2　????????????????? ,????? 3?Web ??????:??????[ 6 ,7] 、WUS ??[ 8]???????[ 9].??????? 1??:? 1　??Web???????????? ?????? WUS?? ????????????? ????????? ???????? ?????????????? (???? 1 , 2 , … , n) (???? 1 , 2 , … , n) (???? 1 , 2 , … , n)????????? ??????? ?????? —??????? ???????? ????????? ???????????? ???? ,????? ?? 、??????? ?????????? ??????? ????????? ??????　　(1)???????????????????????(appli-cation under test , AU T)???? ,??? AU T ?????? ,???????? 、????????.???????“????”(????????????????????)??????????? ,?“????”(???????????)???????.???????????? 、????????????????????????????.????????????? ,????????????????????????AU T ,???????? IP ?? 、???????????????????? ,????????????? ,????????????.??????????? ,???? ,?????????????;??????????????? ,?????.(2)WUS ????“??????(website usage signature ,WUS)”?????????? ,?????????.WUS ???????????????????????? ,??????????????????????? ,???????????/ ??? 、???????? 、????????????/????????????.???????????????.????????????????? ,??????????? 、?????????????????.?????????????????.????? ,???????WUS???????WUS ????? ,????1070 ? ? ? ? ? ? ? ? 2003 ????.??????????????????????? ,???????????????.??????????? ,????????????.(3)???????????? AUT ????????????.??????? 、???? 、??? 、?? 、?? 、?????? 、???.??????????????????.?? Web??????????? ,????????????????????????? ,????????????????.?????? AU T ????? ,????????? 、????? 、???? 、??????????? 、?????????;??????????????????????????.? 1　?????????????3　???????????????Web????????????? ,????????????? ,????????????????? ,?????????? 、?????????———?????????????.??????:?????????????????????????? 、?????? ,???????????????? ,?????????????? ,??? AUT ???.?????????:?????? 、?????? 、?????? 、?????? 、?????? 、?????? 、???????????????.????? 1??.?????????????? ,???????? ———?????????.??????????????:????? 、?? 、?? 、?? 、????? 、???????.3.1　??????????????????.??????????????? 、??????????????????[ 10] ,????????????????????.????????????? WUS??????????? ,?????????????????????:????? 、???? 、???? 、??????????????????????[ 7 , 8].(1)?????:????????????????????????.? 2 ??????????????????? ,?????????????????????????? ,??????/??.????????????????.? 2　?????????????????????????????(?????????)4∶00～6∶006∶00～8∶008∶00～10∶0010∶00～12∶0012∶00～14∶0014∶00～16∶0016∶00～18∶00????? 500 400 800 1200 1000 500 600?? 800 320 1050 2500 950 550 370?? 1000 8000 12000 31200 26000 12000 8000?? 200 1600 8800 16800 13500 3000 2000????? 100 800 1200 3450 2000 3500 2400???? 110 300 570 600 480 510 170?? 100 230 760 1050 660 240 150(2)????:???????????????????? 、????Web????????????? 、????????????.???? 3:? 3　????????????? ??? ??? Web Server?? DB?? ??????? 750 1200 ? ? ??? 1050 2500 ? ? ??? 15600 31200 ? ? ??? 12000 16800 ? ? ?????? 2100 3450 ? ? ????? 200 600 ? ? ??? 800 1050 ? ? ?10717 ? ?　??:???????Web ??????????　　(3)????:?????? 、??? 、???????? 、????????.???? 4.? 4　??????(???)??? ????? ??? ????????/? 600 150 250????????/min 10 30 20?????/(?/??) 1.5 2 —??/(?/??) 2 2 4??/(?/??) 36 14 30??/(?/??) 18 — 24?????/(?/??) — 8 9????/(?/??) — 4 —??/(?/??) — 2 3(4)???????????:??????????? ,???????? 、??????? 、??????????.???? 5 、? 6:? 5　???????????????? ????/ % ??/ s ?????????? 25 8? 50 16? 25 24???????????????????? ,???????? 6　?????????????????? ????/ % ?????? ??????????? 35 2.0?? 55 1.0?? 10 0.5???????????????????????????????(5)???????????:??????? 、???????????.? 7 、? 8?????????????????????:? 7　??????????????? ??/ % ???? ??/ %56Kbps Modem 20 1.544M bps T1/ LAN 5112Kbps Dual ISDN 5 10Mbps LAN 25256Kbps DSL/ Cable 5 45Mbps T3/ LAN 15512Kbps DSL/ Cable 15 ?? 10? 8　????????????????? ??/%Netscape 25Internet Explorer 50?? 25　　?????? AU T ???????????????? 、?????????????????????? ,???????????????????.3.2　??????????????? ,???????????????????????? ,???????? 、???? 、???? 、???? 、?????????? ,?????????.?? IEEE????????(IEEE Std 829-1998)[ 11] ,???????????????????????????(??? A),????????????????.3.3　????????????????????? AU T ??????????? ,??????????:(1)???????????????????;(2)?????????????????????;(3)?????????????????.?????? ,??????????????? 3???:????????????;??????;???????????.3.3.1　???????????????????????????? ,??????????????????????.???????????? 、??? ,????????80%??????? ,????????????5 ～ 10???.??????????????? 、?Web?????????????????????????.??? 3 ,??????????????? 5???:?? 、?? 、????? 、?????????????? ,?? 9?? ,?????????????.? 9　???????????????? ??????????Web Server??DB?? ??????? 750 1200 ? ? ??? 1050 2500 ? ? ??? 15600 31200 ? ? ??? 12000 16800 ? ? ?????? 2100 3450 ? ? ????? 200 600 ? ? ??? 800 1050 ? ? ?3.3.2　???????????????????????:??????????????????;?????????????? ,??????????;??1072 ? ? ? ? ? ? ? ? 2003 ????????????;????????????????? 、?????????.???????????????????.??????????????????????? AUT ???????.??? 3 ～? 8????????????? 9????????? ,???????????????? ,?? 10 、? 11??.???????? 1000?????????? ,???????????????.?? ,? 4?????????????? 14????? ,???????30min???????? 7????(14×30/60).? 5 ??????? 8 s ????????25%,???? 3???????????????????? 15%,5%? 5%,??? 25%??????.? 10　?????????? ????? ??????/min ??????????/??????? 600 10 ?? 6(36×10/ 60)?? 3(18×10/ 60)??? 150 30 ?? 7(14×30/ 60)????? 4(8×30/ 60)???? 2(4×30/ 60)?? 1(2×30/ 60)??? 250 20 ?? 10(30×20/ 60)?? 8(24×20/ 60)????? 3(9×20/ 60)?? 1(3×20/ 60)? 11　??????(?)????(???)??????????? ???? ???? ???????/ s ??? ??/ s ??? ??/ s ??? ??/ s ????????(60)8 15 3 5 56Kbps 20 IE 3516 30 6 35 512Kbps 15 Nets. 1524 15 12 20 10Mbps 25 ?? 10???(15)8 5 3 3 256Kbps 5 IE 516 5 6 5 1.54Mbps 5 Nets. 524 5 12 7 ?? 5 ?? 5???(25)8 5 3 2 112Kbps 5 IE 1016 15 6 15 45Mbps 15 Nets. 524 5 12 8 ?? 5 ?? 10? 12　??????:?????? ????1.??“????”?? ??????????2.??????????? ?????? xxx???? y ? z? w ?Email xxx@xxx.com3.??“??”? ??????????4.??????????? ???????? ??? ?????????? xxxxxx????? xxx????:3?????5.??“??”? ????????????????3.3.3　???????????????????????? ,???????????? ,????????????????????.??????????????.?12???????????.??????????????????.???????????????? ,?????????(????????????),?????????????????.??????????:????????????;?????????????.???????? ,??????;??????????????? ,????????????? ,???????????.3.4　??????????????????? AUT ?????? ,???????????.???????????????.???????:(1)??????????? 、?????;(2)????????? ,?????? 、????? 、?? 、?? 、??????? AUT ???????????;(3)??????????????????.3.5　?????????????????????[ 12].????????? ,??????????????.???????? 3???:???? 、??????.???????????????????:10737 ? ?　??:???????Web ???????????????? AU T ??????? ,???????????????.?????? ,???????.??????????????????????.????????????? ,??????????????? ,????????????????.????????????????????:???????? 、????? 、???? 、???????????.????????????????????? ,????????????? ,??? 、?????.3.6　??/??/???????? 、?????? 、??????????????.????????? ,????????(? 3 ～ 5 ?????)?????(?20%???)?????(???? 100%????120%).????????? ,???????????????????? ,??????.?????????????????? ,??????????????????????????.??????????????????????????????? ,???????.????????????????? ,????????.??????????????????????????.3.7　??????????? ,???????????? ,??????.????????????? 、???????? 、?????????????????.4　???Web??????????????????? 、????????? 、???????????????.?????? ,?????????????????????? ,?????????????? 、?????———????????????? ,??????????????????????? ,?????????????? ,?????????????????.? ? ? ?1 Zona Research Inc.The economic im pacts of unacceptable web-sitedow nload speed. 1999.h ttp://www.keynote.com/ downloads/w hitepapers/ economic im pact of dow nloadspeed.pdf2 A Rudolf , R Pirker.E-Business testing:User perceptions andperformance issues.In:Proc of the 1st Asia-Pacific Conf on Quali-ty S of tw are.Los Alamitos:IEEE Computer Society Press , 2000.315～ 3233 D A M enasce , Virgilio A F Almedia.Capacity Planning for WebPerformance Met rics , Models &Methods.New Jerseyz:PrenticeHall , 1998.1～ 124 H Q Nguyen.Testing Applicat ions on the Web.New Jersey:John Wiley & Sons , 2001.12～ 265 M D Anderson.The top 13 mistakes in load testing applications.Software Test ing &Quality Engineering.1999 , 1(5):31～ 396 Mercury Interact ive Corp.Load testing to predict Web perfor-mance. 2000.ht tp:// www-svca.mercuryinteractive.com/ pdf/products/ whi tepapers/ load testing w eb perform.pdf7 LoadRunner User' s Guide(Version 6.5).Sunnyvale , CA:Mer-cury Interact ive Corporation , 20008 A Savoia.The science of w ebsite load test ing.2000.ht tp:// www.keynote.com/ downloads/ w hitepapers/ science of loadtest ing.pdf9 B M Subraya , S V Subrahmanya.Object driven performance test-ing in Web applications.In:Proc of the 1st Asia-Pacif ic Conf onQuality S of tw are.Los Alamitos, California:IEEE Computer So-ciety Press , 2000.17～ 2610 D Grossman et al.Performance test ing a large finance application.IEEE Netw ork , 1996 , 13(5):50～ 5411 IEEE Standards Softw are Engineering. IEEE S td 829-1998 ,IEEE S tandard for Software Test Documentation.199912 M Few ster , D Graham. S of tw are Test Automat ion. Boston ,MA:Addison-Wesley , 1999.143～ 167　　?　?　? , 1976 ?? ,????? ,????????????? 、??Web??.　　???　? , 1966 ?? , ??? ,????? , ????????????? 、Internet/Web ????? 、????.　　???　? , 1957 ?? ,?? ,???,??????????? 、???? 、????????? 、????、????.　　??? 　? , 1959 ?? , ?? , ??? ,??????????? 、???? 、????.1074 ? ? ? ? ? ? ? ? 2003 ??? A　Web ????????(1)?　??????????????.?????? 、?? 、???????.(2)??????????????? ,???????.(3)????????????? ,??????.??????? ,????????? ,????????????????????.????????????????????? ,????? A1?? A2.? A1　???? 1:???????? ???? ???? ????????????????????????.?????????????.??????????? , ??????????????.??????????? ??? , ???????????.?????????????.? A2　???? 2:???????? ???? ?????? AUT ??????????????;????????????? , ?????.??????????? ,?????????????????????? ,???????????????????.(4)??????????????????????? ,????? 、????????.?????????? 、??????? 、??????? 、???? 、??????????????.(5)?????????????????.?????????.①??:????????? 、???????.②??:????????????.③????:?????? 、???? 、??? 、?????.(6)?????????????????????? ,??????? 、?????.??? A3.? A3　?????????? ??????　??????:??????;??????;??????;??????.????　??AUT ??????:??? 、???????;??????????.?????? ??????;????????.??????　????:??????/??????;????;????;??????.???????　?? 、?????????;??????????.?? ? (? ? 、Web???)???????????(?? 、Web???);?????? ,?????(?? 、Web???)(7)?????????????????? ,?????????????????????? ,?????????.???A4.? A4　??????? ???? ???? ??? ???? ????(1)???? ???? ???? 1(2)???? ??(1) ?????? 2(3)?????? ??(2) ??????? 1(4)?????? ??(3) ?????? 4(5)?????? ??(4), (6) ?????? 2(6)?????? 、????　??(5)　???(?? 、Web???)???4　(7)?????? ??(6) ?????? 1　　　　　?:?????????????????,?????????????????.　　(8)?????????????????? ,??????????????????.?? ,????????????????? ,????????????????.(9)???????????????? ,??????????.10757 ? ?　??:???????Web ??????????

A simulation-driven performance testing method for Web applications

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository