This paper presents findings of an exploratory case study that was undertaken to assess web performance evaluation in three petroleum energy firms in the Southwest. Findings indicate that although senior management has not yet asked for performance justification, some firms are attempting to develop standard measures for web performance. We explore these measures, and examine the link between developer expertise and performance evaluation

Jones, Mary

Kayworth, Timothy

English

AIS Electronic Library (AISeL)

Association for Information SystemsAIS Electronic Library (AISeL)AMCIS 1999 Proceedings Americas Conference on Information Systems(AMCIS)December 1999Corporate Web Performance Evaluation: AnExploratory AssessmentMary JonesMississippi State UniversityTimothy KayworthBaylor UniversityFollow this and additional works at: http://aisel.aisnet.org/amcis1999This material is brought to you by the Americas Conference on Information Systems (AMCIS) at AIS Electronic Library (AISeL). It has been acceptedfor inclusion in AMCIS 1999 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contactelibrary@aisnet.org.Recommended CitationJones, Mary and Kayworth, Timothy, "Corporate Web Performance Evaluation: An Exploratory Assessment" (1999). AMCIS 1999Proceedings. 88.http://aisel.aisnet.org/amcis1999/88247Corporate Web Performance Evaluation: An Exploratory Assessment Mary C. Jones, Mississippi State University, mjones@cobilan.msstate.eduTimothy C. Kayworth, Baylor University, timothy_kayworth@baylor.eduAbstractThis paper presents findings of an exploratory case studythat was undertaken to assess web performance evaluation inthree petroleum energy firms in the Southwest. Findingsindicate that although senior management has not yet askedfor performance justification, some firms are attempting todevelop standard measures for web performance. We explorethese measures, and examine the link between developerexpertise and performance evaluation.IntroductionCommercial use of the World Wide Web has become oneof the fastest growing electronic commerce applications ofthe decade. Total purchases over the web are estimated togrow to $228billion U.S. dollars by 2001 (Commerce byNumbers, 1999). Web sites offer a variety of potentialbenefits to firms, including reduced transaction,advertising,marketing, and distribution costs, elimination ofthird party intermediaries, reduced time to completetransactions, and the ability to monitor customer choices andgather market intelligence (Hoffman, Novak, and Chatterjee,1995). Other benefits include ability to deliver information tostakeholders faster, offer customized advertising, promotion,and customer service, and operational benefits like reducederrors, time, and overhead costs, easier entry to new markets,and faster time to market (Magid, 1995; Wilder, 1995). Inspite of these potential benefits, only 30% of commercialweb sites worldwide are reported to be profitable (Commerceby Numbers, 1999). Informal discussions with web sitemanagers also indicate that firms do not believe that theirweb sites are as effective as they initially thought they wouldbe. Evidence further suggests that consumers often feel thatweb sites do not meet their needs, and that the presentation ofgoods/services seems vague or intangible to them (Jarvenpaaand Todd, 1996). Yet, few firms have performance measuresin place to regularly monitor how effective the site is. This ispartially because the web is still in an evolving stage, andstandard criteria for such evaluation is limited (Ho, 1997).     This paper presents a discussion of findings about webperformance evaluations gathered as part of a larger casestudy of commercial web site innovations. It addresses thetypes of performance measures firms are using, the extent towhich web developers are held accountable by uppermanagement for web performance, and the relationshipbetween web expertise and developers' self-perceptions ofthe performance of their respective web sites. Theoryindicates that self-appraisals may be valid predictors ofperformance: individuals who view themselves as capable ofperforming tasks tend to do so successfully (Bandura, 1986;Schunk, 1989). Thus, developers who view themselves ashaving high web expertise may view their web sites as beingmore successful than those with less expertise.Data Collection and Profile of Firms       We examined firms in a single industry to help control forcross industry effects that might confound the results of thestudy. Three firms in the petroleum energy industry in theSouthwest were chosen. Although products in this industry donot lend themselves to directs sales over the web, these firmscan and do use web sites to provide information to endconsumers and other external stakeholders and to exchangetransactions with business trading partners. Northco is anintegrated, international petroleum company with assets ofover $13 billion and annual revenues of over $15 billion. It isa major oil and gas producer, and one of the largest producersof gas liquids in the United States. Midco is an energy servicesprovider that operates separately within a larger corporationthat operates other non-energy related businesses worldwide.It provides sales, storage, and distribution of energy productssuch as natural gas, electricity, refined products, andpetroleum liquids. It also provides natural gas services such asgathering, processing, and dehydration. It has assets of over$3.9 billion and annual revenues of $1.5 billion. Southco is aprivately held company engaged in the refining, marketing,and transportation of petroleum products including gasoline,diesel fuel, jet turbine fuel, heating oils, petrochemicals,lubricants, and petroleum-based industrial fuels. It is a whollyowned, separately operated subsidiary of one of the world'slargest oil companies. It owns several refineries that areamong the largest in the U.S., and markets branded gasolinethrough over 14,000 independent retail outlets. It has one ofthe largest shares of the U.S. gasoline market and is among thelargest owner/operators of U.S. bulk petroleum storagefacilities.      On site interviews were administered to two keyrespondents at each of the firms over one week: the personmost responsible for managing the firm's web site content andthe person most responsible for the technical management ofthe site (e.g., webmaster). This allowed a richer understandingof the web process by providing both business and technicalperspectives.Web ExpertiseSouthco and Midco both perceive that their internalbusiness units and external users have low levels of experienceboth using and developing Web site content. However, thedifference between these two companies is that Southco isattempting to train its employees and external stakeholders tohelp them better understand the uses and value of the Web. In248contrast to Southco and Midco, Northco perceives a muchhigher level of internal business unit experience with Website use and development of content. There are severalreasons for this. First, a high percentage of employees haveaccess to the internet since many have computers at homeand many workers were upgraded to high end PCs at workrecently due to the implementation of another technologybased application. Northco's employees appear to be morecomfortable with the Web browsing environment than at theother two firms.      Southco and Midco also appear to have high levels ofinternal Web expertise on their Web development teams.Northco, however, indicates that its web development teamhas a low level of Web expertise. Northco feels it missed asignificant window of opportunity to hire skilled Webexpertise because of a corporate hiring freeze in effect whenthe site was first developed. Furthermore, Northco'swebmaster is the only one who does not come from theinformation systems area. In contrast, both Midco's andSouthco's webmasters are highly experienced with regard tothe web. Midco's webmaster gained Web experience workingat Midco, whereas the content manager had helped develop acorporate Web site for another firm prior to joining Midco. One weakness at Southco is that the content manager hadonly held her position six months at the time of the interview,and had no prior web experience. However, she wasattending meetings inside and outside the firm and readingweb-related material to get knowledge necessary to do herjob.     Both Southco and Northco currently use outsidecompanies to provide them with technical support for theirWeb site. Southco uses an external consulting firm forgraphics design and Web architecture. Northco currently usesan external ISP to host their Web page and is considering theuse of an outside consulting firm to assist with the Webcontent and overall design process. Midco currently uses anadvertising agency for graphics, multimedia, and to monitorconsistency among postings. While this ad agency providessome external support, it is not a technology based firm suchas the one Southco uses. Midco also intends to do more in-house work on their Web site and to move away from usingexternal service providers in order to gain more control overthe development process. Thus, although Midco's webmanagers have the greatest web expertise, Southco andNorthco are compensating for any lack of internal knowledgeby seeking expertise outside the company. Both of thesefirms also belong to a sevenfirm web benchmarking group in the industry that meetsregularly to discuss web issues, problems, and successes.Research FindingsThe web benchmarking group Southco and Northcobelong to has not widely addressed the issue of performancemeasurement and has not provided standard tools formeasurement. Thus, it appears that member firms are largelyon their own with regard to how to determine their respectiveweb site's effectiveness. Southco currently monitorsperformance through Web trend reports such as number ofvisits and demographics on visitors. It also gets feedbackabout how satisfied the primary users are with the site.Southco is  planning a major redesign of the site to focus onthe needs of different target audiences, and will expandperformance evaluations to include those new users. Othermeasures Southco is considering include reduced customerservice time, how well the site meets constituent needs, howoften the news media picks up stories from the site, and howoften different stakeholders access the site. They also plan touse return on investment (ROI) when asking seniormanagement for funding for redesign, yet the respondents arenot sure how to measure ROI for the web at this point. Seniormanagement has not yet asked the web development team tojustify the site, thus there has been little incentive to developstrong measures. It is interesting to note that the goals for theredesign of the site are explicitly tied to business objectives,yet little thought has been given to establishing performancemeasures to monitor how well these objectives are met. Thus,Southco relies largely on qualitative performance measures todetermine how effective the site is.     Northco relies primarily on cost as a measure ofperformance. It currently monitors development costs andnumber of daily visits to the site, and compares the cost pervisit to the cost per person who might see an advertisement ina magazine or newspaper. Thus far, senior management hasnot asked for a justification of the site. However, Northco isalso planning to redesign its site, and feels that seniormanagement will ask for performance justification prior tofunding the redesign. Like Southco, Northco wants to use ROIfor this, but is also not sure how to apply ROI to the web.Midco counts the number of visits, how long the visit is,where visitors go on the site, and where they came from.However the web managers believe that this does not reallyshow how effective the site is, because it does not indicatewhether the visit turned into sales. They would like to followup on sales by asking the purchaser what influenced them tobuy, and specifically ask whether the Web influenced them.Just as at Southco and Northco, senior management has notasked for justification of the site, so there has been no pressureto evaluate its effectiveness. Of the three firms, Midco appearsto have the weakest performance measures in that they do notseem to be tied to anything in particular such as usersatisfaction with the site (at Southco) or cost (at Northco).     When asked for self-evaluation of the performance of sites,each of the respondents rated their site as average compared to249others in the industry. Thus, without solid measures, it seemsthat people believe they are doing as well as anyone else. It issomewhat surprising that even Midco, who believes it hassubstantial web expertise on the web development team,rated the site as average, rather than above average. Thus, itappears that expertise has little bearing on these respondent'sperceptions of web performance. However, there weredifferences in respondents' perceptions of their ability toimprove the sites. Southco believes its ability to develop abetter web site is moderate, Northco believes its ability islow, and Midco believes its ability is high. These self ratingsparallel the expertise on the web development team. Northcobelieves it has limited expertise, Southco moderate, andMidco very high expertise.Summary & Directions for Future ResearchThree key findings emerge from this exploratory study.First, although few standard web performance measuresexist, at least some firms are attempting to develop their own.Some are qualitative measures of how well the site meetsuser needs, while others are cost oriented. However, none areexplicitly tied to organizational goals, or even to explicitobjectives for the web site itself. This relates to the secondfinding; that senior management in none of these firms hasyet asked for performance justification. One area for futureresearch is to examine the influence of senior management'srole and perspective on corporate web sites. Web managersare attempting to measure performance, but without directionfrom top managers, it is difficult to tie web performance tocorporate goals, and thus to determine how the web site isactually benefiting the firm.      Another finding is that self-perceptions of expertiseparallel perceptions of ability to develop more effective sites.Although there are many factors that influence webperformance, self-perceptions of ability have been shown toinfluence actual performance in other contexts as well(Schunk, 1989). Thus, if web managers do not believe theycan do much better, then indeed they may not take advantageof opportunities and make use of the resources necessary toimprove site performance. Further research is needed toexplore the strength of this link, and if it is strong, firmsshould take steps to ensure that people responsible for webdevelopment have/receive the education and trainingnecessary to not only increase actual expertise, but tofacilitate realistic perceptions of expertise and its role ineffective web development.     With the phenomenal growth in corporate web site usage,it is becoming increasingly important for firms to measurethe performance of their sites in order to determine whetherthey are realizing projected benefits. Firms need to measureperformance and to understand what factors may influenceperformance in order to make effective decisions about thefuture of their sites. As standards begin to emerge, web siteswill begin to be even more useful, and business will be ableto take better advantage of the opportunities afforded by thisnew technology.References1. Bandura, A., 1986, Social Foundations of Thought andAction - A Social Cognitive Theory, Englewood Cliffs, NJ:Prentice-Hall.2. Commerce by Numbers, 1999, accessed via the WorldWide Web, Internet addresshttp://www.computerworld.com/home/Emmerce.nsf/all/bs,accessed on February 3, 1999.3. Jarvenpaa, S.L. and Todd, P.A., 1996-97, ConsumerReactions to Electronic Shopping on the World Wide Web,International Journal of Electronic Commerce, Winter,1(2), pp. 59-88.4. Ho, J., 1997, Evaluating the World Wide Web: A GlobalStudy of Commercial Sites, Journal of Computer MediatedCommunication, June, 3(1), Internet addresshttp:/www.ascusc.org/jcmc/vol3/issue1/ho.html, accessedvia WWW on January 6, 1999.5. Hoffman, D.L., Novak, T.P, and Chatterjee, P., 1995,Commercial Scenarios for the Web: Opportunities andChallenges, Journal of Computer Mediated Communication,Dec, 3(1), Internet addresshttp:/www.ascusc.org/jcmc/vol1/issue3/hoffman.html,accessed via WWW on January 6, 1999.6. Magid. L. 1995, Populism thrives online,Informationweek, 514, Feb 13, p. 74.7. Schunk, D.H., 1989, Self-efficacy and Cognitive eSkill Learning, in Research on Motivation in Education,Ames, C. and R. Ames (eds.), 3 (1989), 13-14.8. Wilder, C. 1995, An electronic bridge to consumerism,Informationweek, 510, Jan 16, pp. 38-39.

Corporate Web Performance Evaluation: An Exploratory Assessment

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1445&amp;context=amcis1999

Corporate Web Performance Evaluation: An Exploratory Assessment

Abstract

Similar works

Full text

Available Versions

AIS Electronic Library (AISeL)