Irani, Zahir

Jones, Steve

Love, Peter

Themistocleous, Marinos

English

AIS Electronic Library (AISeL)

Association for Information SystemsAIS Electronic Library (AISeL)AMCIS 2002 Proceedings Americas Conference on Information Systems(AMCIS)December 2002INFORMATION SYSTEMS EVALUATION:MINI-TRACK INTRODUCTIONZahir IraniBrunel UniversityMarinos ThemistocleousBrunel UniversityPeter LoveEdith Cowan UniversitySteve JonesConwy County Borough CouncilFollow this and additional works at: http://aisel.aisnet.org/amcis2002This material is brought to you by the Americas Conference on Information Systems (AMCIS) at AIS Electronic Library (AISeL). It has been acceptedfor inclusion in AMCIS 2002 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contactelibrary@aisnet.org.Recommended CitationIrani, Zahir; Themistocleous, Marinos; Love, Peter; and Jones, Steve, "INFORMATION SYSTEMS EVALUATION: MINI-TRACKINTRODUCTION" (2002). AMCIS 2002 Proceedings. 163.http://aisel.aisnet.org/amcis2002/1631182 2002  Eighth Americas Conference on Information SystemsINFORMATION SYSTEMS EVALUATION:MINI-TRACK INTRODUCTIONZahir IraniBrunel UniversityZahir.Irani@Brunel.ac.ukMarinos ThemistocleousBrunel UniversityMarinos.Themistocleous@Brunel.ac.ukPeter E. D LoveEdith Cowan UniversityP.Love@edu.edu.auSteve JonesConwy County Borough Councilsteve.jones@conwy.gov.ukIntroductionThere is an increasing awareness that the implementation of IS forms an integral part of the provision of business andorganisational services. Successful IS can empower users and streamline processes (Hirschheim and Smithson, 1999) to helpimprove business and organisational performance (Storey et al., 2000). However, against this backdrop there is concern that ISinvestment does not always deliver value and that many IS projects do not meet business objectives (Strassman, 1997). Thisphenomenon has been labeled the 'IT productivity paradox' Brynjolffson (1993) and is most accurately linked to research studiesin the early nineties. Investment in information systems is regarded as both costly and risky and any IS investment should be examined for its businessvalue and benefit to the organization (Willcocks and Lester, 1999). However, many IS investments go ahead without the use ofinvestment appraisal, evaluation and risk management techniques (Remenyi et al, 2000). This is worrying, as in todays hi-technology environment, organizations are spending large proportions of their turnover of new systems and technology and thus,need to be confident that such monies are impacting the organization in the best possible way.Information Systems EvaluationIS evaluation necessarily forms part of any overall project to enable issues to be raised and documented, and remedial action takento address and improve the situation where necessary (Turner, 1998). However, many IS investments go ahead without the useof any evaluation techniques (Willcocks and Lester, 1999). Authors such as Hirschheim and Smithson, (1999), Remenyi et al, (2000) and Irani and Love, (2001) contend that IS evaluationis difficult to undertake because reliable estimates of IS costs and benefits are not always available or easy to obtain. This is dueto the often complex nature of the impact of IS, which frequently leads to tangible and intangible benefits and indirect costs (Iraniand Love, 2001). Many methods have been developed to measure IS efficiency and effectiveness, and there have been variousmodels proposed in an attempt to define what is required (Bjorn-Anderson and Davies, 1988; Willcocks, 1996; Strassman, 1997;Graeser, Willcocks and Pisanias, 1998, Remenyi et al, 2000). Land (2001) contends that there are now over 60 methods availableto practitioners. Irani and Love (2002) go one step further, and offer a classification of such techniques. However, Walsham(1999) maintains that there is widespread disagreement over the usefulness of evaluation methods and over which model to adopt.Furthermore, organizations that have undertaken IS evaluation have had mixed results (Balantine et al, 1999). This prevailing situation of no clear consensus of what constitutes meaningful evaluation, makes it difficult to assess the impactof IS, which has led some authors (Walsham, 1999; Wilson and Howcroft, 2000, Irani and Love, 2001) to argue for improvedIS evaluation practice and, furthermore, to identify and address barriers to undertaking IS evaluation in organizations. Irani et al./Information Systems Evaluation:  Introduction2002  Eighth Americas Conference on Information Systems 1183Barriers to Carrying Out an EvaluationAlthough there remains a wide variety of reasons to justify investments in information systems, empirical evidence is offered byIrani and Love (2001), Khalifa et al., (2000) and Serafeimidis and Smithson (2000) to support the lack of widespread evaluationprocesses, financial or otherwise. The increased complexity of information systems combined with the uncertainty and unpredictability associated with informationsystems benefits and costs clearly point to the need for evaluation procedures to offer companies a deeper insight into the impactsurrounding capital investment in IT/IS infrastructure, although it must be noted that there is a changing view from one ofinvestment to consumption (Irani and Love, 2002). Therefore, often placing the justification of IT/IS outside the confines oftraditional budgeting processes, albeit with varying degrees of reliance on investment appraisal techniques.Farbey et al, (1993) suggest that the search for a single best approach is fruitless due to the wide variety of complex interactingvariables. Yet, evaluation methods are constantly being propagated in a hope to find the panacea for the evaluation paradox,which organizations clearly face. Information systems evaluation has not been an explicit topic of any recent AMCIS mini-track [other than the mini track organizedby Irani et al., 2000; 2001] although isolated papers on information systems evaluation have appeared in several AMCISproceedings. These papers have been presented while spanning across different mini-tracks, thereby not allowing the informationsystems evaluation community and interested researchers to readily follow developments in this dynamic and emerging field.We [mini-track chairs] believe that this specific mini-track on Information Systems Evaluation will be highly beneficial to bothAMCIS and the information systems evaluation community.  In doing so, it will enable new and different insights of informationsystems evaluation to be viewed in a more holistic and integrated manner. The idea for organizing a mini-track on informationsystems evaluation originated from a lack of forum to debate the issues associated with information systems evaluation outsideEurope. This is evident with recent (European published journals) announcing special issues, such as:  Information Systems Journal, 12(4), 2002. Guest edited by Irani Z and Fitzgerald G. European Journal of Information Systems, 10(4): 183-236, 2001. Guest Edited by Irani Z and Love P.E.D. Logistics Information Management, 12(1-2): 1-195, 1999. Guest Edited by Irani Z.Much of the research community feels frustration with having to look through many conference programmes [including thoseof AMCIS] to find papers that relate to the information systems evaluation area, as a result, this mini-track proposed to go someway in addressing this critical issue.The themes explored by this mini-track deal with the evaluation and measurement of the effectiveness of emerging technologies,and its implication of the evaluation process. As a result, the purpose of this mini-track is to generate a stream of research orientedtoward the study of measuring effectiveness and impacts of information systems. Specifically, in areas were theoretical modelsmay need to be borrowed from referent disciplines, or where models and associated operationalizations have been proposed, ornot yet tested [conceptual].  Information Systems Evaluation: PurposeThe information systems evaluation mini-track will help researchers and practitioners understand the processes involved in thedecision making of adopting technology in contemporary organizations. Articles that address the justification process necessaryto evaluate IT/IS deployments by identifying the constructs associated with investment decision-making are presented. Emphasishas been placed on investment decision-making in the context of business process change and effective capital budgeting.Strategic frameworks, conceptual and analytical models, and case studies of information systems evaluation were encouragedand form the genesis of the mini-track.It is hoped that this mini-track will encourage the latest thinking and research in information systems evaluation to be presentedto a forum of leading information systems professionals and business executives. The mini-track will provide a potpourri of ideas,models, and case studies, which will be stimulating and useful. Information Systems Evaluation1184 2002  Eighth Americas Conference on Information SystemsInformation Systems Evaluation: Navigating Through the LiteratureAlthough the field of information systems evaluation is not new, it has been arguably re-popularized in recent years. This isevident in the number of conferences and journal special issues devoted to the subject. Only looking back at the last few yearsone can find: Americas Conference on Information Systems (AMCIS-2001), [Mini-track on Information Systems Evaluation andIntegration, Chair: Irani et al.,] - Boston Massachusetts USA, August 2001. Americas Conference on Information Systems (AMCIS-2000), [Mini-track on Information Systems Evaluation, Chair: Iraniet al.,] - Long Beach California, USA, August 2000. Thirty-third annual Hawaii international conference on System Science, [Mini-track on Information Systems Performanceand Evaluation, Chair: Giaglis G, Irani Z and Amaroso D], January 4-7, Island of Maui, Hawaii, USA, 2000. European Journal of Information Systems, vol. 7, no. 3 (1998): Special issue on Information Systems Evaluation, Edited byFarbey B. Sixth European Conference on the Evaluation of Information Technology, Edited by Remenyi D, Irani Z and Brown A.,Brunel University, UK, November 1999.  Fifth European Conference on the Evaluation of Information Technology, Reading University, UK, November 1998.  Fourth European Conference on the Evaluation of Information Technology, Delf University, Netherlands, November 1997. Third European Conference on the Evaluation of Information Technology, Bath University, UK, November 1996. Second European Conference on the Evaluation of Information Technology, City University, UK, November 1995. First European Conference on the Evaluation of Information Technology, Henley Management College, UK, November1994.AcknowledgementsThe authors would like to also acknowledge the support provided by the Engineering and Physical Sciences Research Council(EPSRC) Grant Ref: [GR/R08025].ReferencesAlshawi S, Irani Z, Love P.E.D and Zairi M. 2001. Supporting Supply Chain Management and E-commerce through a IT/ISInfrastructure. International NAISO Congress: Information Science Innovations (ISI2001), March 17-21, Dubai, UAE.Ballantine, J.A., Galliers, R.D. and Stray, S.J. Information Systems/Technology  Evaluation Practices: Evidence from UKOrganisations. In: Beyond the IT Productivity Paradox, L. Willcocks and S. Lester (Eds.) John Wiley and Sons, Chichester,pp. 123-150Bjorn-Andersen, N. and Davies, G.E. 1998. Information Systems Assessment, Elsevier Science Publishers, BV, North-Holland.Broadbent M.F, Weill P and Neo B.S,. 1999. Strategic context and patterns of IT infrastructure capability. Journal of StrategicInformation Systems, 8(2): 157-188.Brynjolfsson, E. 1993. The Productivity Paradox of Information Technology, Communications of the ACM, 35: 9-43.Chircu A.M abd Kauffman R.J. 2000. Limits of value in Electronic Commerce-Related IT Investments. Journal of ManagementInformation Systems, 17(2): 30-59.DIST (1998).  Building for Growth.  A Draft Strategy for the Building and Construction Industry.  Department of Industry,Science and Tourism, Commonwealth of Australia Publication, February, Canberra, Australia.Farbey, B., Land, F. and Targett, D. 1993. How to Assess your IT Investment: A Study of Methods and Practice, Butterworth-Heinmann, Oxford.Garfield M.J. and Watson R.T. 1997. 1997. Differences in national information infrastructures: the reflection of nationalcultures. Journal of Strategic Information Systems, 6(4): 313-338.Hirschheim, R.A. and Smithson, S. 1999. Evaluation of Information Systems: A Critical Assessment.  In: Beyond the ITProductivity Paradox, L. Willcocks and S. Lester (Eds.) Wiley, Chichester.Irani Z and Love P.E.D. 2002. Developing a Frame of Reference for Ex-ante IT/IS Investment Evaluation. European Journalof Information Systems, 11(1): 74-82.Irani, Z, Sharif, A.M and Love P.E.D. 2001. Transforming Failure into Success through Organizational Learning: An analysisof a Manufacturing Information System'. European Journal of Information Systems, 10(1): 55-66.Irani et al./Information Systems Evaluation:  Introduction2002  Eighth Americas Conference on Information Systems 1185Irani Z and Love P.E.D. 2001. The propagation of technology management taxonomies for evaluating investments in informationsystems Journal of Management Information System, 17(3): 161-177.Irani Z, Love P.E.D and Zairi M. 2000. 'Information Systems Evaluation: chair's introduction paper to the minitrack'. Associationfor Information System, 2000 Americas Conference on Information Systems (AMCIS 2000) [CD Proceedings], August 10-13, 2000, Long Beach, California, USA.Irani Z, Love P.E.D and Li. H 1999. IT/IS investment barriers to the decision making process Business Information TechnologyWorld (BITWORLD) Conference, [CD Proceedings], June 30-2, Cape Town, South Africa.Irani Z, Ezingeard J-N and Grieve R.J. 1998. 'Costing the true costs of IT/IS investments: A focus during management decisionmaking'. The Journal of Logistics and Information Management, 11(1): 38-43.Irani Z, Ezingeard J-N and Grieve R.J. 1997. 'Integrating the costs of an IT/IS infrastructure into the investment decision makingprocess'. Technovation, 17(11/12): 695-706.Khalifa G, Irani Z, Baldwin L.P and Jones S. 2000. Evaluating Information Technology with You in Mind'. Seventh EuropeanConference on Information Technology Evaluation (ECITE2000), Trinity College Dublin, September 28-29, pp. 117-132.Kaplan R.S. 1985. 'Financial justification for the factory of the future'. Working Paper 9-785-078, Harvard Business School,USA.Land, F.F. 2001. IS Evaluation: Recent Trends, Unpublished Presentation to NUKAIS Information Systems Evaluation Seminar,Priestley Hall, Leeds Metropolitan University, 27th February 2001. Laurillard, D. 1999. A conversational framework for individual learning applied to the learning organisation and the learningsociety. Systems Research and Behavioral Science, 16, 113-122.Lefley F. 1994. 'Capital investment appraisal of manufacturing technology'. International Journal of Production Research. 32(12):2751-2756.Love P.E.D, Li H, Irani Z and Faniran, O. 2000. 'Total quality management and the learning organisation: A dialogue for changein construction'. Construction Management and Economics, 18(1):321-331.Marosszekey, M., Sauer, C., Johnson, K., Karim, K., and Yetton, P. (2000). Information technology in the building andconstruction industry: the Australian experience. In Implementing IT to gain a competitive advantage in the 21st century, H.Li, Q.P. Shen, D. Scott, and P.E.D. Love (Eds.) Hong Kong Polytechnic University Press, pp.183-196.Raymond, L., Pare, G. and Bergeron, F. 1995. Matching Information Technology and Organisational Structure: An EmpiricalStudy with Implications for Performance, European Journal of Information Systems, 4, pp. 3-16.Remenyi D, Money A, Sherwood-Smith M, Irani Z. 2000. The Effective Measurement and Management of IT Costs andBenefits, Butterworth Heinemann/Computer Weekly - Professional Information Systems Text Books series, Second Edition,UK.Ryan S.D and Harrison D.A. 2000. Considering social subsystem costs and benefits in information technology investmentdecisions: A view from the field of anticipated payoffs. Journal of Management Information Systems, 16(4): 11-40.Serafeimidis V and Smithson S. 2000. Information Systems Evaluation in Practice: a case study of Organizational change, Journalof Information Technology, 15(2): 93-106.Smithson, S. and Hirschheim, R. (1998) Analysing Information Systems Evaluation: Another Look at an Old Problem, EuropeanJournal of Information Systems, 7(3): 158-174.Storey, A., Thompson, J.B., Bokma, A. and Bradnum, J. 2000.  An Evaluation of UK and USA Online Banking and Web Sites.In: Proceedings of the Americas Conference on Information Systems  (AMCIS 2000), August 10-13, Long Beach, California,USA, pp. 221-230.Strassman, P. 1997. The Squandered Computer, Information Economic Press, New Caanan, CN.Sumner, M. 1999. 'Critical success factors in enterprise wide information management systems projects', Proceedings of SIGCPR'99, New Orleans, LA, USA, pp. 297-303.Sircar S, Turnbow J.L and Bordoloi B. A framework for assessing the relationship between IT investments and firmperformance. Journal of Management Information Systems, 16(4): 42-69.Turner, J.R. 1998. The Commercial Project Manager, McGraw-Hill, London.Vandenbosch B and Ginzberg M. 1997. Lotus Notes and collaboration. Journal of Management Information Systems, 13(3):65-81.Walsham, G. 1999. Interpretive Evaluation Design for Information Systems.  In: Beyond the IT Productivity Paradox, L.Willcocks and S. Lester (Eds.) Wiley, Chichester. pp. 363-380.Willcocks, L. 1996.  Investing In Information Systems: Evaluation and Management, Chapman and Hall, London.Willcocks, L., Graeser, V. and Pisanias, N. 1998. Developing the IT Scorecard, Business Intelligence, London.Willcocks, L. and Lester, S. 1999.  IT: Transformer or Sink Hole.  In: Beyond the IT Productivity Paradox, L. Willcocks and S.Lester (Eds.) Wiley, Chichester.Wilson, M. and Howcroft, D. 2000. Politics of IS Evaluation: A Social Shaping Perspective.  In: Proceedings of the International Conference on Information Systems, December, Brisbane, Australia.

INFORMATION SYSTEMS EVALUATION:
MINI-TRACK INTRODUCTION

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1522&amp;context=amcis2002

INFORMATION SYSTEMS EVALUATION: MINI-TRACK INTRODUCTION

Abstract

Similar works

Full text

Available Versions

AIS Electronic Library (AISeL)