Evaluation of software mediated process assessments for IT service management

IT service organisations are cognisant that continual service improvement can be achieved by conducting regular process assessments. However, such assessments are expensive and so we have developed a Decision Support System (DSS) tool which uses the international standard for process assessment ISO/IEC 15504 to offer a transparent and efficient approach. This paper provides evidence of evaluation of this software-mediated process assessment (SMPA) approach which was based on ISO/IEC 15504, ISO/IEC 20000 and the IT Infrastructure Library (ITIL®). In a usability evaluation of the online tool, participants reported largely positive experiences finding the online survey easy to use trustworthy, comfortable, generally effective, and more transparent and less costly to implement than a manual assessment. However, to engage in process improvement, human judgment, and possibly expert assessment facilitators are necessary for assessment validation and improvement, that is, a fully automated online survey that is strictly standard-based is not very useful. Further clarification of the survey questions with relevant examples, clearer answer options and having more visible goal statements on every question page were suggested

Usefulness and reliability of online assessments: a Business Faculty's experience

The usefulness and reliability of online assessment results relate to the clarity, specificity and articulation of assessment purposes, goals and criteria. Cheating and plagiarism are two frequent and controversial issues that arise and there is a view that the online assessments mode inherently lends itself to both these practices. However, reconceptualising practice and redeveloping techniques can pave the way for an authentic assessment approach which minimizes student academic dishonesty. This article describes research which investigated online assessments practice in a business faculty at an Australian university and proposes what might constitute good, sustainable practice and design in university online assessment practices

The impact of using computer decision-support software in primary care nurse-led telephone triage:Interactional dilemmas and conversational consequences

Telephone triage represents one strategy to manage demand for face-to-face GP appointments in primary care. Although computer decision-support software (CDSS) is increasingly used by nurses to triage patients, little is understood about how interaction is organized in this setting. Specifically any interactional dilemmas this computer-mediated setting invokes; and how these may be consequential for communication with patients. Using conversation analytic methods we undertook a multi-modal analysis of 22 audio-recorded telephone triage nurse-caller interactions from one GP practice in England, including 10 video-recordings of nurses' use of CDSS during triage. We draw on Goffman's theoretical notion of participation frameworks to make sense of these interactions, presenting 'telling cases' of interactional dilemmas nurses faced in meeting patient's needs and accurately documenting the patient's condition within the CDSS. Our findings highlight troubles in the 'interactional workability' of telephone triage exposing difficulties faced in aligning the proximal and wider distal context that structures CDSS-mediated interactions. Patients present with diverse symptoms, understanding of triage consultations, and communication skills which nurses need to negotiate turn-by-turn with CDSS requirements. Nurses therefore need to have sophisticated communication, technological and clinical skills to ensure patients' presenting problems are accurately captured within the CDSS to determine safe triage outcomes. Dilemmas around how nurses manage and record information, and the issues of professional accountability that may ensue, raise questions about the impact of CDSS and its use in supporting nurses to deliver safe and effective patient care

Innovation and Employability in Knowledge Management Curriculum Design

During 2007/8, Southampton Solent University worked on a Leadership Foundation project focused on the utility of the multi-functional team approach as a vehicle to deliver innovation in strategic and operational terms in higher education (HE). The Task-Orientated Multi-Functional Team Approach (TOMFTA) project took two significant undertakings for Southampton Solent as key areas for investigation, one academic and one administrative in focus. The academic project was the development of an innovative and novel degree programme in knowledge management (KM). The new KM Honours degree programme is timely both in recognition of the increasing importance to organisations of knowledge as a commodity, and in its adoption of a distinctive structure and pedagogy. The methodology for the KM curriculum design brings together student-centred and market-driven approaches: positioning the programme for the interests of students and requirements of employers, rather than just the capabilities of staff; while looking at ways that courses can be delivered with more flexibility, e.g. accelerated and block-mode; with level-differentiated activities, common cross-year content and material that is multi-purpose for use in short courses. In order to permit context at multiple levels in common, a graduate skills strand is taught separately as part of the University’s business-facing education agenda. The KM portfolio offers a programme of practically-based courses integrating key themes in knowledge management, business, information distribution and development of the media. They develop problem-solving, communications, teamwork and other employability skills as well as the domain skills needed by emerging information management technologies. The new courses are built on activities which focus on different aspects of KM, drawing on existing content as a knowledge base. This paper presents the ongoing development of the KM programme through the key aspects in its conception and design

Virtualising process assessments to facilitate continual service improvement in IT service management

The IT Service Management (ITSM) industry has defined processes as best practices in the widely accepted IT Infrastructure Library (ITIL) framework. However, studies on the measurement of ITSM process improvement are scant. Our research addressed the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the Design Science Research methodology, we developed a Software-mediated Process Assessment (SMPA) approach that enables assessment of ITSM processes. The SMPA approach includes process selection; an online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. We implemented a decision support system (DSS) to automate the SMPA approach and evaluated it at two IT service providers. The evaluations indicated that the SMPA approach supports decision-making on process improvements. The findings provided design knowledge of virtualisation in ITSM process assessment and how this may facilitate continual service improvement

Virtualising Process Assessments to Facilitate Continual Service Improvement in IT Service Management

The IT Service Management (ITSM) industry has defined processes as best practices in the widely-accepted IT Infrastructure Library (ITIL) framework. However, studies on the measurement of ITSM process improvement are scant. Our research addressed the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the Design Science Research methodology, we developed a Software-mediated Process Assessment (SMPA) approach that enables assessment of ITSM processes. The SMPA approach includes process selection; an online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. We implemented a decision support system (DSS) to automate the SMPA approach and evaluated it at two IT service providers. The evaluations indicated that the SMPA approach supports decision-making on process improvements. The findings provided design knowledge of virtualisation in ITSM process assessment and how this may facilitate continual service improvement

Development and evaluation of a software-mediated process assessment approach in IT service management

To operate in a highly competitive business environment, organisations require the support of continually improving IT services. The dominant academic literature on ITService Management (ITSM) focuses on the measurement of the outcome of ITSM implementation. Consequently, there is limited research on the measurement of ITSM processes. The ITSM industry has defined a number of processes as best practices in the IT Infrastructure Library (ITIL®) framework and the international standard forITSM, ISO/IEC 20000. However, there is a lack of a transparent and efficient process assessment method to improve ITSM processes. This research aims to address the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the design science research methodology, an iterative design process was followed to develop a research artefact in the form of a method: the Software-Mediated Process Assessment (SMPA) approach that enables researchers and practitioners to assess the ITSM processes in a transparent and efficient way. The four phases in theSMPA approach include preparation for the assessment; online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. The international standard for process assessment ISO/IEC 15504 and associated assessment models provided support for a transparent method. A Decision Support System (DSS) was implemented to demonstrate efficient use of the SMPA approach. Using a theoretically-grounded fit profile based on the Task-Technology Fit theory, the international standards and DSS technology were implemented in the SMPA approach to address the research problem. The DSS platform was provided by an industry partner Assessment Portal Pty Ltd. that specialises in online assessment services. Two case study organisations provided test sites for the evaluation of the SMPA approach. The two organisations are the Queensland Government’s primary IT service provider, CITEC and the IT service department of an Australian local government authority, Toowoomba Regional Council. Using the quality models from the international standard for software quality evaluation ISO/IEC 25010, the usability and ii outcomes of the SMPA approach were evaluated. Evidence from the case study evaluations indicated that the SMPA approach is usable for ITSM process assessment in order to support decision-making on process improvements. Further discussions of the research findings provided design knowledge that included the emergence of the concept of virtualisability in ITSM process assessments and a proposal of a hybrid ITSM process assessment method. Moreover, iterations ofself-assessments of ITSM processes using the SMPA approach may facilitate continual service improvement. Based on the design knowledge obtained, the contributions of this research to theory and practice were articulated. The SMPA approach extends prior guidelines on ITSM process assessment by providing a fine-grained method to assess ITSM processes. The SMPA approach clarifies the impact of software mediation to support transparency and efficiency in the way process assessments are conducted. This research also demonstrates how the SMPA approach is applied in practice by enabling IT organisations to self-assess the capability of their ITSM processes. Upon reflection, the design science research method was found to be highly suitable to develop an artefact to solve a research problem and to evaluate the practical utility of the artefact. The SMPA approach is a research artefact that is implemented as a DSS; hence it is readily accessible to practitioners. The focus on practical utility provides researchers with results that are more readily endorsed, thus maximising the impact of the research findings in practice