Towards smart assessment: A metamodel proposal

International audienceAssessment initiatives in organisations are focused on the evaluation of organisational aspects aiming to obtain a critic view of their status. The assessment results are used to lead improvement programs or to serve as base for comparative purposes. Assessment approaches may comprise complex tasks demanding a large amount of time and resources. Moreover, assessment results are highly dependent on the assessment input, which may have a dynamic nature due to the constant evolution of organisations. The assessment results should be adaptable to these changes without much effort whilst being able to provide efficient and reliable results. Therefore, providing smart capabilities to the assessment process or to systems in charge of performing assessments represents a step forward in the search for more efficient appraisal processes. This work proposes a metamodel defining the elements of a Smart Assessment, which is guided by elements related to the smartness concept such as knowledge, learning, reasoning and inferring capabilities. The metamodel is further specialized considering a Business Process In-teroperability Smart Assessment scenario

The evolution of the TIPA framework: towards the automation of the assessment process in a design science research project

Managing processes remain a key challenge for most organizations which need to preserve competitiveness. Process assessment frameworks can help by providing instruments guiding process improvement and regulation alignment. Several process assessment frameworks such as TIPA® are based on the ISO Process assessment standard series ISO/IEC 15504, currently revised in the ISO/IEC 330xx family of standards. Following a Design Science Research (DSR) methodology, this paper visits the TIPA Framework evolution throughout iterative cycles in terms of design, rigour and relevance. It investigates how original and new artefacts are being developed and improved over the period of ten years, in particular the new strategic move towards the automation of the assessment process. By demonstrating the evolution of the TIPA framework using a DSR perspective, this paper explicates design knowledge regarding the role and value of the framework within the ISO standards community and in practice

Virtualising Process Assessments to Facilitate Continual Service Improvement in IT Service Management

The IT Service Management (ITSM) industry has defined processes as best practices in the widely-accepted IT Infrastructure Library (ITIL) framework. However, studies on the measurement of ITSM process improvement are scant. Our research addressed the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the Design Science Research methodology, we developed a Software-mediated Process Assessment (SMPA) approach that enables assessment of ITSM processes. The SMPA approach includes process selection; an online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. We implemented a decision support system (DSS) to automate the SMPA approach and evaluated it at two IT service providers. The evaluations indicated that the SMPA approach supports decision-making on process improvements. The findings provided design knowledge of virtualisation in ITSM process assessment and how this may facilitate continual service improvement

Applying design science research methodology in the development of virtual reality forest management services

Peer reviewe

Virtualising process assessments to facilitate continual service improvement in IT service management

The IT Service Management (ITSM) industry has defined processes as best practices in the widely accepted IT Infrastructure Library (ITIL) framework. However, studies on the measurement of ITSM process improvement are scant. Our research addressed the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the Design Science Research methodology, we developed a Software-mediated Process Assessment (SMPA) approach that enables assessment of ITSM processes. The SMPA approach includes process selection; an online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. We implemented a decision support system (DSS) to automate the SMPA approach and evaluated it at two IT service providers. The evaluations indicated that the SMPA approach supports decision-making on process improvements. The findings provided design knowledge of virtualisation in ITSM process assessment and how this may facilitate continual service improvement

The role of international standards to corroborate artefact development and evaluation: experiences from a design science research project in process assessment

International standards were used to corroborate artefact development and evaluation in a Design Science Research (DSR) project within the context of Process Assessments in IT Service Management (ITSM). While there have been significant research efforts towards extending DSR guidelines and the development and revisions of the standards, reports of the application of International Standards to validate DSR artefacts are scant. DSR, akin to any academic research, is required to demonstrate rigour and relevance with the use of theories and prior knowledge. Moreover, DSR presents an artefact as a solution to a class of problems and reports how the artefact is developed and evaluated. Our DSR project demonstrated that concerns about the quality of artefacts can be addressed and thereby the utility and validity of the artefact can be verified with the use of International Standards. Using three International Standards, process assessment ISO/IEC 15504-33000 series, IT Service Management ISO/IEC 20000, and System and Software Quality Models ISO/IEC 25010, this paper presents an account of a real-life DSR project that demonstrates the significant role of International Standards to guide DSR researchers during artefact design, development and evaluation

Development and evaluation of a software-mediated process assessment approach in IT service management

To operate in a highly competitive business environment, organisations require the support of continually improving IT services. The dominant academic literature on ITService Management (ITSM) focuses on the measurement of the outcome of ITSM implementation. Consequently, there is limited research on the measurement of ITSM processes. The ITSM industry has defined a number of processes as best practices in the IT Infrastructure Library (ITIL®) framework and the international standard forITSM, ISO/IEC 20000. However, there is a lack of a transparent and efficient process assessment method to improve ITSM processes. This research aims to address the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the design science research methodology, an iterative design process was followed to develop a research artefact in the form of a method: the Software-Mediated Process Assessment (SMPA) approach that enables researchers and practitioners to assess the ITSM processes in a transparent and efficient way. The four phases in theSMPA approach include preparation for the assessment; online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. The international standard for process assessment ISO/IEC 15504 and associated assessment models provided support for a transparent method. A Decision Support System (DSS) was implemented to demonstrate efficient use of the SMPA approach. Using a theoretically-grounded fit profile based on the Task-Technology Fit theory, the international standards and DSS technology were implemented in the SMPA approach to address the research problem. The DSS platform was provided by an industry partner Assessment Portal Pty Ltd. that specialises in online assessment services. Two case study organisations provided test sites for the evaluation of the SMPA approach. The two organisations are the Queensland Government’s primary IT service provider, CITEC and the IT service department of an Australian local government authority, Toowoomba Regional Council. Using the quality models from the international standard for software quality evaluation ISO/IEC 25010, the usability and ii outcomes of the SMPA approach were evaluated. Evidence from the case study evaluations indicated that the SMPA approach is usable for ITSM process assessment in order to support decision-making on process improvements. Further discussions of the research findings provided design knowledge that included the emergence of the concept of virtualisability in ITSM process assessments and a proposal of a hybrid ITSM process assessment method. Moreover, iterations ofself-assessments of ITSM processes using the SMPA approach may facilitate continual service improvement. Based on the design knowledge obtained, the contributions of this research to theory and practice were articulated. The SMPA approach extends prior guidelines on ITSM process assessment by providing a fine-grained method to assess ITSM processes. The SMPA approach clarifies the impact of software mediation to support transparency and efficiency in the way process assessments are conducted. This research also demonstrates how the SMPA approach is applied in practice by enabling IT organisations to self-assess the capability of their ITSM processes. Upon reflection, the design science research method was found to be highly suitable to develop an artefact to solve a research problem and to evaluate the practical utility of the artefact. The SMPA approach is a research artefact that is implemented as a DSS; hence it is readily accessible to practitioners. The focus on practical utility provides researchers with results that are more readily endorsed, thus maximising the impact of the research findings in practice