Systematic Review of Methodological Approaches for Designing, Assessing and Validating Business Analytics Maturity Models

Context: Applying maturity models to measure and evaluate Business Analytics (BA) in organisations is challenging. There is a lack of empirical studies on how BA maturity models are designed, assessed and validated to determine how BA contributes to business value. Objective: To report on state of research on BA maturity models (BAMMs) and identify how BAMMs can be empirically (1) designed, (2) assessed and (3) validated. Method: Systematic review of BA maturity model studies focuses on methodological approaches used in design, assessment and validation of BA maturity models. Results: (1) A systematic review resulted in nine papers included for analysis. (2) Within these papers the dominant methodological design approaches for maturity models are Rasch analysis and set theory; (3) assessment approaches are Cluster, Additive Logic, Minimum Constraints using Statistical Squared Distance and Euclidian Distance; and (4) validation approaches are variance techniques using regression, correlation coefficients with tests for statistical significance against self-reported maturity, perceived benefits or performance. Conclusion: This research contributes to a deeper understanding of how BAMMs can be designed, assessed and validated in a rigorous manner. Future research should involve more empirical studies that demonstrate the validity and usefulness of BAMMs in contributing to business value

WHOSE MATURITY IS IT ANYWAY? THE INFLUENCE OF DIFFERENT QUANTITATIVE METHODS ON THE DESIGN AND ASSESSMENT OF MATURITY MODELS

This paper presents results from an ongoing empirical study that seeks to understand the influence of different quantitative methods on the design and assessment of maturity models. Although there have been many academic publications on maturity models, there exists a significant lack of understanding of the potential impact of (a) choice of the quantitative approach, and (b) scale of measurement on the design and assessment of the maturity model. To address these two methodological issues, we analysed a social media maturity data set and computed maturity scores using different quantitative methods prescribed in literature. Specifically, we employed five methods (Additive, Variance, Cluster, Minimum Constraint, and RASCH) and compared the sensitivity of measurement scale and maturity stages. Based on our results, we propose a set of methodological recommendations for maturity model designers

Maturity Models Development in IS Research: A Literature Review

Maturity models are widespread in IS research and in particular, IT practitioner communities. However, theoretically sound, methodologically rigorous and empirically validated maturity models are quite rare. This literature review paper focuses on the challenges faced during the development of maturity models. Specifically, it explores maturity models literature in IS and standard guidelines, if any to develop maturity models, challenges identified and solutions proposed. Our systematic literature review of IS publications revealed over hundred and fifty articles on maturity models. Extant literature reveals that researchers have primarily focused on developing new maturity models pertaining to domain-specific problems and/or new enterprise technologies. We find rampant re-use of the design structure of widely adopted models such as Nolan’s Stage of Growth Model, Crosby’s Grid, and Capability Maturity Model (CMM). Only recently have there been some research efforts to standardize maturity model development. We also identify three dominant views of maturity models and provide guidelines for various approaches of constructing maturity models with a standard vocabulary. We finally propose using process theories and configurational approaches to address the main theoretical criticisms with regard to maturity models and conclude with some recommendations for maturity model developers

HOW AGILE IS YOUR IT DEPARTMENT? – DEVELOPMENT AND APPLICATION OF AN FRAMEWORK-INDEPENDENT AGILE SCALING MATURITY MODEL

Many IT departments seek to capitalize on the benefits of agile development by scaling agile practices. To manage the complex scaling, established approaches and frameworks promise guidance. However, although existing works envision a clear target state, they lack relevant capabilities along the scaling process, especially for vertical agile scaling. Managers need these capabilities to assess their company’s status quo and develop a clear scaling roadmap. Thus, within this work, we use the Design Science Research paradigm to build and evaluate a framework-independent agile scaling maturity model that provides management with a tool for ex-ante identification and evaluation of agile scaling capabilities in five maturity stages. To evaluate our model, we applied it at KUKA IT, the IT department of an international provider of automation solutions. As a result, this work provides insights into the application and outlines how IT departments can operationalize and utilize our model to guide agile scaling

A Set Theoretical Approach to Maturity Models: Guidelines and Demonstration

Maturity Model research in IS has been criticized for the lack of theoretical grounding, methodological rigor, empirical validations, and ignorance of multiple and non-linear paths to maturity. To address these criticisms, this paper proposes a novel set-theoretical approach to maturity models characterized by equifinality, multiple conjunctural causation, and case diversity. We prescribe methodological guidelines consisting of a six-step procedure to systematically apply set theoretic methods to conceptualize, develop, and empirically derive maturity models and provide a demonstration of it application on a social media maturity data-set. Specifically, we employ Necessary Condition Analysis (NCA) to identify maturity stage boundaries as necessary conditions and Qualitative Comparative Analysis (QCA) to arrive at multiple configurations that can be equally effective in progressing to higher maturity

An exploratory study for investigating the issues and current practices of service-oriented architecture adoption

Service-Oriented Architecture (SOA) is an approach that can be used to integrate different services across operating systems, platforms, languages, and networks which offer some benefits. However, many organizations fail to fully utilize SOA because the adoption processes are still immature. Therefore, an exploratory study was conducted to investigate current issues and practices of SOA adoption, the use of maturity levels for assessing SOA adoption, and the importance of information technology (IT) and business benefits in SOA adoption. Thus, the Grounded Theory approach was adapted in this study which involved seven software development companies in Malaysia. In this study, 14 SOA practitioners with at least three years of experience in SOA development were interviewed. The collected data was analyzed through three main coding stages: open, axial and selective coding. The theory which emerged from this study revealed SOA adoption issues, current practices, maturity levels, IT and business benefits. The study managed to identify five main issues in SOA adoption which were knowledge, infrastructure, costing, readiness, and documentation issues. The study also portrayed five best practices related to technology, framework, platform, standards, and tools. In addition, results from the study showed five IT and business benefits, consecutively. The findings from the study have led to theories formulation on SOA adoption which may assist researchers and SOA assessors to continuously improve the quality and maturity of SOA adoption in the future

Set-Theoretic Approach to Maturity Models

Despite being widely accepted and applied, maturity models in Information Systems (IS) have been criticized for the lack of theoretical grounding, methodological rigor, empirical validations, and ignorance of multiple and non-linear paths to maturity. This PhD thesis focuses on addressing these criticisms by incorporating recent developments in configuration theory, in particular application of set-theoretic approaches. The aim is to show the potential of employing a set-theoretic approach for maturity model research and empirically demonstrating equifinal paths to maturity. Specifically, this thesis employs Necessary Condition Analysis (NCA) to identify maturity stage boundaries as necessary conditions and Qualitative Comparative Analysis (QCA) to arrive at multiple configurations that can be equally effective in progressing to higher maturity. Furthermore, this thesis prescribes methodological guidelines consisting of detailed procedures to systematically apply set theoretic approaches for maturity model research and provides demonstrations of it application on three datasets. The thesis is a collection of six research papers that are written in a sequential manner. The first paper reviews literature on maturity models in IS, identifies research gaps and proposes use of configurational theory to address these challenges. The second paper conceptualizes stage boundaries as necessary conditions and demonstrates the application of Necessary Condition Analysis (NCA) on a social media maturity dataset. Building on the second paper, the third paper conceptualises maturity stage characteristics in terms of configurations using Qualitative Comparative Analysis (QCA). Overall, the third demonstrates empirically the existence of multiple paths to maturity and provides IS researchers with a six-step procedure and detailed guidelines to systematically apply set theroretic approaches to maturity models (STAMM). The fourth paper then uses the social media maturity dataset, computes maturity scores using different quantitative methods prescribed in maturity models literature and proposes recommendations for maturity model designers. The fifth and sixth papers are demonstrations of applicability of STAMM on different datasets. The fifth replicates and extends a prior research study on ITIL maturity and compares the findings with the results using STAMM. Finally, the sixth paper argues for a multi-method approach by combining STAMM and PLS-SEM in understanding the conditions associated with IT service management (ITSM) maturity. This PhD thesis contributes to the academic discussion on how maturity occurs through configurations. The key contribution is STAMM, a set-theoretic procedure model and method, which employs FsQCA and NCA to empirically demonstrate multiple paths to maturity (or equifinality). It also contributes to set-theoretic approaches, in particular QCA and NCA. Finally, this thesis contributes to multimethod approach by harmoniously integrating PLS-SEM, QCA and NCA, thus adding to the limited body of multi-method literature

An Instrument for Measuring SOA Maturity

Existing empirical research on the business value of Service-Oriented Architectures (SOA) has only measured the extent of SOA adoption – but not maturity – to determine whether typical goals, like increased business agility or IT cost reduction, could be achieved. However, a widely implemented SOA might be less mature than an SOA adopted only in particular areas of the organization, which in turn can lead to mismeasurement and misinterpretation. On the other side, the few existing SOA maturity frameworks that have been specified by previous researchers lack valid operationalizations to make them applicable to empirical research. Ready to use items and scales for evaluating the particular maturity level of an organization are missing. We propose to measure the degree of SOA maturity as a new variable for future empirical research especially in the context of SOA business value. Our analytical approach uses 21 items to classify the maturity of an organization’s SOA in seven maturity levels along seven maturity dimensions derived from The Open Group Service Integration Maturity Model (OSIMM). The applicability of this new instrument is shown using data from 121 organizations. The results show that the majority of the organizations has only reached SOA maturity levels two to four. Also, higher levels of SOA maturity highly and significantly increase the realized business value from SOA in terms of business agility, straight through processing (STP), and reduced IT costs. However, the marginal benefits are decreasing for higher levels of SOA maturity in cases of STP and business agility