Correct and Control Complex IoT Systems: Evaluation of a Classification for System Anomalies

In practice there are deficiencies in precise interteam communications about system anomalies to perform troubleshooting and postmortem analysis along different teams operating complex IoT systems. We evaluate the quality in use of an adaptation of IEEE Std. 1044-2009 with the objective to differentiate the handling of fault detection and fault reaction from handling of defect and its options for defect correction. We extended the scope of IEEE Std. 1044-2009 from anomalies related to software only to anomalies related to complex IoT systems. To evaluate the quality in use of our classification a study was conducted at Robert Bosch GmbH. We applied our adaptation to a postmortem analysis of an IoT solution and evaluated the quality in use by conducting interviews with three stakeholders. Our adaptation was effectively applied and interteam communications as well as iterative and inductive learning for product improvement were enhanced. Further training and practice are required.Comment: Submitted to QRS 2020 (IEEE Conference on Software Quality, Reliability and Security

Evaluate and control service and transaction dependability of complex IoT systems

Observing and controlling the dependability of service provision of complex IoT systems is challenging. In practice, many organizations struggle to derive consumer needs related to quality and to observe and quantify the service provision in the context of the dynamic behavior of a complex distributed system. In this paper, we present an approach to define and evaluate the dependability of complex IoT systems. Our approach is an adaptation of the ISO/IEC 25040, an international standard for the evaluation process for system and software quality, which is part of the systems and software quality requirements and evaluation (SQuaRE) series. Our approach was designed and evaluated with action research in an industrial study at Robert Bosch GmbH. Based on the framework of the SQuaRE series, we integrated different elements of site reliability engineering (SRE) and combined them with distributed tracing as a promising measurement method. Our approach introduces the IoT transaction concept to reduce modeling and observation efforts while increasing operationalization to measure performance against dependability targets. Our adaption was effectively applied, consumer-centricity along different system stakeholders were enhanced, and negative consequences of organizational silos were reduced. This has improved the dependability evaluation of service provision to enable fast feedback cycles for service performance control and improvement.Projekt DEA