Evaluating System Architecture Quality and Architecting Team Performance Using Information Quality Theory

Abstract

As engineering projects grow in complexity, estimating the required engineering effort during the development phase of a project has become more art than science. Predictions of required engineering effort are based on empirical models fitted to historical data with additional subjective factors, such as "team cohesion," applied based on the judgment of the user of the model. Information quality theory can be used to estimate the engineering effort necessary to develop system requirements by evaluating the change in requirements uncertainty during the requirements engineering process. This work begins by generalizing the information quality theory to apply to the variety of systems engineering artifacts generated during the development phase of a program. The generalized form of information quality theory is implemented in a model for application to the systems architecture definition process. Engineering effort required for system architecture definition on ten programs is used to evaluate both the performance of the model and the performance of the systems architecture definition teams. The results of the model and program evaluations show clear benefits in reducing the engineering effort required to define a systems architecture when there is reuse from previous programs. The results of the evaluation also show the architecture definition team benefits from "momentum" and performs more efficiently if the team has completed a systems architecture definition on a recent program. Finally, a set of systems architecting effort estimation equations are developed. These estimators show good prediction performance when applied to a set of ten major defense acquisition programs and could be used to estimate the required systems architecting engineering effort when planning future programs