Accurate Estimates Without Local Data?

The article of record as published may be found at http://dx.doi.org/10.1002/spip.414Models of software projects input project details and output predictions via their internal tunings. The output predictions, therefore, are affected by variance in the project details P and variance in the internal tunings T. Local data is often used to constrain the internal tunings (reducing T). While constraining internal tunings with local data is always the preferred option, there exist some models for which constraining tuning is optional. We show empirically that, for the USC COCOMO family of models, the effects of P dominate the effects of T i.e. the output variance of these models can be controlled without using local data to constrain the tuning variance (in ten case studies, we show that the estimates generated by only constraining P are very similar to those produced by constraining T with historical data). We conclude that, if possible, models should be designed such that the effects of the project options dominate the effects of the tuning options. Such models can be used for the purposes of decision making without elaborate, tedious, and time‐consuming data collection from the local domain. Copyright © 2009 John Wiley & Sons, Ltd

Accurate Estimates Without Calibration?

Most process models calibrate their internal settings using local data. Collecting this data is expensive, tedious, and often an incomplete process. Is it possible to make accurate process decisions without historical data? Variability in model output arises from (a) uncertainty in model inputs and (b) uncertainty in the internal parameters that control the conversion of inputs to outputs. We find that, for USC family process models such as COCOMO and COQUALMO, we can control model outputs by using an AI search engine to adjust the controllable project choices without requiring local tuning. For example, in ten case studies, we show that the estimates generated in this manner are very similar to those produced by traditional methods (local calibration). Our conclusion is that, (a) while local tuning is always the preferred option, there exist some process models for which local tuning is optional; and (b) when building a process model, we should design it such that it is possible to use it without tuning

The business case for automated software engineering

Adoption of advanced automated SE (ASE) tools would be more favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the ”local tuning ” problem. Normally, predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable. This paper shows that assessing the relative merits of different SE methods need not require precise local tunings. STAR1 is a simulated annealer plus a Bayesian post-processor that explores the space of possible local tunings within software prediction models. STAR1 ranks project decisions by their effects on effort and defects and threats. In experiments with NASA systems, STAR1 found one project where ASE were essential for minimizing effort / defect/ threats; and another project were ASE tools were merely optional

Accurate Estimates Without Calibration?

Abstract. Most process models calibrate their internal settings using local data. Collecting this data is expensive, tedious, and often an incomplete process. Is it possible to make accurate process decisions without historical data? Variability in model output arises from (a) uncertainty in model inputs and (b) uncertainty in the internal parameters that control the conversion of inputs to outputs. We find that, for USC family process models such as COCOMO and COQUALMO, we can control model outputs by using an AI search engine to adjust the controllable project choices without requiring local tuning. For example, in ten case studies, we show that the estimates generated in this manner are very similar to those produced by traditional methods (local calibration). Our conclusion is that, (a) while local tuning is always the preferred option, there exist some process models for which local tuning is optional; and (b) when building a process model, we should design it such that it is possible to use it without tuning. Word length: 6525 words (4875 words of text + 7 figures at 250 words per figure).