We want to explore and analyse design decisions that influence maintainability of software. Software maintainability is important because the effort expended on changes and fixes in software is a major cost driver. We take an empirical, qualitative approach, by investigating cases where a change has cost more or less than comparable changes, and analysing the causes for those differences. We will use this analysis of causes as input to following research in which the individual contributions of a selection of those causes will be quantitatively analysed

Hordijk, Wiebe

Wieringa, Roel

English

Crossref

Surveying the factors that influence maintainability

Hordijk, W.T.B.

Wieringa, Roelf J.

University of Twente Research Information

Surveying the Factors that Influence MaintainabilityResearch DesignWiebe Hordijk∗Faculty of Electrical Engineering,Mathematics and Computer ScienceUniversity of Twentewww.cs.utwente.nl/∼hordijkwtbhordijkwtb@cs.utwente.nlRoel WieringaFaculty of Electrical EngineeringMathematics and Computer ScienceUniversity of Twentewww.cs.utwente.nl/∼roelwroelw@cs.utwente.nlABSTRACTWe want to explore and analyse design decisions that influ-ence maintainability of software. Software maintainabilityis important because the effort expended on changes andfixes in software is a major cost driver. We take an em-pirical, qualitative approach, by investigating cases where achange has cost more or less than comparable changes, andanalysing the causes for those differences. We will use thisanalysis of causes as input to following research in which theindividual contributions of a selection of those causes will bequantitatively analysed.Categories and Subject DescriptorsD.2.11 [Software]: Software Architectures; D.2.9 [SoftwareEngineering]: Management—time estimation, productiv-ity ; D.2.7 [Software Engineering]: Distribution, Mainte-nance, and Enhancement—restructuring, reverse engineer-ing, and re-engineering ; D.2.8 [Software Engineering]:Metrics—product metricsGeneral TermsDesign, MeasurementKeywordsSoftware maintainability, software design, case study1. INTRODUCTIONThis research is part of a research project that aims atidentifying and validating relations between design decisionsand software quality. In this part of the project, the qual-ity attribute maintainability has been chosen as the prime∗Sponsored by Ordina (www.ordina.nl)Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy therwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.ESEC-FSE’05,September 5–9, 2005, Lisbon, Portugal.Copyright 2005 ACM 1-59593-014-0/05/0009 ...$5 00.attribute of interest. To establish a solid theory of maintain-ability, the factors that are thought to influence maintain-ability (also known as cost drivers) must be explored first,and later validated.The system of which maintainability is an attribute, isthe combination of the software system and the maintenanceteam, where the team is the (part of the) organisation thatperforms changes to the software. Maintainability cannot beseen as an attribute of the software system alone, because itdepends a great deal on who maintains it: a team that hasa lot of experience with a particular system will maintainit more easily. Both the software and the team have inter-nal attributes that influence maintainability, for example,structural complexity of the software and skill of the teammembers. We want to survey the factors that lead to low orhigh maintainability.As an indicator of maintainability we choose change ef-fort. Change effort is an external attribute of a change.Change effort is the total of analysis and programming ef-fort expended on one particular change. Change effort doesnot include related activities, such as documentation, com-munication or deployment. We do not need a system-levelmaintainability measure, as the individual change efforts arethe object of our study.A change request can be due to a failure, changing re-quirements, prevention or any other reason. The activitiesby the maintenance team include actually performing thechange, but also documenting, testing, and reporting, de-pending on the maintenance procedures. When a system ischanged so extensively that a new team is formed to imple-ment the changes, that is not regarded as a change. Such asituation is more like a new system being developed.Based on our own experience and preliminary literaturestudy, we think that the factors that influence maintainabil-ity can be categorised in the following way (assembled andadapted from [1], [2], [4], [6], [7], [8] and [9]). This categori-sation is used as an aid to direct our exploration and to keepit within reasonable bounds. When during the research wefind that categories should be added or changed, that willbe an outcome of the research.• Software product properties– Specification-level properties, e.g. correctness orfunctional size– Design-level properties, e.g. modularity, coupling385– Code-level properties, e.g. code size, complexity,maturity, decay, duplication• Maintenance process properties– Frequency of changes– Maintenance procedures– Testing process properties, e.g. test coverage• Resource properties– Team properties, e.g. activity rate, communica-tion structure, personnel turnover– Team member properties, e.g. skill level, familiar-ity with the system– Maintenance infrastructure, e.g. development en-vironment, tools• Change properties– Change functional properties, e.g. functional size,change type, functional complexity, performancesensitivity– Change implementation properties, e.g. changesize, change span, fault potentialIn our research, we treat design decisions as independentvariables and change effort as the dependent variable. Weassume the software product properties as intervening vari-ables, linking the design decisions to change effort. Theother factors are either control variables if we can isolatethem, or extraneous variables if we cannot. This is illus-trated in figure 1.To be able to analyse the software product properties asintervening variables, we will need to make them measur-able in some way. There are several approaches to measure-ment of these properties. One notably comprehensive setof instruments is provided by MITRE corporation [4]. Wecurrently evaluate these instruments.The change property functional size deserves special at-tention. We consider it logical, and not a sign of bad designor inefficient procedures, that larger changes take more ef-fort. The functional size of changes, however, is not easyto define. It is a very different measure from the technicalsize, which can be expressed for example in lines of code.The functional size is the size of the change as seen from theuser’s viewpoint. This is important because it is only logicalthat bigger changes need more effort. The maintainabilitywe’re interested in, is the effort per unit volume of functionalchange size. To control this variable, we use a categorisa-tion of functionally similar changes, where changes are onlycompared within their own category.2. GOALOur goal is to investigate which design decisions influencemaintenance effort for software systems. Our scope is lim-ited to custom-built administrative enterprise applications.Our first sub-goal is to explore which factors could influ-ence software maintainability. We need an as complete aspossible overview of these factors, to enable isolation of in-dividual factors in the validation part of the research. Theexploration has been successful if we have a model that atleast covers the categories mentioned in the introduction,and that we can use to base further research on.Our second sub-goal is to validate whether design deci-sions found in the exploration really have an effect on soft-ware maintainability.3. RESEARCH DESIGNTo achieve our first goal we will review the literature foralready investigated factors that influence maintainability,and visualise these in a cause-effect graph. We will ex-pand this graph with factors from our own experience andmaintenance personnel’s opinions about what factors affectmaintainability. In this graph, sources and evidence for thehypothesised cause-effect relations will be visualised.To achieve our second goal we will validate the effects thatdesign decisions have on maintainability. In each validation,we choose one particular design decision as the indepen-dent variable, and the maintainability of the system is thedependent variable. Design decisions are choices between fi-nite numbers of options; they have a nominal scale and canbe measured by inspecting the software product. We op-erationalise maintainability as the change effort of individ-ual changes. Numerous other factors are either interveningor extraneous variables, which is why the exploratory partof this research is important: since there are long cause-effect chains between design decisions and maintainability,and there are many extraneous variables, having a good mapof the factors involved is essential for isolating the variablesunder investigation.We will validate the effects that design decisions have onmaintainability by looking for cases where a difference inoptions chosen for a particular design decision can explaina difference in change efforts. This can be shown by findingpairs of cases where for a certain decision, all else remain-ing equal, in one case one option is chosen and in the othercase the other, and then measuring the change effort in bothcases. This is of course an idealised picture of the researchdesign; in practice, more factors will be different betweeneach two cases, so more than two cases will be needed tosingle out one factor. Some thoughts on isolating individ-ual factors can be given. Team factors, for instance, canbe isolated by picking cases where the same team maintainsseveral systems. Some other factors can be isolated by tak-ing cases from different parts of the same system, or thesame system at different moments in time. This leads tothe case study design depicted in figure 2.The case study design we are currently testing in a pilotcase study is a multiple-case embedded design [11], wherethe top-level units are systems and the embedded units arechanges to those systems. We compare changes to the samesystem using contrasting logic, and between the systems weuse replication logic, as shown in figure 2. The strategyfor analysing individual cases is pattern matching: from thelogical model given by the theoretical framework, we deriveexpected patterns in the data for the hypothesis and the ri-val explanations, and then we show whether the actual datamatches the hypothesis pattern or any of the rival explana-tion patterns.4. SAMPLINGWe will select a sample of cases, based on the followingcriteria:• Whether they have collected data about maintenanceeffort;386Independent variables:Design decisionsIntervening variables:Software product properties(maintenance-friendliness)Dependent variable:Change EffortControl/Extraneous variables:Process, Resource and ChangepropertiesFigure 1: Overview of the relations between the kinds of variablesFigure 2: Overview of the case study design• Whether enough documentation and people are avail-able to investigate the causes of maintenance effortdifferences;• Whether the systems themselves are representative forthe class of systems in our scope;• We will choose cases that have occurred in recent his-tory, to facilitate data collection.5. RATIONALEThe reason we choose case study research as our researchmethod is that case study research is particularly suited forresearch in which the number of variables is greater thanthe number of data points that can feasibly be sampled [11].Since maintainability is influenced by many factors, this iscertainly the case for our research. Also, case studies areappropriate for descriptive research like ours, in which thecauses of phenomena are investigated.We choose cases from history to facilitate data collection.We prefer as recent cases as possible, to improve the rel-evance of the results (as technological advancements makeresults from older cases obsolete) and to improve the accu-racy of the data (as we partly need to rely on the memoriesof people involved).We first planned to focus on maintenance outliers, be-cause we thought that those would yield more qualitativeinformation than normal changes. We figured that a prac-titioner when asked why change X took an average changeeffort might not know what to answer, whereas the same in-terviewee would give lots of valuable information when askedwhy change Y took ten times as much. We have deviatedfrom that approach because of the foreseeable difficulties ingeneralising from outliers to normal cases. When our chosenapproach yields too little results, we can still decide to usethe outlier approach.We operationalise maintainability as the change effort ofindividual changes. We choose this fine level of granularityinstead of a coarser measure like average change effort, tobenefit from the additional information that can be retrievedfrom differences between different parts of the same system.We choose a rather narrow definition for change effort,excluding all activities except analysis and coding. This isbecause for many of the excluded activities, the team canchoose how much effort is expended on them. Take doc-umentation effort as an example. The more documenta-tion effort, the higher the quality of the documentation (wethink). The higher the quality of the documentation, the387more maintainable the system will probably be. So if wewould measure documentation effort as part of change ef-fort, we would contaminate our dependent variable with anindependent variable.We rank changes into similarity classes, instead of using ameasure for functional size of changes based on a ratio scale.This is because we expect considerable difficulties in choos-ing such a ratio scale. Some researchers [1] report successwith using function points for measuring change size in smallmaintenance projects, while others [10] report low fidelity offunction point measures for change size. We do not need aratio scale for change size in this research (although we mayneed one in further research), so we avoid such difficulties fornow by choosing a nominal scale. We still risk interactionbetween the categorisation and change effort, i.e. differentcategorisations of a set of changes may yield different influ-ences of factors. We don’t see a solution to this weakness,but we reduce the problem by being transparent about thecategorisation.6. ACKNOWLEDGEMENTSWe thank prof. dr. Klaas van den Berg for valuable in-sights about measurement of software system properties. Wealso thank dr. Denis Verhoef and dr. Jan-Willem Hubbersfor valuable feedback on early drafts of this research design.We thank the anonymous reviewers for pointing out un-clear parts of the research design and for recommendingsome very interesting previous works.7. REFERENCES[1] Y. Ahn, J. Suh, S. Kim, and H. Kim. The softwaremaintenance project effort estimation model based onfunction points. Journal of Software Maintenance,15(2):71–85, 2003.[2] L. Bass, P. Clements, and R. Kazman. SoftwareArchitecture in Practice. Addison-Wesley, 2nd edition,2003.[3] M. A. Chaumun, H. Kabaili, R. K. Keller,F. Lustman, and G. Saint-Denis.Design properties and object-oriented softwarechangeability. In CSMR ’00: Proceedings of theConference on Software Maintenance andReengineering, page 45. IEEE Computer Society, 2000.[4] M.-A. Côté, W. Suryn, C. Y. Laporte, and R. A.Martin. The evolution path for industrial softwarequality evaluation methods applying ISO/IEC9126:2001 quality model: Example of MITRE’s SQAEmethod. Software Quality Journal, 13:17–30, 2005.[5] S. G. Eick, T. L. Graves, A. F. Karr, J. S. Marron,and A. Mockus. Does code decay? Assessing theevidence from change management data. IEEE Trans.Softw. Eng., 27(1):1–12, 2001.[6] N. E. Fenton and S. L. Pfleeger. Software Metrics: ARigorous and Practical Approach. PWS PublishingCo., 1998.[7] T. L. Graves and A. Mockus. Inferring change effortfrom configuration management databases. InMETRICS ’98: Proceedings of the 5th InternationalSymposium on Software Metrics, page 267. IEEEComputer Society, 1998.[8] M. Lehman. Laws of Software Evolution Revisited. InC. Montangero, editor, Software Process Technology(EWSPT 96), volume 1149 of Lecture Notes inComputer Science, pages 108–124, Nancy, France,1996. Springer-Verlag, Berlin.[9] A. Monden, D. Nakae, T. Kamiya, S. Sato, andK. Matsumoto. Software quality analysis by codeclones in industrial legacy software. In METRICS ’02:Proceedings of the 8th International Symposium onSoftware Metrics, 2002.[10] F. Niessink and H. van Vliet. Predicting maintenanceeffort with function points. In ICSM ’97: Proceedingsof the International Conference on SoftwareMaintenance, pages 32–39. IEEE Computer Society,1997.[11] R. K. Yin. Case study research: design and methods.Sage Publications, 3rd edition, 2003.388

Surveying the factors that influence maintainability: research design

Case study research: design and methods. Sage Publications, 3rd edition,

Design properties and object-oriented software changeability.

Does code decay? Assessing the evidence from change management data.

Inferring change eﬀort from conﬁguration management databases.

Laws of Software Evolution Revisited. In

Predicting maintenance eﬀort with function points.

Software Metrics: A Rigorous and Practical Approach.

Software quality analysis by code clones in industrial legacy software.

The evolution path for industrial software quality evaluation methods applying ISO/IEC 9126:2001 quality model: Example of MITRE’s SQAE method.

The software maintenance project eﬀort estimation model based on function points.

Surveying the factors that influence maintainability: research design

Abstract

Similar works

Full text

Available Versions

Crossref

University of Twente Research Information