This contribution presents initial results in the study of the relationship between size, effort, duration and number of contributors in eleven evolving Free/Libre Open Source Software (FLOSS) projects, in the range from approx. 650,000 to 5,300,000 lines of code. Our initial motivation was to estimate how much effort is involved in achieving a large FLOSS system. Software cost estimation for proprietary projects has been an active area of study for many years. However, to our knowledge, no previous similar research has been conducted in FLOSS effort estimation. This research can help planning the evolution of future FLOSS projects and in comparing them with proprietary systems. Companies that are actively developing FLOSS may benefit from such estimates. Such estimates may also help to identify the productivity ’baseline’ for evaluating improvements in process, methods and tools for FLOSS evolution

Fernandez-Ramil, Juan

Izquierdo-Cortazar, Daniel

Mens, Tom

English

Open Research Online (The Open University)

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsRelationship between size, effort, duration and numberof contributors in large FLOSS projectsConference or Workshop ItemHow to cite:Fernandez-Ramil, Juan; Izquierdo-Cortazar, Daniel and Mens, Tom (2008). Relationship between size, effort,duration and number of contributors in large FLOSS projects. In: 7th BElgian-NEtherlands software eVOLutionworkshop - BENEVOL 2008, 11-12 Dec 2008, Eindhoven, The Netherlands.For guidance on citations see FAQs.c© 2008 The AuthorsVersion: Accepted ManuscriptLink(s) to article on publisher’s website:http://www-set.win.tue.nl/benevol2008/Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.ukRelationship between Size, Effort, Duration andNumber of Contributors in Large FLOSS projectsJuan Fernandez-Ramil1,3, Daniel Izquierdo-Cortazar2 and Tom Mens31 The Open University, Milton Keynes, U.K. j.f.ramil@open.ac.uk2 Universidad Rey Juan Carlos, Madrid, Spain dizquierdo@gsyc.urjc.es3 Universite´ de Mons-Hainaut, Mons, Belgium {j.f.ramil,tom.mens}@umh.ac.be1 IntroductionThis contribution presents initial results in the study of the relationship between size,effort, duration and number of contributors in eleven evolving Free/Libre Open SourceSoftware (FLOSS) projects, in the range from approx. 650,000 to 5,300,000 lines ofcode. Our initial motivation was to estimate how much effort is involved in achievinga large FLOSS system. Software cost estimation for proprietary projects has been anactive area of study for many years (e.g. [1][2][3]). However, to our knowledge, no pre-vious similar research has been conducted in FLOSS effort estimation. This researchcan help planning the evolution of future FLOSS projects and in comparing them withproprietary systems. Companies that are actively developing FLOSS may benefit fromsuch estimates [4]. Such estimates may also help to identify the productivity ’base-line’ for evaluating improvements in process, methods and tools for FLOSS evolution.Table 1 shows the projects that we have considered, together with the programminglanguage(s) primarily used for each system and the time from the first known commit.Table 1. FLOSS systems studied and some of their characteristicsname primary description time fromlanguage 1st commit(years)Blender C/C++ cross-platform tool suite for 3D animation 6Eclipse Java IDE and application framework 6.9FPC Pascal Pascal compiler 3.4GCC C/Java/Ada GNU Compiler Collection 19.9GCL C/Lisp/ASM GNU Common Lisp 8.8GDB C/C++ GNU Debugger 9.5GIMP C GNU Image Manipulation Program 10.9GNUBinUtils C/C++ collection of binary tools 9.4NCBITools C/C++ libraries for biology applications 15.4WireShark C network traffic analyser 10XEmacs Lisp/C text editor and application development system 12.2The measurements extracted for this study are listed in Table 2. We used the SLOC-Count4 tool to measure the size of the software in lines of code. In order to measuresize in number of files, duration, effort and team size, we used CVSAnalY5. This toolstores information extracted from the version control log (CVS, Subversion or Git) ina MySQL database. Specifically, we indirectly used CVSAnalY by downloading datafrom FLOSSMetrics6. For mesuring EFFORT , possibly in a very approximate way,we added one contributor-month for each contributor making one commit or more ina given month. Then, we calculated the number of contributor-years by dividing by 12the total contributor-month values for each project.Table 2. Measured attributes for each systemattribute name description extracted using:KLOC physical lines of source code (in thousands) SLOCCountnetKLOC see explantion in textFILES total number of files in code repository CVSAnalYnetFILES see explanation in textEFFORT effort in contributor-years CVSAnalYDUR time length of ’active’ evolution in years CVSAnalYDEV number of unique contributors CVSAnalY2 Data filteringIn most of the projects, when looking at the total number of files (FILES) in the repos-itory per month, we observed a number of relatively large ‘jumps’. Figure 1 (left handside) shows this for project GCL. It is unlikely that the productivity of the team of con-tributors has increased so suddenly. It is more likely that each ‘jump’ reflects momentsin which the repository receives chunks of externally generated files. It could also bethat non-code files were added. We tried to filter out this phenomenon by subtractingthe size increments in the ‘jumps’. We also subtracted the initial size. We call the re-sult netFILES. For GCL, the result is presented on the right hand side of Figure 1.Since KLOC was measured only at one recent date, we also defined netKLOC asKLOC ∗ netFILESFILES .When measuring project life span or duration DUR since the first commit up toOct 2008 – for Eclipse the cut off month was April 2008 –, we excluded any apparentperiods of no commits or with monthly commits number much lower than during otherperiods. We did this for two of the projects, GCC and GCL. This can be seen on Figure 1for GCL. The left-hand-side plot shows the total duration. The right-hand-side showsthe period of active work, which determined the value of DUR (4.3 years instead of8.8). Similarly, for GCC we assumed a value of DUR of 11.2 instead of 19.9 years.4 www.dwheeler.com/sloccount/5 svn.forge.morfeo-project.org/svn/libresoft-tools/cvsanaly6 data.flossmetrics.org!"#!!!"$!!!"%!!!"&!!!"'!!!!"'#!!!"'$!!!"'%!!!"'" '(" #)" (*" $+" %'" *(" &)" +*"!"#$%&'()'*+%,'#(!-.,'/01'!"#!!"$!!!"$#!!"%!!!"%#!!"$" $&" %#" &'" ()" *$" '&" +#" )'"!"#$%&'()'*+%,'#(!-.,'/01'Fig. 1. Trend in FILES (left) and netFILES (right) per month for GCL.For DEV we counted the number of people making at least one commit since thestart of the repository. We computed two variants, one measuring all the contributors,DEV [100], and one measuring the number of developers which provided 80 percent ormore of the effort in contributor-months, or DEV [80], in order to exclude contributorswho may have made only a few commits.3 Preliminary resultsInitially, we studied the netFILES trends over months. We found a very good fit tolinear, quadratic or exponential models, with coefficient of determination7 (R2) valuesranging between 0.98 to 0.99. Five of the systems had trends that follow linear models,two follow sublinear (quadratic) and three superlinear (2 quadratic and one exponen-tial). The high goodness of fit suggests that size increase in these FLOSS systems takeplace at a predictable rate. This mixture of different trends is in agreement with previousempirical studies of FLOSS growth trends [5].As it is common practice in effort estimation models (e.g., COCOMO [2]), westudied the linear correlation between size (measured in KLOC and FILES) andEFFORT , DUR and DEV . Table 3 shows the parameters of linear models of theform y = (a ∗ size) + b and the corresponding a, b and R2 values, for the eleven sys-tems and for ten systems (excluding Eclipse) The exclusion of the data of the largestsystem, Eclipse, following its identification as a possible outlier in some of the scat-ter plots that we studied, is indicated by an ‘*’ in the attribute’s name (e.g. KLOC∗).Parameters a and b are not reported when R2 is less than 0.1 because they are not mean-ingful. The best models, indicated in bold, are obtained when Eclipse is excluded fromthe dataset.Contrary to what was expected, the sole removal of unusual ‘jumps’ in the monthlygrowth of the studied systems did not lead to visible improvements in the regressionmodels. It is the exclusion of Eclipse that leads to improvement in the correlation in13 out of 16 correlations studied. All the best regression results correspond to net7 This is a measure of goodness of fit, with values from 0, for a very poor fit, and 1 for a verygood fit.Table 3. Linear regression results - parameters a, b and R2 valuesEFFORT DUR DEV [100] DEV [80]size in: (a, b, R2) (a, b, R2) (a, b, R2) (a, b, R2)KLOC (0.0858, 29.909, 0.339) ( -, - , 0.012) (0.027, 88.538, 0.101) (0.0095, 29.57, 0.1331)FILES (0.0039, 119.58, 0.390) ( -, -, 0.001) (0.0017, 103.97, 0.219) (0.0006, 35.142, 0.2845)netKLOC (0.076, 110.39, 0.326) ( - , - , 0.048) (0.0287, 106.56, 0.139) (0.0105, 35.384, 0.196)netFILES (0.0035, 156.61, 0.313) ( - , - , 8.8E-05) (0.0016, 119.2, 0.185) (0.006, 40.066, 0.258)KLOC∗ (0.1327, -53.323, 0.387) (0.0015, 6.1233, 0.153) ( - , - , 0.088) ( - , - , 0.07)FILES∗ (0.0093, 3123,0.66) ( - , - , 0.06) (0.004, 62.831, 0.391) ( 0.0012, 24.787, 0.367)netKLOC∗ (0.1699, 14.247, 0.499) (0.0032, 5.4474, 0.525) (0.0626, 71.879, 0.196) (0.0183, 27.401, 0.185)netFILES∗ (0.0139, 51.455, 0.797) ( - , - , 0.094) (0.0143, -19.009, 0.565) (0.002, 25.998, 0.506)size and with Eclipse excluded. The best regression model obtained is the one in-volving EFFORT as a linear function of netFILES′ (R2 value of 0.797), that is,EFFORT = 0.0139 ∗ netFILES∗+51.455. According to this model, reaching say,8,000 files would require about (0.0139 ∗ 8000) + 51.455 or 163 contributor-years [6].The six worst regression models correspond to DUR vs size, indicating that FLOSSincrease in size at different rates across projects.With regards to external validity (generalisation), eleven arbitrarily chosen FLOSSconstitute a small sample to be able to generalise our conclusions. Regarding threatsto internal validity, EFFORT is likely to be an over-estimation of the actual effort.This is so because many FLOSS contributors are volunteers who work part-time ratherthan full time on the project. One way to improve measuring EFFORT would be toconduct a survey of FLOSS contributors to know better their work patterns and use thisknowledge to adjust our measurements. Surveys, however, may require considerable re-search effort and the willingness of FLOSS contributors to participate. Our EFFORTmeasurement may be more accurate for FLOSS projects like Eclipse, where a portionof contributors are full-time employees of a sponsoring company (in this case, IBM).Despite all our care, the tools used to extract and analyse the data may contain defectsthat may affect the results.4 Conclusion and further workThis paper reports on work in progress in the study of the relationship between size,effort and other attributes for large FLOSS, an issue that does not seem to have beenempirically studied. The preliminary results are encouraging and better models maybe obtained through refinement and improvement of the measurement methodology.If this were succesful, the models could provide a baseline to study, for example, thepossible impact of new or improved processes, methods and tools. The models mayalso help FLOSS communities in systematic planning of the future evolution of theirsystems. The best model excludes data from Eclipse. This suggests that independentmodels for very large systems (greater than 5 million LOC) or for systems based ondifferent technologies (Eclipse was the only Java based system in the sample) are worthexploring.We have started to compare how well existing cost estimation models based on pro-prietary projects (in particular, COCOMO [2]) correspond to our FLOSS data [6]. Inorder to increase the external validity of the results, we would need to study an ad-ditional number of FLOSS projects. The regression models we used are simple. Betterresults may be achievable by using robust regression [7]. Other suitable modelling tech-niques different than regression [8] could be evaluated against the data. Accuracy maybe improved by including other attributes that may impact productivity and duration.One of these is the so-called number of orphaned lines of code [9], the portion of thecode left behind by contributors who have left a project. Currently, our FILES mea-surement considers all files in the repository (i.e. code, configuration, data, web pages).It is likely that better results will be achieved by considering code files only. We wouldalso like to exclude any automatically generated files since they will bias the results(e.g., programming productivity may appear higher than it is). We also plan to exam-ine different approaches to extract the outliers from monthly growth trends. In Figure 1(right) one can identify ’jumps’ that were not apparent when looking at the unfiltereddata (left). One question is how to define formally what is a ‘jump’. In particular, oneneeds to check manually the version repositories to confirm whether the ‘jumps’ areactually corresponding to the inclusion of externally generated code or libraries.AcknowledgementsThis contribution has been prepared during the research visits of two of the authors (DI and JFR) to UMH. One of the au-thors (JFR) acknowledges the Belgian F.R.S.-F.N.R.S. for funding through postdoctoral scholarship 2.4519.05. One of theauthors (DI) has been funded in part by the European Commission, under the FLOSSMETRICS (FP6-IST-5-033547) andQUALOSS (FP6-IST-5-033547) projects, and by the Spanish CICyT, project SobreSalto (TIN2007-66172). TM acknowl-edges partial funding by the Actions de Recherche Concerte´es – Ministe`re de la Communaute´ franc¸aise - Direction ge´ne´ralede l’Enseignement non obligatoire et de la Recherche scientifique.References1. Wolverton, R.W.: The cost of developing large-scale software. IEEE Trans. Computers C-23(6) (June 1974) 615 – 6362. Boehm, B.: Software Engineering Economics. Prentice Hall (1981)3. Jones, C.: Estimating Software Costs - Bringing Realism to Estimating. McGraw Hill (April2007)4. Amor, J.J., Robles, G., Gonzalez-Barahona, J.M.: Effort estimation by characterizing devel-oper activity. In: EDSER ’06: Proceedings of the 2006 international workshop on economicsdriven software engineering research, New York, NY, USA, ACM (2006) 3–65. Herraiz, I., Robles, G., Gonzalez-Barahona, J.M., Capiluppi, A., Ramil, J.F.: Comparisonbetween SLOCs and number of files as size metrics for software evolution analysis. In: Proc.European Conf. Software Maintenance and Reengineering (CSMR), Bari, Italy (March 2006)6. Fernandez-Ramil, J., Izquierdo-Cortazar, D., Mens, T.: How much does it take to achieve onemegaloc in open source? Submitted for publication (November 2008)7. Lawrence, K.D., Arthur, J.L.: Robust Regression: Analysis and Applications. CRC Press(1990)8. MacDonell, S.G., Gray, A.R.: Alternatives to regression models for estimating softwareprojects. In: Proceedings of the IFPUG Fall Conference, Dallas TX, IFPUG. (1996)9. Izquierdo-Cortazar, D., Robles, G., Ortega, F., Gonzalez-Barahona, J.: Using software ar-chaelogy to measure knowledge loss in software projects due to developer turnover. In: Pro-ceedings of the Hawaii International Conference on System Sciences (HICSS-42), Hawaii,USA, forthcoming (January 2009)

Relationship between size, effort, duration and number of contributors in large FLOSS projects

http://oro.open.ac.uk/27317/1/benevol08-final_version.pdf

Relationship between size, effort, duration and number of contributors in large FLOSS projects

Abstract

Similar works

Full text

Available Versions

Open Research Online (The Open University)