Kouters, E.T.M.

Serebrenik, A.

Vasilescu, B.N.

English

Pure OAI Repository

 Who’s who on Gnome mailing lists : identity merging on alarge data setCitation for published version (APA):Kouters, E. T. M., Vasilescu, B. N., & Serebrenik, A. (2013). Who’s who on Gnome mailing lists : identitymerging on a large data set. In T. Mens, M. Claes, M. Goeminne, & S. Drobisz (Eds.), 12th Belgian-NetherlandsSoftware Evolution Seminar (BENEVOL'13, Mons, Belgium, December 16-17, 2013) (pp. 31-32). Université deMons.Document status and date:Published: 01/01/2013Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)Please check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverneTake down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.Download date: 08. Feb. 2024Who’s Who on GNOME Mailing Lists:Identity Merging on a Large Data SetErik Kouters, Bogdan Vasilescu, Alexander SerebrenikTechnische Universiteit Eindhoven,Den Dolech 2, P.O. Box 513,5600 MB Eindhoven, The Netherlandse.t.m.kouters@student.tue.nl, {b.n.vasilescu, a.serebrenik}@tue.nlINTRODUCTIONAn open-source software project often uses multiple channels of communication (e.g., software repos-itory, mailing list, bug tracker) in each of which an individual has to create a new identity. People mightswitch between different email addresses (e.g., private and corporate email addresses), causing a mailinglist to contain multiple identities for a single individual. Identity merging attempts to solve the challengeof aggregating data by individuals instead of identities (e.g., by merging identities throughout multipleconferences [1]).Identity merging is the process of identifying which identities belong to the same individual. Aliasesare values identifying an individual, commonly found in the form of different 〈name, emailAddress〉tuples in mailing lists and source code repositories. By using an identity merging algorithm, two aliaseswill be matched based on the similarity determined by the algorithm. When two aliases are matchedpositive, they are considered as belonging to the same individual.Existing identity merging algorithms are not very robust to noise (e.g., misspelling, diacritics, nick-names, punctuation). We introduce an algorithm which is inspired by information retrieval [2]. We haveevaluated this algorithm’s performance, and compared it to three existing identity merging algorithms.We present preliminary results of evaluating the performance on a large data set. A priori it is unknownhow the algorithms will perform on a data set with this order of magnitude.The identity merging was performed on GNOME’s mailing list archives, which were extracted on April11, 2012. At that time, the mailing list archives contained 2, 202, 746 emails which were sent by 73, 920distinct email addresses. A previously studied data set, GNOME’s software repository logs, was smallerin an order of magnitude, and is expected to contain less noise [2]. The algorithm we introduced isdesigned to be more robust to the types of noise found in the mailing list archives than the existingidentity merging algorithms, and therefore performs better than these existing algorithms. As the data setgrows, more people with the same name will occur in the data; existing algorithms do not take this intoaccount (e.g., aliases containing only a common first name will be matched, generating false positives).EXISTING ALGORITHMSWe consider three existing identity merging algorithms: Simple algorithm by Goeminne and Mens [3],an algorithm by Bird et al. [4], and our interpretation of Bird’s algorithm [2]. After implementing andevaluating performance of our interpretation of Bird’s algorithm [2], we acquired Bird et al.’s originalcode. After evaluating this original code, which we will refer to as Bird Original, the results comparedto our interpretation of Bird’s algorithm were very different. For the sake of completeness, we decidedto keep both versions for evaluation.The simple algorithm bases its merging on the name and email address prefix using simple heuristics. Iftwo aliases share any of the elements, name or prefix, they are considered as a positive match. Additionally,the algorithm has a threshold minLen that filters out short words that would easily match everythingtogether.Bird’s algorithms use more complex heuristics such as splitting the name into the first and last names,and comparing names by using the first letter of the first name, concatenated with the last name (e.g., ErikKouters ⇒ ekouters). These rules are used on both the name and the email address prefix. Additionally,the algorithms use the Levenshtein distance similarity threshold, levThr, to allow for differences in names(e.g., as a result of misspelling).31Algorithm Precision Recall F-measureSimple Algorithm 0.35 0.90 0.50Bird’s Algorithm 0.18 0.92 0.30Bird’s Original Algorithm 0.41 0.90 0.56Kouters’ Algorithm 0.67 0.98 0.80TABLE ITHE RESULTS OF EACH ALGORITHM ON GNOME’S MAILING LIST ARCHIVES.THE ALGORITHMWe introduce an algorithm inspired by information retrieval that is more robust to noise and largerdata sets. The bigger a data set becomes, the more likely two people with the same first/last name occurs.Because of the heuristics used by the existing algorithms, it is expected they scale badly when the dataset grows by an order of magnitude.The data set, consisting of a list of aliases, is transformed to Vector Space Model, essentially creatinga term-document matrix. This matrix is then augmented by the Levenshtein distance similarity used byBird et al., due to name differences as a result misspelling. To add robustness with respect to frequentnames, we apply the term frequency – inverse document frequency (tf-idf) model which scales to themost frequent name. Finally, the similarity between the aliases are computed using the cosine similarity.The algorithm accepts three different parameters: minLen filters out short words, similar to the simplealgorithm; levThr adds robustness with respect to misspelling, similar to Bird et al.’s algorithms; cosThrdefines the threshold of similarity when two aliases are considered as positive or negative matches.EVALUATIONThe algorithms described in this article have been evaluated on GNOME’s mailing list archives. Thepreliminary results are shown in Table I. The parameters used for the algorithms are minLen = 2 forthe simple algorithm; levThr = 0.8 for Bird et al.’s algorithms; minLen = 2, levThr = 0.75 andcosThr = 0.75 for Kouters’ algorithm. The choice of these parameters was based on earlier research[2]. Table I refers to Kouters’ algorithm which is the algorithm described in the previous section.All algorithms have a high recall on GNOME’s mailing list archives. Even the simple algorithm isable to achieve a high recall with its simple heuristics, showing that the people using the mailing listsare consistently using names when sending emails. Furthermore, Bird et al.’s algorithms do not have amuch higher recall than the simple algorithm, despite the more complex heuristics. Kouters’ algorithmhas the highest recall of all algorithms, even higher than Bird et al.’s algorithms. Bird et al.’s algorithmsbase their heuristics on the first and last names. However, in some cultures it is common to have a namewith more than two words (e.g., a Spanish name typically has a first name and two surnames). Betterhandling of these type of names might improve recall for Bird et al.’s algorithms.The precision is what the algorithms differ in the most. Low precision is caused by a high number offalse positive matches. As a result of complex heuristics, our interpretation of Bird’s algorithm is verysensitive to false positives: People with the same last name and first letter of the first name are matched(e.g., Aaron Smith, Alex Smith). Kouters’ algorithm prevents matching on common names by applyingthe tf–idf model; names that occur often are decreased in value. This characteristic scales well with alarger data set.FUTURE WORKThe parameters that were used for the preliminary results presented in Table I were chosen based onearlier research that included a large-scale experiment that tested all combinations of parameters. Doinga similar large-scale experiment on the large data set is considered future work; we do not know howthe parameters will affect the results on a large data set. Ideally, an identity merging algorithm has oneoptimal parameter combination that performs best on all data sets.REFERENCES[1] B. Vasilescu, A. Serebrenik, M. Goeminne, and T. Mens, “On the variation and specialisation of workload – A case study ofthe Gnome ecosystem community,” Empirical Software Engineering, pp. 1–54, 2013.[2] E. Kouters, B. Vasilescu, A. Serebrenik, and M. G. J. van den Brand, “Who’s who in GNOME: Using LSA to merge softwarerepository identities,” in Proc. ICSM, 2012, pp. 592–595.[3] M. Goeminne and T. Mens, “A comparison of identity merge algorithms for software repositories,” Science of ComputerProgramming, 2011.[4] C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A. Swaminathan, “Mining email social networks,” in Proc. MSR. ACM,2006, pp. 137–143. [Online]. Available: http://doi.acm.org/10.1145/1137983.113801632

Who’s who on Gnome mailing lists : identity merging on a large data set

https://pure.tue.nl/ws/files/3838598/2621322772840.pdf

Who’s who on Gnome mailing lists : identity merging on a large data set

Abstract

Similar works

Full text

Available Versions

Pure OAI Repository