Homoglyphs can be used for disguising plagiarized text by

replacing letters in source texts with visually identical letters from other

scripts. Most current plagiarism detection systems are not able to detect

plagiarism when text has been obfuscated using homoglyphs. In this

work, we present two alternative approaches for detecting plagiarism in

homoglyph obfuscated texts. The first approach utilizes the Unicode list

of confusables to replace homoglyphs with visually identical letters, while

the second approach uses a similarity score computed using normalized

hamming distance to match homoglyph obfuscated words with source

words. Empirical testing on datasets from PAN-2015 shows that both

approaches perform equally well for plagiarism detection in homoglyph

obfuscated texts

Alvi, F.

Clough, P.

Stevenson, R.M.

White Rose Research Online

This is a repository copy of Plagiarism Detection in Texts Obfuscated with Homoglyphs.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/112665/Version: Accepted VersionProceedings Paper:Alvi, F., Stevenson, R.M. orcid.org/0000-0002-9483-6006 and Clough, P. (2017) PlagiarismDetection in Texts Obfuscated with Homoglyphs. In: Jose, J.M., Hauff, C., Altıngovde, I.S., Song, D., Albakour, D., Watt, S. and Tait, J., (eds.) ECIR 2017: Advances in Information Retrieval. 39th European Conference on Information Retrieval, 08-13 Apr 2017, Aberdeen,Scotland . Lecture Notes in Computer Science . Springer, Cham , pp. 669-675. ISBN 978-3-319-56608-5 eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Plagiarism Detection in Texts Obfuscated withHomoglyphsFaisal Alvi,a,b Mark Stevenson,a and Paul Cloughaa University of Sheffield, Sheffield S10 2TN, United Kingdom,b King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia.{falvi1, mark.stevenson, p.d.clough}@sheffield.ac.ukAbstract. Homoglyphs can be used for disguising plagiarized text byreplacing letters in source texts with visually identical letters from otherscripts. Most current plagiarism detection systems are not able to detectplagiarism when text has been obfuscated using homoglyphs. In thiswork, we present two alternative approaches for detecting plagiarism inhomoglyph obfuscated texts. The first approach utilizes the Unicode listof confusables to replace homoglyphs with visually identical letters, whilethe second approach uses a similarity score computed using normalizedhamming distance to match homoglyph obfuscated words with sourcewords. Empirical testing on datasets from PAN-2015 shows that bothapproaches perform equally well for plagiarism detection in homoglyphobfuscated texts.1 IntroductionThe notion of ‘Disguised Plagiarism’ refers to a class of methods used for inten-tionally hiding text that has been copied [8]. Furthermore, ‘Technical Disguise’is a particular form of disguised plagiarism, wherein obfuscation techniques areused in order to evade the detection of plagiarized text by changing the compu-tational representation of text. An important method for technically disguisingtext is to substitute characters visually identical to other characters in someother script (i.e., homoglyphs) [5]. For example, the Latin character ‘p’ (Uni-code U+160) and the Cyrillic ‘p’ (Unicode U+0440) have identical glyphs butdistinct Unicode values, making the words ‘paypal’ and ‘paypal’ appear identicalto a human evaluator, but undetectable to an automated plagiarism detectionsystem that has not been designed to deal with such changes. In tests of sev-eral leading plagiarism detection systems most were unable to detect similaritiesbetween source and plagiarized texts obfuscated using homoglyphs [7, 12].In this work we present two alternate approaches for plagiarism detection inhomoglyph obfuscated texts: (1) by using the Unicode list of ‘confusables’ to findand replace homoglyphs with visually identical ASCII letters; and (2) by usinga measure of similarity based on normalized hamming distance to match homo-glyph obfuscated words with source words. Our work shows both approaches per-form equally well for detecting plagiarism in homoglyph obfuscated texts. Bothapproaches have their particular advantages and limitations and may thereforebe applicable in specific application scenarios in homoglyph obfuscated texts.2 Related WorkHomoglyph substitution has been used as part of standard tests for plagiarismdetection systems. For example, Gillam et al. [5] used homoglyph substitutionas an obfuscation strategy for testing plagiarism detection systems. Their resultsdemonstrated that six out of seven plagiarism detection systems were unable todetect any similarity between source and substituted text. Figure 1 gives a listof characters used in their work, with the number of instances of each charactervisually detected by human evaluators stated as well. In the annual ‘PlagiarismDetection Software Test’ by Weber Wulff et al. [12], 13 out of 15 plagiarismdetection systems failed to report any similarity between a given text sourceand its homoglyph substituted version.Fig. 1. Replacement Letters for Visually Similar Characters from [5]Heather et. al [6] describe a variety of techniques for technically disguisingplagiarized text, which include: modifying the character map, rearranging theglyphs in fonts, replacing text with graphical symbols, and inserting charactersin background (white) font between words. Kakoneen and Mozgovoy [7] alsodiscuss a number of ‘technical tricks’ that can be used to obfuscate texts, in-cluding: (1) the insertion of similar looking characters from foreign alphabets(homoglyph substitution); (2) the insertion of background colored characters inbetween spaces; and (3) the use of scanned images in place of text. According totheir results, “None of the evaluated systems were able to detect any instancesof plagiarism from the documents.”In addition to text obfuscation during plagiarism, homoglyphs have also beenused in IDN (Internationalized Domain Name) homograph attacks1 used to di-rect users towards alternative websites. With such an attack, users could bedirected towards the website ‘paypal.com’ which is a Cyrillic substituted versionof the Latin ‘paypal.com’. Existing approaches to deal with IDN homoglyph at-tacks include: (1) Punycode [3] that converts non-ASCII characters into ASCIIcharacters irreversibly (e.g., Ǵooǵle is converted to the ASCII ‘xn–oole-ksbc’);(2) coloring-based strategies that distinguish homoglyphs by assigning variouscolors to foreign script characters [13]; and (3) a Unicode character similaritylist (UC-SimList) [4] to detect homoglyphs in URLs. Some of these approachesmight not be useful for plagiarism detection e.g. Punycode results in loss of in-formation, and coloring requires visual inspection. However, the idea of using alist of Unicode equivalents for detecting IDN homograph attacks can be utilizedfor plagiarism detection in homoglyph obfuscated texts.1 https://en.wikipedia.org/wiki/IDN_homograph_attack3 Methodology3.1 ResourcesThe Unicode List of Confusable Characters. Several lists of homoglyph-alphabet pairs are freely available (e.g., homoglyphs.net). The Unicode consor-tium has released a list of confusables, which is a list of visually similar characterpairs that includes homoglyphs and their corresponding Latin letters [1]. We useVersion 9.0.0 of the list of confusables containing 6167 pairs of confusable char-acters. Figure 2 shows a partial list of letters similar to the letter ‘p’ taken fromthis list.Fig. 2. Visually Confusable characters for ‘p’ from the Unicode List of ConfusablesEvaluation Dataset. We use PAN-2015 evaluation lab [11] dataset submissionby Palkovskii and Belov [10] which is based on the PAN-2013 training datasetwith characters in the suspicious documents replaced with homoglyphs. Thisdataset consists of 5185 document pairs divided into five categories of ‘no pla-giarism’, ‘no obfuscation’, ‘translation’, ‘random’ and ‘summary’ obfuscation.3.2 ApproachesApproach 1: Unicode Confusables. In our first approach (shown in Figure3), we find and replace every non-ASCII character in the suspicious documentswith the corresponding visually matching character from the list of confusablecharacters. This process replaces homoglyphs in the text of the suspicious doc-uments with visually similar ASCII characters. The resulting suspicious docu-ments can then be compared with the source documents for similarity. In ourapproach we use word trigram similarity as the seeding strategy, with mergingand filtering to discard small matches as false positives [2].        Suspicious Documents             Source Documents                   Report Similarity Use the list of confusables to replace homoglyphs  Apply word trigram similarity with merging and filtering  Fig. 3. Block Diagram for Plagiarism Detection using the List of ConfusablesApproach 2: Normalized Hamming Distance. Our second approach usesnormalized hamming distance as an approximate string matching technique.Such techniques are well-suited for this task since homoglyph substitution maypartially change the structure of a word. Hamming Distance (when applied tostrings of characters) detects the number of substitutions (replacements) fromone string into another by finding the number of positions where the two stringsdiffer [9]. We use a similarity score (simh) computed using normalized hammingdistance, defined between two words w1, w2 of equal length as:simh(w1, w2) = 1− Number of substitutions(w1, w2)/length(w1).Compared to other approximate string similarity measures, normalized ham-ming distance has the advantage of significantly reducing the number of falsepositives generated. Hamming distance is undefined for strings of unequal length,(we consider simh = 0 in this case), whereas these strings might be marked assimilar using alternative string similarity techniques, such as character skip grammatching. For example, simh(play, plays) = 0, while simh(play, play) = 0.75.Normalized hamming distance similarity (simh) is used to compare eachword in the suspicious document with the words in the source document. Ifa pair of words have a value of simh greater than or equal to a particularthreshold, we consider them as similar. The threshold value depends on theextent of homoglyph substitution in the dataset. For example, if most of theletters in each word have been replaced by homoglyphs, then a lower thresholdvalue will be required to match these words.Using this procedure for approximate matching of words instead of exactmatching, we apply word trigram similarity with merging and filtering (as usedin the list-based approach) to find the plagdet score between the source and sus-picious documents. We conduct our experiments on the PAN-2015 dataset usedin the list-based approach. Regarding the threshold value of simh for matchingwords in our experiments, we do not pre-select a value for this threshold. Insteadwe calculate plagdet scores for the entire dataset for a range of values of simhas shown in Figure 4.4 Results and Discussion4.1 Approach 1: Unicode ConfusablesTable 1 shows the results of plagiarism detection in terms of Precision, Recalland Plagdet [11] scores. It can be seen that except for summary obfuscation,Plagdet scores for all other categories including that for the entire dataset aremoderately high (≥ 0.60). This can be compared with the performance of mostof the PAN approaches from 2012-2014 [11] on this dataset where the reportedPlagdet scores were mostly 0, suggesting a significant improvement.During the homoglyph replacement phase using the list-based approach, weobserved that a number of replacements were also made for non-Latin charac-ters in suspicious documents which were not intended as homoglyphs in sourcedocuments. For example, the currency symbol ‘’ was replaced by a ‘c’. ThisTable 1. Plagdet-scores using the Homoglyph Replacement ApproachDataset No Obf. Random Obf. Transl. Obf. Summ. Obf.Precision 0.772 0.663 0.953 0.781 0.826Recall 0.727 0.988 0.667 0.643 0.107Plagdet 0.670 0.717 0.707 0.632 0.150observation suggests that the proposed approach of using a list of homoglyph-alphabet pairs to replace characters may not work well when the source textcontains a large number of foreign characters, since these might be convertedto ASCII characters in the substitution phase. However, the approach can beimproved by searching through the source documents to distinguish homoglyphsfrom true source non-Latin characters, at the cost of increased computation time.Fig. 4. Plagdet scores for Plagiarism Detectionusing Normalized Hamming Distance SimilarityCategory PlagdetEntire Dataset 0.644No Obfuscation 0.662Random Obf. 0.688Translate. Obf. 0.626Summary Obf. 0.142Table 2. Plagdet scores forthe case when simh = 0.4504.2 Approach 2: Normalized Hamming DistanceFigure 4 shows Precision, Recall and Plagdet scores for various values of simhthreshold. It can be seen that a threshold value for simh ≈ 0.45 is giving thehighest Plagdet score of 0.644. Table 2 gives Plagdet scores for each category ofplagiarism in the dataset for a threshold value of 0.45. Similar to Table 1, weobserve that except for summary obfuscation, most of these values are moder-ately high (≥ 0.6). Although the scores in Table 1 are somewhat higher thanthose in Table 2, the differences are small enough to consider performance ofthe approaches to be similar for detecting plagiarism in homoglyph obfuscatedtexts. From Figure 4 we observe that a careful selection of threshold value forsimh is important. The Plagdet score rapidly decreases after simh = 0.5 sincehigher threshold values increase the number of true matches being rejected. Forlarge datasets, this problem can alleviated by first applying the approach on asmaller collection of training documents to obtain a suitable initial estimate forthe threshold value.5 Conclusions and Future WorkThe development of techniques for automated plagiarism detection continuesto be an active area of research. In this work we presented two approaches forplagiarism detection in homoglyph obfuscated texts which perform equally wellfor plagiarism detection. One approach utilizes the Unicode list of confusablesto replace homoglyphs with visually identical letters; the other approach uses asimilarity score computed using normalized hamming distance. For future work,improvised versions of these approaches can be incorporated into a set of ap-proaches for detecting multiple forms of technical disguise.References1. Unicode List of Visually Confusable Characters. http://www.unicode.org/Public/security/9.0.0/confusables.txt, Online, Accessed: 2016-10-192. Alvi, F., Stevenson, M., Clough, P.D.: Hashing and Merging Heuristics for TextReuse Detection. In: Working Notes for CLEF 2014 Conference. pp. 939–946 (2014)3. Costello, A.: RFC3492-Punycode: A Bootstring encoding of Unicode for Inter-nationalized Domain Names in Applications (IDNA). Network Working Group.http://www.ietf.org/rfc/rfc3492.txt (2003), Online: Accessed: 2016-10-194. Fu, A.Y., Deng, X., Wenyin, L.: REGAP: A Tool for Unicode-Based Web IdentityFraud Detection. Journal of Digital Forensic Practice 1(2), 83–97 (2006)5. Gillam, L., Marinuzzi, J., Ioannou, P.: Turnitoff–Defeating Plagiarism DetectionSystems. In: Proceedings of the 11th Higher Education Academy-ICS Annual Con-ference. Higher Education Academy (2010)6. Heather, J.: Turnitoff: Identifying and Fixing a Hole in Current Plagiarism De-tection Software. Assessment & Evaluation in Higher Education 35(6), 647–660(2010)7. Kakkonen, T., Mozgovoy, M.: Hermetic and Web Plagiarism Detection Systemsfor Student Essays An Evaluation of the State-of-the-Art. Journal of EducationalComputing Research 42(2), 135–159 (2010)8. Meuschke, N., Gipp, B.: State-of-the-Art in Detecting Academic Plagiarism. Inter-national Journal for Educational Integrity 9(1) (2013)9. Navarro, G.: A Guided Tour to Approximate String Matching. ACM ComputingSurveys (CSUR) 33(1), 31–88 (2001)10. Palkovskii, Y., Belov, A.: Submission to the 7th International Competitionon Plagiarism Detection. http://www.uni-weimar.de/medien/webis/events/pan-15 (2015), Online, Accessed: 2016-10-1511. Potthast, M., Göring, S., Rosso, P., Stein, B.: Towards Data Submissions for SharedTasks: First Experiences for the Task of Text Alignment. In: Working Notes Papersof the CLEF 2015 Evaluation Labs. CEUR Workshop Proceedings (Sep 2015)12. Weber-Wulff, D., Möer, C., Touras, J., Zincke, E.: Plagiarism Detection Soft-ware Test 2013 (2013), http://plagiat.htw-berlin.de/software-en/test2013/report-2013/, Online, Accessed: 2016-10-1513. Wenyin, L., Fu, A.Y., Deng, X.: Exposing Homograph Obfuscation Intentions byColoring Unicode Strings. In: Asia-Pacific Web Conf. pp. 275–286. Springer (2008)

Plagiarism Detection in Texts Obfuscated with Homoglyphs

https://eprints.whiterose.ac.uk/112665/1/paper_247v2.pdf

Plagiarism Detection in Texts Obfuscated with Homoglyphs

Abstract

Similar works

Full text

Available Versions

White Rose Research Online