This paper introduces a new resource for computing textual complexity. It consists in a Python library for calculating different complexity metrics for several languages from plain texts. The resource has been made available to the research community and provides all needed instructions for its installation and use. To our knowledge, it is the first time a resource like this is published, so we expect many researchers can profit from it.Este artículo presenta un nuevo recurso para el cálculo de la complejidad textual. Se trata de una biblioteca de programación en Python que facilita el cómputo de distintas métricas de complejidad para varios idiomas a partir de textos en lenguaje natural. El recurso se ha liberado para su uso por parte de la comunidad científica y proporciona todas las instrucciones necesarias para su instalación y aprovechamiento. Hasta donde sabemos, es la primera vez que un recurso así está disponible, por lo que esperamos sea de utilidad.This work has been partially supported by Fondo Europeo de Desarrollo Regional (FEDER), LIVING-LANG project (RTI2018-094653-B-C21) from the Spanish Government

Collado-Montañez, Jaime

López-Anguita, Rocío

Montejo Ráez, Arturo

English

Repositorio Institucional de la Universidad de Alicante

The Text Complexity LibraryBiblioteca de Complejidad TextualRoc´ıo Lo´pez-Anguita, Jaime Collado-Montan˜ez, Arturo Montejo-Ra´ezCentre for Advanced Information and Communication Technologies (CEATIC)University of Jae´n, Spain{rlanguit, jcollado, amontejo}@ujaen.esAbstract: This paper introduces a new resource for computing textual complexity.It consists in a Python library for calculating different complexity metrics for severallanguages from plain texts. The resource has been made available to the researchcommunity and provides all needed instructions for its installation and use. To ourknowledge, it is the first time a resource like this is published, so we expect manyresearchers can profit from it.Keywords: Demostration, linguistic resources, textual complexity, lexical analysisResumen: Este art´ıculo presenta un nuevo recurso para el ca´lculo de la compleji-dad textual. Se trata de una biblioteca de programacio´n en Python que facilita elco´mputo de distintas me´tricas de complejidad para varios idiomas a partir de textosen lenguaje natural. El recurso se ha liberado para su uso por parte de la comu-nidad cient´ıfica y proporciona todas las instrucciones necesarias para su instalacio´ny aprovechamiento. Hasta donde sabemos, es la primera vez que un recurso as´ı esta´disponible, por lo que esperamos sea de utilidad.Palabras clave: Demostracio´n, recursos lingu¨´ısticos, complejidad textual, ana´lisisle´xico1 IntroductionReading comprehension and reading compe-tence are complex processes that are closelyrelated, according to Pe´rez (2014), as is theconcept of readability. The reading compre-hension is close to the reader’s capabilitiesand the latter is an objective view of the com-plexity of the text.Determining the readability of a text isnot a simple task, as each reader has differ-ent skills or limitations (Cain, Oakhill, andBryant, 2004). It is usually determined bylinguistic features, which are usually groupedinto those related to grammar (in otherwords, syntax) and those related to the lex-icon (i.e. vocabulary) (Alliende Gonza´lez,1994).Currently, we consider that measures ofcomplexity can be a convenient way to modelnatural language in certain applications, suchas authorship detection, text selection forpeople with difficulties associated with lan-guage disorders (autism, cerebral palsy...),or early detection of cognitive impairments,such as Alzheimer’s. Therefore, in this articlewe present a “demo” paper, which consists of12 of the most widely used metrics for lexi-cal and syntactic readability. These measuresand their interpretation are presented below,as well as details on the use of the library.2 Complexity metrics providedIn this section, we introduce the dif-ferent complexity metrics offered in thisPython library, proposed by different au-thors, for different languages (Spanish, En-glish, French...).Lexical complexity: The lexical complex-ity of a text, determined by the frequencyof use and lexical density, was proposed byAnula (2008). It is based on the number ofdifferent content words per sentence (LexicalComplexity Index, LC ) and on measuring thenumber of low frequency words per 100 con-tent words (Index of Low Frequency Words,ILFW ). Consequently, the higher the LC in-dex, the greater the difficulty in reading com-prehension.Spaulding readability: Commonlyknown as the SSR Index, it was proposed bySpaulding (1956). It focuses on measuringvocabulary and sentence structure to predictProcesamiento del Lenguaje Natural, Revista nº 65, septiembre de 2020, pp. 127-130 recibido 28-03-2020 revisado 30-05-2020 aceptado 30-05-2020ISSN 1135-5948. DOI 10.26342/2020-65-19 © 2020 Sociedad Española para el Procesamiento del Lenguaje Naturalthe relative difficulty of a text’s readability.Its formula is an empirically adjusted mea-sure to try to keep the score between 0 and1.Complexity of sentences: The SentenceComplexity Index (SCI) was proposed by An-ula (2008), as a measure of the complexity ofsentences in a literary text aimed at secondlanguage learners.This syntactic complexity measure focuseson measuring the number of words per sen-tence, thus obtaining the sentence length in-dex (Average Sentence Length, ASL), andthe number of complex sentences per sen-tence, from a complex sentence index (Com-plex Sentences, CS ).Automated Readability Index (ARI):Senter and Smith (1967) proposes one of themost used indexes due to its ease of calcula-tion, the Automated Readability Index, bet-ter known as ARI (Automated Readability In-dex ). This index measures the difficulty of atext from the average number of characters(letters and numbers) per word and the av-erage number of words per sentence.Dependency tree depth: This measurewas proposed by Saggion et al. (2015). Itis a very useful metric to capture syntac-tic complexity: long sentences can be syn-tactically complex or contain a large numberof modifiers (adjectives, adverbs or adverbialphrases). It complements the ASL measure,as it captures syntactic complexity in termsof recursive or nested structures.Punctuation Marks: This measure wasalso proposed by Saggion et al. (2015). Inthe complexity of a text, the average numberof punctuation marks is used as one of theindicators of the simplicity of the text.Readability of Ferna´ndez-Huerta:Blanco Pe´rez and Gutie´rrez Couto (2002) yRamı´rez-Puerta et al. (2013) propose thismeasure of complexity as an adaptation toSpanish of Flesch’s readability test (Flesch(1948)).Readability of Flesch-Szigrist (IFSZ):The works of Barrio-Cantalejo et al. (2008)and Ramı´rez-Puerta et al. (2013) proposethe Flesch-Szigristzt readability index as amodification of the Flesch formula (Flesch,1948) adapted to Spanish by Szigriszt-Pazosin 1993. This index is currently considereda reference for the Spanish language. It fo-cuses on measuring the number of syllablesper word and the number of sentences perword in the text.Comprehensibility of Gutie´rrez dePolini: This metric, originally developed in1972, is not an adaptation of English, butwas created from the beginning for Spanish(Rodr´ıguez, 1980). It focuses on measuringthe average number of letters per word andthe average number of words per sentence.µ Readability: It is a formula to calculatethe readability of a text. It provides an in-dex between 0 and 100 and was developedby Mun˜oz (2006). This measure focuses onmeasuring the number of words, the averagenumber of letters per word and their vari-ance.Minimum age to understand: In workof Garc´ıa Lo´pez (2001) we can find anotherformula to measure the age needed to under-stand a text. It is, again, an adaptation intoSpanish of Flesch’s original formula (Flesch(1948)) for English. It measures the averagenumber of syllables per word and the averagenumber of words per sentence to obtain theminimum age needed to understand a text.SOL Readability: Contreras et al. (1999)proposes the SOL metric as an adaptation toSpanish of the SMOG formula proposed byMc Laughlin (1969). It measures the read-ability of a text by means of grade level,which is the number of years of schooling re-quired to understand the text.Years Crawford: This measure was pro-posed by Alan N. Crawford in 1989 (Craw-ford, 1984). It is used to calculate the yearsof school required to understand a text. Mea-sures the number of sentences per hundredwords and the number of syllables per hun-dred words.3 How to obtain itThe library has been released under the Gen-eral Public License (GPL v3.0) license1 andcan be downloaded or cloned from its pub-lic repository2. The library will be updatedwith new features in the future, and you canalways get the latest version from that link.1http://www.gnu.org/licenses/gpl.html2https://gitlab.ujaen.es/amontejo/text-complexityRocío López Anguita, Jaime Collado Montañez, Arturo Montejo Ráez1284 InstallationIn order to use this library, you first need toinstall some previous requirements:• NumPy, Scipy, Pandas, Matplotlib andOpenpyxl for python3 have to be in-stalled in your system.• The FreeLing (Padro´ and Stanilovsky,2012) package, which is a library provid-ing language analysis services that our li-brary makes use of. In order to install it,you have to follow its installation man-ual under the project’s GitHub page35 Usage examplesIn order to test the library and teach how touse it, we have prepared some testing textsfor Spanish under the ./texts folder (youcan use your own texts by modifying thesefiles or adding more to that location).To compute complexity metrics on thesetext samples, modify the FREELINGDIR vari-able in the TextComplexityFreeling.pyscript (line 18) to your own FreeLing instal-lation directory (/usr/local by default).Then, if you run the Jupyter notebookexamples.ipynb you should get some tableswith the metrics for each text provided. Thescript will also generate three MS Excel filescontaining the results in your project’s folder.To use it in your Python scripts, this is assimple as follows:import TextComplexityFreeling as TCF# Create the text complexity calculatortc = TCF.TextComplexityFreeling()# Load text to analyzetext_processed = tc.textProcessing(text)# Compute different metricspmarks = tc.punctuationMarks()lexcomplexity = tc.lexicalComplexity()ssreadability = tc.ssReadability()sencomplexity = tc.sentenceComplexity()autoreadability = tc.autoReadability()embeddingdepth = tc.embeddingDepth()readability = tc.readability()agereadability = tc.ageReadability()yearscrawford = tc.yearsCrawford()For example, for a given sample text:La u´ltima luna llena del an˜o, que se obser-vara´ completa este jueves en el cielo, sera´ espe-cial. Se produce estos d´ıas el feno´meno conocidocomo luna fr´ıa, una coincidencia astrono´mica que3https://github.com/TALP-UPC/FreeLing-User-Manualhace las delicias de quienes atribuyen al astrocualidades esote´ricas. Sucede cuando la Tierrase encuentra ubicada exactamente entre el sol yla luna, de forma que la luna recibe directamentela luz. La luna llena sera´ visible durante todala noche, pero alcanzara´ su magnitud ma´ximacuando se encuentre a medio cielo, de forma que,al reflejar completamente la luz del sol que in-cide en la tierra, se vera´ especialmente grande yluminosa. Se llama luna fr´ıa porque marca la lle-gada del invierno en el hemisferio norte, aunquetambie´n se conoce como luna de las noches largasal ocurrir cerca del solsticio, informa NationalGeographic, que cita a un astro´nomo de la NASA.La luna fr´ıa de 2019 coincide adema´s con la lluviade meteoros Gemı´nidas, visible entre el 7 y el 17de diciembre pero que alcanzara´ su punto ma´ximode actividad entre el 11 y el 13. Es la lluvia deestrellas ma´s masiva, lo que la hace mucho ma´sbrillante. El cielo augura todo un especta´culo estanoche.This is the generated output:Number of words (N_w): 207Punctuation marks: 21Number of low freq. words (N_lfw): 67Number of content words (N_dcw): 71Number of sentences (N_s): 8Number of total content words (N_cw): 93Lexical Distribution Index (LDI): 8.875Index Low Frequency Words (ILFW): 0.72Lexical Complexity Index (LC): 4.7977Number of rare words (N_rw): 65Spaulding Spanish Readability (SSR): 167.82Average Sentence Length (ASL): 25.875Complex Sentences (CS): 23.75Sentence Complexity Index:(SCI): 24.81Automated Readability Index (ARI): 13.66Average embddings depth (MeanDEPTH): 7.25Huerta’s Readability index: 80.86IFSZ Readability: 54.25Polani’s Compressibility (Polani’s): 42.61Mu Readability: 53.19Minimum age to understand: 12.48SOL Readability: 11.66Years needed: 5.766 Conclusions and futureversionsThere are several studies that reflect thestrong influence of the richness of the reader’svocabulary on reading comprehension, be-yond symbols and grammar. However, weconsider that complexity measures can be aconvenient way to model natural language inThe text complexity library129certain applications, such as authorship de-tection, text selection for people with dif-ficulties associated with language disorders(autism, cerebral palsy...), or early detectionof cognitive impairments, such as Alzheimer.Another future line of work is to definecomplexity from computed language models(like RNNs or BERT models). We believethat information measures on the parame-ters of these models may capture the inherentcomplexity of the texts they were trained on.7 AcknowledgementsThis work has been partially supportedby Fondo Europeo de Desarrollo Re-gional (FEDER), LIVING-LANG project(RTI2018-094653-B-C21) from the SpanishGovernment.ReferencesAlliende Gonza´lez, F. 1994. La legibilidadde los textos. Santiago de Chile: Andre´sBello, 24.Anula, A. 2008. Lecturas adaptadas a laensen˜anza del espan˜ol como l2: variableslingu¨´ısticas para la determinacio´n del nivelde legibilidad. La evaluacio´n en el apren-dizaje y la ensen˜anza del espan˜ol como LEL, 2:162–170.Barrio-Cantalejo, I. M., P. Simo´n-Lorda,M. Melguizo, I. Escalona, M. I. Mar-ijua´n, and P. Hernando. 2008. Vali-dacio´n de la Escala INFLESZ para eval-uar la legibilidad de los textos dirigidosa pacientes. In Anales del Sistema Sani-tario de Navarra, volume 31, pages 135–152. SciELO Espa{ na.Blanco Pe´rez, A. and U. Gutie´rrez Couto.2002. Legibilidad de las pa´ginas web so-bre salud dirigidas a pacientes y lectoresde la poblacio´n general. Revista espan˜olade salud pu´blica, 76(4):321–331.Cain, K., J. Oakhill, and P. Bryant. 2004.Children’s reading comprehension abil-ity: Concurrent prediction by workingmemory, verbal ability, and componentskills. Journal of educational psychology,96(1):31.Contreras, A., R. Garcia-Alonso,M. Echenique, and F. Daye-Contreras.1999. The sol formulas for convertingsmog readability scores between healtheducation materials written in spanish,english, and french. Journal of healthcommunication, 4(1):21–29.Crawford, A. N. 1984. A spanish languagefry-type readability procedure: Elemen-tary level. bilingual education paper se-ries, vol. 7, no. 8.Flesch, R. 1948. A new readability yardstick.Journal of applied psychology, 32(3):221.Garc´ıa Lo´pez, J. 2001. Legibilidad de los fol-letos informativos. Pharmaceutical CareEspan˜a, 3(1):49–56.Mc Laughlin, G. H. 1969. Smog grading-anew readability formula. Journal of read-ing, 12(8):639–646.Mun˜oz, M. 2006. Legibilidad y variabilidadde los textos. Bolet´ın de Investigacio´n Ed-ucacional, Pontificia Universidad Cato´licade Chile, 21, 2:13–26.Padro´, L. and E. Stanilovsky. 2012. Freel-ing 3.0: Towards wider multilinguality.In Proceedings of the Eighth InternationalConference on Language Resources andEvaluation (LREC’12), pages 2473–2479.Pe´rez, E. J. 2014. Comprensio´n lectoravs competencia lectora: que´ son y que´relacio´n existe entre ellas. Investigacionessobre lectura, (1):65–74.Ramı´rez-Puerta, M., R. Ferna´ndez-Ferna´ndez, J. Fr´ıas-Pareja, M. Yuste-Ossorio, S. Narbona-Galdo´, and L. Pen˜as-Maldonado. 2013. Ana´lisis de legibilidadde consentimientos informados en cuida-dos intensivos. Medicina Intensiva,37(8):503–509.Rodr´ıguez, T. 1980. Determinacio´n de lacomprensibilidad de materiales de lecturapor medio de variables lingu¨´ısticas. Lec-tura y vida, 1(1):29–32.Saggion, H., S. Sˇtajner, S. Bott, S. Mille,L. Rello, and B. Drndarevic. 2015. Mak-ing it simplext: Implementation and eval-uation of a text simplification system forspanish. ACM Transactions on AccessibleComputing (TACCESS), 6(4):14.Senter, R. and E. A. Smith. 1967. Auto-mated readability index. Technical report,CINCINNATI UNIV OH.Spaulding, S. 1956. A spanish readabilityformula. The Modern Language Journal,40(8):433–441.Rocío López Anguita, Jaime Collado Montañez, Arturo Montejo Ráez130

Biblioteca de Complejidad Textual

http://rua.ua.es/dspace/bitstream/10045/109306/1/PLN_65_19.pdf

Biblioteca de Complejidad Textual

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional de la Universidad de Alicante