INEX Tweet Contextualization Task: Evaluation, Results and Lesson Learned

Microblogging platforms such as Twitter are increasingly used for on-line client and market analysis. This motivated the proposal of a new track at CLEF INEX lab of Tweet Contextualization. The objective of this task was to help a user to understand a tweet by providing him with a short explanatory summary (500 words). This summary should be built automatically using resources like Wikipedia and generated by extracting relevant passages and aggregating them into a coherent summary. Running for four years, results show that the best systems combine NLP techniques with more traditional methods. More precisely the best performing systems combine passage retrieval, sentence segmentation and scoring, named entity recognition, text part-of-speech (POS) analysis, anaphora detection, diversity content measure as well as sentence reordering. This paper provides a full summary report on the four-year long task. While yearly overviews focused on system results, in this paper we provide a detailed report on the approaches proposed by the participants and which can be considered as the state of the art for this task. As an important result from the 4 years competition, we also describe the open access resources that have been built and collected. The evaluation measures for automatic summarization designed in DUC or MUC were not appropriate to evaluate tweet contextualization, we explain why and depict in detailed the LogSim measure used to evaluate informativeness of produced contexts or summaries. Finally, we also mention the lessons we learned and that it is worth considering when designing a task

Evaluación de material didáctico en L2

Resumen: los materiales didácticos utilizados para la enseñanza en niveles del último ciclo de primaria y secundaria de asignaturas curriculares que enseñan nuevos contenidos de distintas áreas (geografía, historia, ciencias, etc.) utilizando una segunda lengua (L2) requieren de unas características sintácticas, de vocabulario y discurso que faciliten el aprendizaje del mismo. Por esta razón, este proyecto surge de la necesidad de evaluar la idoneidad de estos materiales. Por ello, se ha desarrollado una aplicación web (llamada AzterTest) que es capaz de evaluar varios textos en inglés mediante el análisis y cálculo de las distintas métricas e indicadores que sirven, especialmente, para determinar el grado de complejidad de dichos textos, de manera que esta herramienta servirá de apoyo a los profesores que necesiten determinar y clasificar textos según su complejidad.Laburpena: lehen eta bigarren mailakotako azken zikloan, bigarren hizkuntza (L2) batean irakasten diren area desberdineko eduki berrien (geografia, historia, zientziak, etab.) material didaktikoak testu hauen ikasketa errazten duten ezaugarri sintaktiko batzuk dituzte, baita hiztegiarekin eta diskurtsoarekin erlazionatuta dauden ezaugarriak ere. Arrazoi honengatik, proiektu hau agertzen da material hauen egokitasuna ebaluatzeko beharretik. Horregatik, garatu da ingelesez idatzita dauden hainbat testu aztertzeko gai den web-aplikazioa (AzterTest deituta). Aplikazio honek testu baten konplexutasuna zehazteko balio duten hainbat metrika kalkulatzeko gai da. Beraz, erreminta hau behar duten irakasleentzako laguntza gisa balio izango du haien testuak bere konplexutasunaren arabera sailkatzeko.Abstract: the teaching materials used for teaching at the last cycle of primary and secondary levels of curricular subjects that teach new contents from different areas (geography, history, sciences, etc.) using a second language (L2) require syntactic, vocabulary and speech characteristics that facilitate the learning of it. For this reason, this project arises from the need to evaluate the suitability of these materials. Therefore, a web application (called AzterTest) has been developed. This application is capable of evaluating several texts in English through the analysis and calculation of the different metrics and indicators that serve, especially, to determine the degree of complexity of these texts, in such a way that this tool will serve as a support for teachers who need to determine and classify texts according to their complexity

Evaluación de material didáctico en L2

Resumen: los materiales didácticos utilizados para la enseñanza en niveles del último ciclo de primaria y secundaria de asignaturas curriculares que enseñan nuevos contenidos de distintas áreas (geografía, historia, ciencias, etc.) utilizando una segunda lengua (L2) requieren de unas características sintácticas, de vocabulario y discurso que faciliten el aprendizaje del mismo. Por esta razón, este proyecto surge de la necesidad de evaluar la idoneidad de estos materiales. Por ello, se ha desarrollado una aplicación web (llamada AzterTest) que es capaz de evaluar varios textos en inglés mediante el análisis y cálculo de las distintas métricas e indicadores que sirven, especialmente, para determinar el grado de complejidad de dichos textos, de manera que esta herramienta servirá de apoyo a los profesores que necesiten determinar y clasificar textos según su complejidad.Laburpena: lehen eta bigarren mailakotako azken zikloan, bigarren hizkuntza (L2) batean irakasten diren area desberdineko eduki berrien (geografia, historia, zientziak, etab.) material didaktikoak testu hauen ikasketa errazten duten ezaugarri sintaktiko batzuk dituzte, baita hiztegiarekin eta diskurtsoarekin erlazionatuta dauden ezaugarriak ere. Arrazoi honengatik, proiektu hau agertzen da material hauen egokitasuna ebaluatzeko beharretik. Horregatik, garatu da ingelesez idatzita dauden hainbat testu aztertzeko gai den web-aplikazioa (AzterTest deituta). Aplikazio honek testu baten konplexutasuna zehazteko balio duten hainbat metrika kalkulatzeko gai da. Beraz, erreminta hau behar duten irakasleentzako laguntza gisa balio izango du haien testuak bere konplexutasunaren arabera sailkatzeko.Abstract: the teaching materials used for teaching at the last cycle of primary and secondary levels of curricular subjects that teach new contents from different areas (geography, history, sciences, etc.) using a second language (L2) require syntactic, vocabulary and speech characteristics that facilitate the learning of it. For this reason, this project arises from the need to evaluate the suitability of these materials. Therefore, a web application (called AzterTest) has been developed. This application is capable of evaluating several texts in English through the analysis and calculation of the different metrics and indicators that serve, especially, to determine the degree of complexity of these texts, in such a way that this tool will serve as a support for teachers who need to determine and classify texts according to their complexity