INEX Tweet Contextualization Task: Evaluation, Results and Lesson Learned

Microblogging platforms such as Twitter are increasingly used for on-line client and market analysis. This motivated the proposal of a new track at CLEF INEX lab of Tweet Contextualization. The objective of this task was to help a user to understand a tweet by providing him with a short explanatory summary (500 words). This summary should be built automatically using resources like Wikipedia and generated by extracting relevant passages and aggregating them into a coherent summary. Running for four years, results show that the best systems combine NLP techniques with more traditional methods. More precisely the best performing systems combine passage retrieval, sentence segmentation and scoring, named entity recognition, text part-of-speech (POS) analysis, anaphora detection, diversity content measure as well as sentence reordering. This paper provides a full summary report on the four-year long task. While yearly overviews focused on system results, in this paper we provide a detailed report on the approaches proposed by the participants and which can be considered as the state of the art for this task. As an important result from the 4 years competition, we also describe the open access resources that have been built and collected. The evaluation measures for automatic summarization designed in DUC or MUC were not appropriate to evaluate tweet contextualization, we explain why and depict in detailed the LogSim measure used to evaluate informativeness of produced contexts or summaries. Finally, we also mention the lessons we learned and that it is worth considering when designing a task

Beyond Stemming and Lemmatization: Ultra-stemming to Improve Automatic Text Summarization

In Automatic Text Summarization, preprocessing is an important phase to reduce the space of textual representation. Classically, stemming and lemmatization have been widely used for normalizing words. However, even using normalization on large texts, the curse of dimensionality can disturb the performance of summarizers. This paper describes a new method for normalization of words to further reduce the space of representation. We propose to reduce each word to its initial letters, as a form of Ultra-stemming. The results show that Ultra-stemming not only preserve the content of summaries produced by this representation, but often the performances of the systems can be dramatically improved. Summaries on trilingual corpora were evaluated automatically with Fresa. Results confirm an increase in the performance, regardless of summarizer system used.Comment: 22 pages, 12 figures, 9 table

SASI: sumarizador automático de documentos baseado no problema do subconjunto independente de vértices

XLVI Simpósio Brasileiro de Pesquisa OperacionalThis article discusses a summarizer system of documents named SASI. This system features an innovative approach to provide automatic summaries, based on the determination of the maximum independent subset of vertices, modeling the problem a graph of phrases (vertices) and the relationships between them (edges). The concepts and operation of the proposed summarizer and a series of tests comparing the results provided by SASI with others summarizer systems are described. Initial results are promising, evaluating questions of informativeness of the produced summaries on the parameters of time and algorithmic complexity

Evaluating Information Retrieval and Access Tasks

This open access book summarizes the first two decades of the NII Testbeds and Community for Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global team of researchers and hosted by the National Institute of Informatics (NII), Japan. The book is unique in that it discusses not just what was done at NTCIR, but also how it was done and the impact it has achieved. For example, in some chapters the reader sees the early seeds of what eventually grew to be the search engines that provide access to content on the World Wide Web, today’s smartphones that can tailor what they show to the needs of their owners, and the smart speakers that enrich our lives at home and on the move. We also get glimpses into how new search engines can be built for mathematical formulae, or for the digital record of a lived human life. Key to the success of the NTCIR endeavor was early recognition that information access research is an empirical discipline and that evaluation therefore lay at the core of the enterprise. Evaluation is thus at the heart of each chapter in this book. They show, for example, how the recognition that some documents are more important than others has shaped thinking about evaluation design. The thirty-three contributors to this volume speak for the many hundreds of researchers from dozens of countries around the world who together shaped NTCIR as organizers and participants. This book is suitable for researchers, practitioners, and students—anyone who wants to learn about past and present evaluation efforts in information retrieval, information access, and natural language processing, as well as those who want to participate in an evaluation task or even to design and organize one

Vers une représentation du contexte thématique en Recherche d'Information

Quand des humains cherchent des informations au sein de bases de connaissancesou de collections de documents, ils utilisent un système de recherche d information(SRI) faisant office d interface. Les utilisateurs doivent alors transmettre au SRI unereprésentation de leur besoin d information afin que celui-ci puisse chercher des documentscontenant des informations pertinentes. De nos jours, la représentation du besoind information est constituée d un petit ensemble de mots-clés plus souvent connu sousla dénomination de requête . Or, quelques mots peuvent ne pas être suffisants pourreprésenter précisément et efficacement l état cognitif complet d un humain par rapportà son besoin d information initial. Sans une certaine forme de contexte thématiquecomplémentaire, le SRI peut ne pas renvoyer certains documents pertinents exprimantdes concepts n étant pas explicitement évoqués dans la requête.Dans cette thèse, nous explorons et proposons différentes méthodes statistiques, automatiqueset non supervisées pour la représentation du contexte thématique de larequête. Plus spécifiquement, nous cherchons à identifier les différents concepts implicitesd une requête formulée par un utilisateur sans qu aucune action de sa part nesoit nécessaire. Nous expérimentons pour cela l utilisation et la combinaison de différentessources d information générales représentant les grands types d informationauxquels nous sommes confrontés quotidiennement sur internet. Nous tirons égalementparti d algorithmes de modélisation thématique probabiliste (tels que l allocationde Dirichlet latente) dans le cadre d un retour de pertinence simulé. Nous proposonspar ailleurs une méthode permettant d estimer conjointement le nombre de conceptsimplicites d une requête ainsi que l ensemble de documents pseudo-pertinent le plusapproprié afin de modéliser ces concepts. Nous évaluons nos approches en utilisantquatre collections de test TREC de grande taille. En annexes, nous proposons égalementune approche de contextualisation de messages courts exploitant des méthodesde recherche d information et de résumé automatiqueWhen searching for information within knowledge bases or document collections,humans use an information retrieval system (IRS). So that it can retrieve documentscontaining relevant information, users have to provide the IRS with a representationof their information need. Nowadays, this representation of the information need iscomposed of a small set of keywords often referred to as the query . A few wordsmay however not be sufficient to accurately and effectively represent the complete cognitivestate of a human with respect to her initial information need. A query may notcontain sufficient information if the user is searching for some topic in which she is notconfident at all. Hence, without some kind of context, the IRS could simply miss somenuances or details that the user did not or could not provide in query.In this thesis, we explore and propose various statistic, automatic and unsupervisedmethods for representing the topical context of the query. More specifically, we aim toidentify the latent concepts of a query without involving the user in the process norrequiring explicit feedback. We experiment using and combining several general informationsources representing the main types of information we deal with on a dailybasis while browsing theWeb.We also leverage probabilistic topic models (such as LatentDirichlet Allocation) in a pseudo-relevance feedback setting. Besides, we proposea method allowing to jointly estimate the number of latent concepts of a query andthe set of pseudo-relevant feedback documents which is the most suitable to modelthese concepts. We evaluate our approaches using four main large TREC test collections.In the appendix of this thesis, we also propose an approach for contextualizingshort messages which leverages both information retrieval and automatic summarizationtechniquesAVIGNON-Bib. numérique (840079901) / SudocSudocFranceF

Analyse de l’image de marque sur le Web 2.0

Analyse of entities representation over the Web 2.0Every day, millions of people publish their views on Web 2.0 (social networks,blogs, etc.). These comments focus on subjects as diverse as news, politics,sports scores, consumer objects, etc. The accumulation and agglomerationof these notices on an entity (be it a product, a company or a public entity) givebirth to the brand image of that entity. Internet has become in recent years aprivileged place for the emergence and dissemination of opinions and puttingWeb 2.0 at the head of observatories of opinions. The latter being a means ofaccessing the knowledge of the opinion of the world population.The image is here understood as the idea that a person or a group of peopleis that entity. This idea carries a priori on a particular subject and is onlyvalid in context for a given time. This perceived image is different from theentity initially wanted to broadcast (eg via a communication campaign). Moreover,in reality, there are several images in the end living together in parallel onthe network, each specific to a community and all evolve differently over time(imagine how would be perceived in each camp together two politicians edgesopposite). Finally, in addition to the controversy caused by the voluntary behaviorof some entities to attract attention (think of the declarations required orshocking). It also happens that the dissemination of an image beyond the frameworkthat governed the and sometimes turns against the entity (for example,« marriage for all » became « the demonstration for all »). The views expressedthen are so many clues to understand the logic of construction and evolution ofthese images. The aim is to be able to know what we are talking about and howwe talk with filigree opportunity to know who is speaking.viiIn this thesis we propose to use several simple supervised statistical automaticmethods to monitor entity’s online reputation based on textual contentsmentioning it. More precisely we look the most important contents and theirsauthors (from a reputation manager point-of-view). We introduce an optimizationprocess allowing us to enrich the data using a simulated relevance feedback(without any human involvement). We also compare content contextualizationmethod using information retrieval and automatic summarization methods.Wealso propose a reflection and a new approach to model online reputation, improveand evaluate reputation monitoring methods using Partial Least SquaresPath Modelling (PLS-PM). In designing the system, we wanted to address localand global context of the reputation. That is to say the features can explain thedecision and the correlation betweens topics and reputation. The goal of ourwork was to propose a different way to combine usual methods and featuresthat may render reputation monitoring systems more accurate than the existingones. We evaluate and compare our systems using state of the art frameworks: Imagiweb and RepLab. The performances of our proposals are comparableto the state of the art. In addition, the fact that we provide reputation modelsmake our methods even more attractive for reputation manager or scientistsfrom various fields.Image sur le web : analyse de la dynamique des images sur le Web 2.0. En plus d’être un moyen d’accès à la connaissance, Internet est devenu en quelques années un lieu privilégié pour l’apparition et la diffusion d’opinions.Chaque jour, des millions d’individus publient leurs avis sur le Web 2.0 (réseaux sociaux, blogs, etc.). Ces commentaires portent sur des sujets aussi variés que l’actualité, la politique, les résultats sportifs, biens culturels, des objets de consommation, etc. L’amoncellement et l’agglomération de ces avis publiés sur une entité (qu’il s’agisse d’un produit, une entreprise ou une personnalité publique)donnent naissance à l’image de marque de cette entité.L’image d’une entité est ici comprise comme l’idée qu’une personne ou qu’un groupe de personnes se fait de cette entité. Cette idée porte a priori sur un sujet particulier et n’est valable que dans un contexte, à un instant donné.Cette image perçue est par nature différente de celle que l’entité souhaitait initialement diffuser (par exemple via une campagne de communication). De plus,dans la réalité, il existe au final plusieurs images qui cohabitent en parallèle sur le réseau, chacune propre à une communauté et toutes évoluant différemment au fil du temps (imaginons comment serait perçu dans chaque camp le rapprochement de deux hommes politiques de bords opposés). Enfin, en plus des polémiques volontairement provoquées par le comportement de certaines entités en vue d’attirer l’attention sur elles (pensons aux tenues ou déclarations choquantes), il arrive également que la diffusion d’une image dépasse le cadre qui la régissait et même parfois se retourne contre l’entité (par exemple, «le mariage pour tous» devenu « la manif pour tous »). Les opinions exprimées constituent alors autant d’indices permettant de comprendre la logique de construction et d’évolution de ces images. Ce travail d’analyse est jusqu’à présent confié à des spécialistes de l’e-communication qui monnaient leur subjectivité. Ces derniers ne peuvent considérer qu’un volume restreint d’information et ne sont que rarement d’accord entre eux. Dans cette thèse, nous proposons d’utiliser différentes méthodes automatiques, statistiques, supervisées et d’une faible complexité permettant d’analyser et représenter l’image de marque d’entité à partir de contenus textuels les mentionnant. Plus spécifiquement, nous cherchons à identifier les contenus(ainsi que leurs auteurs) qui sont les plus préjudiciables à l’image de marque d’une entité. Nous introduisons un processus d’optimisation automatique de ces méthodes automatiques permettant d’enrichir les données en utilisant un retour de pertinence simulé (sans qu’aucune action de la part de l’entité concernée ne soit nécessaire). Nous comparer également plusieurs approches de contextualisation de messages courts à partir de méthodes de recherche d’information et de résumé automatique. Nous tirons également parti d’algorithmes de modélisation(tels que la Régression des moindres carrés partiels), dans le cadre d’une modélisation conceptuelle de l’image de marque, pour améliorer nos systèmes automatiques de catégorisation de documents textuels. Ces méthodes de modélisation et notamment les représentations des corrélations entre les différents concepts que nous manipulons nous permettent de représenter d’une part, le contexte thématique d’une requête de l’entité et d’autre, le contexte général de son image de marque. Nous expérimentons l’utilisation et la combinaison de différentes sources d’information générales représentant les grands types d’information auxquels nous sommes confrontés sur internet : de long les contenus objectifs rédigés à des informatives, les contenus brefs générés par les utilisateurs visant à partager des opinions. Nous évaluons nos approches en utilisant deux collections de données, la première est celle constituée dans le cadre du projet Imagiweb, la seconde est la collection de référence sur le sujet : CLEFRepLa

Computer Aided Verification

This open access two-volume set LNCS 11561 and 11562 constitutes the refereed proceedings of the 31st International Conference on Computer Aided Verification, CAV 2019, held in New York City, USA, in July 2019. The 52 full papers presented together with 13 tool papers and 2 case studies, were carefully reviewed and selected from 258 submissions. The papers were organized in the following topical sections: Part I: automata and timed systems; security and hyperproperties; synthesis; model checking; cyber-physical systems and machine learning; probabilistic systems, runtime techniques; dynamical, hybrid, and reactive systems; Part II: logics, decision procedures; and solvers; numerical programs; verification; distributed systems and networks; verification and invariants; and concurrency