Human evaluation of Kea, an automatic keyphrasing system.

This paper describes an evaluation of the Kea automatic keyphrase extraction algorithm. Tools that automatically identify keyphrases are desirable because document keyphrases have numerous applications in digital library systems, but are costly and time consuming to manually assign. Keyphrase extraction algorithms are usually evaluated by comparison to author-specified keywords, but this methodology has several well-known shortcomings. The results presented in this paper are based on subjective evaluations of the quality and appropriateness of keyphrases by human assessors, and make a number of contributions. First, they validate previous evaluations of Kea that rely on author keywords. Second, they show Kea's performance is comparable to that of similar systems that have been evaluated by human assessors. Finally, they justify the use of author keyphrases as a performance metric by showing that authors generally choose good keywords

Interactive document summarisation.

This paper describes the Interactive Document Summariser (IDS), a dynamic document summarisation system, which can help users of digital libraries to access on-line documents more effectively. IDS provides dynamic control over summary characteristics, such as length and topic focus, so that changes made by the user are instantly reflected in an on-screen summary. A range of 'summary-in-context' views support seamless transitions between summaries and their source documents. IDS creates summaries by extracting keyphrases from a document with the Kea system, scoring sentences according to the keyphrases that they contain, and then extracting the highest scoring sentences. We report an evaluation of IDS summaries, in which human assessors identified suitable summary sentences in source documents, against which IDS summaries were judged. We found that IDS summaries were better than baseline summaries, and identify the characteristics of Kea keyphrases that lead to the best summaries

Coherent Keyphrase Extraction via Web Mining

Keyphrases are useful for a variety of purposes, including summarizing, indexing, labeling, categorizing, clustering, highlighting, browsing, and searching. The task of automatic keyphrase extraction is to select keyphrases from within the text of a given document. Automatic keyphrase extraction makes it feasible to generate keyphrases for the huge number of documents that do not have manually assigned keyphrases. A limitation of previous keyphrase extraction algorithms is that the selected keyphrases are occasionally incoherent. That is, the majority of the output keyphrases may fit together well, but there may be a minority that appear to be outliers, with no clear semantic relation to the majority or to each other. This paper presents enhancements to the Kea keyphrase extraction algorithm that are designed to increase the coherence of the extracted keyphrases. The approach is to use the degree of statistical association among candidate keyphrases as evidence that they may be semantically related. The statistical association is measured using web mining. Experiments demonstrate that the enhancements improve the quality of the extracted keyphrases. Furthermore, the enhancements are not domain-specific: the algorithm generalizes well when it is trained on one domain (computer science documents) and tested on another (physics documents).Comment: 6 pages, related work available at http://purl.org/peter.turney

Keyphrase Generation: A Multi-Aspect Survey

Extractive keyphrase generation research has been around since the nineties, but the more advanced abstractive approach based on the encoder-decoder framework and sequence-to-sequence learning has been explored only recently. In fact, more than a dozen of abstractive methods have been proposed in the last three years, producing meaningful keyphrases and achieving state-of-the-art scores. In this survey, we examine various aspects of the extractive keyphrase generation methods and focus mostly on the more recent abstractive methods that are based on neural networks. We pay particular attention to the mechanisms that have driven the perfection of the later. A huge collection of scientific article metadata and the corresponding keyphrases is created and released for the research community. We also present various keyphrase generation and text summarization research patterns and trends of the last two decades.Comment: 10 pages, 5 tables. Published in proceedings of FRUCT 2019, the 25th Conference of the Open Innovations Association FRUCT, Helsinki, Finlan

Etiquetado asistido de documentos de investigación mediante procesamiento de lenguaje natural y tecnologías de la web semántica

El presente artículo se basa en la implementación del procesamiento de lenguaje natural (PLN) y las tecnologías de la web semántica, con la intención de facilitar la extracción de palabras claves en documentos de investigación de forma más eficiente y eficaz. Para tal fin, por medio de una matriz de comparación se seleccionó un algoritmo para realizar el proceso de extracción. Se eligió el algoritmo Keyword Extraction Based On Entropy Difference (C#) realizado por Zhen YANG, Jianjun LEI, Kefeng FAN y Yingxu LAI. Este algoritmo fue desarrollado para procesarlos documentos en idioma chino, por lo que fue requerida una adaptación al idioma inglés y español anexando los vocabularios de correspondientes a estos idiomas configurando el código fuente del algoritmo. Adicionalmente se adaptó el algoritmo para que usase una ontología con la terminología propia del dominio de conocimiento de ingenierías. El algoritmo fue evaluado por medio de ejemplos de artículos científicos, obteniendo métricas de recuperación de la información, como son la precisión, exhaustividad y el valor F. Se obtuvo como resultado un valor F promedio 0.63 para una muestra de 13 artículos científicos, lo que valida el algoritmo como óptimo para la tarea propuesta

Concept graphs: Applications to biomedical text categorization and concept extraction

As science advances, the underlying literature grows rapidly providing valuable knowledge mines for researchers and practitioners. The text content that makes up these knowledge collections is often unstructured and, thus, extracting relevant or novel information could be nontrivial and costly. In addition, human knowledge and expertise are being transformed into structured digital information in the form of vocabulary databases and ontologies. These knowledge bases hold substantial hierarchical and semantic relationships of common domain concepts. Consequently, automating learning tasks could be reinforced with those knowledge bases through constructing human-like representations of knowledge. This allows developing algorithms that simulate the human reasoning tasks of content perception, concept identification, and classification. This study explores the representation of text documents using concept graphs that are constructed with the help of a domain ontology. In particular, the target data sets are collections of biomedical text documents, and the domain ontology is a collection of predefined biomedical concepts and relationships among them. The proposed representation preserves those relationships and allows using the structural features of graphs in text mining and learning algorithms. Those features emphasize the significance of the underlying relationship information that exists in the text content behind the interrelated topics and concepts of a text document. The experiments presented in this study include text categorization and concept extraction applied on biomedical data sets. The experimental results demonstrate how the relationships extracted from text and captured in graph structures can be used to improve the performance of the aforementioned applications. The discussed techniques can be used in creating and maintaining digital libraries through enhancing indexing, retrieval, and management of documents as well as in a broad range of domain-specific applications such as drug discovery, hypothesis generation, and the analysis of molecular structures in chemoinformatics

Evaluation of automatic concept extraction tools within a digital library environment

El rápido avance de la tecnología ha originado la proliferación de fuentes de información digital. Esta evolución informática ha provocado la creación de bibliotecas digitales que han ido convirtiendose poco a poco en un gran pilar para la difusión del conocimiento. Sin embargo, la información contenida en las bibliotecas digitales aún no está descrita totalmente y su explotación es aún insuficiente. Recientemente, se ha comprobado que la descripción de la información usando “metadatos” puede ser primordial para el mejoramiento de la consulta de la información dentro de una biblioteca digital. Nuestro enfoque está basado en la creación e introducción de nuevos “metadatos” capaces de describir, en nuestro caso, las tesis doctorales de una biblioteca digital. Estos “metadatos” corresponden a los conceptos más importantes de cada una de las tesis. Actualmente, la identificación manual de conceptos es un largo proceso llevado a cabo por un especialista del área. Por lo tanto, es importante hacer uso de herramientas capaces de extraer automáticamente conceptos. En este artículo analizamos cuatro herramientas de PLN (Procesamiento del Lenguaje Natural) capaces de extraer automáticamente los conceptos claves de un corpus. Estas herramientas son: (1) TerminologyExtractor de Chamblon Systems Inc., (2) Xerox Terminology Suite de Xerox, (3) Nomino de Nomino Technologies y (4) Copernic Summarizer de NRC. Este artículo presenta también un prototipo de herramienta de anotación desarrollado para insertar de manera automática conceptos a las tesis digitales.The rapid advance of technology has led to the proliferation of digital information sources. This computer evolution has led to the creation of digital libraries that have been gradually becoming a great pillar for the dissemination of knowledge. However, the information contained in digital libraries is not yet fully described and its use is still insufficient. Recently, it has been found that the description of information using "metadata" can be essential for improving the query of information inside a digital library. Our approach is based on the creation and introduction of new “metadata” capable of describing, in our case, the doctoral theses of a library digital. These “metadata” correspond to the most important concepts of each of the thesis. Currently, the manual identification of concepts is a long process carried out by an area specialist. Therefore, it is important to make use of tools capable of extracting automatically concepts. In this article we analyze four NLP tools (Natural Language Processing) capable of automatically extracting the key concepts of a corpus. These tools are: (1) TerminologyExtractor from Chamblon Systems Inc., (2) Xerox Terminology Suite from Xerox, (3) Nomino from Nomino Technologies and (4) Copernic NRC Summary. This article also presents a prototype of an annotation tool developed to automatically insert concepts into digital theses

Evaluación de herramientas de extracción automática de conceptos dentro de un ambiente de biblioteca digital

El rápido avance de la tecnología ha originado la proliferación de fuentes de información digital. Esta evolución informática ha provocado la creación de bibliotecas digitales que han ido convirtiendose poco a poco en un gran pilar para la difusión del conocimiento. Sin embargo, la información contenida en las bibliotecas digitales aún no está descrita totalmente y su explotación es aún insuficiente. Recientemente, se ha comprobado que la descripción de la información usando “metadatos” puede ser primordial para el mejoramiento de la consulta de la información dentro de una biblioteca digital.Palabras claves: Biblioteca digital, metadatos, Procesamiento del Lenguaje Natural, extracción de información, anotación, búsqueda de información