24 research outputs found
Estudio de representaciones mediante co-embeddings para estudiantes y contenidos en minerı́a de datos educativos
Tesis (Doctora en Ciencias de la Computación)--Universidad Nacional de Córdoba, Facultad de Matemática, Astronomía, Física y Computación, 2019.Este trabajo es un estudio sobre la generación automática de representaciones basadas en
métodos neuronales, en aplicaciones dentro del área de Minerı́a de Datos Educacionales (EDM).
Se propone utilizar una arquitectura neuronal recurrente para modelar el cambio en el estado
de los estudiantes a medida que interactúan con plataformas de aprendizaje en lı́nea. Al
mismo tiempo, se generan representaciones automáticas para los elementos de los cursos, como
problemas o lecciones, evitando la necesidad de utilizar ejemplos etiquetados con información
adicional, y en consecuencia costosos de obtener. Sobre esta base, se modifica la arquitectura
para modelar explı́citamente la relación entre la representación de los estudiantes y la de los
componentes del curso, proyectando ambos tipos de entidades en el mismo espacio latente. De
esta manera, se espera mejorar el desempeño del clasificador a través de la inyección directa de
conocimiento de dominio en el modelo.
Ambas propuestas son evaluadas para las tareas de estimación de conocimiento (Knowledge
Tracing) y predicción del abandono escolar (dropout) en tutores inteligentes y cursos masivos,
respectivamente. Se observa que las representaciones conjuntas de estudiantes y lecciones obtienen resultados similares a las representaciones disjuntas, mejorando significativamente en
escenarios con pocos datos o con desbalance de clases pronunciado.This work is a study on the automatic generation of representations based on neuronal methods, for applications in the area of Educational Data Mining (EDM). We proposed to use a recurrent neuronal architecture to model the change in the state of students as they interact with online learning platforms. At the same time, automatic representations are generated for
course elements, such as problems or lessons, avoiding the need to use examples labeled with additional information, and consequently costly to obtain. On this basis, the architecture is
modified to explicitly model the relationship between the students’ representation and that of the course components, projecting both types of entities in the same latent space. In this way, the performance of the classifier is expected to improve through the direct injection of domain knowledge into the model. Both proposals are evaluated for knowledge tracing and dropout prediction in intelligent tutor systems and mass open online courses, respectively. It is observed that the joint representations of students and lessons obtain results similar to the disjoint representations, improving significantly in scenarios with fewer training data, partial sequences, or with pronounced class imbalance.Fil: Teruel, Milagro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía, Física y Computación; Argentina
Aprendizaje activo para clasificación de preguntas
Quepy es una librería para construir sistema de respuesta a preguntas sobre datos enlazados, sin embargo utiliza patrones estáticos para reconocer preguntas y alcanzar una gran cobertura es muy costoso. Utilizamos un clasificador bayesiano ingenuo para clasificar reformulaciones de preguntas semánticamente equivalentes, ligarlas a una misma interpretación y aumentar esta cobertura. La falta de un corpus de preguntas etiquetado nos llevó a utilizar aprendizaje activo tanto sobre instancias como sobre características. En la representación de las instancias incluimos también las concordancias con los patrones parciales del Quepy, para capturar la información semántica de la pregunta. En este escenario contamos con muchas de preguntas que no concuerdan con ningún patrón y muchas clases minoritarias con pocos ejemplos cada una. Los resultados indican que balancear el corpus de entrenamiento utilizado por el clasificador evita que la clase mayoritaria se convierta en la única clase reconocida, mientras que el entrenamiento con características aumentó en gran medida el reconocimiento de las clases minoritarias
Reversing uncertainty sampling to improve active learning schemes
Ponencia presentada en el 16º Simposio Argentino de Inteligencia Artificial. 44 Jornadas Argentinas de Informática. Rosario, Argentina, del 31 de agosto al 4 de septiembre de 2015.Active learning provides promising methods to optimize the cost of manually annotating a dataset. However, practitioners in many areas do not massively resort to such methods because they present technical difficulties and do not provide a guarantee of good performance, especially in skewed distributions with scarcely populated minority classes and an undefined, catch-all majority class, which are very common in human-related phenomena like natural language. In this paper we present a comparison of the simplest active learning technique, pool-based uncertainty sampling, and its opposite, which we call reversed uncertainty sampling. We show that both obtain results comparable to the random, arguing for a more insightful approach to active learning.http://44jaiio.sadio.org.ar/asaiFil: Cardellino, Cristian Adrián. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Fil: Teruel, Milagro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Fil: Alonso i Alemany, Laura. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Ciencias de la Computació
Different Flavors of Attention Networks for Argument Mining
International audienceArgument mining is a rising area of Natural Language Processing (NLP) concerned with the automatic recognition and interpretation of argument components and their relations. Neural models are by now mature technologies to be exploited for automating the argument mining tasks, despite the issue of data sparseness. This could ease much of the manual effort involved in these tasks, taking into account heterogeneous types of texts and topics. In this work, we evaluate different attention mechanisms applied over a state-of-the-art architecture for sequence labeling. We assess the impact of different flavors of attention in the task of argument component detection over two datasets: essays and legal domain. We show that attention not only models the problem better but also supports interpretability
Combining semi-supervised and active learning to recognize minority senses in a new corpus
Ponencia presentada en la 24th International Joint Conference on Artificial Intelligence. Workshop on Replicability and Reproducibility in Natural Language Processing: adaptive methods, resources and software. Buenos Aires, Argentina, del 25 al 31 de julio de 2015.In this paper we study the impact of combining active learning with bootstrapping to grow a small annotated corpus from a different, unannotated corpus. The intuition underlying our approach is that bootstrapping includes instances that are closer to the generative centers of the data, while discriminative approaches to active learning include instances that are closer to the decision boundaries of classifiers. We build an initial model from the original annotated corpus, which is then iteratively enlarged by including both manually annotated examples and automatically labelled examples as training examples for the following iteration. Examples to be annotated are selected in each iteration by applying active learning techniques. We show that intertwining an active learning component in a bootstrapping approach helps to overcome an initial bias towards a majority class, thus facilitating adaptation of a starting dataset towards the real distribution of a different, unannotated corpus.Fil: Cardellino, Cristian Adrián. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Fil: Teruel, Milagro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Fil: Alonso i Alemany, Laura. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina.Otras Ciencias de la Computación e Informació
On Improving Summarization Factual Consistency from Natural Language Feedback
Despite the recent progress in language generation models, their outputs may
not always meet user expectations. In this work, we study whether informational
feedback in natural language can be leveraged to improve generation quality and
user preference alignment. To this end, we consider factual consistency in
summarization, the quality that the summary should only contain information
supported by the input documents, as the user-expected preference. We collect a
high-quality dataset, DeFacto, containing human demonstrations and
informational natural language feedback consisting of corrective instructions,
edited summaries, and explanations with respect to the factual consistency of
the summary. Using our dataset, we study three natural language generation
tasks: (1) editing a summary by following the human feedback, (2) generating
human feedback for editing the original summary, and (3) revising the initial
summary to correct factual errors by generating both the human feedback and
edited summary. We show that DeFacto can provide factually consistent
human-edited summaries and further insights into summarization factual
consistency thanks to its informational natural language feedback. We further
demonstrate that fine-tuned language models can leverage our dataset to improve
the summary factual consistency, while large language models lack the zero-shot
learning ability in our proposed tasks that require controllable text
generation.Comment: ACL 2023 Camera Ready, GitHub Repo:
https://github.com/microsoft/DeFact
Learning Slowly To Learn Better: Curriculum Learning for Legal Ontology Population
International audienceIn this paper, we present an ontology population approach for legal ontologies. We exploit Wikipedia as a source of manually annotated examples of legal entities. We align YAGO, a Wikipedia-based ontology, and LKIF, an ontology specifically designed for the legal domain. Through this alignment, we can effectively populate the LKIF ontology, with the aim to obtain examples to train a Named Entity Recognizer and Classifier to be used for finding and classifying entities in legal texts. Since examples of annotated data in the legal domain are very few, we apply a machine learning strategy called curriculum learning aimed to overcome problems of overfitting by learning increasingly more complex concepts. We compare the performance of this method to identify Named Entities with respect to batch learning as well as two other baselines. Results are satisfying and foster further research in this direction
Legal NERC with ontologies, Wikipedia and curriculum learning
International audienceIn this paper, we present a Wikipedia-based approach to develop resources for the legal domain. We establish a mapping between a legal domain ontology, LKIF (Hoekstra et al., 2007), and a Wikipedia-based ontology, YAGO (Suchanek et al., 2007), and through that we populate LKIF. Moreover, we use the mentions of those entities in Wikipedia text to train a specific Named Entity Recognizer and Classifier. We find that this classifier works well in the Wikipedia, but, as could be expected, performance decreases in a corpus of judgments of the European Court of Human Rights. However, this tool will be used as a preprocess for human annotation. We resort to a technique called curriculum learning aimed to overcome problems of overfitting by learning increasingly more complex concepts. However, we find that in this particular setting, the method works best by learning from most specific to most general concepts, not the other way round
Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback
Many approaches to Natural Language Processing (NLP) tasks often treat them
as single-step problems, where an agent receives an instruction, executes it,
and is evaluated based on the final outcome. However, human language is
inherently interactive, as evidenced by the back-and-forth nature of human
conversations. In light of this, we posit that human-AI collaboration should
also be interactive, with humans monitoring the work of AI agents and providing
feedback that the agent can understand and utilize. Further, the AI agent
should be able to detect when it needs additional information and proactively
ask for help. Enabling this scenario would lead to more natural, efficient, and
engaging human-AI collaborations.
In this work, we explore these directions using the challenging task defined
by the IGLU competition, an interactive grounded language understanding task in
a MineCraft-like world. We explore multiple types of help players can give to
the AI to guide it and analyze the impact of this help in AI behavior,
resulting in performance improvements
Different Flavors of Attention Networks for Argument Mining
International audienceArgument mining is a rising area of Natural Language Processing (NLP) concerned with the automatic recognition and interpretation of argument components and their relations. Neural models are by now mature technologies to be exploited for automating the argument mining tasks, despite the issue of data sparseness. This could ease much of the manual effort involved in these tasks, taking into account heterogeneous types of texts and topics. In this work, we evaluate different attention mechanisms applied over a state-of-the-art architecture for sequence labeling. We assess the impact of different flavors of attention in the task of argument component detection over two datasets: essays and legal domain. We show that attention not only models the problem better but also supports interpretability