A Cross-domain and Cross-language Knowledge-based Representation of Text and its Meaning

Tesis por compendioNatural Language Processing (NLP) is a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human languages. One of its most challenging aspects involves enabling computers to derive meaning from human natural language. To do so, several meaning or context representations have been proposed with competitive performance. However, these representations still have room for improvement when working in a cross-domain or cross-language scenario. In this thesis we study the use of knowledge graphs as a cross-domain and cross-language representation of text and its meaning. A knowledge graph is a graph that expands and relates the original concepts belonging to a set of words. We obtain its characteristics using a wide-coverage multilingual semantic network as knowledge base. This allows to have a language coverage of hundreds of languages and millions human-general and -specific concepts. As starting point of our research we employ knowledge graph-based features - along with other traditional ones and meta-learning - for the NLP task of single- and cross-domain polarity classification. The analysis and conclusions of that work provide evidence that knowledge graphs capture meaning in a domain-independent way. The next part of our research takes advantage of the multilingual semantic network and focuses on cross-language Information Retrieval (IR) tasks. First, we propose a fully knowledge graph-based model of similarity analysis for cross-language plagiarism detection. Next, we improve that model to cover out-of-vocabulary words and verbal tenses and apply it to cross-language document retrieval, categorisation, and plagiarism detection. Finally, we study the use of knowledge graphs for the NLP tasks of community questions answering, native language identification, and language variety identification. The contributions of this thesis manifest the potential of knowledge graphs as a cross-domain and cross-language representation of text and its meaning for NLP and IR tasks. These contributions have been published in several international conferences and journals.El Procesamiento del Lenguaje Natural (PLN) es un campo de la informática, la inteligencia artificial y la lingüística computacional centrado en las interacciones entre las máquinas y el lenguaje de los humanos. Uno de sus mayores desafíos implica capacitar a las máquinas para inferir el significado del lenguaje natural humano. Con este propósito, diversas representaciones del significado y el contexto han sido propuestas obteniendo un rendimiento competitivo. Sin embargo, estas representaciones todavía tienen un margen de mejora en escenarios transdominios y translingües. En esta tesis estudiamos el uso de grafos de conocimiento como una representación transdominio y translingüe del texto y su significado. Un grafo de conocimiento es un grafo que expande y relaciona los conceptos originales pertenecientes a un conjunto de palabras. Sus propiedades se consiguen gracias al uso como base de conocimiento de una red semántica multilingüe de amplia cobertura. Esto permite tener una cobertura de cientos de lenguajes y millones de conceptos generales y específicos del ser humano. Como punto de partida de nuestra investigación empleamos características basadas en grafos de conocimiento - junto con otras tradicionales y meta-aprendizaje - para la tarea de PLN de clasificación de la polaridad mono- y transdominio. El análisis y conclusiones de ese trabajo muestra evidencias de que los grafos de conocimiento capturan el significado de una forma independiente del dominio. La siguiente parte de nuestra investigación aprovecha la capacidad de la red semántica multilingüe y se centra en tareas de Recuperación de Información (RI). Primero proponemos un modelo de análisis de similitud completamente basado en grafos de conocimiento para detección de plagio translingüe. A continuación, mejoramos ese modelo para cubrir palabras fuera de vocabulario y tiempos verbales, y lo aplicamos a las tareas translingües de recuperación de documentos, clasificación, y detección de plagio. Por último, estudiamos el uso de grafos de conocimiento para las tareas de PLN de respuesta de preguntas en comunidades, identificación del lenguaje nativo, y identificación de la variedad del lenguaje. Las contribuciones de esta tesis ponen de manifiesto el potencial de los grafos de conocimiento como representación transdominio y translingüe del texto y su significado en tareas de PLN y RI. Estas contribuciones han sido publicadas en diversas revistas y conferencias internacionales.El Processament del Llenguatge Natural (PLN) és un camp de la informàtica, la intel·ligència artificial i la lingüística computacional centrat en les interaccions entre les màquines i el llenguatge dels humans. Un dels seus majors reptes implica capacitar les màquines per inferir el significat del llenguatge natural humà. Amb aquest propòsit, diverses representacions del significat i el context han estat proposades obtenint un rendiment competitiu. No obstant això, aquestes representacions encara tenen un marge de millora en escenaris trans-dominis i trans-llenguatges. En aquesta tesi estudiem l'ús de grafs de coneixement com una representació trans-domini i trans-llenguatge del text i el seu significat. Un graf de coneixement és un graf que expandeix i relaciona els conceptes originals pertanyents a un conjunt de paraules. Les seves propietats s'aconsegueixen gràcies a l'ús com a base de coneixement d'una xarxa semàntica multilingüe d'àmplia cobertura. Això permet tenir una cobertura de centenars de llenguatges i milions de conceptes generals i específics de l'ésser humà. Com a punt de partida de la nostra investigació emprem característiques basades en grafs de coneixement - juntament amb altres tradicionals i meta-aprenentatge - per a la tasca de PLN de classificació de la polaritat mono- i trans-domini. L'anàlisi i conclusions d'aquest treball mostra evidències que els grafs de coneixement capturen el significat d'una forma independent del domini. La següent part de la nostra investigació aprofita la capacitat\hyphenation{ca-pa-ci-tat} de la xarxa semàntica multilingüe i se centra en tasques de recuperació d'informació (RI). Primer proposem un model d'anàlisi de similitud completament basat en grafs de coneixement per a detecció de plagi trans-llenguatge. A continuació, vam millorar aquest model per cobrir paraules fora de vocabulari i temps verbals, i ho apliquem a les tasques trans-llenguatges de recuperació de documents, classificació, i detecció de plagi. Finalment, estudiem l'ús de grafs de coneixement per a les tasques de PLN de resposta de preguntes en comunitats, identificació del llenguatge natiu, i identificació de la varietat del llenguatge. Les contribucions d'aquesta tesi posen de manifest el potencial dels grafs de coneixement com a representació trans-domini i trans-llenguatge del text i el seu significat en tasques de PLN i RI. Aquestes contribucions han estat publicades en diverses revistes i conferències internacionals.Franco Salvador, M. (2017). A Cross-domain and Cross-language Knowledge-based Representation of Text and its Meaning [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/84285TESISCompendi

Beyond Question Answering: Understanding the Information Need of the User

Intelligent interaction between humans and computers has been a dream of artificial intelligence since the beginning of digital era and one of the original motivations behind the creation of artificial intelligence. A key step towards the achievement of such an ambitious goal is to enable the Question Answering systems understand the information need of the user. In this thesis, we attempt to enable the QA system's ability to understand the user's information need by three approaches. First, an clarification question generation method is proposed to help the user clarify the information need and bridge information need gap between QA system and the user. Next, a translation based model is obtained from the large archives of Community Question Answering data, to model the information need behind a question and boost the performance of question recommendation. Finally, a fine-grained classification framework is proposed to enable the systems to recommend answered questions based on information need satisfaction

Language representations for computational argumentation

Argumentation is an essential feature and, arguably, one of the most exciting phenomena of natural language use. Accordingly, it has fascinated scholars and researchers in various fields, such as linguistics and philosophy, for long. Its computational analysis, falling under the notion of computational argumentation, is useful in a variety of domains of text for a range of applications. For instance, it can help to understand users’ stances in online discussion forums towards certain controversies, to provide targeted feedback to users for argumentative writing support, and to automatically summarize scientific publications. As in all natural language processing pipelines, the text we would like to analyze has to be introduced to computational argumentation models in the form of numeric features. Choosing such suitable semantic representations is considered a core challenge in natural language processing. In this context, research employing static and contextualized pretrained text embedding models has recently shown to reach state-of-the-art performances for a range of natural language processing tasks. However, previous work has noted the specific difficulty of computational argumentation scenarios with language representations as one of the main bottlenecks and called for targeted research on the intersection of the two fields. Still, the efforts focusing on the interplay between computational argumentation and representation learning have been few and far apart. This is despite (a) the fast-growing body of work in both computational argumentation and representation learning in general and (b) the fact that some of the open challenges are well known in the natural language processing community. In this thesis, we address this research gap and acknowledge the specific importance of research on the intersection of representation learning and computational argumentation. To this end, we (1) identify a series of challenges driven by inherent characteristics of argumentation in natural language and (2) present new analyses, corpora, and methods to address and mitigate each of the identified issues. Concretely, we focus on five main challenges pertaining to the current state-of-the-art in computational argumentation: (C1) External knowledge: static and contextualized language representations encode distributional knowledge only. We propose two approaches to complement this knowledge with knowledge from external resources. First, we inject lexico-semantic knowledge through an additional prediction objective in the pretraining stage. In a second study, we demonstrate how to inject conceptual knowledge post hoc employing the adapter framework. We show the effectiveness of these approaches on general natural language understanding and argumentative reasoning tasks. (C2) Domain knowledge: pretrained language representations are typically trained on big and general-domain corpora. We study the trade-off between employing such large and general-domain corpora versus smaller and domain-specific corpora for training static word embeddings which we evaluate in the analysis of scientific arguments. (C3) Complementarity of knowledge across tasks: many computational argumentation tasks are interrelated but are typically studied in isolation. In two case studies, we show the effectiveness of sharing knowledge across tasks. First, based on a corpus of scientific texts, which we extend with a new annotation layer reflecting fine-grained argumentative structures, we show that coupling the argumentative analysis with other rhetorical analysis tasks leads to performance improvements for the higher-level tasks. In the second case study, we focus on assessing the argumentative quality of texts. To this end, we present a new multi-domain corpus annotated with ratings reflecting different dimensions of argument quality. We then demonstrate the effectiveness of sharing knowledge across the different quality dimensions in multi-task learning setups. (C4) Multilinguality: argumentation arguably exists in all cultures and languages around the globe. To foster inclusive computational argumentation technologies, we dissect the current state-of-the-art in zero-shot cross-lingual transfer. We show big drops in performance when it comes to resource-lean and typologically distant target languages. Based on this finding, we analyze the reasons for these losses and propose to move to inexpensive few-shot target-language transfer, leading to consistent performance improvements in higher-level semantic tasks, e.g., argumentative reasoning. (C5) Ethical considerations: envisioned computational argumentation applications, e.g., systems for self-determined opinion formation, are highly sensitive. We first discuss which ethical aspects should be considered when representing natural language for computational argumentation tasks. Focusing on the issue of unfair stereotypical bias, we then conduct a multi-dimensional analysis of the amount of bias in monolingual and cross-lingual embedding spaces. In the next step, we devise a general framework for implicit and explicit bias evaluation and debiasing. Employing intrinsic bias measures and benchmarks reflecting the semantic quality of the embeddings, we demonstrate the effectiveness of new debiasing methods, which we propose. Finally, we complement this analysis by testing the original as well as the debiased language representations for stereotypically unfair bias in argumentative inferences. We hope that our contributions in language representations for computational argumentation fuel more research on the intersection of the two fields and contribute to fair, efficient, and effective natural language processing technologies

Mapping vegetation with remote sensing and GIS data using object-based analysis and machine learning algorithms

Remote sensing technology is an efficient tool for various practical applications of environmental resources management. Advances in this technology include the diverse range of high quality data sources and image analysis techniques. Object-based image analysis (OBIA) and machine learning algorithms are recent advances, which this thesis evaluates. OBIA and machine learning algorithms are first tested using a combination of multiple datasets for identifying individual tree species. These datasets include Quickbird, LiDAR, and GIS derived terrain data. Improvements in tree species classification were obtained and the best data combination was terrain context (based on slope, elevation, and wetness), tree height, canopy shape, and branch density (based on LiDAR return intensity). The availability of a range of classifiers and different data pre-processing techniques adds to the complexity of image analysis. The combinations of these techniques result in a large number of potential outcomes and these need to be evaluated. Therefore, the second part of this research investigated and compared tree species classification performance for different methods (Naïve Bayes - NB , Logistic Regression - LR, Random Forest - RF, and Support Vector Machine - SVM), combined with various dimensionality reduction (DR) methods (Correlation-based feature selection filter, Information Gain, Wrapper methods, and Principal Component Analysis). When DR was used prior to classification, only the NB classifier had a significant improvement in accuracy. SVM and RF had the best classification accuracy, and this was achieved without DR. The final part of this thesis demonstrates a new method using OBIA for mapping the biomass change of mangrove forests in Vietnam between 2000 and 2011 from SPOT images. First, three different mangrove associations were identified using two levels of image segmentation followed by a SVM classifier and a range of spectral, texture and GIS information for classification. The RF regression model that integrated spectral, vegetation association type, texture, and vegetation indices obtained the highest accuracy

Text Summarization Across High and Low-Resource Settings

Natural language processing aims to build automated systems that can both understand and generate natural language textual data. As the amount of textual data available online has increased exponentially, so has the need for intelligence systems to comprehend and present it to the world. As a result, automatic text summarization, the process by which a text\u27s salient content is automatically distilled into a concise form, has become a necessary tool. Automatic text summarization approaches and applications vary based on the input summarized, which may constitute single or multiple documents of different genres. Furthermore, the desired output style may consist of a sentence or sub-sentential units chosen directly from the input in extractive summarization or a fusion and paraphrase of the input document in abstractive summarization. Despite differences in the above use-cases, specific themes, such as the role of large-scale data for training these models, the application of summarization models in real-world scenarios, and the need for adequately evaluating and comparing summaries, are common across these settings. This dissertation presents novel data and modeling techniques for deep neural network-based summarization models trained across high-resource (thousands of supervised training examples) and low-resource (zero to hundreds of supervised training examples) data settings and a comprehensive evaluation of the model and metric progress in the field. We examine both Recurrent Neural Network (RNN)-based and Transformer-based models to extract and generate summaries from the input. To facilitate the training of large-scale networks, we introduce datasets applicable for multi-document summarization (MDS) for pedagogical applications and for news summarization. While the high-resource settings allow models to advance state-of-the-art performance, the failure of such models to adapt to settings outside of that in which it was initially trained requires smarter use of labeled data and motivates work in low-resource summarization. To this end, we propose unsupervised learning techniques for both extractive summarization in question answering, abstractive summarization on distantly-supervised data for summarization of community question answering forums, and abstractive zero and few-shot summarization across several domains. To measure the progress made along these axes, we revisit the evaluation of current summarization models. In particular, this dissertation addresses the following research objectives: 1) High-resource Summarization. We introduce datasets for multi-document summarization, focusing on pedagogical applications for NLP, news summarization, and Wikipedia topic summarization. Large-scale datasets allow models to achieve state-of-the-art performance on these tasks compared to prior modeling techniques, and we introduce a novel model to reduce redundancy. However, we also examine how models trained on these large-scale datasets fare when applied to new settings, showing the need for more generalizable models. 2) Low-resource Summarization. While high-resource summarization improves model performance, for practical applications, data-efficient models are necessary. We propose a pipeline for creating synthetic training data for training extractive question-answering models, a form of query-based extractive summarization with short-phrase summaries. In other work, we propose an automatic pipeline for training a multi-document summarizer in answer summarization on community question-answering forums without labeled data. Finally, we push the boundaries of abstractive summarization model performance when little or no training data is available across several domains. 3) Automatic Summarization Evaluation. To understand the extent of progress made across recent modeling techniques and better understand the current evaluation protocols, we examine the current metrics used to compare summarization output quality across 12 metrics across 23 deep neural network models and propose better-motivated summarization evaluation guidelines as well as point to open problems in summarization evaluation

Expanding the Horizons of Manufacturing: Towards Wide Integration, Smart Systems and Tools

This research topic aims at enterprise-wide modeling and optimization (EWMO) through the development and application of integrated modeling, simulation and optimization methodologies, and computer-aided tools for reliable and sustainable improvement opportunities within the entire manufacturing network (raw materials, production plants, distribution, retailers, and customers) and its components. This integrated approach incorporates information from the local primary control and supervisory modules into the scheduling/planning formulation. That makes it possible to dynamically react to incidents that occur in the network components at the appropriate decision-making level, requiring fewer resources, emitting less waste, and allowing for better responsiveness in changing market requirements and operational variations, reducing cost, waste, energy consumption and environmental impact, and increasing the benefits. More recently, the exploitation of new technology integration, such as through semantic models in formal knowledge models, allows for the capture and utilization of domain knowledge, human knowledge, and expert knowledge toward comprehensive intelligent management. Otherwise, the development of advanced technologies and tools, such as cyber-physical systems, the Internet of Things, the Industrial Internet of Things, Artificial Intelligence, Big Data, Cloud Computing, Blockchain, etc., have captured the attention of manufacturing enterprises toward intelligent manufacturing systems

Tune your brown clustering, please

Brown clustering, an unsupervised hierarchical clustering technique based on ngram mutual information, has proven useful in many NLP applications. However, most uses of Brown clustering employ the same default configuration; the appropriateness of this configuration has gone predominantly unexplored. Accordingly, we present information for practitioners on the behaviour of Brown clustering in order to assist hyper-parametre tuning, in the form of a theoretical model of Brown clustering utility. This model is then evaluated empirically in two sequence labelling tasks over two text types. We explore the dynamic between the input corpus size, chosen number of classes, and quality of the resulting clusters, which has an impact for any approach using Brown clustering. In every scenario that we examine, our results reveal that the values most commonly used for the clustering are sub-optimal

Modeling user information needs on mobile devices: from recommendation to conversation

Recent advances in the development of mobile devices, equipped with multiple sensors, together with the availability of millions of applications have made these devices more pervasive in our lives than ever. The availability of the diverse set of sensors, as well as high computational power, enable information retrieval (IR) systems to sense a user’s context and personalize their results accordingly. Relevant studies show that people use their mobile devices to access information in a wide range of topics in various contextual situations, highlighting the fact that modeling user information need on mobile devices involves studying several means of information access. In this thesis, we study three major aspects of information access on mobile devices. First, we focus on proactive approaches to modeling users for venue suggestion. We investigate three methods of user modeling, namely, content-based, collaborative, and hybrid, focusing on personalization and context-awareness. We propose a two-phase collaborative ranking algorithm for leveraging users’ implicit feedback while incorporating temporal and geographical information into the model. We then extend our collaborative model to include multiple cross-venue similarity scores and combine it with our content-based approach to produce a hybrid recommendation. Second, we introduce and investigate a new task on mobile search, that is, unified mobile search. We take the first step in defining, studying, and modeling this task by collecting two datasets and conducting experiments on one of the main components of unified mobile search frameworks, that is target apps selection. To this end, we propose two neural approaches. Finally, we address the conversational aspect of mobile search where we propose an offline evaluation protocol and build a dataset for asking clarifying questions for conversational search. Also, we propose a retrieval framework consisting of three main components: question retrieval, question selection, and document retrieval. The experiments and analyses indicate that asking clarifying questions should be an essential part of a conversational system, resulting in high performance gain

O impacto da inteligência artificial no negócio eletrónico

Pela importância que a Inteligência Artificial exibe na atualidade, revela-se de grande interesse verificar até que ponto ela está a transformar o Negócio Eletrónico. Para esse efeito, delineou-se uma revisão sistemática com o objetivo de avaliar os impactos da proliferação destes instrumentos. A investigação empreendida pretendeu identificar artigos científicos que, através de pesquisas realizadas a Fontes de Dados Eletrónicas, pudessem responder às questões de investigação implementadas: a) que tipo de soluções, baseadas na Inteligência Artificial (IA), têm sido usadas para melhorar o Negócio Eletrónico (NE); b) em que domínios do NE a IA foi aplicada; c) qual a taxa de sucesso ou fracasso do projeto. Simultaneamente, tiveram de respeitar critérios de seleção, nomeadamente, estar escritos em inglês, encontrarem-se no intervalo temporal 2015/2021 e tratar-se de estudos empíricos, suportados em dados reais. Após uma avaliação de qualidade final, procedeu-se à extração dos dados pertinentes para a investigação, para formulários criados em MS Excel. Estes dados estiveram na base da análise quantitativa e qualitativa que evidenciaram as descobertas feitas e sobre os quais se procedeu, posteriormente, à sua discussão. A dissertação termina com as conclusão e discussão de trabalhos futuros.Due to the importance that Artificial Intelligence exhibits today, it is of great interest to see to what extent it is transforming the Electronic Business. To this end, a systematic review was designed to evaluate the impacts of the proliferation of these instruments. The research aimed to identify scientific articles that, through research carried out on Electronic Data Sources, could answer the research questions implemented: a) what kind of solutions, based on Artificial Intelligence, have been used to improve the Electronic Business; b) in which areas of the Electronic Business Artificial Intelligence has been applied; c) what the success rate or failure of the project is. At the same time, they must comply with selection criteria, to be written in English, to be found in the 2015/2021-time interval and to be empirical studies supported by actual data. After a final quality evaluation, the relevant data for the investigation were extracted for forms created in MS Excel. These data were the basis of the quantitative and qualitative analysis that evidenced the findings found and on which they were subsequently discussed. The dissertation ends with the conclusion and discussion of future works

Fuzzy Sets in Business Management, Finance, and Economics

This book collects fifteen papers published in s Special Issue of Mathematics titled “Fuzzy Sets in Business Management, Finance, and Economics”, which was published in 2021. These paper cover a wide range of different tools from Fuzzy Set Theory and applications in many areas of Business Management and other connected fields. Specifically, this book contains applications of such instruments as, among others, Fuzzy Set Qualitative Comparative Analysis, Neuro-Fuzzy Methods, the Forgotten Effects Algorithm, Expertons Theory, Fuzzy Markov Chains, Fuzzy Arithmetic, Decision Making with OWA Operators and Pythagorean Aggregation Operators, Fuzzy Pattern Recognition, and Intuitionistic Fuzzy Sets. The papers in this book tackle a wide variety of problems in areas such as strategic management, sustainable decisions by firms and public organisms, tourism management, accounting and auditing, macroeconomic modelling, the evaluation of public organizations and universities, and actuarial modelling. We hope that this book will be useful not only for business managers, public decision-makers, and researchers in the specific fields of business management, finance, and economics but also in the broader areas of soft mathematics in social sciences. Practitioners will find methods and ideas that could be fruitful in current management issues. Scholars will find novel developments that may inspire further applications in the social sciences