Representation Learning for Natural Language Processing

This open access book provides an overview of the recent advances in representation learning theory, algorithms and applications for natural language processing (NLP). It is divided into three parts. Part I presents the representation learning techniques for multiple language entries, including words, phrases, sentences and documents. Part II then introduces the representation techniques for those objects that are closely related to NLP, including entity-based world knowledge, sememe-based linguistic knowledge, networks, and cross-modal entries. Lastly, Part III provides open resource tools for representation learning techniques, and discusses the remaining challenges and future research directions. The theories and algorithms of representation learning presented can also benefit other related domains such as machine learning, social network analysis, semantic Web, information retrieval, data mining and computational biology. This book is intended for advanced undergraduate and graduate students, post-doctoral fellows, researchers, lecturers, and industrial engineers, as well as anyone interested in representation learning and natural language processing

Proceedings of the First Workshop on Computing News Storylines (CNewsStory 2015)

This volume contains the proceedings of the 1st Workshop on Computing News Storylines (CNewsStory 2015) held in conjunction with the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2015) at the China National Convention Center in Beijing, on July 31st 2015. Narratives are at the heart of information sharing. Ever since people began to share their experiences, they have connected them to form narratives. The study od storytelling and the field of literary theory called narratology have developed complex frameworks and models related to various aspects of narrative such as plots structures, narrative embeddings, characters’ perspectives, reader response, point of view, narrative voice, narrative goals, and many others. These notions from narratology have been applied mainly in Artificial Intelligence and to model formal semantic approaches to narratives (e.g. Plot Units developed by Lehnert (1981)). In recent years, computational narratology has qualified as an autonomous field of study and research. Narrative has been the focus of a number of workshops and conferences (AAAI Symposia, Interactive Storytelling Conference (ICIDS), Computational Models of Narrative). Furthermore, reference annotation schemes for narratives have been proposed (NarrativeML by Mani (2013)). The workshop aimed at bringing together researchers from different communities working on representing and extracting narrative structures in news, a text genre which is highly used in NLP but which has received little attention with respect to narrative structure, representation and analysis. Currently, advances in NLP technology have made it feasible to look beyond scenario-driven, atomic extraction of events from single documents and work towards extracting story structures from multiple documents, while these documents are published over time as news streams. Policy makers, NGOs, information specialists (such as journalists and librarians) and others are increasingly in need of tools that support them in finding salient stories in large amounts of information to more effectively implement policies, monitor actions of “big players” in the society and check facts. Their tasks often revolve around reconstructing cases either with respect to specific entities (e.g. person or organizations) or events (e.g. hurricane Katrina). Storylines represent explanatory schemas that enable us to make better selections of relevant information but also projections to the future. They form a valuable potential for exploiting news data in an innovative way.JRC.G.2-Global security and crisis managemen

Technologies to enhance self-directed learning from hypertext

With the growing popularity of the World Wide Web, materials presented to learners in the form of hypertext have become a major instructional resource. Despite the potential of hypertext to facilitate access to learning materials, self-directed learning from hypertext is often associated with many concerns. Self-directed learners, due to their different viewpoints, may follow different navigation paths, and thus they will have different interactions with knowledge. Therefore, learners can end up being disoriented or cognitively-overloaded due to the potential gap between what they need and what actually exists on the Web. In addition, while a lot of research has gone into supporting the task of finding web resources, less attention has been paid to the task of supporting the interpretation of Web pages. The inability to interpret the content of pages leads learners to interrupt their current browsing activities to seek help from other human resources or explanatory learning materials. Such activity can weaken learner engagement and lower their motivation to learn. This thesis aims to promote self-directed learning from hypertext resources by proposing solutions to the above problems. It first presents Knowledge Puzzle, a tool that proposes a constructivist approach to learn from the Web. Its main contribution to Web-based learning is that self-directed learners will be able to adapt the path of instruction and the structure of hypertext to their way of thinking, regardless of how the Web content is delivered. This can effectively reduce the gap between what they need and what exists on the Web. SWLinker is another system proposed in this thesis with the aim of supporting the interpretation of Web pages using ontology based semantic annotation. It is an extension to the Internet Explorer Web browser that automatically creates a semantic layer of explanatory information and instructional guidance over Web pages. It also aims to break the conventional view of Web browsing as an individual activity by leveraging the notion of ontology-based collaborative browsing. Both of the tools presented in this thesis were evaluated by students within the context of particular learning tasks. The results show that they effectively fulfilled the intended goals by facilitating learning from hypertext without introducing high overheads in terms of usability or browsing efforts

Unsupervised entity linking using graph-based semantic similarity

Nowadays, the human textual data constitutes a great proportion of the shared information resources such as World Wide Web (WWW). Social networks, news and learning resources as well as Knowledge Bases (KBs) are just the small examples that widely contain the textual data which is used by both human and machine readers. The nature of human languages is highly ambiguous, means that a short portion of a textual context (such as words or phrases) can semantically be interpreted in different ways. A language processor should detect the best interpretation depending on the context in which each word or phrase appears. In case of human readers, the brain is quite proficient in interfering textual data. Human language developed in a way that reflects the innate ability provided by the brain’s neural networks. However, there still exist the moments that the text disambiguation task would remain a hard challenge for the human readers. In case of machine readers, it has been a long-term challenge to develop the ability to do natural language processing and machine learning. Different interpretation can change the broad range of topics and targets. The different in interpretation can cause serious impacts when it is used in critical domains that need high precision. Thus, the correctly inferring the ambiguous words would be highly crucial. To tackle it, two tasks have been developed: Word Sense Disambiguation (WSD) to infer the sense (i.e. meaning) of ambiguous words, when the word has multiple meanings, and Entity Linking (EL) (also called, Named Entity Disambiguation–NED, Named Entity Recognition and Disambiguation–NERD, or Named Entity Normalization–NEN) which is used to explore the correct reference of Named Entity (NE) mentions occurring in documents. The solution to these problems impacts other computer-related writing, such as discourse, improving relevance of search engines, anaphora resolution, coherence, and inference. This document summarizes the works towards developing an unsupervised Entity Linking (EL) system using graph-based semantic similarity aiming to disambiguate Named Entity (NE) mentions occurring in a target document. The EL task is highly challenging since each entity can usually be referred to by several NE mentions (synonymy). In addition, a NE mention may be used to indicate distinct entities (polysemy). Thus, much effort is necessary to tackle these challenges. Our EL system disambiguates the NE mentions in several steps. For each step, we have proposed, implemented, and evaluated several approaches. We evaluated our EL system in TAC-KBP4 English EL evaluation framework in which the system input consists of a set of queries, each containing a query name (target NE mention) along with start and end offsets of that mention in the target document. The output is either a NE entry id in a reference Knowledge Base (KB) or a Not-in-KB (NIL) id in the case that system could not find any appropriate entry for that query. At the end, we have analyzed our result in different aspects. To disambiguate query name we apply a graph-based semantic similarity approach to extract the network of the semantic knowledge existing in the content of target document.Este documento es un resumen del trabajo realizado para la construccion de un sistema de Entity Linking (EL) destinado a desambiguar menciones de Entidades Nombradas (Named Entities, NE) que aparecen en un documento de referencia. La tarea de EL presenta una gran dificultad ya que cada entidad puede ser mencionada de varias maneras (sinonimia). Ademas cada mencion puede referirse a mas de una entidad (polisemia). Asi pues, se debe realizar un gran esfuerzo para hacer frente a estos retos. Nuestro sistema de EL lleva a cabo la desambiguacion de las menciones de NE en varias etapas. Para cada etapa hemos propuesto, implementado y evaluado varias aproximaciones. Hemos evaluado nuestro sistema de EL en el marco del TAC-KBP English EL evaluation framework. En este marco la evaluacion se realiza a partir de una entrada que consiste en un conjunto de consultas cada una de las cuales consta de un nombre (query name) que corresponde a una mencion objetivo cuya posicion en un documento de referencia se indica. La salida debe indicar a que entidad en una base de conocimiento (Knowledge Base, KB) corresponde la mencion. En caso de no existir un referente apropiado la respuesta sera Not-in-KB (NIL). La tesis concluye con un analisis pormenorizado de los resultados obtenidos en la evaluacion.Postprint (published version

Emotion word processing: evidence from electrophysiology, eye movements and decision making

A degree of confusion currently exists regarding how the emotionality of a textual stimulus influences its processing. Despite a wealth of research recently being conducted in the area, heterogeneity of stimuli used and methodologies utilized prevented general conclusion from being confidently drawn. This thesis aimed to clarify understanding of cognitive processes associated with emotional textual stimuli by employing well controlled stimuli in a range of simple but innovative paradigms. Emotion words used in this thesis were defined by their valence and arousal ratings. The questions asked here concerned early stages of processing of emotional words, the attention capturing properties of such words, any spill-over effects which would impact the processing of neutral text presented subsequently to the emotional material, and the effect of emotional words on higher cognitive processes such as attitude formation. The first experiment (Chapter 2) manipulated the emotionality of words (positive, negative, neutral) and their frequency (HF – high frequency, LF – low frequency) while ERPs were recorded. An emotion x frequency interaction was found, with emotional LF words responded to fastest, but only positive LF words responded to fastest. Negative HF words were also associated with a large N1 component. Chapter 3 investigated the attention-capturing properties of positive and negative words presented above and below a central fixation cross. The only significant effects appeared when a positive word was presented in the top condition, and a negative word in the bottom condition. Here saccade latencies were longer and there were a fewer number of errors made. Chapter 4 reports an eye tracking study which examined the effect of target words’ emotion (positive, negative, neutral) and their frequency (HF, LF). The pattern of results, produced in a variety of fixation time measurements such as first fixation duration and single fixation duration, was similar to those reported in Chapter 2. The existence of any spill-over effect of emotion onto subsequently presented neutral text was examined in a number of ways. Chapter 5 describes priming with emotional primes and neutral targets but no effect of emotion was found. Chapter 6 employed the same design as Chapter 4 but presented positive, negative or neutral sentences in the middle of neutral paragraphs. It was found that the positive sentences were read fastest, but the neutral sentences following the negative sentences were read faster than those following neutral sentences. Chapters 7 and 8 employed a version of the Velten mood-induction tool to examine the effect of mood when reading emotional text. Chapter 7 was a replication of Chapter 4 with 4 participant groups: positive, negative and neutral mood. While the neutral group showed similar results to those produced in Chapter 4, the positive group only fixated on the positive HF words faster, the negative group showed a frequency effect within each emotional word type, but within HF words positive words were viewed for less time than neutral words. Chapter 8 had participants read 4 product reviews and then afterwards rate each of the products on a set of semantic differentials. This was a 3 (mood: positive, negative, neutral) x 2 (message type: positive negative) x 2 (word type: positive negative). There was no effect of mood but positive messages were read quicker when they contained positive words and negative messages were read quicker when they contained negative words. Participants were asked to recommend each product to individuals in either a prevention in a promotion focus. When the focus was prevention there were additive effects of message and word type, but when the focus was positive there was an interaction, with the positive message conveyed using negative words being rated highest. The same pattern also emerged in the series of semantic differentials. Possible mechanisms to account for these findings are discussed, including many incarnations of McGinnies’s (1949) perceptual defense theory. Future studies should possibly aim to combine the current knowledge with motivational, goal-orientated models such as Higgins’s (1998) theory of regulatory focus

Unravelling the Influence of Online Social Context on Consumer Health Information Technology (CHIT) Implementations

While health information technology research has examined a variety of topics (e.g., adoption and assimilation of technology within healthcare organizations, critical success factors), it has remained unclear how the uniqueness of the online context (e.g., users connecting with strangers for social and emotional support) influences consumer health information technology (CHIT) implementations. Towards this goal, this dissertation examines the influence of online social context on CHIT implementations and outcomes. Using theories from social psychology, this dissertation encompasses two empirical research essays. The first essay draws on the environmental enrichment concept to examine the influential role of the online social context of a gamified CHIT on its success. By surveying existing fitness technology users, we demonstrate the influence of the social context enabled by CHITs on behavioral adherence to exercise. The second essay draws on construal level theory to examine the influence of textual information (such as race, geographic location) in online patient communities on a user’s trust of the community and the system as well as their intentions to participate in them. Using randomized experiments, we identify some of the propinquity-related factors that influence a user’s trust in online patient communities. The key contribution of this dissertation is the advancement of our understanding of the important role played by the social context enabled by the CHITs