Medical WordNet: A new methodology for the construction and validation of information resources for consumer health

A consumer health information system must be able to comprehend both expert and non-expert medical vocabulary and to map between the two. We describe an ongoing project to create a new lexical database called Medical WordNet (MWN), consisting of medically relevant terms used by and intelligible to non-expert subjects and supplemented by a corpus of natural-language sentences that is designed to provide medically validated contexts for MWN terms. The corpus derives primarily from online health information sources targeted to consumers, and involves two sub-corpora, called Medical FactNet (MFN) and Medical BeliefNet (MBN), respectively. The former consists of statements accredited as true on the basis of a rigorous process of validation, the latter of statements which non-experts believe to be true. We summarize the MWN / MFN / MBN project, and describe some of its applications

Deep Memory Networks for Natural Conversations

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 장병탁.Attention-based models are firstly proposed in the field of computer vision. And then they spread into natural language processing (NLP). The first one successfully bringing in attention mechanism from computer vision to NLP is neural machine translation. Such attention-based mechanism is motivated from that, instead of decoding based on the encoding of a whole and a fixed-length sentence during one pass of neural network-based machine translation, one can attend a specific part of the sentence. This specific part is what should currently be attended. These parts could be words or phrases. The basic problem that the attention mechanism solves is that it allows the network to refer back to the input sequence, instead of forcing it to encode all information into one fixed-length vector. The attention mechanism is simply giving the network access to its internal memory, which is the hidden state of the encoder. In this point of view, instead of choosing what to attend to, the network chooses what to retrieve from memory. Unlike typical memory, the memory access mechanism here is soft, which means that the network retrieves a weighted combination of all memory locations, not a value from a single discrete location. Making the memory access soft has the benefit that we can easily train the network end-to-end using backpropagation The trend towards more complex memory structures is now continuing. End-to-End Memory Networks allow the network to read same input sequence multiple times before making an output, updating the memory contents at each step. For example, answering a question by making multiple reasoning steps over an input story. However, when the networks parameter weights are tied in a certain way, the memory mechanism in End-to-End Memory Networks identical to the attention mechanism presented here, only that it makes multiple hops over the memory. In this dissertation, we propose the deep memory network with attention mechanism and word/sentence embedding for attention mechanism. Due to the external memory and attention mechanism, proposed method can handle various tasks in natural language processing, such as question and answering, machine comprehension and sentiment analysis. Usually attention mechanism requires huge computational cost. In order to solve this problem. I also propose novel word and sentence embedding methods. Previous embedding methods only use the Markov assumption. But if we consider the language structure and make use of it, it will be very helpful to reduce the computational cost. Also it does not need strong supervision which means the additional information on important sentences.Chapter 1. Introduction 1 1.1 Background and Motivation 1 1.2 Approach and Contributions 3 1.3 Organization of the Dissertation 5 Chapter 2. Related Work 7 2.1 Memory Networks 7 2.2 End-to-End Memory Networks 10 2.3 Dynamic Memory Networks 13 Chapter 3. Conceptual Word Embedding 20 3.1 Related Work 20 3.2 Dependency-Gram 22 3.3 Experimental Results 26 3.4 Discussion and Summary 29 Chapter 4. Sentence Embedding using Context 31 4.1 Related Work 31 4.2 CR-Gram 35 4.3 Experimental Results 41 4.4 Discussion and Summary 43 Chapter 5. Deep Memory Networks 46 5.1 Related Work 46 5.2 Deep Memory Networks 48 5.3 Experimental Results 54 5.3.1 bAbI Dataset 54 5.3.2 Stanford Sentiment Treebank 57 5.3.3 SQuAD Dataset 58 5.4 Discussion and Summary 60 Chapter 6. Concluding Remarks 62 6.1 Summary and Discussion 62 6.2 Future Work 65 References 65 초록 76Docto

The representation of meaning in episodic memory

In several models of long-term memory it is assumed, either explicitly or implicitly, that different meanings of homonyms and even different senses of nonhomonyms have separate representations in long-term memory. While evidence has accrued, particularly from studies employing lexical decision tasks, to suggest that homonyms are multiply represented in semantic memory, claims for multiple representation of homonyms in episodic memory have tended to be made on a purely post hoc basis. The aim of the present research was to determine the manner in which homonyms are represented in episodic memory. A series of experiments were conducted in which either one or two meanings of homonyms were encoded at input. Retention of the homonyms or their biasing nouns was tested in a variety of retrieval contexts. The results obtained were consistent with a conceptualisation of episodic memory in which successive encodings of the same item are represented within the same memory trace which was established on the first occurrence of the item. When to different meanings of a homonym are encoded at input the encoded meanings will be represented within a single memory trace, with each different meaning being represented by an independent set of encoded semantic features. The generality of the framework for episodic memory which is developed is demonstrated through its interpretive application to a wide range of episodic memory phenomena

Diversity of narrative context disrupts the early stage of learning the meanings of novel words

High quality lexical representations develop through repeated exposures to words in different contexts. This preregistered experiment investigated how diversity of narrative context affects the earliest stages of word learning via reading. Adults (N = 100) learned invented meanings for eight pseudowords, which each occurred in five written paragraphs either within a single coherent narrative context or five different narrative contexts. The words’ semantic features were controlled across conditions to avoid influences from polysemy (lexical ambiguity). Posttests included graded measures of word-form recall (spelling accuracy) and recognition (multiple choice), and word-meaning recall (number of semantic features). Diversity of narrative context did not affect word-form learning, but more semantic features were correctly recalled for words trained in a single context. These findings indicate that learning the meanings of novel words is initially boosted by anchoring them to a single coherent narrative discourse

Cross-lingual priming of cognates and interlingual homographs from L2 to L1

Many word forms exist in multiple languages, and can have either the same meaning (cognates) or a different meaning (interlingual homographs). Previous experiments have shown that processing of interlingual homographs in a bilingual’s second language is slowed down by recent experience with these words in the bilingual’s native language, while processing of cognates can be speeded up (Poort et al., 2016; Poort & Rodd, 2019a). The current experiment replicated Poort and Rodd’s (2019a) Experiment 2 but switched the direction of priming: Dutch–English bilinguals (n = 106) made Dutch semantic relatedness judgements to probes related to cognates (n = 50), interlingual homographs (n = 50) and translation equivalents (n = 50) they had seen 15 minutes previously embedded in English sentences. The current experiment is the first to show that a single encounter with an interlingual homograph in one’s second language can also affect subsequent processing in one’s native language. Cross-lingual priming did not affect the cognates. The experiment also extended Poort and Rodd (2019a)’s finding of a large interlingual homograph inhibition effect in a semantic relatedness task in the participants’ L2 to their L1, but again found no evidence for a cognate facilitation effect in a semantic relatedness task. These findings extend the growing literature that emphasises the high level of interaction in a bilingual’s mental lexicon, by demonstrating the influence of L2 experience on the processing of L1 words. Data, scripts, materials and pre-registration available via https://osf.io/2swyg/?view_only=b2ba2e627f6f4eaeac87edab2b59b236

The Latent Relation Mapping Engine: Algorithm and Experiments

Many AI researchers and cognitive scientists have argued that analogy is the core of cognition. The most influential work on computational modeling of analogy-making is Structure Mapping Theory (SMT) and its implementation in the Structure Mapping Engine (SME). A limitation of SME is the requirement for complex hand-coded representations. We introduce the Latent Relation Mapping Engine (LRME), which combines ideas from SME and Latent Relational Analysis (LRA) in order to remove the requirement for hand-coded representations. LRME builds analogical mappings between lists of words, using a large corpus of raw text to automatically discover the semantic relations among the words. We evaluate LRME on a set of twenty analogical mapping problems, ten based on scientific analogies and ten based on common metaphors. LRME achieves human-level performance on the twenty problems. We compare LRME with a variety of alternative approaches and find that they are not able to reach the same level of performance.Comment: related work available at http://purl.org/peter.turney

Word sense extension

Humans often make creative use of words to express novel senses. A long-standing effort in natural language processing has been focusing on word sense disambiguation (WSD), but little has been explored about how the sense inventory of a word may be extended toward novel meanings. We present a paradigm of word sense extension (WSE) that enables words to spawn new senses toward novel context. We develop a framework that simulates novel word sense extension by first partitioning a polysemous word type into two pseudo-tokens that mark its different senses, and then inferring whether the meaning of a pseudo-token can be extended to convey the sense denoted by the token partitioned from the same word type. Our framework combines cognitive models of chaining with a learning scheme that transforms a language model embedding space to support various types of word sense extension. We evaluate our framework against several competitive baselines and show that it is superior in predicting plausible novel senses for over 7,500 English words. Furthermore, we show that our WSE framework improves performance over a range of transformer-based WSD models in predicting rare word senses with few or zero mentions in the training data

Listeners and readers generalise their experience with word meanings across modalities

Research has shown that adults’ lexical-semantic representations are surprisingly malleable. For instance, the interpretation of ambiguous words (e.g. bark) is influenced by experience such that recently encountered meanings become more readily available (Rodd et al., 2016, 2013). However the mechanism underlying this word-meaning priming effect remains unclear, and competing accounts make different predictions about the extent to which information about word meanings that is gained within one modality (e.g. speech) is transferred to the other modality (e.g. reading) to aid comprehension. In two web-based experiments, ambiguous target words were primed with either written or spoken sentences that biased their interpretation toward a subordinate meaning, or were unprimed. About 20 minutes after the prime exposure, interpretation of these target words was tested by presenting them in either written or spoken form, using word association (Experiment 1, N=78) and speeded semantic relatedness decisions (Experiment 2, N=181). Both experiments replicated the auditory unimodal priming effect shown previously (Rodd et al., 2016, 2013) and revealed significant cross-modal priming: primed meanings were retrieved more frequently and swiftly across all primed conditions compared to the unprimed baseline. Furthermore, there were no reliable differences in priming levels between unimodal and cross-modal prime-test conditions. These results indicate that recent experience with ambiguous word meanings can bias the reader’s or listener’s later interpretation of these words in a modality-general way. We identify possible loci of this effect within the context of models of long-term priming and ambiguity resolution

The Effects of Proficiency and Task Context on L2-L1 Noncognate Masked Translation Priming in Chinese-English Bilinguals

The masked translation priming effect was examined in Chinese-English bilinguals using three experimental paradigms: lexical decision, semantic categorization, and speeded episodic recognition. A machine-learning approach was used to assess the subject- and item-specific factors that contribute to the sizes of translation priming effects across these tasks. The factors that contributed to translation priming effects were found to be task specific. Priming effects in lexical decision were associated with higher self-rated listening and writing abilities in English, especially when primes were high-frequency and targets were low-frequency. Priming effects in semantic categorization were associated with more frequent use of English in daily life, especially when targets were high-frequency and primes were low-frequency. Finally, priming effects in episodic recognition were associated with higher self-rated reading, writing, speaking, and listening abilities in English. These results are discussed within different frameworks of current models of bilingual language processing