CNGL-CORE: Referential translation machines for measuring semantic similarity

We invent referential translation machines (RTMs), a computational model for identifying the translation acts between any two data sets with respect to a reference corpus selected in the same domain, which can be used for judging the semantic similarity between text. RTMs make quality and semantic similarity judgments possible by using retrieved relevant training data as interpretants for reaching shared semantics. An MTPP (machine translation performance predictor) model derives features measuring the closeness of the test sentences to the training data, the difficulty of translating them, and the presence of acts of translation involved. We view semantic similarity as paraphrasing between any two given texts. Each view is modeled by an RTM model, giving us a new perspective on the binary relationship between the two. Our prediction model is the $15$th on some tasks and $30$th overall out of $89$ submissions in total according to the official results of the Semantic Textual Similarity (STS 2013) challenge

CNGL: Grading student answers by acts of translation

We invent referential translation machines (RTMs), a computational model for identifying the translation acts between any two data sets with respect to a reference corpus selected in the same domain, which can be used for automatically grading student answers. RTMs make quality and semantic similarity judgments possible by using retrieved relevant training data as interpretants for reaching shared semantics. An MTPP (machine translation performance predictor) model derives features measuring the closeness of the test sentences to the training data, the difficulty of translating them, and the presence of acts of translation involved. We view question answering as translation from the question to the answer, from the question to the reference answer, from the answer to the reference answer, or from the question and the answer to the reference answer. Each view is modeled by an RTM model, giving us a new perspective on the ternary relationship between the question, the answer, and the reference answer. We show that all RTM models contribute and a prediction model based on all four perspectives performs the best. Our prediction model is the $2$nd best system on some tasks according to the official results of the Student Response Analysis (SRA 2013) challenge

Referential translation machines for predicting translation quality

We use referential translation machines (RTM) for quality estimation of translation outputs. RTMs are a computational model for identifying the translation acts between any two data sets with respect to interpretants selected in the same domain, which are effective when making monolingual and bilingual similarity judgments. RTMs achieve top performance in automatic, accurate, and language independent prediction of sentence-level and word-level statistical machine translation (SMT) quality. RTMs remove the need to access any SMT system specific information or prior knowledge of the training data or models used when generating the translations and achieve the top performance in WMT13 quality estimation task (QET13). We improve our RTM models with the Parallel FDA5 instance selection model, with additional features for predicting the translation performance, and with improved learning models. We develop RTM models for each WMT14 QET (QET14) subtask, obtain improvements over QET13 results, and rank $1$st in all of the tasks and subtasks of QET14

RTM-DCU: referential translation machines for semantic similarity

We use referential translation machines (RTMs) for predicting the semantic similarity of text. RTMs are a computational model for identifying the translation acts between any two data sets with respect to interpretants selected in the same domain, which are effective when making monolingual and bilingual similarity judgments. RTMs judge the quality or the semantic similarity of text by using retrieved relevant training data as interpretants for reaching shared semantics. We derive features measuring the closeness of the test sentences to the training data via interpretants, the difficulty of translating them, and the presence of the acts of translation, which may ubiquitously be observed in communication. RTMs provide a language independent approach to all similarity tasks and achieve top performance when predicting monolingual cross-level semantic similarity (Task 3) and good results in semantic relatedness and entailment (Task 1) and multilingual semantic textual similarity (STS) (Task 10). RTMs remove the need to access any task or domain specific information or resource

Consensus Ontologies in Socially Interacting MultiAgent Systems

This paper presents approaches for building, managing, and evaluating consensus ontologies from the individual ontologies of a network of socially interacting agents. Each agent has its own conceptualization of the world within the multiagent system framework. The interactions between agents are modeled by sending queries and receiving responses and later assessing each other’s performance based on the results. This model enables us to measure the quality of the societal beliefs in the resources which we represent as the expertise in each domain. The dynamic nature of our system allows us to model the emergence of consensus that mimics the evolution of language. We present an algorithm for generating the consensus ontologies which makes use of the authoritative agent’s conceptualization in a given domain. As the expertise of agents changes after a number of interactions, the consensus ontology that we build based on the agents ’ individual views evolves. The resulting approach is concordant with the principles of emergent semantics. We provide formal definitions for the problem of finding a consensus ontology in a step by step manner. We evaluate the consensus ontologies by using different heuristic measures of similarity based on the component ontologies. Conceptual processing methods for generating, manipulating, and evaluating consensus ontologies are given and experimental results are presented. The presented approach looks promising and opens new directions for further research.