Automatic Accuracy Prediction for AMR Parsing

Abstract Meaning Representation (AMR) represents sentences as directed, acyclic and rooted graphs, aiming at capturing their meaning in a machine readable format. AMR parsing converts natural language sentences into such graphs. However, evaluating a parser on new data by means of comparison to manually created AMR graphs is very costly. Also, we would like to be able to detect parses of questionable quality, or preferring results of alternative systems by selecting the ones for which we can assess good quality. We propose AMR accuracy prediction as the task of predicting several metrics of correctness for an automatically generated AMR parse - in absence of the corresponding gold parse. We develop a neural end-to-end multi-output regression model and perform three case studies: firstly, we evaluate the model's capacity of predicting AMR parse accuracies and test whether it can reliably assign high scores to gold parses. Secondly, we perform parse selection based on predicted parse accuracies of candidate parses from alternative systems, with the aim of improving overall results. Finally, we predict system ranks for submissions from two AMR shared tasks on the basis of their predicted parse accuracy averages. All experiments are carried out across two different domains and show that our method is effective.Comment: accepted at *SEM 201

A Mention-Ranking Model for Abstract Anaphora Resolution

Resolving abstract anaphora is an important, but difficult task for text understanding. Yet, with recent advances in representation learning this task becomes a more tangible aim. A central property of abstract anaphora is that it establishes a relation between the anaphor embedded in the anaphoric sentence and its (typically non-nominal) antecedent. We propose a mention-ranking model that learns how abstract anaphors relate to their antecedents with an LSTM-Siamese Net. We overcome the lack of training data by generating artificial anaphoric sentence--antecedent pairs. Our model outperforms state-of-the-art results on shell noun resolution. We also report first benchmark results on an abstract anaphora subset of the ARRAU corpus. This corpus presents a greater challenge due to a mixture of nominal and pronominal anaphors and a greater range of confounders. We found model variants that outperform the baselines for nominal anaphors, without training on individual anaphor data, but still lag behind for pronominal anaphors. Our model selects syntactically plausible candidates and -- if disregarding syntax -- discriminates candidates using deeper features.Comment: In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). Copenhagen, Denmar

Dissecting Content and Context in Argumentative Relation Analysis

When assessing relations between argumentative units (e.g., support or attack), computational systems often exploit disclosing indicators or markers that are not part of elementary argumentative units (EAUs) themselves, but are gained from their context (position in paragraph, preceding tokens, etc.). We show that this dependency is much stronger than previously assumed. In fact, we show that by completely masking the EAU text spans and only feeding information from their context, a competitive system may function even better. We argue that an argument analysis system that relies more on discourse context than the argument's content is unsafe, since it can easily be tricked. To alleviate this issue, we separate argumentative units from their context such that the system is forced to model and rely on an EAU's content. We show that the resulting classification system is more robust, and argue that such models are better suited for predicting argumentative relations across documents.Comment: accepted at 6th Workshop on Argument Minin

An Argument-Marker Model for Syntax-Agnostic Proto-Role Labeling

Semantic proto-role labeling (SPRL) is an alternative to semantic role labeling (SRL) that moves beyond a categorical definition of roles, following Dowty's feature-based view of proto-roles. This theory determines agenthood vs. patienthood based on a participant's instantiation of more or less typical agent vs. patient properties, such as, for example, volition in an event. To perform SPRL, we develop an ensemble of hierarchical models with self-attention and concurrently learned predicate-argument-markers. Our method is competitive with the state-of-the art, overall outperforming previous work in two formulations of the task (multi-label and multi-variate Likert scale prediction). In contrast to previous work, our results do not depend on gold argument heads derived from supplementary gold tree banks.Comment: accepted at *SEM 201

Using Differential Item Functioning to Analyze the Domain Generality of a Common Scientific Reasoning Test

A significant problem that assessments of scientific reasoning face at the level of higher education is the question of domain generality, that is, whether a test will produce biased results for students from different domains. This study applied three recently developed methods of analyzing differential item functioning (DIF) to evaluate the domain generality assumption of a common scientific reasoning test. Additionally, we evaluated the usefulness of these new, tree- and lasso-based, methods to analyze DIF and compared them with methods based on classical test theory. We gave the scientific reasoning test to 507 university students majoring in physics, biology, or medicine. All three DIF analysis methods indicated a domain bias present in about one-third of the items, mostly benefiting biology students. We did not find this bias by using methods based on classical test theory. Those methods indicated instead that all items were easier for physics students compared to biology students. Thus, the tree- and lasso-based methods provide a clear added value to test evaluation. Taken together, our analyses indicate that the scientific reasoning test is neither entirely domain-general, nor entirely domain-specific. We advise against using it in high-stakes situations involving domain comparisons

AMR4NLI: Interpretable and robust NLI measures from semantic graphs

The task of natural language inference (NLI) asks whether a given premise (expressed in NL) entails a given NL hypothesis. NLI benchmarks contain human ratings of entailment, but the meaning relationships driving these ratings are not formalized. Can the underlying sentence pair relationships be made more explicit in an interpretable yet robust fashion? We compare semantic structures to represent premise and hypothesis, including sets of contextualized embeddings and semantic graphs (Abstract Meaning Representations), and measure whether the hypothesis is a semantic substructure of the premise, utilizing interpretable metrics. Our evaluation on three English benchmarks finds value in both contextualized embeddings and semantic graphs; moreover, they provide complementary signals, and can be leveraged together in a hybrid model.Comment: International Conference on Computational Semantics (IWCS 2023); v2 fixes an imprecise sentence below Eq.

Transcriptomic and anatomical complexity of primary, seminal, and crown roots highlight root type-specific functional diversity in maize (Zea mays L.)

Maize develops a complex root system composed of embryonic and post-embryonic roots. Spatio-temporal differences in the formation of these root types imply specific functions during maize development. A comparative transcriptomic study of embryonic primary and seminal, and post-embryonic crown roots of the maize inbred line B73 by RNA sequencing along with anatomical studies were conducted early in development. Seminal roots displayed unique anatomical features, whereas the organization of primary and crown roots was similar. For instance, seminal roots displayed fewer cortical cell files and their stele contained more meta-xylem vessels. Global expression profiling revealed diverse patterns of gene activity across all root types and highlighted the unique transcriptome of seminal roots. While functions in cell remodeling and cell wall formation were prominent in primary and crown roots, stress-related genes and transcriptional regulators were over-represented in seminal roots, suggesting functional specialization of the different root types. Dynamic expression of lignin biosynthesis genes and histochemical staining suggested diversification of cell wall lignification among the three root types. Our findings highlight a cost-efficient anatomical structure and a unique expression profile of seminal roots of the maize inbred line B73 different from primary and crown roots

Complexity and specificity of the maize (Zea mays L.) root hair transcriptome

Root hairs are tubular extensions of epidermis cells. Transcriptome profiling demonstrated that the single cell-type root hair transcriptome was less complex than the transcriptome of multiple cell-type primary roots without root hairs. In total, 831 genes were exclusively and 5585 genes were preferentially expressed in root hairs [false discovery rate (FDR) ≤1%]. Among those, the most significantly enriched Gene Ontology (GO) functional terms were related to energy metabolism, highlighting the high energy demand for the development and function of root hairs. Subsequently, the maize homologs for 138 Arabidopsis genes known to be involved in root hair development were identified and their phylogenetic relationship and expression in root hairs were determined. This study indicated that the genetic regulation of root hair development in Arabidopsis and maize is controlled by common genes, but also shows differences which need to be dissected in future genetic experiments. Finally, a maize root view of the eFP browser was implemented including the root hair transcriptome of the present study and several previously published maize root transcriptome data sets. The eFP browser provides color-coded expression levels for these root types and tissues for any gene of interest, thus providing a novel resource to study gene expression and function in maize roots