Analysing Errors of Open Information Extraction Systems

We report results on benchmarking Open Information Extraction (OIE) systems using RelVis, a toolkit for benchmarking Open Information Extraction systems. Our comprehensive benchmark contains three data sets from the news domain and one data set from Wikipedia with overall 4522 labeled sentences and 11243 binary or n-ary OIE relations. In our analysis on these data sets we compared the performance of four popular OIE systems, ClausIE, OpenIE 4.2, Stanford OpenIE and PredPatt. In addition, we evaluated the impact of five common error classes on a subset of 749 n-ary tuples. From our deep analysis we unreveal important research directions for a next generation of OIE systems.Comment: Accepted at Building Linguistically Generalizable NLP Systems at EMNLP 201

Acetylcodeine as a Urinary Marker to Differentiate the Use of Street Heroin and Pharmaceutical Heroin

Acelylcodeine (ACOD) is a synthesis by product present in street heroin but not in pharmaceutical diacetylmorphine (DAM) as used in the Swiss program Heroin-Assisted Treatment for Opiate Dependent Drug Users (HAT). ACOD was evaluated and validated as an urine marker to detect the consumption of street heroin by HAT participants. A gas chromatography-mass spectrometry method allowing the quantitation of ACOD concentrations as low as 0.2 ng/mL urine has been developed. In opiate-naïve subjects, intravenous (i.v.) ACOD showed a plasma elimination half-life of 237 ± 18 min, urine peak concentrations 2 h after administration, and a detection window of 8 h. Only 0.4 ± 0.1% was excreted unchanged, with codeine (COD) as the main metabolite. ACOD may be formed by transacetylation when i.v. DAM and oral codeine are co-administered. To avoid false-positive results, the calculation of COD/ACOD ratios is recommended. In a study with 105 HAT participants, 14% of the tested urines were ACOD positive. Only a low correlation was found between the anonymously self-declared consumption of street heroin and the ACOD positive rat

Power Analysis for the Wald, LR, Score, and Gradient Tests in a Marginal Maximum Likelihood Framework: Applications in IRT

The Wald, likelihood ratio, score, and the recently proposed gradient statistics can be used to assess a broad range of hypotheses in item response theory models, for instance, to check the overall model fit or to detect differential item functioning. We introduce new methods for power analysis and sample size planning that can be applied when marginal maximum likelihood estimation is used. This allows the application to a variety of IRT models, which are commonly used in practice, e.g., in large-scale educational assessments. An analytical method utilizes the asymptotic distributions of the statistics under alternative hypotheses. We also provide a sampling-based approach for applications where the analytical approach is computationally infeasible. This can be the case with 20 or more items, since the computational load increases exponentially with the number of items. We performed extensive simulation studies in three practically relevant settings, i.e., testing a Rasch model against a 2PL model, testing for differential item functioning, and testing a partial credit model against a generalized partial credit model. The observed distributions of the test statistics and the power of the tests agreed well with the predictions by the proposed methods in sufficiently large samples. We provide an openly accessible R package that implements the methods for user-supplied hypotheses

Social validation in group decision making: differential effects on the decisional impact of preference-consistent and preference-inconsistent information

"Shared information has a stronger impact on group decisions than unshared information. A prominent explanation for this phenomenon is that shared information can be socially validated during group discussion and, hence, is perceived as more accurate and relevant than unshared information. In the present study we argue that this explanation only holds for preference-inconsistent information (i.e., information contradicting the group members’ initial preferences) but not for preference-consistent information. In Experiments 1 and 2 participants studied the protocol of a fictitious group discussion. In this protocol, we manipulated which types of information were socially validated. As predicted, social validation increased the decisional impact of preference-inconsistent but not preference-consistent information. In both experiments the effect of social validation was mediated by the perceived quality of information. Experiment 3 replicated the results of the first two experiments in an interactive setting in which two confederates discussed a decision case face-to-face with one participant." [author's abstract

The Formation and Coarsening of the Concertina Pattern

The concertina is a magnetization pattern in elongated thin-film elements of a soft material. It is a ubiquitous domain pattern that occurs in the process of magnetization reversal in direction of the long axis of the small element. Van den Berg argued that this pattern grows out of the flux closure domains as the external field is reduced. Based on experimental observations and theory, we argue that in sufficiently elongated thin-film elements, the concertina pattern rather bifurcates from an oscillatory buckling mode. Using a reduced model derived by asymptotic analysis and investigated by numerical simulation, we quantitatively predict the average period of the concertina pattern and qualitatively predict its hysteresis. In particular, we argue that the experimentally observed coarsening of the concertina pattern is due to secondary bifurcations related to an Eckhaus instability. We also link the concertina pattern to the magnetization ripple and discuss the effect of a weak (crystalline or induced) anisotropy