New Resources and Perspectives for Biomedical Event Extraction

Event extraction is a major focus of recent work in biomedical information extraction. Despite substantial advances, many challenges still remain for reliable automatic extraction of events from text. We introduce a new biomedical event extraction resource consisting of analyses automatically created by systems participating in the recent BioNLP Shared Task (ST) 2011. In providing for the first time the outputs of a broad set of state-ofthe-art event extraction systems, this resource opens many new opportunities for studying aspects of event extraction, from the identification of common errors to the study of effective approaches to combining the strengths of systems. We demonstrate these opportunities through a multi-system analysis on three BioNLP ST 2011 main tasks, focusing on events that none of the systems can successfully extract. We further argue for new perspectives to the performance evaluation of domain event extraction systems, considering a document-level, “off-the-page ” representation and evaluation to complement the mentionlevel evaluations pursued in most recent work.

Modern Logic and Judicial Decision Making: A Sketch of One View

Two hundred years elapsed before the nineteenth century logicians Boole, De Morgan, and others, finally succeeded in formally developing the calculus of reason-ing first suggested by the German mathematician, Leibniz. It is, perhaps, to the credit of the legal profession that less than one century has subsequently elapsed, and already some lawyers and legal writers, along with other scholars, are beginning to explore the relationship between modern logic and law. What is attempted here is to outline the bare bones of one tentative way of looking at the relationship between modern logic and the judicial decision process. From the useful vantage point of a Lasswellian social process framework of analysis, logic and judicial decision making are considered contextually within that total mani-fold of events that we call the world. Thus viewed, the judicial decision making process is just one constituent of the complex unfolding of events through time. We attempt to represent some of the complexities involved in each of these processes and the relationships between them by means of a series of diagrams. By suggesting that we begin with the world as our context, we make no claim to describing it in complete detail. To the contrary, the sketch presented here-we would emphasize the word sketch and the word tentative -is rough, incomplete, and subject to considerable improvement. But one of our purposes will be served if the outline points the way toward cumulative efforts to achieve a comprehensive description of the judicial decision process. In addition to this broad look at logic, judicial decision making, and the world, a more modest aim is to describe, in some detail and with reasonable clarity, one aspect of the relation between logic and judicial decision making

ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters

To bridge the gap between the capabilities of the state-of-the-art in factoid question answering (QA) and what users ask, we need large datasets of real user questions that capture the various question phenomena users are interested in, and the diverse ways in which these questions are formulated. We introduce ComQA, a large dataset of real user questions that exhibit different challenging aspects such as compositionality, temporal reasoning, and comparisons. ComQA questions come from the WikiAnswers community QA platform, which typically contains questions that are not satisfactorily answerable by existing search engine technology. Through a large crowdsourcing effort, we clean the question dataset, group questions into paraphrase clusters, and annotate clusters with their answers. ComQA contains 11,214 questions grouped into 4,834 paraphrase clusters. We detail the process of constructing ComQA, including the measures taken to ensure its high quality while making effective use of crowdsourcing. We also present an extensive analysis of the dataset and the results achieved by state-of-the-art systems on ComQA, demonstrating that our dataset can be a driver of future research on QA.Comment: 11 pages, NAACL 201