We introduce an incremental model for coreference resolution that competed in the CoNLL 2011 shared task (open regular). We decided to participate with our baseline model, since it worked well with two other datasets. The beneﬁts of an incremental over a mention-pair architecture are: a drastic reduction of the number of candidate pairs, a means to overcome the problem of underspeciﬁed items in pairwise classiﬁcation and the natural integration of global constraints such as transitivity. We do not apply machine learning, instead the system uses an empirically derived salience measure based on the dependency labels of the true mentions. Our experiments seem to indicate that such a system already is on par with machine learning approaches

Klenner, M

Tuggener, D

ZORA

Proceedings of the 15th Conference on Computational Natural Language Learning: Shared Task, pages 81–85,Portland, Oregon, 23-24 June 2011. c©2011 Association for Computational LinguisticsAn Incremental Model for Coreference Resolution with RestrictiveAntecedent AccessibilityManfred KlennerInstitute of Computational LinguisticsUniversity of Zurichklenner@cl.uzh.chDon TuggenerInstitute of Computational LinguisticsUniversity of Zurichtuggener@cl.uzh.chAbstractWe introduce an incremental model for coref-erence resolution that competed in the CoNLL2011 shared task (open regular). We decidedto participate with our baseline model, since itworked well with two other datasets. The ben-efits of an incremental over a mention-pair ar-chitecture are: a drastic reduction of the num-ber of candidate pairs, a means to overcomethe problem of underspecified items in pair-wise classification and the natural integrationof global constraints such as transitivity. Wedo not apply machine learning, instead thesystem uses an empirically derived saliencemeasure based on the dependency labels of thetrue mentions. Our experiments seem to indi-cate that such a system already is on par withmachine learning approaches.1 IntroductionWith notable exceptions (Luo et al., 2004; Yang etal., 2004; Daume III and Marcu, 2005; Culotta etal., 2007; Klenner, 2007; Rahman and Ng, 2009;Klenner and Ailloud, 2009; Cai and Strube, 2010;Raghunathan et al., 2010) supervised approaches tocoreference resolution are often realized by pairwiseclassification of anaphor-antecedent candidates. Apopular and often reimplemented approach is pre-sented in (Soon et al., 2001). As recently discussedin (Ng, 2010), the so called mention-pair model suf-fers from several design flaws which originate fromthe locally confined perspective of the model:• Generation of (transitively) redundant pairs, asthe formation of coreference sets (coreferenceclustering) is done after pairwise classification• Thereby generation of skewed training setswhich lead to classifiers biased towards nega-tive classification• No means to enforce global constraints such astransitivity• Underspecification of antecedent candidatesThese problems can be remedied by an incremen-tal entity-mention model, where candidate pairs areevaluated on the basis of the emerging coreferencesets. A clustering phase on top of the pairwise clas-sifier no longer is needed and the number of candi-date pairs is reduced, since from each coreferenceset (be it large or small) only one mention (the mostrepresentative one) needs to be compared to a newanaphor candidate. We form a ’virtual prototype’that collects information from all the members ofeach coreference set in order to maximize ’repre-sentativeness’. Constraints such as transitivity andmorphological agreement can be assured by just asingle comparison. If an anaphor candidate is com-patible with the virtual prototype, then it is by defini-tion compatible with all members of the coreferenceset.We designed our system to work purely with asimple, yet empirically derived salience measure. Itturned out that it outperformed (for German and En-glish, using CEAF, B-cubed and Blanc) the systemsfrom the 2010’s SemEval shared task1 on ’corefer-ence resolution in multiple languages’. Only withthe more and more questioned (Luo, 2005; Cai and1We have carried out a post task evaluation with the dataprovided on the SemEval web page.81Strube, 2010) MUC measure our system performedworse (at least for English). Our system uses realpreprocessing (i.e. a dependency parser (Schneider,2008)) and extracts markables (nouns, named enti-ties and pronouns) from the chunks and based onPOS tags delivered by the preprocessing pipeline.Since we are using a parser, we automatically takepart in the open regular session. Please note that thedependency labels are the only additional informa-tion being used by our system.2 Our Incremental ModelFig. 1 shows the basic algorithm. Let I be thechronologically ordered list of markables, C be theset of coreference sets (i.e. the coreference partition)and B a buffer, where markables are stored, if theyare not found to be anaphoric (but might be validantecedents, still). Furthermore mi is the currentmarkable and ⊕ means concatenation of a list anda single item. The algorithm proceeds as follows: aset of antecedent candidates is determined for eachmarkable mi (steps 1 to 7) from the coreference setsand the buffer. A valid candidate rj or bk must becompatible with mi. The definition of compatibilitydepends on the POS tags of the anaphor-antecedentpair (in order to be coreferent, e.g. two pronounsmust agree in person, number and gender etc.).In order to reduce underspecification, mi is com-pared to a virtual prototype of each coreference set.The virtual prototype bears information accumu-lated from all elements of the coreference set. Forinstance, assume a candidate pair ’she .. Clinton’.Since the gender of ’Clinton’ is unspecified, the pairmight or might not be a good candidate. But if thereis a coreference set already including ’Clinton’, let’ssay: {’Hilary Clinton’, her, she} then we know thegender from the other members and are more savein our decision. The virtual prototype here would besomething like: singular, feminine, human.From the set of candidates, Cand, the most salientantei ∈ Cand is selected (step 10) and the coref-erence partition is augmented (step 11). If anteicomes from a coreference set, mi is added to thatset. Otherwise (antei is from the buffer), a new set isformed, {antei,mi}, and added to the set of coref-erence sets.2.1 Restricted Accessibility of AntecedentCandidatesAs already discussed, access to coreference setsis restricted to the virtual prototype - the concretemembers are invisible. This reduces the number ofconsidered pairs (from the cardinality of a set to 1).Moreover, we also restrict the access to buffer el-ements: if an antecedent candidate, rj , from a coref-erence set exists, then elements from the buffer, bk,are only licensed if they are more recent than rj . Ifboth appear in the same sentence, the buffer elementmust be more salient in order to get licensed.2.2 Filtering based on Anaphora TypeThere is a number of conditions not shown in thebasic algorithm from Fig. 1 that define compatibil-ity of antecedent and anaphor candidates based onPOS tags. Reflexive pronouns must be bound in thesubclause they occur, more specifically to the sub-ject governed by the same verb. Personal and pos-sessive pronouns are licensed to bind to morphologi-cally compatible antecedent candidates (named enti-ties, nouns2 and pronouns) within a window of threesentences.We use the information given by CoNLL inputdata to identify ’speaker’ and the person adressed by’you’. ’I’ refers to one of the coreference sets whosespeaker is the person who, according to the CoNLLdata, is the producer of the sentence. ’You’ refersto the producer of the last sentence not being pro-duced by the current ’speaker’. If one didn’t haveaccess to these data, it would be impossible to cor-rectly identify the reference of ’I’, since turn takingis not indicated in the pure textual data.As we do not use machine learning, we onlyapply string matching techniques to match nom-inal NPs and leave out bridging anaphora (i.e.anaphoric nouns that are connected to their an-tecedents through a semantic relation such as hy-ponymy and cannot be identified by string matchingtherefore). Named entities must either match com-pletely or the antecedent must be longer than onetoken and all tokens of the anaphor must be con-tained in the antecedent (to capture relations such2To identify animacy and gender of NEs we use a list ofknown first names annotated with gender information. To ob-tain animacy information for common nouns we conduct aWordNet lookup.821 for i=1 to length(I)2 for j=1 to length(C)3 rj := virtual prototype of coreference set Cj4 Cand := Cand ⊕ rj if compatible(rj ,mi)5 for k= length(B) to 16 bk:= the k-th licensed buffer element7 Cand := Cand ⊕ bk if compatible(bk,mi)8 if Cand = {} then B := B ⊕mi9 if Cand 6= {} then10 antei := most salient element of Cand11 C := augment(C,antei,mi)Figure 1: Incremental Model: Base Algorithmas ’Hillary Clinton ... Clinton’). Demonstrative NPsare mapped to nominal NPs by matching their heads.Definite NPs match with noun chunks that are longerthan one token3 and must be contained completelywithout the determiner (e.g. ’Recent events ... theevents’). From the candidates that pass these filtersthe most salient one is selected as antecedent. If twoor more candidates with equal salience are available,the closest one is chosen.2.3 Binding Theory as a FilterThere is another principle that help reduce the num-ber of candidates even further: binding theory. Weknow that ’He’ and ’him’ cannot be coreferent inthe sentence ’He gave him the book’. Thus, the pair’He’-’him’ need not be considered at all. Actually,there are subtle restrictions to be captured here. Wehave not implemented a full-blown binding theoryon top of our dependency parser, yet. Instead, weapproximated binding restrictions by subclause de-tection. ’He’ and ’him’ in the example above are inthe same subclause (the main clause) and are, thus,exclusive. This is true for nouns and personal pro-nouns, only. Possesive and reflexive pronouns areallowed to be bound in the same subclause.2.4 An Empirically-based Salience MeasureSince we look for a simple and fast salience measureand do not apply machine learning in our baselinesystem, our measure is solely based on the gram-matical functions (given by the dependency labels)of the true mentions. Grammatical functions have3If we do not apply this restriction too many false positivesare produced.played a major role in calculating salience, espe-cially in rule based system such as (Hobbs, 1976;Lappin and Leass, 1994; Mitkov et al., 2002; Sid-dharthan, 2003). Instead of manually specifyingthe weights for the dependency labels like (Lappinand Leass, 1994), we derived them empirically fromthe coreference CoNLL 2011 gold standard (train-ing data). The salience of a dependency label, D,is estimated by the number of true mentions in thegold standard that bear D (i.e. are connected to theirheads with D), divided by the total number of truementions. The salience of the label subject is thuscalculated by:Number of truementions bearing subjectTotal number of truementionsFor a given dependency label, this fraction indicateshow strong is the label a clue for bearing an an-tecedent. This way, we get a hierarchical order-ing of the dependency labels (subject > object >pobject > ...) according to which antecedents areranked. Clearly, future work will have to establisha more elaborate calculation of salience. To oursurprise, however, this salience measure performedquite well, at least together with our incremental ar-chiteture.3 EvaluationThe results of our evaluation over the CoNLL 2011shared task development set are given in Fig. 2 (de-velopment set) and 3 (official results on the test set).The official overall score of our system in theopen regular setting is 51.77.Our results are mediocre. There are several rea-83Metric R P F1CEAFM 49.73 49.73 49.73CEAFE 44.26 37.70 40.72BCUB 59.17 71.66 66.06BLANC 62.70 72.74 64.82MUC 42.20 49.21 45.44Figure 2: CoNLL 2011 Development Set ResultsMetric R P F1CEAFM 50.03 50.03 50.03CEAFE 41.28 39.70 40.48BCUB 61.70 68.61 64.97BLANC 66.05 73.90 69.05MUC 49.04 50.71 49.86Figure 3: CoNLL 2011 Test Set Resultssons for that. First and foremost, the scorer requireschunk extensions to match perfectly. That is, evenif the head of an antecedent is found, this does notcount if the chunk extension of that noun phrase wasnot correctly identified. Since chunks do not play amajor role in depencendy parsing, our approxima-tion might be faulty4. Another shortcomming arenominal anaphora that can not be identified by stringmatching (e.g. Obama ... The president). Our sim-ple salience-based approach does not cope at all withthis type of anaphora.4 Related Work(Ng, 2010) discusses the entity-mention modelwhich operates on emerging coreference sets to cre-ate features describing the relation of an anaphorcandidate and established coreference sets. (Luoet al., 2004) implemented such a model but it per-formed worse than the mention-pair model. (Yanget al., 2004) presented an incremental model whichused some coreference set specific features, namelyintroducing the number of mentions in a set as afeature besides checking for morphological compat-ibility with all mentions in a set. They also reportthat the set size feature only marginally improves orin some combinations even worsens system perfor-mance. (Daume III and Marcu, 2005) introduceda wide range of set specific features, capturing set4Especially Asiatic names pose problems to our parser, quiteoften the extensions could not get correctly fixed.count, size and distribution amongst others, in a jointmodel for the ACE data.All the above mentioned systems use an incre-mental model to generate features describing theemerging coreference sets and the anaphor candi-date. In contrast, we use an incremental architectureto control pair generation in order to prevent gener-ation of either redundant or irrelevant pairs.5 ConclusionsWe have introduced an incremental model for coref-erence resolution based on an empirically derivedsalience measure that is meant as a simple andvery fast baseline system. We do not use machinelearning, nor do we resolve more complex nominalanaphora such as ’Obama ... The president’ (but wehandle those that can be resolved by simple patternmatching, e.g. Hilary Clinton .. Clinton). Giventhese restrictions, our system performed well.The central idea of our approach is that the evolv-ing coreference sets should restrict the access to an-tecedent candidates in a twofold way: by use of vir-tual prototypes that accumulate the properties of allmembers of a coreference set (e.g. wrt. animacy),but also by restricting reachable buffer elements (i.e.yet unattached markables).The benefits of our incremental model are:• due to the restricted access to antecedent candi-dates, the number of generated candidate pairscan be reduced drastically5• no coreference clustering phase is needed• the problem of underspecification that exists forany pair-wise model can be compensated by avirtual prototype that accumulates the proper-ties of the elements of a coreference setThese benefits are independent of the underly-ing classification scheme, be it a simple salience-based one or a more advanced machine learning one.The work presented here thus would like to opt forfurther research based on incremental architectures.Web demos for English and German are available6.5We observed a reduction over 75% in some experimentswhen moving from a mention-pair to an incremental entity-mention model.6http://kitt.cl.uzh.ch/kitt/coref/84ReferencesJie Cai and Michael Strube. 2010. Evaluation metricsfor end-to-end coreference resolution systems. In Pro-ceedings of the SIGdial 2010 Conference: The 11thAnnual Meeting of the Special Interest Group on Dis-course and Dialogue.Aron Culotta, Michael Wick, and Andrew McCallum.2007. First-order probabilistic models for coreferenceresolution. In Human Language Technologies 2007:The Conference of the North American Chapter of theAssociation for Computational Linguistics; Proceed-ings of the Main Conference, pages 81–88, Rochester,New York, April. Association for Computational Lin-guistics.Hal Daume III and Daniel Marcu. 2005. A large-scaleexploration of effective global features for a joint en-tity detection and tracking model. In HLT ’05: Pro-ceedings of the conference on Human Language Tech-nology and Empirical Methods in Natural LanguageProcessing, pages 97–104, Morristown, NJ, USA. As-sociation for Computational Linguistics.Jerry R. Hobbs. 1976. Pronoun resolution. TechnicalReport 76-1, Research Report, Department of Com-puter Sciences, City College, City University of NewYork.Manfred Klenner and Etienne Ailloud. 2009. Opti-mization in Coreference Resolution Is Not Needed: ANearly-Optimal Zero-One ILP Algorithm with Inten-sional Constraints. In Proc. of the EACL.Manfred Klenner. 2007. Enforcing consistency on coref-erence sets. In In Recent Advances in Natural Lan-guage Processing (RANLP), pages 323–328.Shalom Lappin and Herbert J Leass. 1994. An algorithmfor pronominal anaphora resolution. ComputationalLinguistics, 20:535–561.Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, NandaKambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm basedon the bell tree. In Proceedings of the 42nd AnnualMeeting on Association for Computational Linguis-tics.Xiaoqiang Luo. 2005. On coreference resolution perfor-mance metrics. In HLT ’05: Proceedings of the con-ference on Human Language Technology and Empir-ical Methods in Natural Language Processing, pages25–32, Morristown, NJ, USA. Association for Com-putational Linguistics.Ruslan Mitkov, Richard Evans, and Constantin Orasan.2002. A new, fully automatic version of mitkov’sknowledge-poor pronoun resolution method. In CI-CLing ’02: Proceedings of the Third InternationalConference on Computational Linguistics and Intel-ligent Text Processing, pages 168–186, London, UK.Springer-Verlag.Vincent Ng. 2010. Supervised noun phrase coreferenceresearch: The first fifteen years. In Proceedings of the48th Annual Meeting of the Association for Computa-tional Linguistics.Karthik Raghunathan, Heeyoung Lee, Sudarshan Ran-garajan, Nathanael Chambers, Mihai Surdeanu, DanJurafsky, and Christopher Manning. 2010. A multi-pass sieve for coreference resolution. In Proceedingsof the 2010 Conference on Empirical Methods in Nat-ural Language Processing, EMNLP ’10, pages 492–501, Stroudsburg, PA, USA. Association for Compu-tational Linguistics.Altaf Rahman and Vincent Ng. 2009. Supervised mod-els for coreference resolution. In Proceedings of the2009 Conference on Empirical Methods in NaturalLanguage Processing: Volume 2 - Volume 2, EMNLP’09, pages 968–977, Stroudsburg, PA, USA. Associa-tion for Computational Linguistics.Gerold Schneider. 2008. Hybrid Long-Distance Func-tional Dependency Parsing. Doctoral Thesis, Instituteof Computational Linguistics, Univ. of Zurich.Advaith Siddharthan. 2003. Resolving pronouns ro-bustly: Plumbing the depths of shallowness. In Pro-ceedings of the Workshop on Computational Treat-ments of Anaphora, 11th Conference of the EuropeanChapter of the Association for Computational Linguis-tics (EACL 2003).Wee M. Soon, Hwee T. Ng, and Daniel. 2001. A ma-chine learning approach to coreference resolution ofnoun phrases. Computational Linguistics, 27(4):521–544, December.Xiaofeng Yang, Jian Su, Guodong Zhou, and Chew LimTan. 2004. An np-cluster based approach to corefer-ence resolution. In Proceedings of the 20th interna-tional conference on Computational Linguistics.85

An incremental model for coreference resolution with restrictive antecedent accessibility

http://www.zora.uzh.ch/51280/1/W11-1912.pdf

An incremental model for coreference resolution with restrictive antecedent accessibility

Abstract

Similar works

Full text

Available Versions

ZORA