Summary: METIS is a web-based integrated annotation tool. From single query sequences, the PRECIS component allows users to generate structured protein family reports from sets of related Swiss-Prot entries. These reports may then be augmented with pertinent sentences extracted from online biomedical literature via support vector machine and rule-based sentence classification systems. Availability: Contact: mitchell@ebi.ac.uk Supplementary informatio

Attwood, T. K.

Divoli, A.

Hilario, M.

Kim, J.-H

Mitchell, A. L.

Selimas, I.

RERO DOC Digital Library

Vol. 21 no. 22 2005, pages 4196–4197doi:10.1093/bioinformatics/bti675BIOINFORMATICSAPPLICATIONS NOTEData and Text MiningMETIS: multiple extraction techniques for informative sentencesA. L. Mitchell1,2,, A. Divoli1, J.-H. Kim3, M. Hilario3, I. Selimas1 and T. K. Attwood1,21Faculty of Life Sciences and School of Computer Science, University of Manchester, Oxford Road, ManchesterM13 9PT, UK, 2European Bioinformatics Institute,Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD,UK and 3Artificial Intelligence Laboratory, University of Geneva, CH-1211 Geneva 4, SwitzerlandReceived on June 29, 2005; revised on August 31, 2005; accepted on September 8, 2005Advance Access publication September 13, 2005ABSTRACTSummary: METIS is a web-based integrated annotation tool. Fromsingle query sequences, the PRECIS component allows users to gen-erate structured protein family reports from sets of related Swiss-Protentries.Thesereportsmay thenbeaugmentedwithpertinent sentencesextracted from online biomedical literature via support vector machineand rule-based sentence classification systems.Availability: http://umber.sbs.man.ac.uk/dbbrowser/metis/Contact: mitchell@ebi.ac.ukSupplementary information: http://umber.sbs.man.ac.uk/dbbrowser/metis/supp_inf_results.html1 INTRODUCTIONThere is a pressing need for computational tools to facilitate anno-tation of sequence data, a task that, for each sequence or set ofsequences, involves culling information from various sources,including the literature. A major challenge for such tools is totrace pertinent papers and to extract relevant information fromthem. Several automated approaches tackle the information extrac-tion problem [e.g. PASTA (Gaizauskas et al., 2003) for proteinstructure and MedMiner (Tanabe et al., 1999) for gene expressionprofiling], but these tools have a specific focus that is not directlyapplicable for database annotation.To this end we have developed METIS, building on an existingannotation tool PRECIS (Mitchell et al., 2003) that automaticallycreates protein reports from related entries in Swiss-Prot [the manu-ally annotated component of UniProt (Apweiler et al., 2004)].Although PRECIS gathered the linked literature from each entry,it never directly exploited that information. An innovation inMETIS is to use the data in the Swiss-Prot entries to find relevantliterature, or to find search terms with which to seek this out. Theliterature (in the form of abstracts) is then collected and passed totwo sentence classification components that extract informativesentences and present them to the user.2 IMPLEMENTATIONFigure 1 shows an overview of METIS. The software takes as inputa FastA format sequence or a Swiss-Prot identifier. The PRECIScomponent performs a BLAST (Altschul et al., 1997) search ofSwiss-Prot and digests the related entries to create a structuredreport that details protein structure, function and disease, keywords,and database and literature cross-references.Using PubMed identifiers from each Swiss-Prot entry, corre-sponding abstracts are retrieved and passed to the sentenceclassifiers. Refineable PubMed query terms are also produced byanalysing the Swiss-Prot entries. These allow users to performwiderliterature searches and to run the sentence classifiers on the output.The first sentence classification component is a set of supportvector machines (SVMs), built as part of the BioMinT text-mining project. It was developed on three specialized corpora forstructure, function and disease, totalling 2406 positive and 5681negative sentences extracted from 934 abstracts. 80% of each corpuswas used for training with 20% reserved for final blind testing. Thetraining process involved 10-fold cross-validation, i.e. at each itera-tion, 90% of the training instances were used to build a large numberof models and 10% were used to estimate their performance. Exten-sive sentence classification experiments were performed involvingdifferent feature representations, learning algorithms (e.g. neuralnetworks, decision trees, Naı¨ve Bayes classifiers, K-nearest-neighbours), and different SVM kernels and hyperparameter values.Linear SVMs with a C parameter value of 0.1 performed best (pre-cision/recall values of 62/70, 53/66 and 60/69% for structure, func-tion and disease, respectively, when evaluated using the final blindtesting sets) and hence were used in METIS.The second classification component, BioIE (Divoli and Attwood,2005), uses manually predefined templates and rules to identifysentences relating to the categories of interest. Users may extractSwiss-Prot identifier/FastA sequenceSwiss-ProtCoreReportSearch terms forabstract retrievalCited LiteraturePRECISPubMedSentences:StructureFunctionDiseaseSentenceClassifiers:BioIE / SVMsFig. 1. Flow chart showing the sequence of actions performed by METIS.To whom correspondence should be addressed.4196  The Author 2005. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.orgall the sentences from each category, or specify keywords to refinethe extraction. A link to GPSDB (Pillet et al., 2004) allows users toextend their search terms by seeking possible protein synonyms.The templates and user-specified keywords are marked up onthe selected sentences, which are in turn ranked according to thenumber and type/complexity of templates found in them.3 RESULTSTo further evaluate the performance of the sentence classificationcomponents, 20 sets of abstracts were generated by running UniProtsequences through METIS. Precision and recall values were thencalculated. An overview of the results is given in Table 1. A listof identifiers used and the full results obtained are available athttp://umber.sbs.man.ac.uk/dbbrowser/metis/supp_inf_ids.html andhttp://umber.sbs.man.ac.uk/dbbrowser/metis/supp_inf_results.html4 DISCUSSIONAnnotating protein sequences and families is onerous and time-consuming, typically involving BLAST searches to gather sequen-ces related to a query, examining hits to primary databases, findingpapers and scrutinizing them for relevant information. METIS hasbeen designed to do this automatically. Moreover, it is easy to usesince it requires only a single sequence or an ID as input.By using the biomedical literature cited in Swiss-Prot, METIScircumvents the problems associated with finding relevant publica-tions automatically — effectively, the Swiss-Prot curators havealready performed this task for us manually. This approach is simi-lar to that underpinning the MedBlast literature-mining tool (Tu etal., 2004); however, MedBlast does not extract any informationfrom the literature it finds. The ability to suggest wider searchterms and run the sentence classifiers on any gathered abstractsmeans that although METIS is Swiss-Prot-based, it is not cons-trained by the database — should the cited literature prove toonarrow in scope or out of date, further information can be gatheredand analysed very easily.Use of online abstracts rather than full texts was a pragmaticchoice, as accessing full texts online often involves licensing andsubscription issues. Similarly, performing sentence classificationrather than information extraction (IE) proper was a practicaldecision — sentence classification can yield useful results quicklyand itself provides an appropriate foundation for true IE, a far moredifficult task that is being tackled in BioMinT. Meanwhile, lists ofextracted sentences are helpful to annotators, as they are usuallyconcise, semantically complete entities that can be used directly toaugment core PRECIS reports.Our results show that the sentence classifiers embedded in thesystem perform differently, depending on the sentence types evalu-ated, e.g. under the test conditions, BioIE performs better at clas-sifying disease-related sentences than the SVM component(precision 56 versus 48%), while for structure-related sentencesthe opposite is true (precision 33 versus 51%). This finding validatesour use of multiple extraction techniques.The relatively low precision of the function sentence classi-fiers is disappointing and clearly requires improvement. A likelyproblem is that the terms used to convey functional information inthese sentences are polysemic (not specific to descriptions of func-tion alone) compared with those for structure and disease.Although further SVM training and revision of the function tem-plates may help to improve precision, some syntactic analysis offunction sentences will probably also be required to classify themcorrectly.Currently, the BioIE component of METIS is flexible—its pre-cision can be increased by manually supplying specific search terms(e.g. a protein name) so that only sentences containing those termsare considered for sentence classification. We are now exploringhow we might extend it to perform such specific extraction auto-matically, using the query terms already suggested by the system forwider literature searching.METIS is a significant next step towards the automation of data-base annotation: it reduces the time required to seek out and readrelevant literature; it is versatile, yet easy to use; and its output isEnglish-like (being made from Swiss-Prot information and sen-tences extracted directly from the literature), rendering it immedi-ately useful in the annotation process. As we continue to enhance itsperformance, the value of METIS will therefore grow as an anno-tator’s assistant.ACKNOWLEDGEMENTSThis work was supported by European Commission grant numberQLRI-CT-2002-02770 BioMinT, EPSRC grant number GR/R80810/01 and the Swiss SER.Conflict of Interest: none declared.REFERENCESAltschul,S.F. et al. (1997) Gapped BLAST and PSI-BLAST: a new generation ofprotein database search programs. Nucleic Acids Res., 25, 3389–3402.Apweiler,R. et al. (2004) UniProt: the Universal Protein Knowledgebase. Nucleic AcidsRes., 32, D115–D119.Divoli,A. and Attwood,T.K. (2005) BioIE: extracting informative sentences from thebiomedical literature. Bioinformatics, 21, 2138–2139.Gaizauskas,R. et al. (2003) Protein structures and information extraction from bio-logical texts: the PASTA system. Bioinformatics, 19, 135–143.Mitchell,A.L. et al. (2003) PRECIS—an automatic tool for generating protein reportsengineered from concise information in Swiss-Prot. Bioinformatics, 19,1664–1671.Pillet,V. et al. (2005) GPSDB: a new database for synonyms expansion of gene andprotein names. Bioinformatics, 21, 1743–1744.Tanabe,L. et al. (1999) MedMiner: an Internet text-mining tool for biomedical informa-tion, with application to gene expression profiling. Biotechniques, 27, 1210–12141216–1217.Tu,Q. et al. (2004) MedBlast: searching articles related to a biological sequence.Bioinformatics, 20, 75–77.Table 1. Sentence classification resultsTopic Precision (%) Recall (%) nSVMs BioIE SVMs BioIEStructure 51 ± 3 33 ± 3 74 ± 3 85 ± 3 20Function 31 ± 3 16 ± 2 61 ± 5 91 ± 2 18Disease 48 ± 7 56 ± 5 56 ± 7 79 ± 6 16METIS: extraction of informative sentences4197

METIS: multiple extraction techniques for informative sentences

Abstract

Similar works

Full text

Available Versions

RERO DOC Digital Library