Hinrichs, Erhard

Kübler, Sandra

English

Hochschulschriftenserver - Universität Frankfurt am Main

EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ]ESSLLI 2002 WorkshopMachine Learning Approaches inComputational LinguisticsIntroductionErhard W. Hinrichs, Sandra K¨ ublereh,kuebler @sfs.uni-tuebingen.de.Seminar f¨ ur SprachwissenschaftUniversity of T¨ ubingenGermanyESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 1EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Motivation for ML in CLmanually developed NLP systems and languageresources for NLPrequire considerable human effortare often based on limited inspection of thedata with an emphasis on prototypicalexamplesoften fail to reach sufﬁcient domain coverageoften lack sufﬁcient robustness when inputdata are noisyESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 2EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Motivation for ML in CL (2)NLP systems and language resources for NLPbased on machine learning techniquesrequire less human effortare data-driven and require large-scale datasourcesachieve coverage directly proportional to therichness of the data sourceare more adaptive to noisy dataESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 3EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Machine Learningdo not give the computer explicit ruleslet it extract knowledge from datalearning = classiﬁcation taskESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 4EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Machine Learningdo not give the computer explicit ruleslet it extract knowledge from datalearning = classiﬁcation taskfrom labeled data supervised learningfrom unlabeled data unsupervised learningESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 4EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Machine Learningdo not give the computer explicit ruleslet it extract knowledge from datalearning = classiﬁcation taskfrom labeled data supervised learningfrom unlabeled data unsupervised learningabstract over data eager learningdo not abstract over data lazy learningESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 4EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] ML Methodssupervised learning methods:decision tree learningmemory-based learningtransformation-based error-driven learningneural networksinductive logic programmingmaximum entropy learningESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 5EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] ML Methodssupervised learning methods:decision tree learningmemory-based learningtransformation-based error-driven learningneural networksinductive logic programmingmaximum entropy learningunsupervised learning methods:conceptual clusteringmimimum description lengthneural networksESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 5EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Example: Part-of-Speech Taggingtask: ﬁnd the appropriate POS tag for a word incontextThey man the boat. versus The man inthe boat.for English, accuracy > 96 %for morphologically rich languages: many POStagsESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 6EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Example: Part-of-Speech Taggingtask: ﬁnd the appropriate POS tag for a word incontextThey man the boat. versus The man inthe boat.for English, accuracy > 96 %for morphologically rich languages: many POStagsSample instance:feature word - 2 word - 1 word POS tagvalue NULL/NULL They/PRP man VBESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 6EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Probleminstance: a vector of feature valueswhere the values are takenfrom the discrete or real-valued domain of theth featureESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 7EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Probleminstance: a vector of feature valueswhere the values are takenfrom the discrete or real-valued domain of theth featurelet X be the space of possible instanceslet Y be the set of classesESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 7EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Probleminstance: a vector of feature valueswhere the values are takenfrom the discrete or real-valued domain of theth featurelet X be the space of possible instanceslet Y be the set of classesthe goal of the ML system is to learn a targetfunctionESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 7EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Problem (2)training example: instance labeled withthe correct class c(x)ESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 8EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Problem (2)training example: instance labeled withthe correct class c(x)let D be the set of all training exampleshypothesis space, H: set of functionsof possible deﬁnitionsESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 8EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] The Learning Problem (2)training example: instance labeled withthe correct class c(x)let D be the set of all training exampleshypothesis space, H: set of functionsof possible deﬁnitionsthe goal is to ﬁnd an such that for all,ESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 8EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] CL Problems Approached with MLgrapheme-phoneme conversion (Stanﬁll & Waltz1986, van den Bosch & Daelemans 1993)POS tagging (Brants 1998, Cardie 1996,Daelemans et al. 1996)PP attachment (Hindle & Rooth 1993, Brill &Resnik 1994, Volk 2001)word sense disambiguation (Escudero et al.2000, Mooney 1996, Veenstra et al. 2000)noun phrase chunking (Ramshaw & Marcus1995, CoNLL 2000)ESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 9EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Evaluationgold standard: data against which the MLprogram is evaluatedtraining set: data on which the ML program istrainedtest set: data on which the performance of theML program is measuredtenfold cross validation: split data into 10parts; 10 rounds: use 1 part as test set andremaining parts as training setESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 1 0EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Evaluation Metricsaccuracy: percentage of correctly classiﬁedinstances from test setrecall: percentage of the items in the goldstandard that were found by the ML programprecision: percentage of the items selected bythe ML program that are correctESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 1 1EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Workshop ProgramMo. E. Hinrichs, S. Kübler (Tübingen): IntroductionW. Daelemans (Antwerpen): Machine Learning of Language:A Model and a ProblemTu. M. Rössler (Duisburg): Using Markov models for namedentity recognition in German newspapersP. Osenova, K. I. Simov (Soﬁa): Learning a token classiﬁcationfrom a large corpusWe. O. Streiter (Bolzano): Abduction, induction and memorizing incorpus-based parsingJ. Veenstra, F. H. Müller, T. Ule (Tübingen): Topological ﬁeldchunking for GermanESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 1 2EBERHARDKARLSUNIVERSITÄTTÜBINGENSeminarf¨urSprachwissenschaft[CL SfS ] Workshop Program (2)Th. A. Wagner (Tübingen): Learning thematic role relations forwordnetsC. Sporleder (Edinburgh): Learning lexical inheritancehierarchies with maximum entropy modelsFr. P. Lendvai, A. van den Bosch, E. Krahmer (Tilburg), M. Swerts(Eindhoven/Antwerpen): Improving machine-learned detectionof miscommunications in human-machine dialogues throughinformed data splittingK. Simov (Soﬁa): Grammar extraction and reﬁnement from anHPSG corpusFinal DiscussionESSLLI 2002 Workshop Machine Learning Approaches in Computational Linguistics –p . 1 3

Machine learning approaches in computational linguistics : introduction

Abstract

Similar works

Full text

Available Versions

Hochschulschriftenserver - Universität Frankfurt am Main