In this paper we introduce our attempts to incorporate the participant role information in multiparty meetings for document modeling using the hierarchical Dirichlet process. The perplexity and automatic speech recognition results demonstrate that the participant role information is a promising prior knowledge source to be combined with language models for automatic speech recognition and interaction modeling for multiparty meetings

Huang, Songfang

Renals, Steve

English

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Using Participant Role in Multiparty Meetings as Prior Knowledgefor Nonparametric Topic ModelingCitation for published version:Huang, S & Renals, S 2008, Using Participant Role in Multiparty Meetings as Prior Knowledge forNonparametric Topic Modeling. in Proc. ICML/UAI/COLT Workshop on Prior Knowledge for Text andLanguage Processing. pp. 21-24.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Proc. ICML/UAI/COLT Workshop on Prior Knowledge for Text and Language ProcessingGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Using Participant Role in Multiparty Meetingsas Prior Knowledge for Nonparametric Topic ModelingSongfang Huang s.f.huang@ed.ac.ukSteve Renals s.renals@ed.ac.ukThe Centre for Speech Technology Research, University of Edinburgh, Edinburgh, EH8 9LW, United KingdomAbstractIn this paper we introduce our attempts to in-corporate the participant role information inmultiparty meetings for document modelingusing the hierarchical Dirichlet process. Theperplexity and automatic speech recognitionresults demonstrate that the participant roleinformation is a promising prior knowledgesource to be combined with language modelsfor automatic speech recognition and interac-tion modeling for multiparty meetings.1. IntroductionIn recent years there has been growing research in-terest in the automatic speech recognition (ASR) formultiparty meetings, which is of essential importancefor the subsequent meeting processing such as con-tent analysis, summarisation, discourse analysis, andinformation retrieval. In this paper, we consider animproved language model (LM) in a state-of-the-artlarge vocabulary ASR system for meetings, based onthe prior knowledge of participant roles. More specifi-cally, we estimate the word distribution over the role ofeach participant, i.e., P (w|r), and use this as unigrammarginals to adapt a conventional n-gram LM.The AMI and AMIDA (http://www.amiproject.org)projects are dedicated to the development of tech-nologies to enhance the recognition and interpreta-tion of interactions between people in multiparty meet-ings (Renals et al., 2007). The AMI Meeting Cor-pus collected by the AMI project consists of 100 hoursof multimodal meeting recordings with comprehensiveannotations at a number of different levels. About 70%of the corpus was elicited using a design scenario, inAppearing in the Workshop on Prior Knowledge forText and Language Processing at ICML/UAI/COLT 2008,Helsinki, Finland, 2008.which the participants play the roles of employees, i.e.,project manager (PM), marketing expert (ME), userinterface designer (UI), and industrial designer (ID),in an electronics company that decides to develop anew type of television remote control. Our intuitionis that, since different participants play different roles,there may be a different word distribution, and in turndifferent dominant words, specific to each role. For ex-ample, we expect a project manager is more likely tospeak words relating to the coordination of meetings,i.e., meeting, project, or present, while a user interfacedesigner may favor words on interaction mediums likescreen, voice, or speech.Topic models have received much attention in the ma-chine learning community, which follows the “bag-of-words” assumption, i.e., words in a document are ex-changeable. In this paper we attempt to incorporatethe participant role as prior knowledge in topic models,by assigning role information to exchangeable words ina document. This could be achieved within the flexi-ble framework of topic models, by introducing an ad-ditional observed variable for the role into the graphi-cal model. By assuming that each role has a mixturedistribution over the latent topics, we could infer thetopic distribution specific to each role. We could fur-ther estimate P (w|r) for each role r by integrating outthe latent topics. Moreover, incorporating the role intopic models enables not only the document modeling,but also the interaction modeling in meetings.An alternative approach to modeling the relationshipbetween the participant role information and lexicalwords is to directly estimate the conditional probabil-ity P (w|r) based on the co-occurrence statistics of rolesand words, using the maximum likelihood principle.As a comparison to the probabilistic topic models, wealso introduce in this paper a deterministic approachto modeling roles and words, by regarding the role asan additional feature (factor) of lexical words in anMLE-based LM.21Workshop on Prior Knowledge for Text and Language Processing2. Modeling ApproachesWe consider two modeling approaches to the estima-tion of P (w|r): one is a hierarchical Bayesian modelusing the hierarchical Dirichlet process (HDP) as theprior, and the other is a factored approach using thefactored language model (FLM).2.1. Hierarchical Bayesian ModelThe hierarchical Dirichlet process (Teh et al., 2006)is a nonparametric generalization of latent Dirichletallocation (LDA), which extends the standard LDAmodel to infinite and hierarchical topic modeling.Conversational speech consists of sequences of utter-ances, which do not comprise well-defined documents.We used the following procedure to obtain documents:for each scenario meeting, first align all the wordsin it along a common timeline; then for each sen-tence/segment, collect those non-stop words belongingto a window of length L as the document, by backtrac-ing from the end time of the sentence/segment. Therole that has been assigned to the most of words in thewindow is selected as the role for that document. Weuse a moving window with L = 20 seconds over thesequences of words to obtain documents.We incorporate the participant role by extending the2-level HDP (Teh et al., 2006) in Figure 1(A) to a thirdlevel, as shown in Figure 1(B), role-HDP. An DP Gris assigned for each of the four roles (PM,ME,UI,ID),which then served as the parent DP (the base proba-bility measure) in the HDP hierarchy for all those DPscorresponding to documents belonging to that role.(A)01(B)Figure 1. The graphical model depictions for (A) 2-levelHDP, and (B) role-HDP.2.2. Factored Language ModelOne straightforward method for modeling words androles is to use the maximum likelihood estimationbased on the co-occurrences of words w and therole information r, i.e., training a bigram-like modelP (w|r) = Count(w, r)/Count(r). More generally, wecan use a factored language model (Bilmes & Kirch-hoff, 2003) to model words and role deterministically.The FLM, initially developed to address the languagemodeling problems faced by morphologically rich orinflected languages, is a generalization of standard n-gram language models, in which each word wt is de-composed into a bundle of K word-related features(called factors), wt ≡ f1:Kt = {f1t , f1t , . . . , fKt }. Fac-tors may include the word itself. Each word in an FLMis dependent not only on a single stream of its pre-ceding words, but also on additional parallel streamsof factors. Combining with interpolation or general-ized parallel backoff (GPB) (Bilmes & Kirchhoff, 2003)strategies, multiple backoff paths may be used simul-taneously.We exploit two factors for word w at time t: the wordwt itself and the corresponding role rt, as shown inFigure 2. All the words in a sentence share a com-mon role, i.e., rt = rt−1 = . . . = r1 in Figure 2. Weuse a simple backoff strategy, for example, by movingfrom the model P (wt|rt) directly down to the unigrammodel P (wt). We refer this model to the role-FLM.gtmin=1kndiscountgtmin=1kndiscountFigure 2. The graphical model representation and backoffpath for role-FLM.2.3. Combination with n-gram LMsAs in (Kneser et al., 1997), we use the dynamic uni-gram marginals P (w|r), from either the role-HDP orthe role-FLM, for LM adaptation:Padapt(w|h) = Pback(w|h) ·(P (w|r)Pback(w))µ/z(h) (1)where h is the history of w, Pback(w|h) the baselinen-gram, Padapt(w|h) the adapted n-gram, and z(h)a normalisation factor. For the role-HDP, P (w|r) ≈∑Kk=1 φkw · θdk with φk estimated during training andremaining fixed in testing, while θd are document-dependent (and in turn are role-dependent because θd22Workshop on Prior Knowledge for Text and Language Processingare derived from Gr) and thus are calculated dynami-cally for each test document.3. Experiments and Results13.1. Empirical ExperimentWe first carried out some empirical analyses for theHDP and the role-HDP. The HDP was implementedas an extension to the SRILM toolkit2. We trained theHDP and the role-HDP models using different values(k = 1, . . . , 100) for the initial number of topics. Weused uniform distribution for H, i.e., Hw = 1/W . Allmodels were trained using the fold-2−4 of the AMIscenario meetings, with a fixed size vocabulary of 7,910words, by the Markov Chain Monte Carlo (MCMC)sampling method. The concentration parameters weresampled using the auxiliary variable sample scheme in(Teh et al., 2006). We ran 3,000 iterations to burn-in, then collected 10 samples from the posteriors tocalculate the unigram perplexity on the fold-1 testingdata, with the sample step of 5.Figure 3 shows the perplexity results, from which wecan see that the role-HDP produced better results thanthe HDP. Our understanding for the improvement isthat by using the role prior knowledge, documents withthe same role share the strengths in the HDP frame-work. In addition, we show in Figure 4 the top twotopics for each role. It is interesting to find out thateach role has some specific topics with high probabil-ities, while they also tend to interact with each otheron some common topics, i.e., the button topic appearswith high probability for all the four roles in Figure 4.0 20 40 60 80 1005005506006507007508008509009501000Initial number of topics in the HDP/role−HDPPerplexity  HDProle−HDPFigure 3. The perplexity results for the HDP/role-HDP.1Some of the results here were appearing in anotherpaper by the authors in (Huang & Renals, 2008).2http://www.speech.sri.com/projects/srilmPM ME UI ID(29) 0.28 (14) 0.13 (20) 0.27 (14) 0.16 (14) 0.25 (19) 0.15 (14) 0.17 (6) 0.16DESIGNMEETINGPROJECTMINUTESUSERINTERFACEPRESENTPRODUCTDESIGNERLOOKSTARTSUREBITGUESSTHANKBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDREMOTECONTROLLOOKMARKETCONTROLSMEANEASYPRODUCTDESIGNBUTTONSFANCYIMPORTANTPERCENTFINDLOTBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDRECOGNITIONSPEECHVOICEL_C_D_SCREENMICROPHONECONTROLLERREMOTEPROBLEMTECHNOLOGYCOFFEESPEAKERCOSTCONTROLSYSTEMBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDCHIPSIMPLEADVANCEDREGULARREMOTEL_C_D_BATTERYSPEAKEREXPENSIVEINFRAREDSTATIONSAMPLEBATTERIESCOSTCONTROLFigure 4. The example topics for the four roles using therole-HDP.3.2. ASR ExperimentTo further investigate the effectiveness of employingthe role as prior knowledge for topic modeling, we per-formed ASR experiments on multiparty meetings. Weused part of the AMI Meeting Corpus for our exper-iments. There are 138 scenario meetings in total, ofwhich 118 were used for training and the other 20 fortesting (about 11 hours). The procedure and param-eters used to train the HDP/role-HDP were the sameas those used in Section 3.1, except that we used a dif-ferent split-up of the AMI scenario meetings for ASR.We trained two baseline LMs: the first one used theFisher conversational telephone speech data (fisher-03-p1+p2), and the second used three datasets from theAMI training data, the Fisher, and the Hub-4 broad-cast news data (hub4-lm96). The two baseline LMswere trained with standard parameters using SRILM:trigrams, cut-off value of 2 for trigram counts, mod-ified Kneser-Ney smoothing, interpolated model. Acommon vocabulary with 56,168 words was used forthe two LMs, which has 568 out-of-vocabulary (OOV)words for the AMI test data.We investigated the effectiveness of the adapted LMsbased on topic and role information from meetings ona practical large vocabulary ASR system. The AMI-ASR system (Hain & et al., 2007) was used as the base-line system. We began from the lattices for the wholeAMI Meeting Corpus, generated by the AMIASR sys-tem using a trigram LM trained on a large set of datacoming from Fisher, Hub4, Switchboard, webdata, andvarious meeting sources including AMI. We then gen-erated 500-best lists from the lattices for each utter-ance. We adapted the two baseline LMs (Fisher andAMI+Fisher+Hub4) using Equation (1) according tothe unigram marginals from the role-FLM, the HDP,and the role-HDP respectively. For the HDP, we usedP (w|d) ≈ ∑Kk=1 φkw · θdk as the unigram marginals,i.e., the difference from P (w|r) by the role-HDP isthat in the HDP θd are only document-dependent23Workshop on Prior Knowledge for Text and Language ProcessingTable 1. The %WER results of ASR experiments usingadapted LMs on the AMI scenario meetings.LMs SUB DEL INS WERFisher 22.7 11.4 5.8 39.9role-FLM-adapted 22.5 11.1 5.9 39.5HDP-adapted 22.2 11.3 5.6 39.1role-HDP-adapted 22.3 11.3 5.6 39.2AMI+Fisher+Hub4 21.6 11.1 5.4 38.2role-FLM-adapted 21.4 10.9 5.6 37.9HDP-adapted 21.2 11.1 5.3 37.6role-HDP-adapted 21.2 11.1 5.3 37.5but not role-dependent. The topics were extractedby the HDP/role-HDP models based on the previousASR outputs, using a moving document window witha length of 10 seconds. Three adapted LMs togetherwith the baseline LM were then used to rescore the500-best lists with a common language model weightof 14 (the same as for lattice generation) and no wordinsertion penalty. The adapted LM was destroyed af-ter it was used to rescore the current N-best lists.Table 1 shows the word error rate (WER) results. Wecan see that both the HDP and the role-HDP adaptedLMs yield significant reductions (p < 0.01 according toa matched-pair significance test 3) in WER, comparingto the baseline LMs. Although the role-FLM adaptedLMs also reduce the WER, this deterministic approachis not as effective as the probabilistic topic modeling byintroducing a latent variable – topic. However, thereis no significant difference between the HDP and therole-HDP.4. DiscussionAlthough the approach we used here to incorporate therole as prior knowledge for topic modeling is straight-forward, the preliminary experiments demonstratednot only the better perplexity and WER results, butalso the ability for modeling specific topic distribu-tions for each role. This suggests some future work onthe use of role information as prior knowledge is worthfurther investigation in the following aspects:Probabilistic. The fact that we only assign one rolefor each document implies that we may lose some in-formation, because there are potentially multiple rolesfor a document by using a moving window to obtaindocuments. Therefore, it is better to use a more prob-abilistic way, for example, each document is regarded3http://www.icsi.berkeley.edu/speech/faq/signiftest.htmlas a multinomial distribution over roles, and each rolea multinomial distribution over topics. Moreover, thishelps to model the interactions between roles.Observed vs. Latent. Depending on whether wetreat the role variable as observed or latent, we canexploit the role as prior knowledge for topic modeling(as in this paper), or use other information to infer therole for each document. The latter is useful for mod-eling the human interactions in multiparty meetings.Application. Even if we observed reductions in per-plexity, it is not trivial to transfer the advantage ofusing the role prior knowledge for topic modeling toreal applications such as on language modeling for au-tomatic speech recognition in meetings. We observedno significant difference between the HDP and the role-HDP for ASR. We are interested in either a method ofexplicitly conditioning on the role for language model-ing, or an approach to tightly combining topic modelsand n-gram models.AcknowledgmentsThis work is jointly supported by the Wolfson Micro-electronics Scholarship and the European IST Pro-gramme Project FP6-033812 (AMIDA). This paperonly reflects the authors’ views and funding agenciesare not liable for any use that may be made of theinformation contained herein.ReferencesBilmes, J. A., & Kirchhoff, K. (2003). Factored lan-guage models and generalized parallel backoff. Pro-ceedings of HLT/NACCL (pp. 4–6).Hain, T., & et al. (2007). The AMI system for the tran-scription of speech in meetings. Proc. of ICASSP’07.Huang, S., & Renals, S. (2008). Modeling topic androle information in meetings using the hierarchicalDirichlet process. Proc. of Machine Learning forMultimodal Interaction (MLMI’08).Kneser, R., Peters, J., & Klakow, D. (1997). Languagemodel adaptation using dynamic marginals. Proc. ofEurospeech. Rhodes.Renals, S., Hain, T., & Bourlard, H. (2007). Recogni-tion and interpretation of meetings: The AMI andAMIDA projects. Proc. IEEE Workshop on Auto-matic Speech Recognition and Understanding.Teh, Y., Jordan, M., Beal, M., & Blei, D. (2006). Hi-erarchical Dirichlet processes. Journal of the Amer-ican Statistical Association, 101, 1566–1581.24

Using Participant Role in Multiparty Meetings as Prior Knowledge for Nonparametric Topic Modeling

Edinburgh Research Archive

Using Participant Role in Multiparty Meetingsas Prior Knowledge for Nonparametric Topic ModelingSongfang Huang s.f.huang@ed.ac.ukSteve Renals s.renals@ed.ac.ukThe Centre for Speech Technology Research, University of Edinburgh, Edinburgh, EH8 9LW, United KingdomAbstractIn this paper we introduce our attempts to in-corporate the participant role information inmultiparty meetings for document modelingusing the hierarchical Dirichlet process. Theperplexity and automatic speech recognitionresults demonstrate that the participant roleinformation is a promising prior knowledgesource to be combined with language modelsfor automatic speech recognition and interac-tion modeling for multiparty meetings.1. IntroductionIn recent years there has been growing research in-terest in the automatic speech recognition (ASR) formultiparty meetings, which is of essential importancefor the subsequent meeting processing such as con-tent analysis, summarisation, discourse analysis, andinformation retrieval. In this paper, we consider animproved language model (LM) in a state-of-the-artlarge vocabulary ASR system for meetings, based onthe prior knowledge of participant roles. More specifi-cally, we estimate the word distribution over the role ofeach participant, i.e., P (w|r), and use this as unigrammarginals to adapt a conventional n-gram LM.The AMI and AMIDA (http://www.amiproject.org)projects are dedicated to the development of tech-nologies to enhance the recognition and interpreta-tion of interactions between people in multiparty meet-ings (Renals et al., 2007). The AMI Meeting Cor-pus collected by the AMI project consists of 100 hoursof multimodal meeting recordings with comprehensiveannotations at a number of different levels. About 70%of the corpus was elicited using a design scenario, inAppearing in the Workshop on Prior Knowledge forText and Language Processing at ICML/UAI/COLT 2008,Helsinki, Finland, 2008.which the participants play the roles of employees, i.e.,project manager (PM), marketing expert (ME), userinterface designer (UI), and industrial designer (ID),in an electronics company that decides to develop anew type of television remote control. Our intuitionis that, since different participants play different roles,there may be a different word distribution, and in turndifferent dominant words, specific to each role. For ex-ample, we expect a project manager is more likely tospeak words relating to the coordination of meetings,i.e., meeting, project, or present, while a user interfacedesigner may favor words on interaction mediums likescreen, voice, or speech.Topic models have received much attention in the ma-chine learning community, which follows the “bag-of-words” assumption, i.e., words in a document are ex-changeable. In this paper we attempt to incorporatethe participant role as prior knowledge in topic models,by assigning role information to exchangeable words ina document. This could be achieved within the flexi-ble framework of topic models, by introducing an ad-ditional observed variable for the role into the graphi-cal model. By assuming that each role has a mixturedistribution over the latent topics, we could infer thetopic distribution specific to each role. We could fur-ther estimate P (w|r) for each role r by integrating outthe latent topics. Moreover, incorporating the role intopic models enables not only the document modeling,but also the interaction modeling in meetings.An alternative approach to modeling the relationshipbetween the participant role information and lexicalwords is to directly estimate the conditional probabil-ity P (w|r) based on the co-occurrence statistics of rolesand words, using the maximum likelihood principle.As a comparison to the probabilistic topic models, wealso introduce in this paper a deterministic approachto modeling roles and words, by regarding the role asan additional feature (factor) of lexical words in anMLE-based LM.21Workshop on Prior Knowledge for Text and Language Processing2. Modeling ApproachesWe consider two modeling approaches to the estima-tion of P (w|r): one is a hierarchical Bayesian modelusing the hierarchical Dirichlet process (HDP) as theprior, and the other is a factored approach using thefactored language model (FLM).2.1. Hierarchical Bayesian ModelThe hierarchical Dirichlet process (Teh et al., 2006)is a nonparametric generalization of latent Dirichletallocation (LDA), which extends the standard LDAmodel to infinite and hierarchical topic modeling.Conversational speech consists of sequences of utter-ances, which do not comprise well-defined documents.We used the following procedure to obtain documents:for each scenario meeting, first align all the wordsin it along a common timeline; then for each sen-tence/segment, collect those non-stop words belongingto a window of length L as the document, by backtrac-ing from the end time of the sentence/segment. Therole that has been assigned to the most of words in thewindow is selected as the role for that document. Weuse a moving window with L = 20 seconds over thesequences of words to obtain documents.We incorporate the participant role by extending the2-level HDP (Teh et al., 2006) in Figure 1(A) to a thirdlevel, as shown in Figure 1(B), role-HDP. An DP Gris assigned for each of the four roles (PM,ME,UI,ID),which then served as the parent DP (the base proba-bility measure) in the HDP hierarchy for all those DPscorresponding to documents belonging to that role.(A)01(B)Figure 1. The graphical model depictions for (A) 2-levelHDP, and (B) role-HDP.2.2. Factored Language ModelOne straightforward method for modeling words androles is to use the maximum likelihood estimationbased on the co-occurrences of words w and therole information r, i.e., training a bigram-like modelP (w|r) = Count(w, r)/Count(r). More generally, wecan use a factored language model (Bilmes & Kirch-hoff, 2003) to model words and role deterministically.The FLM, initially developed to address the languagemodeling problems faced by morphologically rich orinflected languages, is a generalization of standard n-gram language models, in which each word wt is de-composed into a bundle of K word-related features(called factors), wt ≡ f1:Kt = {f1t , f1t , . . . , fKt }. Fac-tors may include the word itself. Each word in an FLMis dependent not only on a single stream of its pre-ceding words, but also on additional parallel streamsof factors. Combining with interpolation or general-ized parallel backoff (GPB) (Bilmes & Kirchhoff, 2003)strategies, multiple backoff paths may be used simul-taneously.We exploit two factors for word w at time t: the wordwt itself and the corresponding role rt, as shown inFigure 2. All the words in a sentence share a com-mon role, i.e., rt = rt−1 = . . . = r1 in Figure 2. Weuse a simple backoff strategy, for example, by movingfrom the model P (wt|rt) directly down to the unigrammodel P (wt). We refer this model to the role-FLM.gtmin=1kndiscountgtmin=1kndiscountFigure 2. The graphical model representation and backoffpath for role-FLM.2.3. Combination with n-gram LMsAs in (Kneser et al., 1997), we use the dynamic uni-gram marginals P (w|r), from either the role-HDP orthe role-FLM, for LM adaptation:Padapt(w|h) = Pback(w|h) ·(P (w|r)Pback(w))µ/z(h) (1)where h is the history of w, Pback(w|h) the baselinen-gram, Padapt(w|h) the adapted n-gram, and z(h)a normalisation factor. For the role-HDP, P (w|r) ≈∑Kk=1 φkw · θdk with φk estimated during training andremaining fixed in testing, while θd are document-dependent (and in turn are role-dependent because θd22Workshop on Prior Knowledge for Text and Language Processingare derived from Gr) and thus are calculated dynami-cally for each test document.3. Experiments and Results13.1. Empirical ExperimentWe first carried out some empirical analyses for theHDP and the role-HDP. The HDP was implementedas an extension to the SRILM toolkit2. We trained theHDP and the role-HDP models using different values(k = 1, . . . , 100) for the initial number of topics. Weused uniform distribution for H, i.e., Hw = 1/W . Allmodels were trained using the fold-2−4 of the AMIscenario meetings, with a fixed size vocabulary of 7,910words, by the Markov Chain Monte Carlo (MCMC)sampling method. The concentration parameters weresampled using the auxiliary variable sample scheme in(Teh et al., 2006). We ran 3,000 iterations to burn-in, then collected 10 samples from the posteriors tocalculate the unigram perplexity on the fold-1 testingdata, with the sample step of 5.Figure 3 shows the perplexity results, from which wecan see that the role-HDP produced better results thanthe HDP. Our understanding for the improvement isthat by using the role prior knowledge, documents withthe same role share the strengths in the HDP frame-work. In addition, we show in Figure 4 the top twotopics for each role. It is interesting to find out thateach role has some specific topics with high probabil-ities, while they also tend to interact with each otheron some common topics, i.e., the button topic appearswith high probability for all the four roles in Figure 4.0 20 40 60 80 1005005506006507007508008509009501000Initial number of topics in the HDP/role−HDPPerplexity  HDProle−HDPFigure 3. The perplexity results for the HDP/role-HDP.1Some of the results here were appearing in anotherpaper by the authors in (Huang & Renals, 2008).2http://www.speech.sri.com/projects/srilmPM ME UI ID(29) 0.28 (14) 0.13 (20) 0.27 (14) 0.16 (14) 0.25 (19) 0.15 (14) 0.17 (6) 0.16DESIGNMEETINGPROJECTMINUTESUSERINTERFACEPRESENTPRODUCTDESIGNERLOOKSTARTSUREBITGUESSTHANKBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDREMOTECONTROLLOOKMARKETCONTROLSMEANEASYPRODUCTDESIGNBUTTONSFANCYIMPORTANTPERCENTFINDLOTBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDRECOGNITIONSPEECHVOICEL_C_D_SCREENMICROPHONECONTROLLERREMOTEPROBLEMTECHNOLOGYCOFFEESPEAKERCOSTCONTROLSYSTEMBUTTONBUTTONSCHANNELSCREENSCROLLVOLUMEMENUL_C_D_REMOTEWHEELMEANCHANNELSCONTROLPRESSKINDCHIPSIMPLEADVANCEDREGULARREMOTEL_C_D_BATTERYSPEAKEREXPENSIVEINFRAREDSTATIONSAMPLEBATTERIESCOSTCONTROLFigure 4. The example topics for the four roles using therole-HDP.3.2. ASR ExperimentTo further investigate the effectiveness of employingthe role as prior knowledge for topic modeling, we per-formed ASR experiments on multiparty meetings. Weused part of the AMI Meeting Corpus for our exper-iments. There are 138 scenario meetings in total, ofwhich 118 were used for training and the other 20 fortesting (about 11 hours). The procedure and param-eters used to train the HDP/role-HDP were the sameas those used in Section 3.1, except that we used a dif-ferent split-up of the AMI scenario meetings for ASR.We trained two baseline LMs: the first one used theFisher conversational telephone speech data (fisher-03-p1+p2), and the second used three datasets from theAMI training data, the Fisher, and the Hub-4 broad-cast news data (hub4-lm96). The two baseline LMswere trained with standard parameters using SRILM:trigrams, cut-off value of 2 for trigram counts, mod-ified Kneser-Ney smoothing, interpolated model. Acommon vocabulary with 56,168 words was used forthe two LMs, which has 568 out-of-vocabulary (OOV)words for the AMI test data.We investigated the effectiveness of the adapted LMsbased on topic and role information from meetings ona practical large vocabulary ASR system. The AMI-ASR system (Hain & et al., 2007) was used as the base-line system. We began from the lattices for the wholeAMI Meeting Corpus, generated by the AMIASR sys-tem using a trigram LM trained on a large set of datacoming from Fisher, Hub4, Switchboard, webdata, andvarious meeting sources including AMI. We then gen-erated 500-best lists from the lattices for each utter-ance. We adapted the two baseline LMs (Fisher andAMI+Fisher+Hub4) using Equation (1) according tothe unigram marginals from the role-FLM, the HDP,and the role-HDP respectively. For the HDP, we usedP (w|d) ≈∑Kk=1 φkw · θdk as the unigram marginals,i.e., the difference from P (w|r) by the role-HDP isthat in the HDP θd are only document-dependent23Workshop on Prior Knowledge for Text and Language ProcessingTable 1. The %WER results of ASR experiments usingadapted LMs on the AMI scenario meetings.LMs SUB DEL INS WERFisher 22.7 11.4 5.8 39.9role-FLM-adapted 22.5 11.1 5.9 39.5HDP-adapted 22.2 11.3 5.6 39.1role-HDP-adapted 22.3 11.3 5.6 39.2AMI+Fisher+Hub4 21.6 11.1 5.4 38.2role-FLM-adapted 21.4 10.9 5.6 37.9HDP-adapted 21.2 11.1 5.3 37.6role-HDP-adapted 21.2 11.1 5.3 37.5but not role-dependent. The topics were extractedby the HDP/role-HDP models based on the previousASR outputs, using a moving document window witha length of 10 seconds. Three adapted LMs togetherwith the baseline LM were then used to rescore the500-best lists with a common language model weightof 14 (the same as for lattice generation) and no wordinsertion penalty. The adapted LM was destroyed af-ter it was used to rescore the current N-best lists.Table 1 shows the word error rate (WER) results. Wecan see that both the HDP and the role-HDP adaptedLMs yield significant reductions (p < 0.01 according toa matched-pair significance test 3) in WER, comparingto the baseline LMs. Although the role-FLM adaptedLMs also reduce the WER, this deterministic approachis not as effective as the probabilistic topic modeling byintroducing a latent variable – topic. However, thereis no significant difference between the HDP and therole-HDP.4. DiscussionAlthough the approach we used here to incorporate therole as prior knowledge for topic modeling is straight-forward, the preliminary experiments demonstratednot only the better perplexity and WER results, butalso the ability for modeling specific topic distribu-tions for each role. This suggests some future work onthe use of role information as prior knowledge is worthfurther investigation in the following aspects:Probabilistic. The fact that we only assign one rolefor each document implies that we may lose some in-formation, because there are potentially multiple rolesfor a document by using a moving window to obtaindocuments. Therefore, it is better to use a more prob-abilistic way, for example, each document is regarded3http://www.icsi.berkeley.edu/speech/faq/signiftest.htmlas a multinomial distribution over roles, and each rolea multinomial distribution over topics. Moreover, thishelps to model the interactions between roles.Observed vs. Latent. Depending on whether wetreat the role variable as observed or latent, we canexploit the role as prior knowledge for topic modeling(as in this paper), or use other information to infer therole for each document. The latter is useful for mod-eling the human interactions in multiparty meetings.Application. Even if we observed reductions in per-plexity, it is not trivial to transfer the advantage ofusing the role prior knowledge for topic modeling toreal applications such as on language modeling for au-tomatic speech recognition in meetings. We observedno significant difference between the HDP and the role-HDP for ASR. We are interested in either a method ofexplicitly conditioning on the role for language model-ing, or an approach to tightly combining topic modelsand n-gram models.AcknowledgmentsThis work is jointly supported by the Wolfson Micro-electronics Scholarship and the European IST Pro-gramme Project FP6-033812 (AMIDA). This paperonly reflects the authors’ views and funding agenciesare not liable for any use that may be made of theinformation contained herein.ReferencesBilmes, J. A., & Kirchhoff, K. (2003). Factored lan-guage models and generalized parallel backoff. Pro-ceedings of HLT/NACCL (pp. 4–6).Hain, T., & et al. (2007). The AMI system for the tran-scription of speech in meetings. Proc. of ICASSP’07.Huang, S., & Renals, S. (2008). Modeling topic androle information in meetings using the hierarchicalDirichlet process. Proc. of Machine Learning forMultimodal Interaction (MLMI’08).Kneser, R., Peters, J., & Klakow, D. (1997). Languagemodel adaptation using dynamic marginals. Proc. ofEurospeech. Rhodes.Renals, S., Hain, T., & Bourlard, H. (2007). Recogni-tion and interpretation of meetings: The AMI andAMIDA projects. Proc. IEEE Workshop on Auto-matic Speech Recognition and Understanding.Teh, Y., Jordan, M., Beal, M., & Blei, D. (2006). Hi-erarchical Dirichlet processes. Journal of the Amer-ican Statistical Association, 101, 1566–1581.24

https://www.pure.ed.ac.uk/ws/files/20126415/ptkl.pdf

Using Participant Role in Multiparty Meetings as Prior Knowledge for Nonparametric Topic Modeling

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer

Edinburgh Research Archive