National Institute for Japanese Language and LinguisticsNational Institute of InformaticsLREC 2018 Special Speech Sessions "Speech Resources Collection in Real-World Situations"; Phoenix Seagaia Conference Center, Miyazaki; 2018-05-09In this paper, we introduce representative corpora of spontaneous speech, which have been provided publically in Japan. A large amount of spontaneous speech data is required for research on various themes in speech studies such as speech analysis, speech recognition systems, and natural language processing in recent years. However, it is difficult to collect spontaneous speech data, and few corpora of spontaneous speech are available. Considering the diversity of speech in real-world situations, the data remain insufficient. We show the characteristics of spontaneous Japanese speech corpora gathered and distributed by two organizations: the Speech Resources Consortium at the National Institute of Informatics, and the National Institute for Japanese Language and Linguistics. Then, we describe prospects for the development of spontaneous speech resources

Tomoko Ohsuga

Yuichi Ishimoto

LREC 2018 Special Speech Sessions "Speech Resources Collection in Real-World Situations"; Phoenix Seagaia Conference Center, Miyazaki; 2018-05-09application/pdfNational Institute for Japanese Language and LinguisticsNational Institute of InformaticsIn this paper, we introduce representative corpora of spontaneous speech, which have been provided publically in Japan. A large amount of spontaneous speech data is required for research on various themes in speech studies such as speech analysis, speech recognition systems, and natural language processing in recent years. However, it is difficult to collect spontaneous speech data, and few corpora of spontaneous speech are available. Considering the diversity of speech in real-world situations, the data remain insufficient. We show the characteristics of spontaneous Japanese speech corpora gathered and distributed by two organizations: the Speech Resources Consortium at the National Institute of Informatics, and the National Institute for Japanese Language and Linguistics. Then, we describe prospects for the development of spontaneous speech resources.conference pape

Ishimoto, Yuichi

Ohsuga, Tomoko

Academic Repository of the National Institute for Japanese Language and Linguistics / 国立国語研究所学術情報リポジトリ

国立国語研究所学術情報リポジトリSpontaneous speech resources in Japanjournal orpublication titleProceedings of the LREC 2018 Special SpeechSessionspage range 1-5year 2018-05-09URL http://doi.org/10.15084/00001909Spontaneous Speech Resources in JapanYuichi Ishimoto†, Tomoko Ohsuga‡†National Institute for Japanese Language and Linguistics10–2 Midori-cho, Tachikawa, Tokyo 190–8561, Japanyishi@ninjal.ac.jp‡National Institute of Informatics2–1–2 Hitotsubashi, Chiyoda-ku, Tokyo 101–8430, Japanosuga@nii.ac.jpAbstractIn this paper, we introduce representative corpora of spontaneous speech, which have been provided publically in Japan.A large amount of spontaneous speech data is required for research on various themes in speech studies such as speechanalysis, speech recognition systems, and natural language processing in recent years. However, it is difficult to collectspontaneous speech data, and few corpora of spontaneous speech are available. Considering the diversity of speech inreal-world situations, the data remain insufficient. We show the characteristics of spontaneous Japanese speech corporagathered and distributed by two organizations: the Speech Resources Consortium at the National Institute of Informatics,and the National Institute for Japanese Language and Linguistics. Then, we describe prospects for the development ofspontaneous speech resources.Keywords: Japanese corpus, spontaneous speech, natural conversation, corpus distribution1. IntroductionSpeech resources are necessary to promote speech re-search; therefore various speech corpora have beencompiled. Initially, most of the corpora consisted ofwords and sentences read aloud such as numbers, greet-ings, place names, and phonetically balanced phrasesbecause in the past, some providers usually collectedthem for use in constructing early speech recognitionsystems. Although prior data used to be effective, itis no longer sufficient for systems to show high perfor-mance in real-world situations.Read-aloud speeches have different characteristics fromthose of the words and sentences that we utter ineveryday conversations; consequently, the old systemderived from speech data did not exhibit competentperformance for real-life situations. Moreover, spon-taneous utterances are more complex and have moredisfluency than sentences prepared in advance.Spontaneous speech data have thus been required byresearchers; however, it takes much more time torecord spontaneous speech than read-aloud speech.The recorder needs to prepare an environment in whichthe speaker makes spontaneous utterances, or to visita place in which natural conversations occur. Few cor-pora of spontaneous Japanese speeches exist.In this paper, we introduce several spontaneousJapanese speech corpora that are publically dis-tributed and describe their characteristics. Then, wedescribe prospects for the development of spontaneousresources.2. Spontaneous Japanese SpeechCorporaIn this section, we introduce representative speech re-sources of spontaneous Japanese gathered and dis-tributed by two organizations in Japan.2.1. Corpora from NII-SRCThe Speech Resources Consortium at the National In-stitute of Informatics (NII-SRC) was established in2006. It aims to collect speech resources from re-searchers who belong to universities, as well as com-panies that record speech sounds for various purposes,and to distribute them to researchers who need speechdata suitable for their investigations. Although mostresearchers record speech for their purposes only andutilize it, they do not have the means or knowledge todistribute their data.NII-SRC has distributed 43 corpora as of May 2018;Table 1 shows a list of them. As described in the in-troduction, corpora distributed earlier consist of read-aloud speeches, mainly because the providers aimedto apply words and sentences uttered fluently to fun-damental research on speech. Subsequently, sponta-neous speech was collected to apply speech informa-tion processing in an actual environment. Most of theearlier corpora for spontaneous speech were composedof role-play in different situations (such as navigationand shopping) because a question-and-response formatwas preferred for human-computer dialogue systemsbased on speech recognition technology. For example,RWCP-SP96 and RWCP-SP97 — the formal namesof which are “RWCP Spoken Dialogue Corpus, 1996edition” and “1997 edition” — contain face-to-face di-LREC 2018 Special Speech Sesssions 1Name Launched Contents Style Situation NotePASL-DSR 2006 Words, Sentences Read-aloud —UT-ML 2006 Words, Sentences Read-aloud —TMW 2006 Words Read-aloud —GSR-JD 2006 Words, Dialogue Read-aloud, Spontaneous Natural DialectRWCP-SP96 2006 Dialogue Spontaneous Role-playRWCP-SP97 2006 Dialogue Spontaneous Role-playRWCP-SP99 2006 Monologue Read-aloud —RWCP-SP01 2006 Dialogue Spontaneous Role-playPASD 2006 Dialogue Spontaneous Role-playCIAIR-VCV 2006 Words, Sentences Read-aloud —CENSREC-1 2006 Words Read-aloud —CENSREC-1-C 2006 Words Read-aloud —CENSREC-2 2006 Words Read-aloud —CENSREC-3 2006 Words, Sentences Read-aloud —JNAS 2006 Sentences Read-aloud —FW03 2006 Words Read-aloud —RWCP-SSD 2007 Sentences, Non-speech Read-aloud —UME-ERJ 2007 Words, Sentences Read-aloud —UME-JRF 2007 Words, Sentences Read-aloud —RIKEN-DLG 2007 Monologue, Dialogue Spontaneous Role-playMapTask 2007 Dialogue Spontaneous Natural Task-orientedS-JNAS 2007 Sentences Read-aloud —ASJ-JIPDEC 2007 Sentences, Dialogue Read-aloud, Spontaneous Role-playFW07 2007 Words Read-aloud —CENSREC-4 2008 Words Read-aloud —UUDB 2008 Dialogue Spontaneous Natural Task-orientedETL-WD 2008 Words Read-aloud —Tsuruoka91-92 2008 Words, Sentences Read-aloud —INFANT 2008 Dialogue Spontaneous NaturalX-Ray 2010 Sentences Read-aloud —MULTEXT-J 2010 Monologue Acted —MULTEXT-C 2010 Monologue Acted —CENSREC-1-AV 2011 Words Read-aloud —Keio-ESD 2011 Words Acted —JVPD 2011 Words Read-aloud —TITML-IDN 2011 Sentences Read-aloud —TITML-ISL 2012 Sentences Read-aloud —AWA-LTR 2012 Words, Sentences Read-aloud —Aragusuku 2013 Words, Sentences Read-aloud —Oogami 2013 Words, Sentences Read-aloud —OGVC 2013 Dialogue Acted, Spontaneous NaturalChiba3Party 2014 Dialogue Spontaneous NaturalJWC 2017 Words Read-aloud —Table 1: Corpora distributed by the NII-SRC (as of May 2018). Note: “Launched” means the first year ofdistribution by the NII-SRC, rather than the year in which the speech was recorded or distributed directly bythe developers.alogues involving two people: a professional and a cus-tomer who asks questions about purchasing a car andoverseas travel plans. The Priority Areas “Spoken Di-alogue” Simulated Spoken Dialogue Corpus (PASD)also contains conversations between a user and varioussystems (such as those that involve a secretary system,scheduling appointments, travel guides, and telephoneshopping); two people simulate the user and the sys-tem.Although the speakers in these dialogues played rolesin simulated situations, they produced spontaneous ut-terances because they improvised what to say. Thesecorpora have performed to some extent; however, theyare still insufficient for general studies on spontaneousspeech. The critical point of such investigations is notonly to demonstrate the spontaneity of utterances, butalso their naturalness and diversity; it is difficult toachieve these goals in role-playing situations.Through these circumstances, various natural conver-sations have been collected as a new trend. As shownLREC 2018 Special Speech Sesssions 2in Table 1, in recent years, natural situations have be-come more popular than role-playing1. We introducefive corpora, as follows.The Chiba University Japanese Map Task DialogueCorpus (MapTask) (Ichikawa et al., 2000) is a Japaneseversion derived from the Home Communications Re-search Centre (HCRC) Map Task Corpus, which wasdeveloped by a group at the University of Edinburgh,mainly for linguistic research (Thompson et al., 1993).It contains task-oriented dialogues using maps, withtwo participants involved: an instruction-giver whohas a map with a route, and an instruction-followerwho has a map without one. Although the partici-pants had the roles of giver and follower, this was notrole-play because they talked spontaneously in orderto simulate how they naturally speak in everyday life.The Utsunomiya University Spoken DialogueDatabase for Paralinguistic Information Studies(UUDB) (Mori et al., 2008) also consists of task-oriented dialogues. The dialogues were produced from“four-frame cartoon sorting tasks” (Mori et al., 2003)in which two participants have four cards extractedfrom a four-frame cartoon and they estimate theoriginal order. The unique characteristic of thiscorpus is that it was designed to collect spontaneousand emotional utterances for studies on paralinguisticbehavior.The NTT Infant Speech Database (INFANT) (Amanoet al., 2009) contains speech data uttered by five chil-dren (from three families) who are native Japanesespeakers. The data were recorded for more than onehour per month since they were born until they werefive years old. From this corpus, we can obtain the chil-dren’s spontaneous utterances in daily conversationsand the changes they experienced that are associatedwith growing up.The Online Gaming Voice Chat Corpus with Emo-tional Labels (OGVC) (Arimoto et al., 2012) is a col-lection of natural and acted speeches used for emo-tional studies. The natural speech dialogues wererecorded from voice chats that took place in an onlinegame involving 2–3 players. The players expressed alot of emotions because they were absorbed in play-ing the game. In addition, the speech that profes-sional actors uttered in accordance with transcriptionsof the natural speech is also included. For applicationsof emotional research, perceptual emotion labels andtheir intensity rates are appended to the utterances.The Chiba Three-party Conversation Corpus(Chiba3Party) (Den and Enomoto, 2007) is acollection of casual conversations among three peo-ple of the same gender who are friendly with eachother. The recording operator tried to avoid placing1 The GSR(A) “Regional Differences in SpokenJapanese Dialects” Spoken Japanese Dialect Corpus (GSR-JD) aimed to record dialects in each region of Japan andcompare them. Although the launch year of GSR-JD isolder than that of other natural speech corpora, the spon-taneity of the collected conversations is not the primarypurpose.any restrictions on the content and progress of theconversations; thus, the conversations have a highdegree of spontaneity. This corpus aims to contributeto descriptions and modeling of human interactions.Consequently, the transcriptions and morphologi-cal information based on conversation analysis aresubstantial.Hence, recent corpora have been constructed that takediversity of speech into account.2.2. CSJThe National Institute for Japanese Language and Lin-guistics (NINJAL) is a comprehensive research orga-nization. Collaborative efforts between NINJAL andthe Communications Research Laboratory led to thedevelopment of a large-scale, spontaneous speech cor-pus called the Corpus of Spontaneous Japanese (CSJ).This corpus is useful for the investigation and mod-eling of spontaneous speech, as well as the studyof speech recognition and summarization technology;NINJAL has publically distributed the CSJ since 2004(Maekawa, 2003). The CSJ contains monologues con-sisting of academic presentations, simulated publicspeech, and dialogues (such as interviews with speakersand free-form conversations). The academic presen-tations were recorded live in nine different academicsocieties covering the fields of engineering, the socialsciences, and the humanities. The public speeches arestudio recordings of paid laypeople on everyday topicspresented in front of a small audience.One of the special features of the CSJ is that it is thelargest spontaneous speech corpus in Japan. Its speechsignals amount to about 660 hours and were utteredby around 1,400 different speakers. This quantity ofdata satisfies the construction of the language modelfor recognition of spontaneous speech, as well as ap-plications to natural language processing studies onspontaneous speech. Furthermore, the wide range ofspeakers is useful for investigations on phonetic andlinguistic variation caused by spontaneity.Another unique quality of this corpus is its abundanceof linguistic, phonetic, and prosodic labels aligned tothe data. As for the linguistic labels, transcriptiontexts were annotated using two types of part-of-speechsystems, and differed regarding the length of morpho-logical units that reflect the complex word boundariesof the Japanese language. In addition, transcriptiontags that were designed to represent fillers and dis-fluency particular to spontaneous speech were embed-ded in the transcriptions. As for the phonetic labels,phoneme labels considering phonetic events — suchas the release of stop closure, the distinction betweenvoiced affricates and fricatives, and the voicing of vow-els — were assigned to the speech signals. Regardingthe prosodic labels, X-JToBI labels (Maekawa et al.,2002) — which were extended from the J_ToBI schemerepresenting the intonational structure of Japanese— were appended to the transcriptions to representprosodic variations observed in spontaneous speech.Although all of these labels have been adopted intoLREC 2018 Special Speech Sesssions 3only a subset of the CSJ (called the CSJ-Core) due tothe high cost of labeling, there is no other corpus withas many types of labels as these.The CSJ is useful for research on speech recognition,natural language processing, prosodics, linguistics, andthe paralinguistics of spontaneous speech3. ProspectsAs described in Section 2., some spontaneous speechresources were developed. However, considering thediversity of speech in real-world situations, the dataremain insufficient. For example, although INFANTprovides utterances of children under six years old,utterances of children who are a little older, as wellas elderly speakers, are necessary to represent the di-versity caused by the growth and ages of speakers.The Chiba3Party provides casual conversations amongthree participants sitting face-to-face, but conversa-tions in everyday life do not always happen this way.The CSJ is mostly limited to presentations; there-fore, it is possible that the data do not representgeneral spontaneous speech. We believe that sponta-neous speech resources should be developed by manyresearchers in various organizations to satisfy the di-versity of utterances, because single organizations mayproduce biased data.Currently, studies are investigating the followingthemes related to spontaneous Japanese speech:• Emotional speech (Arimoto, 2018)• Elderly speech (Kitaoka et al., 2018)• Areal dialects (Kibe et al., 2018)• Everyday conversations (Koiso et al., 2018)• Multi-party interactions (Bono et al., 2018)• Human-machine (i.e., robot and speech assis-tant systems) interactions (Funakoshi, 2018; Hi-gashinaka et al., 2018)The refereed papers provide details of each study.4. ConclusionWe introduced representative speech resources of spon-taneous Japanese that are publically distributed, anddescribed the characteristics of each resource. In re-cent years, the amount of corpora containing sponta-neous Japanese speech have increased; however, thequantity of speech resources is still insufficient to meetthe demands of studies examining topics such as auto-matic speech recognition and natural language process-ing. We expect that more speech corpora that gatherimprovised utterances will gradually be developed tocover the diversity of spontaneous speech.5. AcknowledgementThis work was supported by JSPS KAKENHI GrantNumber 17H00914, and a project of the Center forCorpus Development, NINJAL.6. Bibliographical ReferencesAmano, S., Kondo, T., Kato, K., and Nakatani,T. (2009). Development of Japanese infant speechdatabase using longitudinal recordings from birth tofive years old. In 2009 Oriental COCOSDA Inter-national Conference on Speech Database and Assess-ments, pages 31–37, Aug.Arimoto, Y., Kawatsu, H., Ohno, S., and Iida, H.(2012). Naturalistic emotional speech collectionparadigm with online game and its psychological andacoustical assessment. Acoustical Science and Tech-nology, 33(6):359–369.Arimoto, Y. (2018). Challenges on building authen-tic emotional speech corpus of spontaneous Japanesedialog. In Proceedings of LREC2018 Special SpeechSessions, pages 6–13.Bono, M., Sakaida, R., Makino, R., and Joh, A.(2018). Miraikan SC corpus: A trial for data col-lection in a semi-open and semi-controlled environ-ment. In Proceedings of LREC2018 Special SpeechSessions, pages 30–34.Den, Y. and Enomoto, M. (2007). A scientific ap-proach to conversational informatics: Description,analysis, and modeling of human conversation. InT. Nishida, editor, Conversational informatics: Anengineering approach, pages 307–330. John Wiley &Sons.Funakoshi, K. (2018). A multimodal multipartyhuman-robot dialogue corpus for real world inter-action. In Proceedings of LREC2018 Special SpeechSessions, pages 35–39.Higashinaka, R., Ishii, R., Matsumura, N., Nunobiki,T., Itoh, A., Inagawa, R., and Tomita, J. (2018).Speech and language resources for the developmentof dialogue systems and problems arising from theirdeployment. In Proceedings of LREC2018 SpecialSpeech Sessions, pages 40–46.Ichikawa, A., Horiuchi, Y., and Tutiya, S. (2000). TheJapanese map task dialogue corpus. Journal of thePhonetic Society of Japan, 4(2):4–15.Kibe, N., Otsuki, T., and Sato, K. (2018). Intona-tional variations at the end of interrogative sen-tences in Japanese dialects: From the “corpus ofjapanese dialects”. In Proceedings of LREC2018 Spe-cial Speech Sessions, pages 21–28.Kitaoka, N., Iribe, Y., and Nishizaki, H. (2018). Con-struction of a corpus of elderly Japanese speechfor analysis and recognition. In Proceedings ofLREC2018 Special Speech Sessions, pages 14–20.Koiso, H., Den, Y., Iseki, Y., Kashino, W., Kawa-bata, Y., Nishikawa, K., Tanaka, Y., and Usuda,Y. (2018). Construction of the corpus of everydayJapanese conversation: An interim report. In Pro-ceedings of LREC2018 (in print).Maekawa, K., Kikuchi, H., Igarashi, Y., and Venditti,J. (2002). X-JToBI: An extended J_ToBI for spon-taneous speech. In Proc. ICSLP2002, pages 1545–1548.Maekawa, K. (2003). Corpus of spontaneous JapaneseLREC 2018 Special Speech Sesssions 4: its design and evaluation. Proceedings of The ISCA& IEEE Workshop on Spontaneous Speech Process-ing and Recognition, pages 7–12.Mori, H., Kasuya, H., Nakamura, M., and Amanuma,M. (2003). Some considerations for designing spo-ken dialogue database from the viewpoint of paralin-guistic information. Acoustical Science and Technol-ogy, 24(6):376–378.Mori, H., Satake, T., Nakamura, M., and Kasuya,H. (2008). Uu database: A spoken dialogue cor-pus for studies on paralinguistic information in ex-pressive conversation. In Proceedings of the 11thInternational Conference on Text, Speech and Di-alogue, TSD ’08, pages 427–434, Berlin, Heidelberg.Springer-Verlag.Thompson, H. S., Anderson, A., Bard, E. G., Doherty-Sneddon, G., Newlands, A., and Sotillo, C. (1993).The HCRC map task corpus: Natural dialogue forspeech recognition. In Proceedings of the Workshopon Human Language Technology, HLT ’93, pages 25–30, Stroudsburg, PA, USA. Association for Compu-tational Linguistics.LREC 2018 Special Speech Sesssions 5

Spontaneous speech resources in Japan

https://repository.ninjal.ac.jp/?action=repository_action_common_download&item_id=1925&item_no=1&attribute_id=48&file_no=1

Spontaneous speech resources in Japan

Abstract

Similar works

Full text

Available Versions

Academic Repository of the National Institute for Japanese Language and Linguistics / 国立国語研究所学術情報リポジトリ