How to improve TTS systems for emotional expressivity

Several experiments have been carried out that revealed weaknesses of the current Text-To-Speech (TTS) systems in their emotional expressivity. Although some TTS systems allow XML-based representations of prosodic and/or phonetic variables, few publications considered, as a pre-processing stage, the use of intelligent text processing to detect affective information that can be used to tailor the parameters needed for emotional expressivity. This paper describes a technique for an automatic prosodic parameterization based on affective clues. This technique recognizes the affective information conveyed in a text and, accordingly to its emotional connotation, assigns appropriate pitch accents and other prosodic parameters by XML-tagging. This pre-processing assists the TTS system to generate synthesized speech that contains emotional clues. The experimental results are encouraging and suggest the possibility of suitable emotional expressivity in speech synthesis

Automated assessment of second language comprehensibility: Review, training, validation, and generalization studies

Whereas many scholars have emphasized the relative importance of comprehensibility as an ecologically valid goal for L2 speech training, testing, and development, eliciting listeners’ judgments is time-consuming. Following calls for research on more efficient L2 speech rating methods in applied linguistics, and growing attention toward using machine learning on spontaneous unscripted speech in speech engineering, the current study examined the possibility of establishing quick and reliable automated comprehensibility assessments. Orchestrating a set of phonological (maximum posterior probabilities and gaps between L1 and L2 speech), prosodic (pitch and intensity variation), and temporal measures (articulation rate, pause frequency), the regression model significantly predicted how naïve listeners intuitively judged low, mid, high, and nativelike comprehensibility among 100 L1 and L2 speakers’ picture descriptions. The strength of the correlation (r = .823 for machine vs. human ratings) was comparable to naïve listeners’ interrater agreement (r = .760 for humans vs. humans). The findings were successfully replicated when the model was applied to a new dataset of 45 L1 and L2 speakers (r = .827) and tested under a more freely constructed interview task condition (r = .809)

〈共同研究プロジェクト紹介〉領域指定型 : 日本語教育のためのコーパスを利用したオンライン日本語アクセント辞書の開発 オンライン日本語アクセント辞書OJADの開発と利用

東京大学大学院工学系研究科「自然に聞こえる発音を身に付けたい」と考える学生が多い一方で,発音教育,特に韻律教育に当てる時間が限られている。また教師本人が発音(韻律)教育を受けていないために,効果的な指導方法を模索せざるを得ない状況にある。発話の自然さを上げるためには,フレージング(句を単位として「へ」の字型のピッチパターンを意識して発声させる)+ポージング(「へ」と「へ」の間には意識的にポーズを置く)の習得が重要になる。より自然な発声を目指す場合,フレーズを構成する個々の単語のアクセントも習得することが望ましい。しかし日本語は文脈によってアクセントが頻繁に変わる特性を持ち,このアクセント変形に十分対応した教材は存在していなかった。本研究では,日本語音声教育,特に,イントネーション+アクセント教育を強力に支援するインフラストラクチャであるオンライン日本語アクセント辞書(OJAD)を開発した。現在,世界中の日本語教育機関で利用されるに至っている。Many learners of Japanese want to acquire the ability to speak "natural" Japanese, but only a limited amount of time is allowed for pronunciation and prosody instruction in class. Also, teachers themselves have often not received good instruction in pronunciation and prosody, so it is difficult for them to provide such instruction to learners. It is well known that emphasis on phrasing and pausing is very effective and efficient in improving naturalness. To improve it even more, knowledge of word accent is required, but Japanese word accent often varies depending on context. At present there are no good textbooks for teaching Japanese accent and its variability. To help provide learners with better materials, this project developed the Online Japanese Accent Dictionary (OJAD) to support effective Japanese pronunciation and prosody instruction. The OJAD is currently being used in Japanese language teaching programs around the world

New Approach to Teaching Japanese Pronunciation in the Digital Era - Challenges and Practices

Pronunciation has been a black hole in the L2 Japanese classroom on account of a lack of class time, teacher\u2019s confidence, and consciousness of the need to teach pronunciation, among other reasons. The absence of pronunciation instruction is reported to result in fossilized pronunciation errors, communication problems, and learner frustration. With an intention of making a contribution to improve such circumstances, this paper aims at three goals. First, it discusses the importance, necessity, and e ectiveness of teaching prosodic aspects of Japanese pronunciation from an early stage in acquisition. Second, it shows that Japanese prosody is challenging because of its typological rareness, regardless of the L1 backgrounds of learners. Third and finally, it introduces a new approach to teaching L2 pronunciation with the goal of developing L2 comprehensibility by focusing on essential prosodic features, which is followed by discussions on key issues concerning how to implement the new approach both inside and outside the classroom in the digital era

Open-source Software for Developing Anthropomorphic Spoken Dialog Agents

An architecture for highly-interactive human-like spoken-dialog agent is discussed in this paper. In order to easily integrate the modules of different characteristics including speech recognizer, speech synthesizer, facial-image synthesizer and dialog controller, each module is modeled as a virtual machine that has a simple common interface and is connected to each other through a broker (communication manager). The agent system under development is supported by the IPA and it will be publicly available as a software toolkit this year

コーパス日本語学のための言語資源 : 形態素解析用電子化辞書の開発とその応用

千葉大学国立国語研究所国立国語研究所京都高度技術研究所東京大学情報通信研究機構国立国語研究所Chiba UniversityThe National Institute for Japanese LanguageThe National Institute for Japanese LanguageASTEMThe University of TokyoNational Institute of Information and Communications TechnologyThe National Institute for Japanese Languageコーパス日本語学への応用を指向した形態素解析用電子化辞書UniDicを開発した。大規模コーパスに対する形態論情報付与作業には,計算機を用いた形態素解析システムの利用が不可欠であるが,既存の形態素解析システム用辞書には,コーパス日本語学への応用を考える上でさまざまな不都合がある。1つは,単位の認定がある場合には長く,ある場合には短いといった不揃いがあることであり,もう1つは,異表記や異形態に対して同一の見出しが与えられないということである。言語研究で重要な要件となる,このような単位の斉一性や見出しの同一性への対処といったことを中心に,本電子化辞書の設計方針とそれを実装した辞書データベースシステムについて述べる。さらに,この設計の有用性を示すため,表記や語形の変異に関するコーパス分析の事例を紹介する。In this paper, we describe the design and the implementation of an electronic dictionary for morphological analysis, UniDic, which aims particularly at application to Japanese corpus linguistics. It has been indispensable for the development of a large-scale corpus to utilize an automatic morphological analyzer on computer. The existing dictionaries for morphological analyzers, however, reveal lots of problems when used in corpus linguistics, such as unevenness in defining a unit and failure in handling allomorphs and orthographic variants. Our dictionary, in contrast, deals with the uniformity of units and the identity of indexes, which are important requirements for linguistic analysis of corpora. We adopt multi-level definition of word units, consisting of short-, middle-, and long-unit words, and structured representation of indexes, composed of lemma, word form, orthography, and pronunciation. We develop a database system that straight-forwardly implements this design of the dictionary and a friendly user-interface for dictionary builders to be capable of searching and registering entries with grasping the complex structure of the indexes. We also show how this structured representation benefits us in analyzing morphologically annotated corpora, presenting case studies that investigate the variation of word form in spoken language corpus and the variation of orthography in written language corpus