A serious game for children with speech disorders and hearing problems

Tezin basılısı İstanbul Şehir Üniversitesi Kütüphanesi'ndedir.Speechimpedimentaffectingchildrenwithhearingdifficultiesandspeechdisordersrequires speech therapy and much practice to overcome. In fact, speech therapy via serious games gives an opportunity to children with speech disorders and hearing problems to overcome their problems. As far as children are more inclined to play games, so we intend to learn them by entertainments like serious games. In this thesis, we have designed and implemented a serious game that can be used both as a therapy and as a tool to measure the performance of children with speech impediments in which children will learn to speak specific words that they are expected to know before the age of 7. And then we will teach them how to make sentences. The game consists of three steps. The first step provides information for parents or therapists to decide if their child needs speech therapy or not. In the second step, the child starts to learn specific words while playing the game. The third step aims to measure the performance of the child and evaluate how much the child has learned at the end of the game. The game has an avatar which can be controlled by the child through speech, with the objective of moving the avatar around the environment to earn coins. The avatar is controlled by both voice commands such as Jump, Ahead, Back, Left, Right, and arrow keys of the keyboard. The child will be guided by an arrow during the game instead of getting help from a therapist or a teacher to guide the child to the next goal. This allows the child to practice longer hours, compared to clinical approaches under the supervision of a therapist, which are time-limited. Our preliminary performance measurements indicate an improvement of 40% for children who play our game at least 5 times and a specific period of time.Declaration of Authorship ii Abstract iv Öz v Acknowledgments vii List of Figures x List of Tables xi Abbreviations xii 1 Introduction 1 1.1 Introduction ................................... 1 1.1.1 Learning definition ........................... 1 1.1.2 Does gamification work? ........................ 2 1.2 Introduction to Serious Games: ........................ 4 1.2.1 What is serious games? ........................ 4 1.2.2 First Serious Game ........................... 5 1.2.3 Background on Serious Games .................... 5 1.3 Research Problems ............................... 7 1.4 Motivation.................................... 8 1.5 Research Contributions............................. 9 1.5.1 Research Publications ......................... 9 1.6 Thesis Outline ................................. 9 2 Background 11 2.1 Related Works ................................. 11 2.2 An overview of Serious Games in health ................... 13 2.3 Does speech therapy and language recovery work? .............. 14 2.4 A literature survey of serious games for speech disorder ........... 14 2.5 Main Characteristics of Into the Forest Game ................ 15 3 Proposed System 19 3.1 Game engine analysis .............................. 19 3.2 Avatar ...................................... 20 3.3 Proposed Game ................................. 21 4 Implementation 30 4.1 Preliminary Testing............................... 30 4.2 Testing ...................................... 32 5 Conclusion and Future Work 37 5.1 Conclusion.................................... 37 5.1.1 Future Work .............................. 38 Bibliography 3

Automatic analysis of pathological speech

De ernst van een spraakstoornis wordt vaak gemeten a.d.h.v. spraakverstaanbaarheid. Deze maat wordt in de klinische praktijk vaak bepaald met een perceptuele test. Zo’n test is van nature subjectief vermits de therapeut die de test afneemt de (stoornis van de) patiënt vaak kent en ook vertrouwd is met het gebruikte testmateriaal. Daarom is het interessant te onderzoeken of men met spraakherkenning een objectieve beoordelaar van verstaanbaarheid kan creëren. In deze thesis wordt een methodologie uitgewerkt om een gestandaardiseerde perceptuele test, het Nederlandstalig Spraakverstaanbaarheidsonderzoek (NSVO), te automatiseren. Hiervoor wordt gebruik gemaakt van spraakherkenning om de patiënt fonologisch en fonemisch te karakteriseren en uit deze karakterisering een spraakverstaanbaarheidsscore af te leiden. Experimenten hebben aangetoond dat de berekende scores zeer betrouwbaar zijn. Vermits het NSVO met nonsenswoorden werkt, kunnen vooral kinderen hierdoor leesfouten maken. Daarom werden nieuwe methodes ontwikkeld, gebaseerd op betekenisdragende lopende spraak, die hiertegen robuust zijn en tegelijk ook in verschillende talen gebruikt kunnen worden. Met deze nieuwe modellen bleek het mogelijk te zijn om betrouwbare verstaanbaarheidsscores te berekenen voor Vlaamse, Nederlandse en Duitse spraak. Tenslotte heeft het onderzoek ook belangrijke stappen gezet in de richting van een automatische karakterisering van andere aspecten van de spraakstoornis, zoals articulatie en stemgeving