Game design and the gamification of content : assessing a project for learning sign language

Comunicação apresentada na EDULEARN 2015, realizada em Barcelona de 6-8 de julho de 2015This paper discusses the concepts of game design and gamification of content, based on the development of a serious game aimed at making the process of learning sign language enjoyable and interactive. In this game the player controls a character that interacts with various objects and non- player characters, with the aim of collecting several gestures from the Portuguese Sign Language corpus. The learning model used pushes forward the concept of gamification as a learning process valued by students and teachers alike, and illustrates how it may be used as a personalized device for amplifying learning. Our goal is to provide a new methodology to involve students and general public in learning specific subjects using a ludic, participatory and interactive approach supported by ICT- based tools. Thus, in this paper we argue that perhaps some education processes could be improved by adding the gaming factor through technologies that are able to involve students in a way that is more physical (e.g. using Kinect and sensor gloves), so learning becomes more intense and memorable

Leveraging 2D pose estimators for American Sign Language Recognition

Most deaf children born to hearing parents do not have continuous access to language, leading to weaker short-term memory compared to deaf children born to deaf parents. This lack of short-term memory has serious consequences on their mental health and employment rate. To this end, prior work has explored CopyCat, a game where children interact with virtual actors using sign language. While CopyCat has been shown to improve language generation, reception, and repetition, it uses expensive hardware for sign language recognition. This thesis explores the feasibility of using 2D off-the-shelf camera-based pose estimators such as MediaPipe for complementing sign language recognition and moving towards a ubiquitous recognition framework. We compare MediaPipe with 3D pose estimators such as Azure Kinect to determine the feasibility of using off-the-shelf cameras. Furthermore, we develop and compare Hidden Markov Models (HMMs) with state-of-the-art recognition models like Transformers to determine which model is best suited for American Sign Language Recognition in a constrained environment. We find that MediaPipe marginally outperforms Kinect in various experimental settings. Additionally, HMMs outperform Transformers by on average 17.0% on recognition accuracy. Given these results, we believe that a widely deployable game using only a 2D camera is feasible.Undergraduat

Using serious games for learning sign language combining video, enhanced interactivity and VR technology

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with or they disordered they acquired during their life. 900,000 people in the UK are severely or profoundly deaf and based on a study by Action On Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress, and an even larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a serious game (SG) that aims to close the communication gap between able hearing people and people with a hearing impediment by providing a tool that facilitates BSL learning targeting adult population. The paper presents the theoretical framework supporting adult learning based on which a SG game using Virtual Reality (VR) technology has been developed. It explains the experimental framework of the study and presents the creation of the research instruments to facilitate the study comprising of a SG that integrates video and conventional video based educational material. It reports and analyses the study results that demonstrate the advantage of the SG in effectively supporting users learning a set of BSL signs and it presents qualitative outcomes that inform the further development of the game to serve learning needs. The paper closes with conclusions, directions for further development of this educational resource and future studies

Using Serious Games for Learning British Sign Language Combining Video, Enhanced Interactivity, and VR Technology

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with or they disordered they acquired during their life. 900,000 people in the UK are severely or profoundly deaf and based on a study by Action On Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress, and an even larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a serious game (SG) that aims to close the communication gap between able hearing people and people with a hearing impediment by providing a tool that facilitates BSL learning targeting adult population. The paper presents the theoretical framework supporting adult learning based on which a SG game using Virtual Reality (VR) technology has been developed. It explains the experimental framework of the study and presents the creation of the research instruments to facilitate the study comprising of a SG that integrates video and conventional video based educational material. It reports and analyses the study results that demonstrate the advantage of the SG in effectively supporting users learning a set of BSL signs and it presents qualitative outcomes that inform the further development of the game to serve learning needs. The paper closes with conclusions, directions for further development of this educational resource and future studies

Reverse-Engineering Satire, or "Paper on Computational Humor Accepted Despite Making Serious Advances"

Humor is an essential human trait. Efforts to understand humor have called out links between humor and the foundations of cognition, as well as the importance of humor in social engagement. As such, it is a promising and important subject of study, with relevance for artificial intelligence and human-computer interaction. Previous computational work on humor has mostly operated at a coarse level of granularity, e.g., predicting whether an entire sentence, paragraph, document, etc., is humorous. As a step toward deep understanding of humor, we seek fine-grained models of attributes that make a given text humorous. Starting from the observation that satirical news headlines tend to resemble serious news headlines, we build and analyze a corpus of satirical headlines paired with nearly identical but serious headlines. The corpus is constructed via Unfun.me, an online game that incentivizes players to make minimal edits to satirical headlines with the goal of making other players believe the results are serious headlines. The edit operations used to successfully remove humor pinpoint the words and concepts that play a key role in making the original, satirical headline funny. Our analysis reveals that the humor tends to reside toward the end of headlines, and primarily in noun phrases, and that most satirical headlines follow a certain logical pattern, which we term false analogy. Overall, this paper deepens our understanding of the syntactic and semantic structure of satirical news headlines and provides insights for building humor-producing systems.Comment: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 201

User interface evaluation of serious games for students with intellectual disability

We have designed and evaluated around 10 serious games under the EU Leonardo Transfer of Innovation Project: Game On Extra Time (GOET) project http://goet-project.eu/. The project supports people with learning disabilities and additional sensory impairments in getting and keeping a job by helping them to learn, via games-based learning; skills that will help them in their working day. These games help students to learn how to prepare themselves for work, dealing with everyday situations at work, including money management, travelling independently etc. This paper is concerned with the potential of serious games as effective and engaging learning resources for people with intellectual disabilities. In this paper we will address questions related to the design and evaluation of such games, and our design solutions to suit the individual learning needs of our target audiences

Spartan Daily, September 21, 2004

Volume 123, Issue 16https://scholarworks.sjsu.edu/spartandaily/10020/thumbnail.jp

Using Serious Games for Learning British Sign Language Combining Video, Enhanced Interactivity, and VR Technology

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with, or they developed during their life. Nine hundred thousand people in the UK are severely or profoundly deaf. Based on a study by Action on Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress. It also leaves even a larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a Serious Game (SG) that aims to close the communication gap between able hearing people and people with hearing impairment by providing a tool that facilitates BSL learning targeting the adult population. The paper presents the theoretical framework supporting adult learning based on which a SG game using Virtual Reality (VR) technology has been developed. The paper explains the experimental framework of the study. It presents the creation of the research instruments to facilitate the study comprising of a SG that integrates video and conventional video-based educational material. It reports and analyses the study results that demonstrate the advantage of the SG in effectively supporting users learning a set of BSL signs. It also presents qualitative outcomes that inform the further development of the game to serve learning needs. The paper closes with conclusions, directions for further development of this educational resource, and future studies