HCI for the deaf community: developing human-like avatars for sign language synthesis

With ever increasing computing power and advances in 3D animation technologies it is no surprise that 3D avatars for sign language (SL) generation are advancing too. Traditionally these avatars have been driven by somewhat expensive and inflexible motion capture technologies and perhaps this is the reason avatars do not feature in all but a few user interfaces (UIs). SL synthesis is a competing technology that is less costly, more versatile and may prove to be the answer to the current lack of access for the Deaf in HCI. This paper outlines the current state of the art in SL synthesis for HCI and how we propose to advance this by improving avatar quality and realism with a view to ameliorating communication and computer interaction for the Deaf community as part of a wider localisation project

Building a sign language corpus for use in machine translation

In recent years data-driven methods of machine translation (MT) have overtaken rule-based approaches as the predominant means of automatically translating between languages. A pre-requisite for such an approach is a parallel corpus of the source and target languages. Technological developments in sign language (SL) capturing, analysis and processing tools now mean that SL corpora are becoming increasingly available. With transcription and language analysis tools being mainly designed and used for linguistic purposes, we describe the process of creating a multimedia parallel corpus specifically for the purposes of English to Irish Sign Language (ISL) MT. As part of our larger project on localisation, our research is focussed on developing assistive technology for patients with limited English in the domain of healthcare. Focussing on the first point of contact a patient has with a GP’s office, the medical secretary, we sought to develop a corpus from the dialogue between the two parties when scheduling an appointment. Throughout the development process we have created one parallel corpus in six different modalities from this initial dialogue. In this paper we discuss the multi-stage process of the development of this parallel corpus as individual and interdependent entities, both for our own MT purposes and their usefulness in the wider MT and SL research domains

(MU-CTL-01-12) Towards Model Driven Game Engineering in SimSYS: Requirements for the Agile Software Development Process Game

Software Engineering (SE) and Systems Engineering (Sys) are knowledge intensive, specialized, rapidly changing disciplines; their educational infrastructure faces significant challenges including the need to rapidly, widely, and cost effectively introduce new or revised course material; encourage the broad participation of students; address changing student motivations and attitudes; support undergraduate, graduate and lifelong learning; and incorporate the skills needed by industry. Games have a reputation for being fun and engaging; more importantly immersive, requiring deep thinking and complex problem solving. We believe educational games are essential in the next generation of e-learning tools. An extensible, freely available, engaging, problem-based game platform that provides students with an interactive simulated experience closely resembling the activities performed in a (real) industry development project would transform the SE/Sys education infrastructure. Our goal is to extend the state-of-the-art research in SE/Sys education by investigating a game development platform (GDP) from an interdisciplinary perspective (education, game research, and software/systems engineering). A meta-model has been proposed to provide a rigourous foundation that integrates the three disciplines. The GDP is intended to support the semi-automated development of collections of scripted games and their execution, where each game embodies a specific set of learning objectives. The games are scripted using a template based approach. The templates integrate three approaches: use cases; storyboards; and state machines (timed, concurrent, hierarchical state machines). The specification templates capture the structure of the game (Game, Acts, Scenes, Screens, Challenges), storyline, characters (player, non-player, external), graphics, music/sound effects, rules, and so on. The instantiated templates are (manually) transformed into XML game scripts that can be loaded into the SimSYS Game Play Engine. As a game is played, the game play events are logged; they are analyzed to automatically assess a player’s accomplishments and automatically adapt the game play script. Currently, we are manually defining a collection of games. The games are being used to ensure the GDP is flexible and reliable (i.e., the prototype can load and correctly run a variety of game scripts), the ontology is comprehensive, and the templates assist in defining well-organized, modular game scripts. In this report, we present the initial part of an Agile Software Development Process game (Act I, Scenes 1 and 2) that embodies learning objectives related to SE fundamentals (requirements, architecture, testing, process); planning with Gantt charts; working with budgets; and selecting a team for an agile development project. A student player is rewarded in the game by getting hired, scoring points, or getting promoted to lead a project. The game has a variety of settings including a classroom, job fair, and a work environment with meeting rooms, cubicles, and a water cooler station. The main non-player characters include a teacher, boss, and an evil peer. In the future, semi-automated support for creating new game scripts will be explored using a wizard interface. The templates will be formally defined, supporting automated transformation into XML game scripts that can be loaded into the SimSYS Game Engine. We also plan to explore transforming the requirements into a notation that can be imported into a commercial tool that supports Statechart simulation