Virtual sign translator in serious games

InforAbERTA 2014 - IV Jornadas de InformáticaApresentação efetuada nas IV Jornadas de Informática da Universidade Abert

Margin to Margin: Arts-Based Research for Digital Outreach to Marginalised Communities

This article discusses the artistic activity titled ‘Conversations with the edge’ that was executed by communities in Australia, Russia and Finland, and curated for an exhibition at the Helinä Rautavaara Museum in Espoo, Finland in 2017. This activity was created in the context of Margin to Margin: Women living on the edges of the world, a larger arts-based research project that took place between four geographical margins: outback South Australia, Finnish Lapland, Russian Kola Peninsula and Namibia. Margin to Margin was a collaboration between artist communities with the aim to explore the relationship between art-making and empowerment of makers living and working ‘on the edges’. The aim of the project was to understand the realities marginalised communities face whilst giving voice to these communities by exhibiting their art in various formats, stimulating digital participation and utilising technology for digital inclusion. The purpose of the article is to develop a model that will guide virtual arts-based project mediation for digital outreach in both urban and regionally situated marginalised communities

Developing undergraduate projects in multinational teams to enhance employability

Our society is experiencing sudden changes in work organization in part due to the growing ease with which people can collaborate. Many successful cases of peer-to-peer models of organization arise and assume leading positions in world economy replacing, in many cases, the traditional hierarchical organization. People are evolving and interacting within heterogeneous teams composed by members from many different cultural groups and with distinct skills and backgrounds. Modern economy requires engineers to excel in collaborative and communication skills at an international setting. However, these competences are not usually addressed in most engineering curricula. We believe that in such a demanding and culturally diverse environment as the labour market is today, it is essential to promote team work and communication skills at an international and intercultural level. In the Multinational Undergraduate Team Work course, MUTW, students develop their capstone project as members of an international team while working at their home institutions. MUTW projects are to be developed by teams of final-year-undergraduate students from a multinational group of higher education institutions working to solve some engineering problem. Team members are geographically spread to assure heterogeneous teams and to promote international cooperation. This paradigm can be applied in any project/internship course unit. The results from the first edition are very encouraging supporting our initial hypothesis that MUTW significantly promotes students soft skills without requiring any change to prior degree curricula.info:eu-repo/semantics/publishedVersio

Representação de uma identidade cultural - etnografia e arte multimédia

Mestrado em Comunicação Multimédia“Je est un autre” é uma instalação audiovisual, cujo objectivo é representar uma identidade cultural, através da arte multimédia. Para a execução desta obra, eu pergunto: Como pode ser representada uma identidade? Esta dissertação levanta os conceitos teóricos chave que sustentam a criação da instalação, discutindo a definição de identidade, e o modo como esta identidade pode ser traduzida numa criação artística. Esta dissertação apresenta uma solução conceptual para a representação de uma identidade colectiva, através da refleção da identidade individual e de técnicas para a desconstrução das estruturas narrativas clássicas. A dissertação estabelece um processo investigativo com base em processos de “Material Thinking”, pensar sobre os materiais utilizados no processo criativo. Este trabalho toma em consideração as relações entre linguagem e interpretação da realidade, através das contribuições de Derrida e semiótica.“Je est un autre” is an interactive installation, aiming to represent cultural identity through multimedia art. In this dissertation I ask: How can we represent identity? The present text raises the key theoretical concepts that sustain this artistic creation, discussing the definition of identity, and how this identity may be translated into artistic creation. The dissertation establishes a research process, based on “Material Thinking”, a method for rationalizing over the materials used in the creative process. This dissertation presents a conceptual proto-methodological solution for representing collective identity, through a reflection of individual identity and presenting a solution for the use of structure against itself. It takes into account the different actors that take part in the representation process and the relationship between language and reality, through the contributions of Derrida and Semiotics

Virtual sign translator

In this paper authors present the overall study that includes the model developed (VS Model) and the experiences performed, with an automatic bidirectional sign language translator, between written and sign language, which is being supervised by the research group GILT (Graphics, interaction & learning technologies) under the frame of a national project called Virtual Sign (VS project). This project aims to develop and evaluate a model that facilitates access for the deaf and hearing impaired to digital content - in particular the educational content and learning objects - creating the conditions for greater social inclusion of deaf and hearing impaired people. Access to digital content will be supported by an automatic translator between Portuguese Writing (LEP) and Portuguese Sign Language (LGP) supported by an interaction model.FCT - Projeto VirtualSign - Ref. PTDC/CPE-CED/121878/201

Using games to make the process of learning sign language enjoyable and interactive

Conferência realizada em Wellington, na Nova Zelândia, de 8-10 de dezembro de 2014The work presented in this paper consists in the development of a game to make the process of learning sign language enjoyable and interactive. In this game the player controls a character that interacts with various objects and non-player characters with the aim of collecting several gestures from the Portuguese Sign Language. This interaction is supported by data gloves and Kinect. These gestures can then be represented by the character. This allows the user to visualize and learn or train the various existing gestures. To improve the interactivity and to make the game more interesting and motivating, several checkpoints were placed along game levels. This will provide the players a chance to test the knowledge they have acquired so far on the checkpoints by performing the signs using Kinect. A High Scores system was also created as well as a history to ensure that the game is a continuous motivating process as well as a learning process

Virtual sign : a real time bidirectional translator of portuguese sign language

Promoting equity, equal opportunities to all and social inclusion of people with disabilities is a concern of modern societies at large and a key topic in the agenda of European Higher Education. Despite all the progress, we cannot ignore the fact that the conditions provided by the society for the deaf are still far from being perfect. The communication with deaf by means of written text is not as efficient as it might seem at first. In fact, there is a very deep gap between sign language and spoken/written language. The vocabulary, the sentence construction and the grammatical rules are quite different among these two worlds. These facts bring significant difficulties in reading and understanding the meaning of text for deaf people and, on the other hand, make it quite difficult for people with no hearing disabilities to understand sign language. The deployment of tools to assist the daily communication, in schools, in public services, in museums and other, between deaf people and the rest may be a significant contribution to the social inclusion of the deaf community. The work described in this paper addresses the development of a bidirectional translator between Portuguese Sign Language and Portuguese text. The translator from sign language to text resorts to two devices, namely the Microsoft Kinect and 5DT Sensor Gloves in order to gather data about the motion and shape of the hands. The hands configurations are classified using Support Vector Machines. The classification of the movement and orientation of the hands are achieved through the use of Dynamic Time Warping algorithm. The translator exhibits a precision higher than 90%. In the other direction, the translation of Portuguese text to Portuguese Sign Language is supported by a 3D avatar which interprets the entered text and performs the corresponding animations

Virtual sign: using a bidirectional translator in serious games

The work presented in this paper is one of the outcomes of the virtual sign project that aims to assist the communication with deaf students in the classroom. The project is being developed by Portuguese researchers. The main goal of the virtual sign project is the creation of a bidirectional Portuguese sign language translator. This translator supports the development of a serious game that was developed to facilitate the sign language learning process. In the game, it is possible to gather different gestures and reproduce them. The game includes three different scenarios. These scenarios are linked to increasing levels of proficiency in sign language. The first one addresses the alphabet, the second one goes a step forward to teach words, and the third one introduces full sentences. The game experience can be enhanced by using the Kinect to perform the gestures. This game intends to be of great assistance in the process of learning the Portuguese sign language.This work was supported by Engineering Institute of Oporto and GILT (Graphics, Interaction and Learning Technologies) under Grant Nos. 60973089. FCT (Fundação para a ciência e Tecnologia) project.info:eu-repo/semantics/publishedVersio

Game design and the gamification of content : assessing a project for learning sign language

Comunicação apresentada na EDULEARN 2015, realizada em Barcelona de 6-8 de julho de 2015This paper discusses the concepts of game design and gamification of content, based on the development of a serious game aimed at making the process of learning sign language enjoyable and interactive. In this game the player controls a character that interacts with various objects and non- player characters, with the aim of collecting several gestures from the Portuguese Sign Language corpus. The learning model used pushes forward the concept of gamification as a learning process valued by students and teachers alike, and illustrates how it may be used as a personalized device for amplifying learning. Our goal is to provide a new methodology to involve students and general public in learning specific subjects using a ludic, participatory and interactive approach supported by ICT- based tools. Thus, in this paper we argue that perhaps some education processes could be improved by adding the gaming factor through technologies that are able to involve students in a way that is more physical (e.g. using Kinect and sensor gloves), so learning becomes more intense and memorable