Shamanic interfaces for computers and gaming platforms

Simpósio de informática, realizado no Porto de 4-5 de setembro de 2014Natural user interfaces are becoming widespread as a focus of research in human-computer interaction. Gestural interaction is an important part of this field, but generally done by mimicry. This raises concerns such as the necessity of creating abstractions for non-imitable commands and the difficulty of finding gestures that are meaningful for a worldwide audience. Cultural backgrounds impart different meanings to gestures. In this research , we explore the concept of allowing individuals to interact with computer systems using gesture from the individual’s own culture, focusing on a software engineering approach to support this idea. The aim is to leverage the rich semantics of non-mimicry cultural gestures to expand gestural interaction to support abstract commands for instructions that do not have a matching gestural imitation. This approach also holds the potential to support the learning of gestural commands, by linking them to the cultural background of each user. The proposed software engineering approach demonstrates the feasibility of planning applications with commands in mind, not specific gestures, separating concerns between gestural identification (which can include cultural background elements) and actual commands

Separating gesture detection and application control concerns with a multimodal architecture

Comunicação apresentada na conferência internacional - CIT 2015, realizada de 26-28 de outubro de 2015Gesture-controlled applications typically are tied to specific gestures, and also tied to specific recognition methods and specific gesture-detection devices. We propose a concernseparation architecture, which mediates the following concerns: gesture acquisition; gesture recognition; and gestural control. It enables application developers to respond to gesture-independent commands, recognized using plug-in gesture-recognition modules that process gesture data via both device-dependent and deviceindependent data formats and callbacks. Its feasibility is demonstrated with a sample implementation

eXtended new reality

This article addresses some aspects of the potential of Extended Reality (XR) technologies in the context of the accelerated ongoing digital transformation, with public awareness and wider acceptance being prompted by current pandemic, due to the widespread adoption of teleworking, distance learning, and virtual conferences. The state-of-the-art of XR technologies and immersive environments is briefly addressed from the perspective of their sustained adoption in multiple scenarios, including education and training, well-being and active aging, and business.info:eu-repo/semantics/publishedVersio

Avaliação com videojogos para interpretar conhecimento no contexto de vivências e decisões

Apesar do valor em si que um aluno pode vivenciar com algo dinâmico e interativo como um videojogo, esta aplicação é um desaproveitamento das potencialidades do meio. De facto, numa visita de estudo não é possível parar o tempo ou congelar o espaço, pela necessidade de desfrutar da experiência ao vivo, enquanto ela está a decorrer; mas um videojogo pode ser repetido, pode ser registado na sua plenitude, pode ser planeado como experiência social, grupal ou individual. Neste texto reflete-se sobre este potencial e como a sua concretização pode permitir maior disseminação do próprio uso educativo dos videojogos e de novas formas de avaliação das aprendizagens por recurso a eles.info:eu-repo/semantics/publishedVersio

An Approach of a Research Tool based in a Shamanic Interface

Hoje em dia os sistema de reconhecimento de gestos são desenvolvidos para diversos fins, como entretenimento, segurança ou educação. Estas interfaces são controladas através de gestos realizados por um utilizador, e estes gestos não estão só associados a um comando no sistema, mas também têm um significado intrínseco para o utilizador.Uma vez que estes gestos têm diferentes significados para os utilizadores com base nas suas culturas, esta tese adota a utilização de uma interface xamânica. A interface xamânica considera a cultura da pessoa para uma interação gestual. De um ponto de vista da arquitectura do software, a interface xamânica usa uma camada cultural que considera a cultura do utilizador quando liga gestos a um certo comando. Isto suporta diferentes utilizadores experienciarem com a mesma interface o uso de gestos ligados a sua cultura.Sendo uma proposta recente, a interface xamânica carece de testes e validações empíricas, assim esta tese contribui com a criação de uma ferramenta que permite uma pesquisa empírica do conceito. Após o desenvolvimento da camada cultural, foi implementado e avaliado num jogo para testar a sua performance e aplicabilidade. Para o reconhecimento de gestos, a camada cultural depende de um Leap Motion Controller. Um Leap Motion Controller consegue rastrear gestos com alguma precisão, sem necessitar que o utilizador utilize um dispositivo incómodo. Para reconhecer e classificar gestos, foram usados Modelos Ocultos de Markov e um respectivo classificador.Ao desenvolver um jogo com uma interface xamânicas e realizando alguns testes de usabilidade a utilizadores com diferentes culturas, foi possível concluir que esta ferramenta pode ser usada por investigadores para testar e desenvolver o conceito. Interfaces xamânicas são promissoras, mas ainda é necessário ser realizado um trabalho mais aprofundado na área de reconhecimento gestual como também na definição de gestos culturais significativos.Nowadays gesture recognition systems are developed for several purposes, e.g. entertainment, security or education. Such interfaces are controlled by gestures performed by a user, and these gestures are not only associated with a command in the system but also have an intrinsic meaning to the user. Since gestures can have different meanings for users based on their cultural background, this thesis adopted the perspective of using a shamanic interface. A shamanic interface considers a person's cultural background for gesture-based interaction. From a software architecture perspective, shamanic interfaces use a cultural layer that considers a user's culture when linking a gesture to a certain command. This supports different user experiences with the same interface using gestures linked to a user's culture. Being a recent proposal, the shamanic interface lacks empirical testing and validation, hence this thesis' contribution is the creation of a research tool to enable empirical research of the concept. After developing a cultural layer, a game was implemented and evaluated to test its performance and applicability. For recognising gestures, the cultural layer relies on a Leap Motion Controller. A Leap Motion Controller can track gestures with a certain precision without requiring the user to wear cumbersome tracking devices. For recognising and classifying gestures, Hidden Markov Models and a respective classifier were used. By developing a game with a shamanic interface and performing usability tests with users with different cultural backgrounds, it can be concluded that this research tool can be used by empirical researchers to test and develop the concept. Shamanic interfaces are promising, but further work needs to be done on the gesture recognition area as well as in the definition of meaningful cultural gestures

Desenvolvimento de interfaces multimodais para jogos de estratégia

As interfaces multimodais têm tido um grande incremento ao nível dos jogos digitais, pelas mecânicas e dinâmicas inovadoras que possibilitam. No entanto, o seu desenvolvimento é complexo. A título de exemplo, nem todos os jogos que utilizam o Kinect exploram as suas capacidades na totalidade. Como tal, nesta dissertação apresenta-se uma solução baseada numa interface gestual para a exploração de jogos de estratégia, que possibilita a criação de mecânicas e dinâmicas inovadoras.Para validar a solução apresentada fez-se um estudo de caso, sobre o protótipo de um novo jogo de estratégia baseado na História de Portugal. Constatou-se que com esta solução é possível tornar a interção jogador-computador mais intuitiva e dinâmica. São exemplos, a utilização dos gestos de ambas as mãos para movimentar o mapa, seleccionar unidades, aumentar ou diminuir o zoom, entre outras.Multimodal Interfaces is having a great increase when it comes to digital games, because of the innovative mechanics and dynamics that they allow. However, its development is complex. For example, not all the games that use Kinect explore its capabilities in total.As such, in the dissertation a solution based in a gestural interface for the exploration of strategy games, which allows the creation of new mechanics and dynamics, will be presented.In order to validate the presented solution, a study case about the prototype of a new strategy game based on the History of Portugal has been done. It was possible to conclude that with this solution it is possible to make the gamer-computer interaction more intuitive and dynamic. For instance, the use of gestures of both hands to move the map, select units, increase or decrease zoom, amongst other functionalities

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

Adult Learning Sign Language by combining video, interactivity and play

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with or a disorder they acquired during their life. 900,000 people in the UK are severely or profoundly deaf and based on a study by Action On Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress, and an even larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a theoretical framework for the design of interactive games to support learning BSL supporting the entire learning cycle, instruction, practice and assessment. It then describes the proposed design of a game based on this framework aiming to close the communication gap between able hearing people and people with a hearing impediment, by providing a tool that facilitates BSL learning targeting adult population. The paper concludes with the planning of a large scale study and directions for further development of this educational resource