Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Proceedings of the International Workshop “Re-Thinking Technology in Museums: towards a new understanding of people’s experience in museums"

Proceedings of the International Workshop “Re-Thinking Technology in Museums: towards a new understanding of people’s experience in museums

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room

NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas

In Gameplay : the invariant structures and varieties of the video game gameplay experience

This dissertation is a multidisciplinary study on video game gameplay as an autonomous form of vernacular experience. Plays and games are traditional research subjects in folkloristics, but commercial video games have not been studied yet. For this reason, methods and concepts of the folkloristic research tradition have remained unknown in contemporary games studies. This thesis combines folkloristics, game studies and phenomenological enactive cognitive science in its investigations into player–game interaction and the video game gameplay experience at large. In this dissertation, three representative survey samples (N=2,594, N=845, N=1,053) on “Rewarding gameplay experience” are analyzed using statistical analysis methods. The samples were collected in 2014–2017 from Finnish and Danish adult populations. This dissertation also analyzes data from 32 interviews, through which the survey respondents’ gameplay preferences, gaming memories, and motivations to play were further investigated. By combining statistical and qualitative data analyses, this work puts forward a mixed-methods research strategy and discusses how the findings relate to prior game research from several disciplines and schools of thought. Based on theoretical discussions, this dissertation argues that the video game gameplay experience as a cultural phenomenon consists of eight invariants in relation to which each individual gameplay experience can be interpreted: The player must demonstrate a lusory attitude (i), and a motivation to play (ii). The gameplay experience consists of explorative and coordinative practices (iii), which engender a change in the player’s self-experience (iv). This change renders the gameplay experience inherently emotional (v) and performative (vi) in relation to the gameworld (vii). The gameplay experience has the dramatic structure of a prototypical narrative (viii) although a game as an object cannot be regarded a narrative in itself. As a key result of factor analytical studies and qualitative interview analyses, a novel approach to understanding player–game interaction is put forward. An original gameplay preference research tool and a player typology are introduced. This work argues, that, although video games as commercial products would not be intuitive research subjects for folkloristics, video game gameplay, player–game interaction, and the traditions in experiencing and narrating gameplay do not differ drastically from those of traditional social games. In contrast to this, all forms of gameplay are argued to be manifestations of the same vernacular phenomenon. Indeed, folkloristic research could pay more attention to how culture is experienced, modified, varied and expressed, regardless of whether the research subject is a commercial product or not.Käsillä oleva väitöskirja on monitieteellinen tutkimus videopelien pelaamisesta itsenäisenä kansanomaisen kokemuksen muotona. Pelien ja leikkien tutkimus on perinteikäs tutkimusaihe folkloristiikassa, mutta kaupallisten videopelien tutkimusta ei ole juuri tehty. Tästä syystä folkloristiikan tutkimusmenetelmät ja -käsitteet ovat jääneet tuntemattomaksi nykyaikaisessa pelitutkimuksessa. Tutkimus yhdistää folkloristiikan ja pelitutkimuksen näkökulmien lisäksi enaktiivisen kognition fenomenologista teoriaa pelaaja–peli-vuorovaikutuksen tutkimukseen sekä pelikokemuksen analyysiin. Tutkimuksessa analysoidaan tilastotieteellisin menetelmin kolmea aikuisväestöä edustavaa ”Palkitseva pelikokemus” -kyselytutkimusaineistoa (N=2,594, N=845, N=1,053), jotka kerättiin Suomesta ja Tanskasta vuosina 2014–2017. Kyselytutkimusaineiston rinnalla analysoidaan 32 teemahaastattelun aineistoa. Haastatteluilla tuotettiin syvempää ymmärrystä kyselyyn vastanneiden henkilöiden pelimieltymyksistä, pelimuistoista ja pelimotivaatioista. Tilastoaineiston ja haastatteluaineiston analyysi tuodaan yhteen monimenetelmällisellä ja dialogisella tutkimusotteella, joka yhdistää havainnot usealla eri tutkimusalalla tehtyyn pelitutkimukseen. Teoreettisen analyysin tuloksena argumentoidaan, että videopelien pelikokemusta ilmiönä määrittää kahdeksan muuttumatonta ominaisuutta, joiden suhteen kunkin yksittäisen pelikokemuksen ainutlaatuisuutta voidaan tarkastella: Pelaajalla tulee olla leikkisä asenne (i) ja motivaatio pelaamiseen (ii). Pelaamisen kokemus rakentuu tutkivista ja suorittavista käyntänteistä (iii), jotka tuovat väliaikaisen muutoksen pelaavan henkilön minäkokemukseen (iv). Tämän muutoksen myötä pelaajuudesta muodostuu emotionaalinen (v) ja performatiivinen (vi) positio suhteessa pelimaailmaan (vii). Näin syntyvän omakohtaisen pelikokemuksen rakenne vastaa kertomuksen dramaattista perusrakennetta (viii), vaikka peliä itsessään ei voida pitää kertomuksena. Tutkimuksen empiirisenä tuloksena esitellään faktorianalyyttisiin tapaustutkimuksiin ja laadullisten aineistojen analyysiin perustuva uudenlainen näkökulma ja menetelmä pelaaja–peli-vuorovaikutuksen ja pelimieltymyksen tutkimukseen, sekä edelliseen perustuva pelaajatyyppiluokittelu. Samalla väitetään, että vaikka videopelit kaupallisina esineinä eivät olisi folkloristiikan tutkimuskohteita, videopelien pelaaminen, pelaaja–peli-vuorovaikutus ja pelien kokemisen tavat eivät eroa ratkaisevasti pihaleikeistä vaan ovat saman kansanomaisen ilmiön esiintymiä. Folkloristisen tutkimuksen soisikin kiinnittävän nykyistä painokkaampaa huomiota kulttuurin kokemisen, muokkaamisen ja ilmaisun tapoihin riippumatta siitä, onko tarkastelun kohteena kaupallinen tuote vai ei