Sistemi immersivi per contesti sociali: come progettare e sviluppare nuovi tipi di esperienze ed interazioni

In questo lavoro verr\ue0 affrontato e discusso il concetto di esperienza immersiva e delle tematiche ad esso collegate con l\u2019obiettivo finale di individuare e descrivere i principali fattori da tenere in considerazione durante la progettazione e lo sviluppo di artefatti digitali immersivi in contesti sociali. Nella prima parte del lavoro verranno quindi introdotti i concetti di immersivit\ue0, le tecnologie che consentono la creazione della sensazione di immersione e i sensi che vengono stimolati. La seconda parte del lavoro introduce invece quella che \ue8 una classificazione dei contesti in cui \ue8 possibile fruire questo tipo di esperienze, permettendo quindi di evidenziare possibili differenze tra le necessit\ue0, i vincoli e gli agenti presenti all\u2019interno di ciascuno. La combinazione e l\u2019applicazione dei primi due argomenti ha portato a dover affrontare e definire il concetto di esperienza immersiva, le teorie e i concetti importanti da seguire per una sua buona realizzazione. Durante l\u2019argomentazione delle tematiche relative al concetto di esperienza immersiva si \ue8 arrivati a delineare quelli che sono gli elementi caratterizzanti di questo tipo di esperienze (social contexts, environment, content, fruition, people relation, people disabilities, involved display, tracking e outcomes) e che si vanno a riflettere su quattro variabili principali da tenere in considerazione durante la progettazione, lo sviluppo e l\u2019analisi. Queste riflessioni hanno portato alla concezione di un modello che si basa sulla descrizione delle informazioni legate al contesto in cui l\u2019esperienza viene fruita (Context), la tipologia di utenti coinvolti (Users), la tecnologia necessarie per lo sviluppo (Technology), e le competenze necessarie per la sua realizzazione (Expertise). Tale modello, abbreviato in CUTE (Context, Users, Technology, Expertise), \ue8 stato poi quindi utilizzato per la progettazione e lo sviluppo delle esperienze immersive realizzate durante l\u2019intero percorso di dottorato oltre che per l\u2019analisi di progetti esistenti a cui si \ue8 contribuito, permettendo di evidenziare con facilit\ue0 le alternative che i singoli elementi sono in grado assumere e consentendo l\u2019identificazione di bisogni, requisiti e problematiche in maniera agile per ciascuna delle singole variabili. Le singole esperienze hanno riguardato lo sviluppo di funzionalit\ue0 aggiuntive per un simulatore di operazioni videolaparoscopiche; la progettazione e sviluppo di un prototipo di simulatore di parto; lo sviluppo di un\u2019esperienza di realt\ue0 aumentata in contesto education per l\u2019apprendimento; la creazione di un virtual coach per il recupero motorio; un survey sulle possibili alternative d\u2019uso di Natural User Interfaces su dispositivi mobile; uno studio su un\u2019applicazione per la fruizione di luoghi non accessibili a causa di limiti strutturali degli ambienti o fisici/cognitivi degli utenti; la realizzazione di un prototipo immersivo per la progettazione di spazi dedicati a eventi pubblici; lo sviluppo di un\u2019applicazione dedicata al immersive journalism; ed infine lo studio di un sistema intelligente per lo smaltimento dei rifiuti. Nelle conclusioni verranno i ripresi i concetti introdotti nelle varie sezioni e che insieme ai risultati emersi dalle singole esperienze permetteranno di fare delle riflessioni sulla readiness, cio\ue8 il livello di prontezza operativa, dei singoli contesti rispetto all\u2019introduzione delle tecnologie immersive oltre che delle considerazioni sull\u2019applicabilit\ue0 e le possibili evoluzioni del modello CUTE sviluppato.This work will analyze the concept of \u201cimmersive experience\u201d and notions related to it, with the final aim to identify and describe the key elements to consider when designing and developing immersive digital artifacts in social contexts. The first part of the dissertation introduces the concepts of immersiveness and immersion, with a specific focus on technologies able to stimulate various human senses, thus enabling a perception called \u201csense of immersion\u201d. Then, I will classify the possible social contexts in which an immersive experience can happen, highlighting the differences in terms of needs, constraints and agents involved. The combination of the knowledge belonging to immersion and social contexts leads to a deeper concept of immersive experience with theories and best practices useful for its development. This insight allowed the categorization of nine key elements that shape an immersive experience: social contexts, environment, content, fruition, people relation, people disabilities, involved display, tracking and outcomes. Such elements converge in four different variables to consider during the stages of analysis, design and development. Such variables refer to the information related to the context in which the experience occurs, the kind of users involved, the technologies and expertise required to complete the development. Starting from these considerations, I developed a model named CUTE (Context, Users, Technologies, Expertise) that can be used to analyze existing products and develop new projects. The model has been used for all the immersive experiences I realized during these years of PhD. In particular, the development of additional features for a videolaparoscopic simulator; the design and development of a birth simulator; the development of an augmented reality experience in educational context; a survey on the possible alternatives to enable Natural User Interfaces on mobile devices; the creation of a virtual coach for motor recovery; the development of an application to visit inaccessible places; the creation of an immersive solution for the design of public spaces and events; the development of an application dedicated to immersive journalism; the study of an intelligent system for waste disposal. Altogether the concepts introduced in the various sections and the results of single experiences gives hints about the readiness of the individual contexts regarding the introduction of immersive technologies as well as considerations on the applicability and possible evolutions of the CUTE model developed

Towards a virtual coach for boccia: developing a virtual augmented interaction based on a boccia simulator

Science and Technology Publications, Lda. All rights reserved. Disability can be a factor that leads to social exclusion. Considering that involvement in society is paramount for a person with disability, participation in sports can be a powerful tool for inclusion. Based on this premise, the authors propose an intelligent virtual coach for Boccia to encourage the practice of this sport on persons with disabilities, while promoting social inclusion and shortening the learning curve for individuals new to the sport by learning about game strategy. The envisioned virtual coach will rely on Artificial Intelligence models, thus requiring the creation of large datasets, namely for ball placement and throwing movement recommendations. To answer these problems, this work is focused on the development of a Boccia simulator. With this simulator, it is possible to generate artificial gameplay images and allow the user to control an avatar with body tracking. Gesture recognition was implemented with a state-machine, thus enabling the player to throw the ball, with customizable physics, by performing one of two different throwing movements. This functionality can allow the recording of data describing the body movement associated with the placement of the ball in a certain position within the virtual court, which is essential for the proposed recommendation system.FCT - Fundação para a Ciência e a Tecnologia(690874).This article is supported by the project Deus ex Machina: NORTE – 01 – 0145 – FEDER - 000026, supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF) and by national funds through FCT – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2019. Vinicius Silva also thanks FCT for the PhD scholarship SFRH/BD/133314/2017

Enabling touchless interfaces for mobile platform: State of the art and future trends

In this work, we discuss the existing methods to implement and use a touchless gesture-based interaction on mobile platforms such as smartphones and tablets, specifically for situations where you cannot have a direct interaction with devices. The purpose of this work is to investigate the actual solutions and establish the most convenient way to enable these interactions on mobile devices with minimally-invasive procedures. The paper is organized in three parts: a survey of the current technological possibilities; a description of trials made to know advantages and limitations of the existing solutions; a final discussion concerning the evolution of the natural user interface on mobile platforms in the next few years

eBSim: Development of a low-cost obstetric simulator

This work presents the design a low cost obstetric simulator allowing precise identification of the fetal position in a simulated birth canal. The system consists on a female pelvis, a custom-made fetal mannequin, a virtual 3d representation, a visual display and an user interface to show in real time what happens inside the birth canal. Students are often unable not only to identify correctly the fetal head position, but also to discriminate between the two fontanels, for this reason the simulator eBSim can help them to train this ability and could be an important instrument for the instructors to objectively assess the clinical skill of each student

PUPPET MENTORING: A NEW SIMULATION SCENARIO FOR LEARNING SURGICAL ABILITIES

Surgical simulators are now able to teach in a way that the learning curve of young surgeons can progress in a lab faster than when using other teaching models (cadaveric or animal) or real patients. The impact of surgical simulators is confirmed by the fact that, in the US, standardized training courses are needed to acquire the Board of Surgery certification. The virtual simulator set up at the University of Genoa (eLaparo4D) is based on two key features: a convincing haptic feedback and a limited cost. Nevertheless, the main issue of eLaparo4D is the \u201csimplicity\u201d of the virtual scenario. To improve it, a new model of simulation is proposed in this project: the \u201cpuppet mentoring\u201d, that might enhance its characteristics. The \u201cpuppet mentoring\u201d is based on the recording of the movements of the surgeon in the real clinical scenario, that are transferred to the virtual machine. The apprentice, in his learning session, could be led through the operation by the simulator itself, in a scenario and in a way is the same of the real one