Open Medical Gesture: An Open-Source Experiment in Naturalistic Physical Interactions for Mixed and Virtual Reality Simulations

Mixed Reality (MR) and Virtual Reality (VR) simulations are hampered by requirements for hand controllers or attempts to perseverate in use of two-dimensional computer interface paradigms from the 1980s. From our efforts to produce more naturalistic interactions for combat medic training for the military, USC has developed an open-source toolkit that enables direct hand controlled responsive interactions that is sensor independent and can function with depth sensing cameras, webcams or sensory gloves. Natural approaches we have examined include the ability to manipulate virtual smart objects in a similar manner to how they are used in the real world. From this research and review of current literature, we have discerned several best approaches for hand-based human computer interactions which provide intuitive, responsive, useful, and low frustration experiences for VR users.Comment: AHFE 202

Multi-Level Representation of Gesture as Command for Human Computer Interaction

oai:ojs.cai.ui.sav.sk:article/16The paper addresses the multiple forms of representation that human gesture takes at different levels for human computer interaction, ranging from gesture acquisition to mathematical model for analysis, pattern for recognition, record for database up to end-level application event triggers. A mathematical model for gesture as command is presented. We equally identify and provide particular models for four different types of gestures by considering both posture information and underlying motion trajectories. The problem of constructing gesture dictionaries is further addressed by taking into account similarity measures and dictionary discriminative features

User-Defined Gestural Interaction: a Study on Gesture Memorization

8 pagesInternational audienceIn this paper we study the memorization of user created gestures for 3DUI. Wide public applications mostly use standardized gestures for interactions with simple contents. This work is motivated by two application cases for which a standardized approach is not possible and thus user specific or dedicated interfaces are needed. The first one is applications for people with limited sensory-motor abilities for whom generic interaction methods may not be adapted. The second one is creative arts applications, for which gesture freedom is part of the creative process. In this work, users are asked to create gestures for a set of tasks, in a specific phase, prior to using the system. We propose a user study to explore the question of gesture memorization. Gestures are recorded and recognized with a Hidden Markov Model. Results show that it seems difficult to recall more than two abstract gestures. Affordances strongly improve memorization whereas the use of colocalization has no significant effect

3D HAPTIC INTERFACES SIMULATION FOR RECOVERY OF CULTURAL HERITAGE TRAIN IN RIOBAMBA

[EN] The present work had studied and determined various levels of usability and interactivity in haptic interfaces for manipulating 3D applications. It was applied to a Heritage Cultural system composed by Train Station and Steam Locomotive. A physical model and a railroad model were built. The system was operated by high school students. The information gathering was performed by the deductive method, collecting data about preferences, needs, and requirements for developing a 3D virtual environment using 3DSMAX 2011 Trial (3D Modeled Software) and Unity 4.3 Trial (Simulation Software), capable to engage and transmit information relevant to geographical, social and cultural aspects of the central highlands. Subsequently connected to a real model controlled by electronic systems. The inquiry made from high school students shows that 56 % possess ability and willingness for interfaces manipulation. Also, it was found an 80% of criteria for usability acceptance, and interactivity of 84% towards the current applications on mobile devices. As the final evaluation of the application, it was shown that the intervention of haptic interfaces on threedimensional applications connected to tangible environments, increased 95% of the usability and 100% of interactivity. We conclude that combining virtual systems with real ones accelerates learning processes at the secondary education level. It improves retention of information and allows the user to actively participate in virtual environments, converting recreational applications into academic and research tools.[ES] El presente trabajo estudió y determino diferentes El presente trabajo estudió y determino diferentes niveles de usabilidad e interactividad en interfaces hápticas para manipulación de aplicaciones 3D. Se aplicó a un Patrimonio Cultural que comprende la Estación del tren y una locomotora a vapor, los mismos que fueron representados en una maqueta física y un sistema de trenes eléctricos respectivamente. Este sistema fue operado por estudiantes secundarios. La obtención de información se realizó mediante el método deductivo, recolectando datos sobre necesidades y requerimientos para elaborar un entorno virtual 3D utilizando 3DSMAX 2011 Trial (Software Modelado 3D) y Unity 4.3 Trial (Software Simulación), capaz de vincularse y transmitir información relevante sobre aspectos geográficos, sociales y culturales de la serranía central. Posteriormente conectada a una maqueta real controlada por sistemas electrónicos. La indagación realizada a estudiantes, muestra que el 56% poseen capacidad y disposición de manipulación de interfaces, también se encontró aceptación del 80% en criterios de usabilidad y 84% en interactividad hacia aplicaciones actuales en dispositivos móviles, como evaluación final a la aplicación se demostró que la intervención de interfaces hápticas en aplicaciones tridimensionales conectadas a entornos tangibles, incrementaron la usabilidad al 95% y la interactividad al 100%. Se concluye que combinar sistemas virtuales con reales acelera procesos de aprendizaje a nivel de educación secundaria, mejora la retención de información y permite al usuario participar activamente con entornos virtuales, convirtiendo aplicaciones lúdicas en herramientas académicas e investigativas.Duque, MA.; Proaño, F.; Santos, R. (2016). SIMULACIÓN 3D CON INTERFACES HÁPTICAS PARA LA RECUPERACION DEL PATRIMONIO CULTURAL DEL TREN EN RIOBAMBA. En 8th International congress on archaeology, computer graphics, cultural heritage and innovation. Editorial Universitat Politècnica de València. 191-196. https://doi.org/10.4995/arqueologica8.2015.3564OCS19119

La usabilidad de las interfaces basadas en visión

Las interfaces basadas en visión hacen uso de gestos para la comunicación del usuario con el sistema interactivo sin la necesidad de dispositivos que requieran contacto físico. El modelado de gestos suele realizarse en laboratorios y es importante que se lleve a cabo la evaluación de su usabilidad, pero la gran diversidad de usos y aplicaciones de estas interfaces hace que resulte difícil decidir qué factores tener en cuenta cuando se mide la usabilidad. En este artículo se presenta una revisión de la literatura cuyo objetivo es clasificar i recopilar los factores y las métricas de usabilidad que se utilizan para validar las interfaces basadas en visión.Peer ReviewedPostprint (published version

Interfaces baseadas em gestos e movimento

Dissertação de Mestrado em Engenharia InformáticaEsta tese estuda novas formas de interacção pessoa-máquina, baseadas em sensores de infravermelhos. O objectivo foi criar uma interface que tornasse a interacção com o computador mais natural e divertida, utilizando gestos e movimentos que são usados intuitivamente no dia-a-dia. Foi necessário o desenho e implementação de um sistema flexível e modular, que permite detectar as posições e movimentos das mãos e cabeça do utilizador. Adicionalmente, esta interface tambem permite a utilização de botões e fornece feedback háptico ao utilizador. Foram encontrados vários problemas durante a realização do hardware, que levaram à utilização de novas abordagens e à construcção e teste de vários protótipos Paralelamente à construção dos protótipos do hardware, foi implementada uma biblioteca que permite detectar a posição das mãos e cabeça cabeça do utilizador, num espaço tridimensional. Esta biblioteca trata de toda a comunicação com o hardware, fornecendo funções e callbacks simples ao programador das aplicações. Foram desenvolvidas quatro aplicações que permitiram testar e demonstrar as várias funcionalidades desta interface em diferentes cenários. Uma destas aplicações foi um jogo, que foi demonstrado publicamente durante o dia aberto da FCT/UNL, tendo sido experimentado e avaliado por um grande número de utilizadores

Using the functional resonance analysis method (FRAN) to model and analyze lifeboat training in a simulator

Lifeboat operation is a complex procedure in which safety and rescue are of utmost importance. Training coxswain to perform these operations and acquiring sufficient skills and competencies to face unforeseen risks in harsh weather conditions is challenging. However, lifeboat simulators facilitate the training by removing the risk of training in real environments and improving training courses and trainees' skills. In this study, a Functional Resonance Analysis Method (FRAM) model for launching a lifeboat and on-water tasks was created based on the approved lifeboat training course materials, rubric grading, and lifeboat course scenarios. Two scenarios were used to identify some essential functions in a lifeboat operation. Launch a lifeboat, get away from the platform and drive to a safe zone, pick up Person In Waters (PIWs), recover people from the life raft, tow a life raft, stop by a vessel and transfer the PIW are some tasks covered in this FRAM model. The model was tested with the simulator to identify variabilities in terms of accuracy and time. Five volunteers were asked to perform these scenarios. FRAM signatures of different performances were created to visualize various ways of doing an operation. Successful and unsuccessful operations were monitored using the FRAM, and key elements to having successful and unsuccessful outcomes were determined. Identifying functions and their variations helped to discover where and how trainees act differently in the lifeboat operation. The results of building the FRAM model showed that four categories of functions contributed to lifeboat operation, including action, assessment, decision-making, and skill. The comprehensive model presented in this study enables the researcher to better understand lifeboat operations and helps identify the variations that can affect an operation. Effective processes and key features to diagnose acceptable vs. unacceptable performance extracted by FRAM can be considered a perfect source of observational learning to inform trainees. The FRAM approach used in this study can be employed to determine work practices that are more or less effective, allowing for the adaptation of processes and techniques to steer lifeboat training in the direction of routes that would provide better results