Natural user interfaces for interdisciplinary design review using the Microsoft Kinect

As markets demand engineered products faster, waiting on the cyclical design processes of the past is not an option. Instead, industry is turning to concurrent design and interdisciplinary teams. When these teams collaborate, engineering CAD tools play a vital role in conceptualizing and validating designs. These tools require significant user investment to master, due to challenging interfaces and an overabundance of features. These challenges often prohibit team members from using these tools for exploring designs. This work presents a method allowing users to interact with a design using intuitive gestures and head tracking, all while keeping the model in a CAD format. Specifically, Siemens\u27 Teamcenter® Lifecycle Visualization Mockup (Mockup) was used to display design geometry while modifications were made through a set of gestures captured by a Microsoft KinectTM in real time. This proof of concept program allowed a user to rotate the scene, activate Mockup\u27s immersive menu, move the immersive wand, and manipulate the view based on head position. This work also evaluates gesture usability and task completion time for this proof of concept system. A cognitive model evaluation method was used to evaluate the premise that gesture-based user interfaces are easier to use and learn with regards to time than a traditional mouse and keyboard interface. Using a cognitive model analysis tool allowed the rapid testing of interaction concepts without the significant overhead of user studies and full development cycles. The analysis demonstrated that using the KinectTM is a feasible interaction mode for CAD/CAE programs. In addition, the analysis pointed out limitations in the gesture interfaces ability to compete time wise with easily accessible customizable menu options

Virtual Meeting Rooms: From Observation to Simulation

Much working time is spent in meetings and, as a consequence, meetings have become the subject of multidisciplinary research. Virtual Meeting Rooms (VMRs) are 3D virtual replicas of meeting rooms, where various modalities such as speech, gaze, distance, gestures and facial expressions can be controlled. This allows VMRs to be used to improve remote meeting participation, to visualize multimedia data and as an instrument for research into social interaction in meetings. This paper describes how these three uses can be realized in a VMR. We describe the process from observation through annotation to simulation and a model that describes the relations between the annotated features of verbal and non-verbal conversational behavior.\ud As an example of social perception research in the VMR, we describe an experiment to assess human observers’ accuracy for head orientation

Application-driven visual computing towards industry 4.0 2018

245 p.La Tesis recoge contribuciones en tres campos: 1. Agentes Virtuales Interactivos: autónomos, modulares, escalables, ubicuos y atractivos para el usuario. Estos IVA pueden interactuar con los usuarios de manera natural.2. Entornos de RV/RA Inmersivos: RV en la planificación de la producción, el diseño de producto, la simulación de procesos, pruebas y verificación. El Operario Virtual muestra cómo la RV y los Co-bots pueden trabajar en un entorno seguro. En el Operario Aumentado la RA muestra información relevante al trabajador de una manera no intrusiva. 3. Gestión Interactiva de Modelos 3D: gestión online y visualización de modelos CAD multimedia, mediante conversión automática de modelos CAD a la Web. La tecnología Web3D permite la visualización e interacción de estos modelos en dispositivos móviles de baja potencia.Además, estas contribuciones han permitido analizar los desafíos presentados por Industry 4.0. La tesis ha contribuido a proporcionar una prueba de concepto para algunos de esos desafíos: en factores humanos, simulación, visualización e integración de modelos

Implementing intelligent pedagogical agents in virtual worlds: Tutoring natural science experiments in OpenWonderland

Intelligent Pedagogical Agents (IPAs) can be thought of as embodied intelligent agents that are designed for pedagogical purposes to support learning. They can be designed in particular for virtual worlds. Virtual worlds are becoming an interesting medium for engineering education for the properties of visual collaboration abilities providing authentic learning experiences and for the opportunity of providing active learning. However, virtual worlds need more educational support to be more inhabited with increased learning services. Incorporating intelligent pedagogical agents into virtual worlds adds such learning support by adding intelligence, improving believability, and the opportunity to increase communication with an artificial educator. However the implementation of intelligent pedagogical agents and adopting them in a virtual world require several efforts with different aspects of implementation. This paper reports our first prototype implementation of an IPA interacting with a learner and a learning object in natural science experiment in a virtual world while providing supporting multi-modal communication abilities. The IPA has features of text chat based on the Artificial Intelligence Markup Language (AIML), a text-to-speech synthesis function, and non-verbal communication abilities through gesture animation. The implementation is presented through explained scenarios of the IPA tutoring an experiment or monitoring a learner avatar interaction with a learning object in a Virtual World. The IPA & the learning scenarios are implemented in the open source of Open Wonderland

Supporting Memorization and Problem Solving with Spatial Information Presentations in Virtual Environments

While it has been suggested that immersive virtual environments could provide benefits for educational applications, few studies have formally evaluated how the enhanced perceptual displays of such systems might improve learning. Using simplified memorization and problem-solving tasks as representative approximations of more advanced types of learning, we are investigating the effects of providing supplemental spatial information on the performance of learning-based activities within virtual environments. We performed two experiments to investigate whether users can take advantage of a spatial information presentation to improve performance on cognitive processing activities. In both experiments, information was presented either directly in front of the participant or wrapped around the participant along the walls of a surround display. In our first experiment, we found that the spatial presentation caused better performance on a memorization and recall task. To investigate whether the advantages of spatial information presentation extend beyond memorization to higher level cognitive activities, our second experiment employed a puzzle-like task that required critical thinking using the presented information. The results indicate that no performance improvements or mental workload reductions were gained from the spatial presentation method compared to a non-spatial layout for our problem-solving task. The results of these two experiments suggest that supplemental spatial information can support performance improvements for cognitive processing and learning-based activities, but its effectiveness is dependent on the nature of the task and a meaningful use of space