Systematic Review of Intelligent Tutoring Systems for Hard Skills Training in Virtual Reality Environments

Advances in immersive virtual reality (I-VR) technology have allowed for the development of I-VR learning environments (I-VRLEs) with increasing fidelity. When coupled with a sufficiently advanced computer tutor agent, such environments can facilitate asynchronous and self-regulated approaches to learning procedural skills in industrial settings. In this study, we performed a systematic review of published solutions involving the use of an intelligent tutoring system (ITS) to support hard skills training in an I-VRLE. For the seven solutions that qualified for the final analysis, we identified the learning context, the implemented system, as well as the perceptual, cognitive, and guidance features of the utilized tutoring agent. Generally, the I-VRLEs emulated realistic work environments or equipment. The solutions featured either embodied or embedded tutor agents. The agents’ perception was primarily based on either learner actions or learner progress. The agents’ guidance actions varied among the solutions, ranging from simple procedural hints to event interjections. Several agents were capable of answering certain specific questions. The cognition of the majority of agents represented variations on branched programming. A central limitation of all the solutions was that none of the reports detailed empirical studies conducted to compare the effectiveness of the developed training and tutoring solutions.Peer reviewe

Investigation of visually induced motion sickness: a comparison of mitigation techniques in real and virtual environments

Motion sickness affects almost all users of virtual reality, and can be a limiting factor in the use of virtual reality environments in applications for training, therapy and entertainment. However, some actions can be taken to reduce the severity of the motion sickness, known as mitigation techniques. One of the mitigation techniques examined in this thesis is an active hand-eye coordination task. The other is passive recovery, by way of removing one\u27s self from the sickening stimuli and allowing time to pass, referred to as natural decay. Both tasks were used in physical reality and virtual reality settings, in order to rank the efficacy of each. The hypothesis was that a virtual mitigation task can be as effective as a physical mitigation task. Forty people participated in a within-subjects experimental design over two visits. Responses on the Simulator Sickness Questionnaire served as the measure for their motion sickness symptom severity. The research found significant differences between the physical and virtual hand-eye tasks, but no significant difference between the physical and virtual natural decay tasks. Further investigation of the differences in the physical and virtual hand-eye tasks is necessary to explain the significant differences; more analysis is required to conclude that natural decay while in a virtual environment is as effective as natural decay in the physical world

Une approche à base de modèles synergiques pour la prise en compte simultanée de l'utilisabilité, la fiabilité et l'opérabilité des systèmes interactifs critiques

Dans le cadre de la conception et du développement de systèmes interactifs critiques, lorsque le coût d'une erreur potentielle d'utilisation ou d'un dysfonctionnement du système peut dépasser le coût de développement de ce système ou se chiffrer en pertes humaines, les techniques, méthodes et processus actuellement proposés dans le domaine de l'IHM sont difficilement exploitables. D'une part, ils ne permettent pas de garantir simultanément les propriétés d'utilisabilité et de sûreté du système développé. D'autre part, la formation et la qualification des utilisateurs du système avant sa mise en opération n'est pas envisagée. Enfin, ces techniques, méthodes et processus ne fournissent pas les moyens de traçabilité exigés pour le développement de systèmes critiques. L'argumentaire de cette thèse s'appuie sur les avantages et limitations des approches existantes en termes de processus et notations de modélisation. Nous proposons une approche et montrons sa réalisation à travers un processus de développement d'un système interactif critique et de son programme de formation associé. Ce processus fournit un cadre conceptuel, une association d'étapes, des notations, et un environnement logiciel pour : le développement d'un système utilisable et sûr, le développement du programme de formation associé ainsi que la traçabilité des exigences et des choix de conception tout au long des différentes étapes. Il utilise certains principes de la conception centrée utilisateur et exploite de manière synergique les modèles des tâches, les modèles formels du comportement du système et le modèle de développement du programme de formation.In the field of interactive critical systems, the cost of a usage error or of a system failure can overcome the cost of the development of the system itself, and can result in loss of life, injury or damage to the system and its environment. Then, currently available Human Computer Interaction techniques, methods and processes are not sufficient, as they are not handling all of the design and development issues that are associated to interactive critical systems. First of all, these techniques, methods and processes do not enable to guarantee that the system will fulfil both usability and reliability properties. Then, they do not consider training and qualification of the users of the system. At last, they do not provide means for traceability of the needs and requirements through the whole development process. We propose an approach to develop interactive critical systems that are usable, reliable and operable and we describe the associated conceptual framework of our approach. We propose an implementation of this approach with a development process, notations and a software environment. The development process integrates phases for the development of the associated training program, and it provides support for the traceability of requirements and design choices during the whole phases of the process. This approach takes advantages from the User Centered Design paradigm and uses, in a synergistic way, task models, system's behaviour formal models and training program development model

A multi-modelS based approach for the modelling and the analysis of usable and resilient partly autonomous interactive systems

La croissance prévisionnelle du trafic aérien est telle que les moyens de gestion actuels doivent évoluer et être améliorés et l'automatisation de certains aspects de cette gestion semble être un moyen pour gérer cet accroissement du trafic tout en gardant comme invariant un niveau de sécurité constant. Toutefois, cette augmentation du trafic pourrait entraîner un accroissement de la variabilité de la performance de l'ensemble des moyens de gestion du trafic aérien, en particulier dans le cas de dégradation de cette automatisation. Les systèmes de gestion du trafic aérien sont considérés comme complexes car ils impliquent de nombreuses interactions entre humains et systèmes, et peuvent être profondément influencés par les aspects environnementaux (météorologie, organisation, stress ...) et tombent, de fait, dans la catégorie des Systèmes Sociotechniques (STS) (Emery & Trist, 1960). A cause de leur complexité, les interactions entre les différents éléments (humains, systèmes et organisations) de ces STS peuvent être linéaires et partiellement non linéaires, ce qui rend l'évolution de leur performance difficilement prévisible. Au sein de ces STS, les systèmes interactifs doivent être utilisables, i.e. permettre à leurs utilisateurs d'accomplir leurs tâches de manière efficace et efficiente. Un STS doit aussi être résilient aux perturbations telles que les défaillances logicielles et matérielles, les potentielles dégradations de l'automatisation ou les problèmes d'interaction entre les systèmes et leurs opérateurs. Ces problèmes peuvent affecter plusieurs aspects des systèmes sociotechniques comme les ressources, le temps d'exécution d'une tâche, la capacité à d'adaptation à l'environnement... Afin de pouvoir analyser l'impact de ces perturbations et d'évaluer la variabilité de la performance d'un STS, des techniques et méthodes dédiées sont requises. Elles doivent fournir un support à la modélisation et à l'analyse systématique de l'utilisabilité et de la résilience de systèmes interactifs aux comportements partiellement autonomes. Elles doivent aussi permettre de décrire et de structurer un grand nombre d'informations, ainsi que de traiter la variabilité de chaque élément du STS et la variabilité liée à leurs interrelations. Les techniques et méthodes existantes ne permettent actuellement ni de modéliser un STS dans son ensemble, ni d'en analyser les propriétés d'utilisabilité et de résilience (ou alors se focalisent sur un sous-ensemble du STS perdant, de fait, la vision systémique). Enfin, elles ne fournissent pas les moyens d'analyser la migration de tâches suite à l'introduction d'une nouvelle technologie ou d'analyser la variabilité de la performance en cas de dégradation de fonctions récemment automatisées. Ces arguments sont développés dans la thèse et appuyés par une analyse détaillée des techniques de modélisation existantes et des méthodes qui leurs sont associées. La contribution présentée est basée sur l'identification d'un ensemble d'exigences requises pour pouvoir modéliser et analyser chacun des éléments d'un STS. Certaines de ces exigences ont été remplies grâce à l'utilisation de techniques de modélisation existantes, les autres grâce à l'extension et au raffinement d'autres techniques. Cette thèse propose une approche qui intègre 3 techniques en particulier : FRAM (centrée sur les fonctions organisationnelles), HAMSTERS (centrée les objectifs et activités humaines) et ICO (dédiée à la modélisation du comportement des systèmes interactifs). Cette approche est illustrée par un exemple mettant en œuvre les extensions proposées et l'intégration des modèles. Une étude de cas plus complexe sur la gestion du trafic aérien (changement de route d'un avion en cas de mauvaises conditions météorologiques) est ensuite présentée pour montrer le passage à l'échelle de l'approche. Elle met en avant les bénéfices de l'intégration des modèles pour la prise en compte de la variabilité de la performance des différents éléments d'un STSThe current European Air Traffic Management (ATM) System needs to be improved for coping with the growth in air traffic forecasted for next years. It has been broadly recognised that the future ATM capacity and safety objectives can only be achieved by an intense enhancement of integrated automation support. However, increase of automation might come along with an increase of performance variability of the whole ATM System especially in case of automation degradation. ATM systems are considered complex as they encompass interactions involving humans and machines deeply influenced by environmental aspects (i.e. weather, organizational structure) making them belong to the class of Socio-Technical Systems (STS) (Emery & Trist, 1960). Due to this complexity, the interactions between the STS elements (human, system and organisational) can be partly linear and partly non-linear making its performance evolution complex and hardly predictable. Within such STS, interactive systems have to be usable i.e. enabling users to perform their tasks efficiently and effectively while ensuring a certain level of operator satisfaction. Besides, the STS has to be resilient to adverse events including potential automation degradation issues but also interaction problems between their interactive systems and the operators. These issues may affect several STS aspects such as resources, time in tasks performance, ability to adjust to environment, etc. In order to be able to analyse the impact of these perturbations and to assess the potential performance variability of a STS, dedicated techniques and methods are required. These techniques and methods have to provide support for modelling and analysing in a systematic way usability and resilience of interactive systems featuring partly autonomous behaviours. They also have to provide support for describing and structuring a large amount of information and to be able to address the variability of each of STS elements as well as the variability related to their interrelations. Current techniques, methods and processes do not enable to model a STS as a whole and to analyse both usability and resilience properties. Also, they do not embed all the elements that are required to describe and analyse each part of the STS (such as knowledge of different types which is needed by a user for accomplishing tasks or for interacting with dedicated technologies). Lastly, they do not provide means for analysing task migrations when a new technology is introduced or for analysing performance variability in case of degradation of the newly introduced automation. Such statements are argued in this thesis by a detailed analysis of existing modelling techniques and associated methods highlighting their advantages and limitations. This thesis proposes a multi-models based approach for the modelling and the analysis of partly-autonomous interactive systems for assessing their resilience and usability. The contribution is based on the identification of a set of requirements needed being able to model and analyse each of the STS elements. Some of these requirements were met by existing modelling techniques, others were reachable by extending and refining existing ones. This thesis proposes an approach which integrates 3 modelling techniques: FRAM (focused on organisational functions), HAMSTERS (centred on human goals and activities) and ICO (dedicated to the modelling of interactive systems). The principles of the multi-models approach is illustrated on an example for carefully showing the extensions proposed to the selected modelling techniques and how they integrate together. A more complex case study from the ATM World is then presented to demonstrate the scalability of the approach. This case study, dealing with aircraft route change due to bad weather conditions, highlights the ability of the integration of models to cope with performance variability of the various parts of the ST

Una propuesta de modelado del estudiante basada en ontologías y diagnóstico pedagógico-cognitivo no monótono

Los recientes avances tecnológicos han encontrado un potencial campo de explotación en la educación asistida por computador. A finales de los años 90 surgió un nuevo campo de investigación denominado Entornos Virtuales Inteligentes para el Entrenamiento y/o Enseñanza (EVIEs), que combinan dos áreas de gran complejidad: Los Entornos Virtuales (EVs) y los Sistemas de Tutoría Inteligente (STIs). De este modo, los beneficios de los entornos 3D (simulación de entornos de alto riesgo o entornos de difícil uso, etc.) pueden combinarse con aquéllos de un STIs (personalización de materias y presentaciones, adaptación de la estrategia de tutoría a las necesidades del estudiante, etc.) para proporcionar soluciones educativas/de entrenamiento con valores añadidos. El Modelo del Estudiante, núcleo de un SIT, representa el conocimiento y características del estudiante, y refleja el proceso de razonamiento del estudiante. Su complejidad es incluso superior cuando los STIs se aplican a EVs porque las nuevas posibilidades de interacción proporcionadas por estos entornos deben considerarse como nuevos elementos de información clave para el modelado del estudiante, incidiendo en todo el proceso educativo: el camino seguido por el estudiante durante su navegación a través de escenarios 3D; el comportamiento no verbal tal como la dirección de la mirada; nuevos tipos de pistas e instrucciones que el módulo de tutoría puede proporcionar al estudiante; nuevos tipos de preguntas que el estudiante puede formular, etc. Por consiguiente, es necesario que la estructura de los STIs, embebida en el EVIE, se enriquezca con estos aspectos, mientras mantiene una estructura clara, estructurada, y bien definida. La mayoría de las aproximaciones al Modelo del Estudiante en STIs y en IVETs no consideran una taxonomía de posibles conocimientos acerca del estudiante suficientemente completa. Además, la mayoría de ellas sólo tienen validez en ciertos dominios o es difícil su adaptación a diferentes STIs. Para vencer estas limitaciones, hemos propuesto, en el marco de esta tesis doctoral, un nuevo mecanismo de Modelado del Estudiante basado en la Ingeniería Ontológica e inspirado en principios pedagógicos, con un modelo de datos sobre el estudiante amplio y flexible que facilita su adaptación y extensión para diferentes STIs y aplicaciones de aprendizaje, además de un método de diagnóstico con capacidades de razonamiento no monótono. El método de diagnóstico es capaz de inferir el estado de los objetivos de aprendizaje contenidos en el SIT y, a partir de él, el estado de los conocimientos del estudiante durante su proceso de aprendizaje. La aproximación almodelado del estudiante propuesta ha sido implementada e integrada en un agente software (el agente de modelado del estudiante) dentro de una plataforma software existente para el desarrollo de EVIEs denominadaMAEVIF. Esta plataforma ha sido diseñada para ser fácilmente configurable para diferentes aplicaciones de aprendizaje. El modelado del estudiante presentado ha sido implementado e instanciado para dos tipos de entornos de aprendizaje: uno para aprendizaje del uso de interfaces gráficas de usuario en una aplicación software y para un Entorno Virtual para entrenamiento procedimental. Además, se ha desarrollado una metodología para guiar en la aplicación del esta aproximación de modelado del estudiante a cada sistema concreto.---ABSTRACT---Recent technological advances have found a potential field of exploitation in computeraided education. At the end of the 90’s a new research field emerged, the so-called Intelligent Virtual Environments for Training and/or Education (IVETs), which combines two areas of great complexity: Virtual Environments (VE) and Intelligent Tutoring Systems (ITS). In this way, the benefits of 3D environments (simulation of high risk or difficult-to-use environments, etc.) may be combined with those of an ITS (content and presentation customization, adaptation of the tutoring strategy to the student requirements, etc.) in order to provide added value educational/training solutions. The StudentModel, core of an ITS, represents the student’s knowledge and characteristics, and reflects the student’s reasoning process. Its complexity is even higher when the ITSs are applied on VEs because the new interaction possibilities offered by these environments must be considered as new key information pieces for student modelling, impacting all the educational process: the path followed by the student during their navigation through 3D scenarios; non-verbal behavior such as gaze direction; new types of hints or instructions that the tutoring module can provide to the student; new question types that the student can ask, etc. Thus, it is necessary for the ITS structure, which is embedded in the IVET, to be enriched by these aspects, while keeping a clear, structured and well defined architecture. Most approaches to SM on ITSs and IVETs don’t consider a complete enough taxonomy of possible knowledge about the student. In addition, most of them have validity only in certain domains or they are hard to be adapted for different ITSs. In order to overcome these limitations, we have proposed, in the framework of this doctoral research project, a newStudentModeling mechanism that is based onOntological Engineering and inspired on pedagogical principles, with a wide and flexible data model about the student that facilitates its adaptation and extension to different ITSs and learning applications, as well as a rich diagnosis method with non-monotonic reasoning capacities. The diagnosis method is able to infer the state of the learning objectives encompassed by the ITS and, fromit, the student’s knowledge state during the student’s process of learning. The proposed student modelling approach has been implemented and integrated in a software agent (the student modeling agent) within an existing software platform for the development of IVETs called MAEVIF. This platform was designed to be easily configurable for different learning applications. The proposed student modeling has been implemented and it has been instantiated for two types of learning environments: one for learning to use the graphical user interface of a software application and a Virtual Environment for procedural training. In addition, a methodology to guide on the application of this student modeling approach to each specific system has been developed