Fidelity metrics for virtual environment simulations based on spatial memory awareness states

This paper describes a methodology based on human judgments of memory awareness states for assessing the simulation fidelity of a virtual environment (VE) in relation to its real scene counterpart. To demonstrate the distinction between task performance-based approaches and additional human evaluation of cognitive awareness states, a photorealistic VE was created. Resulting scenes displayed on a headmounted display (HMD) with or without head tracking and desktop monitor were then compared to the real-world task situation they represented, investigating spatial memory after exposure. Participants described how they completed their spatial recollections by selecting one of four choices of awareness states after retrieval in an initial test and a retention test a week after exposure to the environment. These reflected the level of visual mental imagery involved during retrieval, the familiarity of the recollection and also included guesses, even if informed. Experimental results revealed variations in the distribution of participants’ awareness states across conditions while, in certain cases, task performance failed to reveal any. Experimental conditions that incorporated head tracking were not associated with visually induced recollections. Generally, simulation of task performance does not necessarily lead to simulation of the awareness states involved when completing a memory task. The general premise of this research focuses on how tasks are achieved, rather than only on what is achieved. The extent to which judgments of human memory recall, memory awareness states, and presence in the physical and VE are similar provides a fidelity metric of the simulation in question

Assessing Professionalism: A theoretical framework for defining clinical rotation assessment criteria

Although widely accepted as an important graduate competence, professionalism is a challenging outcome to define and assess. Clinical rotations provide an excellent opportunity to develop student professionalism through the use of experiential learning and effective feedback, but without appropriate theoretical frameworks, clinical teachers may find it difficult to identify appropriate learning outcomes. The adage “I know it when I see it” is unhelpful in providing feedback and guidance for student improvement, and criteria that are more specifically defined would help students direct their own development. This study sought first to identify how clinical faculty in one institution currently assess professionalism, using retrospective analysis of material obtained in undergraduate teaching and faculty development sessions. Subsequently, a faculty workshop was held in which a round-table type discussion sought to develop these ideas and identify how professionalism assessment could be improved. The output of this session was a theoretical framework for teaching and assessing professionalism, providing example assessment criteria and ideas for clinical teaching. This includes categories such as client and colleague interaction, respect and trust, recognition of limitations, and understanding of different professional identities. Each category includes detailed descriptions of the knowledge, skills, and behaviors expected of students in these areas. The criteria were determined by engaging faculty in the development of the framework, and therefore they should represent a focused development of criteria already used to assess professionalism, and not a novel and unfamiliar set of assessment guidelines. The faculty-led nature of this framework is expected to facilitate implementation in clinical teaching

Situation awareness measurement: A review of applicability for C4i environments

The construct of situation awareness (SA) has become a core theme within the human factors (HF) research community. Consequently, there have been numerous attempts to develop reliable and valid measures of SA but there is a lack of techniques developed specifically for the assessment of SA in command, control, communication, computers and intelligence (C4i) environments. During the design, development and evaluation of novel systems, technology and procedures, valid and reliable situation awareness measurement techniques are required for the assessment of individual and team SA, in order to determine the improvements (or in some cases decrements) resulting from proposed design and technological interventions. The paper presents a review of existing situation awareness measurement techniques for their suitability for use in the assessment of SA in C4i environments. Seventeen SA measures were evaluated against a set of HF methods criteria. It was concluded that current SA measurement techniques are inadequate by themselves for use in the assessment of SA in C4i environments, and a multiple-measure approach utilising different approaches is recommended

Exploring the link between language anxiety and learner self-management in open language learning contexts

The Department of Languages at the Open University/UK was set up in 1991 and presented its first course in French for lower intermediate learners[1] in 1995. Intensive course production over a number of years has resulted in a portfolio of courses from beginner to degree level in French, German and Spanish. Language learning resources are varied and include print, video and audio materials as well as ICT components such as, for example, course websites. All students are individually assigned a tutor who advises on learning, marks coursework and holds tutorials. Until recently all tutorials took place face-to-face in the Open University's regional centres throughout the UK and Continental Western Europe, but since 2002 tutorials for some courses have been conducted online using Lyceum, an Internet-based conferencing system originally developed by the university's Knowledge Media Institute which provides multiple synchronous audio channels as well as synchronous text chat and several shared graphic interfaces (for a more detailed description of the tool see Hauck & Hampel, 2005). As a result of the continuing success of this tutorial mode the Department of Languages now offers a choice of face-to-face or Lyceum-based tuition to all students. This paper seeks to enhance our understanding of the interrelationship between affective issues, language anxiety in particular, and successful learner self-management in these learning environments. The findings are based on two phenomenographic studies, with the first one on language anxiety prompting the second one which explores the role of successful learner self-management. After some background information about the Open University's approach to distance education in general and the Department of Languages' approach to teaching languages at a distance in particular, this article looks at the theory underpinning the phenomena of language anxiety and learner self-management. This is followed by a presentation of the aforementioned studies, a discussion of their results and some preliminary conclusions

The one minute mentor : a pilot study assessing medical students’ and residents’ professional behaviours through recordings of clinical preceptors’ immediate feedback

Introduction: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners. Goal: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors’ brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach. Methods: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar’s clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner’s performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process. Results: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach α around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming. Discussion: This method of assessing learners’ professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed

Evaluation of post-license advanced driver training in Italy

Post-license advanced driver training addresses different categories of road users such as: novice drivers, professional drivers, company employers and recidivists. These training courses can be carried out on-track or on the road. On-track courses allow participants to gain knowledge on driving physics and experience limits in a safe road environment. On-road courses are more focused on hazard perception and situation awareness. Although extensive research has been done in this field, knowledge of the effects of these courses on road accident risk remains unclear. Previous evaluation of on-track courses did not always show a positive effect on crash rate. For example, post-license training focused on mastery of driving skills can lead to an increase of accident risk, especially on young males. However, research identified several factors that may enhance the effectiveness of driving training. In Europe a new framework for driver education and training has been proposed based on a safe driver hierarchical model (the GADGET model) and the development of a strategy for continuous learning. According to this framework, an evaluation study of on-track post-license advanced driver training has been undertaken in Italy with the main goal of assessing the safety effects of these courses and identifying training aspects to be improved. Besides crash rate, the study aims at assessing also driver behavior, knowledge of risks, self-evaluation and training quality. This paper presents the results of the possible effects of advanced driver training on driving behavior, considering in particular the number and type of violations. For each driver, data on age, gender and driving violations history were extracted from the platform and the national violations database. Three cases were addressed through a before-after analysis with control group. Case 1 considers all drivers who attended an ADT course. Case 2 aimed at understanding the effects of the courses on a specific target group: the traffic violators. Case 3 is similar to Case 2, however the control group was selected in a way that drivers characteristics and the violation rate was similar to the violation rate of the treatment group in the before period. The significance of the differences highlighted was assessed through appropriate statistical tests (i.e. paired t-test and the Wilcoxon signed-rank test). The study showed in general a higher propensity to commit traffic violations after attending an ADT course. These results are in contrast to what expected and show the necessity to diversify the training classes according to the different needs of participant

Assessment of the Angolan (CHERRT) Mobile Laboratory Curriculum for Disaster and Pandemic Response

Introduction: As of April 5, 2020, the World Health Organization reported over one million confirmed cases and more than 62,000 confirmed coronavirus (COVID-19) deaths affecting 204 countries/ regions. The lack of COVID-19 testing capacity threatens the ability of both the United States (US) and low middle income countries (LMIC) to respond to this growing threat, The purpose of this study was to assess the effectiveness through participant self-assessment of a rapid response team (RRT) mobile laboratory curriculumMethods: We conducted a pre and post survey for the purpose of a process improvement assessment in Angola, involving 32 individuals. The survey was performed before and after a 14-day training workshop held in Luanda, Angola, in December 2019. A paired t-test was used to identify any significant change on six 7-point Likert scale questions with α< 0.05 (95% confidence interval).Results: All six of the questions –  1) “I feel confident managing a real laboratory sample test for Ebola or other highly contagious sample;” 2) “I feel safe working in the lab environment during a real scenario;” 3) “I feel as if I can appropriately manage a potentially highly contagious laboratory sample;” 4)“I feel that I can interpret a positive or negative sample during a suspected contagious outbreak;” 5) “I understand basic Biobubble/mobile laboratory concepts and procedures;” and 6) “I understand polymerase chain reaction (PCR) principles” – showed statistical significant change pre and post training. Additionally, the final two questions – “I can more effectively perform my role/position because of the training I received during this course;” and “This training was valuable”  – received high scores on the Likert scale.Conclusion: This Angolan RRT mobile laboratory training curriculum provides the nation of Angola with the confidence to rapidly respond and test at the national level a highly infectious contagion in the region and perform on-scene diagnostics. This mobile RRT laboratory provides a mobile and rapid diagnostic resource when epidemic/pandemic resource allocation may need to be prioritized based on confirmed disease prevalence