Training high performance skills using above real-time training

The Above Real-Time Training (ARTT) concept is a unique approach to training high performance skills. ARTT refers to a training paradigm that places the operator in a simulated environment that functions at faster than normal time. Such a training paradigm represents a departure from the intuitive, but not often supported, feeling that the best practice is determined by the training environment with the highest fidelity. This approach is hypothesized to provide greater 'transfer value' per simulation trial, by incorporating training techniques and instructional features into the simulator. These techniques allow individuals to acquire these critical skills faster and with greater retention. ARTT also allows an individual trained in 'fast time' to operate at what appears to be a more confident state, when the same task is performed in a real-time environment. Two related experiments are discussed. The findings appear to be consistent with previous findings that show positive effects of task variation during training. Moreover, ARTT has merit in improving or maintaining transfer with sharp reductions in training time. There are indications that the effectiveness of ARTT varies as a function of task content and possibly task difficulty. Other implications for ARTT are discussed along with future research directions

The use of acoustic cues in phonetic perception: Effects of spectral degradation, limited bandwidth and background noise

Hearing impairment, cochlear implantation, background noise and other auditory degradations result in the loss or distortion of sound information thought to be critical to speech perception. In many cases, listeners can still identify speech sounds despite degradations, but understanding of how this is accomplished is incomplete. Experiments presented here tested the hypothesis that listeners would utilize acoustic-phonetic cues differently if one or more cues were degraded by hearing impairment or simulated hearing impairment. Results supported this hypothesis for various listening conditions that are directly relevant for clinical populations. Analysis included mixed-effects logistic modeling of contributions of individual acoustic cues for various contrasts. Listeners with cochlear implants (CIs) or normal-hearing (NH) listeners in CI simulations showed increased use of acoustic cues in the temporal domain and decreased use of cues in the spectral domain for the tense/lax vowel contrast and the word-final fricative voicing contrast. For the word-initial stop voicing contrast, NH listeners made less use of voice-onset time and greater use of voice pitch in conditions that simulated high-frequency hearing impairment and/or masking noise; influence of these cues was further modulated by consonant place of articulation. A pair of experiments measured phonetic context effects for the "s/sh" contrast, replicating previously observed effects for NH listeners and generalizing them to CI listeners as well, despite known deficiencies in spectral resolution for CI listeners. For NH listeners in CI simulations, these context effects were absent or negligible. Audio-visual delivery of this experiment revealed enhanced influence of visual lip-rounding cues for CI listeners and NH listeners in CI simulations. Additionally, CI listeners demonstrated that visual cues to gender influence phonetic perception in a manner consistent with gender-related voice acoustics. All of these results suggest that listeners are able to accommodate challenging listening situations by capitalizing on the natural (multimodal) covariance in speech signals. Additionally, these results imply that there are potential differences in speech perception by NH listeners and listeners with hearing impairment that would be overlooked by traditional word recognition or consonant confusion matrix analysis

The Effects of Emotion and Action on Binding in Memory

The ability to successfully bind features and objects at different levels of abstraction is important for everyday functioning of memory. The current study examined how actions and emotional arousal influence item recognition and between-item binding across two experiments. According to the Arousal-Biased Competition Theory (ABC; Mather and Sutherland, 2011), binding can be enhanced by emotional arousal, depending upon what is the focus of attention within a scene. In the current study, participants viewed a series of slides, each of which depicted a person performing an action with an object, as well as an object that is not interacted with. All of the actions performed were emotionally neutral. According to ABC, this difference between attended and non-attended items should be enhanced when in the presence of an emotionally arousing stimulus. In the current experiments, emotional arousal was manipulated using sounds presented before slides depicting a person performing an action using an object. In Experiment 1, actions led to enhanced attention to (and better item memory for) manipulated objects relative to objects that were not the focus of attention. Participants also gave higher confidence ratings for recognized interacted objects than recognized non-interacted objects. However, the predicted interaction between emotional arousal and item type was not obtained. The one impact of emotional arousal was that participants responded more conservatively to faces and items associated with a negative sound. In Experiment 2, there was no evidence for enhanced associative memory as a function of either emotion or action, although overall performance was very poor. The implications of these findings for theoretical views of memory are discussed, as well as future directions for research

The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

Objective and subjective measures of performance in virtual reality environments increase as more sensory cues are delivered and as simulation fidelity increases. Some cues (colour or sound) are easier to present than others (object weight, vestibular cues) so that substitute cues can be used to enhance informational content in a simulation at the expense of simulation fidelity. This study evaluates how substituting cues in one modality by alternative cues in another modality affects subjective and objective performance measures in a highly immersive virtual reality environment. Participants performed a wheel change in a virtual reality (VR) environment. Auditory, haptic and visual cues, signalling critical events in the simulation, were manipulated in a factorial design. Subjective ratings were recorded via questionnaires. The time taken to complete the task was used as an objective performance measure. The results show that participants performed best and felt an increased sense of immersion and involvement, collectively referred to as 'presence', when substitute multimodal sensory feedback was provided. Significant main effects of audio and tactile cues on task performance and on participants' subjective ratings were found. A significant negative relationship was found between the objective (overall completion times) and subjective (ratings of presence) performance measures. We conclude that increasing informational content, even if it disrupts fidelity, enhances performance and user's overall experience. On this basis we advocate the use of substitute cues in VR environments as an efficient method to enhance performance and user experience

Issues in development, evaluation, and use of the NASA Preflight Adaptation Trainer (PAT)

The Preflight Adaptation Trainer (PAT) is intended to reduce or alleviate space adaptation syndrome by providing opportunities for portions of that adaptation to occur under normal gravity conditions prior to space flight. Since the adaptation aspects of the PAT objectives involve modification not only of the behavior of the trainee, but also of sensiomotor skills which underly the behavioral generation, the defining of training objectives of the PAT utilizes four mechanisms: familiarization, demonstration, training and adaptation. These mechanisms serve as structural reference points for evaluation, drive the content and organization of the training procedures, and help to define the roles of the PAT instructors and operators. It was determined that three psychomotor properties are most critical for PAT evaluation: reliability; sensitivity; and relevance. It is cause for concern that the number of measures available to examine PAT effects exceed those that can be properly studied with the available sample sizes; special attention will be required in selection of the candidate measure set. The issues in PAT use and application within a training system context are addressed through linking the three training related mechanisms of familiarization, demonstration and training to the fourth mechanism, adaptation

Design For Auditory Displays: Identifying Temporal And Spatial Information Conveyance Principles

Designing auditory interfaces is a challenge for current human-systems developers. This is largely due to a lack of theoretical guidance for directing how best to use sounds in today\u27s visually-rich graphical user interfaces. This dissertation provided a framework for guiding the design of audio interfaces to enhance human-systems performance. This doctoral research involved reviewing the literature on conveying temporal and spatial information using audio, using this knowledge to build three theoretical models to aid the design of auditory interfaces, and empirically validating select components of the models. The three models included an audio integration model that outlines an end-to-end process for adding sounds to interactive interfaces, a temporal audio model that provides a framework for guiding the timing for integration of these sounds to meet human performance objectives, and a spatial audio model that provides a framework for adding spatialization cues to interface sounds. Each model is coupled with a set of design guidelines theorized from the literature, thus combined, the developed models put forward a structured process for integrating sounds in interactive interfaces. The developed models were subjected to a three phase validation process that included review by Subject Matter Experts (SMEs) to assess the face validity of the developed models and two empirical studies. For the SME review, which assessed the utility of the developed models and identified opportunities for improvement, a panel of three audio experts was selected to respond to a Strengths, Weaknesses, Opportunities, and Threats (SWOT) validation questionnaire. Based on the SWOT analysis, the main strengths of the models included that they provide a systematic approach to auditory display design and that they integrate a wide variety of knowledge sources in a concise manner. The main weaknesses of the models included the lack of a structured process for amending the models with new principles, some branches were not considered parallel or completely distinct, and lack of guidance on selecting interface sounds. The main opportunity identified by the experts was the ability of the models to provide a seminal body of knowledge that can be used for building and validating auditory display designs. The main threats identified by the experts were that users may not know where to start and end with each model, the models may not provide comprehensive coverage of all uses of auditory displays, and the models may act as a restrictive influence on designers or they may be used inappropriately. Based on the SWOT analysis results, several changes were made to the models prior to the empirical studies. Two empirical evaluation studies were conducted to test the theorized design principles derived from the revised models. The first study focused on assessing the utility of audio cues to train a temporal pacing task and the second study combined both temporal (i.e., pace) and spatial audio information, with a focus on examining integration issues. In the pace study, there were four different auditory conditions used for training pace: 1) a metronome, 2) non-spatial auditory earcons, 3) a spatialized auditory earcon, and 4) no audio cues for pace training. Sixty-eight people participated in the study. A pre- post between subjects experimental design was used, with eight training trials. The measure used for assessing pace performance was the average deviation from a predetermined desired pace. The results demonstrated that a metronome was not effective in training participants to maintain a desired pace, while, spatial and non-spatial earcons were effective strategies for pace training. Moreover, an examination of post-training performance as compared to pre-training suggested some transfer of learning. Design guidelines were extracted for integrating auditory cues for pace training tasks in virtual environments. In the second empirical study, combined temporal (pacing) and spatial (location of entities within the environment) information were presented. There were three different spatialization conditions used: 1) high fidelity using subjective selection of a best-fit head related transfer function, 2) low fidelity using a generalized head-related transfer function, and 3) no spatialization. A pre- post between subjects experimental design was used, with eight training trials. The performance measures were average deviation from desired pace and time and accuracy to complete the task. The results of the second study demonstrated that temporal, non-spatial auditory cues were effective in influencing pace while other cues were present. On the other hand, spatialized auditory cues did not result in significantly faster task completion. Based on these results, a set of design guidelines was proposed that can be used to direct the integration of spatial and temporal auditory cues for supporting training tasks in virtual environments. Taken together, the developed models and the associated guidelines provided a theoretical foundation from which to direct user-centered design of auditory interfaces

Toward Simulation-Based Training Validation Protocols: Exploring 3d Stereo with Incremental Rehearsal and Partial Occlusion to Instigate and Modulate Smooth Pursuit and Saccade Responses in Baseball Batting

“Keeping your eye on the ball” is a long-standing tenet in baseball batting. And yet, there are no protocols for objectively conditioning, measuring, and/or evaluating eye-on-ball coordination performance relative to baseball-pitch trajectories. Although video games and other virtual simulation technologies offer alternatives for training and obtaining objective measures, baseball batting instruction has relied on traditional eye-pitch coordination exercises with qualitative “face validation”, statistics of whole-task batting performance, and/or subjective batter-interrogation methods, rather than on direct, quantitative eye-movement performance evaluations. Further, protocols for validating transfer-of-training (ToT) for video games and other simulation-based training have not been established in general ― or for eye-movement training, specifically. An exploratory research study was conducted to consider the ecological and ToT validity of a part-task, virtual-fastball simulator implemented in 3D stereo along with a rotary pitching machine standing as proxy for the live-pitch referent. The virtual-fastball and live-pitch simulation couple was designed to facilitate objective eye-movement response measures to live and virtual stimuli. The objective measures 1) served to assess the ecological validity of virtual fastballs, 2) informed the characterization and comparison of eye-movement strategies employed by expert and novice batters, 3) enabled a treatment protocol relying on repurposed incremental-rehearsal and partial-occlusion methods intended to instigate and modulate strategic eye movements, and 4) revealed whether the simulation-based treatment resulted in positive (or negative) ToT in the real task. Results indicated that live fastballs consistently elicited different saccade onset time responses than virtual fastballs. Saccade onset times for live fastballs were consistent with catch-up saccades that follow the smooth-pursuit maximum velocity threshold of approximately 40-70˚/sec while saccade onset times for virtual fastballs lagged in the order of 13%. More experienced batters employed more deliberate and timely combinations of smooth pursuit and catch-up saccades than less experienced batters, enabling them to position their eye to meet the ball near the front edge of home plate. Smooth pursuit and saccade modulation from treatment was inconclusive from virtual-pitch pre- and post-treatment comparisons, but comparisons of live-pitch pre- and post-treatment indicate ToT improvements. Lagging saccade onset times from virtual-pitch suggest possible accommodative-vergence impairment due to accommodation-vergence conflict inherent to 3D stereo displays

Varying Feedback Strategy and Scheduling in Simulator Training: Effects on Learner Perceptions, Initial Learning, and Transfer

This experimental study investigated the effects of visual feedback on initial learning, perceived self-efficacy, workload, near transfer, far transfer, and perceived realism during a simulator-based training task. Prior studies indicate that providing feedback is critical for schema development (Salmoni, Schmidt, & Walter 1984; Sterman, 1994). However, its influence has been shown to dissipate and is not directly proportionate to the frequency at which it is given (Wulf, Shea, & Matschiner, 1998). A total of 54 participants completed the study forming six treatment groups. The independent treatment, visual feedback, was manipulated as scheduling (absolute—every practice trial or relative—every third trial) and strategies (gradual decrease of visual cues within the interface, gradual increase of visual cues within the interface, or a single consistent cue for each trial). Participants completed twelve practice trials of welding under one of six feedback manipulations; then, participants completed twelve practice trials of welding without it. Lastly, participants performed the weld task on actual equipment in a shop area. No treatment showed significant difference among groups with regard to initial learning, retention, near transfer, and far transfer measures. However, a statistical significance was found during initial learning and retention within each treatment group. Findings support empirical evidence that a variability of practice paradigm promotes learning (Lee & Carnahan, 1990; Shea & Morgan, 1979). Learner perceptions of realism suggest that novice learners perceive simulator fidelity as high, however, these perceptions may dissipate as the learner practices. Those groups that involved the greatest number of cues at the onset of practice or having cues available at every other trial reported the greatest amount of workload. All groups reported increases in perceptions of self-efficacy during practice on the simulator, but those perceptions decreased when participants performed the weld task on actual equipment. Findings suggest that contextual-interference of increasing, decreasing, or changing feedback counteracts the guidance effect of feedback as found in previous studies

An Examination of a Theory of Embodied Social Presence in Virtual Worlds

In this article, we discuss and empirically examine the importance of embodiment, context, and spatial proximity as they pertain to collaborative interaction and task completion in virtual environments. Specifically, we introduce the embodied social presence (ESP) theory as a framework to account for a higher level of perceptual engagement that users experience as they engage in activity-based social interaction in virtual environments. The ESP theory builds on the analysis of reflection data from Second Life users to explain the process by which perceptions of ESP are realized. We proceed to describe implications of ESP for collaboration and other organizational functions