Modified Cooper Harper scales for assessing unmanned vehicle displays

Unmanned vehicle (UV) displays are often the only information link between operators and vehicles, so their design is critical to mission success. However, there is currently no standardized methodology for operators to subjectively assess a display's support of mission tasks. This paper proposes a subjective UV display evaluation tool: the Modified Cooper-Harper for Unmanned Vehicle Displays (MCH-UVD). The MCH-UVD is adapted from the Cooper-Harper aircraft handling scale by shifting focus to support of operator information processing. An experiment was conducted to evaluate and refine the MCH-UVD, as well as assess the need for mission-specific versus general versions. Participants (86%) thought that MCH-UVD helped them identify display deficiencies, and 32% said that they could not have identified the deficiencies without the tool. No major additional benefits were observed with mission-specific versions over the general scale.U.S. Army Aberdeen Test Cente

Modified Cooper Harper Scales for Assessing Unmanned Vehicle Displays

In unmanned vehicle (UV) operations, displays are often the only information link between operators and vehicles. It is essential these displays present information clearly and efficiently so that operators can interact with the UVs to achieve mission objectives. While there are a variety of metrics to evaluate displays, there is no current standardized methodology for operators to subjectively assess a display’s support and identify specific deficiencies. Such a methodology could improve current displays and ensure that displays under development support operator processes. This report presents a quasi- subjective display evaluation tool called the Modified Cooper-Harper for Unmanned Vehicle Displays (MCH-UVD) diagnosis tool. This tool, adapted from the Cooper-Harper aircraft handling scale, allows operators to assess a display, translating their judgments on potential display shortcomings into a number corresponding to a particular deficiency in operator support. The General MCH-UVD can be used to diagnose deficiencies for any UV display, while the Specific MCH-UVD is UV and mission specific in its evaluation of displays. This report presents the General MCH-UVD and provides guidance on how to adapt it to create a Specific MCH-UVD through the use of UV mission taxonomies and a questioning method. A UGV search mission case study provides a how-to guide example for generating a Specific MCH-UVD. The report also presents an experiment conducted to validate the MCH-UVD and assess if a mission-specific version is necessary, or if the general form of the MCH-UVD is sufficient for different UV display evaluation. The report concludes with discussion on how to administer the scale, when a Specific scale is necessary, MCH-UVD diagnosis tool limitations, and future work.Prepared for US Army Aberdeen Testing Cente

Selecting Metrics to Evaluate Human Supervisory Control Applications

The goal of this research is to develop a methodology to select supervisory control metrics. This methodology is based on cost-benefit analyses and generic metric classes. In the context of this research, a metric class is defined as the set of metrics that quantify a certain aspect or component of a system. Generic metric classes are developed because metrics are mission-specific, but metric classes are generalizable across different missions. Cost-benefit analyses are utilized because each metric set has advantages, limitations, and costs, thus the added value of different sets for a given context can be calculated to select the set that maximizes value and minimizes costs. This report summarizes the findings of the first part of this research effort that has focused on developing a supervisory control metric taxonomy that defines generic metric classes and categorizes existing metrics. Future research will focus on applying cost benefit analysis methodologies to metric selection. Five main metric classes have been identified that apply to supervisory control teams composed of humans and autonomous platforms: mission effectiveness, autonomous platform behavior efficiency, human behavior efficiency, human behavior precursors, and collaborative metrics. Mission effectiveness measures how well the mission goals are achieved. Autonomous platform and human behavior efficiency measure the actions and decisions made by the humans and the automation that compose the team. Human behavior precursors measure human initial state, including certain attitudes and cognitive constructs that can be the cause of and drive a given behavior. Collaborative metrics address three different aspects of collaboration: collaboration between the human and the autonomous platform he is controlling, collaboration among humans that compose the team, and autonomous collaboration among platforms. These five metric classes have been populated with metrics and measuring techniques from the existing literature. Which specific metrics should be used to evaluate a system will depend on many factors, but as a rule-of-thumb, we propose that at a minimum, one metric from each class should be used to provide a multi-dimensional assessment of the human-automation team. To determine what the impact on our research has been by not following such a principled approach, we evaluated recent large-scale supervisory control experiments conducted in the MIT Humans and Automation Laboratory. The results show that prior to adapting this metric classification approach, we were fairly consistent in measuring mission effectiveness and human behavior through such metrics as reaction times and decision accuracies. However, despite our supervisory control focus, we were remiss in gathering attention allocation metrics and collaboration metrics, and we often gathered too many correlated metrics that were redundant and wasteful. This meta-analysis of our experimental shortcomings reflect those in the general research population in that we tended to gravitate to popular metrics that are relatively easy to gather, without a clear understanding of exactly what aspect of the systems we were measuring and how the various metrics informed an overall research question

Evaluation of Unmanned Aircraft Flying Qualities Using a Stitched Learjet Model

In recent years, military UAVs have taken over missions that were too dull, dirty, or dangerous for manned aircraft. The increased demand has led to a build-fly-fix-fly development mentality, plaguing the early lifecycle with staggering mishap rates. Currently, MIL-STD-1797 lists flying qualities for UAVs as TBD, and the standards for manned fixed wing are inadequate when applied to UAVs. In an effort to expand the database of UAV flying qualities, an analysis was completed on a Simulink model of an LJ-25D developed from Calspans Variable Stability System aircraft at the United States Test Pilot School. Three maneuvers were simulated: (1) a non-precision, non-aggressive climbing spiral, (2) a precision, non-aggressive side step landing, and (3) a precision, non-aggressive aerial refueling task. These maneuvers were chosen to evaluate the performance and workload of the aircraft as four stability and control parameters were scaled. The data were utilized in identifying trends between the scaled stability and control parameters and resulting workload and performance metrics. Thumbprint plots were generated to identify Level 1, Level 2, and Level 3 flying qualities and compared to MIL-STD-1797 plots. Results point to utilizing a combination of classical aircraft literal factors, such as ςsp and CAP, with newly developed mathematical techniques, such as L2 norm and TIC, to assess the workload of the flight controller and performance during the maneuver

The Integration Of Audio Into Multimodal Interfaces: Guidelines And Applications Of Integrating Speech, Earcons, Auditory Icons, and Spatial Audio (SEAS)

The current research is directed at providing validated guidelines to direct the integration of audio into human-system interfaces. This work first discusses the utility of integrating audio to support multimodal human-information processing. Next, an auditory interactive computing paradigm utilizing Speech, Earcons, Auditory icons, and Spatial audio (SEAS) cues is proposed and guidelines for the integration of SEAS cues into multimodal systems are presented. Finally, the results of two studies are presented that evaluate the utility of using SEAS cues, developed following the proposed guidelines, in relieving perceptual and attention processing bottlenecks when conducting Unmanned Air Vehicle (UAV) control tasks. The results demonstrate that SEAS cues significantly enhance human performance on UAV control tasks, particularly response accuracy and reaction time on a secondary monitoring task. The results suggest that SEAS cues may be effective in overcoming perceptual and attentional bottlenecks, with the advantages being most revealing during high workload conditions. The theories and principles provided in this paper should be of interest to audio system designers and anyone involved in the design of multimodal human-computer systems

A Quantitative analysis of the mental workload demands of MRAP vehicle drivers using physiological, subjective, and performance assessments

United States Special Operations Command (USSOCOM) Operators and vehicle Commanders are specially trained United States military Warfighters that have the demanding task of operating or working onboard Mine Resistant Ambush Protected (MRAP) All Terrain Vehicles (M-ATVs). Their missions encounter significant mental demands resulting from fatigue, highly stressful situations, and interactions with Government Furnished Equipment (GFE). Excessive mental demands can be the primary factor leading to compromised vehicle communication, missed improvised explosive device (IED) detection, and increased incidents of vehicle roll-over. Research has demonstrated the consequences of mental overloading including increased errors, performance decrements, distraction, cognitive tunneling and inadequate time to appropriately process information. The objectives of this thesis were to evaluate the extent to which task-related factors impact the mental workload of Warfighters and to evaluate the consistency among the three categories of mental workload metrics. The 14 participants studied in this research were Marine Corps personnel who had heavy vehicle driving experience. Physiological, subjective and performance measures were collected during a four-segment course that progressed in difficulty and analyzed across all participants to assess changes in mental workload. It was found that task-related factors impacted the mental workload of Warfighters. The subjective metric was able to capture changes in workload more accurately than biosignals. Due to technical problems with the biosignal data, comparison of consistency across metrics was inconclusive. The subjective workload ratings were significantly different between course segments and experience levels. The experiment resulted in workload ratings that increased by as much as 94% between segments and were 18% higher among novice drivers. This study showed that mental workload fluctuates while driving in a stressful situation, despite training and experience, and consequently, detection performance will be impacted which could have very adverse consequences. There is the need for additional research to have a better understanding of the true impact of mental workload on MRAP vehicle drivers, especially in an operational environment

Evaluation of Pilot and Quadcopter Performance from Open Loop Mission Oriented Flight Testing

Title from PDF of title page viewed January 30, 2019Thesis advisor: Travis FieldsVitaIncludes bibliographical references (pages 69-78)Thesis (M.S.)--School of Computing and Engineering. University of Missouri--Kansas City, 2018Ease of control, portability and efficiency in versatile applications have made Unmanned Aerial Vehicle (UAV) very popular. Considering various usefulness, safe operation of UAV is important and to ensure safe operation, proper synergy between pilot and UAV is mandatory. For this reason, individual evaluation of both pilot and UAV performance is vital so that pilot can accomplish a task with the assigned system without any accident. In this study, a new evaluation technique of pilot and UAV performance is presented based on flight test results of a mission task of following a desired path. Seven pilots are categorized into two groups based on their experience level and a quadcopter is categorized into three groups based on level of autonomy associated with it. Path error is calculated in time domain to distinguish between pilot levels and level of autonomy of UAV. Path error metrics show that novice pilots make more error than experienced pilots and error increases from more autonomous to less autonomous UAV. For frequency do main analysis, transfer function modeling is done including human operator in the open loop so that full scenario of the flight, from pilot to UAV can be analyzed. Frequency domain analysis helps to identify system complexity, stability and fastness based on level of autonomy as well as pilot performance based on experience level. Apart from time and frequency domain analysis, Cooper-Harper rating scale is used by the pilots to rate the UAV based on ease of control. Along with time and frequency domain variables, Cooper-Harper rating is included as predictors in the modeling of evaluation of pilot and quadcopter performance. The parameter estimation of regression model shows the change in model outcome for both pilot and UAV level with the variation of predictor values. In the end, a verification test case is included where an eighth pilot flies the same quadcopter to complete the same task and variables derived from the flight data of this single flight test are placed in the binary logistic regression model equation to predict pilot experience level and multinoial logistic regression model equation to predict UAV autonomy level. The established model can predict pilot experience level and UAV autonomy level correctly that matches with the real case. The evaluation technique developed in this thesis shows a path to evaluate pilot and quadcopter performance individually, that can be used to train pilots to accomplish a specific task with the assigned UAV system.Introduction -- Literature review -- Methodology -- Results and discussions -- Conclusion -- Future wor

Pengukuran Metode Beban Kerja Mental Modified Cooper Harper (MCH) dan Manfaatnya

Mental workload is a branch of ergonomics that focuses on the psychological load that a person experiences at work. This is important to research because humans will definitely experience psychological fatigue in addition to physical fatigue. One method that can be used in examining a person's mental workload is the modified cooper harper method. The purpose of using this method is to determine the mental workload conditions of workers, especially when using tools or facilities that support their work. And the subject chosen in applying this method is a nurse. It is hoped that after knowing the workload of nurses, researchers can provide recommendations for the future

A Quantitative Evaluation of Pilot-in-the-Loop Flying Tasks Using Power Frequency and NASA TLX Workload Assessment

While all manner of both qualitative and quantitative assessment tools exist to measure pilot performance during aircraft flight test, the argument to mathematically correlate two such diametrically different metrics is strong. By definitively connecting a pilot’s written handling qualities or task loading feedback with measured performance data, researchers can more accurately examine any of a whole host of flight research topics. Building upon past research which shows a positive correlation between Cooper-Harper Handling Qualities Ratings and calculated values for power frequency using a group of experienced test pilots, it is valuable to examine whether power frequency correlates with other metrics such as the NASA Task Loading Index (TLX). TLX provides a measure of a pilot’s self-assessed workload and is routinely used in modern flight test experimentation to measure perceived pilot workload. Using data from twenty-nine instructor pilots flying the NASA Ice Contamination Effects Flight Training Device (ICEFTD), the data set examined showed little connection between power frequency values and the TLX scores assigned by the pilots to each approach. Among the group of pilots flying the ICEFTD, self-assessed workload was a poor indicator of measured work load – such a trend indicates that non-test pilot self-measurement in workload assessment may not be as valuable as trained test pilot measurements. A number of influential causal factors were evident in the use of this recycled data set, and an ideal retest scenario is discussed at length

The Effect of Control and Display Lag on UAS Internal Pilot Manual Landing Performance

An important characteristic of UASs is lag because it can become a considerable challenge to successful human-in-the-loop control. As such, UASs are designed and configured to minimize system lag, though this can increase acquisition and operation costs considerably. In an effort to cut costs, an organization may choose to accept greater risk and deploy a UAS with high system lag. Before this risk can be responsibly accepted, it must be quantified. While many studies have examined system lag, very few have been able to quantify the risk that various levels of lag pose to an internally piloted, manually landed UAS. This study attempted to do so by evaluating pilot landing performance in a simulator with 0 ms, 240 ms, and 1000 ms of additional lag. Various measures were used, including a novel coding technique. Results indicated that 1000 ms of lag was unsafe by all measures. They also indicate that 240 ms of lag degrades performance, but participants were able to successfully land the simulated aircraft. This study showed the utility of using several measures to evaluate the effect of lag on landing performance and it helped demonstrate that while 1000 ms poses a high risk, 240 ms of lag may be a much more manageable risk. Future research suggested by this research includes: investigating lag between 240 ms and 1000 ms, introducing different weather phenomena, developing system lag training techniques for operators, and investigating the effect of aides such as predictive displays and autopilot-assisted recovery