Stability, reliability and cross-mode correlations of tests in a recommended 8-minute performance assessment battery

A need exists for an automated performance test system to study drugs, agents, treatments, and stresses of interest to the aviation, space, and environmental medical community. The purpose of this present study is to evaluate tests for inclusion in the NASA-sponsored Automated Performance Test System (APTS). Twenty-one subjects were tested over 10 replications with tests previously identified as good candidates for repeated-measure research. The tests were concurrently administered in paper-and-pencil and microcomputer modes. Performance scores for the two modes were compared. Data from trials 1 to 10 were examined for indications of test stability and reliability. Nine of the ten APT system tests achieved stability. Reliabilities were generally high. Cross-correlation of microbased tests with traditional paper-and-pencil versions revealed similarity of content within tests in the different modes, and implied at least three cognition and two motor factors. This protable, inexpensive, rugged, computerized battery of tests is recommended for use in repeated-measures studies of environmental and drug effects on performance. Identification of other tests compatible with microcomputer testing and potentially capable of tapping previously unidentified factors is recommended. Documentation of APTS sensitivity to environmental agents is available for more than a dozen facilities and is reported briefly. Continuation of such validation remains critical in establishing the efficacy of APTS tests

Development of microcomputer-based mental acuity tests for repeated-measures studies

The purpose of this report is to detail the development of the Automated Performance Test System (APTS), a computer battery of mental acuity tests that can be used to assess human performance in the presence of toxic elements and environmental stressors. There were four objectives in the development of APTS. First, the technical requirements for developing APTS followed the tenets of the classical theory of mental tests which requires that tests meet set criteria like stability and reliability (the lack of which constitutes insensitivity). To be employed in the study of the exotic conditions of protracted space flight, a battery with multiple parallel forms is required. The second criteria was for the battery to have factorial multidimensionality and the third was for the battery to be sensitive to factors known to compromise performance. A fourth objective was for the tests to converge on the abilities entailed in mission specialist tasks. A series of studies is reported in which candidate APTS tests were subjected to an examination of their psychometric properties for repeated-measures testing. From this work, tests were selected that possessed the requisite metric properties of stability, reliability, and factor richness. In addition, studies are reported which demonstrate the predictive validity of the tests to holistic measures of intelligence

Microcomputer-based tests for repeated-measures: Metric properties and predictive validities

A menu of psychomotor and mental acuity tests were refined. Field applications of such a battery are, for example, a study of the effects of toxic agents or exotic environments on performance readiness, or the determination of fitness for duty. The key requirement of these tasks is that they be suitable for repeated-measures applications, and so questions of stability and reliability are a continuing, central focus of this work. After the initial (practice) session, seven replications of 14 microcomputer-based performance tests (32 measures) were completed by 37 subjects. Each test in the battery had previously been shown to stabilize in less than five 90-second administrations and to possess retest reliabilities greater than r = 0.707 for three minutes of testing. However, all the tests had never been administered together as a battery and they had never been self-administered. In order to provide predictive validity for intelligence measurement, the Wechsler Adult Intelligence Scale-Revised and the Wonderlic Personnel Test were obtained on the same subjects

Develop real-time dosimetry concepts and instrumentation for long term missions

The development of a rugged portable instrument to evaluate dose and dose equivalent is described. A tissue-equivalent proportional counter simulating a 2 micrometer spherical tissue volume was operated satisfactorily for over a year. The basic elements of the electronic system were designed and tested. And finally, the most suitable mathematical technique for evaluating dose equivalent with a portable instrument was selected. Design and fabrication of a portable prototype, based on the previously tested circuits, is underway

Users manual for the Automated Performance Test System (APTS)

The characteristics of and the user information for the Essex Automated Performance Test System (APTS) computer-based portable performance assessment battery are given. The battery was developed to provide a menu of performance test tapping the widest possible variety of human cognitive and motor functions, implemented on a portable computer system suitable for use in both laboratory and field settings for studying the effects of toxic agents and other stressors. The manual gives guidance in selecting, administering and scoring tests from the battery, and reviews the data and studies underlying the development of the battery. Its main emphasis is on the users of the battery - the scientists, researchers and technicians who wish to examine changes in human performance across time or as a function of changes in the conditions under which test data are obtained. First the how to information needed to make decisions about where and how to use the battery is given, followed by the research background supporting the battery development. Further, the development history of the battery focuses largely on the logical framework within which tests were evaluated

Surrogate measures: A proposed alternative in human factors assessment of operational measures of performance

Surrogate measures are proposed as an alternative to direct assessment of operational performance for purposes of screening agents who may have to work under unusual stresses or in exotic environments. Such measures are particularly proposed when the surrogate can be empirically validated against the operational criterion. The focus is on cognitive (or throughput) performances in humans as opposed to sensory (input) or motor (output) measures, but the methods should be applicable for development of batteries which will tap input/output functions. A menu of performance tasks is under development for implementation on a battery-operated portable microcomputer, with 21 tests currently available. The tasks are reliable and become stable in minimum amounts of time; appear sensitive to some agents; comprise constructs related to actual job tasks; and are easily administered in most environments. Implications for human factors engineering studies in environmental stress are discussed

A menu of self-administered microcomputer-based neurotoxicology tests

This study examined the feasibility of repeated self-administration of a newly developed battery of mental acuity tests. Researchers developed this battery to be used to screen the fitness for duty of persons in at-risk occupations (astronauts, race car drivers), or those who may be exposed to environmental stress, toxic agents, or disease. The menu under study contained cognitive and motor tests implemented on a portable microcomputer including: a five-test core battery, lasting six minutes, which had demonstrable reliabilities and stability from several previous repeated-measures studies, and also 13 new tests, lasting 42 minutes, which had appeared in other batteries but had not yet been evaluated for repeated-measures implementation in this medium. Sixteen subjects self-administered the battery over 10 repeated sessions. The hardware performed well throughout the study and the tests appeared to be easily self-administered. Stabilities and reliabilities of the test from the core battery were comparable to those obtained previously under more controlled experimental conditions. Analyses of metric properties of the remaining 13 tests produced eight additional tests with satisfactory properties. Although the average retest reliability was high, cross-correlations between tests were low, indicating factorial richness. The menu can be used to form batteries of flexible total testing time which are likely to tap different mental processes and functions

Development of a menu of performance tests self-administered on a portable microcomputer

Eighteen cognitive, motor, and information processing performance subtests were screened for self-administration over 10 trials by 16 subjects. When altered presentation forms of the same test were collectively considered, the battery composition was reduced to 10 distinctly different measures. A fully automated microbased testing system was employed in presenting the battery of subtests. Successful self-administration of the battery provided for the field testing of the automated system and facilitated convenient data collection. Total test administration time was 47.2 minutes for each session. Results indicated that nine of the tests stabilized, but for a short battery of tests only five are recommended for use in repeated-measures research. The five recommended tests include: the Tapping series, Number Comparison, Short-term Memory, Grammatical Reasoning, and 4-Choice Reaction Time. These tests can be expected to reveal three factors: (1) cognition, (2) processing quickness, and (3) motor. All the tests stabilized in 24 minutes, or approximately two 12-minute sessions

An automated stall-speed warning system

The development and testing of a stall-speed warning system for the OV-1C was examined. NASA designed and built an automated stall-speed warning system which presents both airspeed and stall speed to the pilot. The airspeed and stall speed are computed in real time by monitoring the basic aerodynamic parameters (dynamic pressure, horizontal and vertical accelerations, and pressure altitude) and other parameters (elevator and flap positions, engine torques, and fuel flow). In addition, an aural warning at predetermined stall margins is presented to the pilot through a voice synthesizer. Once the system was designed and installed in the aircraft, a flight-test program of less than 20 hrs was anticipated to determine the stall-speed software coefficients. These coefficients would then be inserted in the system's software and then test flown over a period of about 10 hr for the purpose of evaluation