21 research outputs found
Recommended from our members
A usability comparison of PDA-based quizzes and paper-and-pencil quizzes
In the last few years, many schools and universities have incorporated personal
digital assistants (PDAs) into their teaching curricula, in an attempt to enhance
students' learning experience and reduce instructors' workload. One of the
most common uses of PDAs in the classroom is as a test administrator. This
study compared the usability effectiveness, efficiency, and satisfaction of a
PDA-based quiz application to that of standard paper-and-pencil quizzes in a
university course in order to determine whether it was advisable to invest time
and money in PDA-based testing. The effects of computer anxiety, age, gender,
and ethnicity on usability were also evaluated, to ascertain that these factors do
not discriminate against individuals taking PDA-based tests.
Five quizzes were administered to students participating in an engineering
introductory course. Of these, students took two PDA-based quizzes and three
paper-and-pencil quizzes. One PDA-based quiz and one paper-and-pencil quiz
were compared in terms of their effectiveness, measured as students' quiz
scores and through a mental workload questionnaire; their efficiency, which
was the time it took students to complete each quiz; and their satisfaction,
evaluated using a subjective user satisfaction questionnaire. Computer anxiety
was also measured, using an additional questionnaire.
It was hypothesized that the PDA-based quiz would be more effective and
efficient than the paper-and-pencil quiz and that students' satisfaction with the
PDA-based quiz would be greater. The study showed the PDA-based quiz to be
more efficient, that is, students completed it in less time than they needed to
complete the paper-and-pencil quiz. No differences in effectiveness and
satisfaction were found between the two quiz types.
It was also hypothesized that for PDA-based quizzes, as computer anxiety
increased, effectiveness and satisfaction would decrease; for paper-and-pencil
quizzes there would be no relationship between computer anxiety and
effectiveness and no relationship between computer anxiety and satisfaction.
Findings showed an increase in quiz score (increase in effectiveness) and an
increase in mental workload (decrease in effectiveness) as computer anxiety
increased for both quiz types. No relationship was found between computer
anxiety and satisfaction for either paper-and-pencil or PDA-based quizzes.
The final hypothesis suggested that user satisfaction would be positively
correlated with effectiveness (quiz score and mental workload) for both PDA-based
and paper-and-pencil quizzes. No relationship was found between quiz
score and satisfaction for either quiz type. User satisfaction was positively
correlated with mental workload, regardless of quiz type.
The usability comparison of paper-and-pencil and PDA-based quizzes found
the latter to be equal, if not superior, to the former. The effort students put into
taking the quiz was the same, regardless of administration method, and scores
were not affected. In addition, different demographic groups performed almost
equally well in both quiz types (white students' PDA-based quiz scores were
slightly lower than those of the other ethnic groups). Computer anxiety was not
affected by the quiz type. For these reasons, as well as other advantages to both
students (e.g. real-time scoring) and teachers (e.g. spending less time on
grading), PDAs are an attractive test administration option for schools and
universities
Measurement of Situation Awareness Effects of Adaptive Automation of Air Traffic Control Information Processing Functions
The goal of this research was to define a measure of situation awareness (SA) in an air traffic control (ATC) task and to investigate the effect of adaptive automation (AA) of various information processing functions on SA. An ATC simulation was used that was capable of presenting four different modes of control, including information acquisition, information analysis, decision making and action implementation automation, and a manual mode. Eight subjects completed two trials under each mode of control. Operator workload, assessed using a secondary task, was used to trigger automation of the primary ATC task. The SA measure was an adaptation of the Situation Awareness Global Assessment Technique (SAGAT), involving cueing of aircraft positions as well as objective weighting of the relevance of aircraft to controllers for queries. The SA response measure revealed a significant effect of AA on subject perception and overall SA, with superior SA under the information acquisition mode of automation. ATC performance was significantly superior (p\u3c0.05) when automation was applied to lower-order sensory processing functions, including information acquisition and action implementation, as compared to higher-order functions, specifically information analysis. During manual control periods as part of AA trials, ATC performance was significantly superior when following automation of information acquisition and information analysis functions. Secondary task performance was significantly worse under information analysis and decision making automation
Data from: Development and usability testing of an audit and feedback tool for anesthesiologists
Background: We describe the creation and evaluation of a personal audit & feedback (A&F) tool for anesthesiologists. Methods: A survey aimed at capturing barriers for personal improvement efforts and feedback preferences was administered to attending anesthesiologists. The results informed the design and implementation of 4 dashboards that display information on individual practice characteristics as well as comparative performance on several quality metrics. The dashboards’ usability was then tested using the human-centered design framework. Results: Anesthesiologists listed lack of information on current practice as the main barrier for improvement. Regarding usability, participants gave the dashboards an average score of 3.8 (scale 1-5) on consistency, learnability, and information organization, and performed the assigned tasks well, with an average score of 89% (range, 79-100%). Conclusions: We describe the design, implementation and usability testing of an innovative tool that utilizes data derived from the EHR system to provide A&F to anesthesiology providers
Situation Awareness Implications of Adaptive Automation of Air Traffic Controller Information Processing Functions
The goal of this research was to define a measure of situation awareness (SA) in an air traffic control (ATC) task and to assess the influence of adaptive automation (AA) of various information processing functions on controller perception, comprehension and projection. The measure was also to serve as a basis for defining and developing an approach to triggering dynamic control allocations, as part of AA, based on controller SA. To achieve these objectives, an enhanced version of an ATC simulation (Multitask (copyright)) was developed for use in two human factors experiments. The simulation captured the basic functions of Terminal Radar Approach Control (TRACON) and was capable of presenting to operators four different modes of control, including information acquisition, information analysis, decision making and action implementation automation, as well as a completely manual control mode. The SA measure that was developed as part of the research was based on the Situation Awareness Global Assessment Technique (SAGAT), previous goal-directed task analyses of enroute control and TRACON, and a separate cognitive task analysis on the ATC simulation. The results of the analysis on Multitask were used as a basis for formulating SA queries as part of the SAGAT-based approach to measuring controller SA, which was used in the experiments. A total of 16 subjects were recruited for both experiments. Half the subjects were used in Experiment #1, which focused on assessing the sensitivity and reliability of the SA measurement approach in the ATC simulation. Comparisons were made of manual versus automated control. The remaining subjects were used in the second experiment, which was intended to more completely describe the SA implications of AA applied to specific controller information processing functions, and to describe how the measure could ultimately serve as a trigger of dynamic function allocations in the application of AA to ATC. Comparisons were made of the sensitivity of the SA measure to automation manipulations impacting both higher-order information processing functions, such as information analysis and decision making, versus lower-order functions, including information acquisition and action implementation. All subjects were exposed to all forms of AA of the ATC task and the manual control condition. The approach to AA used in both experiments was to match operator workload, assessed using a secondary task, to dynamic control allocations in the primary task. In total, the subjects in each experiment participated in 10 trials with each lasting between 45 minutes and 1 hour. In both experiments, ATC performance was measured in terms of aircraft cleared, conflicting, and collided. Secondary task (gauge monitoring) performance was assessed in terms of a hit-to-signal ratio. As part of the SA measure, three simulation freezes were conducted during each trial to administer queries on Level 1, 2, and 3 SA