The purpose of this research was to investigate the factors that contributed to engineering education students’ reliance on technology while learning new concepts. The researcher hypothesized that students would give reliance to their technology, even in the face of evidence that the technology was not working as intended. This research used a mixed-methods approach to answer the research questions. Three questions guided the research: (1) How are the participant’s level of automation complacency and the correctness of the simulation that participant is using related?; (2) How is automation bias related to a participant’s ability to recognize errors in a simulation?; and (3) What factors explain the automation bias and automation complacency that the participants are experiencing? The third research question had two subquestions: (a) What factors explain the correlation between a participant’s level of automation complacency and the correctness of the simulation that participant is using?; and (b) What factors explain the impact that automation bias has on a participant’s ability to recognize errors in that simulation?
This study was based on the Theory of Technology Dominance, which states that people are more likely to rely on their technology the less experience they have with the task, the higher the complexity of the task, the lower the familiarity with the technology, and the further the technology is from the skillsets needed to solve the problem. This framework is built on the automation bias and automation complacency given by an individual towards technology. Automation bias is an overreliance on automation results despite contradictory information being produced by humans, while automation complacency is the acceptance of results from automation because of an unjustifiable assumption that the automation is working satisfactorily.
To ensure that the study could gather the information necessary, the mixed-study utilized deception techniques to divide participants into separate groupings. Four groupings were created, with some participants being given a properly functioning simulation with others being given a faulty simulation. Half of each grouping were informed that the simulation may have errors, while others were not. All participants who completed the study were debriefed about the real purpose of the study, but only after information had been gathered for analysis. The simulation given to all participants was designed to help students learn and practice the Method of Joints.
Students participating in the statics courses taught in the College of Engineering courses at Utah State University were invited to participate in the program over Spring and Fall semesters of 2022. Sixty-nine participants began the study, but only thirty-four remained in the study through to completion. Each participant took a pre-questionnaire, worked with a provided simulation that was either correct or incorrect, were possibly informed of potential errors in the simulation, and took a post-questionnaire. A few participants were invited to participate in an interview.
The findings of this study revealed that students often have high levels of automation bias and automation complacency. Participants changed their answers from wrong answers to right answers more often when using correct simulations and from right answer to wrong answers more often when using faulty simulations. The accuracy of each participant’s responses was also higher for those with correct simulations than faulty simulations. And most participants expressed that they checked their work and changed their answers when the simulation asked them to. These findings were confirmed through the use of the post-questionnaire results and in interview analysis between the groups
