Impact of Retrieval Augmented Generation and Large Language Model Complexity on Undergraduate Exams Created and Taken by AI Agents

The capabilities of large language models (LLMs) have advanced to the point where entire textbooks can be queried using retrieval-augmented generation (RAG), enabling AI to integrate external, up-to-date information into its responses. This study evaluates the ability of two OpenAI models, GPT-3.5 Turbo and GPT-4 Turbo, to create and answer exam questions based on an undergraduate textbook. 14 exams were created with four true-false, four multiple-choice, and two short-answer questions derived from an open-source Pacific Studies textbook. Model performance was evaluated with and without access to the source material using text-similarity metrics such as ROUGE-1, cosine similarity, and word embeddings. Fifty-six exam scores were analyzed, revealing that RAG-assisted models significantly outperformed those relying solely on pre-trained knowledge. GPT-4 Turbo also consistently outperformed GPT-3.5 Turbo in accuracy and coherence, especially in short-answer responses. These findings demonstrate the potential of LLMs in automating exam generation while maintaining assessment quality. However, they also underscore the need for policy frameworks that promote fairness, transparency, and accessibility. Given regulatory considerations outlined in the European Union AI Act and the NIST AI Risk Management Framework, institutions using AI in education must establish governance protocols, bias mitigation strategies, and human oversight measures. The results of this study contribute to ongoing discussions on responsibly integrating AI in education, advocating for institutional policies that support AI-assisted assessment while preserving academic integrity. The empirical results suggest not only performance benefits but also actionable governance mechanisms, such as verifiable retrieval pipelines and oversight protocols, that can guide institutional policies

Influence of Personnel Factors on Air Force Fighter Mishaps via Bayesian Regression

Aviation safety in the United States (U.S.) military has received growing attention in recent years due to numerous high-profile mishaps. Despite the increased attention, there have been few quantitative analyses of the relationship between pilot attributes and mishap rates. In this study, we use nearly 15 years of U.S. Air Force (USAF) safety and administrative records to investigate the relationship between pilot attributes and fighter aviation mishap rates. First, we present an analysis of flight mishap rates for different mishap classes and fighter aircraft types, referred to as a mission design series (MDS). Second, we quantify pilot attributes and present an analysis of fighter pilot populations across time and MDS. We then model the association between pilot attributes and annual rate of class A, B, and C flight mishaps, which we refer to as high-class mishaps (HCMs), using a Bayesian regression framework. Our results show prior flight experience and key characteristics of an MDS pilot community are associated with the rate of HCMs. Specifically, we find that MDS pilot communities with 10 more flight hours in the past year are, on average, associated with a 5% decrease in HCM rate. Additionally, we find that a 0.1 standard deviation increase in the proportion of pilots who are instructor pilots, distinguished graduates from commissioning source, and graduate degree recipients is associated with a reduction in major aviation mishaps by 2.1%, 2.0%, and 1.3%, respectively. These findings have significant financial implications, given that the cost of a single HCM starts at $50K and can be as high as$200M. In addition to our model results, our efforts to quantify pilot attributes and model the relationship between personnel factors and mishap rates using Bayesian regression and predictive projection for feature selection represent a valuable methodological contribution to aviation accident analysis

Factor-Graph Optimization for Robust Navigation via High-Precision GNSS/IMU Corroboration

Excerpt: This paper proposes an approach for resilient navigation through a factor-graph formulation that incorporates high-quality IMU measurements into a robust GNSS factor-graph formulation. The approach effectively enables the corroboration of GNSS measurements based on their consistency with the precise relative-motion information available from a high-quality IMU

AFIT Generative AI Teaching Guidebook

AFIT is proud to highlight the Generative AI Teaching Guidebook, a resource designed to provide military educators with practical insights, strategies, and use cases for integrating Generative AI (Gen AI) into their teaching practices. Developed through a collaborative effort involving AFIT faculty across various departments within the Graduate School of Engineering and Management and the School of Systems and Logistics, this digital resource serves as a starting point for educators exploring how to leverage Gen AI in their classrooms. It offers accessible examples and best practices, ensuring utility for instructors of all technical backgrounds. The guidebook provides a comprehensive overview of how Gen AI can enhance education, offering actionable use cases, illustrative examples, and best practices tailored to diverse teaching environments. By addressing both opportunities and challenges—such as ethical considerations and data privacy—it empowers educators to design meaningful learning experiences while fostering discussions about AI\u27s potential and limitations. The release of this guidebook comes at a key moment as the adoption of Gen AI in education accelerates. It highlights how these tools can be integrated into both traditional academic settings and professional continuing education, such as utilizing Gen AI for educational simulations or modeling in systems engineering. By fostering critical thinking, innovation, and ethical awareness, the Generative AI Teaching Guidebook empowers educators to prepare students for an AI-driven world while advancing AFIT’s defense-focused educational mission

Global Sporadic-E Prediction and Climatology Using Deep Learning

Sporadic-E (Es) is an ionospheric phenomenon defined by strong layers of plasma which may interfere with radio wave propagation. In this work, we develop deep learning models to improve the understanding of Es, including the presence, intensity and height of the layers. We developed three separate models. The first, building off earlier work in (J. A. Ellis et al., 2024, link in AFIT Scholar, 10.1029/2023sw003669), includes only the main features from radio occultation (RO) measurements. The second adds to that time, date, location, geomagnetic and solar indices, solar winds, x-ray flux, weather and lightning. A third model excludes RO measurements but includes the rest. In training the first two models, the Es ordinary mode critical frequency (foEs), a measure of intensity, and height (hEs) parameters extracted from ionosondes were used as the “ground truth” target variables. In training the third model, estimated foEs and hEs values from the RO model were added as target variables to augment the data set and produce physically reasonable model predictions globally. We find that the second model performs well with intensity prediction tasks, but struggles with height estimations, which is likely due to the tangent point assumption made during RO signal processing and errors inherent in the ionosonde extracted virtual heights. The third model performed reasonably well considering the lack of in situ RO measurement. The third model performs the best on height predictions, which points to the height being very climatologically driven, whereas the intensity is a more complex interaction of several variables

Book Talk on Brief: \u3ci\u3eMake a Bigger Impact by Saying Less\u3c/i\u3e by Jonathan McCormack: Ten Key Concepts and Six Lessons for Improving Communication in a World Full of Distractions

This Lunch and Learn presentation, delivered by Wheeler Hall from the D’Azzo Research Library (DRL) and hosted by the Center for Innovation in Education (CIE), focuses on the principles outlined in Joseph McCormack’s Brief: Make a Bigger Impact by Saying Less (2014). The session serves as a summary for the book and explores strategies for effective communication in a distraction-filled world, emphasizing brevity, clarity, and audience engagement. After an introduction from Jonathan Zemmer (CIE), Mr. Hall relays McCormack’s background, the practical workplace applications in Brief, and key concepts from the book, including the BRIEF mnemonic (Background, Reason, Information, Ending, Follow-up), the seven capital sins of communication, and the importance of storytelling. The presentation concludes with six lessons for refining communication, including thorough research, concise delivery, and respect for others’ time. Viewers are encouraged to borrow copies of the book from the library to further enhance their communication skills, applicable in both professional and personal contexts

Performance Evaluation of Utilizing Rust for PCAP Analysis in Satellite Cybersecurity

Previously, launched satellites were not designed with the necessary resource capacity or safety protocols to integrate essential Intrusion Detection Systems (IDS). This paper proposes the use of Rust to develop a statistics-based IDS, leveraging the language’s fast, efficient, and memory-safe attributes. The paper begins by providing an overview of cybersecurity threats to space infrastructure, introducing the fundamentals of intrusion detection, and outlining the architecture of space systems as background knowledge. It then details the proposed methodology for using Rust to build a statistics-based IDS. By comparing this approach with traditional methods, such as Python’s pandas, the paper aims to evaluate Rust’s speed and efficiency, advocating for its adoption in the development of more secure space systems

Structural Index Parameter for Capturing Structural & Aerothermal Effects in Conceptual Level Vehicle Design

The three phases of vehicle conceptual design include parametric sizing, configuration layout, and configuration evaluation. During the parametric sizing phase, the ability to define and quantify the technology level of an aerospace system allows the assessment of candidate designs based on feasibility given current technology or indicates if one must advance a particular technology. To meet this need, the structural Index (Istr) parameter merits exploration to consider structural and aerothermal effects during the parametric sizing phase of conceptual design given materials, structural concepts, and manufacturing capability. This study showcases the utility of this structural/materials technology parameter for high-speed vehicles by modernizing and expanding upon Paul Czysz\u27s original structural index (Istr) versus the surface temperature map. The modernized and expanded structural index (Istr) map is constructed by selecting a temperature-through-thickness method for a given thermal protection system (TPS) that simplifies a given surface temperature and atmospheric pressure profile into a constant heat pulse. One can then size the TPS to keep the structural temperature within material limits. The newly generated structural index (Istr) maps allow one to observe trends with variations in surface temperature, cruise time, average atmospheric pressure (Pavg), and TPS materials

Magnetic Field Variability as a Consistent Predictor of Solar Flares

Solar flares are intense bursts of electromagnetic radiation that occur due to a rapid destabilization and reconnection of the magnetic field. While preflare signatures and trends have been investigated from magnetic observations prior to flares for decades, analysis that characterizes the variability of the magnetic field in the hours prior to flare onset has not been included in the literature. Here, the 3D magnetic field is modeled using a nonlinear force-free field extrapolation for 6 hr before and 1 hr after 18 on-disk solar flares and flare quiet windows for each active region. Parameters are calculated directly from the magnetic field from two field isolation methods: the “active region field,” which isolates field lines where the photospheric field magnitude is ≥200 Gauss, and the “high current region,” which isolates field lines in the 3D field where the current, nonpotential field, twist, and shear exceed predefined thresholds. For this small pool of clean events, there is a significant increase in variation starting 2–4 hr before flare onset for the current, twist, shear, and free energy, and the variation continues to increase through the flare start time. The current, twist, shear, and free energy are also significantly stronger through the lower corona and their separation from flare quiet height curves scales with flare strength. Methods are proposed to combine variation of the magnetic fields with variation of other data products prior to flare onset, suggesting a new potential flare prediction capability