Research Methodology: ChatGPT as a Tool for Coding 4,000 Dissertations

Abstract

This research study is in the process of examining over 4,000 Ed.D. dissertations collected from 56 doctoral programs nationwide. The long-term goal is to conduct a meta-analysis, but the project is currently focused on developing a methodology for large-scale data extraction. Initially, a graduate student was manually reviewing and coding each dissertation into a spreadsheet. Over the course of a year, this yielded data from approximately 400 dissertations. This data includes the population of the study (P-12 or higher education), research methodology (quantitative, qualitative, or mixed methods), chapter lengths, and the number of citations used. While this process was effective, the time to do so makes it impractical to analyze the full dataset. To address this, the researchers have explored using artificial intelligence to automate this process. Using ChatGPT, the data was extracted from the PDFs of the dissertations into tables that can be exported. To ensure the reliability and accuracy of using AI in this process, the ChatGPT-coded data was compared against the 400 manually coded dissertations. The initial findings suggest that AI-assisted coding can accurately extract such data while drastically reducing the time to do so. This presentation will discuss this methodology and findings