Effects of Structural Flaws on the Psychometric Properties of Multiple-Choice Questions

The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students. Adviser: Allen Steckelber

Feasibility of a Readiness Exam for Predicting Radiography Program Success: A Pilot Study

ABSTRACT Background: Research investigating predictors of academic success in rigorous health science education is valuable for curricular intervention for identified at-risk students. Various predictors of success have been investigated, but the literature is insufficient when examining anatomy and physiology readiness scores as they correlate to radiography curricular success. This pilot study assessed the correlation between readiness exam scores and programmatic course GPA to determine if the scores could be used as a metric for identifying academic success resources for incoming students. Cohorts of the radiography program at a midwestern health sciences center demonstrated a longitudinal trend of difficulty with anatomy and physiology programmatic coursework. Therefore, researchers set out to investigate whether or not readiness exam scores, in addition to the metrics they were already utilizing, could be used as a tool for early academic remediation. Objective: The objective of this study was to determine if the anatomy and physiology readiness exam scores would be reliable indicators of programmatic success in anatomy and physiology program coursework. Design: This investigation occurred in two phases: a retrospective correlational phase and a quasi-experimental phase. Methods: Retrospective data from cohorts that matriculated between 2013 and 2017 (n=91) was collected and de-identified. Data included prerequisite grade point average (GPA) and grades from anatomy and physiology course taken during the program. During the quasi-experimental phase, a sample of students (n=18) completed a readiness examination. The scores from this examination were correlated with prerequisite GPA and program anatomy and physiology GPA. Results: Data analysis revealed prerequisite GPA and the anatomy and physiology section of the readiness examination to be strong and moderate predictors of programmatic anatomy and physiology course grades, respectively. Conclusion: Predictors of curricular success in a radiography program’s anatomy and physiology coursework are essential factors to consider in relation to admissions practices, curricular prerequisite standards, and on-boarding of new students, especially those identified as at-risk

Student Builders of Online Curriculum Content. What Are Their Perceptions and Motivations?

Objective/Purpose: 1. Describe the benefits and challenges of establishing a campus-wide student-centered program to create online curricular content. 2. Gain insight about student perceptions and motivations for becoming curriculum content developers. Need for Innovation: Faculty receive constant encouragement to discover methods for transforming health science instructional materials using active learning. Faculty identify lack of time as the most significant barrier to developing blended and online materials. In 2014, our academic medical center issued a student-centered call for proposals to create online materials for the Interactive E-Learning Program as a means to include students in the curricular change process. By pairing student creators, many of whom are fluent consumers of technology and digital education, with content expert faculty, the program uniquely capitalized on the students desire to contribute with the identified need of faculty for production time. Instructional Methods/Materials Used: The first call for student proposals was conducted in the fall of 2014, with the second in the fall of 2015. The competitive application required the following information: project members, faculty advisors, learning objectives, budget worksheet, description of the proposed module, and implementation plans for the curriculum. Awardees received $1000 per project for project expenses or student stipends. Students developed modules in six months with guidance from faculty advisors using resources available in the campus e-learning studio. Program completion was recognized by a letter of commendation. During the summer of 2016, students from both cohorts took part in a survey to assess their motivations and perceived benefits of participating in the Interactive E-Learning Program. Educational Outcomes: The two calls for proposals resulted in 30 funded student projects covering curricular topics for medicine, nursing, pharmacy, dentistry, allied health, and public health. A total of 58 students were involved and worked solo or in groups of 2-4 people. Survey results indicated students were motivated to participate by the desire to contribute to or enhance the curriculum and to develop their own e-learning skills. Students perceived an enhanced relationship with their faculty advisors and increased e-learning skills to be benefits of the program. Students also perceived participation in the program as a positive addition to their CV. Strengths/Areas for Improvement: The energy and creativity with which students approached the modules they designed resulted in many novel projects. The centralized e-learning studio and instructional design staff are strengths of the program, ensuring the modules meet instructional design principles, university branding requirements, copyright guidelines, and accessibility rules. An internal rubric and project checklist provided a guide for development and served as a tool for evaluation and feedback. Areas for improvement include the development of more interprofessional student teams that can lead to content applicable in more than one curriculum (eg professionalism skills, vital signs, medical history, etc). Approaches for identifying opportunities to implement the student-developed materials into courses on campus is also under consideration. Feasibility of Program Maintenance/Transferability: Many health sciences programs are facing curriculum redesign to meet the learning needs of current and future students or are seeking ways to engage students in the learning process. Based on our findings, extending the opportunity to students to become creators in partnership with faculty content experts was highly successful. Students are now more than just learners, but are also demonstrating altruistic behaviors to contribute and enhance curriculum for future students. After initial expenditures are made to acquire the proper software and hardware needed to support a formal e-learning program, the cost of maintaining such equipment is of small consequence when considering the impact of this program for the student developers, their faculty mentors, and the entire campus community

Spouses, Children, and the Pursuit of a Doctorate: Real or Perceived Barriers for Women in Academia

A group of six women, each of them working full-time while pursuing a doctoral degree, began meeting every other week in the fall of 2017. Their backgrounds were unique, and their programs varied, but their goal was the same: to complete their dissertations. Along the way, these women supported each other through writing challenges, committee woes, and balancing the demands of work, life, and the dissertation writing process. A proven model for success, the interdisciplinary writing group these women formed was based on a few basic principles: respect and positive regard for each other’s work, trust that the frustrations shared during meetings were kept confidential, and accountability to each other for the writing goals they set each week. Experts from campus spoke with the group about topics such as: authorship guidelines, research resources, turning the dissertation into publication, life after graduation, and technology tools. The goal of this Ignite Session is to unpack the barriers that may be unique to women and discuss potential strategies for supporting women during these challenges. These personal, vulnerable conversations are timely and significant because more and more women are in pursuit of a doctoral degree. In 2015, 46% of all U.S. doctorates were awarded to women (National Science Foundation). Leonard (2001) offers that women, more so than their male counterparts, face a balancing act between family and scholarship commitments. Moreover, Brown and Watson (2010) report that women tend to time their pursuit of a doctoral degree based upon domestic demands. Wall (2008) adds that for female doctoral students, juggling the demands of professional scholarship and personal life often requires them to make choices between the head and heart. This emotional struggle can feed into self-doubt and bruise self-efficacy for some women, which may slow or entirely halt their progress through a doctoral program. Research culture tends to be deeply founded on objectivity and autonomy by overlooking emotions (Johnson, Lee, & Green, 2000), but channeling emotions via a support group may be a successful strategy for many women. According to Mewburn (2011), many women actively find or form supportive groups where they can engage in honest dialogue to minimize emotional dissonance and share personal feelings and emotions. Historically, dissertation writing has been viewed as an independent venture; often one without external guidance and support. The notion of writing groups as they pertain to the dissertation stage have only recently been recognized as an acceptable support mechanism. Maher, Fallucca, & Mulhern Halasz, (2013) contend that dissertation writing groups can result in the development of a vibrant, intellectual community where commitment to degree completion and scholarly productivity soar. This implication is echoed by the session presenters. Support groups such as the one described by the presenters serve to hold participants accountable to writing outcomes; but potentially more important, a supportive group can dually function as a safe space for doctoral students to find support through the more sensitive challenges of writing the dissertation. To achieve the objectives of the session, presenters will begin by engaging attendees in a discussion of the unique challenges women who are pursuing an advanced degree face. Secondly, group facilitators will focus conversation on the construction and success of the writing group represented and solicit success stories from other attendees with similar approaches. Last, specific strategies from the writing group experience and from the collective experiences of attendees will be cultivated in order to deepen understanding of approaches that each attendee can bring back to his or her campus community for action. It is the ultimate goal of the presenters to validate the fears and concerns commonly shared by women in academia who aspire to develop through formal educational opportunities or through professional development activities and to equip them with multiple strategies for success as individuals and as members of a campus community. This equipment is imperative for faculty affairs officers, faculty developers, and other stakeholders at major medical centers because the success of our female faculty and staff members may well depend on proven, achievable strategies for supporting them through the juggling act they perceive as a significant barrier to achievement

Thrive: Success Strategies for the Modern-Day Faculty Member

The THRIVE collection is intended to help faculty thrive in their roles as educators, scholars, researchers, and clinicians. Each section contains a variety of thought-provoking topics that are designed to be easily digested, guide personal reflection, and put into action. Please use the THRIVE collection to help: Individuals study topics on their own, whenever and wherever they want Peer-mentoring or other learning communities study topics in small groups Leaders and planners strategically insert faculty development into existing meetings Faculty identify campus experts for additional learning, grand rounds, etc. If you have questions or want additional information on a topic, simply contact the article author or email [email protected]://digitalcommons.unmc.edu/facdev_books/1000/thumbnail.jp

Effects of Structural Flaws on the Psychometric Properties of Multiple-Choice Questions

The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students. Adviser: Allen Steckelber

Effects of Structural Flaws on the Psychometric Properties of Multiple-choice Questions

The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students

Effects of Structural Flaws on the Psychometric Properties of Multiple-choice Questions

The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students

The Nebraska Educator, Volume 1: 2014 (complete issue)

The Nebraska Educator is an open access peer-reviewed academic education journal at the University of Nebraska-Lincoln. This journal is produced by UNL graduate students and publishes articles on a broad range of education topics that are timely and have relevance in the field of all levels of education. We seek original research that covers topics which include but are not limited to: (a) curriculum, teaching and professional development; (b) education policy, practice and analysis; (c) literacy, language and culture; (d) school, society and reform; and (e) teaching and learning with technologies. Urban Immersion: Changing Pre-service Teachers\u27 Perceptions of Urban Schools -- Connie Schaffer, Deborah Gleich-Bope and Cindy B Copich 4 Exploring Public Middle School English Language Learner Teachers\u27 Grammar Teaching: An Instrumental Case Study -- Sevda Budak 32 Eyes Never Lie: Eye-tracking Technology Reveals how Students Study Displays -- Linlin Luo, Markeya S Peteranetz, Abraham E Flanigan, Amanda L Witte, and Kenneth A Kiewra 60 Faculty Perceptions to Imposed Pedagogical Change: A Case Study -- Mary L Sinclair and Sarah R Faltin Osborn 78 Finding the Connections Between Art and Teaching: A Case Study -- Tareq Daher and Stephanie A Baer 101 Transforming Field Experiences to Create Authentic Teaching Opportunities -- Connie Schaffer and Kelly Welsh 116 Currere and the Beauty of Soulful Classroom Moments -- Jessica Sierk 135 Teaching with Poetic Insight: A Practitioner Reflection and Dream of Possibility -- Dorothy Marie Bossma