Q Methodology as a Tool for Program Assessment

Program assessment is now commonplace at most colleges and universities and is required for accreditation of specific degree programs. Key aspects of program assessment include program improvement, improved student learning, and adequate student preparation for the workforce. Thus, program assessment is a key ingredient to program health. Although surveys are often used within program assessment in higher education, this study demonstrates the weaknesses of this method and, instead, introduces Q methodology as a means of program assessment especially in the area of needs assessment. In essence, Q offers an objective way to measure subjectivity about any topic. Unlike Likert-scale surveys, Q is a mixed method that reveals the multiple unique views as well as consensus within the group of participants. In this study, Q was used to determine views of a construction engineering technology program. How the results will be used to improve the program is presented

Divergent student views of cybersecurity

Cybersecurity is a worldwide issue and concern. Prior studies indicate that many people do not use cybersecurity best practices. Although these prior studies used large-scale surveys or interviews, this study used Q methodology [Q] because Q provides greater insight than Likert-format surveys. In fact, Q was created to scientifically study subjectivity. Within a Q study, various stages as well as philosophical, epistemological, and ontological principles represent a complete methodology. At first, Q researchers collect items that represent the broad range of communications about the topic (called the concourse). Although the items can be pictures, scents, or other means of communication, statements are the most common. Q researchers reduce the items of the concourse to create the Q-sample while preserving the range of communications. Subsequently, participants sort these items into a grid to provide a snapshot of their viewpoint on the topic. Statistical analysis reveals the multiple, diverse viewpoints in a way that allows for detailed descriptions of those views. In this study, the researchers collected statements about cybersecurity. Students in technical degree programs, including computer information systems (CIS), sorted these statements into a grid with a range of “most like my view” to “most unlike my view” of cybersecurity. Items placed on the extreme ends of this grid represent those statements most salient with each student’s views. Analyses revealed three divergent viewpoints: 1) Cybersecurity best practices, 2) No worries, and 3) No sense of urgency. Although the CIS majors identified with View 1, the other technical degree program students were represented across all three views. Certainly, students who hold the No worries and No sense of urgency viewpoints are unprepared to deal with cybersecurity issues in the workplace. The descriptions of these views have implications for cybersecurity course and program development, including assessments. Additionally, this study’s outcomes indicate a need to replicate this investigation in other settings to estimate risk of employees introducing cyber threats at their workplace. Similarly, these outcomes have implications for workforce development training regarding improved cybersecurity viewpoints and, therefore, behaviors

STEM High School Teachers’ Views of Implementing PBL: An Investigation Using Anecdote Circles

Problem-based learning (PBL) has been gaining in popularity, especially within the context of STEM-based (science, technology, engineering, and mathematics) schools. Program assessments for these schools typically focus on student standardized test scores rather than the needs of the teachers. This study utilized anecdote circles, storytelling via moderated group discussions, to investigate teachers’ needs related to developing and implementing authentic, interdisciplinary PBL activities in an urban, public STEM high school. Teacher experiences and viewpoints were explored within three broad themes: assessment; coaching and training; and authentic learning. The analyses provide insights for transitioning a school for effective PBL implementation as well as improving teaching and learning best practices. Recommendations to improve PBL practice based on the study’s results are also included

Strength Training: An Inquiry of Females Perceptions

Topics in Exercise Science and Kinesiology Volume 4: Issue 1, Article 10, 2023. While traditionally considered a male-dominated activity, women have increasingly embraced strength training. Although strength training has numerous physical and mental benefits to women, their subjective decision-making about using strength training have not been investigated thoroughly, especially using a unique methodology, Q, that is designed specifically to study subjectivity such as viewpoints and decision-making. There is limited research focused on females’ viewpoints about strength training. Capturing feelings and perceptions females have towards resistance training in an objective manner requires a specialized, mixed method called Q methodology [Q] that was created 90 years ago to scientifically study subjectivity. In Q, participants sort subjective statements about the topic into a grid to offer their internal viewpoints. Statistical analyses group similar sorts and provide empirical evidence to describe divergent viewpoints at a level usually reserved for qualitative studies. In this study, analyses revealed two distinct views: The Equipped Exercisers, who purposefully seek the benefits of exercise, and Road-blockers, who value exercise but implement constraints upon their participation. Understanding these views can facilitate expanding female participation in strength training through interventions

Inter-rater variability as mutual disagreement: identifying raters' divergent points of view

Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting 'idiosyncratic rater variance' is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical assessments have used open response formats to gather raters' comments and justifications. This design choice allows participants to use idiosyncratic response styles that could result in a distorted representation of the underlying rater cognition and skew subsequent analyses. In this study we explored rater variability using the structured response format of Q methodology. Physician raters viewed video-recorded clinical performances and provided Mini Clinical Evaluation Exercise (Mini-CEX) assessment ratings through a web-based system. They then shared their assessment impressions by sorting statements that described the most salient aspects of the clinical performance onto a forced quasi-normal distribution ranging from "most consistent with my impression" to "most contrary to my impression". Analysis of the resulting Q-sorts revealed distinct points of view for each performance shared by multiple physicians. The points of view corresponded with the ratings physicians assigned to the performance. Each point of view emphasized different aspects of the performance with either rapport-building and/or medical expertise skills being most salient. It was rare for the points of view to diverge based on disagreements regarding the interpretation of a specific aspect of the performance. As a result, physicians' divergent points of view on a given clinical performance cannot be easily reconciled into a single coherent assessment judgment that is impacted by measurement error. If inter-rater variability does not wholly reflect error of measurement, it is problematic for our current measurement models and poses challenges for how we are to adequately analyze performance assessment ratings

HbA(1c) Measured in Stored Erythrocytes Is Positively Linearly Associated with Mortality in Individuals with Diabetes Mellitus

Introduction: Observational studies have shown that glycated haemoglobin (HbA(1c)) is related to mortality, but the shape of the association is less clear. Furthermore, disease duration and medication may modify this association. This observational study explored the association between HbA(1c) measured in stored erythrocytes and mortality. Secondly, it was assessed whether disease duration and medication use influenced the estimates or were independently associated with mortality. Methods: Within the European Prospective Investigation into Cancer and Nutrition a cohort was analysed of 4,345 individuals with a confirmed diagnosis of diabetes at enrolment. HbA(1c) was measured in blood samples stored up to 19 years. Multivariable Cox proportional hazard regression models for all-cause mortality investigated HbA(1c) in quartiles as well as per 1% increment, diabetes medication in seven categories of insulin and oral hypoglycaemic agents, and disease duration in quartiles. Results: After a median follow-up of 9.3 years, 460 participants died. Higher HbA(1c) was associated with higher mortality: Hazard Ratio for 1%-increase was 1.11 (95% CI 1.06, 1.17). This association was linear (P-nonlinearity = 0.15) and persistent across categories of medication use, disease duration, and co-morbidities. Compared with metformin, other medication types were not associated with mortality. Longer disease duration was associated with mortality, but not after adjustment for HbA(1c) and medication. Conclusion: This prospective study showed that persons with lower HbA(1c) had better survival than those with higher HbA(1c). The association was linear and independent of disease duration, type of medication use, and presence of co-morbidities. Any improvement of HbA(1c) appears to be associated with reduced mortality risk