Report from the STEM 2026 Workshop on Assessment, Evaluation, and Accreditation

A gathering of science, technology, engineering, and math (STEM) higher education stakeholders met in November 2018 to consider the relationship between innovation in education and assessment. When we talk about assessment in higher education, it is inextricably linked to both evaluation and accreditation, so all three were considered. The first question we asked was can we build a nation of learners? This starts with considering the student, first and foremost. As educators, this is a foundation of our exploration and makes our values transparent. As educators, how do we know we are having an impact? As members and implementers of institutions, programs and professional societies, how do we know students are learning and that what they are learning has value? The focus of this conversation was on undergraduate learning, although we acknowledge that the topic is closely tied to successful primary and secondary learning as well as graduate education. Within the realm of undergraduate education, students can experience four-year institutions and two-year institutions, with many students learning at both at different times. Thirty-seven participants spent two days considering cases of innovation in STEM education, learning about the best practices in assessment, and then discussing the relationship of innovation and assessment at multiple levels within the context of higher education. Six working groups looked at course-level, program-level, and institution-level assessment, as well as cross-disciplinary programs, large-scale policy issues, and the difficult-to-name “non-content/cross-content” group that looked at assessment of transferable skills and attributes like professional skills, scientific thinking, mindset, and identity, all of which are related to post-baccalaureate success. These conversations addressed issues that cut across multiple levels, disciplines, and course topics, or are otherwise seen as tangential or perpendicular to perhaps “required” assessment at institutional, programmatic, or course levels. This report presents the context, recommendations, and “wicked” challenges from the meeting participants and their working groups. Along with the recommendations of workshop participants, these intricate challenges weave a complex web of issues that collectively need to be addressed by our community. They generated a great deal of interest and engagement from workshop participants, and act as a call to continue these conversations and seek answers that will improve STEM education through innovation and improved assessment. This material is based upon work supported by the National Science Foundation under Grant No. DUE-1843775. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation

Report from the STEM 2026 Workshop on Assessment, Evaluation, and Accreditation

A gathering of science, technology, engineering, and math (STEM) higher education stakeholders met in November 2018 to consider the relationship between innovation in education and assessment. When we talk about assessment in higher education, it is inextricably linked to both evaluation and accreditation, so all three were considered. The first question we asked was can we build a nation of learners? This starts with considering the student, first and foremost. As educators, this is a foundation of our exploration and makes our values transparent. As educators, how do we know we are having an impact? As members and implementers of institutions, programs and professional societies, how do we know students are learning and that what they are learning has value? The focus of this conversation was on undergraduate learning, although we acknowledge that the topic is closely tied to successful primary and secondary learning as well as graduate education. Within the realm of undergraduate education, students can experience four-year institutions and two-year institutions, with many students learning at both at different times. Thirty-seven participants spent two days considering cases of innovation in STEM education, learning about the best practices in assessment, and then discussing the relationship of innovation and assessment at multiple levels within the context of higher education. Six working groups looked at course-level, program-level, and institution-level assessment, as well as cross-disciplinary programs, large-scale policy issues, and the difficult-to-name “non-content/cross-content” group that looked at assessment of transferable skills and attributes like professional skills, scientific thinking, mindset, and identity, all of which are related to post-baccalaureate success. These conversations addressed issues that cut across multiple levels, disciplines, and course topics, or are otherwise seen as tangential or perpendicular to perhaps “required” assessment at institutional, programmatic, or course levels. This report presents the context, recommendations, and “wicked” challenges from the meeting participants and their working groups. Along with the recommendations of workshop participants, these intricate challenges weave a complex web of issues that collectively need to be addressed by our community. They generated a great deal of interest and engagement from workshop participants, and act as a call to continue these conversations and seek answers that will improve STEM education through innovation and improved assessment. This material is based upon work supported by the National Science Foundation under Grant No. DUE-1843775. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation

Assessing Working Memory in Mild Cognitive Impairment with Serial Order Recall.

BACKGROUND: Working memory (WM) is often assessed with serial order tests such as repeating digits backward. In prior dementia research using the Backward Digit Span Test (BDT), only aggregate test performance was examined. OBJECTIVE: The current research tallied primacy/recency effects, out-of-sequence transposition errors, perseverations, and omissions to assess WM deficits in patients with mild cognitive impairment (MCI). METHODS: Memory clinic patients (n = 66) were classified into three groups: single domain amnestic MCI (aMCI), combined mixed domain/dysexecutive MCI (mixed/dys MCI), and non-MCI where patients did not meet criteria for MCI. Serial order/WM ability was assessed by asking participants to repeat 7 trials of five digits backwards. Serial order position accuracy, transposition errors, perseverations, and omission errors were tallied. RESULTS: A 3 (group)×5 (serial position) repeated measures ANOVA yielded a significant group×trial interaction. Follow-up analyses found attenuation of the recency effect for mixed/dys MCI patients. Mixed/dys MCI patients scored lower than non-MCI patients for serial position 3 (p \u3c 0.003) serial position 4 (p \u3c 0.002); and lower than both group for serial position 5 (recency; p \u3c 0.002). Mixed/dys MCI patients also produced more transposition errors than both groups (p \u3c 0.010); and more omissions (p \u3c 0.020), and perseverations errors (p \u3c 0.018) than non-MCI patients. CONCLUSIONS: The attenuation of a recency effect using serial order parameters obtained from the BDT may provide a useful operational definition as well as additional diagnostic information regarding working memory deficits in MCI

Partial Volume Correction in Quantitative Amyloid Imaging.

Amyloid imaging is a valuable tool for research and diagnosis in dementing disorders. As positron emission tomography (PET) scanners have limited spatial resolution, measured signals are distorted by partial volume effects. Various techniques have been proposed for correcting partial volume effects, but there is no consensus as to whether these techniques are necessary in amyloid imaging, and, if so, how they should be implemented. We evaluated a two-component partial volume correction technique and a regional spread function technique using both simulated and human Pittsburgh compound B (PiB) PET imaging data. Both correction techniques compensated for partial volume effects and yielded improved detection of subtle changes in PiB retention. However, the regional spread function technique was more accurate in application to simulated data. Because PiB retention estimates depend on the correction technique, standardization is necessary to compare results across groups. Partial volume correction has sometimes been avoided because it increases the sensitivity to inaccuracy in image registration and segmentation. However, our results indicate that appropriate PVC may enhance our ability to detect changes in amyloid deposition

White Matter Hyperintensities in Vascular Contributions to Cognitive Impairment and Dementia (VCID): Knowledge Gaps and Opportunities

White matter hyperintensities (WMHs) are frequently seen on brain magnetic resonance imaging scans of older people. Usually interpreted clinically as a surrogate for cerebral small vessel disease, WMHs are associated with increased likelihood of cognitive impairment and dementia (including Alzheimer\u27s disease [AD]). WMHs are also seen in cognitively healthy people. In this collaboration of academic, clinical, and pharmaceutical industry perspectives, we identify outstanding questions about WMHs and their relation to cognition, dementia, and AD. What molecular and cellular changes underlie WMHs? What are the neuropathological correlates of WMHs? To what extent are demyelination and inflammation present? Is it helpful to subdivide into periventricular and subcortical WMHs? What do WMHs signify in people diagnosed with AD? What are the risk factors for developing WMHs? What preventive and therapeutic strategies target WMHs? Answering these questions will improve prevention and treatment of WMHs and dementia

Comparing cortical signatures of atrophy between late-onset and autosomal dominant Alzheimer disease

Defining a signature of cortical regions of interest preferentially affected by Alzheimer disease (AD) pathology may offer improved sensitivity to early AD compared to hippocampal volume or mesial temporal lobe alone. Since late-onset Alzheimer disease (LOAD) participants tend to have age-related comorbidities, the younger-onset age in autosomal dominant AD (ADAD) may provide a more idealized model of cortical thinning in AD. To test this, the goals of this study were to compare the degree of overlap between the ADAD and LOAD cortical thinning maps and to evaluate the ability of the ADAD cortical signature regions to predict early pathological changes in cognitively normal individuals. We defined and analyzed the LOAD cortical maps of cortical thickness in 588 participants from the Knight Alzheimer Disease Research Center (Knight ADRC) and the ADAD cortical maps in 269 participants from the Dominantly Inherited Alzheimer Network (DIAN) observational study. Both cohorts were divided into three groups: cognitively normal controls (nADRC = 381; nDIAN = 145), preclinical (nADRC = 153; nDIAN = 76), and cognitively impaired (nADRC = 54; nDIAN = 48). Both cohorts underwent clinical assessments, 3T MRI, and amyloid PET imaging with either 11C-Pittsburgh compound B or 18F-florbetapir. To generate cortical signature maps of cortical thickness, we performed a vertex-wise analysis between the cognitively normal controls and impaired groups within each cohort using six increasingly conservative statistical thresholds to determine significance. The optimal cortical map among the six statistical thresholds was determined from a receiver operating characteristic analysis testing the performance of each map in discriminating between the cognitively normal controls and preclinical groups. We then performed within-cohort and cross-cohort (e.g. ADAD maps evaluated in the Knight ADRC cohort) analyses to examine the sensitivity of the optimal cortical signature maps to the amyloid levels using only the cognitively normal individuals (cognitively normal controls and preclinical groups) in comparison to hippocampal volume. We found the optimal cortical signature maps were sensitive to early increases in amyloid for the asymptomatic individuals within their respective cohorts and were significant beyond the inclusion of hippocampus volume, but the cortical signature maps performed poorly when analyzing across cohorts. These results suggest the cortical signature maps are a useful MRI biomarker of early AD-related neurodegeneration in preclinical individuals and the pattern of decline differs between LOAD and ADAD.Fil: Dincer, Aylin. Washington University in St. Louis; Estados UnidosFil: Gordon, Brian A.. Washington University in St. Louis; Estados UnidosFil: Hari-Raj, Amrita. Ohio State University; Estados UnidosFil: Keefe, Sarah J.. Washington University in St. Louis; Estados UnidosFil: Flores, Shaney. Washington University in St. Louis; Estados UnidosFil: McKay, Nicole S.. Washington University in St. Louis; Estados UnidosFil: Paulick, Angela M.. Washington University in St. Louis; Estados UnidosFil: Shady Lewis, Kristine E.. University of Kentucky; Estados UnidosFil: Feldman, Rebecca L.. Washington University in St. Louis; Estados UnidosFil: Hornbeck, Russ C.. Washington University in St. Louis; Estados UnidosFil: Allegri, Ricardo Francisco. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia; ArgentinaFil: Ances, Beau M.. Washington University in St. Louis; Estados UnidosFil: Berman, Sarah B.. University of Pittsburgh; Estados UnidosFil: Brickman, Adam M.. Columbia University; Estados UnidosFil: Brooks, William S.. Neuroscience Research Australia; Australia. University of New South Wales; AustraliaFil: Cash, David M.. UCL Queen Square Institute of Neurology; Reino UnidoFil: Chhatwal, Jasmeer P.. Harvard Medical School; Estados UnidosFil: Farlow, Martin R.. Indiana University; Estados UnidosFil: Fougère, Christian la. German Center for Neurodegenerative Diseases; Alemania. University Hospital of Tübingen; AlemaniaFil: Fox, Nick C.. UCL Queen Square Institute of Neurology; Reino UnidoFil: Fulham, Michael J.. Royal Prince Alfred Hospital; Australia. University of Sydney; AustraliaFil: Jack, Clifford R.. Mayo Clinic; Estados UnidosFil: Joseph-Mathurin, Nelly. Washington University in St. Louis; Estados UnidosFil: Karch, Celeste M.. Washington University in St. Louis; Estados UnidosFil: Lee, Athene. University Brown; Estados UnidosFil: Levin, Johannes. German Center for Neurodegenerative Diseases; Alemania. Ludwig Maximilians Universitat; Alemania. Munich Cluster for Systems Neurology; AlemaniaFil: Masters, Colin L.. University of Melbourne; AustraliaFil: McDade, Eric M.. Washington University in St. Louis; Estados UnidosFil: Oh, Hwamee. University Brown; Estados UnidosFil: Perrin, Richard J.. Washington University in St. Louis; Estados Unido

Comparing cortical signatures of atrophy between late-onset and autosomal dominant Alzheimer disease

Defining a signature of cortical regions of interest preferentially affected by Alzheimer disease (AD) pathology may offer improved sensitivity to early AD compared to hippocampal volume or mesial temporal lobe alone. Since late-onset Alzheimer disease (LOAD) participants tend to have age-related comorbidities, the younger-onset age in autosomal dominant AD (ADAD) may provide a more idealized model of cortical thinning in AD. To test this, the goals of this study were to compare the degree of overlap between the ADAD and LOAD cortical thinning maps and to evaluate the ability of the ADAD cortical signature regions to predict early pathological changes in cognitively normal individuals. We defined and analyzed the LOAD cortical maps of cortical thickness in 588 participants from the Knight Alzheimer Disease Research Center (Knight ADRC) and the ADAD cortical maps in 269 participants from the Dominantly Inherited Alzheimer Network (DIAN) observational study. Both cohorts were divided into three groups: cognitively normal controls (nADRC = 381; nDIAN = 145), preclinical (nADRC = 153; nDIAN = 76), and cognitively impaired (nADRC = 54; nDIAN = 48). Both cohorts underwent clinical assessments, 3T MRI, and amyloid PET imaging with either 11C-Pittsburgh compound B or 18F-florbetapir. To generate cortical signature maps of cortical thickness, we performed a vertex-wise analysis between the cognitively normal controls and impaired groups within each cohort using six increasingly conservative statistical thresholds to determine significance. The optimal cortical map among the six statistical thresholds was determined from a receiver operating characteristic analysis testing the performance of each map in discriminating between the cognitively normal controls and preclinical groups. We then performed within-cohort and cross-cohort (e.g. ADAD maps evaluated in the Knight ADRC cohort) analyses to examine the sensitivity of the optimal cortical signature maps to the amyloid levels using only the cognitively normal individuals (cognitively normal controls and preclinical groups) in comparison to hippocampal volume. We found the optimal cortical signature maps were sensitive to early increases in amyloid for the asymptomatic individuals within their respective cohorts and were significant beyond the inclusion of hippocampus volume, but the cortical signature maps performed poorly when analyzing across cohorts. These results suggest the cortical signature maps are a useful MRI biomarker of early AD-related neurodegeneration in preclinical individuals and the pattern of decline differs between LOAD and ADAD

Parent Report and Audio Recording Patterns in Bilingual Children Who Speak a Minority Language at Home

Bilingualism is when an individual is able to utilize at least two languages. If a person speaks two languages, it is beneficial for them both socially and for their brain development. This study focuses on the language input of sequential bilingual children in a bilingual Head Start program that promotes early childhood development and school readiness for low income families. There were six participants and their families in this study. To examine these children\u27s use of language, parental questionnaires and auditory recordings from the children\u27s day were utilized. Each family that participated in this study completed a questionnaire about their child which included family history and information on languages that are spoken at home. These parent reports gave insight to clinicians about the child directly from someone who was close to them. The audio recordings were retrieved from a device that the children were required to wear that depicted a typical day of language use. These recordings included linguistic information from both their school and home environments. Both the audio recordings and the parent reports were examined to compare data from a child\u27s day as recorded in the audio recordings to what the parent report stated. The results provides some information about children\u27s language input in L1 (Cantonese) and in L2 (English) at home and in school. The findings suggested that there are advantages and disadvantages of auditory recording and questionnaires. The results could be useful for clinicians and teachers who work with young bilingual children