Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol

Citation: Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., . . . Cooper, M. M. (2016). Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol. Plos One, 11(9), 21. doi:10.1371/journal.pone.0162333Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not

Determination of an optimal response cut-off able to predict progression-free survival in patients with well-differentiated advanced pancreatic neuroendocrine tumours treated with sunitinib: an alternative to the current RECIST-defined response.

BACKGROUND: Sunitinib prolongs progression-free survival (PFS) in patients with advanced pancreatic neuroendocrine tumours (pNET). Response Evaluation Criteria in Solid Tumors (RECIST)-defined partial responses (PR; classically defined as ⩾30% size decrease from baseline) are infrequent. METHODS: Individual data of pNET patients from the phase II [NCT00056693] and pivotal phase III [NCT00428597] trials of sunitinib were analysed in this investigator-initiated, post hoc study. The primary objective was to determine the optimal RECIST (v.1.0) response cut-off value to identify patients who were progression-free at 11 months (median PFS in phase III trial); and the most informative time-point (highest area under the curve (AUC) by receiver operating characteristic (ROC) analysis and logistic regression) for prediction of benefit (PFS) from sunitinib. RESULTS: Data for 237 patients (85 placebo; 152 sunitinib (n=66.50 mg \u274-weeks on/2-weeks off\u27 schedule; n=86 \u2737.5 mg continuous daily dosing (CDD)\u27)) and 788 scans were analysed. The median PFS for sunitinib and placebo were 9.3 months (95% CI 7.6-12.2) and 5.4 months (95% CI 3.5-6.01), respectively (hazard ratio (HR) 0.43 (95% CI 0.29-0.62); P CONCLUSIONS: A 10% reduction within marker lesions identifies pNET patients benefiting from sunitinib treatment with implications for maintenance of dose intensity and future trial design

Evaluating the extent of a large-scale transformation in gateway science courses

We evaluate the impact of an institutional effort to transform undergraduate science courses using an approach based on course assessments. The approach is guided by A Framework for K-12 Science Education and focuses on scientific and engineering practices, crosscutting concepts, and core ideas, together called three-dimensional learning. To evaluate the extent of change, we applied the Three-dimensional Learning Assessment Protocol to 4 years of chemistry, physics, and biology course exams. Changes in exams differed by discipline and even by course, apparently depending on an interplay between departmental culture, course organization, and perceived course ownership, demonstrating the complex nature of transformation in higher education. We conclude that while transformation must be supported at all organizational levels, ultimately, change is controlled by factors at the course and departmental levels

Aligning Practice to Policies: Changing the Culture to Recognize and Reward Teaching at Research Universities

Recent calls for improvement in undergraduate education within STEM (science, technology, engineering, and mathematics) disciplines are hampered by the methods used to evaluate teaching effectiveness. Faculty members at research universities are commonly assessed and promoted mainly on the basis of research success. To improve the quality of undergraduate teaching across all disciplines, not only STEM fields, requires creating an environment wherein continuous improvement of teaching is valued, assessed, and rewarded at various stages of a faculty member’s career. This requires consistent application of policies that reflect well-established best practices for evaluating teaching at the department, college, and university levels. Evidence shows most teaching evaluation practices do not reflect stated policies, even when the policies specifically espouse teaching as a value. Thus, alignment of practice to policy is a major barrier to establishing a culture in which teaching is valued. Situated in the context of current national efforts to improve undergraduate STEM education, including the Association of American Universities Undergraduate STEM Education Initiative, this essay discusses four guiding principles for aligning practice with stated priorities in formal policies: 1) enhancing the role of deans and chairs; 2) effectively using the hiring process; 3) improving communication; and 4) improving the understanding of teaching as a scholarly activity. In addition, three specific examples of efforts to improve the practice of evaluating teaching are presented as examples: 1) Three Bucket Model of merit review at the University of California, Irvine; (2) Evaluation of Teaching Rubric, University of Kansas; and (3) Teaching Quality Framework, University of Colorado, Boulder. These examples provide flexible criteria to holistically evaluate and improve the quality of teaching across the diverse institutions comprising modern higher education

Moderators, Mediators, and Other Predictors of Risperidone Response in Children with Autistic Disorder and Irritability

Objective/Background: The National Institute of Mental Health (NIMH) Research Units on Pediatric Psychopharmacology (RUPP) Autism Network found an effect size of d = 1.2 in favor of risperidone on the main outcome measure in an 8-week double-blind, placebo-controlled trial for irritabilityin autistic disorder. This paper explores moderators and mediators of this effect. Method: Intention-to-treat (ITT) analyses were conducted with suspected moderators and mediators entered into the regression equations. MacArthur Foundation Network subgroup guidelines were followed in the evaluation of the results. Results: Only baseline severity moderated treatment response: Higher severity showed greater improvement for risperidone but not for placebo. Weight gain mediated treatment response negatively: Those who gained more weight improved less with risperidone and more with placebo. Compliance correlated with outcome for risperidone but not placebo. Higher dose correlated with worse outcome for placebo, but not risperidone. Of nonspecific predictors, parent education, family income, and low baseline prolactin positively predicted outcome; anxiety, bipolar symptoms, oppositional-defiant symptoms, stereotypy, and hyperactivity negatively predicted outcome. Risperidone moderated the effect of change in 5'-nucleotidase, a marker of zinc status, for which decrease was associated with improvement only with risperidone, not with placebo. Conclusion: The benefit–risk ratio of risperidone is better with greater symptom severity. Risperidone can be individually titrated to optimal dosage for excellent response in the majority of children. Weight gain is not necessary for risperidone benefit and may even detract from it. Socioeconomic advantage, low prolactin, and absence of co-morbid problems non-specifically predict better outcome. Mineral interactions with risperidone deserve further study

Cognitive Effects of Risperidone in Children with Autism and Irritable Behavior

Objective: The objective of this research was to explore the effects of risperidone on cognitive processes in children with autism and irritable behavior. Method: Thirty-eight children, ages 5-17 years with autism and severe behavioral disturbance, were randomly assigned to risperidone (0.5 to 3.5 mg/day) or placebo for 8 weeks. This sample of 38 was a subset of 101 subjects who participated in the clinical trial; 63 were unable to perform the cognitive tasks. A double-blind placebo-controlled parallel groups design was used. Dependent measures included tests of sustained attention, verbal learning, hand-eye coordination, and spatial memory assessed before, during, and after the 8-week treatment. Changes in performance were compared by repeated measures ANOVA. Results: Twenty-nine boys and 9 girls with autism and severe behavioral disturbance and a mental age ≥18 months completed the cognitive part of the study. No decline in performance occurred with risperidone. Performance on a cancellation task (number of correct detections) and a verbal learning task (word recognition) was better on risperidone than on placebo (without correction for multiplicity). Equivocal improvement also occurred on a spatial memory task. There were no significant differences between treatment conditions on the Purdue Pegboard (hand-eye coordination) task or the Analog Classroom Task (timed math test). Conclusion: Risperidone given to children with autism at doses up to 3.5 mg for up to 8 weeks appears to have no detrimental effect on cognitive performance

A phase II study of the bispecific antibody MDX-H210 (anti-HER2 × CD64) with GM-CSF in HER2+ advanced prostate cancer

The proto-oncogene HER2 presents a novel therapeutic target. We report results in 25 patients with HER2+ advanced prostate cancer treated with the bispecific antibody MDX-H210 15 μg m−2by intravenous infusion plus GM-CSF 5 μg kg−1day−1by subcutaneous injection for 4 days repeated weekly for 6 weeks. Patients with stable disease or better received further cycles of treatment until disease progression or study withdrawal. 1 patient received no treatment and 4 received less than 1 cycle and are included in the toxicity analysis only. Median duration of follow up was 105+ (range 21–188) days. Toxicity was generally NCI-CTG 0–2. There were 2 grade 4 adverse events (heart failure and dyspnoea) and 1 grade 3 event (allergic reaction) resulting in discontinuation of the study medication. There were 9 further grade 3 events not resulting in trial withdrawal. There were no treatment-related deaths. 7/20 (35%) evaluable patients had a >50% PSA response of median duration 128 (range 71–184+) days. 7/12 (58%) patients with evaluable pain had improvements in pain scores. The PSA relative velocity on therapy decreased in 15/18 (83%) assessable patients compared to pre-study. GM-CSF and MDX-H210 is active in hormone refractory prostate carcinoma with acceptable toxicity; further studies are warranted. © 2001 Cancer Research Campaign http://www.bjcancer.co

Characterizing college science instruction: The Three-Dimensional Learning Observation Protocol

The importance of improving STEM education is of perennial interest, and to this end, the education community needs ways to characterize transformation efforts. Three-dimensional learning (3DL) is one such approach to transformation, in which core ideas of the discipline, scientific practices, and crosscutting concepts are combined to support student development of disciplinary expertise. We have previously reported on an approach to the characterization of assessments, the Three-Dimensional Learning Assessment Protocol (3D-LAP), that can be used to identify whether assessments have the potential to engage students in 3DL. Here we present the development of a companion, the Three-Dimensional Learning Observation Protocol (3D-LOP), an observation protocol that can reliably distinguish between instruction that has potential for engagement with 3DL and instruction that does not. The 3D-LOP goes beyond other observation protocols, because it is intended not only to characterize the pedagogical approaches being used in the instructional environment, but also to identify whether students are being asked to engage with scientific practices, core ideas, and crosscutting concepts. We demonstrate herein that the 3D-LOP can be used reliably to code for the presence of 3DL; further, we present data that show the utility of the 3D-LOP in differentiating between instruction that has the potential to promote 3DL from instruction that does not. Our team plans to continue using this protocol to evaluate outcomes of instructional transformation projects. We also propose that the 3D-LOP can be used to support practitioners in developing curricular materials and selecting instructional strategies to promote engagement in three-dimensional instruction