Educational Knowledge Brokerage and Mobilization: The Marshall Memo Case

The importance of intermediation between communities primarily engaged in research production and those primarily engaged in practice is increasingly acknowledged, yet our understanding of the nature and influence of this work in education remains limited. Accordingly, this study utilizes case study methodology and aspires to understand the activities and signature product (the Marshall Memo) of a particularly influential mediator of current educational research, news, and ideas: Mr. Kim Marshall. The article also examines the memo’s meaning to subscribing educators. Data analyses suggest subscribers greatly appreciate several aspects of the memo, which was found to draw from a wide range of source material that varies in terms of its research centredness and its practical implications

Review of Climate Change Adaptation in the Canadian North (Report): Identifying common themes, actions, and opportunities to improve access to adaptation knowledge

Throughout northern Canada, a variety of documents speaking to climate change adaptation have been published. These range from peer review articles over presentations, websites, and reports to online tools. This project represents an effort to tie all these together. A literature review, including academic as well as grey literature, was conducted and organized into a database. The over 300 records were analyzed for common themes, adaptation enablers and barriers, project types, and geographical distribution. Additionally, online knowledge brokering platforms were reviewed and analyzed for their effectiveness.repor

VLEngagement: A Dataset of Scientific Video Lectures for Evaluating Population-based Engagement

With the emergence of e-learning and personalised education, the production and distribution of digital educational resources have boomed. Video lectures have now become one of the primary modalities to impart knowledge to masses in the current digital age. The rapid creation of video lecture content challenges the currently established human-centred moderation and quality assurance pipeline, demanding for more efficient, scalable and automatic solutions for managing learning resources. Although a few datasets related to engagement with educational videos exist, there is still an important need for data and research aimed at understanding learner engagement with scientific video lectures. This paper introduces VLEngagement, a novel dataset that consists of content-based and video-specific features extracted from publicly available scientific video lectures and several metrics related to user engagement. We introduce several novel tasks related to predicting and understanding context-agnostic engagement in video lectures, providing preliminary baselines. This is the largest and most diverse publicly available dataset to our knowledge that deals with such tasks. The extraction of Wikipedia topic-based features also allows associating more sophisticated Wikipedia based features to the dataset to improve the performance in these tasks. The dataset, helper tools and example code snippets are available publicly at https://github.com/sahanbull/context-agnostic-engagemen

Dealing with multimodal assignments in writing centres

We can no longer confine literacy pedagogy to the realm of language alone, as we need to take into account the role of images and other modes of meaning-making in texts. Nowadays, the tasks set for students’ assignments in higher education often require complex multimodal competencies (Archer 2006). Many assignments use images as evidence, whilst other assignments are predominantly visual in nature, such as posters, storyboards, or assignments that include CD-roms or other media. New technologies also enable a range of possibilities for individuals creating documents, including variety in layout, image, color, typeface, sound. The challenge for writing centers is to train the tutors to utilize these technologies effectively themselves so that they can deal with the changing nature of assignments

A Critical Examination of How U.S. Schools Structure Inequalities in Science Learning Opportunities Based on Intersectional Student Backgrounds

For decades, U.S. education policy has focused on the persistent achievement gap based on race and class in public schools. Within this test-based accountability context, math and reading achievement have been prioritized, and students have experienced inequitable access to rigorous science learning opportunities. Some scholars have drawn on cultural reproduction theory to examine the relationship between student background and achievement without accounting for the role of U.S. schools in structuring differential access to learning opportunities. This study aims to fill a gap in the literature by employing a critical quantitative lens and intersectional framework to examine how school structures, norms, and instructional practices contribute to stratification and systematic inequality in schools based on student background, shifting the focus from the achievement gap to the opportunity gap in U.S. schools. Using nationally representative U.S. data from PISA 2015, this dissertation employs latent class analysis (LCA) with auxiliary variables to examine the relationship between intersectional student background profiles, student sense of belonging, and student learning opportunities in science for 15-year-olds. A structural equation model (SEM) is used to extend these findings by examining potential mediators of intersectional student background and science achievement – opportunity to learn (OTL), sense of belonging, and student perceptions of academic climate – to account for inequitable learning environments in schools. Multilevel structural equation modeling (SEM) is then used to analyze science learning opportunities and academic press as mediators of intersectional student background and scientific literacy outcomes, as well as the school norms and structures that contribute to these experiences and outcomes. The findings from these studies revealed systemic inequality highlighted by a wealth gap between intersectional background groups of similar affluence based on parent occupational status and education. Further, gender disparities in OTL, sense of belonging to school, perceptions of academic climate, and scientific literacy outcomes consistently emerged across studies. Academic press was identified as an important mediator of student background and science achievement, and was a negative predictor of scientific literacy outcomes. Finally, while academic tracking predicted school mean academic press and OTL, school-level academic climate predicted school mean science achievement. However, there were significant differences in school-level academic climate between school contexts, pointing to a potential focal area to improve equity in schools. By identifying malleable school structures, norms, and instructional practices that shape students’ educational experiences and subsequent outcomes, this study provides potential policy levers for addressing concerns about equity in science education, including gaps in science opportunity to learn, engagement, achievement, and postsecondary outcomes

Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature

The quickly-expanding nature of published medical literature makes it challenging for clinicians and researchers to keep up with and summarize recent, relevant findings in a timely manner. While several closed-source summarization tools based on large language models (LLMs) now exist, rigorous and systematic evaluations of their outputs are lacking. Furthermore, there is a paucity of high-quality datasets and appropriate benchmark tasks with which to evaluate these tools. We address these issues with four contributions: we release Clinfo.ai, an open-source WebApp that answers clinical questions based on dynamically retrieved scientific literature; we specify an information retrieval and abstractive summarization task to evaluate the performance of such retrieval-augmented LLM systems; we release a dataset of 200 questions and corresponding answers derived from published systematic reviews, which we name PubMed Retrieval and Synthesis (PubMedRS-200); and report benchmark results for Clinfo.ai and other publicly available OpenQA systems on PubMedRS-200.Comment: Preprint of an article published in Pacific Symposium on Biocomputing copyright 2024 World Scientific Publishing Co., Singapore, http://psb.stanford.edu

Science-Practice Gap: Does Innovative Academic Knowledge Diffuse?

It has been noted that few practitioners read academic research (Rynes, Colbert, & Brown, 2002). It has also been noted that the gap has grown so wide that science and practice are now specialized autonomous systems operating in isolation from each other (Siedl, 2005). Hambrick (1994) believes that science operates in such isolation that it is a closed loop. This closed loop presents challenges with science effectively communicating with practice (Kieser & Leiner, 2009). As most in the field recognize this as an issue (Hambrick, 1994; Siedl, 2005; Van De Ven, 2007; Kieser & Leiner, 2009) there seems to be a feeling among certain academics that the situation is simply the new norm of academic life (Siedl, 2005; Kieser & Leiner, 2009).While I embrace the fact that the two systems have fundamental differences, this should not preclude some level of communication between the two. In fact, we know that some robust and rigorous scholarship is effectively communicated to practitioners. I hold that once we understand the anatomy of communicative scholarship, we can begin to close the gap more effectively.By framing scholarship as an innovation (Rogers, 2010), I can investigate the underlying components, (i.e., relative advantage, compatibility, complexity, trialability, and observability) of academic research. With this framework, I will contrast the profiles of adopted research with non-adopted research. Subsequent to this, among the adopted research, I evaluate the impact of the components on the time and extent of diffusion

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity

Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided by empirically unsupported intuitions. To address this, we pretrain 28 1.5B parameter decoder-only models, training on data curated (1) at different times, (2) with varying toxicity and quality filters, and (3) with different domain compositions. First, we quantify the effect of pretraining data age. A temporal shift between evaluation data and pretraining data leads to performance degradation, which is not overcome by finetuning. Second, we explore the effect of quality and toxicity filters, showing a trade-off between performance on standard benchmarks and risk of toxic generations. Our findings indicate there does not exist a one-size-fits-all solution to filtering training data. We also find that the effects of different types of filtering are not predictable from text domain characteristics. Lastly, we empirically validate that the inclusion of heterogeneous data sources, like books and web, is broadly beneficial and warrants greater prioritization. These findings constitute the largest set of experiments to validate, quantify, and expose many undocumented intuitions about text pretraining, which we hope will help support more informed data-centric decisions in LM development