MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education

Since the introduction of the original BERT (i.e., BASE BERT), researchers have developed various customized BERT models with improved performance for specific domains and tasks by exploiting the benefits of transfer learning. Due to the nature of mathematical texts, which often use domain specific vocabulary along with equations and math symbols, we posit that the development of a new BERT model for mathematics would be useful for many mathematical downstream tasks. In this resource paper, we introduce our multi-institutional effort (i.e., two learning platforms and three academic institutions in the US) toward this need: MathBERT, a model created by pre-training the BASE BERT model on a large mathematical corpus ranging from pre-kindergarten (pre-k), to high-school, to college graduate level mathematical content. In addition, we select three general NLP tasks that are often used in mathematics education: prediction of knowledge component, auto-grading open-ended Q&A, and knowledge tracing, to demonstrate the superiority of MathBERT over BASE BERT. Our experiments show that MathBERT outperforms prior best methods by 1.2-22% and BASE BERT by 2-8% on these tasks. In addition, we build a mathematics specific vocabulary 'mathVocab' to train with MathBERT. We discover that MathBERT pre-trained with 'mathVocab' outperforms MathBERT trained with the BASE BERT vocabulary (i.e., 'origVocab'). MathBERT is currently being adopted at the participated leaning platforms: Stride, Inc, a commercial educational resource provider, and ASSISTments.org, a free online educational platform. We release MathBERT for public usage at: https://github.com/tbs17/MathBERT.Comment: Accepted by NeurIPS 2021 MATHAI4ED Workshop (Best Paper

The Impact of PISA on Students' Learning: a Chinese Perspective

This thesis investigates PISA’s impact on students’ learning in a local context, Fangshan District of Beijing, in China. PISA’s growing influence on educational policymaking in domestic education systems has been widely discussed, but concerns about policy borrowing and PISA’s governing power on education have been raised. However, these discussions rarely look further into local contexts to investigate PISA’s impact on students’ learning. Through mixed methods research, this thesis presents an investigation into the impact PISA has had on students’ learning and how this impact occurs. A conceptual framework employing theories about washback effect and ecological systems theory was developed to underpin the research. Sixteen local educational policymakers and practitioners were interviewed to identify how PISA is used in the local context, and to gather their views on the perceived impact of PISA on students’ learning. Fangshan PISA data were used to triangulate their perceptions, and also to expand the understanding of their use and translation of PISA data in policymaking. Thematic analysis of interview data reveals that PISA is perceived as a new perspective, which is different from domestic assessment in some aspects, for benchmarking the quality of local education. PISA is also used as an impetus motivating local initiatives to improve educational quality, in which some PISA concepts are integrated. The mechanism of PISA’s impact on students’ learning was conceptualised via making the factors of different contextual layers which negotiate PISA’s impact explicit. Interviewees perceived that through school enactment of the local PISA-motivated initiatives, and reforms of national curriculum and assessment, to some extent, students’ learning has been gradually fostered. These perceptions are largely supported by the trend analysis and multilevel modelling of PISA data. Research findings also indicate the challenges that the local PISA data users face for appropriately interpreting and translating PISA data to inform educational policymaking

Enacting the Semantic Web: Ontological Orderings, Negotiated Standards, and Human-machine Translations

Artificial intelligence (AI) that is based upon semantic search has become one of the dominant means for accessing information in recent years. This is particularly the case in mobile contexts, as search based AI are embedded in each of the major mobile operating systems. The implications are such that information is becoming less a matter of choosing between different sets of results, and more of a presentation of a single answer, limiting both the availability of, and exposure to, alternate sources of information. Thus, it is essential to understand how that information comes to be structured and how deterministic systems like search based AI come to understand the indeterminate worlds they are tasked with interrogating. The semantic web, one of the technologies underpinning these systems, creates machine-readable data from the existing web of text and formalizes those machine-readable understandings in ontologies. This study investigates the ways that those semantic assemblages structure, and thus define, the world. In accordance with assemblage theory, it is necessary to study the interactions between the components that make up such data assemblages. As yet, the social sciences have been slow to systematically investigate data assemblages, the semantic web, and the components of these important socio-technical systems. This study investigates one major ontology, Schema.org. It uses netnographic methods to study the construction and use of Schema.org to determine how ontological states are declared and how human-machine translations occur in those development and use processes. This study has two main findings that bear on the relevant literature. First, I find that development and use of the ontology is a product of negotiations with technical standards such that ontologists and users must work around, with, and through the affordances and constraints of standards. Second, these groups adopt a pragmatic and generalizable approach to data modeling and semantic markup that determines ontological context in local and global ways. This first finding is significant in that past work has largely focused on how people work around standards’ limitations, whereas this shows that practitioners also strategically engage with standards to achieve their aims. Second, the particular approach that these groups use in translating human knowledge to machines, differs from the formalized and positivistic approaches described in past work. At a larger level, this study fills a lacuna in the collective understanding of how data assemblages are constructed and operate

The student-produced electronic portfolio in craft education

The authors studied primary school students’ experiences of using an electronic portfolio in their craft education over four years. A stimulated recall interview was applied to collect user experiences and qualitative content analysis to analyse the collected data. The results indicate that the electronic portfolio was experienced as a multipurpose tool to support learning. It makes the learning process visible and in that way helps focus on and improves the quality of learning. © ISLS.Peer reviewe

Teaching Citizens: Exploring the Relationships Between Teacher Professional Learning, Interactive Civics, and Student Achievement on NAEP Civics

Thesis advisor: Laura M. O'DwyerYouth civic participation is at alarmingly low levels. In 2014, nearly 80% of eligible 18-29 year-olds did not vote in the midterm election (CIRCLE, 2014). Other forms of civic engagement are also at starkly low levels: less than one in ten 18-29 year-olds report contacting a public official, boycotting a product, or frequently expressing political opinions on the internet (U.S. Census Bureau, 2014). Historically, schools have been tasked with preparing students with the knowledge and skills to be active democratic citizens. However, few studies have examined the role of teachers in fostering students' civic knowledge and skills. This study used data from the 2010 National Assessment of Educational Progress (NAEP) 8th grade civics assessment to analyze the relationships between teacher participation in professional learning, use of interactive instructional practices, and student achievement in civics. Participation in professional learning significantly predicted both interactive instruction and student achievement: a one standard deviation increase in professional learning was associated with a predicted .32 standard deviation increase in interactive instructional practices, and a predicted .045 standard deviation increase in student achievement. There was no significant difference between more traditional and communities of practice based forms of professional development in their relationships with interactive instructional practices and student civic achievement. Interactive instructional practices were also significantly associated with increases in student achievement on NAEP civics, but the effect size was small: a one standard deviation increase in interactive instruction was related to a predicted .03 standard deviation increase in student achievement. Moreover, the relationship between interactive instruction and student achievement was curvilinear; high levels of interactive instruction were associated with decreases in student achievement. The study did not find any evidence that teacher participation in professional learning increased the effectiveness of interactive instructional practices.Thesis (PhD) — Boston College, 2015.Submitted to: Boston College. Lynch School of Education.Discipline: Educational Research, Measurement and Evaluation

Reasoning and Religion: The Relevance of the Academic Study of Religion to Critical Thinking Pedagogy

In this thesis I inquire into method and theory debates within the academic study of religion, arguing for the potential of this broad field to enrich critical thinking pedagogy, especially with regard to worldviews and problems associated with the influences of worldviews on reasoning. I highlight the growing recognition within critical thinking literature that critical thinking pedagogy ought to include a sizable component devoted to worldviews, and argue for the relevance of the academic study of religion to this worldview component. An inquiry into three popular method and theory debates within the academic study of religion concerning the definition, comparison, and evaluation of religion(s), and into interreligious dialogue, helps cement this assertion. In my treatment of definition I attempt to put the close connection between religions and worldviews on firm ground. I also describe common misconceptions of religion and worldviews that should be of concern to critical thinking educators, and for which the academic study of religion is particularly apposite. Next I concentrate on how comparison of religions and worldviews is justified within the academic study of religion and on good and bad forms of comparison. My discussion of evaluation repeats this pattern: I look at debates over what constitutes good and bad evaluation in and of religions and worldviews, and the relevance of this subject to critical thinking. My foray into interreligious or cross-worldview dialoguing focuses on difficulties that are germane within the critical thinking domain. Although I do recommend the inclusion of these four subjects within critical thinking pedagogy, I take them primarily as a sample of a wider field of inquiry. This sample is meant to support a broader recommendation, namely: Just as education should be infused with critical thinking, so too should critical thinking be infused with the philosophy of the academic study of religion, and the fruits of its inquiry. This recommendation does not come without reservation. In the last chapter I discuss some of the problems that my suggestions bring up, first among which is the religious bias evident in much of what can be found under the auspices of the academic study of religion

Advancing Human Assessment: The Methodological, Psychological and Policy Contributions of ETS

​This book describes the extensive contributions made toward the advancement of human assessment by scientists from one of the world’s leading research institutions, Educational Testing Service. The book’s four major sections detail research and development in measurement and statistics, education policy analysis and evaluation, scientific psychology, and validity. Many of the developments presented have become de-facto standards in educational and psychological measurement, including in item response theory (IRT), linking and equating, differential item functioning (DIF), and educational surveys like the National Assessment of Educational Progress (NAEP), the Programme of international Student Assessment (PISA), the Progress of International Reading Literacy Study (PIRLS) and the Trends in Mathematics and Science Study (TIMSS). In addition to its comprehensive coverage of contributions to the theory and methodology of educational and psychological measurement and statistics, the book gives significant attention to ETS work in cognitive, personality, developmental, and social psychology, and to education policy analysis and program evaluation. The chapter authors are long-standing experts who provide broad coverage and thoughtful insights that build upon decades of experience in research and best practices for measurement, evaluation, scientific psychology, and education policy analysis. Opening with a chapter on the genesis of ETS and closing with a synthesis of the enormously diverse set of contributions made over its 70-year history, the book is a useful resource for all interested in the improvement of human assessment

Developing Students\u27 Understanding of Similar Figures: a Perceptual Approach.

Children encounter and recognize similar figures in their everyday experiences with such things as basketballs, soccer balls, tennis balls, ping-pong balls; or a candy bar that comes in various sizes of the same shape. Yet their school experience with the mathematics of similarity generally does not build on these perceptual intuitions. Traditional mathematics curricula bypass students\u27 visual intuitions and their quantitative understandings, proceeding directly to set piece problems solved by formal algebraic methods. The result for many students is that the topic of similarity contributes to their evolving view of mathematics as a domain of complex procedural methods divorced from their intuitive sense of quantity and space. The purpose of this study was to explore how to develop students\u27 mathematical understandings of similarity by having quantitative methods evolve from students\u27 visual intuitions about similar figures. The foundation of this curricular approach was a perceptual analysis of similarity consisting of within relationships which are the static relationships within one figure that may be recognized in another, and between relationships which stem from the dynamic perception of one figure as resulting from uniform growth of the other. The curricular strategy encouraged students to verbalize and quantify these perceptual attributes of similar figures, eventually applying quantitative methods to the standard problems encountered in traditional curricula. The subjects were secondary school students enrolled in two separate geometry classes (one classified as college-bound) which were taught by the researcher over a period of three weeks. Qualitative data were collected and analyzed from video recordings, students\u27 work, and journals. The results indicated that the students were generally able to represent their perceptual orientations quantitatively, and to utilize quantitative methods to solve problems. However, the between representation was selected by the plurality of students even when a within strategy would have been computationally more convenient. This highlights the observations of other researchers concerning the difficulty of static representations, and suggests a developmental model in which the more accessible dynamic representations of similarity should precede the static approaches. The students in the college-bound class exhibited more overall flexibility than the students in the non-college-bound class