FeedbackMap: a tool for making sense of open-ended survey responses

Analyzing open-ended survey responses is a crucial yet challenging task for social scientists, non-profit organizations, and educational institutions, as they often face the trade-off between obtaining rich data and the burden of reading and coding textual responses. This demo introduces FeedbackMap, a web-based tool that uses natural language processing techniques to facilitate the analysis of open-ended survey responses. FeedbackMap lets researchers generate summaries at multiple levels, identify interesting response examples, and visualize the response space through embeddings. We discuss the importance of examining survey results from multiple perspectives and the potential biases introduced by summarization methods, emphasizing the need for critical evaluation of the representation and omission of respondent voices.Comment: Demo at CSCW 202

Understanding Communication Patterns in MOOCs: Combining Data Mining and qualitative methods

Massive Open Online Courses (MOOCs) offer unprecedented opportunities to learn at scale. Within a few years, the phenomenon of crowd-based learning has gained enormous popularity with millions of learners across the globe participating in courses ranging from Popular Music to Astrophysics. They have captured the imaginations of many, attracting significant media attention - with The New York Times naming 2012 "The Year of the MOOC." For those engaged in learning analytics and educational data mining, MOOCs have provided an exciting opportunity to develop innovative methodologies that harness big data in education.Comment: Preprint of a chapter to appear in "Data Mining and Learning Analytics: Applications in Educational Research

Structural limitations of learning in a crowd: communication vulnerability and information diffusion in MOOCs

Massive Open Online Courses (MOOCs) bring together a global crowd of thousands of learners for several weeks or months. In theory, the openness and scale of MOOCs can promote iterative dialogue that facilitates group cognition and knowledge construction. Using data from two successive instances of a popular business strategy MOOC, we filter observed communication patterns to arrive at the "significant" interaction networks between learners and use complex network analysis to explore the vulnerability and information diffusion potential of the discussion forums. We find that different discussion topics and pedagogical practices promote varying levels of 1) "significant" peer-to-peer engagement, 2) participant inclusiveness in dialogue, and ultimately, 3) modularity, which impacts information diffusion to prevent a truly "global" exchange of knowledge and learning. These results indicate the structural limitations of large-scale crowd-based learning and highlight the different ways that learners in MOOCs leverage, and learn within, social contexts. We conclude by exploring how these insights may inspire new developments in online education.Comment: Pre-print version. Published version available at http://dx.doi.org/10.1038/srep0644

All a-board: sharing educational data science research with school districts

Educational data scientists often conduct research with the hopes of translating findings into lasting change through policy, civil society, or other channels. However, the bridge from research to practice can be fraught with sociopolitical frictions that impede, or altogether block, such translations -- especially when they are contentious or otherwise difficult to achieve. Focusing on one entrenched educational equity issue in US public schools -- racial and ethnic segregation -- we conduct randomized email outreach experiments and surveys to explore how local school districts respond to algorithmically-generated school catchment areas ("attendance boundaries") designed to foster more diverse and integrated schools. Cold email outreach to approximately 4,320 elected school board members across over 800 school districts informing them of potential boundary changes reveals a large average open rate of nearly 40%, but a relatively small click-through rate of 2.5% to an interactive dashboard depicting such changes. Board members, however, appear responsive to different messaging techniques -- particularly those that dovetail issues of racial and ethnic diversity with other top-of-mind issues (like school capacity planning). On the other hand, media coverage of the research drives more dashboard engagement, especially in more segregated districts. A small but rich set of survey responses from school board and community members across several districts identify data and operational bottlenecks to implementing boundary changes to foster more diverse schools, but also share affirmative comments on the potential viability of such changes. Together, our findings may support educational data scientists in more effectively disseminating research that aims to bridge educational inequalities through systems-level change.Comment: In Proceedings of the Tenth ACM Conference on Learning at Scale (L@S '23

Uncovering latent features in massive open online courses

Massive Open Online Courses (MOOCs) bring together a global crowd of thousands of learners for several weeks or months. We use social network analysis and community detection to uncover the latent features of online discussions in MOOCs. We begin by using data from two successive instances of a popular business strategy MOOC to filter observed communication patterns and arrive at the "significant" interaction networks between learners. We then use complex network analysis to explore the vulnerability and information diffusion potential of the discussion forums. While network analysis offers a vibrant post-hoc analytical framework, it fails to answer a fundamental question: can we devise a model to represent the generation of the dataset at hand? Moving from the structural properties of global-scale discussion to the discussion content itself, we employ existing educational theories to qualitatively content-analyse over 6,500 forum posts from a particular MOOC. We then use a generative model - Bayesian Non-negative Matrix Factorization (BNMF) - to extract communities of learners based on the nature of their online forum posts. We observe that the inferred communities are differentiated by the nature and topic of dialogue, as well as their composite students' demographic and course performance indicators. While qualitative analysis confirms these detected communities, additional quantitative sensitivity analysis shows that they are not crisply defined, illuminating key challenges of applying Machine Learning techniques to model noisy and incomplete learner data. We conclude by discussing the key insights of this work for online education, namely, that different discussion topics and pedagogical practices promote varying levels of peer-to-peer engagement. Additional qualitative investigations reveal that many learners feel a sense of "content-overload" when deciding to participate in online discussions, often leading to their disengagement. These insights call for an interdisciplinary effort to help create relevant and personalized learning experiences in massive scale online settings.</p

Designing for a new "ZIP code destiny"

We live immersed in “cocoons”: tight-knit, segregated psychosocial units that shape who we encounter, the media we consume, what we believe, and ultimately, the opportunities we are able to access in order to positively shape ourselves, families, and communities. These cocoons are fractal in nature, manifesting geographically as schools and neighborhoods segregated by race and income; as social networks that shape which media, mentors and role models we are exposed to; and even in our minds as cloistered concepts that spur biases and make us less welcoming of difference. They are one of the reasons that America is the land of “ZIP code destinies”: the geographic and social contexts in which a child grows up often dramatically affect the opportunities they are able to capitalize on. Advances in social media and communications platforms were supposed to create new connective tissues between cocoons to enable a freer flow of knowledge and opportunity between disparate groups. In some ways, this has happened, and many of these advances have also spun cocoons where marginalized groups can build solidarity and offer mutual support. Yet these advances have also produced social media ecosystems that are highly fragmented, amplifying a priori preferences for which information to consume, and from whom. They have also enabled those with various privileges to more easily access and act on information to obtain education, healthcare, jobs, and other critical resources. This dissertation explores how the analysis of data from digitally and physically-mediated social environments might help inform the design of new technologies to mitigate cocoons across two domains: politics and education. We start with an analysis of political fragmentation on Twitter in the wake of the 2016 US Presidential Election. This analysis motivates the design of a web application, Social Mirror, to probe how prompting social media users to reflect on their own “echo chambers” might help mitigate such fragmentation—which has become a crippling feature of US society, often impeding policies that could positively shape children’s futures. Social fragmentation, of course, is not only rampant in our social media ecosystems: the neighborhoods in which many children grow up are fragmented by race and income, creating cocoons that impede access to quality role models, schools, and other educational opportunities. First, we investigate existing neighborhood-level datasets detailing the importance of exposure to role models to inform the design of INSPIRE, a new video-based social network for middle schoolers to enhance exposure to role models as they start to think about their future aspirations. Next, given the importance of schools and the role parents play in school choice—turning both to personal networks and online resources to inform their choices—we use recent advances in natural language processing to analyze parents’ reviews of schools posted online. Our analyses, however, reveal that affluent parents are more likely to post reviews, and that reviews recapitulate well-documented racial and income disparities in education. We use these insights to inform the design of EdMirror, a “community-sourcing” platform that seeks to surface less biased, more actionable insights from Boston Public Schools parents to other parents and school leaders in ways that might help spark sustainable, positive changes in schools. A central theme across these efforts is the role of prompted reflection and introspection as a potential mechanism for mitigating the biases and other psychological barriers that perpetuate cocoons. The dissertation concludes by exploring how the analysis and design of communications platforms can inform new tools for reflection—i.e., “mirrors”—and how these mirrors might combine with other structural interventions (like policy change) to fuel designs for a new ZIP code destiny.Ph.D