Facilitating teacher participation in intelligent computer tutor design : tools and design methods.

This work addresses the widening gap between research in intelligent tutoring systems (ITSs) and practical use of this technology by the educational community. In order to ensure that ITSs are effective, teachers must be involved in their design and evaluation. We have followed a user participatory design process to build a set of ITS knowledge acquisition tools that facilitate rapid prototyping and testing of curriculum, and are tailored for usability by teachers. The system (called KAFITS) also serves as a test-bed for experimentation with multiple tutoring strategies. The design includes novel methodologies for tutoring strategy representation (Parameterized Action Networks) and overlay student modeling (a layered student model), and incorporates considerations from instructional design theory. It also allows for considerable student control over the content and style of the information presented. Highly interactive graphics-based tools were built to facilitate design, inspection, and modification of curriculum and tutoring strategies, and to monitor the progress of the tutoring session. Evaluation of the system includes a sixteen-month case study of three educators (one being the domain expert) using the system to build a tutor for statics (forty topics representing about four hours of on-line instruction), testing the tutor on a dozen students, and using test results to iteratively improve the tutor. Detailed throughput analysis indicates that the amount of effort to build the statics tutor was, surprisingly, comparable to similar figures for building (non-intelligent) conventional computer aided instructional systems. Few ITS projects focus on educator participation and this work is the first to empirically study knowledge acquisition for ITSs. Results of the study also include: a recommended design process for building ITSs with educator participation; guidelines for training educators; recommendations for conducting knowledge acquisition sessions; and design tradeoffs for knowledge representation architectures and knowledge acquisition interfaces

Natural Language Tutoring and the Novice Programmer

For beginning programmers, inadequate problem solving and planning skills are among the most salient of their weaknesses. Novices, by definition, lack much of the tacit knowledge that underlies effective programming. This dissertation examines the efficacy of natural language tutoring (NLT) to foster acquisition of this tacit knowledge. Coached Program Planning (CPP) is proposed as a solution to the problem of teaching the tacit knowledge of programming. The general aim is to cultivate the development of such knowledge by eliciting and scaffolding the problem solving and planning activities that novices are known to underestimate or bypass altogether. ProPL (pro-PELL), a dialogue-based intelligent tutoring system based on CPP, is also described. In an evaluation, the primary findings were that students who received tutoring from ProPL seemed to exhibit an improved ability compose plans and displayed behaviors suggestive of thinking at greater levels of abstraction than students in a read-only control group. The major finding is that NLT appears to be effective in teaching program composition skills

Tool Support for Finding and Preventing Faults in Rule Bases

This thesis analyzes challenges for the correct creation of rule bases. Based on experiences and data from three rule base development projects, dedicated experiments and a survey of developers, ten main problem areas are identified. Four approaches in the area of Testing, Debugging, Anomaly Detection and Visualization are proposed and evaluated as remedies for these problem areas

A Grounded Conceptual Model for Ownership Types in Rust

Programmers learning Rust struggle to understand ownership types, Rust's core mechanism for ensuring memory safety without garbage collection. This paper describes our attempt to systematically design a pedagogy for ownership types. First, we studied Rust developers' misconceptions of ownership to create the Ownership Inventory, a new instrument for measuring a person's knowledge of ownership. We found that Rust learners could not connect Rust's static and dynamic semantics, such as determining why an ill-typed program would (or would not) exhibit undefined behavior. Second, we created a conceptual model of Rust's semantics that explains borrow checking in terms of flow-sensitive permissions on paths into memory. Third, we implemented a Rust compiler plugin that visualizes programs under the model. Fourth, we integrated the permissions model and visualizations into a broader pedagogy of ownership by writing a new ownership chapter for The Rust Programming Language, a popular Rust textbook. Fifth, we evaluated an initial deployment of our pedagogy against the original version, using reader responses to the Ownership Inventory as a point of comparison. Thus far, the new pedagogy has improved learner scores on the Ownership Inventory by an average of 9% ($N = 342, d = 0.56$).Comment: Published at OOPSLA 202

"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Code-generating large language models translate natural language into code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the specific context of data analysis in spreadsheets, in a system that maps the users natural language query to Python code using the Codex generator, executes the code, and shows the result. We propose grounded abstraction matching, which bridges the abstraction gap by translating the code back into a systematic and predictable naturalistic utterance. In a between-subjects, think-aloud study (n=24), we compare grounded abstraction matching to an ungrounded alternative based on previously established query framing principles. We find that the grounded approach improves end-users' understanding of the scope and capabilities of the code-generating model, and the kind of language needed to use it effectively

Critical Programming: Toward a Philosophy of Computing

Beliefs about the relationship between human beings and computing machines and their destinies have alternated from heroic counterparts to conspirators of automated genocide, from apocalyptic extinction events to evolutionary cyborg convergences. Many fear that people are losing key intellectual and social abilities as tasks are offloaded to the everywhere of the built environment, which is developing a mind of its own. If digital technologies have contributed to forming a dumbest generation and ushering in a robotic moment, we all have a stake in addressing this collective intelligence problem. While digital humanities continue to flourish and introduce new uses for computer technologies, the basic modes of philosophical inquiry remain in the grip of print media, and default philosophies of computing prevail, or experimental ones propagate false hopes. I cast this as-is situation as the post-postmodern network dividual cyborg, recognizing that the rational enlightenment of modernism and regressive subjectivity of postmodernism now operate in an empire of extended mind cybernetics combined with techno-capitalist networks forming societies of control. Recent critical theorists identify a justificatory scheme foregrounding participation in projects, valorizing social network linkages over heroic individualism, and commending flexibility and adaptability through life long learning over stable career paths. It seems to reify one possible, contingent configuration of global capitalism as if it was the reflection of a deterministic evolution of commingled technogenesis and synaptogenesis. To counter this trend I offer a theoretical framework to focus on the phenomenology of software and code, joining social critiques with textuality and media studies, the former proposing that theory be done through practice, and the latter seeking to understand their schematism of perceptibility by taking into account engineering techniques like time axis manipulation. The social construction of technology makes additional theoretical contributions dispelling closed world, deterministic historical narratives and requiring voices be given to the engineers and technologists that best know their subject area. This theoretical slate has been recently deployed to produce rich histories of computing, networking, and software, inform the nascent disciplines of software studies and code studies, as well as guide ethnographers of software development communities. I call my syncretism of these approaches the procedural rhetoric of diachrony in synchrony, recognizing that multiple explanatory layers operating in their individual temporal and physical orders of magnitude simultaneously undergird post-postmodern network phenomena. Its touchstone is that the human-machine situation is best contemplated by doing, which as a methodology for digital humanities research I call critical programming. Philosophers of computing explore working code places by designing, coding, and executing complex software projects as an integral part of their intellectual activity, reflecting on how developing theoretical understanding necessitates iterative development of code as it does other texts, and how resolving coding dilemmas may clarify or modify provisional theories as our minds struggle to intuit the alien temporalities of machine processes

An Ethnographic Study of Everyday Interactions in Innovative Learning Spaces

For the last 10 years universities and colleges in the UK have generated significant investment in designing innovative learning spaces. These spaces have been created to accommodate a student-centered pedagogical approach that is intended to promote formal and informal learning activities, collaboration and socializing by means of flexible technological infrastructure and architectural design. Various assessments have already been realized to investigate the outcomes of this investment and the impact of those spaces on learning. Yet, there is a persevering need to better understand the role of the technological infrastructure and the architectural design in innovative learning spaces as a lived experience by those who use and inhabit them; and to establish whether they are used as anticipated. This work takes up on this challenge and investigates three innovative learning spaces through an ethnographic approach that, following the analytic orientation of Suchman’s situated action, considers and juxtaposes anticipated versus actual use. More specifically, this work addresses the following research questions: • How do people interact with the architectural and technological infrastructure and with each other in innovative learning spaces on an everyday basis? • How do everyday interactions compare with those envisioned by the designers and managers of these spaces? • How do we account for the differences between actual and anticipated use of the spaces? • How can spaces be designed or recover from breakdowns so that actual and anticipated use (re) align? By addressing those questions, the present work contributes to an empirically-grounded understanding of how innovative learning spaces are being used and appropriated compared to the envisioned usage. The analysis reveals tensions between actual and anticipated use, the situated nature of flexible design, as well as the complex and contested processes through which interactions in innovative learning spaces are accomplished, adapted or superseded. The findings suggest a set of critical factors that account for the tensions between desired and actual use of such spaces. Issues of legibility, legitimacy and sense of ownership and appropriation supersede the existing views and guidelines of adaptable design as presented in the current literature and can be used to inform the design and evaluation of innovative learning spaces