Investigating the role of model-based reasoning while troubleshooting an electric circuit

We explore the overlap of two nationally-recognized learning outcomes for physics lab courses, namely, the ability to model experimental systems and the ability to troubleshoot a malfunctioning apparatus. Modeling and troubleshooting are both nonlinear, recursive processes that involve using models to inform revisions to an apparatus. To probe the overlap of modeling and troubleshooting, we collected audiovisual data from think-aloud activities in which eight pairs of students from two institutions attempted to diagnose and repair a malfunctioning electrical circuit. We characterize the cognitive tasks and model-based reasoning that students employed during this activity. In doing so, we demonstrate that troubleshooting engages students in the core scientific practice of modeling.Comment: 20 pages, 6 figures, 4 tables; Submitted to Physical Review PE

Explanation-based learning for diagnosis

Diagnostic expert systems constructed using traditional knowledge-engineering techniques identify malfunctioning components using rules that associate symptoms with diagnoses. Model-based diagnosis (MBD) systems use models of devices to find faults given observations of abnormal behavior. These approaches to diagnosis are complementary. We consider hybrid diagnosis systems that include both associational and model-based diagnostic components. We present results on explanation-based learning (EBL) methods aimed at improving the performance of hybrid diagnostic problem solvers. We describe two architectures called EBL_IA and EBL(p). EBL_IA is a form fo "learning in advance" that pre-compiles models into associations. At run-time the diagnostic system is purely associational. In EBL(p), the run-time diagnosis system contains associational, MBD, and EBL components. Learned associational rules are preferred but when they are incomplete they may produce too many incorrect diagnoses. When errors cause performance to dip below a give threshold p, EBL(p) activates MBD and explanation-based "learning while doing". We present results of empirical studies comparing MBD without learning versus EBL_IA and EBL(p). The main conclusions are as follows. EBL_IA is superior when it is feasible but it is not feasible for large devices. EBL(p) can speed-up MBD and scale-up to larger devices in situations where perfect accuracy is not required

Characterizing lab instructors' self-reported learning goals to inform development of an experimental modeling skills assessment

The ability to develop, use, and refine models of experimental systems is a nationally recognized learning outcome for undergraduate physics lab courses. However, no assessments of students' model-based reasoning exist for upper-division labs. This study is the first step toward development of modeling assessments for optics and electronics labs. In order to identify test objectives that are likely relevant across many institutional contexts, we interviewed 35 lab instructors about the ways they incorporate modeling in their course learning goals and activities. The study design was informed by the Modeling Framework for Experimental Physics. This framework conceptualizes modeling as consisting of multiple subtasks: making measurements, constructing system models, comparing data to predictions, proposing causes for discrepancies, and enacting revisions to models or apparatus. We found that each modeling subtask was identified by multiple instructors as an important learning outcome for their course. Based on these results, we argue that test objectives should include probing students' competence with most modeling subtasks, and test items should be designed to elicit students' justifications for choosing particular modeling pathways. In addition to discussing these and other implications for assessment, we also identify future areas of research related to the role of modeling in optics and electronics labs.Comment: 24 pages, 2 figures, 5 tables; submitted to Phys. Rev. PE

Qualitative mechanism models and the rationalization of procedures

A qualitative, cluster-based approach to the representation of hydraulic systems is described and its potential for generating and explaining procedures is demonstrated. Many ideas are formalized and implemented as part of an interactive, computer-based system. The system allows for designing, displaying, and reasoning about hydraulic systems. The interactive system has an interface consisting of three windows: a design/control window, a cluster window, and a diagnosis/plan window. A qualitative mechanism model for the ORS (Orbital Refueling System) is presented to coordinate with ongoing research on this system being conducted at NASA Ames Research Center

Applying Formal Methods to Networking: Theory, Techniques and Applications

Despite its great importance, modern network infrastructure is remarkable for the lack of rigor in its engineering. The Internet which began as a research experiment was never designed to handle the users and applications it hosts today. The lack of formalization of the Internet architecture meant limited abstractions and modularity, especially for the control and management planes, thus requiring for every new need a new protocol built from scratch. This led to an unwieldy ossified Internet architecture resistant to any attempts at formal verification, and an Internet culture where expediency and pragmatism are favored over formal correctness. Fortunately, recent work in the space of clean slate Internet design---especially, the software defined networking (SDN) paradigm---offers the Internet community another chance to develop the right kind of architecture and abstractions. This has also led to a great resurgence in interest of applying formal methods to specification, verification, and synthesis of networking protocols and applications. In this paper, we present a self-contained tutorial of the formidable amount of work that has been done in formal methods, and present a survey of its applications to networking.Comment: 30 pages, submitted to IEEE Communications Surveys and Tutorial

Using think-aloud interviews to characterize model-based reasoning in electronics for a laboratory course assessment

Models of physical systems are used to explain and predict experimental results and observations. The Modeling Framework for Experimental Physics describes the process by which physicists revise their models to account for the newly acquired observations, or change their apparatus to better represent their models when they encounter discrepancies between actual and expected behavior of a system. While modeling is a nationally recognized learning outcome for undergraduate physics lab courses, no assessments of students' model-based reasoning exist for upper-division labs. As part of a larger effort to create two assessments of students' modeling abilities, we used the Modeling Framework to develop and code think-aloud problem-solving activities centered on investigating an inverting amplifier circuit. This study is the second phase of a multiphase assessment instrument development process. Here, we focus on characterizing the range of modeling pathways students employ while interpreting the output signal of a circuit functioning far outside its recommended operation range. We end by discussing four outcomes of this work: (1) Students engaged in all modeling subtasks, and they spent the most time making measurements, making comparisons, and enacting revisions; (2) Each subtask occurred in close temporal proximity to all over subtasks; (3) Sometimes, students propose causes that do not follow logically from observed discrepancies; (4) Similarly, students often rely on their experiential knowledge and enact revisions that do not follow logically from articulated proposed causes.Comment: 18 pages, 5 figure

Expert system technology

The expert system is a computer program which attempts to reproduce the problem-solving behavior of an expert, who is able to view problems from a broad perspective and arrive at conclusions rapidly, using intuition, shortcuts, and analogies to previous situations. Expert systems are a departure from the usual artificial intelligence approach to problem solving. Researchers have traditionally tried to develop general modes of human intelligence that could be applied to many different situations. Expert systems, on the other hand, tend to rely on large quantities of domain specific knowledge, much of it heuristic. The reasoning component of the system is relatively simple and straightforward. For this reason, expert systems are often called knowledge based systems. The report expands on the foregoing. Section 1 discusses the architecture of a typical expert system. Section 2 deals with the characteristics that make a problem a suitable candidate for expert system solution. Section 3 surveys current technology, describing some of the software aids available for expert system development. Section 4 discusses the limitations of the latter. The concluding section makes predictions of future trends

Live, virtual, and constructive environments for performance support

As military systems become more complex, the operation and support of these systems becomes intrinsically more difficult. The U.S. Army\u27s current procurement process relies on industry to provide embedded training and performance support tools for the systems they produce. These tools are relatively new and in the early stages of development. As yet, they have failed to meet the needs of the technicians that are required to support these complex systems. Current efforts to provide enabling technologies that enhance the capabilities of automotive maintenance technicians are concentrated in three professional communities. First is the Performance Improvement community where work is focused on developing and implementing performance support system technologies that deliver information that is stored in information systems. Second is the Knowledge Management community working on organizational knowledge management techniques that capture, store, and map information that is delivered to workers within an organization. The third is the Training and Education community focusing on developing curriculum and delivery systems that support life-long-learning requirements. This dissertation addresses an essential component of performance systems, namely the ability to deliver the knowledge needed to guide a problem solver to a solution state, thereby enhancing worker capabilities. This objective is met by developing the LockTel Framework that provides a construct for segmenting knowledge into three environments for performance support, the live, the virtual, and the constructive environments. It provides a means for the maintenance technician to gain knowledge associated with completing a given task. Seventy-eight maintenance technician trainees at an U.S. Army training center tested the framework. The hypothesis behind the proposed construct was strongly supported, thereby establishing the foundation for future work in live, virtual, and constructive environments for performance support