The Problems with Problem Solving: Reflections on the Rise, Current Status, and Possible Future of a Cognitive Research Paradigm

The research paradigm invented by Allen Newell and Herbert A. Simon in the late 1950s dominated the study of problem solving for more than three decades. But in the early 1990s, problem solving ceased to drive research on complex cognition. As part of this decline, Newell and Simon’s most innovative research practices – especially their method for inducing subjects’ strategies from verbal protocols - were abandoned. In this essay, I summarize Newell and Simon’s theoretical and methodological innovations and explain why their strategy identification method did not become a standard research tool. I argue that the method lacked a systematic way to aggregate data, and that Newell and Simon’s search for general problem solving strategies failed. Paradoxically, the theoretical vision that led them to search elsewhere for general principles led researchers away from studies of complex problem solving. Newell and Simon’s main enduring contribution is the theory that people solve problems via heuristic search through a problem space. This theory remains the centerpiece of our understanding of how people solve unfamiliar problems, but it is seriously incomplete. In the early 1970s, Newell and Simon suggested that the field should focus on the question where problem spaces and search strategies come from. I propose a breakdown of this overarching question into five specific research questions. Principled answers to those questions would expand the theory of heuristic search into a more complete theory of human problem solving

Beyond Evidence-Based Belief Formation: How Normative Ideas Have Constrained Conceptual Change Research

The cognitive sciences, including psychology and education, have their roots in antiquity. In the historically early disciplines like logic and philosophy, the purpose of inquiry was normative. Logic sought to formalize valid inferences, and the various branches of philosophy sought to identify true and certain knowledge. Normative principles are irrelevant for descriptive, empirical sciences like psychology. Normative concepts have nevertheless strongly influenced cognitive research in general and conceptual change research in particular. Studies of conceptual change often ask why students do not abandon their misconceptions when presented with falsifying evidence. But there is little reason to believe that people evolved to conform to normative principles of belief management and conceptual change. When we put the normative traditions aside, we can consider a broader range of hypotheses about conceptual change. As an illustration, the pragmatist focus on action and habits is articulated into a psychological theory that claims that cognitive utility, not the probability of truth, is the key variable that determines belief revision and conceptual change

Rules and principles in cognitive diagnoses

Cognitive simulation is concerned with constructing process models of human cognitive behavior. Our work on the ACM system (Automated Cognitive Modeler) is an attempt to automate this process. The basic assumption is that all goal-oriented cognitive behavior involves search through some problem space. Within this framework, the task of cognitive diagnosis is to identify the problem space in which the subject is operating, identify solution paths used by the subject, and find conditions on the operators that explain those solution paths and that predict the subject's behavior on new problems. The work presented in this paper uses techniques from machine learning to automate the tasks of finding solution paths and operator conditions. We apply this method to the domain of multi-column subtraction and present results that demonstrate ACM's ability to model incorrect subtraction strategies. Finally, we discuss the difference between procedural bugs and misconceptions, proposing that errors due to misconceptions can be viewed as violations of principles for the task domain

Verbal IQ of a Four-Year Old Achieved by an AI System

Abstract One view of common-sense reasoning ability is that it is the ability to perform those tasks with verbal inputs and outputs that have traditionally been difficult for computer systems, but are easy for fairly young children. We administered the verbal part of the Wechsler Preschool and Primary Scale of Intelligence (WPPSI-III, Third Edition) to the ConceptNet 4 system. The IQ test's questions (e.g., "Why do we shake hands?" or "What do apples and bananas have in common") were translated into ConceptNet 4 inputs using a combination of the simple natural language processing tools that come with ConceptNet together with short Python programs that we wrote. The question-answering primarily used the part of the ConceptNet system that represents the knowledge as a matrix based on spectral methods (AnalogySpace). We found that the system has a Verbal IQ that is average for a four-year-old child, but below average for 5, 6, and 7 yearolds. Large variations from subtest to subtest indicate potential areas of improvement. In particular, results were strongest for the Vocabulary and Similarities subtests, intermediate for the Information subtest, and lowest for the Comprehension and Word Reasoning subtests. Comprehension is the subtest most strongly associated with common sense. Children's verbal IQ tests offer a new, objective, third-party metric for the evaluation and comparison of common-sense AI systems

Towards an Intelligent Tutor for Mathematical Proofs

Computer-supported learning is an increasingly important form of study since it allows for independent learning and individualized instruction. In this paper, we discuss a novel approach to developing an intelligent tutoring system for teaching textbook-style mathematical proofs. We characterize the particularities of the domain and discuss common ITS design models. Our approach is motivated by phenomena found in a corpus of tutorial dialogs that were collected in a Wizard-of-Oz experiment. We show how an intelligent tutor for textbook-style mathematical proofs can be built on top of an adapted assertion-level proof assistant by reusing representations and proof search strategies originally developed for automated and interactive theorem proving. The resulting prototype was successfully evaluated on a corpus of tutorial dialogs and yields good results.Comment: In Proceedings THedu'11, arXiv:1202.453

An intelligent interactive system for delivering individualized information to patients

This paper is a report on the first phase of a long-term, interdisciplinary project whose goal is to increase the overall effectiveness of physicians ’ time, and thus the quality of health care, by improving the information exchange between physicians and patients in clinical settings. We are focusing on patients with long-term and chronic conditions, initially on migraine patients, who require periodic interaction with their physicians for effective management of their condition. We are using medical informatics to focus on the information needs of patients, as well as of physicians, and to address problems of information exchange. This requires understanding patients’ concerns to design an appropriate system, and using state-of-the-art artificial intelligence techniques to build an interactive explanation system. In contrast to many other knowledge-based systems, our system’s design is based on empirical data on actual information needs. We used ethnographic techniques to observe explanations actually given in clinic settings, and to conduct interviews with migraine sufferers and physicians. Our system has an extensive knowledge base that contains both general medical terminology and specific knowledge about migraine, such a