The effect of multiple knowledge sources on learning and teaching

Current paradigms for machine-based learning and teaching tend to perform their task in isolation from a rich context of existing knowledge. In contrast, the research project presented here takes the view that bringing multiple sources of knowledge to bear is of central importance to learning in complex domains. As a consequence teaching must both take advantage of and beware of interactions between new and existing knowledge. The central process which connects learning to its context is reasoning by analogy, a primary concern of this research. In teaching, the connection is provided by the explicit use of a learning model to reason about the choice of teaching actions. In this learning paradigm, new concepts are incrementally refined and integrated into a body of expertise, rather than being evaluated against a static notion of correctness. The domain chosen for this experimentation is that of learning to solve "algebra story problems." A model of acquiring problem solving skills in this domain is described, including: representational structures for background knowledge, a problem solving architecture, learning mechanisms, and the role of analogies in applying existing problem solving abilities to novel problems. Examples of learning are given for representative instances of algebra story problems. After relating our views to the psychological literature, we outline the design of a teaching system. Finally, we insist on the interdependence of learning and teaching and on the synergistic effects of conducting both research efforts in parallel

Measuring measuring: Toward a theory of proficiency with the Constructing Measures framework

This paper is relevant to measurement educators who are interested in the variability of understanding and use of the four building blocks in the Constructing Measures framework (Wilson, 2005). It proposes a uni-dimensional structure for understanding Wilson’s framework, and explores the evidence for and against this conceptualization. Constructed and fixed choice response items are utilized to collect responses from 72 participants who range in experience and expertise with constructing measures. The data was scored by two raters and was analyzed with the Rasch partial credit model using ConQuest (1998). Guided by the 1999 Testing Standards, analyses of validity and reliability evidence provide support for the construct theory and limited uses of the instrument pending item design modifications

Attention and automation: New perspectives on mental underload and performance

There is considerable evidence in the ergonomics literature that automation can significantly reduce operator mental workload. Furthermore, reducing mental workload is not necessarily a good thing, particularly in cases where the level is already manageable. This raises the issue of mental underload, which can be at least as detrimental to performance as overload. However, although it is widely recognized that mental underload is detrimental to performance, there are very few attempts to explain why this may be the case. It is argued in this paper that, until the need for a human operator is completely eliminated, automation has psychological implications relevant in both theoretical and applied domains. The present paper reviews theories of attention, as well as the literature on mental workload and automation, to synthesize a new explanation for the effects of mental underload on performance. Malleable attentional resources theory proposes that attentional capacity shrinks to accommodate reductions in mental workload, and that this shrinkage is responsible for the underload effect. The theory is discussed with respect to the applied implications for ergonomics research

Rationalizing Noneconomic Damages: A Health-Utilities Approach

Studdert et al examine why making compensation of noneconomic damages in personal-injury litigation more rational and predictable is socially valuable. Noneconomic-damages schedules as an alternative to caps are discussed, several potential approaches to construction of schedules are reviewed, and the use of a health-utilities approach as the most promising model is argued. An empirical analysis that combines health-utilities data created in a previous study with original empirical work is used to demonstrate how key steps in construction of a health-utilities-based schedule for noneconomic damages might proceed

Analysis of classifiers' robustness to adversarial perturbations

The goal of this paper is to analyze an intriguing phenomenon recently discovered in deep networks, namely their instability to adversarial perturbations (Szegedy et. al., 2014). We provide a theoretical framework for analyzing the robustness of classifiers to adversarial perturbations, and show fundamental upper bounds on the robustness of classifiers. Specifically, we establish a general upper bound on the robustness of classifiers to adversarial perturbations, and then illustrate the obtained upper bound on the families of linear and quadratic classifiers. In both cases, our upper bound depends on a distinguishability measure that captures the notion of difficulty of the classification task. Our results for both classes imply that in tasks involving small distinguishability, no classifier in the considered set will be robust to adversarial perturbations, even if a good accuracy is achieved. Our theoretical framework moreover suggests that the phenomenon of adversarial instability is due to the low flexibility of classifiers, compared to the difficulty of the classification task (captured by the distinguishability). Moreover, we show the existence of a clear distinction between the robustness of a classifier to random noise and its robustness to adversarial perturbations. Specifically, the former is shown to be larger than the latter by a factor that is proportional to \sqrt{d} (with d being the signal dimension) for linear classifiers. This result gives a theoretical explanation for the discrepancy between the two robustness properties in high dimensional problems, which was empirically observed in the context of neural networks. To the best of our knowledge, our results provide the first theoretical work that addresses the phenomenon of adversarial instability recently observed for deep networks. Our analysis is complemented by experimental results on controlled and real-world data