Influence factors for local comprehensibility of process models

The main aim of this study is to investigate human understanding of process models and to develop an improved understanding of its relevant influence factors. Aided by assumptions from cognitive psychology, this article attempts to address specific deductive reasoning difficulties based on process models. The authors developed a research model to capture the influence of two effects on the cognitive difficulty of reasoning tasks: (i) the presence of different control-flow patterns (such as conditional or parallel execution) in a process model and (ii) the interactivity of model elements. Based on solutions to 61 different reasoning tasks by 155 modelers, the results from this study indicate that the presence of certain control-flow patterns influences the cognitive difficulty of reasoning tasks. In particular, sequence is relatively easy, while loops in a model proved difficult. Modelers with higher process modeling knowledge performed better and rated subjective difficulty of loops lower than modelers with lower process modeling knowledge. The findings additionally support the prediction that interactivity between model elements is positively related to the cognitive difficulty of reasoning. Our research contributes to both academic literature on the comprehension of process models and practitioner literature focusing on cognitive difficulties when using process models

On Cognitive Preferences and the Plausibility of Rule-based Models

It is conventional wisdom in machine learning and data mining that logical models such as rule sets are more interpretable than other models, and that among such rule-based models, simpler models are more interpretable than more complex ones. In this position paper, we question this latter assumption by focusing on one particular aspect of interpretability, namely the plausibility of models. Roughly speaking, we equate the plausibility of a model with the likeliness that a user accepts it as an explanation for a prediction. In particular, we argue that, all other things being equal, longer explanations may be more convincing than shorter ones, and that the predominant bias for shorter models, which is typically necessary for learning powerful discriminative models, may not be suitable when it comes to user acceptance of the learned models. To that end, we first recapitulate evidence for and against this postulate, and then report the results of an evaluation in a crowd-sourcing study based on about 3.000 judgments. The results do not reveal a strong preference for simple rules, whereas we can observe a weak preference for longer rules in some domains. We then relate these results to well-known cognitive biases such as the conjunction fallacy, the representative heuristic, or the recogition heuristic, and investigate their relation to rule length and plausibility.Comment: V4: Another rewrite of section on interpretability to clarify focus on plausibility and relation to interpretability, comprehensibility, and justifiabilit

Disentangling accent from comprehensibility

The goal of this study was to determine which linguistic aspects of second language speech are related to accent and which to comprehensibility. To address this goal, 19 different speech measures in the oral productions of 40 native French speakers of English were examined in relation to accent and comprehensibility, as rated by 60 novice raters and three experienced teachers. Results showed that both constructs were associated with many speech measures, but that accent was uniquely related to aspects of phonology, including rhythm and segmental and syllable structure accuracy, while comprehensibility was chiefly linked to grammatical accuracy and lexical richness

Research methods and intelligibility studies

This paper first briefly reviews the concept of intelligibility as it has been employed in both English as a Lingua Franca (ELF) and world Englishes (WE) research. It then examines the findings of the Lingua Franca Core (LFC), a list of phonological features that empirical research has shown to be important for safeguarding mutual intelligibility between non-native speakers of English. The main point of the paper is to analyse these findings and demonstrate that many of them can be explained if three perspectives (linguistic, psycholinguistic and historical-variationist) are taken. This demonstration aims to increase the explanatory power of the concept of intelligibility by providing some theoretical background. An implication for ELF research is that at the phonological level, internationally intelligible speakers have a large number of features in common, regardless of whether they are non-native speakers or native speakers. An implication for WE research is that taking a variety-based, rather than a features-based, view of phonological variation and its connection with intelligibility is likely to be unhelpful, as intelligibility depends to some extent on the phonological features of individual speakers, rather than on the varieties per se

The Pragmatic Turn in Explainable Artificial Intelligence (XAI)

In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post hoc interpretability that seems to be predominant in most recent literature

Asymmetrical cognitive load Imposed by processing native and non-native speech

Intonation affects information processing and comprehension. Previous research has found that some international teaching assistants (ITAs) fail to exploit English intonation, potentially posing processing difficulties to students who are native English speakers. However, researchers have also found that non-native listeners found it easier to process sentences given by a non-native speaker with a shared language background, leading to an interlanguage speech intelligibility benefit (ISIB). Therefore, how native speaker teaching assistant (NSTA)’s and ITA’s classroom speech affects the processing, comprehension, and attitudes of listeners with different language backgrounds needs to be further investigated. Using a dual-task paradigm, a comprehension questionnaire, and an attitudinal questionnaire, the present study investigates how the pronunciation and intonation of a NSTA and an ITA affect native English speakers’ and Mandarin-speaking English learners’ processing and comprehension of a lecture, and attitudes towards the two instructors. The present study found shared processing advantages when the listeners shared the L1 of the speaker, but overall lecture comprehension and attitude were unaffected. These findings support and extend prior research studies surveying ITAs’ intonational patterns and ISIB. These findings also have implications for research on the teaching of English pronunciation to non-native instructors.Published versio

Adolescent Literacy and Textbooks: An Annotated Bibliography

A companion report to Carnegie's Time to Act, provides an annotated bibliography of research on textbook design and reading comprehension for fourth through twelfth grade, arranged by topic. Calls for a dialogue between publishers and researchers