Experiment and bias: the case of parsimony in comparative cognition

Comparative cognition is the interdisciplinary field of animal cognition and behavior studies, which includes comparative psychology and branches of ethology, biology, and neuroscience. My dissertation shows that the quasi-epistemic value of parsimony plays a problematic role in the experimental setting of comparative cognition. More specifically, I argue that an idiosyncratic interpretation of the statistical hypothesis-testing method, known as the Neyman-Pearson Method (NPM), embeds an Occamist parsimony preference into experimental methodology in comparative cognition, which results in an underattribution bias, or a bias in favor of allegedly simple cognitive ontologies. I trace this parsimony preference to the content of the null hypothesis within the NPM, and defend a strategy for modifying the NPM to guard against the underattribution bias. I recommend adopting an evidence-driven strategy for choosing the null hypothesis. Further, I suggest a role for non-empirical values, such as ethical concerns, in the weighting of Type I and Type II error-rates. I contend that statistical models are deeply embedded in experimental practice and are not value-free. These models provide an often overlooked door through which values, both epistemic and non-epistemic, can enter scientific research. Since statistical models generally, and the NPM in particular, play a role in a wide variety of scientific disciplines, this dissertation can also be seen as a case study illustrating the importance of attending to the choice a particular statistical model. This conclusion suggests that various philosophical investigations of scientific practice - from inquiry into the nature of scientific evidence to analysis of the role of values in science - would be greatly enriched by increased attention to experimental methodology, including the choice and interpretation of statistical models

Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation

Determining the plausibility of causal relations between clauses is a commonsense reasoning task that requires complex inference ability. The general approach to this task is to train a large pretrained language model on a specific dataset. However, the available training data for the task is often scarce, which leads to instability of model training or reliance on the shallow features of the dataset. This paper presents a number of techniques for making models more robust in the domain of causal reasoning. Firstly, we perform adversarial training by generating perturbed inputs through synonym substitution. Secondly, based on a linguistic theory of discourse connectives, we perform data augmentation using a discourse parser for detecting causally linked clauses in large text, and a generative language model for generating distractors. Both methods boost model performance on the Choice of Plausible Alternatives (COPA) dataset, as well as on a Balanced COPA dataset, which is a modified version of the original data that has been developed to avoid superficial cues, leading to a more challenging benchmark. We show a statistically significant improvement in performance and robustness on both datasets, even with only a small number of additionally generated data points.Comment: 7 pages + pages references, 4 figures, 3 tables, paper accepted at AAAI202

Problems of deep machine learning

This article discusses the main difficulties in the development and training of ML-models associated with the problem of generalization. The causes of these problems and the main difficulties caused by them are analyzed. The problem of general non-concretization of ML tasks and the vulnerability of neural networks to adversarial attacks is also considered

Standards of Proof Revisited

This Essay focuses not on how fact-finders process evidence but on how they apply the specified standard of proof to their finding. The oddity that prompts speculation is that, in noncriminal cases, the common law asks only that the fact appear more likely than not, while the Civil Law seems to apply the same high standard in these cases as it does in criminal cases. As a psychological explanation of the cognitive processes involved, some theorists posit that the bulk of fact-finding is an unconscious process, powerful but dangerous, which generates a level of confidence against which the fact-finder could apply the standard of proof. But this foggy confidence-based theory fails because standards of proof should, and factfinders arguably do, concern themselves with probability rather than confidence. Psychology also cannot explain the divide between the common law and the Civil Law because the real explanation likely lies in the different goals that the two procedural systems pursue through their standards of proof