An evolutionary model of reinforcer value

Within the field of evolutionary biology, natural selection is often thought to favor traits that lead to individuals behaving as if they were maximizing their evolutionary fitness. The concept of the individual as a maximizer is also popular in behavioral psychology, especially when it comes to theories of operant learning. Here, the individual is taken to adapt its behavior to the local environment, such that the expected amount of reinforcer value is maximized. Whereas there is a considerable consensus concerning the formal properties of an evolutionary maximand (‘fitness’), there is no generally accepted conceptualization of a corresponding behavioral maximand (‘reinforcer value’). However, such theoretical clarification is crucial to the development and empirical testing of learning theories, since it is impossible to decide whether the concept of reinforcer maximization is adequate, as long as the maximand is not well defined. This paper presents a formal model of reinforcer value that is consistent with existing work on the nature of reinforcement and provides an explicit link between behavioral psychology and evolutionary biology. The main result is that the reinforcer value of an additional time unit spent at a behavior equals its expected marginal effects on evolutionary fitness. Applying the model to matching behavior, it is further demonstrated how the established link between reinforcer value and evolutionary fitness can be used to derive new hypotheses

Evolutionary dynamics of the Trivers–Willard effect : A nonparametric approach

The Trivers–Willard hypothesis (TWH) states that parents in good condition tend to bias their offspring sex ratio toward the sex with a higher variation in reproductive value, whereas parents in bad condition favor the opposite sex. Although the TWH has been generalized to predict various Trivers–Willard effects (TWE) depending on the life cycle of a species, existing work does not sufficiently acknowledge that sex-specific reproductive values depend on the relative abundances of males and females in the population. If parents adjust their offspring sex ratio according to the TWE, offspring reproductive values will also change. This should affect the long-term evolutionary dynamics and might lead to considerable deviations from the original predictions. In this paper, I model the full evolutionary dynamics of the TWE, using a published two-sex integral projection model for the Columbian ground squirrel (Urocitellus columbianus). Offspring sex ratio is treated as a nonparametric continuous function of maternal condition. Evolutionary change is treated as the successive invasion of mutant strategies. The simulation is performed with varying starting conditions until an evolutionarily stable strategy (ESS) is reached. The results show that the magnitude of the evolving TWE can be far greater than previously predicted. Furthermore, evolutionary dynamics show considerable nonlinearities before settling at an ESS. The nonlinear effects depend on the starting conditions and indicate that evolutionary change is fastest when starting at an extremely biased sex ratio and that evolutionary change is weaker for parents of high condition. The results show neither a tendency to maximize average population fitness nor to minimize the deviation between offspring sex ratio and offspring reproductive value ratio. The study highlights the importance of dynamic feedback in models of natural selection and provides a new methodological framework for analyzing the evolution of continuous strategies in structured populations

Quantitative and Qualitative Approaches to Generalization and Replication – A Representationalist View

In this paper, we provide a re-interpretation of qualitative and quantitative modeling from a representationalist perspective. In this view, both approaches attempt to construct abstract representations of empirical relational structures. Whereas quantitative research uses variable-based models that abstract from individual cases, qualitative research favors case-based models that abstract from individual characteristics. Variable-based models are usually stated in the form of quantified sentences (scientific laws). This syntactic structure implies that sentences about individual cases are derived using deductive reasoning. In contrast, case-based models are usually stated using context-dependent existential sentences (qualitative statements). This syntactic structure implies that sentences about other cases are justifiable by inductive reasoning. We apply this representationalist perspective to the problems of generalization and replication. Using the analytical framework of modal logic, we argue that the modes of reasoning are often not only applied to the context that has been studied empirically, but also on a between-contexts level. Consequently, quantitative researchers mostly adhere to a top-down strategy of generalization, whereas qualitative researchers usually follow a bottom-up strategy of generalization. Depending on which strategy is employed, the role of replication attempts is very different. In deductive reasoning, replication attempts serve as empirical tests of the underlying theory. Therefore, failed replications imply a faulty theory. From an inductive perspective, however, replication attempts serve to explore the scope of the theory. Consequently, failed replications do not question the theory per se, but help to shape its boundary conditions. We conclude that quantitative research may benefit from a bottom-up generalization strategy as it is employed in most qualitative research programs. Inductive reasoning forces us to think about the boundary conditions of our theories and provides a framework for generalization beyond statistical testing. In this perspective, failed replications are just as informative as successful replications, because they help to explore the scope of our theories

Beyond quality and quantity : Representing empirical structures by embedded typologies

In this article, we propose a new approach to the problem of integration in mixed methods research that builds on a representational understanding of empirical science. From this perspective, qualitative and quantitative modeling strategies constitute two different ways to represent empirical structures. Whereas qualitative representations focus on the construction of types from cases, quantitative representations focus on the construction of dimensions from variables. We argue that types and dimensions should be integrated within a joint representation of the data that equally acknowledges qualitative and quantitative aspects. We outline how the proposed representational framework can be used to embed qualitative types in quantitative dimensions using an empirical study on teachers’ epistemological beliefs

The formal foundation of an evolutionary theory of reinforcement

Reinforcement learning is often described by analogy to natural selection. However, there is no coherent theory relating reinforcement learning to evolution within a single formal model of selection. This paper provides the formal foundation of such a unified theory. The model is based on the most general description of natural selection as given by the Price equation. We extend the Price equation to cover reinforcement learning as the result of a behavioral selection process within individuals and relate it to the principle of natural selection via the concept of statistical fitness predictors by means of a multilevel model of behavioral selection. The main result is the covariance-based law of effect, which describes reinforcement learning on a molar level by means of the covariance between behavioral allocation and a statistical fitness predictor. We further demonstrate how this abstract principle can be applied to derive theoretical explanations of various empirical findings, like conditioned reinforcement, blocking, matching and response deprivation. Our model is the first to apply the abstract principle of selection to derive a unified description of reinforcement learning and natural selection within a single model. It provides a general analytical tool for behavioral psychology in a similar way that the theory of natural selection does for evolutionary biology. We thus lay the formal foundation of a general theory of reinforcement as the result of behavioral selection on multiple levels

Meaningful measurement requires substantive formal theory

In this article, we take the opportunity to elaborate on some aspects of our article “Squaring the Circle: From Latent Variables to Theory-Based Measurement” (Borgstede & Eggert, 2023) that gave rise to the concerns uttered by Hasselman (2023) and Slaney (2023), and to clarify why we think that theory-based measurement is indeed necessary and sufficient for the establishment of meaningful psychological measurement procedures. Moreover, we will illustrate how theory-based measurement might be accomplished in psychology by means of an example from behavioral selection theory

Squaring the circle: From latent variables to theory-based measurement

Psychometrics builds on the fundamental premise that psychological attributes are unobservable and need to be inferred from observable behavior. Consequently, psychometric procedures consist primarily in applying latent variable modeling, which statistically relates latent variables to manifest variables. However, latent variable modeling falls short of providing a theoretically sound definition of psychological attributes. Whereas in a pragmatic interpretation of latent variable modeling latent variables cannot represent psychological attributes at all, a realist interpretation of latent variable modeling implies that latent variables are empty placeholders for unknown attributes. The authors argue that psychological attributes can only be identified if they are defined within the context of substantive formal theory. Building on the structuralist view of scientific theories, they show that any successful application of such a theory necessarily produces specific values for the theoretical terms that are defined within the theory. Therefore, substantive formal theory is both necessary and sufficient for psychological measurement

The Covariance Based Law of Effect : A Fundamental Principle of Behavior

Building on George Price's formal account of selection, we present an abstract theoretical account of behavioral selection that integrates the domains of individual learning and evolution. From the perspective of the multilevel model of behavioral selection (MLBS), we argue that the covariance based law of effect (CLOE) qualifies as a fundamental principle of behavior in that it provides a general formal framework for selectionist thinking and model building. We demonstrate the feasibility of our approach by means of a covariance based model of choice behavior that explains the effects of changeover delays on operant matching

The Trivers-Willard Effect for Educational Investment : Evidence from an African Sample

The Trivers-Willard hypothesis (TWH) states that individuals in good condition favor offspring of the sex that has a higher variance in reproductive value. Empirical studies with historical human populations suggest that the TWH might explain biased birth-ratios as well as biased parental investment in male or female offspring. However, empirical tests of the TWH in modern human populations are less conclusive. In this study, we investigate whether parental investment in education might be skewed according to the TWH in an African sample (N = 314) that includes students from 8 different countries. The data show that male students who rate their family’s wealth high tend to report more parental involvement in their own education, whereas the opposite is true for female students. This pattern is in accordance with the TWH for parental investment. The results support the validity of evolutionary explanations of behavioral bias in the context of parental investment in offspring education

Why Do Individuals Seek Information? : A Selectionist Perspective

Several authors have proposed that mechanisms of adaptive behavior, and reinforcement learning in particular, can be explained by an innate tendency of individuals to seek information about the local environment. In this article, I argue that these approaches adhere to an essentialist view of learning that avoids the question why information seeking should be favorable in the first place. I propose a selectionist account of adaptive behavior that explains why individuals behave as if they had a tendency to seek information without resorting to essentialist explanations. I develop my argument using a formal selectionist framework for adaptive behavior, the multilevel model of behavioral selection (MLBS). The MLBS has been introduced recently as a formal theory of behavioral selection that links reinforcement learning to natural selection within a single unified model. I show that the MLBS implies an average gain in information about the availability of reinforcement. Formally, this means that behavior reaches an equilibrium state, if and only if the Fisher information of the conditional probability of reinforcement is maximized. This coincides with a reduction in the randomness of the expected environmental feedback as captured by the information theoretic concept of expected surprise (i.e., entropy). The main result is that behavioral selection maximizes the information about the expected fitness consequences of behavior, which, in turn, minimizes average surprise. In contrast to existing attempts to link adaptive behavior to information theoretic concepts (e.g., the free energy principle), neither information gain nor surprise minimization is treated as a first principle. Instead, the result is formally deduced from the MLBS and therefore constitutes a mathematical property of the more general principle of behavioral selection. Thus, if reinforcement learning is understood as a selection process, there is no need to assume an active agent with an innate tendency to seek information or minimize surprise. Instead, information gain and surprise minimization emerge naturally because it lies in the very nature of selection to produce order from randomness