Replication, Communication, and the Population Dynamics of Scientific Discovery

Many published research results are false, and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts---suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics

Social Conformity Despite Individual Preferences for Distinctiveness

We demonstrate that individual behaviors directed at the attainment of distinctiveness can in fact produce complete social conformity. We thus offer an unexpected generative mechanism for this central social phenomenon. Specifically, we establish that agents who have fixed needs to be distinct and adapt their positions to achieve distinctiveness goals, can nevertheless self-organize to a limiting state of absolute conformity. This seemingly paradoxical result is deduced formally from a small number of natural assumptions, and is then explored at length computationally. Interesting departures from this conformity equilibrium are also possible, including divergence in positions. The effect of extremist minorities on these dynamics is discussed. A simple extension is then introduced, which allows the model to generate and maintain social diversity, including multimodal distinctiveness distributions. The paper contributes formal definitions, analytical deductions, and counterintuitive findings to the literature on individual distinctiveness and social conformity.Comment: 11 pages, 6 figures, appendi

Open science and modified funding lotteries can impede the natural selection of bad science.

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behaviour on the part of individuals, via 'the natural selection of bad science.' Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow. However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements, which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favour of lotteries. We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modelling. We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigour, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review. In the absence of funding that targets rigour, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications

Adoption as a Social Marker: Innovation Diffusion with Outgroup Aversion

Social identities are among the key factors driving behavior in complex societies. Signals of social identity are known to influence individual behaviors in the adoption of innovations. Yet the population-level consequences of identity signaling on the diffusion of innovations are largely unknown. Here we use both analytical and agent-based modeling to consider the spread of a beneficial innovation in a structured population in which there exist two groups who are averse to being mistaken for each other. We investigate the dynamics of adoption and consider the role of structural factors such as demographic skew and communication scale on population-level outcomes. We find that outgroup aversion can lead to adoption being delayed or suppressed in one group, and that population-wide underadoption is common. Comparing the two models, we find that differential adoption can arise due to structural constraints on information flow even in the absence of intrinsic between-group differences in adoption rates. Further, we find that patterns of polarization in adoption at both local and global scales depend on the details of demographic organization and the scale of communication. This research has particular relevance to widely beneficial but identity-relevant products and behaviors, such as green technologies, where overall levels of adoption determine the positive benefits that accrue to society at large.Comment: 26 pages, 10 figure

The Origins of Options

Most research on decision making has focused on how human or animal decision makers choose between two or more options, posed in advance by the researchers. The mechanisms by which options are generated for most decisions, however, are not well understood. Models of sequential search have examined the trade-off between continued exploration and choosing one’s current best option, but still cannot explain the processes by which new options are generated. We argue that understanding the origins of options is a crucial but untapped area for decision making research. We explore a number of factors which influence the generation of options, which fall broadly into two categories: psycho-biological and socio-cultural. The former category includes factors such as perceptual biases and associative memory networks. The latter category relies on the incredible human capacity for culture and social learning, which doubtless shape not only our choices but the options available for choice. Our intention is to start a discussion that brings us closer toward understanding the origins of options