1 research outputs found
You Can't Publish Replication Studies (and How to Anyways)
Reproducibility has been increasingly encouraged by communities of science in
order to validate experimental conclusions, and replication studies represent a
significant opportunity to vision scientists wishing contribute new perceptual
models, methods, or insights to the visualization community. Unfortunately, the
notion of replication of previous studies does not lend itself to how we
communicate research findings. Simple put, studies that re-conduct and confirm
earlier results do not hold any novelty, a key element to the modern research
publication system. Nevertheless, savvy researchers have discovered ways to
produce replication studies by embedding them into other sufficiently novel
studies. In this position paper, we define three methods -- re-evaluation,
expansion, and specialization -- for embedding a replication study into a novel
published work. Within this context, we provide a non-exhaustive case study on
replications of Cleveland and McGill's seminal work on graphical perception. As
it turns out, numerous replication studies have been carried out based on that
work, which have both confirmed prior findings and shined new light on our
understanding of human perception. Finally, we discuss how publishing a true
replication study should be avoided, while providing suggestions for how vision
scientists and others can still use replication studies as a vehicle to
producing visualization research publications