The coevolution of effort and replication, recreated, replicated and corrected

<p>We replicated &ldquo;The natural selection of bad science&rdquo; by&nbsp;Paul Smaldino&nbsp;and&nbsp;Richard McElreath (2016). The replication was successful with one exception. We find that selection acting on scientist&rsquo;s propensity for replication frequency caused a brief period of exuberant replication not observed in the original paper due to a coding error. This difference does not, however, change the authors&rsquo; original conclusions.</p> <p>The three panels displayed here concern Figure 5 &ndash; titled &ldquo;The coevolution of effort and replication&rdquo; &ndash; of the original study. Panel (a) is based on the original data and is a recreation of the original figure. Panel (b) is the result of a replication based on the same coding error and panel (c) displays the corrected result.</p> <p>While effort, false positive rate (&alpha;), and false-discovery rate converge to the originally reported values when the simulated steps are extended beyond the original 1e6 time steps, the replication rate&rsquo;s progress and convergence are different from the original.</p> <p>The results of this corrected model show the following pattern: Starting with a high effort, low effort replications are more attractive than conducting novel research (that is, employing this strategy received higher payoffs), which results in the replication rate reaching nearly 100% after ~730,000 steps. At this point the decline of effort has made low-effort novel research more attractive than low-effort replications (because publishing a novel positive result is associated with a higher payof than publishing a replication) and consequently the replication rate decreases again. With the decline of effort, alpha rises up to 0.67, comparable with the value reported in the study by Smaldino and McElreath (2016).</p> <p>Smaldino, P. E., &amp; McElreath, R. (2016). The natural selection of bad science. <em>Royal Society Open Science</em>, <em>3</em>(9), 160384. <a href="https://doi.org/10.1098/rsos.160384">https://doi.org/10.1098/rsos.160384</a></p>