rsabsence.Rnw

different questions and are better in different senses; the equivalence test is
calibrated to have certain frequentist error rates, which the Bayes factor is
not. The Bayes factor, on the other hand, seems to be a more natural measure of
evidence as it treats the null and alternative hypotheses symmetrically and
represents the factor by which rational agents should update their beliefs in
light of the data. Conclusions about whether or not a study can be replicated
should ideally be drawn using multiple methods. Replications that are successful
with respect to all methods provide more convincing support for the original
finding, while replications that are successful with only some methods require
closer examination. Fortunately, the use of multiple methods is already standard
practice in replication assessment (\eg{} the RPCB used seven different
methods), so our proposal does not require a major paradigm shift.



While the equivalence test and the Bayes factor are two principled methods for
analyzing original and replication studies with null results, they are not the
only possible methods for doing so. A straightforward extension would be to
first synthesize the original and replication effect estimates with a
meta-analysis, and then apply the equivalence and Bayes factor tests to the
meta-analytic estimate. This could potentially improve the power of the tests,
but consideration must be given to the threshold used for the
\textit{p}-values/Bayes factors, as naive use of the same thresholds as in the
standard approaches may make the tests too liberal.
% Furthermore, more advanced methods such as the
% reverse-Bayes approach from \citet{Micheloud2022} specifically tailored to
% equivalence testing in the replication setting may lead to more appropriate
% inferences as it also takes into account the compatibility of the effect
% estimates from original and replication studies. In addition, various other
% Bayesian methods have been proposed, which could potentially improve upon the
% considered Bayes factor approach
% \citep{Lindley1998,Johnson2010,Morey2011,Kruschke2018}.
Furthermore, there are various advanced methods for quantifying evidence for
absent effects which could potentially improve on the more basic approaches
considered here \citep{Lindley1998,Johnson2010,Morey2011,Kruschke2018,
  Micheloud2022}.
% For example, Bayes factors based on non-local priors \citep{Johnson2010} or
% based on interval null hypotheses \citep{Morey2011, Liao2020}, methods for
% equivalence testing based on effect size posterior distributions
% \citep{Kruschke2018}, or Bayesian procedures that involve utilities of
% decisions \citep{Lindley1998}.



\section*{Acknowledgements}
We thank the RPCB contributors for their tremendous efforts and for making their
data publicly available. We thank Maya Mathur for helpful advice on data
preparation. We thank Benjamin Ineichen for helpful comments on drafts of the
manuscript. Our acknowledgement of these individuals does not imply their
endorsement of our work. We thank the Swiss National Science Foundation for
financial support (grant
\href{https://data.snf.ch/grants/grant/189295}{\#189295}).



\section*{Conflict of interest}
We declare no conflict of interest.

\section*{Software and data}
The code and data to reproduce our analyses is openly available at
\url{https://gitlab.uzh.ch/samuel.pawel/rsAbsence}. A snapshot of the repository
at the time of writing is available at
\url{https://doi.org/10.5281/zenodo.XXXXXX}. We used the statistical programming
language R version \Sexpr{paste(version$major, version$minor, sep = ".")}
\citep{R} for analyses. The R packages \texttt{ggplot2} \citep{Wickham2016},
\texttt{dplyr} \citep{Wickham2022}, \texttt{knitr} \citep{Xie2022}, and
\texttt{reporttools} \citep{Rufibach2009} were used for plotting, data
preparation, dynamic reporting, and formatting, respectively. The data from the
RPCB were obtained by downloading the files from
\url{https://github.com/mayamathur/rpcb} (commit a1e0c63) and extracting the
relevant variables as indicated in the R script \texttt{preprocess-rpcb-data.R}
which is available in our git repository.

\bibliography{bibliography}

<< "sessionInfo1", eval = Reproducibility, results = "asis" >>=
## print R sessionInfo to see system information and package versions
## used to compile the manuscript (set Reproducibility = FALSE, to not do that)
cat("\\newpage \\section*{Computational details}")
@

<< "sessionInfo2", echo = Reproducibility, results = Reproducibility >>=
cat(paste(Sys.time(), Sys.timezone(), "\n"))
sessionInfo()
@

\end{document}