Skip to content
Snippets Groups Projects
Commit 6a7491cd authored by Rachel Heyard's avatar Rachel Heyard
Browse files

final bit of polishing

parent 06b278fd
No related branches found
No related tags found
1 merge request!2Revision
......@@ -248,7 +248,7 @@ hypothesis testing --- that can address the limitations of the non-significance
criterion. We use the null results replicated in the RPCB to illustrate the
problems of the non-significance criterion and how they can be addressed. We
conclude the paper with practical recommendations for analyzing replication
studies of original null results, including R code for applying the proposed
studies of original null results, including simple R code for applying the proposed
methods.
<< "data" >>=
......@@ -298,7 +298,7 @@ conflevel <- 0.95
Figure~\ref{fig:2examples} shows effect estimates on standardized mean
difference (SMD) scale with \Sexpr{round(100*conflevel, 2)}\% confidence
intervals from two RPCB study pairs. In both study pairs, the original and
replications studies are ``null results'' and therefore meet the
replication studies are ``null results'' and therefore meet the
non-significance criterion for replication success (the two-sided
\textit{p}-values are greater than 0.05 in both the original and the
replication study). However, intuition would suggest that the conclusions in the
......@@ -598,7 +598,7 @@ mean difference effect estimates with \Sexpr{round(conflevel*100, 2)}\%
confidence intervals for all 15 effects which were treated as null results by
the RPCB.\footnote{There are four original studies with null effects for which
two or three ``internal'' replication studies were conducted, leading in total
to 20 replications of null effects. As in the RPCB main analysis
to 20 replications of null effects. As done in the RPCB main analysis
\citep{Errington2021}, we aggregated their SMD estimates into a single SMD
estimate with fixed-effect meta-analysis and recomputed the replication
\textit{p}-value based on a normal approximation. For the original studies and
......@@ -714,7 +714,7 @@ much different from one indicates absence of evidence for either hypothesis
% the alternative over the null $\BF_{10}$. These have to be either interpreted
% in opposite direction or can be reoriented by $\BF_{01} = 1/\BF_{10}$.}.
A reasonable criterion for successful replication of a null result may hence be
to require a Bayes factor larger than some level $\gamma > 1$ from both studies,
to require both studies to report a Bayes factor larger than some level $\gamma > 1$,
for example, $\gamma = 3$ or $\gamma = 10$ which are conventional levels for
``substantial'' and ``strong'' evidence, respectively \citep{Jeffreys1961}. In
contrast to the non-significance criterion, this criterion provides a genuine
......@@ -1099,7 +1099,7 @@ translate into margins of $\Delta = % \log(1.3)\sqrt{3}/\pi =
\Sexpr{round(log(1.3)*sqrt(3)/pi, 2)}$ and $\Delta = % \log(1.25)\sqrt{3}/\pi =
\Sexpr{round(log(1.25)*sqrt(3)/pi, 2)}$ on the SMD scale, respectively, using
the $\text{SMD} = (\surd{3} / \pi) \log\text{OR}$ conversion \citep[p.
233]{Cooper2019}. Similarly, for the Bayesian factor we specified a normal
233]{Cooper2019}. Similarly, for the Bayes factor we specified a normal
unit-information prior under the alternative while other normal priors with
smaller/larger standard deviations could have been considered. Here, we
therefore investigate the sensitivity of our conclusions with respect to these
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment