Add pre-study odds and study power beside significance claims

Add annotation that states the pre-study odds and study power behind a claimed statistically significant finding. For example, pair the p-value threshold crossing with an estimated PPV or false-positive-risk readout and name the assumed R value used to interpret the result.

A significance threshold alone invites readers to equate statistical significance with truth when the paper shows that truth depends on more than the p-value.

Mechanism: Adding pre-study odds and power changes the readout from “the result passed 0.05” to “the result is more or less believable under explicit assumptions.”

Evidence: The paper argues that research should not be interpreted based only on p-values and shows that the post-study probability a finding is true depends on prior probability, study power, and the significance level (Ioannidis, 2005).

• User Goal: Judge whether a claimed relationship is likely true after one study.
• Task: Interpret a significance claim rather than only detect threshold crossing.
• Data: One or a few statistically significant findings from a single study.
• Chart Setting: A chart or table reports p-values, significance labels, or claimed effects.
• Audience: Readers assessing research credibility.
• Success Criterion: Readers can see the assumptions that make the claim more or less believable.

Break it when: The display does not make a positive research claim reaching statistical significance. Why: The paper defines PPV for claimed findings and focuses its argument on relationships investigators claim exist.

Sacrifice: You must expose assumptions about pre-study odds and power. Risk: Those assumptions can be debated because they are partly subjective. Mitigation: State the assumed values directly on the display.

Mistake: Report only p-values or threshold labels without stating prior-odds or power context. Why it fails: It leaves out the terms that the paper says determine whether a claimed finding is likely true.

Failure Sign: A significance claim has no stated R, power, PPV, or false-positive-risk context. Quick Check: Ask whether a reader could explain why the result is likely true without using only p < 0.05. Stronger Test: Verify that the display provides enough information to compute or inspect PPV under the stated assumptions.

• Add a PPV or false-positive-risk annotation next to each claimed finding.
• State the assumed pre-study odds or R value used for that readout.
• Add the study power used to interpret the claim.