Use a multi-view evidence summary when several studies address the same question
For comparing evidence on one research question across several studies, use a multi-view structure on grouped-result displays to improve trust and mitigate overinterpretation of isolated statistically significant findings for audiences judging research credibility.
- purpose:select
- basis:empirical
- task:compare
- structure:multi-view:use
- structure:single-view:avoid
- quality:trust:use
- lever:layout-structure
- communication:credibility
advice
Show all relevant studies together
Use a multi-view structure to show all relevant studies on the same question together. For example, replace a single highlighted significant study with one grouped evidence view that includes independent studies on the question and keeps both positive and negative results visible.
reason
Why grouped evidence works
A grouped evidence view keeps readers from treating one significant study as conclusive when several teams have tested the same question.
Mechanism: Showing the full set of related studies shifts interpretation from isolated discovery to total evidence, so contradictory and nonconfirming results remain part of the reading task.
Evidence: The paper states that emphasizing isolated discoveries by single teams is misleading when many teams study similar questions and that what matters is the totality of the evidence rather than one statistically significant result in isolation (Ioannidis, 2005).
context
Use when several teams have tested one question
- User Goal: Judge whether a claimed relationship is supported across studies.
- Task: Compare confirming, nonconfirming, and refuting results on the same question.
- Data: Several independent studies or teams have addressed the same or similar question.
- Chart Setting: A result summary currently highlights one study or one significant finding.
- Audience: Readers deciding how credible the research claim is.
- Success Criterion: Readers can see the pattern of support and refutation without treating one result as conclusive.
exceptions
Do not use when the question is not a repeated-test case
Break it when: Only one study exists on the question or the display already summarizes the totality of the evidence. Why: The source warning is about isolated single-study emphasis in settings with repeated independent testing.
costs
Costs of grouping the evidence
Sacrifice: You give up the simplicity of a single isolated discovery. Risk: The grouped view may surface contradictory results instead of one clean answer. Mitigation: Keep the grouping scoped to one research question.
mistakes
Common single-study framing mistake
Mistake: Keep the statistically significant study as the only visible result even though several teams have examined the question. Why it fails: It reproduces the isolated-discovery framing the paper identifies as misleading.
check
Check whether the display overstates one study
Failure Sign: A multi-study question is shown with one statistically significant study and no neighboring results. Quick Check: Build a version that places all independent studies on the same question together; if the interpretation changes, the single-view version is failing. Stronger Test: Ask whether a reader can tell from the display how many teams found, failed to find, or refuted the relationship.
fix
Fix the isolated-discovery view
- Replace the single highlighted result with one grouped view for the full question.
- Add the independent studies that tested the same or similar question.
- Keep non-significant or refuting results visible instead of showing only the first positive claim.