3  Organising principles ✎ Very rough draft

3.1 Organising heuristic

  • Judgements and conclusions from forensic methods can be organised around three key concepts

3.2 Possibility

  • Can the reported results actually be as described?
  • Example: GRIM test — are the reported means possible given the sample size and measurement scale?

3.3 Plausibility

  • Are the reported results plausible, even if technically possible?
  • Example: Weird distributions under supposed randomisation at baseline
  • Example: p-values from multiple studies in a paper

3.4 Misalignment

  • Do the different parts of the article align with each other?

3.4.1 Measure-construct misalignment

  • Does the measure actually capture the construct it claims to?

3.4.2 Test-inference misalignment

  • Does the statistical test support the inference being drawn?

3.4.3 Claim misalignment

  • Do the claims in the article match what the methods and results can actually support?
  • Methods related? Design can’t tell you that.