20 Validating meta-science methods using simulated data ✎ Very rough draft
This chapter will explain, with worked examples and R/tidyverse/purrr code, how to validate implementations of methods against simulated data. It differentiates between exhaustive grid sweeps and probabilistic Monte Carlo simulations.
This form of validation is essential to understanding whether a method works as expected under idea conditions (i.e., assuming no text extraction, parsing, etc issues; just perfect inputs), in terms of false positives, false negatives, coverage, bias, etc.