Generalizing Research Findings for Enhanced Reproducibility: A Translational Medicine Case Study
14 Pages Posted: 12 Sep 2017 Last revised: 18 Jan 2022
Date Written: September 10, 2017
Abstract
Reproducibility of research findings is a much-debated topic. A related term in these discussions is replicability of results. A ramification of this discussion has led to statistical concerns over the interpretation of p-values and significance of effects used to identify factors, X, with effects on responses, Y. We suggest adapting a different perspective and focus on how to present findings that are claimed to be reproducible. Using examples from clinical research and translational medicine, we demonstrate this approach to the generalization of findings. Specifically, we map research findings into a conceptual boundary of meaning (BOM) that delineates a demarcation line between alternative representations of the research claims. Such a mapping of reproducibility claims can be statistically evaluated by tests of hypotheses controlling for sign type (S-type) errors determined by the study design. S-type errors apply to the verbally expressed direction (sign) of an effect. We suggest here that in scientific reports, details on the experimental set up of a study design, should be offered as supplementary material in order to support replicability studies. Our main message is that reproducibility is more important than replicability and that to inform about reproducibility claims, findings should be generalized with a BOM.
Keywords: Reproducible research, generalizability of findings, information quality, boundary of meaning
Suggested Citation: Suggested Citation