Guidelines for Science: Evidence and Checklists

Scholarly Commons, 1-24-17

37 Pages Posted: 20 Oct 2017

See all articles by J. Scott Armstrong

J. Scott Armstrong

University of Pennsylvania - Marketing Department

Kesten C. Green

University of South Australia - UniSA Business; Ehrenberg-Bass Institute

Date Written: January 24, 2017


Problem: The scientific method is unrivalled as a basis for generating useful knowledge, yet research papers published in management, economics, and other social sciences fields often ignore scientific principles. What, then, can be done to increase the publication of useful scientific papers?

Methods: Evidence on researchers’ compliance with scientific principles was examined. Guidelines aimed at reducing violations were then derived from established definitions of the scientific method.

Findings: Violations of the principles of science are encouraged by: (a) funding for advocacy research; (b) regulations that limit what research is permitted, how it must be designed, and what must be reported; (c) political suppression of scientists’ speech; (d) universities’ use of invalid criteria to evaluate research — such as grant money and counting of publications without regard to usefulness; (e) journals’ use of invalid criteria for deciding which papers to publish — such as the use of statistical significance tests.

Solutions: We created a checklist of 24 evidence-based operational guidelines to help researchers comply with scientific principles (valid inputs). Based on the definition of science, we then developed a checklist of seven criteria to evaluate whether a research paper provides useful scientific findings (valuable outputs). That checklist can be used by researchers, funders, courts, legislators, regulators, employers, reviewers, and journals.

Originality: This paper provides the first comprehensive evidence-based checklists of operational guidelines for conducting scientific research and for evaluating the scientific quality and usefulness of research efforts.

Usefulness: Journals could increase the publication of useful papers by including a section committed to publishing all relevant and useful papers that comply with science. By using the Criteria for Useful Science checklist, those who support science could more effectively evaluate the contributions of scientists.

Keywords: advocacy, econometrics, evidence-based checklists, experiments, multiple hypotheses, objectivity, reg ression analysis, replication, statistical significance

Suggested Citation

Armstrong, J. Scott and Green, Kesten C., Guidelines for Science: Evidence and Checklists (January 24, 2017). Scholarly Commons, 1-24-17, Available at SSRN: or

J. Scott Armstrong (Contact Author)

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-5087 (Phone)
215-898-2534 (Fax)


Kesten C. Green

University of South Australia - UniSA Business ( email )

GPO Box 2471
Adelaide, SA 5001
+61 8 83012 9097 (Phone)


Ehrenberg-Bass Institute ( email )



Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
PlumX Metrics