Non-Standard Errors

Tinbergen Institute Discussion Paper 2021-102/IV

63 Pages Posted: 14 Dec 2021

See all articles by Albert J. Menkveld

Albert J. Menkveld

Vrije Universiteit Amsterdam

Anna Dreber

Stockholm School of Economics - Department of Economics

Felix Holzmeister

University of Innsbruck - Department of Economics

Juergen Huber

University of Innsbruck

Magnus Johannesson

Stockholm School of Economics - Department of Economics

Michael Kirchler

University of Innsbruck

Sebastian Neusüss

Aalto University

Michael Razen

University of Innsbruck

Utz Weitzel

VU University Amsterdam

Multiple version iconThere are 3 versions of this paper

Date Written: November 13, 2021

Abstract

In statistics, samples are drawn from a population in a data- generating process (DGP). Standard errors measure the uncer- tainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.

JEL Classification: G1, C12, C18

Suggested Citation

Menkveld, Albert J. and Dreber, Anna and Holzmeister, Felix and Huber, Juergen and Johanneson, Magnus and Kirchler, Michael and Neusüss, Sebastian and Razen, Michael and Weitzel, Utz, Non-Standard Errors (November 13, 2021). Tinbergen Institute Discussion Paper 2021-102/IV, Available at SSRN: https://ssrn.com/abstract=3981597 or http://dx.doi.org/10.2139/ssrn.3981597

Albert J. Menkveld (Contact Author)

Vrije Universiteit Amsterdam ( email )

De Boelelaan 1105
Amsterdam, 1081HV
Netherlands
+31 20 5986130 (Phone)
+31 20 5986020 (Fax)

Anna Dreber

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden

Felix Holzmeister

University of Innsbruck - Department of Economics ( email )

Universitaetsstrasse 15
Innsbruck, 6020
Austria

HOME PAGE: http://www.holzmeister.biz

Juergen Huber

University of Innsbruck ( email )

Universitätsstraße 15
Innsbruck, Innsbruck 6020
Austria

Magnus Johanneson

Stockholm School of Economics - Department of Economics ( email )

P.O. Box 6501
Sveavagen 65
S-113 83 Stockholm
Sweden
+46 8 736 9443 (Phone)

HOME PAGE: http://www.hhs.se/Faculty/showperson.htm?personid=198

Michael Kirchler

University of Innsbruck ( email )

Universitätsstraße 15
Innsbruck, Innsbruck 6020
Austria

Sebastian Neusüss

Aalto University ( email )

P.O. Box 21210
Helsinki, 00101
Finland

Michael Razen

University of Innsbruck ( email )

Innsbruck
Austria

Utz Weitzel

VU University Amsterdam ( email )

De Boelelaan 1105
Amsterdam
Netherlands

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
580
Abstract Views
2,677
Rank
456
PlumX Metrics