Permutation Tests (and Sampling Without Replacement) Orders of Magnitude Faster Using SAS®

Abstract

Six permutation test algorithms coded in SAS® are compared. The fastest (“OPDN”), which uses no modules beyond Base SAS®, achieves speed increases orders of magnitude faster than the relevant “built-in” SAS® procedures (over 215x faster than Proc SurveySelect, over 350x faster than NPAR1WAY (which crashes on datasets less than a tenth the size OPDN can handle), and over 720x faster than Proc Multtest). OPDN combines SAS®’s fast sequential access, its most efficient array manipulation and memory usage, and a simple draw-by-draw procedure to very quickly and efficiently perform simple random sampling without replacement (SRSWOR). The particular draw-by-draw method used allows for the repeated creation of many SRSWOR permutation samples without requiring any additional storage or memory space. Based on these results, there appear to be no faster or more scalable permutation test or SRSWOR algorithms in SAS®.

Keywords: Permutation, SAS, Scalable, Without Replacement, Sampling, SRSWOR

JEL Classification: C12, C14, C15, C63, C88