Evidence-Based Recommendations for Designing Free-Sorting Experiments
Behavior Research Methods, Forthcoming.
48 Pages Posted: 17 Feb 2015 Last revised: 4 Aug 2015
Date Written: July 22, 2015
The card-sorting task is a very flexible research tool that is widely used across many of the subfields of psychology. Yet this same great flexibility requires researchers to make several (seemingly arbitrary) decisions in their designs, such as fixing a sufficient number of objects to sort, setting task requirements, and creating task instructions for participants. In the present research, we provide a systematic empirical investigation of the consequences of typical researcher design choices as they administer sorting tasks. Specifically, we study the effect of seven sorting task design factors by collecting data from over 1000 online participants assigned to one of 36 sorting tasks as part of a fractional factorial experimental design. Analyses show effects of the various researcher decisions on the probability that participants quit the task, the amount of time spent on the task, the number of piles made, and post-task measures such as satisfaction and depletion. Research design recommendations are provided.
Keywords: free-sorting, card sorting, experimental design, sorting task
JEL Classification: C9, C90, C91
Suggested Citation: Suggested Citation