Exploring DIF: Comparison of CTT and IRT Methods
36 Pages Posted: 6 Sep 2010
Date Written: September 6, 2010
Assessment of test bias is important to establish the construct validity of tests. Assessment of differential item functioning (DIF) is an important first step in this process. DIF is present when examinees from different groups have differing probabilities of success on an item, after controlling for overall ability level. The study was conducted to answer the following questions: To what extent do the four methods (i.e. area differential index procedure for the 2- parameter Logistic model, TID, b-difference, and Chi-square) agree or disagree in the identification of DIF? Are there gender differences in mathematical proficiency? What is the content or nature of those items identified as revealing DIF? Achievement test covering the following subjects: Relations and functions, polynomial, Trigonometric functions, and triangles was developed. The test was administered to a sample of 1228 tenth grade students (656 males and 624 females) in Jordan. The study pointed out: (1) the percentage of agreement among the four methods in detecting DIF were from 41% to 85%. The highest agreement was between Chi-square and b-parameter difference methods (85%), whereas the lowest agreement was between Area index and TID methods (41%). The agreement among IRT based methods and CTT based methods were convergent. (3) females showed a statistically significant and consistent advantage over males on items involving Relations and functions, polynomial, Trigonometric functions, whereas men showed a less consistent advantage on items involving triangles, however It was concluded that gender differences in mathematics may well be linked to content.
Keywords: Transformed Item Difficulty, Area index , Chi-Square, b-Parameter Difference
Suggested Citation: Suggested Citation