Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data
The International Journal of Educational and Psychological Assessment, Vol. 1, No. 1, pp. 1-11, 2009
11 Pages Posted: 28 Jun 2009 Last revised: 6 Jul 2009
Date Written: May 26, 2009
Abstract
The present report demonstrates the difference between classical test theory (CTT) and item response theory (IRT) approach using an actual test data for chemistry junior high school students. The CTT and IRT were compared across two samples and two forms of test on their item difficulty, internal consistency, and measurement errors. The specific IRT approach used is the one-parameter Rasch model. Two equivalent samples were drawn in a private school in the Philippines and these two sets of data were compared on the tests’ item difficulty, split-half coefficient, Cronbach’s alpha, item difficulty using the Rasch model, person and item reliability (using Rasch model), and measurement error estimates. The results demonstrate certain limitations of the classical test theory and advantages of using the IRT. It was found in the study that (1) IRT estimates of item difficulty do not change across samples as compared with CTT with inconsistencies; (2) difficulty indices were also more stable across forms of tests than the CTT approach; (3) IRT internal consistencies are very stable across samples while CTT internal consistencies failed to be stable across samples; (4) IRT had significantly less measurement errors than the CTT approach. Perspectives for stakeholders in test and measurement are discussed.
Keywords: Rasch Model, IRT, item response theory, Chemistry test, item difficulty
Suggested Citation: Suggested Citation