Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data

The International Journal of Educational and Psychological Assessment, Vol. 1, No. 1, pp. 1-11, 2009

11 Pages Posted: 28 Jun 2009 Last revised: 6 Jul 2009

Carlo Magno

De La Salle University, Manila

Date Written: May 26, 2009

Abstract

The present report demonstrates the difference between classical test theory (CTT) and item response theory (IRT) approach using an actual test data for chemistry junior high school students. The CTT and IRT were compared across two samples and two forms of test on their item difficulty, internal consistency, and measurement errors. The specific IRT approach used is the one-parameter Rasch model. Two equivalent samples were drawn in a private school in the Philippines and these two sets of data were compared on the tests’ item difficulty, split-half coefficient, Cronbach’s alpha, item difficulty using the Rasch model, person and item reliability (using Rasch model), and measurement error estimates. The results demonstrate certain limitations of the classical test theory and advantages of using the IRT. It was found in the study that (1) IRT estimates of item difficulty do not change across samples as compared with CTT with inconsistencies; (2) difficulty indices were also more stable across forms of tests than the CTT approach; (3) IRT internal consistencies are very stable across samples while CTT internal consistencies failed to be stable across samples; (4) IRT had significantly less measurement errors than the CTT approach. Perspectives for stakeholders in test and measurement are discussed.

Keywords: Rasch Model, IRT, item response theory, Chemistry test, item difficulty

Suggested Citation

Magno, Carlo, Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data (May 26, 2009). The International Journal of Educational and Psychological Assessment, Vol. 1, No. 1, pp. 1-11, 2009. Available at SSRN: https://ssrn.com/abstract=1426043

Carlo Magno (Contact Author)

De La Salle University, Manila ( email )

2401 Taft. Ave.
Manila, 4002
Philippines

Paper statistics

Downloads
837
Rank
21,887
Abstract Views
4,444