Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data

The International Journal of Educational and Psychological Assessment, Vol. 1, No. 1, pp. 1-11, 2009

11 Pages Posted: 28 Jun 2009 Last revised: 6 Jul 2009

See all articles by Carlo Magno

Carlo Magno

De La Salle University, Manila; National University

Date Written: May 26, 2009

Abstract

The present report demonstrates the difference between classical test theory (CTT) and item response theory (IRT) approach using an actual test data for chemistry junior high school students. The CTT and IRT were compared across two samples and two forms of test on their item difficulty, internal consistency, and measurement errors. The specific IRT approach used is the one-parameter Rasch model. Two equivalent samples were drawn in a private school in the Philippines and these two sets of data were compared on the tests’ item difficulty, split-half coefficient, Cronbach’s alpha, item difficulty using the Rasch model, person and item reliability (using Rasch model), and measurement error estimates. The results demonstrate certain limitations of the classical test theory and advantages of using the IRT. It was found in the study that (1) IRT estimates of item difficulty do not change across samples as compared with CTT with inconsistencies; (2) difficulty indices were also more stable across forms of tests than the CTT approach; (3) IRT internal consistencies are very stable across samples while CTT internal consistencies failed to be stable across samples; (4) IRT had significantly less measurement errors than the CTT approach. Perspectives for stakeholders in test and measurement are discussed.

Keywords: Rasch Model, IRT, item response theory, Chemistry test, item difficulty

Suggested Citation

Magno, Carlo and Magno, Carlo, Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data (May 26, 2009). The International Journal of Educational and Psychological Assessment, Vol. 1, No. 1, pp. 1-11, 2009, Available at SSRN: https://ssrn.com/abstract=1426043

Carlo Magno (Contact Author)

National University ( email )

11255 North Torrey Pines Road
La Jolla, CA 92037
United States

De La Salle University, Manila ( email )

2401 Taft. Ave.
Manila, 4002
Philippines

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,410
Abstract Views
6,972
Rank
25,556
PlumX Metrics