Constructing Tests that Can Measure and Compare the Maths and Physics Skills of Engineering Students in Russia and China

28 Pages Posted: 3 Sep 2015

See all articles by Elena Kardanova

Elena Kardanova

National Research University Higher School of Economics (Moscow)

Ekaterina Enchikova

National Research University Higher School of Economics (Moscow)

H. Shi

Stanford University

Natalie Johnson

Stanford University

Lydia Liu

Educational Testing Service

Liyang Mao

Educational Testing Service

Prashant Kumar Loyalka

Stanford University - Freeman Spogli Institute for International Studies

Date Written: September 3, 2015

Abstract

Although the number of engineering graduates has expanded rapidly in the last two decades, relatively little is known about the quality of engineering programs worldwide. In particular, few studies look at differences in the degree to which students are learning skills across different engineering programs within and between countries. There is particular interest in the investigation of the engineering education quality in the countries with the rapidly growing economy, such as BRICS countries. Until now, there was little research in this field and one of the main reasons for this is the difficulty in developing an assessment approach and the accompanying set of instruments, which would allow for measurement and international comparison. Our study describes a set of procedures for developing such an assessment framework of instruments, to measure and compare skill levels and gains across engineering programs.

We first describe a systematic approach for constructing cross-nationally comparable instruments in maths and physics for students in the first two years of their undergraduate engineering programs. The approach includes both a priori procedures (including expert assessments to avoid construct, method, and item bias), and a posteriori procedures (including the psychometric analysis of test quality, differential item functioning, and identifying and reducing bias in the data).

In addition to describing this set of procedures in theory, we also show how we systematically implemented these procedures. Drawing on data that we collected from over 24 engineering experts and 3,600 engineering students across Russia and China, we provide evidence that it is possible to create tests that are cross-culturally valid, equate-able, and free from bias.

Keywords: engineering education, BRIC countries, quality of education, cross-cultural measurement

JEL Classification: Z

Suggested Citation

Kardanova, Elena and Enchikova, Ekaterina and Shi, H. and Johnson, Natalie and Liu, Lydia and Mao, Liyang and Loyalka, Prashant, Constructing Tests that Can Measure and Compare the Maths and Physics Skills of Engineering Students in Russia and China (September 3, 2015). Higher School of Economics Research Paper No. WP BRP 28/EDU/2015, Available at SSRN: https://ssrn.com/abstract=2655534 or http://dx.doi.org/10.2139/ssrn.2655534

Elena Kardanova (Contact Author)

National Research University Higher School of Economics (Moscow) ( email )

Myasnitskaya street, 20
Moscow, Moscow 119017
Russia

Ekaterina Enchikova

National Research University Higher School of Economics (Moscow) ( email )

Myasnitskaya street, 20
Moscow, Moscow 119017
Russia

H. Shi

Stanford University ( email )

Stanford, CA 94305
United States

Natalie Johnson

Stanford University ( email )

Stanford, CA 94305
United States

Lydia Liu

Educational Testing Service ( email )

United States

Liyang Mao

Educational Testing Service ( email )

United States

Prashant Loyalka

Stanford University - Freeman Spogli Institute for International Studies ( email )

Stanford, CA 94305
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
70
Abstract Views
816
Rank
594,428
PlumX Metrics