Can Multifactor Models of Teaching Improve Teacher Effectiveness Measures?

14 Pages Posted: 10 Sep 2014 Last revised: 12 Sep 2014

See all articles by Val Lazarev

Val Lazarev

Empirical Education Inc.

Denis Newman

Empirical Education Inc.

Date Written: March 1, 2014

Abstract

NCLB waiver requirements have led to development of teacher evaluation systems, in which student growth is a significant component. Composite teacher evaluation scores commonly sum up the results of measurements made using several instruments. We hypothesize that, across different measures, there is more than one underlying factor and each measure can be decomposed into distinct factors. By performing factor analysis on the disaggregated evaluation data (observation components and survey items), we can identify several orthogonal factors, of which only one is associated with student test performance. We use teacher evaluation data collected by the Measures of Effective Teaching (MET) project as a model of state teacher evaluation system, which includes a value-added score as a measure of student performance, an observational rubric, and a student survey.

We find that one possible model of the latent data structure has three factors, of which only one is associated with value-added scores. This factor is also strongly associated with observation and survey items that deal with classroom control. The second factor is associated with such aspects of pedagogy as classroom dialog, questioning techniques, etc. The third factor is associated with items related to the notions of teacher sensitivity to students’ well-being and includes mostly student survey items. Those factors can be interpreted as reflecting “effective,” “constructive,” and “positive” dimensions of teaching respectively. Each of them is an important and independent input into child development, while only first of them leads to achievement gains measurable in the short run.

The importance for policy of identifying orthogonal underlying factors is, first, that it provides precise knowledge of what exactly the evaluation system is measuring. This knowledge can be used by policy-makers to make an informed decision on how to combine the aspects of teaching into a single “teacher utility” function or, alternatively, if a multidimensional (matrix) evaluation system should be used. Second, it makes clear that teacher effectiveness consists of more than just the ability to promote student growth as measured by test scores. Additional factors may be weighted differently, for example, in identifying a teacher to become a mentor vs. to become a principal, which may require interpersonal capabilities unrelated to promoting student growth. Third, evaluating teachers using several independent factor scores may help target resources (such as professional development) more accurately.

NCLB waiver requirements have led to development of teacher evaluation systems, in which student growth is a significant component. Composite teacher evaluation scores commonly sum up the results of measurements made using several instruments. We hypothesize that, across different measures, there is more than one underlying factor and each measure can be decomposed into distinct factors. By performing factor analysis on the disaggregated evaluation data (observation components and survey items), we can identify several orthogonal factors, of which only one is associated with student test performance. We use teacher evaluation data collected by the Measures of Effective Teaching (MET) project as a model of state teacher evaluation system, which includes a value-added score as a measure of student performance, an observational rubric, and a student survey.

We find that one possible model of the latent data structure has three factors, of which only one is associated with value-added scores. This factor is also strongly associated with observation and survey items that deal with classroom control. The second factor is associated with such aspects of pedagogy as classroom dialog, questioning techniques, etc. The third factor is associated with items related to the notions of teacher sensitivity to students’ well-being and includes mostly student survey items. Those factors can be interpreted as reflecting “effective,” “constructive,” and “positive” dimensions of teaching respectively. Each of them is an important and independent input into child development, while only first of them leads to achievement gains measurable in the short run.

The importance for policy of identifying orthogonal underlying factors is, first, that it provides precise knowledge of what exactly the evaluation system is measuring. This knowledge can be used by policy-makers to make an informed decision on how to combine the aspects of teaching into a single “teacher utility” function or, alternatively, if a multidimensional (matrix) evaluation system should be used. Second, it makes clear that teacher effectiveness consists of more than just the ability to promote student growth as measured by test scores. Additional factors may be weighted differently, for example, in identifying a teacher to become a mentor vs. to become a principal, which may require interpersonal capabilities unrelated to promoting student growth.

Keywords: empirical education, Measures of Effective Teaching (MET) project, teacher evaluation, disaggregated evaluation data, NCLB waiver requirements, teacher evaluation systems, Composite teacher evaluation scores, evaluation system, multidimensional (matrix) evaluation

Suggested Citation

Lazarev, Valeriy and Newman, Denis, Can Multifactor Models of Teaching Improve Teacher Effectiveness Measures? (March 1, 2014). Available at SSRN: https://ssrn.com/abstract=2493544 or http://dx.doi.org/10.2139/ssrn.2493544

Valeriy Lazarev (Contact Author)

Empirical Education Inc.

425 Sherman Ave, #210
Palo Alto, CA 94306
United States
16503281734 (Phone)

HOME PAGE: http://empiricaleducation.com/

Denis Newman

Empirical Education Inc. ( email )

425 Sherman Ave, #210
Palo Alto, CA 94306
United States

HOME PAGE: http://empiricaleducation.com/

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
54
Abstract Views
581
rank
408,075
PlumX Metrics