Monday, July 20, 2015

The Difference a Teacher Makes

I had an interesting natural experiment this past spring semester: I was teaching two remedial algebra courses, one in the afternoon, and one in the evening. Same calendar days, same class sizes, identical lectures from day to day, exact same tests, exact same numbers taking the final exam. In one class, only 23% of the registrants passed the final exam, while in the other class 60% passed. (Median scores on the final were 48% in one class and 80% in the other.)

This got me to wondering: How much difference does the teacher make in these classes? And the honest answer is: not very much. To be humble about it, I could do everything humanly possible both inside and outside the classroom as a teacher, work at maximal effort all the time (and that is generally my goal), and have it make very little difference in the overall classroom result. The example here of enormous variation between two sections, identically treated by me as an instructor, really highlights this fact.

On this point, I found a 2013 paper from ETS by Edward H. Haertel -- principally about the unreliability of teacher VAM scores -- that summarizes several studies as finding that the difference in test scores attributable to teacher proficiency is only about 10% (see p. 5). That actually seems about right based on my recent experiences.

1 comment:

  1. I suppose of more advanced research question might be: can we identify students who are truly prepared to benefit from a class (that is: are not handicapped by overriding out-of-school factors), and considering those students, how much difference does the teacher make for them?