Friday, August 27, 2010

One number can't illustrate teacher effectiveness

Los Angeles Times article "One number can't illustrate teacher effectiveness" (August 25th, 2010) criticizes Richard Buddin's scoring of Los Angeles teachers, arguing that it isn't taking everything into account and that there's an "ethical issue" publishing his research:

Given analytic weaknesses, the ethical question that arises is whether The Times is on sufficiently firm empirical ground to publish a single number, purporting to gauge the sum total of a teacher's effect on children.

Corrections had a number of issues with the article's criticism of Buddin's procedure. However, the more interesting issue was the author's raising of "ethicality" in the Times decision to publish Buddin's research. Theory suggests that unions may raise short-run pay but they flatten skill differentials. Perhaps to that end, teacher's unions have nearly uniformly opposed the ranking of teachers and anything that might help that process.

One reason for this is so fellow travelers might then criticize any empirical ranking that breaks a union's power by encouraging best practices and getting rid of bad teachers. What, then, might a strategy be to break this cartel of unions denying statistics, and education professors criticizing studies using bad statistics? To publish the best studies we can using the statistics we are given. Why?

Let us imagine that there are two types of teachers, good teachers, and bad teachers. They both have the same baseline utility. Both their utilities may be decreased by increased supervision. However, given that they'll be ranked, good teachers would rather have good statistics. The idea is displayed graphically below (click to enlarge).


What might this do? If they aren't being graded, both good teachers and bad teachers prefer to obstruct good statistics being collected. However, if they are being graded, good teachers now prefer good statistics while bad teachers like them even less. However, a wedge has now been created, and it's an empirical question whether or not good teachers will be able to outvote bad teachers in the quest for the collection better statistics.

In this vein, publishing well-done, competent research that admits its flaws (which an article criticizing it then rehashes) and encourages the destruction of union power to the detriment of bad teachers and benefit of students would appear the only "moral" choice.

1 comment:

  1. Over the decades we have massively increased the real investment in public schools per child and yet have very little gain in performance. Anybody who has had children in public schools and has seen the sustained tolerance of notably incompetent teachers and their protection by the unions understands that no matter how much money is poured into the system nothing useful will come from the investment as long as gross incompetence is tolerated. If one's focus is on the interest of the students or on the interest of the nation there is no doubt that the same normal means of using compensation as an incentive to improved performance employed in every business should be applied to education. The main caveat is that the testing instruments be well conceived. Current levels of competence among students are so appalling that even simple measured asking questions that have undeniable face validity are a start.

    ReplyDelete