April 17, 2014

Professional statisticians come out against public school wreckers

Valerie Strauss, Washington Post - You can be certain that members of the American Statistical Association, the largest organization in the United States representing statisticians and related professionals, know a thing or two about data and measurement. That makes the statement that the association just issued very important for school reform.

The ASA just slammed the high-stakes “value-added method” (VAM) of evaluating teachers that has been increasingly embraced in states as part of school-reform efforts. VAM purports to be able to take student standardized test scores and measure the “value” a teacher adds to student learning through complicated formulas that can supposedly factor out all of the other influences and emerge with a valid assessment of how effective a particular teacher has been.

These formulas can’t actually do this with sufficient reliability and validity, but school reformers have pushed this approach and now most states use VAM as part of teacher evaluations. Because math and English test scores are available, reformers have devised bizarre implementation methods in which teachers are assessed on the test scores of students they don’t have or subjects they don’t teach. When Michelle Rhee was chancellor of D.C. public schools (2007-10), she was so enamored with using student test scores to evaluate adults that she implemented a system in which all adults in a school building, including the custodians, were in part evaluated by test scores.

Assessment experts have been saying for years that this is an unfair way to evaluate anybody, especially for high-stakes purposes such as pay, employment status, tenure or even the very survival of a school. But reformers went ahead anyway on the advice of some economists who have embraced the method (though many other economists have panned it).

You can be certain that members of the American Statistical Association, the largest organization in the United States representing statisticians and related professionals, know a thing or two about data and measurement. That makes the statement that the association just issued very important for school reform.

The ASA just slammed the high-stakes “value-added method” (VAM) of evaluating teachers that has been increasingly embraced in states as part of school-reform efforts. VAM purports to be able to take student standardized test scores and measure the “value” a teacher adds to student learning through complicated formulas that can supposedly factor out all of the other influences and emerge with a valid assessment of how effective a particular teacher has been.

These formulas can’t actually do this with sufficient reliability and validity, but school reformers have pushed this approach and now most states use VAM as part of teacher evaluations. Because math and English test scores are available, reformers have devised bizarre implementation methods in which teachers are assessed on the test scores of students they don’t have or subjects they don’t teach. When Michelle Rhee was chancellor of D.C. public schools (2007-10), she was so enamored with using student test scores to evaluate adults that she implemented a system in which all adults in a school building, including the custodians, were in part evaluated by test scores.

Assessment experts have been saying for years that this is an unfair way to evaluate anybody, especially for high-stakes purposes such as pay, employment status, tenure or even the very survival of a school. But reformers went ahead anyway on the advice of some economists who have embraced the method (though many other economists have panned it)....  Now the statisticians have come out with recommendations for the use of VAM for teachers, principals and schools that school reformers should — but most likely won’t — take to heart.

1 comment:

Anonymous said...

Why the hell did it take them so long??!? It's been known ever since Bloom did his Taxonomy of Educational Objectives that standardised tests don't measure what they purport to measure.