February 08, 2011

Report: L.A. Times teacher rankings deeply flawed

Source: NYSUT Communications

From the Great Lakes Center for Education, Research and Practice:

L.A. Times teacher rankings deeply flawed, new report finds

February 8, 2011

"How Effective Are Los Angeles Elementary Teachers and Schools?" report is an inaccurate measure of teacher quality

EAST LANSING, Mich. (Feb. 8, 2011) - A widely reported analysis done of teachers in the Los Angeles Unified School District is a deeply flawed measure of teacher effectiveness, according to a report released today.

On Aug. 14, 2010, the Los Angeles Times published a controversial article that highlighted the results of a study conducted by Richard Buddin, senior economist at the RAND Corporation. Buddin's analysis erroneously asserted that teacher quality can be validly measured using student performance on the math and reading portions of the California Standardized Test. In conjunction with the article, the L.A. Times published a database of LAUSD teachers by name and their individual effectiveness ratings, which were based on Buddin's flawed analysis.

The review of Buddin's analysis was performed by Derek C. Briggs, chair of the Research and Evaluation Methodology Program at the University of Colorado at Boulder, and Colorado-Boulder doctoral student Ben Domingue. Key among their findings was that Buddin's analysis failed to consider several critical factors that impact student achievement, such as long-term achievement patterns and influence from their peers.

Buddin's model led to overly broad and simplistic generalizations about teacher effectiveness, often resulting in negative consequences for many potentially effective teachers and their students, Briggs and Domingue found. Buddin's approach, known as "value-added modeling," can lead to a number of false positives and negatives – many average teachers may have been falsely categorized as either effective or ineffective.

Briggs and Domingue developed a stronger alternative model that included students' long-term achievement patterns and peer influence. The researchers then compared the results of their model and Buddin's, and found that only 60.8 percent of teachers would retain the same effectiveness rating in both models for math outcomes, while only 46.4 percent of teachers would retain the same effectiveness rating for reading outcomes.

Notably, Briggs and Domingue found that Buddin's model could not be independently replicated, thus casting doubts about the scientific validity of the study altogether.

According to a Jan. 10 New York Times article, the New York City Board of Education is planning to follow Los Angeles' lead and publish ratings of 12,000 New York City teachers, based on an approach similar in nature to Buddin's.

Briggs and Domingue's report, "Due Diligence and the Evaluation of Teachers: A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by the Los Angeles Times," was produced by the National Education Policy Center (NEPC), housed at the University of Colorado at Boulder School of Education, with funding from the Great Lakes Center for Education Research and Practice.