Like many New York City teachers, Caitlin Duffy was upset when she first saw her middling rating on her performance report. But alarm bells truly started ringing when she saw the list of students it was based on, and realized she had not taught half of them.
Ms. Duffy, who teaches at The Computer School, a Manhattan middle school, is one of 18,000 public school teachers whose ratings the city released last week, after the teachers' union lost a court battle to keep them private. The idea behind the reports was simple: take students’ test scores, match them to their teachers and use the gains as a measure of job performance. But in the city's school system, the country’s largest, the task of extracting accurate information from multiple data systems and old files has proved more complicated.
By the Education Department’s calculations, 2,700 of the teachers who received data reports for the 2009-10 school year found at least one error.
Teachers had access to their reports a year ago, but many never checked. Of the 12,170 teachers who were given ratings for that year, just under half went on the department’s Web site to verify that the information was correct, officials said.
Ms. Duffy said that when she called the Education Department last year to correct her class list, she was told the city did not have a data report for her. So she was surprised last week to receive a rating that held her responsible for teaching most of the sixth grade, when she in fact taught only half. The report also listed some seventh-grade students as sixth graders. Another sixth-grade math teacher at her school did not get a data report.
“We’re talking upwards of 50 percent are students I didn’t teach,” she said. “And with the kinds of adjustments they’re making, if there are even a few inaccuracies, it would throw your data off.”
After The New York Times published the ratings, dozens of teachers wrote in to say that their scores were based on inaccurate information.
Teachers who work in teams of two — one person teaching math, the other English — said they were rated for the wrong subject. Others had the correct subject, but the wrong grade level. And women who went on maternity leave several years back made the unwelcome discovery that they had been rated when they were not in the classroom for the entire year.
Education officials said the most common mistake that teachers corrected was having too few students on their class rosters, but they said that at this point, changes would not be made to the reports just released.
Since the city began ranking teachers in the 2007-08 school year, there have been complaints about inaccurate class lists and teaching assignments. After teachers submitted corrections last year, the department was able to update those reports.
The rankings that were just released are nearly two years out of date, and education officials acknowledge that they have a wide margin of error. So far there have yet to be reported cases of parents knocking down principals’ doors, demanding their children switch into an “above average” teacher’s class. Still, teachers say that these records are the first to be made public, and have become a permanent assessment of their careers, whether accurate or not.
Under a new state evaluation system, the rankings will be used to calculate at least 20 percent of their overall scores and two years of poor ratings could lead to dismissal. With their scores about to become high stakes, many teachers are questioning the fairness and accuracy of the value-added formula.
City officials said they had developed a program called Class List Reporting to generate accurate student and classroom data for the state reports. But some school leaders said their confidence in the program was shaken during a January trial when teachers’ names and class lists were missing. Officials said they were aware of the problems and had until June to submit records to the state, giving them time to correct technical glitches.
“I think they have to be more careful in collecting data,” said Jamie Lilly, a teacher at Public School 89 in Manhattan, who was rated “average” and discovered that several of her students were not included in her report. “Personally, I tried to let it roll off my back because I know it’s wrong anyway.”