Streams

Teacher Ratings: What Are Other Cities Doing?

Wednesday, December 08, 2010

New York City isn't the only district using student test scores to measure which teachers are most effective. The Obama administration is using federal Race to the Top grants and other funds to encourage states and districts to develop similar systems. Some are farther along than others. Below is a brief description of a few places that use the data.

Los Angeles

In August, The Los Angeles Times published the names and ratings for 6,000 teachers whose effectiveness it determined through its own analysis. The Times linked individual teachers to their students' test scores, because the city doesn't normally use value-added data to evaluate teachers.

California has its own system of grading schools based on how their students perform on standardized tests each year. The Times requested detailed records it could match with teachers. Under the California Public Records Act, information pertaining to public employees must be made available to the public. The Los Angeles Times created its own "value-added" analysis by measuring the difference between a student's expected test performance and the actual score. This determined the teacher’s value-added score.

The Times hired Richard Buddin, senior economist at Rand Corporation, to crunch the data (Rand was not involved in the analysis). The paper found teaching quality varied more within individual schools than across the the district. On December 5, the paper published a story finding some of the most effective teachers were the ones laid-off because of budget cuts.

Buddin stands by the accuracy of his analysis, but warns that value-added scores are only a portion of a teacher’s overall evaluation and were never intended to stand alone. But they get a lot of attention because other "soft" data, such as classroom observations, aren't quantified the same way.

The teachers union called releasing the data "dangerous," according to an article in the Times. "Publishing the database is irresponsible and disrespectful to the hard working teachers of Los Angeles," said David Sanchez, President of the California Teachers Association. A fifth grade teacher took his own life shortly after the data was publicized. He had been ranked less effective than his peers. No direct link was ever established, though, between the teacher's suicide and the evaluation.

Those who support publicizing value-added data claim that parents have the right to know a teacher's scores. Education Secretary Arne Duncan said "this is an extraordinary opportunity to take teaching in L.A. to another level -- to identify those who year after year are getting great results in difficult circumstances," he told the Times. "It can really empower teachers to strengthen their craft and find out who are the great teachers around them." The state's education secretary, Bonnie Reiss, also told the paper, "Publishing this data is not about demonizing teachers. It's going to create a more market-driven approach to results."

Los Angeles is the first city where teacher's individual ratings were publicized.

Tennessee

Tennessee is considered the national leader in value-added data, which it has used since 1993. In fact, the methodology was first developed in Tennessee by Dr. William Sanders, a former math professor.

Individual teacher scores are available only to school administrators, but district and school information is available to the public. These data do not include demographic information on students, such as race or income, because the state says all students should progress at the same rate. Teachers aren't given incentive bonuses, but schools can get extra money if their scores go up.

Value-added data is used to account for eight percent of a teacher's evaluation. But starting this year, it accounts for a greater share, following an agreement between the governor and the unions to qualify for federal Race to the Top funding (Tennessee and Delaware were the first states to receive these coveted grants). The Tennessee Education Association has supported relying more on value-added data. Jerry Winters, TEA chief lobbyist, says "I think we’ve stepped up to the plate and we’re being accountable.”

Houston

Houston gives bonuses to teachers based on several criteria, one of which is how effective the teachers are in raising student achievement, or "adding value." The bonuses ranged this year from $25 on the low end to as much as $15,530 with the typical award averaging $3,606. But the district doesn't teacher ratings with the public.

In 2008, the Houston Chronicle tried to get around that barrier by requesting the amounts paid to each teacher through the district's performance pay program. By looking at who got the bonuses, the paper could figure out which teachers got the highest ratings. The Houston United School District denied the request. But it was overruled by the Texas state attorney general’s office, allowing the Chronicle to publish the names of all the winners and the amounts they received as a bonus. Parents were therefore able to see which teachers were most effective based on bonus size.

In 2009, the Chronicle made a second request for a list of teachers who were in the top and bottom 10 percent in value-added rankings, along with their names and scores. This request was denied by the attorney general's office, on the grounds that state law classifies the evaluations as “confidential” and not public record. Once again, the paper only published the names of those teachers who received the bonuses.

In January 2010, a total of 88 percent of eligible teachers received bonuses. Houston’s use of value-added data has been met with consistent opposition from the Houston Federation of Teachers and its president, Gayle Fallon. She told WNYC that Houston’s use of data is “divisive” and an unreliable resource for making high-stakes personnel decisions. She opposes its lack of transparency, because teachers don't know the formula used by the district's consultant, and she says the data is a poor representation of the complexities of the Houston schools.

But the group Parent Visionaries disagrees. A spokesperson, Mary Nesbit, says “information is really insightful." Nesbit calls the data a "very useful tool for evaluating a school’s quality." But she acknowledges, "it’s not the only tool. I think it’s important to understand what conclusions you can’t draw from this information. There can be a lot of misinformation.”

Ultimately, she says the data "changed the conversation about what parents can expect from schools and what they insist upon from schools.” There's now an expectation, she says, that students can grow every year. "Parents will no longer accept a bad year,” she says.

Denver

Denver uses a teacher bonus program similar to that used in Houston. It's called "ProComp," an alternative pay program. As in Houston, there are several factors that are weighted in determining who gets the bonus. Denver teachers can be rewarded for advancing their teaching qualifications by pursuing a master's or attending seminars, teaching in hard-to-serve communities and receiving successful evaluations. The teacher's impact on student test scores is also a factor.

In the 2009-10 school year, $25 million in bonus money was distributed among qualified teachers. But no breakdown has been provided to the public about who received those incentive bonuses or their value-added ratings. According to The Denver Post, 75 percent of teachers opted to take part in the bonus program and the average payout was $7,277 in the 2009-10 school year. The newspaper figured out that one special ed teacher made more than twice that amount (though not specifically for student growth). The paper has not published the names of teachers.

Washington, DC

Washington DC's approach to rating teachers has been controversial. The city uses a program called IMPACT, a rigorous evaluation system combining value-added methods with observations. More teachers were labeled as ineffective after the system went into effect in 2009.

Value-added data has been used to make personnel decisions in D.C. Former schools chancellor Michelle Rhee came under fire from the teachers union last summer when she fired the bottom five percent of the city's teachers, who rated lowest under IMPACT. This information was not made available to the public. The release of evaluative records is not allowed under D.C. law, but Rhee told The Washington Post that if made public, value-added scores could be "the right sort of pressure we want to see to reform the system."

However, like many educators around the country, Rhee acknowledged that value-added data can be misread and can lead to complications, such as parents demanding the highest-scoring teachers. DC's interim chancellor Kaya Henderson, who worked under Rhee in developing the IMPACT program, has said she will uphold Rhee's evaluation system and pay initiatives (which, unlike IMPACT, must be worked out with the union). Teacher's union president Nathan Saunders says he will take a more aggressive opposition to the district's evaluation system. He says it penalizes those who teach under the most difficult social conditions, such as impoverished communities with high numbers of English language learners. "The best teachers are empowered teachers," he told the Post.

With reporting by Annalies Winny

Tags:

More in:

Comments [13]

Phil from Houston

One dirty secret about Houston ISD's award program is that non-TAKS tested subjects like 6th grade, 7th grade Social Studies uses the nationally taken Stanford test to judge teachers. Well, 6th grade SS is World Cultures & 7th is Texas History. The Stanford asks no World Culture questions in its 6th or 7th grade Social Studies sections! Many 6th grade subjects have negative growths, I reckon because of the Elemetary to Middle school transition. Also, SS is blown-off at the elementary level to focus on the TAKS-test subjects (Math & Reading). The 6th grade SS section of the Stanford is mostly what they should've done in 5th grade (U.S. history) & the rest is pre-5th grade SS! Also, the local media & Houston ISD's administrators love to point to the KIPP & other chart schools as examples of how great their bonus program is. The thing is that those parents commit to community service & if their kids do bad academically or behavior-wise, thry're kicked out! You can't do that at a "regular school"! It's not fair at all & if it ever went to a class-action suit, for many different reasons, Houston ISD would get their butt kicked in court. More egg on the face of this wonderful school district, shameful. And a slap in the face of all teachers. Politicians are partly to blame with all their non-sense about schools needed to be reformed. This all started because of that, all things considered, inaccurate list of the quality of education in different countries. Well, look how great this country is! We're only 17th or so on that list because we have certain populations that do not value education much and places like Sweden & Japan don't.

Jan. 03 2011 04:23 AM
Jenn from Denver

I hear all the feedback about one test not being an effective way to measure teacher performance. What I wonder is how folks think we can improve teacher performance? There are a lot of teachers working hard to implement research based practices that do improve student achievement. There are a lot of teachers that don't implement researched based practices. We need better teacher evaluation systems.

If our job is to improve student achievement, then a measure of this must be part of our evaluation. State tests should be one part of the portfolio of assessments to measure student growth. The other part should be assessments that are common among content areas and developed to be fair and equitable. This is a hard task -- and that's where I say to the unions -- fight for teacher training to develop common assessments that measure student growth. If the political will is changing, then make sure teachers get the professional development they need.

Dec. 15 2010 11:28 PM
Maureen from Florida

The teacher grading system is too simplistic. My teacher friend has two mentally ill students in her room who are in the process of being placed elsewhere. Unfortunately they have caused havoc for months even with support from administration. She is a top notch teacher's teacher. She says she has not made her usual classroom gains. Who's to blame here?

Dec. 14 2010 12:09 PM
Steven Eugene from Houston

I can tell you how special ed classes are handled in Houston. At this time, there is no means to track the growth of Taks M students. I had a double digit growth with my students. I had 89% of my students pass their taks. For the last 2 years, I have been rated an exceeding expectations teacher, and yet, my growth for students is null. I would be rated a developing or below par teacher because of the incentive pay. The data used is faulty, and should be reevaluated.

Dec. 13 2010 03:19 PM
Will from Houston

This all boils down to one big "blame game." How can I be held accountable for a student's scores when they have been suspended for half the year or have the desire to go to summer school because they are bored at home? If I'm a kid and realize the system, that I can jack around for 9 months, fail my grade and go to summer school for 4 weeks and pass no matter what, then what is my motivation to pass a test when I know in the end it will not matter? How about we start a system where parents are ticketed for every failing grade their child brings home, hold mandatory tutorials for any student below a "C" average, increase the school day for schools that continuosly perform poorly, make parents pay for the classes their children will take in summer school. How about before you broadcast my bonus you also put the number of times I've come in hours before my actual duty starts as well as the number of times I've stayed after my actual duty is over. How about you put the number of times I've attended extracurricular activities to support the kids who have absentee parents. Put the number of times I've given money to a kid that needed to eat lunch, how many supplies I've given out, how many times I've helped a young man or young woman make a better decision than what they were going to, because they feel more comfortable talking to me than their own parent....... What number can you put on that? Where do you measure that? How big of a bonus would you give your child's teacher if you knew they had your child's best interest at heart?

Dec. 13 2010 02:22 PM
LivinginBrooklyn from Brooklyn

Lots of smart people write articles on what the status is, but no one looks at the absolutely obvious: What motivation do the STUDENTS have on these standardized tests? What is their investment in these tests? Do they get you into college? No. Are the tests fun? No. Are they fascinating and interesting? No. Are they boring and monotonous? Yes.... So what are we really measuring? How invested kids are in standardized testing, which means we are measuring, nothing.

Dec. 12 2010 10:05 AM
Robin from Indiana

Are the names of the parents also in the paper? Educating a child should be a partnership. After all, a teacher is held responsible for a student's performance on one assessment after less than 160 days. A parent has had years to help a child develop physically, emotionally, and intellectually. (good or bad)

Dec. 10 2010 08:11 AM
V. Mahoney from Avon, MA

Why does a test score from one time indicate the effectiveness of either a student or a teacher over the educational lifespan of the student? How is this accurate? If we want 21st century thinkers and creators, we MUST STOP evaluating this way. It encourages 'inside the box' thinking and cowering to limits. This is not the way to increase the effectiveness of any education. It may be part of the way, but definitely NOT the only way.

Dec. 10 2010 07:35 AM
Robin from Missouri

I believe in good teaching and being held accountable for good teaching. Let me ask those who believe in paying by rising scores and data - I teach students that are in honors. They have normal households, pretty supportive parents, but most of all a desire to succeed based on their background history. I teach grade level students too. Some of them come from normal households, but... the majority of my grade level (I have college prep between honors and grade level) come from homes where drugs are common. Most of them are on public assistance. Most of them have one parent households with a parent that didn't graduate high school. The could give a rip about education. I try to engage, enrich, and inspire, but sometimes that isn't enough. I pour my heart and soul, and own money into my class. I just bought my own projector! They don't want to be in school, they don't care about their grades or tests. How come I am the one suffering.

Why don't we go to a system where if you drop out of fail-out you aren't qualified for public assistance. Then more of the apathetic students would try.

Dec. 09 2010 08:30 PM
Anne Henly

I never hear special education being mentioned in these discussions. How are alternative programs like that handled?

Dec. 09 2010 07:04 PM
Richard V from Redwood city, CA

Always interesting to read. Wonder how the whole merit pay would ever work with specialist classes such as music, art, drama, pe, tec. One can can set benchmarks but assessments are often subjective. Also, all teachers know that every once in a while the "Class from Hell" comes through and skews the numbers. Does any evaluation scheme have enough flexibility to consider the outlyers?

Dec. 09 2010 04:07 PM
DJ from Texas

It would be inteeesting to break the data down further, i.e., student percentages of cohorts, or high mobility rates factored in. Also open charter vs public schools.

Dec. 09 2010 02:26 PM
Robert Hill

This is a neice, brief summary of current merit pay systems.

Dec. 09 2010 01:44 PM

Leave a Comment

Register for your own account so you can vote on comments, save your favorites, and more. Learn more.
Please stay on topic, be civil, and be brief.
Email addresses are never displayed, but they are required to confirm your comments. Names are displayed with all comments. We reserve the right to edit any comments posted on this site. Please read the Comment Guidelines before posting. By leaving a comment, you agree to New York Public Radio's Privacy Policy and Terms Of Use.

Sponsored

Feeds

Supported by