Making the Grades

Tuesday, December 15, 2009

Todd Farley talks about his experience working for the standardized testing industry. His memoir Making the Grades: My Misadventures in the Standardized Testing Industry raises serious questions about the validity of many large-scale standardized tests, which have become a central focus in our education system.


Todd Farley

Comments [17]

Marc Naimark from Paris

Hmmm.... your guest said "for my colleagues and I", but is scoring and writing questions for standardized tests?

Jan. 12 2010 02:12 PM
Bob from Brooklyn, NY

As a teacher in the NYC Dept of Ed, I have to say that this article is a very welcome breath of fresh air.

For years we've been skeptical of the ever expanding role of the large scale testing systems in student assessment. Now, with the administration's push to tie tenure, raises, and a school's actual survival to these test scores, the stakes have been raised even higher.

I can only hope that this and similar interviews/books will reach the public ear and reverse this trend of reliance on the easily quantified, large scale test apparatus. The vast majority of teachers that I've worked with (and I've worked with many) are dedicated, hard workers. The section at the end, concerning Florida, could easily to be transferred to our current situation in NYC.

Dec. 17 2009 08:40 PM
Gabriel Reich from Richmond VA

One more thing!

Agreement among readers is subjective, but social science would be impossible without it.

In criticizing standardized tests, your guest is advocating from a position that supports the idea that purely objective measurement in social science is possible, whether it be by a test or a survey etc. This is just not true! Test developers write questions and these questions are agreed upon and approved by "experts." This is essentially the same process as the scoring. These people can make errors, they can select questions that have more than one correct answer, etc. The only difference is that the ambiguity and subjectivity happens in the creation phase rather than in the scoring phase.

Dec. 16 2009 10:35 AM
Gabriel Reich from Richmond VA

Your guest made one major error in judgment. His problem is with scoring essays because scoring is not "objective" (i.e. scientific). I would argue that multiple-choice questions only offer the illusion of objectivity. The subjective decisions have been made when creating the question and choosing among the possible answers. While this may not pose much of a problem in physics, it poses a real problem in regards to testing more loosely constructed disciplines such as history and English.

Dec. 16 2009 10:28 AM
Marybeth from West Islip, Long Island

I heard this discussion on a day where my 6th grade daughter got 20 points taken off a math quiz because she didn't show the work (as required by the NYS math test) reducing fractions (ie. 4/8ths is equal to 1/2). Her teacher wanted her to show 4/8 DIVIDED BY 2/2 to equal 1/2. The teacher's rationale was that my daughter would lose credit had this been the standardized test. She should have gotten 100 on the quiz - all of her work was correct! My daughter was in tears and learned that she had to "dumb down" her answers to fit the standardized requirement. Great lesson!
I'm going to buy Mr. Farley's book and tell everyone I know about it!

Dec. 15 2009 07:44 PM

Lots of weaknesses in testing, true. But it's quite possible to have a team of 25-30 readers norm essay replies, and score with a high degree of agreement. If you have a head reader who's constantly checking everyone's scoring, at the end of the day you will have results that mirror the morning's work. If you farm this stuff out all over the country, you face an almost impossible task of maintaining integrity. But bringing people together to score, and spending an hour/hour and a half on norming costs. The public isn't willing to bear such. Yet... they want accurate results.Blecch.

Dec. 15 2009 06:39 PM
Kippens from Canada

As a university English instructor, I can tell you first-hand that learning is a dynamic activity and can never be reduced to testing data.

This type of test-taking also encourages uniform rather than engaged and rebellious thinking (the best and really only kind). Hence, the subject goes deeper than even Farley realizes. It perhaps goes some way in explaining why we have such a powerless and passive citizenry in the face of what amounts to a crisis in our democracy.

More than this, standardized testing reduces the 'worth' of children to meaningless scores. Responsible educators, parents, and children should band together and demand a richer education. That act of engagement in and of itself would require more learning than all the time wasted worrying on these superficial tests.

Dec. 15 2009 03:37 PM
Edgeoforever from Manhattan

Two words summed up what's wrong with this: "testing industry". Making a business out education is as wrong as making it out of healthcare.
Thank you for shining the light on this.
I know there is a better way and some schools and organizations in NYC are bravely fighting to maintain it.

Dec. 15 2009 02:41 PM
bk from nyc

Thanks to Mr. Farley for having the courage to write this book. After hearing the interview, I will be odering a copy. As a parent, educator, and concerned citizen, I would love this to be required reading not just for the NYCDOE, but for every educator and teaching college in America.

When we don't adequately assess our children, we ultimately fail them.

There's got to be a better way.

Thanks for the great segment.

Dec. 15 2009 02:12 PM

What's hideous is that the so-called "learning" that these tests are supposed to measure is going to make us fall flat on our faces on the world scene - where are all our inventors and problem solvers? In India and China, for starters...

Dec. 15 2009 01:36 PM
KK from brooklyn

This segment is brilliant. I wish Arne Duncan and Obama--not to mention the Bloomberg/Klein's NYCDOE Department of Accountability--were required to read the book/listen to this interview.

--NYC Public school parent extremely concerned about the overreliance on high-stakes testing (and, way back, before I realized its evil, employee of test prep company)

Dec. 15 2009 12:47 PM
ted from manhattan

i'd love to more technology used to create webfolios to show the accomplishments of our scholars.

the connection between the publishers and he politicians has made this what we have now. think mcgraw and bush friendship.

think portfolio/webfolio


Dec. 15 2009 12:35 PM
EAA from New York

I just tuned in to the show during my study break for a multiple choice licensing exam I have tomorrow. So much for not thinking about testing!

I'm happy its multiple choice only. I'm also happy that I"m not one of the kids having their essay graded by a time pressured, apathetic grader!

I agree these are money making ventures. I paid for a prep course/materials and the exam is $500.00 to sit for it.

Dec. 15 2009 12:31 PM
Jgarbuz from Queens, NY

I was a good test taker in my youth, but that didn't translate into having much talent or becoming a big success. Standardized tests do have an important role in checking the literacy and arithmetic proficiency of students, but not much more than that. Part of schooling should be trying to uncover the hidden potential of children and exposing them to activities that bring them to the fore. Reading, writing and doing basic maths are essential skills that every citizen should have to acquire, but not at the cost of being oblivious or suppressing other skills and talents that are not so obvious to uncover. How we better do that should be the focus of 21st century education.

Dec. 15 2009 12:31 PM

The essay part of these tests are useless. As anybody who has ever taken one of these, unless you follow the "blueprint" expected and taught by all the test prep companies, you will be scored really low. Not to mention, such "blueprint" results in a very amateur essay which any decent college professor would reject.

All together, these so called standarized tests are nothing more than a nice racket run by the testing company which btw, enjoys every advantage that monopolies provide.
Hopefully more and more educational institutions will follow the example of some universities that have finally figured out this is a useless expense (a very expensive once by the way) and not necessary. These tests do not tell you anything new other than what a close look at a student's transcript would not, other than how much money was spent on the other "part" of the racket: test prep classes.

Dec. 15 2009 12:24 PM
Hugh Sansom from Brooklyn NY

Twenty years ago, I knew a woman who was renowned for her test-taking abilities. She aced them all.

She took the LSAT in 1991... got one question wrong. She contacted the testing board to explain why THEY were wrong and she was right. They conceded and instantly offered her a job.

For all her test taking skills, she flunked out of a PhD program at the world's most famous engineering instition (in Cambridge, MA). People who did significantly worse did better in the program. And she did not get into law school.

Dec. 15 2009 12:23 PM
Neil from NYC

If standardized testing for Americans is like this, then imagine what the foreigners have to suffer through.

My nephew has been trying pass the Test of English as a Foreign Language (TOEFL) written part for over 2 years. He speak reasonably good English and has failed the test over a dozen times. Each time he has paid $200 for the test.

The company administering the test (Educational Testing Service) is not accountable to anyone and it has found a big cash cow in newcomers to America who need to pass the TOEFL and are afraid to speak up about the lack of transparency in the grading.

Dec. 15 2009 12:21 PM

Leave a Comment

Email addresses are required but never displayed.

Get the WNYC Morning Brief in your inbox.
We'll send you our top 5 stories every day, plus breaking news and weather.