When Test Prep Requires Test Prep

Email a Friend

It’s that time of year again -- time for students, and their teachers, to become anxious about the standardized tests they are about to take.

I have been a parent in the public school system in New York for 10 years now, and as April rolls around, no matter how calm teachers try to be, tension rises. And now that teacher grades are being made public -- well, no one wants their students to do poorly.

So it was a particularly worried fourth-grader who read me his test preparation question last week, about (really!) seeing-eye horses.

“You probably know about seeing-eye or guide dogs,” the passage read. “These dogs are trained to help people with vision problems. You may not have heard about seeing-eye horses, though. They’re not the same size as a usual horse. In fact they’re only about as big as a medium-size dog. They can be house-trained, like a dog! They don’t bark to be let out, but they do go to the door and neigh. They don’t get fleas like dogs. And unlike dogs, they shed only twice a year.”

And a lot more in this vein. (You can read the whole thing, and test yourself, in the query attached to this post.)

When he got to the question about the passage, he was stumped.

“Q. Why might a person with vision problems choose a guide horse rather than a guide dog? Why might someone prefer a guide dog? Use details from the passage to support your answer.”

While the passage provides reasons someone might choose a guide horse, it doesn’t say anything about why someone would prefer a seeing eye dog. Or at least the fourth grader and I could not figure it out from what we read.

So I brought it to my office and showed it to my colleagues -- journalists with many years of schooling, all familiar with texts and words, and professionally trained to look only for what is actually said.

There was something in it about how horses wear “tiny sneakers to protect the floors” in the house. (I am not making this up!) Maybe a visually-impaired person wouldn’t want to have to deal with putting sneakers on a horse? That was the only thing I could come up with.

But it seemed a crazy answer, and not soothing to my fourth-grade son.

I called the Department of Education press office. Turns out they don’t choose the test prep books, they only give schools a list of approved vendors. So I contacted my son’s principal. This is what she told me, in an e-mail:

“We actually have to spend precious money from our budget on them! Especially because of how high stakes this has become for teachers, they wanted us to purchase some materials. They looked through some sample text prep books from different companies and selected this one. I wanted to try to make their test prep as simple as possible by not requiring them to do Xeroxing of materials, so we purchased them. They didn’t read all the passages of course. Several teachers have come to me over the past couple of weeks talking about the errors in these materials, and I doubt we’ll buy them again.”

But that still didn't end it for me. I called the publisher, Curriculum Associates, which publishes the “New York Ready” series of test prep books.

Renee Foster, the senior vice president for product development and marketing, and Adam Berkin, the vice president of product development, called me back.

Ms. Foster jumped right in. “I agree,” she told me. “The answer to the second piece is there is no answer.”

“This is a guided instruction piece, not a practice test,” she explained. “The idea of our booklet is to facilitate a classroom discussion.”

Mr. Berkin (who pointed out that he used to be a fourth-grade teacher) added: “Our training materials are designed to get kids to be thinking about what evidence is backing up their answer.”

O.K. then, did they craft the question this way on purpose? Did they want to get kids to say there’s no evidence as part of a teaching technique? Is it a trick question?

They wouldn’t use the word “trick.”

“It’s to facilitate a conversation about textual evidence,” they said.

There were many rounds of this. And then, finally, from Mr. Berkin: “If I was writing the question, I wouldn’t have included that part” (about the dog).

I told this to my son. Much relief. Fourth-graders are very literal-minded. If there’s a question, there must be an answer, which does not include “there is no answer.”

Hopefully, there won’t be anything as crazy on the actual test. But if there is, he now knows how to answer it.