|
|
|
|
 n
public schools across the country, there’s a new skill being
taught as early as the primary grades: how to use a number-two pencil
to fill in the “bubble form,” an answer sheet for multiple-choice
tests that can be instantly scanned and scored by computer. In North
Carolina, some second-graders have been practicing this task and other
test-taking strategies in anticipation of the upcoming school year,
when all third-graders in the state will take a single, multiple-choice,
end-of-grade (EOG) test that will likely determine whether they are
promoted to the fourth grade or held back a year.
At a time when some top colleges
are questioning how much weight to give standardized-test scores in
admissions decisions, standardized testing for K-12 students is increasing
in nationwide influence. A study conducted by Jay P. Heubert M.A.T.
’74, associate professor of education and adjunct professor of
law at Columbia University, finds that a rapidly growing number of
states are engaged in “high stakes” testing, requiring students
to pass standardized tests as a condition of grade-to-grade promotion.
In some states, including North Carolina, test scores are also being
used to determine teacher bonuses and school rankings. According to
The Chronicle of Higher Education, at least twenty-seven states now
use standardized tests in promotion and funding decisions, and President
Bush has recently proposed to withhold federal funds from schools
that repeatedly fail to meet minimum standards.
Standardized testing as a diagnostic measure of overall
student knowledge and skill is nothing new, but “high stakes”
assessment is a relatively recent and highly controversial phenomenon.
While national surveys have repeatedly found that a majority of Americans
favor raising academic standards and holding schools accountable for
student achievement, a serious backlash against high-stakes testing,
particularly in the early grades, has begun to emerge.
In Florida, one group of teachers sent
back the end-of-year bonuses they received as a result of their students’
higher test scores, saying that one-time, standardized paper-and-pencil
tests are an affront to their daily, professional assessment of individual
student achievement. In Michigan, Ohio, and Massachusetts, parents
have organized boycotts of state-mandated exams. In Milwaukee, parents,
teachers, and others stonewalled a plan to expand standardized testing
to kindergarten, first, and second grades. In Durham this May, representatives
of at least four different parent groups from across the state spoke
out at a hearing designed to alert legislators to their concerns about
high-stakes testing. The groups favor fair testing, but reject the
use of standardized assessments as the primary indicators of a child’s
curriculum mastery.
Across the nation, critics
have argued that testing reduces teacher and student creativity, focuses
too much on basic skills rather than higher-order thinking, and confines
teachers to “teach to the test” rather than to a more broadly-based
curriculum.
The roots of the current standards movement go back to
1981, when a study commission convened by the Reagan administration
issued A Nation at Risk. The report sounded the alarm that “a
rising tide of mediocrity” in the nation’s public education
system would eventually lead to erosion of the nation’s workforce
competitiveness.
For its part, North Carolina
had been measuring school performance by district as early as 1978,
but in the early 1990s the state legislature mandated a more rigorous
form of statewide testing designed to measure individual school performance.
In response, the North Carolina Department of Public Instruction introduced
“The New ABC’s of Public Education” in May 1995. The
plan requires that all North Carolina public school students take
end-of-grade tests that are tied directly to North Carolina’s
curriculum or “Standard Course of Study.” (In some states,
more general achievement tests not directly linked to curricular objectives
and content are administered.)
After North Carolina and Texas
were designated as the two pacesetter states in the advancement of
school accountability at the 1999 National Education Summit, both
major party candidates referred repeatedly to North Carolina’s
accomplishments during the 2000 presidential debates. George W. Bush
in particular hailed high-stakes testing as the most effective means
to end social, or unearned, promotion and restore confidence in the
nation’s public education system.
TEACHERS ON TESTING
What’s the stance of America’s
largest teachers’ union, the National Education Association,
toward high-stakes testing? The NEA acknowledges
“a fine (and ill-defined) line about what is appropriate
and inappropriate” in test preparation. Officially, it
urges teachers to:
• Teach test-taking skills well in advance of the test
date.
• Discuss with students the purpose and relevance of the
tests and how they will be used.
• Expect good results and model positive attitudes.
• Tell parents/guardians about the tests’ purposes,
how results will be used, and how they can help prepare students
for the tests.
• Acquaint students with the style and format of the tests.
• Teach students to follow test directions carefully.
• Help students learn how to budget the time they have
to complete the tests.
• Teach students how to read test items, reading passages,
and answer choices.
• Teach students to answer easier questions first and persist
to the end of the tests.
• Teach students the importance of making educated guesses.
• Help students develop strategies for checking their test
answers.
• Provide students with opportunities to practice test-taking
skills.
The NEA says it officially supports “ongoing, comprehensive”
assessments of student growth that are “directly linked
to lessons and materials used by teachers.” Beyond that
general statement of philosophy, it says that:
• Students should have adequate resources to learn that
which tests measure.
• Sufficient reliability must be established for each intended
use of a test.
• Tests must be validated for intended uses.
• Explicit rules must exist for determining which students
are to be tested.
• Cut-off or passing scores must be valid for each achievement
level.
• Appropriate provisions must be made for students with
disabilities.
• Appropriate provisions must be made for language differences
among students.
• High-stakes tests must be aligned with
the curriculum.
• High-stakes decisions should not be based on single test
results.
• There must be full disclosure of likely
negative consequences of high-stakes testing programs.
• Extra support for students who fail high-stakes tests
must be available.
• There must be ongoing evaluation of intended and unintended
effects of high-stakes testing. |
Until this year, North Carolina’s
ABC’s test scores have been used only to measure overall school
performance in reading comprehension and math skills among students
in third through eighth grades. Writing test scores administered in
the fourth, seventh, and tenth grades are also entered into the complex
formula that has been used to measure and reward individual school
performance and to determine teacher bonuses. At the high-school level,
End of Course (EOC) tests are administered in Algebra I; Algebra II;
Biology; Chemistry; Economic, Legal, and Political Systems; English
I; English II; Geometry; U.S. History; Physical Science; and Physics.
Today, certified teachers in North Carolina can receive
an end-of-year bonus of $1,500 if their school meets the “exemplary”
or highest-growth standard in their scores (set by the state at 10
percent above the statewide average growth). Teachers in schools that
meet a predetermined “expected growth standard” earn a $750
bonus. Below these two designations are “adequate performing”
schools where at least 50 percent of students are performing at grade
level, and “low performing” schools that do not meet their
goals or have less than 50 percent of students performing at grade
level. No teacher rewards are associated with the latter two designations,
though the fifteen lowest-performing schools statewide are targeted
to receive special assistance from advisory teams deployed by the
state.
Does this business incentive model work for teachers?
Brett Jones, who has taught at Duke for the last two years in the
education department, is now working on a book that considers the
unintended consequences of high-stakes testing, based on his study
of twenty-three teachers in Durham County. “There is some evidence
that teachers are expecting more of their students and are working
harder,” he says. But he suspects that down the road, we may
see teachers opting to work in higher-performing schools where the
bonuses may be less challenging to come by. The bonus system also
may make it more difficult to recruit both new and veteran teachers
to work in low-performing schools, a subject yet to be investigated
fully, says Jones.
North Carolina’s school officials, bolstered by their
success with overall school performance measurements, have just raised
the stakes another notch. They are now phasing in the use of EOG and
EOC scores in individual student promotion decisions.
Beginning with the school year just ended, individual
students’ EOG test scores in reading, mathematics, and writing
are the “gateway” to student promotion for all fifth-graders
in North Carolina. Next year, the third- and eighth-grade gateways
will take effect.
At the high-school level, some EOC exams are also now
being weighted as a full 25 percent of a student’s year-long
grade for the first time. So a student could go into the state exam
with a low “C “ average, perform very poorly on the EOC,
and actually flunk the course, earning no credit for a year’s
work in that subject area. (In prior years, high-school EOC tests
were only marginally weighted in a student’s final grade.) Effective
with the graduating class of 2004-05, students will also have to pass
an exit exam of essential skills to earn a high-school diploma.
Public-instruction officials have repeatedly emphasized
that students in grades three, five, and eight who do not pass the
EOG the first time may retake the test several days later to determine
if the initial score was a fluke or if the student simply had a bad
test day. Students who do not pass in the second round, however, must
participate in remedial work over the summer and are then retested
before the next school year begins. If they do not pass the test on
the third attempt, they are held back.
Parents or teachers who believe a student has been unfairly
judged may then request a portfolio review of the student’s overall
work during the year just ended. Under state policy, this evaluation
must be conducted by an independent committee of teachers and a principal
from another school, who then recommend pass or fail. Ultimately,
the final decision about promotion rests with the principal of the
school that the child attends.
David Malone Ph.D.’84, assistant professor of the
practice in Duke’s education department, teaches a number of
undergraduates who are working toward their Master of Arts in Teaching
(M.A.T.). As part of their training, M.A.T. candidates enter local
classrooms to practice teaching. In addition, Malone oversees a grant-funded
project involving some 100 Duke undergraduates—many of whom are
also students in his educational psychology class. These students
tutor fourth- and fifth-grade students from Durham twice a week for
an hour helping to prepare them for the EOG exams.
 |
“Now that the gateway ABC’s
are finally here,” Malone says, “people in North Carolina
are asking a lot more questions about each individual school’s
resources, the inequities we have in early childhood opportunities
for individual students, and what happens when a class loses a teacher
during the year and has a disproportionate number of days with substitute
teachers who are not familiar with test preparation. With so many
of my undergraduates witnessing the system firsthand, it seems like
testing is all we ever talk about around here.”
Malone had his students take a North Carolina benchmark
or practice exam designed to test fifth-grade reading comprehension.
“Out of my forty undergraduates in Education 118,” he says,
“only half got all the questions right. Their critical-thinking
skills were too sophisticated for some of the questions. Taking this
test quickly demonstrated to them that it is very difficult, even
for a team of experts, to construct a fair assessment of a kid’s
reading ability with a multiple-choice test.” Moreover, says
Malone, new test questions and multiple versions of the tests must
be constantly developed and field-tested. To maintain confidentiality
of content and to avoid cheating, parents and teachers alike are not
permitted to view the EOG and EOC multiple-choice tests, either before
or after their administration.
“Not only was the reading comprehension test we took
culturally biased, in my opinion, and highly ambiguous in places,”
says Malone student Dennis Davis, a Duke senior from Greensboro, North
Carolina, “but it was clear to me that on one particular question,
I could imagine how the fifth-grade kid I tutored would completely
project his own dysfunctional family experience into the answer. There’s
no way he would give the ‘correct’ answer. Heck, I didn’t
get it right, either!” The question to which Davis refers asked
students to judge what might be an absent father’s response to
a family disaster that took place in Colonial America.
“Non-instructional factors account for a significant
amount of the variance among EOG test scores when schools or districts
are compared,” says Steven Pfeiffer, adjunct professor of psychology
and education and executive director of Duke’s Talent Identification
Program for gifted middle- and high-school students. “Factors
such as parents’ educational background, type of community, and
poverty level account for more than 50 percent of the difference in
test scores.” According to Pfeiffer, “an appropriate use
for this kind of testing” is “for screening purposes to
determine if schools or groups of students may need additional support
and in what specific areas.”
But what happens when tests become a measure of individual
student performance for promotion decisions, as they have now in North
Carolina?
Early in the school year, the North Carolina School Psychology
Association issued a position statement arguing strongly against the
use of EOG tests in individual promotion decisions because, they revealed,
the test itself has not been statistically validated as an accurate
measure of individual student performance. The test has only been
validated as a screening tool for students in the aggregate, along
the lines that Pfeiffer says are useful.
Using tests not validated for the measurement of an individual
student’s performance is a common problem nationwide, Pfeiffer
explains. “The tests simply lack a level of precision—an
acceptable level of scientific or technical rigor—necessary for
making decisions on individuals. Norm-referenced EOG tests were not
developed and were never intended to measure the quality of learning
or instruction. And decisions that affect
a student’s life or educational opportunities should never be
made on the basis of a single test score, no matter how reliable or
valid. To ensure fairness, students should have multiple opportunities
to display their skill or competence, particularly on decisions that
carry serious consequences such as promotion and graduation.”
That North Carolina allows students to take the EOG test multiple
times is not the same as multiple measures of performance, Pfeiffer
says.
|
“Decisions
that affect a student’s life or educational opportunities
should never be made on the basis of a single test score, no
matter how reliable or valid.”
—STEPHEN PFEIFFER
Executive Director, Duke’s Talent Identification Program
and adjunct professor of psychology and education |
|
Marvin Pittman is the assistant superintendent
of North Carolina Schools with the special charge of helping North
Carolina close the much-discussed “achievement gap” between
poor and minority students and their more affluent, generally white,
counterparts. “My goal is not to make anyone love this [testing]
policy,” Pittman says. “We know that a single measure can’t
do it, and we’re looking at other ways to measure performance.
The State Board of Education understands that you must look at other
areas. Portfolio assessment [evaluating a range of examples of students’
work] may be the way to go, but we’re not there yet. As we have
looked at the research, we really don’t see anyone doing this
very well. In the interim, we are using the EOG. The overriding part
of this policy is identifying where we need to do extensive interventions
to help create better-performing students and schools.”
Joe Johnson ’70, M.A.T. ’71, Ed.D. ’78
is superintendent of the Wilkes County schools, in the foothills of
North Carolina. “Right now, we can honestly say that our students
are reading, doing math, and writing better than they ever have,”
he says. “Because of the tests, the dialogue between teachers,
students, and home has increased. We are now obliged to give parents
more information. One downside of the process, however, is that administering
the test falls to the guidance counselors, which means they have less
time to attend to individual needs of students, their academic concerns,
and any problems that might be happening at home. At a time when you
are increasing the awareness of individual children and the importance
of learning, you are, ironically, removing one of the people who should
be most involved. Testing takes enormous effort and resources, and
we need more state resources for administrative help with the tests.”
In North Carolina, early in the school year the State
Department of Public Instruction did ask schools to identify students
they expected to have trouble passing the EOG. They obtained an additional
$31 million from the state legislature to reduce class sizes and hire
more teachers in low-performing schools, and to provide tutoring and
special Saturday classes to assist at-risk students. For the upcoming
school year, says Pittman, they have requested an additional $39 million
for the same purpose.
Still, the North Carolina School Psychology Association
argues that while “retention with extensive remediation has been
effective with certain groups of children, promotion with similar
remediation is more effective and has fewer negative effects.”
“The single strongest predictor of whether students
will drop out of school is whether they have been retained in grade,”
writes Columbia’s Jay Heubert, citing several recent studies.
“Those retained in grade even once are much likelier to drop
out later than are students not retained, and the effects are even
greater for students retained more than once. Moreover, much of the
increase in dropout rates show up only years later, and the harm is
thus largely invisible at the time the retention occurs.”
Joseph DiBona, an associate professor of education at
Duke, puts it this way: “Those kids who fail are going to need
lots more help to pass the tests on the third try or after repeating
a grade, which will be very costly. We’re in a terrible box.
We have let people pass through the system for years, but after the
testing is completely phased in at the lower grades in North Carolina,
we may see these same kids, when they get to high school, simply drop
out in frustration.”
Fred Jones ’81 is the assistant principal at Jordan
High School in Durham. “This is a very costly policy in terms
of having to serve any given student for an extra year or more,”
he says. “And at the secondary level, we are nervous that we
may begin to see students who have been retained two or three times,
and may finally be entering high school at age sixteen or seventeen.
State law prohibits our serving any student over twenty-one, so we
may have students coming into ninth grade who are simply not eligible
to graduate unless the law is changed.”
According to the North Carolina School Psychology Association,
the cost of retaining 60,000 students in grades K-12 each year in
North Carolina has previously been in the neighborhood of $360 million.
Increasing the education budget much further may be prohibitive. North
Carolina is already strapped for funds—in part from the widespread
devastation in eastern North Carolina created in 1999 by Hurricane
Floyd, the continuing erosion of the state’s tax base due to
the decline in tobacco sales, and the wholesale movement offshore
of much of the state’s textile and furniture manufacturing. How
costly it will be to make good on remediation promises and support
the expansion of testing to grades three and eight next year is unclear.
At the same time, if the tests do not actually increase intervention,
and in some cases, retention, then how can the new system be considered
more rigorous?
• continues on page two
|
|
