Published March 24 2012
Fargo schools’ test scores ‘suspicious’Hundreds of school systems nationwide, including Fargo and Bismarck public schools, exhibit suspicious standardized test scores that point to the possibility of cheating, according to a major investigation by The Atlanta Journal-Constitution.
The newspaper examined test results for 70,000 public schools and found high concentrations of abnormal scores in school systems from coast to coast.
The analysis doesn’t prove cheating. It reveals that scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.
The AJC reported in 2008 and 2009 about statistically improbable jumps in test scores within the 109-school Atlanta Public Schools system. Those reports led to an investigation by Georgia officials, which found that at least 180 principals, teachers and other staff took part in widespread test-tampering in the 50,000-student district.
Published first online Saturday, and in print today, the AJC reports that 196 of the nation’s 3,125 largest school districts – including Fargo and Bismarck – had enough suspect test results that the odds of the results occurring naturally were less than one in 1,000.
Fargo and Bismarck were the only North Dakota districts with more than 10 percent of their classes flagged in a given year from 2008 to 2011.
Being flagged represents a test-score shift outside the norm. In the analysis, classes with scores rising or dropping with a probability of less than 0.05 were flagged as unusual.
A “class” was defined as a group of students in the same school from one year to the next – for example, fourth-graders in 2009 and fifth-graders in 2010.
Fargo had 10 percent of its 60 classes flagged in 2010. In the other three years, the analysis flagged 8.33 percent of classes in 2008, 7.41 percent in 2009 and 5 percent last year.
Bismarck had 11.11 percent of classes flagged in 2009, 5.68 percent in 2008, 5.26 percent in 2010 and 2.3 percent last year.
Dakota Draper, who taught in Fargo Public Schools for 14 years before being elected in 2007 to his current position as president of the North Dakota Education Association, said he’s unaware of any cheating on standardized tests or allegations of such activity in North Dakota.
“As I’ve gone through the (AJC) article and read it, there’s really not a shred of evidence that Bismarck or Fargo have been engaging in cheating,” he said. “It’s kind of just being suggested because of statistics. But we’re certainly not aware of any.”
“We wouldn’t condone that in any way, shape or form,” he added.
Fargo Public Schools Superintendent Rick Buresh did not return a message left on his cellphone Saturday afternoon.
Among other North Dakota cities examined, West Fargo had 8.33 percent of classes flagged in 2008, none in 2009 and 6.25 percent in each of the last two years.
In Grand Forks, the analysis flagged 1.85 percent of classes in 2008, 4.08 percent in 2009, 8 percent in 2010 and 3.7 percent in 2011.
The analysis found no concentrations of suspicious test score swings in Minnesota that met the reasonably high threshold for being flagged, according to AJC’s managing editor, Bert Roughton Jr.
For 33 districts nationwide, the odds of their test scores occurring naturally were worse than one in a million.
Standardized test scores have been at the forefront of national and local efforts to improve schools. Test performance was the centerpiece of the federal No Child Left Behind Act of 2001, which demanded higher classroom accountability. Tougher teacher evaluations that many states are rolling out place more weight than ever on the tests.
But the AJC report found that the sweeping policy shifts rely on test results that may be unreliable.
While the federal government requires states to use standardized testing, it does not require educators to screen scores for anomalies or investigate those that turn up.
“If we are going to make important decisions based on test results – and we ought to be doing that – we have to make important decisions about how we are going to ensure their trustworthiness,” said Daria Hall, director of K-12 policy with the nonprofit Education Trust.
“That means districts and states taking ownership of the test security issue in a way that they haven’t to date.”
In nine districts – Atlanta, Baltimore, Dallas, Detroit, East St. Louis, Ill., Gary, Ind., Houston, Los Angeles and Mobile County, Ala. – scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were virtually zero, the newspaper found.
In Houston, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis showed. When children moved to a new grade the next year, their scores plummeted – a finding that suggests the gains were not due to learning.
“These findings are concerning,” U.S. Secretary of Education Arne Duncan said in a statement after being briefed on the AJC’s analysis. He added that “states, districts, schools and testing companies should have sensible safeguards in place to ensure tests accurately reflect student learning.”
Many school district officials contacted by the AJC disputed any conclusion that cheating was to blame for the swings.
Some school leaders attributed steep gains to exemplary teaching. But experts said instruction isn’t likely to move scores to the degree seen in the AJC’s analysis.
Cheating is one of only a few plausible explanations for such dramatic changes in scores for so many students within a district, said James Wollack, a University of Wisconsin-Madison expert in testing and cheating who reviewed the newspaper’s analysis.
“I can say with some confidence,” he said, “cheating is something you should be looking at.”
In each state, the newspaper used statistics to identify unusual score jumps and drops on state math and reading tests by grade and school. Declines can signal cheating the previous year. The calculations also took into account other factors that can lead to big score shifts, such as small classes and dramatic changes in class size.
The newspaper also developed a statistical method to identify school systems with far more unusual tests than expected, which could signal endemic cheating similar to what occurred in Atlanta. In its approach, the score analysis used conservative measures that highlighted extremes. The methodology is more likely to overlook possible indications of cheating than to suggest problems where none exist.
The newspaper’s methodology was reviewed by outside experts.
The AJC’s analysis suggests that tens of thousands of children may have been harmed by inflated scores that could have kept them from getting the academic help they needed.
In 2010 alone, the grade-wide reading scores of 24,618 children nationwide – enough to populate a mid-sized school district – swung so improbably that the odds of it happening by chance were less than 1 in 10,000.
Experts said the findings warrant deeper investigation at the local level.
Statistical checks for highly improbable scores are like medical tests, said Gary Phillips, a vice president and chief scientist for the large nonprofit the American Institutes for Research, who advised the AJC on its methodology.
“This is a broad screening,” he said. “If you find something, you’re supposed to go to the doctor and follow up with a more detailed diagnostic process.”
Forum reporter Mike Nowatzki contributed to this article. He can be reached at (701) 241-5528.