Lawmakers who tried to shed light on the murky reasons for Tennessee’s botched transition to online testing generated more questions than answers on Wednesday.
Members of the Senate education and government operations committees pressed State Department of Education officials on what went wrong with the state’s brand new testing system, and why they had been confident in a testing company that had never before undertaken a project of that scale.
Education Commissioner Candice McQueen said her department had done everything it could to prepare for testing, including administering millions of student practice tests and taking “painstaking efforts” to fix bugs reported by students and teachers.
But when testing began on Feb. 8, the testing system crashed while thousands of students were mid-test. Education officials quickly blamed Measurement Inc, the North Carolina vendor hired to develop the test, but have not explained what caused the system to fail.
“Who had oversight over this?” asked Sen. Mike Bell. “Were we just trusting the vendor to get it done?”
(While the hearing was underway, Gov. Bill Haslam announced in an email that test scores would not factor into teachers’ ratings this year because of the testing snafus.)
Department officials said they worked closely with Measurement Inc. throughout the development process, alerting its representatives of technological problems they saw and helping the company test how many students could log on to its system without crashing.
But the problem that crippled servers on Day One of the testing was not due to overload or any bug they’d seen before, said Cliff Lloyd, chief information officer for the department.
“The system was not under tremendous load when it fell over,” Lloyd said. “Unfortunately, what we saw wasn’t a consequence of load. The system was running at about 15 percent of its expected load. What we saw were a combination of factors that could have been avoided, and that is probably the saddest aspect of this.
“We didn’t see a server blow up; we didn’t see insufficient money spent on server capacity; we didn’t see problems with broadband in our schools; we didn’t see all the things you would’ve thought would have been the problem,” Lloyd continued. “What we saw were some bad choices … that really could have been avoided.”
“What we saw were bad choices ... that really could have been avoided.”
Cliff Lloyd, chief information officer
Lloyd did not specific what those choices were. But he signaled that he did not believe anyone at the state department had made them. “That talks really to the internal processes of another organization that we really don’t have control over,” he said.
Measurement Inc. won the contract with the state in 2014 after receiving the highest technical score from an anonymous group of evaluators, and offering to create the test for the lowest cost — about $20 million less than the next lowest offer.
The company had never taken on a statewide online assessment contract before. But Stephen Smith, deputy commissioner of policy, said the State Department of Education had no reason to doubt that the company was up for the job.
“We went through a very extensive (selection) process … and it was really an unprecedented process,” Smith said. “We had leaders from local districts on the content side and the technical side. … At the time, we felt very positive about the selection.”
McQueen, who became commissioner months after the contract was signed, suggested during the hearing that the department’s confidence had been quickly shaken — and then restored.
“There was a bit of a concern there early on about the capacity for them to understand what the issues were and for them to fix them quickly,” she told senators.
So the state worked with Measurement Inc. to do several “stress tests,” ensuring that the company’s servers would not be overwhelmed by the high volume of traffic when students began testing. They also worked to fix bugs that caused the program to freeze during practice tests.
“We were as ready as we could be for Monday, Feb. 8,” McQueen said.
That is little consolation for students and school personnel who were flummoxed when exams started last week. Now, schools that prepared to take the test in early February are waiting for the arrival of paper tests that have yet to be printed — which are actually more expensive for the state to administer. Smith said officials are working to ensure that the state won’t be left with a larger-than-anticipated bill.
In the meantime, students and teachers alike are confused, said Sen. Lee Harris of Memphis. He said his fifth-grade son had brought up concerns about TNReady and the state’s response at the dinner table.
“This is the first time my son has taken an interest with what’s going on the state level because he’s prepared, and his classmates have prepared, with a great deal of effort, for this test. So he wonders what’s going on, and he brings that to the dinner table,” Harris said.
“Assure your son, he will take TNReady,” McQueen said. “He will get the paper version of the test, and he will be taking it.”
“Can you tell him when?” Harris countered.
Assistant Commissioner Nakia Towns said officials hope that Shelby County students will take the test in mid-March before spring break — a month after they originally planned.