essa watch

Growth plus proficiency? Why states are turning to a hybrid strategy for judging schools (and why some experts say they shouldn’t)

PHOTO: Alan Petersime

A compromise in a long-running debate over how to evaluate schools is gaining traction as states rewrite their accountability systems. But experts say it could come with familiar drawbacks — especially in fairly accounting for the challenges poor students face.

Under No Child Left Behind, schools were judged by the share of students deemed proficient in math and reading. The new federal education law, ESSA, gives states new flexibility to consider students’ academic growth, too.

This is an approach that some advocates and researchers have long pushed for, saying that is a better way to judge schools that serve students who start far below proficiency.

But some states are proposing measuring academic growth through a hybrid approach that combines both growth and proficiency. (That’s in addition to using proficiency metrics where they are required.) A Chalkbeat review of ESSA plans found that a number of places plan to use a hybrid metric to help decide which of their schools are struggling the most, including Arizona, Connecticut, Delaware, Louisiana, Massachusetts, and Washington D.C.

The idea has a high-profile supporter: The Education Trust, a civil rights and education group now headed by former U.S. Education Secretary John King. But a number of researchers say the approach risks unfairly penalizing high-poverty schools and maintaining some of the widely perceived flaws of No Child Left Behind.

These questions have emerged because ESSA, the new federal education law, requires states to use academic and other measures to identify 5 percent of their schools as struggling. States have the option to include “academic progress” in their accountability systems, and many are doing so.

This is a welcome trend, says Andrew Ho of Harvard, who has written a book on the different ways to measure student progress. Systems that use proficiency percentages alone, rather than accounting for growth, “are a disaster both for measurement and for usefulness,” Ho said. “They are extremely coarse and dangerously misleading.”

Under a growth-to-proficiency model, Student A would be considered on track to proficiency by grade 6 based on the growth from grades 3 to 4, but students B and C would not. (Image: Ho’s “A Practitioner’s Guide to Growth Models”)

States that propose using this hybrid measure — commonly called “growth to proficiency” or “growth to standard” — have offered varying degrees of specificity in their plans about how they will calculate it. The basic idea is to measure whether students will meet or maintain proficiency within a set period of time, assuming they continue to grow at the same rate. Schools are credited for students deemed on track to meet the standard in the not-too-distant future, even if the students aren’t there yet.

This tends to rewards schools that serve students who are already near, at, or above the proficiency standard, meaning that schools with a large number of students in poverty will likely get lower scores on average.

It also worries researchers wary of re-creating systems that incentivize schools to focus on students near the proficiency bar, as opposed to those far below or above it. That phenomenon has been observed in some research on accountability systems focused on proficiency.

“As an accountability metric, growth-to-proficiency is a terrible idea for the same reason that achievement-level metrics are a bad idea — it is just about poverty,” said Cory Koedel, an economist at the University of Missouri who has studied school accountability. He has argued that policymakers should try to ensure ratings are not correlated with measures of poverty.

Researchers tend to say that the strongest basis for sorting out the best and worst schools (at least as measured by test scores) is to rely on sophisticated value-added calculations. Those models control for where students start, as well as demographic factors like poverty.

“If there are going to be high stakes — and I don’t suggest that there should be — then the more technically rigorous value-added models become the best way to approach teacher- and school-level accountability,” said Ho.

A large share of states are planning to use a value-added measure or similar approach as part of their accountability systems, in several cases alongside the growth-to-proficiency measure.

Some research has found that these complex statistical models can be an accurate gauge of how teachers and schools affect students’ test scores, though it remains the subject of significant academic debate.

But The Education Trust, which has long backed test-based accountability, is skeptical of these growth models, saying that they water down expectations for disadvantaged students and don’t measure whether students will eventually reach proficiency.

“Comparisons to peers won’t reveal whether that student will one day meet grade-level standards,” the group’s Midwest chapter stated in a report on Michigan’s ESSA state plan. “This risks setting lower expectations for students of color and low-income students, and does not incentivize schools to accelerate learning for historically underserved student groups.”

In an email Natasha Ushomirsky, EdTrust’s policy director, said the group supports measures like growth to proficiency over value-added models “because a) they do a better job of communicating expectations for raising student achievement, and b) they can be used to understand whether schools are accelerating learning for historically underserved students, and prompt them to do so.”

Of the value-added approach, Ushomirsky said, “A lower-scoring student is likely to be compared only to other lower-scoring students, while a higher-scoring student is compared to other higher-scoring students. This means that the same … score may represent very different amounts of progress for these two students.”

Marty West, a professor at Harvard, says the most prudent approach is to report proficiency data transparently, but to use value-added growth to identify struggling schools for accountability purposes.

“There are just too many unintended consequences from using [proficiency] or any hybrid approach as the basis of your performance evaluation system,” he said.

“The most obvious is making educators less interested in teaching in [high-poverty] schools because they know they have an uphill battle with respect to any accountability rating — and that’s the last thing we want.”

This story has been updated to include additional information from Education Trust.

pushing back

State’s most drastic school intervention plans won’t work, say Memphis board members

PHOTO: Laura Faith Kebede
Shelby County Schools board member Stephanie Love

School board members in Memphis are pushing back on the state’s plan to intervene in two low-performing schools.

In their first public discussion of an intervention plan outlined this month by the Tennessee Department of Education, members of Shelby County’s board of education said they aren’t convinced the most drastic recommendations will work for Hawkins Mill Elementary and American Way Middle schools.

The state has recommended closing Hawkins Mill because of its low enrollment and poor academic performance. American Way is on the state’s track either for takeover by Tennessee’s Achievement School District or transfer to a charter organization chosen by Shelby County Schools beginning in the fall of 2019.

But school board members said they’d rather move both schools to the Innovation Zone, a turnaround program run by the local district which has had some success since launching in 2012.

And Superintendent Dorsey Hopson said he wants to keep Hawkins Mill open because the Frayser school is in its first year under his “critical focus” plan to invest in struggling schools instead of just closing them.

“I would prefer to stay the course,” he told board members Tuesday evening. “I don’t think the board should be forced to close something by the state.”

Whether local school leaders can make that call is up for debate, though.

The intervention plan is the first rolled out under Tennessee’s new tiered school improvement model created in response to a 2015 federal education law. State officials say it’s designed for more collaboration between state and local leaders in making school improvement decisions, with the state education commissioner ultimately making the call.

But Rodney Moore, the district’s chief lawyer, said the state does not have the authority to close a school if the board votes to keep it open.

Both Hawkins Mill and American Way are on the state’s most intensive track for intervention. The state’s plan includes 19 other Memphis schools, too, with varying levels of state involvement, but only Hawkins Mill and American Way sparked discussion during the board’s work session.

Until this year, Hawkins Mill was one of the few schools in the Frayser community that hadn’t been under a major improvement plan in the last decade — unlike the state-run, charter, and iZone schools that surround it. But last year, Hopson’s “critical focus” plan set aside additional resources for Hawkins Mill and 18 other struggling schools and set a three-year deadline to turn themselves around or face possible closure.

School board members Stephanie Love, whose district includes Hawkins Mill, said that timeline needs to play out. “I am in no support of closing down Hawkins Mill Elementary,” she said. “We have what it takes to fully educate our children.”

PHOTO: Tajuana Cheshier
Protests over the state takeover of American Way Middle School in 2014, which is in Rep. Raumesh Akbari’s district in Memphis, motivated her to file legislation designed to limit the power of the state’s Achievement School District.

American Way Middle has been on the radar of local and state officials for some time. In 2014, the state explored moving it to the ASD, but that didn’t happen because the southeast Memphis school had higher-than-average growth on student test scores. American Way has not kept up that high growth, however, and Chief of Schools Sharon Griffin considered it last year for the iZone.

Board member Miska Clay Bibbs, whose district includes American Way, was opposed to both of the state’s intervention options.

“What you’re suggesting is something that’s not working,” Bibbs said of the ASD’s track record of school turnaround based on its charter-driven model.

Bibbs added that any improvement plan for American Way must be comprehensive and offered up a resolution for consideration next week to move the school into the iZone next school year.

“We can no longer be: change a principal, tack on an extra hour. It has to be a holistic approach,” she said, adding that feeder patterns of schools should be part of the process.

Turnaround 2.0

McQueen outlines state intervention plans for 21 Memphis schools

PHOTO: TN.gov
Candice McQueen has been Tennessee's education commissioner since 2015 and oversaw the restructure of its school improvement model in 2017.

Tennessee Education Commissioner Candice McQueen has identified 21 Memphis schools in need of state intervention after months of school visits and talks with top leaders in Shelby County Schools.

In its first intervention plan under the state’s new school improvement model, the Department of Education has placed American Way Middle School on track either for state takeover by the Achievement School District or conversion to a charter school by Shelby County Schools.

The state also is recommending closure of Hawkins Mill Elementary School.

And 19 other low-performing schools would stay under local control, with the state actively monitoring their progress or collaborating with the district to design improvement plans. Fourteen are already part of the Innovation Zone, the Memphis district’s highly regarded turnaround program now in its sixth year.

McQueen outlined the “intervention tracks” for all 21 Memphis schools in a Feb. 5 letter to Superintendent Dorsey Hopson that was obtained by Chalkbeat.

Almost all of the schools are expected to make this fall’s “priority list” of Tennessee’s 5 percent of lowest-performing schools. McQueen said the intervention tracks will be reassessed at that time.

McQueen’s letter offers the first look at how the state is pursuing turnaround plans under its new tiered model of school improvement, which is launching this year in response to a new federal education law.

The commissioner also sent letters outlining intervention tracks to superintendents in Nashville, Chattanooga, Knoxville, and Jackson, all of which are home to priority schools.

Under its new model, Tennessee is seeking to collaborate more with local districts to develop improvement plans, instead of just taking over struggling schools and assigning them to charter operators under the oversight of the state-run Achievement School District. However, the ASD, which now oversees 29 Memphis schools, remains an intervention of last resort.

McQueen identified the following eight schools to undergo a “rigorous school improvement planning process,” in collaboration between the state and Shelby County Schools. Any resulting interventions will be led by the local district.

  • A.B. Hill Elementary
  • A. Maceo Walker Middle
  • Douglass High
  • Georgian Hills Middle
  • Grandview Heights Middle
  • Holmes Road Elementary
  • LaRose Elementary
  • Sheffield Elementary
  • Wooddale High

These next six iZone schools must work with the state “to ensure that (their) plan for intervention is appropriate based on identified need and level of evidence.”

  • Sheffield Elementary
  • Raleigh-Egypt High
  • Lucie E. Campbell Elementary
  • Melrose High
  • Sherwood Middle
  • Westwood High

The five schools below will continue their current intervention plan within the iZone and must provide progress reports to the state:

  • Hamilton High
  • Riverview Middle
  • Geeter Middle
  • Magnolia Elementary
  • Trezevant High

The school board is expected to discuss the state’s plan during its work session next Tuesday. And if early reaction from board member Stephanie Love is any indication, the discussion will be robust.

“We have what it takes to improve our schools,” Love told Chalkbeat on Friday. “I think what they need to do is let our educators do the work and not put them in the situation where they don’t know what will happen from year to year.”

Among questions expected to be raised is whether McQueen’s recommendation to close Hawkins Mill can be carried out without school board approval, since her letter says that schools on the most rigorous intervention track “will implement a specific intervention as determined by the Commissioner.”

Another question is why the state’s plan includes three schools — Douglass High, Sherwood Middle, and Lucie E. Campbell Elementary — that improved enough last year to move off of the state’s warning list of the 10 percent of lowest-performing schools.

You can read McQueen’s letter to Hopson below: