Place matters: Changing schools can change student test scores, research shows | St. Louis Public Radio

Place matters: Changing schools can change student test scores, research shows

Aug 22, 2013

This article first appeared in the St. Louis Beacon: Now that hundreds of students have started their long bus rides from Normandy and Riverview Gardens to accredited districts, can they expect to have greater academic success in their new schools?

Nothing is certain, of course, but educational research – and the long experience the St. Louis area has with the voluntary desegregation transfer plan – suggest that where students attend class can have a definite positive effect on how much they learn.

“There is a boatload of literature that shows where you go to school matters,” says William Tate, chair of the department of education at Washington University. “There is no credible study that doesn’t come up with that conclusion. Place matters a lot.”

The difference, education researchers say, is more than a matter of just lesson plans and textbooks.

Students who leave unaccredited school districts for classrooms with a higher level of success are likely to encounter a different atmosphere and different expectations as well. Such improved surroundings can occur anywhere, but they are more likely to be present in areas where poor economic conditions don’t tend to overwhelm other concerns.

“There is nothing wrong with high-poverty schools in and of themselves,” says Susan Eaton of Harvard’s Charles Hamilton Houston Institute for Race and Justice, who has followed the transfer situation in St. Louis. “You see isolated examples of high-poverty schools that beat the odds they face. But by and large, students who go to middle-class schools perform better on average in math, English language arts and graduation rates.

“That is not to say that in this particular case that is going to happen. I think it’s important that when students enter a school, they feel a sense of being welcome, a sense of belonging in the school and that they have teachers who are equipped to handle them.”

If those conditions are present, says Amy Stuart Wells of Columbia University, who has written about the St. Louis school desegregation plan that started in the 1980s, academic improvement is more likely to occur.

“One of the things we did learn through the interdistrict plan is that there are a lot of things suburban districts can do to help the families and get better outcomes for students,” she said. “They have to pay attention to that and understand the fact that students don’t live in this community and this may be a real cultural shift for them.

“You don’t just dump the kids in there and say, ‘Swim.’ What suburban districts learned is that there has to be some transition and some assessment of where the students are in terms of learning. In St. Louis County, some of the districts have been doing this for quite some time. It’s not like this is happening in a place that has no experience in the suburbs, so that is a big plus, or should be.”

What the numbers say

Last year, figures were released that for the first time gave numerical evidence of what many educators had suspected for years: Black students who leave St. Louis to attend suburban schools under the deseg program do better on standardized tests than their counterparts who remain in classrooms in the city.

A report by the Voluntary Interdistrict Choice Corp., issued in May and updated in November, released numbers that broke out MAP test scores for students who transferred from the city to St. Louis County districts.

November’s report concluded that the data “suggests significant academic benefits (as measured by state standardized tests) resulting from being in the transfer program. These benefits are in addition to past studies which have shown both a higher graduation rate and a higher attendance rate for transfer students as well.”

Specifically, VICC-- which runs the deseg program -- said average MAP test scores for transfer students are higher than for black students who remain in the St. Louis Public Schools. The differences remains pretty much consistent from grade 3 to grade 8 but narrows in the high school years, perhaps because of a higher dropout rate by high school students in the city.

The numbers also showed that MAP scores for transfer students in individual districts that are still taking part in the deseg program generally are higher than for black students remaining in the city schools, in most grades and in most content areas that are tested.

In Parkway, for example, third-grade MAP scores in communication arts averaged 728.6 for transfer students compared with 694.4 for students in the city schools. By eighth grade, the comparison was 731.8 for students who transferred to Parkway to 706.7 for city students.

By high school, the scores were closer, with city students pulling ahead of Parkway slightly in the end-of-course communications arts test score.

The VICC report had a few caveats. It noted that the St. Louis totals included students in the city’s magnet schools and the non-magnet schools. Further, there was no effort to gauge differences in the scores of black students versus white students, only to compare transfer students to black students who stayed in the city.

Also, the report noted that test scores can be affected by a number of variables, such as student turnover, the socioeconomic status of students’ families, the parents’ level of education and the length of time a student has been in his or her current district. But the limitations of the study did not allow for those variables to be taken into account.

David Glaser, head of the VICC program, has also noted that the original purpose of the interdistrict transfers was not improved academic achievement but increased racial desegregation.

Not surprisingly, the city schools have a different view of the test scores.

Officials at the St. Louis Public Schools say that rather than comparing transfer students in the county to students who remain in the city, transfers’ test scores should be matched against MAP scores for students in the city magnet schools. The reason, they say, is that the VICC program sometimes rejects or ejects students with behavior problems, and that ability would make the VICC scores tend to be higher.

Under its analysis, MAP scores for communication arts in the magnet schools start higher than those in Parkway in grade 3 and continue to be higher – or in one grade, tied – through high school.

Asked by the Beacon to comment on the assumption that transfer students can be expected to score higher in their new district, city school superintendent Kelvin Adams said such a view often is a simplistic answer to a complex problem. The transfer program, he said, “gives that message. Whether it is true or not is another story.

“Every school district has isolated pockets of achievement and success. The challenge is to replicate that.”

The difference, Adams said, often comes down to a question of a school’s culture.

“You have to change the culture,” he said. “Part of what moving kids to a different district does is change the culture and help them believe they will rise to the level of achievement in their new school.

“It could work in St. Louis Public Schools. It could work in Riverview Gardens. It could work in Normandy. It could happen anywhere if you work to change the culture.”

A question of culture

That issue, what a school district’s culture is and how that culture can contribute to the success of transfer students, is crucial as the school year gets underway. This year, more than 2,600 students who live in Normandy and Riverview Gardens have signed up to attend classes in nearby accredited schools.

Kathleen Sullivan Brown, an education professor at the University of Missouri at St. Louis, says the first few weeks are difficult for any students who enter a new school, but the transfer students – and their teachers – face an enhanced set of circumstances.

“I just hung up the phone with someone from Francis Howell,” she said, mentioning one of the districts with a large share of transfers from Normandy. “He felt bad for the kids. I think that’s typical in new situations. And he also knows that none of us can process very well cognitively when we are in a state of fear or anxiety. The brain kind of freezes up.

“There is a whole constellation of things involved – a new discipline policy, new buildings, finding your way around, worrying about whether you will be late to class. All that can eventually find its way into your grades somehow. It may take a few days or weeks for kids to feel comfortable in their new environment.”

Once that initial newness wears off, Brown added, she hopes the preparations that the receiving districts have made, even though they were compressed into an unusually short period of time, will pay off.

“I would hope that the receiving districts have wonderful structures in place that are reflected in their academic results,” Brown said. “Once the commotion and the confusion that have been in everyone’s minds all summer diminish, teachers and principals know how to deal with these kids. One said to me, what’s the big deal? We’ve been dealing with this with the voluntary transfer students for 20 years. Give us the children and we’ll do our jobs.

“After a month or so, if there are significant kinds of structural problems, I think we will know that, and I really don’t expect to have those.”

A big factor, said Tate at Washington U., is which kids made the decision to go to a school away from their own neighborhood.

“Add in the fact that if you decide to transfer, and your parents or guardians are engaged enough in the system to get you out of your old school, you have a robust family structure,” he said. “So the people who transfer are different from those who stay.”

Once the students get established in their new surroundings, he added, their classmates will help set the tone.

“If you have peers who are pushing you academically,” Tate said, “you tend to behave like your peers. Let’s say a student who is a third grader transfers from Normandy to a district like Clayton and stays through high school. On average, I think their academic profile would be better by the time they graduate than it would have been if they had stayed.”

And, adds Eaton at Harvard, if academic performance can improve, more broad-based aims can be achieved as well.

“The goal is not just to raise student test scores,” she said. “The goal is to create a more cohesive region and give students long-range opportunities in their lives. It’s important to look at what happens to students over the long term.

“What we find is that student feel more secure, have more contacts in life, are more willing to enter colleges and persist in those environments when they see there is an opportunity for them to gain. All students have less prejudice and stereotyping, and students say they desire interracial neighborhoods and schools for themselves and their children.”

Move students or move teachers?

One criticism of the transfer programs has been that it represents a misplaced effort. Rather than remove students from their home districts, particularly those who are motivated to get a better education, attention and resources should focus on making failing districts better.

An option to do that exists in new school legislationthat takes effect next week. It allows an unaccredited district taken over by a special administrative board to contract with an accredited district “to deliver high quality educational programs to the residents of the district.” That could mean sending personnel into failing districts instead of letting students transfer out.

Given the importance of school culture and peer pressure on student success, would such an approach work? Would it bring better results for students than transfers would? Education researchers say there isn’t much evidence to go on to predict the possible outcome.

“That really doesn’t exist,” says Tate, “so you can’t get any data on it. We just don’t know. It would change the culture and the nature of that district, and I don’t even know whether it’s feasible to do that. They wouldn’t be the same. If it were that simple, every time you change superintendents and school boards, things would get better. You would think they would bring in smart people to run the system.

“But another way of thinking about that is if policies were implemented and stayed in place, and those districts get bankrupted and lose their population, what would happen to those kids? Would that be a better outcome? I don’t know because I don’t know about the capacity of those districts to take those kids.”

One flaw in such a plan, says Eaton at Harvard, would be finding personnel with the skills and the willingness to make such a switch.

“One of the big challenges that high-poverty schools face is retaining teachers,” she said. “There are a lot of reasons for that. Teachers who are under pressure to keep test scores up are less likely to go into districts if they feel like they are competing with a whole host of challenges that kids face that could affect why the test scores are low.

“In a perfect world, it sounds like it might be a solution worth trying, and it may help a little bit, but I don’t think it would necessarily solve the problem here. Teachers are only one of the factors. So I don’t think that would make a huge difference, and the research up until now doesn’t show that it would.”

Wells, from Columbia, said that one of the most important factors is the context in which learning occurs, so getting students out of their failing districts may in the end prove to be more successful than bringing in personnel to improve their home schools.

“Neither teachers nor students operate outside of their context,” she said. “Schools are very complicated organizations. It’s not something you can come in for an hour a day and do.

“So much of what we learned from the history of the deseg plan in St. Louis is that a lot of it was being in a place where the facility was nicer and the expectations were higher and students were with kids who could help them get internships in the summer. A lot of networking is going on.

“While maybe you could bring in personnel from other districts, I think it’s more of a systemic problem. When you have a lot of kids like you do in Riverview Gardens and Normandy, with schools spiraling down, and it builds up to a point where it’s not so surprising that they are low-achieving.”

What would you ask the data?

As part of our reporting on school transfers, Beacon presentation editor Brent Jones has been digging into the wealth of data provided by the Missouri Department of Elementary and Secondary Education.

As the transfer discussion continues, we'll continue to mine the data for more facts and context. You can help by letting us know what you want to know. If you could sit down with the data and ask it five hard questions, what would they be? Email questions and thoughts to education reporter Dale Singer, dsinger@stlbeacon.org.

Here's what we've looked at so far:

Beacon Back Story: School transfers have raised questions about violence

School transfers: The numbers game