The ‘Best’ High School Academically Irrelevant?

But what if the high-school rat race is largely for naught? That’s the provocative conclusion of a new study that examined students who attended public high schools in Chicago. Surprisingly, students at selective-enrollment schools didn’t seem to benefit academically compared with similar students at different schools. “There is a lot of competition for students to get into selective-enrollment high schools,” said Marisa de la Torre, a co-author of the study and an associate director at the University of Chicago Consortium on School Research. “What the study says is, it’s not the end of the world if you cannot get into selective-enrollment high schools.”

There are, of course, some benefits to attending a competitive high school. The study found that students at selective schools were less likely to be suspended or to miss school and reported that they felt safer. In addition, students who attended high-scoring non-selective schools did see academically significant benefits relative to those at low-achieving schools. The study highlights the challenge of measuring school quality and suggests that many parents and education policymakers are judging schools by the wrong metrics.

* * *

The study, conducted by four researchers affiliated with the consortium, was published this month in the peer-reviewed journal Educational Evaluation and Policy Analysis. The researchers examined ninth-graders who entered Chicago’s public high schools, including charter schools, between 2008 and 2011. Interestingly, about two thirds did not attend their zoned neighborhood high school in the 2011–12 school year. The researchers divided schools into four groups: selective, top-tier, middle-tier, and bottom-tier. The first group consisted of schools that admit students based largely on test scores. The latter three groups were ranked by their students’ ACT scores and high-school graduation rates.

The study compared students against peers who attended different-tier schools but were otherwise similar based on traits including past test scores, degree of parental involvement, and home neighborhood. This approach isn’t perfect, but it allows researchers to estimate the impact of schools while holding student characteristics constant.

When simply making raw comparisons between students at selective-enrollment versus other city schools, the differences appear stark: Students at selective schools scored more than seven points higher on the ACT, which has a maximum score of 36. Yet when researchers controlled for a variety of factors to isolate the effect of attending a selective school, the disparities all but vanished. Attending a selective-enrollment school led to only a statistically insignificant bump in the ACT of half a point. The selective schools also seemed to have little or no effect on the likelihood of taking Advanced Placement classes, graduating from high school, or enrolling and staying in college.

Selective schools did, however, produce a variety of non-academic gains: Students had higher attendance and lower suspension rates, and they trusted their teachers more. Students also reported that their peers’ behavior was much better and that they felt safer in school—this suggests that insofar as selective schools are beneficial, it may be because of higher-achieving peers rather than better-quality instruction.

Again, to get these results, the study compared students at selective schools with similar students at different schools. Notably, though, the second group of students largely ended up at relatively high-achieving schools in the city—very few went to Chicago’s lowest-scoring schools. And so, the lack of academic boost students gained from the selective schools may be attributable to the relative quality of the fallback schools in comparison.

* * *

The researchers also looked at the outcomes of three levels of non-selective schools in Chicago: top-tier vs. middle-tier, top-tier vs. bottom-tier, and middle-tier vs. bottom-tier. The differences between attending a top- and bottom-tier school were vast. However, the gulf between a top-tier school and a middle-tier one was mixed: Attendance at the high-achieving schools led to gains on the ACT but produced small reductions in high-school graduation rates compared with middling schools. Differences in college outcomes generally favored top-tier schools but were usually small and not statistically significant. Similarly, students didn’t seem to make big gains from attending a mid-tier school vs. a bottom-tier one.

This suggests, unsurprisingly, that students are most likely to benefit when moving from a very low-achieving school to a high-achieving one. Still, this isn’t the first research to suggest that selective schools may not be as good as their reputations suggest. A study of New York City’s specialized high schools, including Stuyvesant, found that students who just made the exam cutoff for admission were no more likely to attend or complete college than students who just missed the cutoff score.

In other words, schools matter, but perception of the best schools may not line up with reality. One partial explanation, reflected in the latest Chicago study, is that the fallback school options for students considering attending a selective school are still quite high-achieving. But another answer is that the common measurement of schools—raw test scores, unadjusted for differences in the level where students start—is misleading. (And this doesn’t even get into the limits of exclusively using test scores.) In a 2008 survey, an overwhelming majority of researchers said that schools should be evaluated instead based on how much their students grow academically over time.

As long as the popular conception of a good school is based on a flawed measure, excellent schools that don’t screen students by test scores or real-estate prices won’t get the credit they deserve—and schools with such careful selection will get far too much credit.

Matt Barnum