Tablet Magazine today published the first detailed, independent analysis of two articles that have been widely invoked as demonstrating large academic benefits from an ethnic studies course. The new analysis, by Richard Sander (UCLA) and Abraham Wyner (University of Pennsylvania) finds that the beneficial claims are wholly unsupported by the evidence. The data, they say, could just as easily be argued to show a harmful effect from the course as a beneficial effect — but the soundest scientific conclusion is that the two original studies show nothing at all.
The studies in question are by Thomas Dee, an education professor at Stanford, and Emily Penner, an education professor at the University of California at Irvine, who in 2017 published a paper evaluating a pilot ethnic studies course implemented in five San Francisco public high schools. Dee and Penner were joined by Sade Bonilla (University of Massachusetts) in a 2021 follow-up study.
Both studies were used to support California’s 2021 law, AB 101, which seeks to make ethnic studies a high school graduation requirement. And both studies are also being used to support a controversial proposal to add ethnic studies to the requirements for admission to the University of California, which is being considered this week by the University of California Academic Council. If the proposal is approved, it will be the first time in more than 20 years, and only the second time in nearly a century, that a subject area is added to the requirements for UC admission. Currently, all 10 UC campuses require incoming freshmen to have taken a designated number of high school courses in seven subject areas, such as history, English, mathematics and science. The new proposal would add a “Liberated Ethnic Studies” overlay on a UC-required course.
The Dee, Penner, and Bonilla studies were based on tracking a group of eighth graders who were given the option of taking an ethnic studies course in ninth grade. One hundred and twelve students who struggled academically in eighth grade (i.e., had GPAs below 2.0) were “assigned” to the course, but over a third of these opted out. The 92 percent of students with GPAs above 2.0 were allowed to “opt-in” to the course, and about one in 10 of these students did so. The authors then compared the ninth-grade (and later) outcomes of students who were on either side of the 2.0 “assignment” threshold. The authors claimed that the results showed huge benefits from assignment to the ethnic studies course – an increase of 1.3 points in GPA (e.g., “C” students becoming “B+” students), and similarly dramatic increases in attendance, credits taken, and graduation rates.
Sander and Wyner, in a detailed, 5,000-word analysis of the two papers, find none of the claimed findings supported. Dee and Penner repeatedly confuse the distinction between being “assigned” to the ethnic studies course and actually taking it; on the one occasion where they disaggregate results by those who did and did not take the course, they find no statistically significant difference in outcomes. Far from “raising” ninth-grade GPA by 1.3 points, Dee and Penner’s data show that students in the treatment group generally had much lower grades in ninth grade than in eighth grade. On one outcome where treated students do appear to perform better – high school graduation — the effect occurs for students regardless of whether they take the ethnic studies course.
According to Sander and Wyner, all of the key analyses in the Dee-Penner-Bonilla papers violate basic rules of statistical inference. Just as disturbingly, these authors have not made their data available for other scholars to check or replicate their research, and do not report basic statistics, such as the distribution of outcomes for persons taking and not taking the ethnic studies course. Yet these two studies – which even the lead author does not think shows benefits that can be generalized to all students – have been relied upon, by legislators, California’s Governor Newsom, and academic committees at the University of California, to justify universal ethnic studies for high school students. Sander and Wyner observe that “the publication of these two astoundingly shoddy works and their importance to the ethnic studies movement should raise the alarm – not only that the editing and peer-review process at [the two journals that published the articles] needs overhauling, but that California parents are not being told the truth about a potentially significant change in the education of their children.”