Pre-K Interventions Harder than Hoped

Grover Whitehurst at Brookings:

On the Tennessee pre-K follow up: "Using the state test data and the full randomized sample, the evaluators report negative impacts for reading, math, and science scores at the end of third grade for children assigned to TVPK.[6] The negative impacts on math and science are statistically significant and substantive: children randomly assigned as preschoolers to TVPK had lost ground to their peers who had randomly not been offered admission to the pre-K program. The loss was equal to about 15 percent of the expected gap in test scores between black and white students at that age. In other words, children who were given the opportunity to attend TVPK were, on average, harmed by the experience in terms of their academic skills in elementary school. The reasons for this are not clear. One speculation is that the TVPK program was developmentally inappropriate, i.e., too much like kindergarten or first grade.[7]
There is no longer a methodological escape hatch for people who want to dismiss the results of the evaluation. It, along with the national Head Start Impact study,[8] are the only two large sample studies in the literature that have applied a random assignment design to modern scaled-up pre-K programs and followed children’s progress through school. Both show sizable positive effects for four-year-olds at the end of the pre-K year, but these effects either have either diminished to zero by the end of kindergarten year and stay there in later grades (Head Start) or actually turn negative (Tennessee). Advocates of greater public investment in state pre-K programs are beginning to incorporate these results into their thinking.[9]"

On Heckman and Abecedarian: "Abecedarian was a full-day, year-round program with a very low teacher-child ratio (1:3 for infants and 1:6 for five-year-olds), and unusually qualified staff and management [at UNC]. Children received their primary medical care on site through staff pediatricians, nurses, and physical therapists. There was a defined curriculum intended to foster language and cognitive skills and thereby increase IQ. Many efforts were made to involve families through voluntary programs. Supportive social services were provided to families facing problems with housing, food, and transportation. The annual direct expenditure per participant has been reported as roughly $19,000 in present dollars. ...
"Abecedarian is not a childcare intervention, nor a realistic model for one. It is a hothouse university-based program from nearly a half century ago for a few dozen children from very challenging circumstances who were deemed to be at risk of mental retardation. Its relevance to present day policies on childcare for the general population is uncertain, at best. Even ignoring this failure of external validity, the reported results from Abecedarian favoring the treatment participants are in doubt because the evaluation of Abecedarian’s impacts on participants is seriously compromised by a large imbalance in decline-to-participate rates by those assigned to the treatment vs. the control condition; and by the presence in the treatment group of an appreciable proportion children who were not randomized into that condition."

He concludes: "Go forward with promising ideas with a public acknowledgement of uncertainty and an approach designed to learn from error. Don’t place big and irrevocable bets on conclusions and recommendations that are far out in front of what a careful reading of the underlying evidence can support."