A recent article by Professor David Armor repeats many of the common arguments made by researchers opposed to current proposals for expanding preschool.  The article was published online by the Cato Institute, a libertarian think tank.

The article’s arguments have been frequently made by opponents of universal preschool. In particular, these arguments are similar to those made by Russ Whitehurst of the Brookings Institution, who has previously co-authored a blog post on preschool research with Professor Armor.

The argument in brief is as follows. First, these critics admit that the Perry and Abecedarian experiments provide evidence for the long-term benefits of these preschool programs, but this evidence is argued to be irrelevant to current preschool proposals. Second, the many “regression discontinuity” studies that show strong effects of many current preschool programs on kindergarten entrance test scores are argued to be biased.  Third, the Head Start and Tennessee experiments are argued to show that the effects of current preschool programs are likely to quickly fade.

I have already responded to these arguments in the past. For a more academic take on the research evidence, see my blog post responding to Whitehurst’s weighting of the research evidence, and explaining how I weight the evidence. For a more accessible discussion aimed at a broad audience, see chapter 4 of my new book, From Preschool to Prosperity, and also endnote 21 in that book.

Let me emphasize here just a few points.

First, regarding the Perry and Abecedarian experiments, these critics argue that the strong benefits of these programs are irrelevant to the current policy debate because the Perry and Abecedarian programs are more expensive per child than most current preschool programs. But why do these critics mainly use this argument as a reason to oppose current preschool programs?  If the critics accept that these more expensive preschool programs have high benefit-cost ratios, they should be arguing for spending more per child on highly-intensive preschool programs. Rather than an argument for doing nothing, the Perry and Abecedarian programs provide an argument for doing more.

Furthermore, the argument for today’s preschool programs is not that they will fully replicate the very large benefits of the Perry and Abecedarian programs. Rather, the argument is that at a lesser cost, current preschool programs will still provide a high benefit-cost ratio. As reviewed in my new book, Perry is estimated to increase adult earnings by 19% and the Abecedarian program by 26%. Most current preschool programs are estimated to increase adult earnings by 5 or 10%, but at a much lower cost. A full-day program with similar effectiveness to Tulsa’s preschool program is estimated to increase adult earnings by around 10%, but at a sufficiently lower cost that its benefit-cost ratio is somewhat higher than that of the Abecedarian program (see chapter 3 of my new book).

Second, regarding the “regression discontinuity design” (RDD) studies, Armor and other critics argue that these studies are biased by attrition from the treatment group. “Regression discontinuity” studies look at test scores at pre-K entrance, and at kindergarten entrance for children who attended pre-K the previous year. The studies than estimate how much of the increase in test scores for the latter group is due to the one year of aging, and how much is due to the pre-K program. This separation is done by looking at how test scores vary with age in both the pre-K entrant group and the kindergarten entrant group.

Armor’s point is that the “treatment group” of kindergarten entrants may miss some children who dropped out of preschool or who left the school district. This point is true. Armor argues that the omitted children will have lower test scores, which, if true, would bias the RDD results towards finding larger effects of pre-K. This argument, however, is by no means obvious. For example, in these studies of urban school districts, more upwardly mobile families who move to the suburbs are less likely to be in the kindergarten entrant group, and these children would probably tend to have above average test scores.

But Armor’s argument of bias in RDD estimates overlooks two important points. One point is that even when we redo regression discontinuity studies and only include children observed at both pre-K entrance and kindergarten entrance, we get similar estimated effects. This is shown in a study I did of a Kalamazoo pre-K program. Although attrition in principle might bias RDD estimates of preschool’s benefits, in practice attrition doesn’t seem to make much difference.

A second point is that the observed pattern of results in RDD studies of pre-K is inconsistent with attrition bias explaining these results. In Tulsa, the estimated RDD effects on test scores imply that half-day pre-K increases adult earnings by about 6%, and that full-day pre-K increases adult earnings by about 10%. (See p. 48 of my new book.)This pattern of effects is quite reasonable – one would expect full-day pre-K to have larger effects than half-day pre-K, but perhaps not twice as high effects due to some diminishing returns. But there is no reason that attrition bias should result in this pattern of effects. Why would attrition bias lead to a higher positive bias for full-day pre-K students than for half-day pre-K students?  There is no reason for this pattern to occur for attrition bias.

Third, Armor argues that the Head Start results imply that Head Start has no lasting effects compared to no preschool at all. This argument is based on a reanalysis of the Head Start results in a dissertation by Peter Bernardy, a former student of Armor’s. I’ve read Dr. Bernardy’s dissertation. In the dissertation, he does a non-experimental match of Head Start participants in the Head Start treatment group to children in the control group who attended no preschool.

Although Dr. Bernardy’s results are interesting, they are potentially subject to serious selection bias. As researchers know, the problem in evaluating preschool, or any educational or social intervention, is that there are unobserved pre-existing differences between program participants and non-participants that we can’t control for, because by definition we don’t observe these differences. Preschool families may be either more ambitious or needier than non-preschool families in ways we don’t observe, which will bias comparisons of preschool participants versus non-participants. The post-preschool differences between the two groups could be due to preschool, or could be due to the pre-existing differences.

The ideal solution to this selection bias problem is a randomized control trial. This is the “gold standard” for overcoming selection bias. If preschool participants are selected randomly, and the sample size is large enough, we have a reasonable expectation that the preschool participants and non-participants will be effectively the same on average in unobserved pre-existing characteristics.

Another good solution to the selection bias problem is some “natural experiment”, where preschool access varies with some factor that is arguably unrelated to unobserved characteristics. These natural experiments provide a “silver standard” for dealing with selection bias. For example, the RDD studies of preschool are really natural experiments, because the timing of pre-K access varies with age. Both the treatment group (kindergarten entrants who were in pre-K the previous year), and the control group (pre-K entrants) were similarly selected into preschool.

A less satisfactory solution to the problem of selection bias due to unobserved characteristics is to statistically control for observed characteristics. This is a “bronze standard” for dealing with selection bias. We may hope that by controlling for observed characteristics, we may also reduce differences in unobserved characteristics. But it is impossible to tell if this is so, and in practice, we know from many previous studies that “bronze standard” studies of program effectiveness have often proved misleading.

As Dr. Bernardy appropriately acknowledges in his dissertation, his estimates of the effects of Head Start vs. no-preschool may be biased by unobserved family and child characteristics.   (p. 89-90:  “It should be acknowledged that the omission of unobserved characteristics could lead to bias in the resulting estimation. For instance, varying levels of motivation among children’s parents that is not captured by the included covariates could yield a biased result.”) What direction or size might this bias have? We have no idea.

Professor Armor, along with Dr. Whitehurst and other critics of preschool, usually puts a heavy emphasis on “gold standard” results from randomized control trials. It is therefore odd that so much emphasis is placed in Professor Armor’s paper on Head Start results that are not based on a random experiment, or even on a natural experiment.

What is a non-researcher to make of all this debate over what studies to believe? A natural reaction is to say “we just don’t know anything”. This then argues for the need for more research before going forward with any ambitious preschool plan.

But what should be kept in mind is that Professor Armor and Dr. Whitehurst are distinctly in the minority of researchers in their interpretation of the overall research evidence. The research consensus is much more represented by a report in 2013 by ten of the most distinguished early childhood researchers, Investing in our future: The evidence base on preschool education. This research consensus finds, quoting from the main conclusions of the Executive Summary, that:

“Large-scale public preschool programs can have substantial impacts on children’s early learning….Quality preschool education is a profitable investment… Quality preschool education can benefit middle-class children as well as disadvantaged children… Long-term benefits occur despite convergence of test scores.”

Marketing doubt is easier than moving forward. But what must be always kept in mind is that there are costs not only to moving forward, but also to doing nothing. Children are only age 4 once. If we fail to provide preschool that could positively affect a child’s life course, that opportunity is lost forever. We can of course intervene later, but the main consensus conclusion from the early childhood research literature is that early interventions at ages 3 and 4 can have a particularly high benefit-cost ratio and can reach more children.

There are risks to moving forward. There are risks to doing nothing. Overall, the research consensus is that high-quality preschool has much higher benefits than costs. These benefits occur not only for the participating children and families, but for our overall economy and society.