Professor Bruce Fuller had an op-ed on preschool in the Washington Post on February 9. Professor Fuller’s interpretations of preschool research omit some important research.

Specifically, Professor Fuller argues that “youngsters from middle-class and well-off homes benefit little from preschool”.  He goes on to say that “young children attending quality half-day programs display the same learning gains as those attending full-day programs”.  Therefore, “we must avoid squandering scarce dollars on full-day programs for children who gain little from preschool”.

Professor Fuller cites some studies that support his arguments. But he fails to mention other studies that go against his arguments.

For example, Professor Fuller does not mention the research studies in Tulsa and Boston that find that universal preschool produces benefits for middle-class children that are only slightly less than the benefits for low-income children. Professor Fuller also does not mention a research study from New Jersey that finds significantly greater benefits from full-day preschool compared to half-day preschool.

An obvious and important question is: which studies should you believe? Should we believe the studies that Professor Fuller cites, or the studies that I cite? Or should we just say that the evidence is mixed and uncertain, which can be interpreted as an argument for inaction until more research is done?

The key problem in any preschool research is what social scientists call “selection bias”. The families that choose preschool differ from those who do not choose preschool, due to both family characteristics that we can observe, and family characteristics that we can’t observe. In addition, programs may choose to select preschool participants due to both observed and unobserved family characteristics.

For example, perhaps families that are more ambitious choose preschool. Or perhaps some preschool programs try to choose children who are easier to manage. Either source of selection would tend to mean that preschool participants will tend to do better than non-participants because of pre-existing family and child characteristics, above and beyond the true effect of the preschool program. Selection bias in estimating program effects would be positive.

Alternatively, perhaps families that are having more trouble with their children tend to try to put their children in preschool. Or perhaps preschool programs with a social mission try to choose needier children. These sources of selection will tend to produce a negative selection bias in estimating the true effects of preschool.

How can this selection bias be dealt with? If there are infinite resources and time, the ideal method is a large and perfectly-run randomized control trial. Preschool applicants would be randomly divided into a treatment and control group. As a result, we would expect average observed and unobserved characteristics in both the treatment and control group to be similar, and as the sample size gets larger, that expectation is increasingly likely to be realized.

But randomized trials are expensive and difficult to run, particularly on a large scale. Therefore, an alternative is to rely on natural experiments, in which some aspect of the world has resulted in different children having differing access to preschool, for reasons that have nothing to do with unobserved characteristics of the child and his or her family.  The treatment and comparison groups, with different access to preschool, will differ in preschool participation, but not observed and unobserved characteristics, and therefore we can interpret the outcome differences as being due to preschool, not pre-existing differences between the two groups.

A third method of trying to control for selection bias is to control for observed characteristics of the child and family.  Such controls help, but by their very nature cannot control for unobserved pre-existing differences between the treatment and comparison groups. Hence, such estimates may be subject to selection biases of unknown size and sign.

The Tulsa and Boston evidence that I am citing on middle-class benefits is based on natural experiments. Access to preschool and to kindergarten is based on an age cutoff.  The essence of the methodology used in these two studies is to compare the test scores of children who just missed the kindergarten age cut-off and are therefore just entering preschool, with test scores of similar children who just made the kindergarten age cut-off, who are just entering kindergarten, and who participated in preschool the preceding year.  These two groups are arguably similar in unobserved as well as observed characteristics because they were similarly selected into the same preschool program. The timing of their preschool access was based on age, and a few days of age in either direction should not make a big direct difference in test scores. The “jump” in test scores that is observed for the slightly older group in such studies is therefore reasonably attributable to the preschool participation the preceding year.

The New Jersey evidence I am citing on full-day versus half-day preschool is based on a randomized control trial. Excess applicants for a full-day preschool opportunity were randomly assigned to either receive full-day preschool, or only receive half-day preschool. The results showed significantly greater test score effects of full-day preschool. In Bartik (2011), I used these estimates to predict that full-day preschool produces 56% greater earnings benefits than half-day preschool.  Therefore, there are some diminishing returns to preschool time (benefits are not doubled), but there are benefits to full-day preschool over half-day preschool.

Most of the evidence that Professor Fuller cites is from the third category of studies, which only can control for observable child and family characteristics. These studies may be biased upwards or downwards by selection bias. Therefore, I would not weigh these studies as heavily.

In my view, the research studies that should receive the greatest weight use randomized or natural experiments to examine the causal effects of preschool, which avoids problems due to selection bias. The research studies that use such evidence support middle-class benefits of preschool, and support greater benefits for full-day programs.