In response to a reader request, I looked at a recent article on Obama’s preschool proposal, written by Grover J. “Russ” Whitehurst of the Brookings Institution.  Dr. Whitehurst is a child psychologist who previously directed the research arm of the Department of Education.

The purpose of the article is to see what student achievement effects might be plausible for President Obama’s proposal to help states expand their pre-K programs. Dr. Whitehurst is critical of Secretary of Education Duncan for citing Nobel Prize winning economist James Heckman’s research that programs such as the Perry Preschool program have a return of $7 for every dollar invested.

Dr. Whitehurst comments that

“I’m willing to bet the farm that the typical state pre-K program for four-year-olds that would be expanded if the Obama administration’s proposal were enacted isn’t going to have the impact of Perry…”

My comment: I agree that the typical state pre-K program would probably not have as great an impact as Perry. Of course, the typical state pre-K program that would be expanded probably will spend only a fraction of what Perry would cost. Perry had a class size of 13 students per 2 certified teachers paid public school wages, and lasted for 2 years (ages 3 and 4) for most child participants. The total cost in today’s dollars was over $11,000 per student per year, or $22,000 per student for the two years. The typical state pre-K program does not spend anywhere near that amount per student. So, expanding state pre-K programs could still have very high benefit-cost ratios even if their benefits are only a small fraction of Perry’s benefits.

Dr. Whitehurst goes on to do some analysis of the correlation between state pre-K enrollments in 2006, and 4th grade test scores on the National Assessment of Educational Progress in 2011, 5 years later, when those former pre-K participants would have been in 4^th grade.  He controls for state median income, the percentage of the state’s school-aged population that is non-white, and the percentage of the state’s 4^th graders who qualify for a free or reduced price lunch. He finds that with these controls, higher state pre-K participation in 2006 is associated with higher NAEP test scores in 2011 in reading and math.

As Dr. Whitehurst acknowledges, it is hard to be sure that this analysis is picking up causal effects of state pre-K participation on subsequent test scores. There are many unobservable state characteristics that may also affect test scores, and that might be correlated with state pre-K participation.  For example, Dr. Whitehurst is concerned in an endnote that “states that have invested the most in their pre-K programs [may] have also been more active than other states in instituting other education reforms”.

My comment: Given the problems with unobserved state characteristics, this analysis should be considered much less authoritative than previous pre-K studies that do have good comparison groups.  Of most relevance are programs similar to most state pre-K programs. This includes the Chicago Child-Parent Center program, which has a good comparison group, and both short-term and long-term evidence for significant benefits.  It also includes the many regression discontinuity studies of state pre-K programs that show short-term test score benefits, which I summarized in my recent paper on Kalamazoo’s pre-K program.  If we’re looking to predict test score effects of the Obama pre-K program, we should be looking at these studies with good comparison groups, not relying on regressions with aggregate state data that cannot control for unobserved state characteristics.

Dr. Whitehurst’s spin on his analysis is that it only shows modest effects of state’s pre-K enrollment on NAEP test scores, which means that any proposed expansion of such programs will need to be carefully targeted and designed in order to have net benefits:

“What do I conclude?

1. There are modest positive associations between enrollment levels in state pre-K and later academic achievement once demographic differences among states are taken into account.

2. If these associations reflect a cause and effect relationship then raising the level of state pre-K enrollment would enhance academic achievement.

3. The impact of very substantial increases in the level of state pre-K enrollment (e.g., two standard deviations of current enrollment levels or about 32%) would likely be no more than a few points on NAEP.

4. Raising NAEP scores a couple of points is worth pursuing in the context that national NAEP scores on reading in 4^th grade were only four points higher in 2011 than they were in 1992, but this falls far short of the impacts that advocates of the expansion of state pre-K have touted based on extrapolations of findings from studies of a few high-cost, multiyear, boutique preschool programs from many years ago, such as Perry.

5. If we are to move forward with a new federal program to support state pre-K programs we need to think carefully about the costs and benefits and figure out how to minimize the former while maximizing the latter. “

My comment: Of course, I agree we need to think carefully about benefits and costs of any new federal program, whether pre-K or any other proposed program. (I’m an economist and policy wonk; how could I be against benefit-cost analysis?)

However, I think that Dr. Whitehurst’s numbers as presented actually suggest extremely high benefit-cost ratios for expansion of state pre-K programs, perhaps even comparable to some figures for Perry.  The underlying reason is that even very slight increases in NAEP test scores have large predicted future benefits, while the “very substantial” increases in state pre-K enrollment he discusses would have quite modest costs.

To see this, I think it’s easier to do the analysis in terms of the benefits and costs from having one more child in a state pre-K program.  Whitehurst’s model as presented assumes that benefits and costs are scaled up or down proportionately with the number of children participating, so benefit-cost ratios do not depend on program size.

Whitehurst’s estimates imply that going from 0% of a state’s 4-year-olds in pre-K, to 100% of all 4-year-olds in pre-K, would raise NAEP scores by 7 points. It is as if each child who participates in pre-K raises his or her NAEP scores by 7 points. (Or alternatively, that child may raise his or her own test scores by less than 7 points, but have spillover benefits for other students that add up to a total of a 7 point increase in NAEP scores.)

A seven point increase in NAEP scores is about an “effect size” of 0.2.  (“Effect size” is education jargon for the test score increase divided by the “standard deviation” of test scores among children, in this case the standard deviation of 4^th grade test scores.) This is equivalent near the median test score to an increase in test scores of about 8 percentiles. For example, if the child would have scored at the median or 50^th percentile, pre-K participation would increase their 4^th grade test scores by 8 percentiles to the 58^th percentile.

What are the benefits from an increase in 4^th grade test scores of 8 percentiles? One benefit is the predicted increase in future earnings because students with higher test scores will have higher educational attainment, higher employment rates and higher wage rates.  The analysis by Chetty et al. suggests that an increase of one percentile in 4^th grade test scores increases adult earnings by about 0.6%. (Wonk note: This takes the Chetty et al. figures from column 1 of Appendix Table V, and divides by mean earnings in Table I.) Therefore, an increase in a child’s 4^th grade test scores of 8 percentiles would increase his or her future adult earnings by a little less than 5%.

Using some Current Population Survey figures on adult earnings by age, developed for my 2011 book on pre-K, the average present value for an individual worker of adult earnings in the U.S., discounted back to age 4 at a 3% real discount rate, is about one-half million dollars. A 5% increase would increase the present value of earnings by about $25,000. (All figures have been adjusted to 2012 dollars.) Even a slight increase in worker skills and earnings, when added over a worker’s entire career, can sum to a considerable benefit.

Obviously one could tweak these numbers to get different estimates. But the result remains that one would expect that these NAEP test score increases would be associated with adult earnings increases whose present value is at least in the tens of thousands of dollars per individual worker.

What does pre-K cost for this one child? Well, back in 2006, state spending per pre-K participant averaged a little less than $4,000 per child. (Source: The same NIEER report used by Whitehurst, but I adjusted the dollar figures to 2012 dollars.) There also was some local spending per pre-K participant. However, it seems unlikely that average local spending totaled more than $1,000 per participant on average. So the total spending per child was probably less than $5,000.

What do I conclude?

1. Based on Whitehurst’s regression, state pre-K programs would be predicted to increase the present value of future adult earnings by over $25,000 per individual child, at a cost of $5,000 per child, for a benefit-cost ratio of 5 to 1.
2. The earnings benefits might be even greater if one looked at kindergarten test score effects. Pre-K studies show substantial fading of test score effects from kindergarten to 4^th grade. Yet the Chetty study as well as other studies (for example, of the Chicago Child Parent program) suggest that the initial test score effects are a better predictor of long-run effects on adult earnings.
3. In addition, if state pre-K programs have any anti-crime benefits or other benefits, this would substantially increase the benefit-cost ratio. In many studies, including Perry and the Chicago program, the anti-crime benefits of pre-K are of similar magnitude to the earnings benefits (see Bartik, Gormley, and Adelstein article for review and links).
4. As presented, the Whitehurst estimates numbers provide no basis for different benefit cost ratios for different scales of expanding pre-K programs. The benefits and costs per participating child would seem to be the same, keeping the benefit-cost ratio the same. Increasing pre-K participation by only 32% rather than 100% would have only 32% as great benefits, but also 32% as great costs.  Whitehurst gets more modest test score effects of a 32% increase in pre-K participation because these test score increases are averaged over all kids. But he does not point out that the costs averaged over all kids will also be scaled back.
5. All these calculations rest on a regression that may be biased due to unobserved state characteristics. Better calculations should be based on the Chicago CPC studies and the state pre-K regression discontinuity studies and other state studies. These studies often show high benefit-cost ratios.  For example, a recent CPC study shows a benefit-cost ratio from a one-year pre-K program of over 13 to 1.  Even if typical state programs were less cost-effective than CPC, they could still have very large benefit-cost ratios.

Dr. Whitehurst finishes his article with his brief thoughts on increasing the efficiency of pre-K programs. His suggested pre-K reforms are not provided in his article with much justification. But his thoughts are rationalized by the notion that expanding pre-K programs as they currently exist, without dramatic reforms, would not have high enough benefit-cost ratios, which he believes to be the implication of his analysis of NAEP test scores. However, as I have argued here, his analysis actually implies that state pre-K programs as they currently exist have high benefit-cost ratios. This analysis is based on weak empirical evidence, as he acknowledges, but his results are consistent with previous research with better comparison groups.  Dramatically increasing the efficiency of pre-K programs is a desirable goal if it is achievable with well-researched policies. But in my opinion, even without dramatic reforms, we already know how to run pre-K programs at a state level that have high benefit-cost ratios, and that offer broad access to many income groups.