5 Results

5.1 Enrollment and Graduation Rates

To visualize the enrollment impacts of the Adams Scholarship, we plot in Figure 5 the proportion

of 2005-06 graduates for each value of Gap who enroll in four-year colleges immediately following

high school graduation.24 There is clear visual evidence that students at the eligibility threshold

are substantially more likely to enroll in an Adams (i.e., in-state public) college than students just

below the threshold. Such students are, however, similarly less likely to enroll in a non-Adams

(i.e., in-state private or out-of-state) college, the net result of which is little apparent difference in

overall college enrollment rates between these two groups of students.

The first row of Table 4 reports estimates of these differences in the 2005-06 sample. Scholarship

eligibility induced 6.9 percent of students at the threshold to enroll in Adams colleges, a more than

one-fourth increase over the 23.8 percent enrollment rate of students just below the threshold.25

More than six-sevenths of these marginal students, or 6.0 percent, would have attended other fouryear colleges if not for the scholarship.

The net result is a statistically insignificant 0.9 percentage

point increase in the fraction of students enrolling in any four-year college. Many of these marginal

students switched their enrollment from out-of-state colleges, leading to a 4.8 percentage point

increase in the fraction of students enrolling in-state four-year colleges. The Adams Scholarship

therefore did induce a substantial number of students to enroll in the public sector and succeeded

23The 2005-08 sample looks quite similar, as seen in Figure A.10.

24Immediate enrollment was a requirement of the scholarship.

25Table A.2 shows that nearly half of these marginal students enrolled in U. Mass. Amherst, the flagship campus,

and another third enroll in the various state colleges.

15

in keeping some students in state who otherwise would have left.

The scholarship also induces a statistically insignificant 0.8 percentage point increase in the

fraction of students who enrolled in two-year community colleges. That, combined with the slight

rise in four-year college enrollment rates, implies that the scholarship raised overall immediate

college enrollment rates by 1.7 percentage points. In the second row, we define as the outcome

enrollment within two years of high school graduation, rather than immediately following graduation. The estimates in columns 1 and 3 fall by 0.6 and 0.9 percentage points respectively, suggesting that a small number of marginal students induced to enroll immediately in Adams colleges

because of the scholarship would have enrolled within the next two years in the absence of the

scholarship. The estimates in column 3 suggest that the scholarship may have accelerated enrollment in four-year colleges for a small number of students but did not induce enrollment in

four-year colleges for any students who would not have enrolled within two years. Interestingly,

the two-year college effect is unchanged by the shift in definition from immediate enrollment

to enrollment within two years.

This suggests that scholarship eligibility may have induced a

small number of students to enroll in community colleges who would not otherwise have enrolled

within two years.

Turning from enrollment to graduation, we plot in Figure 6 the proportion of students for each

value of Gap who graduate from four-year colleges within six years of high school graduation.

Students just above the eligibility threshold are more likely to have graduated from Adams colleges than those just below the threshold, an unsurprising result given that the former are much

more likely to enroll in that sector than the latter. Scholarship eligibility also lowers graduation

rates from non-Adams colleges, for the same reason that eligibility reduces initial enrollment in

that sector. More surprising is that the decrease in graduation rates from non-Adams colleges is

larger in magnitude than the increase in graduation rates from Adams colleges. The net result is

that scholarship eligibility lowers overall graduation rates, as shown by the top line in Figure 6,

where points to the right of the eligibility threshold are generally lower than would be predicted

if extrapolating from points to the left of the threshold.26

26One anomaly is that students exactly on the threshold (GAP = 0) have higher graduation rates than students just to

the left of the threshold (GAP = −1). We believe this is either an artifact of noisy data or driven by the slight bunching

16

The third through fifth rows of Table 4 confirm this decrease in graduation rates. The third

row uses as an outcome an indicator for the student being enrolled in a given college sector as of

the spring of the fourth year after her high school graduation, which we interpret as a measure

of persistence. The fourth and fifth rows use as outcomes indicators for whether a student has

graduated from a given college sector within four or six years. The three rows tell a consistent

story. Though scholarship eligibility increased enrollment in Adams colleges by nearly seven

percentage points, it increased persistence and six-year graduation rates by only three percentage

points, suggesting that the majority of marginal students did not successfully graduate from that

sector. Scholarship eligibility reduced persistence and graduation rates in the private sector by

over five percentage points.

The net result is that scholarship eligibility reduced the probability of

earning a four-year college degree within six years by 2.5 percentage points. That the persistence

and four-graduation rate measures show similar declines suggests this is not merely a matter of

delaying graduation but instead is driven by a subset of students who have dropped out of the

four-year college sector entirely.

We note three other important findings. First, although scholarship eligibility increased the

number of students enrolling in state, it had no ultimate effect on the probability of earning a

degree in state. Second, none of the increased enrollment in community colleges translated into

increased completion of two-year college degrees, even six years out of high school. Third, as a

result, scholarship eligibility lowered by 2.5 percentage points the probability that a student had

any college degree six years after high school graduation.

Table 5 explores these enrollment and graduation impacts over time, with the first four columns

analyzing each high school class separately, the fifth pooling the classes of 2005-08, and the sixth

showing enrollment effects for the classes of 2009-11, the most recent for which data are available.

Panel A shows that scholarship eligibility increased enrollment in four-year Adams colleges for

all graduating high school classes. There is, however, a gradual monotonic decrease in the impact

at zero described earlier in the text. Regression estimates that we discuss below, which show a clear discontinuity in the

overall graduation rate, become even larger in a “doughnut hole” regression that excludes students on the threshold.

For evidence of this, see Table A.3. Though not show here, difference-in-difference estimates that use the 2004 cohort as

a pre-policy control group show similarly negative impacts on graduation rates, confirming that students to the right

of the threshold are graduating at lower rates than would otherwise be predicted.

17

of scholarship over time, with the effect in 2005 three times that of the effect over 2009-11. This

gradually shrinking effect size may be driven by the fact that rapidly rising fees have shrunk the

proportion of college costs covered by the scholarship. Also worth noting is that much of increase

in overall four-year college enrollment is driven by the first treated class, with subsequent classes

showing smaller and insignificant impacts on this margin.

Panel B estimates the impact of scholarship eligibility on persistence and graduation after four,

five and six years. Three findings are worth noting. First, that the magnitude of the persistence

and various graduation rates do not vary much within classes implies that the negative impact of

scholarship eligibility on graduation rates is driven largely

by dropout rather than delay. Second,

that the negative graduation effect is not driven solely by the first high school class makes much

less likely the possibility that the effect was generated by confusion about the meaning of “free tuition” in the scholarship letter. If such language was deceiving students into making uninformed

decisions, we would expect such negative graduation effects to diminish across classes as information about the true value of the scholarship spread. There is no clear evidence of such a pattern.

Third, estimated impacts on enrollment and persistence rates generated by the full 2005-08sample

are similar to those generated by the 2005-06 sample. Figures A.11 and A.12 confirm this, replicating Figures 5 and 6 for the larger sample.

The two sets of figures look quite similar. As a whole,

this evidence suggests a fairly stable impact of the scholarship on enrollment, persistence and

graduation.

Table A.3 tests the robustness of our central results to a variety of alternative specifications.

Panel A replicates our default local linear regression specification for a variety of bandwidths,

beginning with the Imbens-Kalyaramanan optimal bandwidth. In the 2005-06 sample, that optimal bandwidth is about 14 for enrollment outcomes and about 10 for persistence and graduation

outcomes, hence our choice of 12 as the default bandwidth in earlier tables. The magnitude and

statistical significant of these estimates are generally quite robust to these changes in the bandwidth.

Panel B replicates our default specification, using a bandwidth of 12, with three variations.

First, we include school district by high school class fixed effects to account for the fact that the

18

eligibility threshold differs by district and class. This has very little impact on the estimates. Second, we include demographic controls, which also change the estimates very little. Third, we run

a doughnut hole regression in which we exclude students exactly on the boundary, because of the

small amount of bunching observed in the running variable. This actually increases the magnitude of our enrollment and graduation estimates by roughly a percentage point, suggesting that

the mild bunching was, if anything, causing us to underestimate the impacts of the scholarship.

Finally, panel C fits quadratic, cubic and quartic polynomials on either side of the threshold, using

the entire sample and a rectangular kernel. This yields similar estimates to those generated by the

local linear regression used as our default specification.

As a final piece of evidence, Figure 7 exploits as a placebo test data from the high school class

of 2004, the one cohort in our data that graduated prior to the scholarship’s existence. In panel A,

we see no evidence of a discontinuity in Adams college enrollment for the class of 2004, compared

to the previously observed clear discontinuity for the classes of 2005-06. Similarly, panel B shows

that students below the threshold have similar six-year graduation rates across the three classes,

whereas students above the threshold in 2005-06 have lower rates than such students in 2004. That

the discontinuities in enrollment and graduation appear only in the years when the scholarship

existed strengthens the case that it is due to the policy itself and not other unobserved factors.

Panel A also highlights that the magnitude of the enrollment impact is large even for students

somewhat far from the threshold. Our regression discontinuity estimates, as well as those based

on difference-in-difference calculations following Figure 7,

suggest that the Adams Scholarship

induced about 1,000 additional students to enroll in in-state public colleges. IPEDS data reported

by Massachusetts’ public colleges themselves confirms this. Figure 8 plots the reported freshman

enrollment across all Massachusetts public four-year colleges, both for all students and for those

from Massachusetts. There is a clear trend break in 2005, when the Adams Scholarship begins,

due entirely to increased numbers of Massachusetts freshman and of magnitude nearly identical

to our estimate. This implies that the additional students induced into in-state public colleges

did not crowd out other students, instead simply adding to each campus at most a few hundred

students who would not otherwise have enrolled there.