5 Results
5.1 Enrollment and Graduation Rates
To visualize the enrollment impacts of the Adams Scholarship, we plot in Figure 5 the proportion
of 2005-06 graduates for each value of Gap who enroll in four-year colleges immediately following
high school graduation.24 There is clear visual evidence that students at the eligibility threshold
are substantially more likely to enroll in an Adams (i.e., in-state public) college than students just
below the threshold. Such students are, however, similarly less likely to enroll in a non-Adams
(i.e., in-state private or out-of-state) college, the net result of which is little apparent difference in
overall college enrollment rates between these two groups of students.
The first row of Table 4 reports estimates of these differences in the 2005-06 sample. Scholarship
eligibility induced 6.9 percent of students at the threshold to enroll in Adams colleges, a more than
one-fourth increase over the 23.8 percent enrollment rate of students just below the threshold.25
More than six-sevenths of these marginal students, or 6.0 percent, would have attended other fouryear colleges if not for the scholarship.
The net result is a statistically insignificant 0.9 percentage
point increase in the fraction of students enrolling in any four-year college. Many of these marginal
students switched their enrollment from out-of-state colleges, leading to a 4.8 percentage point
increase in the fraction of students enrolling in-state four-year colleges. The Adams Scholarship
therefore did induce a substantial number of students to enroll in the public sector and succeeded
23The 2005-08 sample looks quite similar, as seen in Figure A.10.
24Immediate enrollment was a requirement of the scholarship.
25Table A.2 shows that nearly half of these marginal students enrolled in U. Mass. Amherst, the flagship campus,
and another third enroll in the various state colleges.
15
in keeping some students in state who otherwise would have left.
The scholarship also induces a statistically insignificant 0.8 percentage point increase in the
fraction of students who enrolled in two-year community colleges. That, combined with the slight
rise in four-year college enrollment rates, implies that the scholarship raised overall immediate
college enrollment rates by 1.7 percentage points. In the second row, we define as the outcome
enrollment within two years of high school graduation, rather than immediately following graduation. The estimates in columns 1 and 3 fall by 0.6 and 0.9 percentage points respectively, suggesting that a small number of marginal students induced to enroll immediately in Adams colleges
because of the scholarship would have enrolled within the next two years in the absence of the
scholarship. The estimates in column 3 suggest that the scholarship may have accelerated enrollment in four-year colleges for a small number of students but did not induce enrollment in
four-year colleges for any students who would not have enrolled within two years. Interestingly,
the two-year college effect is unchanged by the shift in definition from immediate enrollment
to enrollment within two years.
This suggests that scholarship eligibility may have induced a
small number of students to enroll in community colleges who would not otherwise have enrolled
within two years.
Turning from enrollment to graduation, we plot in Figure 6 the proportion of students for each
value of Gap who graduate from four-year colleges within six years of high school graduation.
Students just above the eligibility threshold are more likely to have graduated from Adams colleges than those just below the threshold, an unsurprising result given that the former are much
more likely to enroll in that sector than the latter. Scholarship eligibility also lowers graduation
rates from non-Adams colleges, for the same reason that eligibility reduces initial enrollment in
that sector. More surprising is that the decrease in graduation rates from non-Adams colleges is
larger in magnitude than the increase in graduation rates from Adams colleges. The net result is
that scholarship eligibility lowers overall graduation rates, as shown by the top line in Figure 6,
where points to the right of the eligibility threshold are generally lower than would be predicted
if extrapolating from points to the left of the threshold.26
26One anomaly is that students exactly on the threshold (GAP = 0) have higher graduation rates than students just to
the left of the threshold (GAP = −1). We believe this is either an artifact of noisy data or driven by the slight bunching
16
The third through fifth rows of Table 4 confirm this decrease in graduation rates. The third
row uses as an outcome an indicator for the student being enrolled in a given college sector as of
the spring of the fourth year after her high school graduation, which we interpret as a measure
of persistence. The fourth and fifth rows use as outcomes indicators for whether a student has
graduated from a given college sector within four or six years. The three rows tell a consistent
story. Though scholarship eligibility increased enrollment in Adams colleges by nearly seven
percentage points, it increased persistence and six-year graduation rates by only three percentage
points, suggesting that the majority of marginal students did not successfully graduate from that
sector. Scholarship eligibility reduced persistence and graduation rates in the private sector by
over five percentage points.
The net result is that scholarship eligibility reduced the probability of
earning a four-year college degree within six years by 2.5 percentage points. That the persistence
and four-graduation rate measures show similar declines suggests this is not merely a matter of
delaying graduation but instead is driven by a subset of students who have dropped out of the
four-year college sector entirely.
We note three other important findings. First, although scholarship eligibility increased the
number of students enrolling in state, it had no ultimate effect on the probability of earning a
degree in state. Second, none of the increased enrollment in community colleges translated into
increased completion of two-year college degrees, even six years out of high school. Third, as a
result, scholarship eligibility lowered by 2.5 percentage points the probability that a student had
any college degree six years after high school graduation.
Table 5 explores these enrollment and graduation impacts over time, with the first four columns
analyzing each high school class separately, the fifth pooling the classes of 2005-08, and the sixth
showing enrollment effects for the classes of 2009-11, the most recent for which data are available.
Panel A shows that scholarship eligibility increased enrollment in four-year Adams colleges for
all graduating high school classes. There is, however, a gradual monotonic decrease in the impact
at zero described earlier in the text. Regression estimates that we discuss below, which show a clear discontinuity in the
overall graduation rate, become even larger in a “doughnut hole” regression that excludes students on the threshold.
For evidence of this, see Table A.3. Though not show here, difference-in-difference estimates that use the 2004 cohort as
a pre-policy control group show similarly negative impacts on graduation rates, confirming that students to the right
of the threshold are graduating at lower rates than would otherwise be predicted.
17
of scholarship over time, with the effect in 2005 three times that of the effect over 2009-11. This
gradually shrinking effect size may be driven by the fact that rapidly rising fees have shrunk the
proportion of college costs covered by the scholarship. Also worth noting is that much of increase
in overall four-year college enrollment is driven by the first treated class, with subsequent classes
showing smaller and insignificant impacts on this margin.
Panel B estimates the impact of scholarship eligibility on persistence and graduation after four,
five and six years. Three findings are worth noting. First, that the magnitude of the persistence
and various graduation rates do not vary much within classes implies that the negative impact of
scholarship eligibility on graduation rates is driven largely
by dropout rather than delay. Second,
that the negative graduation effect is not driven solely by the first high school class makes much
less likely the possibility that the effect was generated by confusion about the meaning of “free tuition” in the scholarship letter. If such language was deceiving students into making uninformed
decisions, we would expect such negative graduation effects to diminish across classes as information about the true value of the scholarship spread. There is no clear evidence of such a pattern.
Third, estimated impacts on enrollment and persistence rates generated by the full 2005-08sample
are similar to those generated by the 2005-06 sample. Figures A.11 and A.12 confirm this, replicating Figures 5 and 6 for the larger sample.
The two sets of figures look quite similar. As a whole,
this evidence suggests a fairly stable impact of the scholarship on enrollment, persistence and
graduation.
Table A.3 tests the robustness of our central results to a variety of alternative specifications.
Panel A replicates our default local linear regression specification for a variety of bandwidths,
beginning with the Imbens-Kalyaramanan optimal bandwidth. In the 2005-06 sample, that optimal bandwidth is about 14 for enrollment outcomes and about 10 for persistence and graduation
outcomes, hence our choice of 12 as the default bandwidth in earlier tables. The magnitude and
statistical significant of these estimates are generally quite robust to these changes in the bandwidth.
Panel B replicates our default specification, using a bandwidth of 12, with three variations.
First, we include school district by high school class fixed effects to account for the fact that the
18
eligibility threshold differs by district and class. This has very little impact on the estimates. Second, we include demographic controls, which also change the estimates very little. Third, we run
a doughnut hole regression in which we exclude students exactly on the boundary, because of the
small amount of bunching observed in the running variable. This actually increases the magnitude of our enrollment and graduation estimates by roughly a percentage point, suggesting that
the mild bunching was, if anything, causing us to underestimate the impacts of the scholarship.
Finally, panel C fits quadratic, cubic and quartic polynomials on either side of the threshold, using
the entire sample and a rectangular kernel. This yields similar estimates to those generated by the
local linear regression used as our default specification.
As a final piece of evidence, Figure 7 exploits as a placebo test data from the high school class
of 2004, the one cohort in our data that graduated prior to the scholarship’s existence. In panel A,
we see no evidence of a discontinuity in Adams college enrollment for the class of 2004, compared
to the previously observed clear discontinuity for the classes of 2005-06. Similarly, panel B shows
that students below the threshold have similar six-year graduation rates across the three classes,
whereas students above the threshold in 2005-06 have lower rates than such students in 2004. That
the discontinuities in enrollment and graduation appear only in the years when the scholarship
existed strengthens the case that it is due to the policy itself and not other unobserved factors.
Panel A also highlights that the magnitude of the enrollment impact is large even for students
somewhat far from the threshold. Our regression discontinuity estimates, as well as those based
on difference-in-difference calculations following Figure 7,
suggest that the Adams Scholarship
induced about 1,000 additional students to enroll in in-state public colleges. IPEDS data reported
by Massachusetts’ public colleges themselves confirms this. Figure 8 plots the reported freshman
enrollment across all Massachusetts public four-year colleges, both for all students and for those
from Massachusetts. There is a clear trend break in 2005, when the Adams Scholarship begins,
due entirely to increased numbers of Massachusetts freshman and of magnitude nearly identical
to our estimate. This implies that the additional students induced into in-state public colleges
did not crowd out other students, instead simply adding to each campus at most a few hundred
students who would not otherwise have enrolled there.
