Regression Discontinuity Design
Lukˊaˇs Laffˊers
Matej Bel University, Dept. of Mathematics
MUNI Brno
2.12.2021 6.1.2022
Nature does not make jumps.
(Natura non facit saltus)
Some things do not occur naturally and requires some explanation.
Example: Campbell (1960)
”Certificate of merit” is given to students above some threshold in an
aptitude test.
High-scoring students went for ”National merit scholarship” which is
not included on the vertical axis =⇒ downward slope.
Students right below and right above the thresholds are similar.
It is as if they were assigned to group below and the group above randomly.
Sharp RDD
6
7
8
9
10
0 100 200
Test score
Grade
Regression Discontinuity Design Demonstration
6
7
8
9
10
0 100 200
Test score
Grade
Regression Discontinuity Design Demonstration
6
7
8
9
10
0 100 200
Test score
Grade
Regression Discontinuity Design Demonstration
Sharp RDD
ˆδ = lim
z↓0
ˆE[Y|Z = z]
treated
−lim
z↑0
ˆE[Y|Z = z]
control
Random variation is coming from the real world constraints/rules
Treatment D = 1 or D = 0 if defined in terms of running variable Z:
D = 1 ⇐⇒ Z > 0
Sharp RDD
In the previous example we used linear model.
Yet in practice we have only seldom reasons to believe that such model
is correct
We wish to have a sufficiently flexible model.
Too flexible?
5
6
7
8
9
10
0 100 200
Test score
Grade
RDD Demonstration (5th degree polynomial)
Sensitive? (add 2 points)
5
6
7
8
9
10
0 100 200
Test score
Grade
RDD Demonstration (5th degree polynomial) + 2 obs
Sharp RDD
It is the threshold that is important
This is where all the action takes place
You have to defend that there is no manipulation around the threshold
It’s all about the threshold
6
7
8
9
10
100 110 120 130 140
Test score
Grade
RDD Demonstration (Zoomed)
7.0
7.5
8.0
8.5
110 120 130 140
Test score
Grade
RDD Demonstration (linear)
6.5
7.0
7.5
8.0
8.5
9.0
110 120 130 140
Test score
Grade
RDD Demonstration (quadratic)
7
8
9
110 120 130 140
Test score
Grade
RDD Demonstration (cubic)
6
7
8
9
110 120 130 140
Test score
Grade
RDD Demonstration (quartic)
Sharp RDD
But how do we choose the model for ˆE[Y|Z = z]
on the left and on the right?
Clearly, there are many many ways how we can do this!
Gelman and Imbens (2019) warns against the higher order polynomials.
Obviously: you can cook up the results according to your liking.
And that is always a bad thing.
Just look at the title: ”Why high-order polynomials should not be used
in regression discontinuity designs.”
Gelman and Imbens (2019)
Issue 1: Some observations are given excessive weights → Check the
weights.
Issue 2: Results are sensitive to the degree of polynomial (see below) →
Use local regression.
Issue 3: Confidence intervals are too narrow. → Global regressions are
simply not precise enough at the cut-off point, use local regression
instead.
Local linear regression (loess)
ˆβ0(x), ˆβ1(x) = arg min
β0,β1
n
∑
i=1
Sum of
K
xi −x
h
weighted
·(yi −(β0 +β1xi))2
squared errors
ˆE[Y|X = x] = ˆβ0(x)+ ˆβ1(x)·x
Kernel functions:
0.00
0.25
0.50
0.75
1.00
−2 −1 0 1 2
x
Gaussian
0.00
0.25
0.50
0.75
1.00
−2 −1 0 1 2
x
Uniform
0.00
0.25
0.50
0.75
1.00
−2 −1 0 1 2
x
Triangular
0.0
0.2
0.4
0.6
−2 −1 0 1 2
x
Epanechnikov
Bandwidth h: bias vs variance tradeoff.
In practice, the choice of the bandwidth is more important than the
choice of the kernel (weighting function)
Local linear regression methods (and other semi/non-parametrics
methods) have problems at the boundary, fewer points → more bias.
There are different ways (fixes) how to deal with this (Noack and Rothe,
2021).
If the running variable is discrete, one should account for it (Kolesar and
Rothe, 2018)
Example: Mask mandates
c - county
s - state
t - week
k lag (k = 1 for cases, k = 2 for hospital admissions, k = 4 for deaths)
maskst - mask mandate in state s at time t
distcst - minimum distance of country c is state s to a county with a
different mask policy at time t
Xc,s,t - controls using lagged mobility
Discontinuity?
Niels-Jakob H Hansen ; Rui C. Mano: ”Mask Mandates Save Lives.” IMF Working Papers 2021.205 (2021).
Results
Niels-Jakob H Hansen ; Rui C. Mano: ”Mask Mandates Save Lives.” IMF Working Papers 2021.205 (2021).
Fuzzy RDD
Random variation is coming from the real world constraints/rules
Treatment probability Pr(D = 1|Z = z) here discontinuously changes at
the cut-off value Z = 0 (it is just an IV!!!)
Very specific subpopulation - those marginal people at the cut-off
(compliers)
The question is: How do we extrapolate??
Fuzzy RDD as an IV
Wald estimator:
ˆβWald =
ˆE[Y|Z = 1]− ˆE[Y|Z = 0]
ˆE[D|Z = 1]− ˆE[D|Z = 0]
Or
ˆβIV = (XT
PZ X)−1
XT
PZ y
where X = [1,X,D] Z = [1,X,Z] yi = Xiβ +ei
Example: Hoekstra (2009)
log(earnings) = ψyear +φexperience +θcohort +ε
We have exogenous variation in the prob. of enrollment rate. What is the
effect on earnings 15 years later?
How to quantify the effect?
We wish to estimate this:
Outcome = β0 +β1(Above)+γ1(
Some flexible function
h(AdjustedSATscore
Cut-off set to 0.
))+γ2GPA+γ3(SAT Score)+ε
But taking into account variation in year of admission, experience and
cohort. Hoekstra did it in two steps:
Step 1 For every individual, regress outcome and independent variables on
ψyear , φexperience,θcohort and obtain residuals. Average these residuals
across different years for every single individual.
Step 2 Plug in residuals from these regressions in this equation and estimate it
via OLS.
Step 3 Scale the estimate of coefficient of interest ˆβ1 by the size of the
discontinuous jump.
Two stage estimator. Standard errors? Bootstrap.
Table 1 in Hoekstra (2009)
Three well-known papers
Angrist and Lavy (1999) - Maimonides rule - what is the effect of
classroom size on academic achievements. Classrooms cannot have
more than 40 students.
Black (1999) - used impact of school districts on the house prices.
Houses at the district boundaries, similar but differ in terms of
elementary school that the child attends. ”...parents are willing to pay
2.5 percent more for a 5 percent increase in test scores”
Van Der Klaauw (2002) - effect of financial aid on student enrollments.
Exploiting the discontinuity in financial aid eligibility in student’s test
performance.
RDD
Only popular since 2000s.
Visually very appealing. The details matters. Work hard on compelling
figures, they do matter a lot.
You should not see a jump in the density of a running variable around
the threshold - it may suggest some manipulation.
There seems to be many choices to consider: estimation, bandwidth,
kernel. Stick to defaults unless you have very good reasons to not to.
There are many many situations with cut-offs.
RDD
Data hungry - we need many points around the threshold to provide
credible evidence on the jump.
Cunningham (2021) suggests you to work on relationships with people
who have access to this kind of data. These are built on trust and
honesty.
Korting et al. (2020) - people are quite good at spotting the
discontinuities - they conducted some tests.
Thank you for your attention!
References
Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology,
51(6), 309–317. https://doi.org/10.1037/h0044319
Angrist, Joshua D., and Victor Lavy. ”Using Maimonides’ rule to estimate the effect of class size on scholastic achievement.” The Quarterly journal of economics 114.2
(1999): 533-575.
Black, Sandra E. ”Do better schools matter? Parental valuation of elementary education.” The quarterly journal of economics 114.2 (1999): 577-599.
Van der Klaauw, Wilbert. ”Estimating the effect of financial aid offers on college enrollment: A regression–discontinuity approach.” International Economic Review
43.4 (2002): 1249-1287.
Gelman, Andrew, and Guido Imbens. ”Why high-order polynomials should not be used in regression discontinuity designs.” Journal of Business & Economic Statistics
37.3 (2019): 447-456.
Niels-Jakob H Hansen ; Rui C. Mano: ”Mask Mandates Save Lives.” IMF Working Papers 2021.205 (2021).
Noack, Claudia, and Christoph Rothe. Bias-Aware Inference in Fuzzy Regression Discontinuity Designs. arXiv. org, 2021.
Kolesˊar, Michal, and Christoph Rothe. ”Inference in regression discontinuity designs with a discrete running variable.” American Economic Review 108.8 (2018):
2277-2304.
Korting, Christina, et al. ”Visual Inference and Graphical Representation in Regression Discontinuity Designs.” (2020).
The chapter in Cunningham, Scott. Causal Inference. Yale University Press, 2021 is very engaging and fun to read.