Synthetic controls method
Lukˊaˇs Laffˊers
Matej Bel University, Dept. of Mathematics
MUNI Brno
3.12.2021 7.1.2022
In many situations the treatment happens on an aggregate level (city,
state).
We may not have a natural unit to use as a control
We create it artificially (hence synthetic) by weighting other units so
that the characteristics of the weighted unit resembles the one of the
treated unit
Example: Tabacco control program and cigarettes sales
Abadie, Diamond and Hainmueller (2010)
Example: The economic cost of a conflict
Abadie and Gardezabal (2003)
Example: Reunification of Germany and Economic growth
Abadie, Diamond and Hainmueller (2015)
time 1,2,...,T
J +1 units, 1 is treated in T0 +1,...,T
Synthetic control is a weighted average of the J control units.
(w2,...,wJ+1) with wj ≥ 0,∑J+1
j=2 wj = 1
Weights w∗
j are chosen optimally to make the synthetic control similar
to the control one in observed characteristics.
Synthetic control estimator is
ˆτ1t = Y1t −
J+1
∑
j=2
w∗
j ·Yjt
Choosing the weights
What does optimally mean?
We need some metric. Assume k variables X1,...,Xk . E.g. we can choose
weighted Euclidean metric.
Pre-intervention outcomes are also included in the set of predictors!
Larger weights on more important predictors.
argmin
w
k
∑
h=1
vh · Xh1 −
J+1
∑
j=2
wh ·Xhj
2
Assuming a linear factor model: If you manage to match controls and
outcome in the pre-treatment periods (T = 1,...,T0) then you can bound the
bias of the synthetic control method (Abadie, Diamond, and Hainmueller
2010).
Example: Tabacco again
Abadie, Diamond and Hainmueller (2010)
Example: Tabacco again
Abadie, Diamond and Hainmueller (2010)
Weights
Ysynth,t = 0.164YColorado,t +0.069YConnecticut,t +0.1999YMontana,t +
0.234YNevada,t +0.334YUtah,t
ˆτCalifornia,t
effect
= YCalifornia,t
real outcome
− Ysynth,t
synthetic control
Balance
Abadie, Diamond and Hainmueller (2010)
Inference
Use permutation method.
Consider every control as a ”fake” treatment and estimate placebo
effect
Compare the effect for treated unit with those placebo effects
Effect for the treated should be much larger than the placebo units
But the pre-treatment fits may be different for different control units
Abadie et al. (2010) suggests to look a the distribution of ratio of post
vs pre-treatment fit
Yes, we look at the whole distribution, not only p-values.
Placebos
Abadie, Diamond and Hainmueller (2010)
Placebos
Abadie, Diamond and Hainmueller (2010)
Inference
Abadie, Diamond and Hainmueller (2010)
If the fit is poor in the pre-intervention period. Do not do SCM, do
something else.
Small T0 and large J → risk of overfitting
Homogenise your pool of potential controls. Make them similar to the
control unit.
Again make comparison more plausible.
But why not regression instead?
Predictors X0 (with intercept) are used to predict y0,t (post intervention
outcomes for J control units at time t ∈ T0 +1,...,T):
ˆβOLS,t = (XT
0 X0)−1
XT
0 y0,t
X1
1×K
ˆβOLS,t
K×1
= X1(XT
0 X0)−1
XT
0
wT
≡ OLS weights
y0,t = wT
1×J
y0,t
J×1
Let us denote Y0 = y0,T0+1 y0,T0+2 ··· y0,T which is J ×(T −T0) matrix.
ˆBOLS
K×(T−T0)
= ( XT
0
K×J
X0
J×K
)−1
XT
0
K×J
Y0
J×(T−T0)
X1
1×K
ˆBOLS
K×(T−T0)
= X1(XT
0 X0)−1
XT
0
wT
≡ OLS weights
Y0 = wT
1×J
Y0
J×(T−T0)
But why not regression instead?
Abadie (2021)
From OLS we have also weights (!)
May be negative → difficult to interpret
OLS weights are not sparse
Sparsity is nice for interpretation
Sparsity?
Abadie (2021)
Induce sparsity (penalized estimator)
We may induce the sparsity, so penalize for large differences.
argmin
w


k
∑
h=1
vh · Xh1 −
J+1
∑
j=2
wh ·Xhj
2


1
2
Regular SCM
+λ
J+1
∑
j=2
wh
k
∑
h=1
vh ·(Xh1 −Xhj)2
1
2
Penalty for non-sparse solution
We are in between the two extreme cases:
λ → 0 - synthetic control method
λ → ∞ - nearest neighbor matching
Alberto Abadie on DAGs
”Synthetic controls,... like in any other method for causal inference, what you
won’t be able to do is to whisper a question in a microphone to a computer and
DAG will produce the answer for you. You have to make design decisions about
what is a good comparison and what is not. And that’s the case here too.”
(Abadie in https://www.youtube.com/watch?v=nKzNp-qpE-I (from 59:50))
Advantages
No extrapolation is made
The weights make it transparent
We know exactly how much each control unit contributes
Weights are non-negative (unlike for OLS)
You can fix the weights before the change has occurred.
Thus you avoid specification fishing.
You don’t need many units, but the right units
You are relatively close to the data → the method is simple
We keep getting back to the most important question:
What do we need to do in order to have a
meaningful comparison?
What do many of these methods (RDD, DiD, SCM) have in common??
[dramatic pause]
They are very visual.
Professional graphics sells. Make sure to produce beautiful graphs. (See the
works of Jonathan Schwabish on how to make great visualizations).
Schwabish, Jonathan A. ”An economist’s guide to visualizing data.” Journal of Economic Perspectives 28.1 (2014): 209-34.
Schwabish, Jonathan. Better presentations. Columbia University Press, 2016.
Schwabish, Jonathan. Better Data Visualizations: A Guide for Scholars, Researchers, and Wonks. Columbia University Press, 2021.
Synthetic controls and experimentation
What is the impact of a new policy?
We can only experiment on larger units (say cities).
We choose some units (cities) and weight them to construct synthetic
treatment unit, that resembles the population of interest.
Construct synthetic control unit for this synthetic treatment unit
And compare them. Yes, that’s it.
This has been used in the industry for a longer time.
Abadie and Zhao (2021) worked out the math.
SCM is new
It is very popular and constantly getting more traction
Much will be done in the next few years
It became a standard in econometrics toolbox
Thank you for your attention!
References
Original paper: Abadie, Alberto, and Javier Gardeazabal. ”The economic costs of conflict: A case study of the Basque Country.” American economic review 93.1
(2003): 113-132.
Paper where theory is worked out: Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. ”Synthetic control methods for comparative case studies: Estimating the
effect of California’s tobacco control program.” Journal of the American statistical Association 105.490 (2010): 493-505.
Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. ”Comparative politics and the synthetic control method.” American Journal of Political Science 59.2 (2015):
495-510.
Recent review article: Abadie, Alberto. ”Using synthetic controls: Feasibility, data requirements, and methodological aspects.” Journal of Economic Literature 59.2
(2021): 391-425.
Instructive video by inventor of SCM himself https://www.youtube.com/watch?v=nKzNp-qpE-I
Similar, slightly longer video also by Abadie at 2021 NBER Summer Institute lecture series https://www.youtube.com/watch?v=T2p9Wg650bY
Abadie, Alberto, and Jinglong Zhao. ”Synthetic controls for experimental design.” arXiv preprint arXiv:2108.02196 (2021).
Chapter 10 in S.Cunningham’s book: https://mixtape.scunning.com/synthetic-control.html
It is not often that WSJ writes about econometric methods:
https://www.wsj.com/articles/how-an-analysis-of-basque-terrorism-helps-economists-understand-brexit-1541587068