Field Experiments in Economics
-- RCT / Randomized Control Trials / Randomize Experiments --
BPV_IEBE_2021
April 29, 2021
Tommaso Reggiani
tommaso.reggiani@econ.muni.cz
Main References
--- Methodology ---
• List, J. A. (2007). Field experiments: a bridge between lab and naturally occurring data. The BE Journal of Economic
Analysis & Policy, 6(2).
• List, J. A. (2011). Why economists should conduct field experiments and 14 tips for pulling one off. Journal of Economic
Perspectives, 25(3), 3-16.
--- application: Labour Economics ---
• Falk, A., & Ichino, A. (2006). Clean evidence on peer effects. Journal of Labor Economics, 24(1), 39-57.
• Heinz, M., Jeworrek, S., Mertins, V., Schumacher, H., & Sutter, M. (2020). Measuring the Indirect Effects of Adverse
Employer Behaviour on Worker Productivity: a field Experiment. The Economic Journal, 130(632), 2546-2568.
--- application: Public Policy ---
• Giulietti, C., Tonin, M., & Vlassopoulos, M. (2019). Racial discrimination in local public services: A field experiment in
the United States. Journal of the European Economic Association, 17(1), 165-204.
• Bott, K. M., Cappelen, A. W., Sørensen, E. Ø., & Tungodden, B. (2020). You’ve got mail: A randomized field experiment
on tax evasion. Management Science, 66(7), 2801-2819.
Experimental Research
• Lab research
• Clinical Trials
• Confirmatory Studies
• Lab Experiments
• Field Experiments
• “Natural” Experiment
→ Causality
- Diff-in-Diff
- IV
- Matching
→ Medicine → Economics / Behavioral Science
Social
Incentives
in the
Workplace
Lab “style” experiments
- lab-lab (e.g. dictator game) / online-exp
- framed (e.g. taxation experiments)
- artefactual (salient pool of subjects)
- lab in the field (conflict areas)
→ Incentive compatible settings
Subjects know they are involved in a study
(experimental demand effect)
→Standard informed consent form
Field experiments
- large scale (thousands of participants)
- small scale (hundreds of participants)
- organisations (e.g. uni) / firms / communities
Subjects are not aware they are involved in a study
(“ecological” behaviour)
→IRB ethical approval & pre-registration
→ research partner from the “real world”
• John LIST (U. Chicago) - reading list -
Control → External Validity
• Methodological trade-off
Control: lab experiments > field experiments > “natural” experiments
External validity: “natural” experiments >(=) field exp > lab exp
→The specific research question remains the most crucial factor
• Costs
• Temporal dimension
• Practical feasibility / Data availability
• Research partnerships
“A/B” between- vs within- subject design
betweendifferent
subjects/groups are exposed to different treatments
withinsame
subjects are exposed to different treatment (temporal dimension)
→ increase in “efficiency”: statistical power, reduced heterogeneity on
unobservable, budget optimization, ordering concerns
case #1: Peer-Effects
- Do we exert more effort when our colleagues are highly productive?
Lab:
Van Veldhuizen, R., Oosterbeek, H., & Sonnemans, J. (2018).
“Peers at work: Evidence from the lab”. PloS one, 13(2), e0192038.
Task: solving problems
Treatments: different peer-monitoring settings
→result: positive effect of peer-effect on performance
→positive aspect: full control over context variables
→limitation: very (!) unnatural setting - limited external validity –
“Natural”:
Mas, A., & Moretti, E. (2009). “Peers at work”.
American Economic Review, 99(1), 112-45.
Treatments: “pseudo” treatments
→result: positive effect of peer-effect
→positive aspect: long micro-panel dataset
→limitation: strong statistical assumptions
mix of advanced methods
demanding data scouting
Field:
Falk, A., & Ichino, A. (2006). “Clean evidence on peer effects”.
Journal of Labor Economics,24(1), 39-57.
Task: filling envelops
Treatments: different peer-monitoring settings
Treatments: Individual / Multiple / Peer
→result: positive effect of peer-effect
→positive aspect: easy and clean / small scale
→limitation: no employment “relationship”
case #2: Lay-off
- Do workers decrease their effort when their colleagues have been
dismissed?
Lab:
Drzensky, F., & Heinz, M. (2016). “The hidden costs of downsizing”
The Economic Journal, 126(598), 2324-2341.
Treatments: control vs lay-off option
→result: negative effect of lay-off policies (-43%)
→limitation: very stylised setting and task
“Natural”
Akerlof, R., Ashraf, A., Macchiavello, R., & Rabbani, A. (2020).
Layoffs and productivity at a Bangladeshi sweater factory.
CESifo WP 8492
Treatments: “pseudo” treatments
→result: sizable negative effect of lay-off measures
→positive aspects: more than 400 workers
daily data > 1 year
→downsides: data scouting
statistical assumptions
data quality
Field:
Heinz, M., et al. (2020). “Measuring the Indirect Effects of Adverse Employer
Behaviour on Worker Productivity: a field Experiment”.
The Economic Journal, 130(632), 2546-2568.
Treatments: control vs lay-off round
→result: negative effect of lay-off policies (-12%)
→positive aspects: easy and clean / realistic
→downsides: expensive, organizational costs
2-day job contract
RACIAL DISCRIMINATION IN LOCAL PUBLIC
SERVICES: A FIELD EXPERIMENT IN THE
UNITED STATES
Giuglietti C., Tonin M., Vlassopoulos M. (2019)
Journal of the European Economic Association
2019 17(1):165–204 DOI: 10.1093/jeea/jvx045
Research question
• Do public officials discriminate (by race) when providing an
online public service?
• Field experiment (correspondence / audit study)
• USA, 2015
- Control group:
White citizens
- Treatment group:
Black citizens
Boot et al. (2019)
.
“You’ve got mail:
A randomized field experiment on tax evasion”
.
Management Science
https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2019.3390
::: SURGERY ::::
Issue
- How can policymakers fight tax evasion?
- Hard policy (“rational approach”)→ detection, inspections, penalties
...for politicians, it’s an expensive and unpopular option
- Soft policy (“moral/psychological approach”)→ stressing moral concerns related to
public contributions
…cost-free intervention, it does not hurt the electorate
→Which strategy is more effective?
…standard economics would tell us that economic agents are more reactive
to detection and penalties than to moral (cheap) appeals.
Empirical test / field experiment
- country: Norway, 2013
- more than 15,000 sampled taxpayers
- goal: increase the volume of -foreign- income declarations
- randomly selected into 4 main groups (see Tab 5.)
- No Letter – no tax reminder was sent to taxpayers in this group
- Base – a very plain tax reminder was sent to taxpayers in this group
- Moral – a tax reminder with a moral appeal was sent to taxpayers in this group
- Detection – a tax reminder with penalty threat was sent to taxpayers in this group
Treatment variations
Treatment groups were comparable on salient
characteristics (randomization)
Results (Extensive margin VS Intensive margin)
Extensive
margin
Intensive
margin
More taxpayers submitted the tax forms under DETECTION …but taxpayers under MORAL reported on average higher income
Total Effect (extensive + intensive)
On the one hand, DETECTION and MORAL are
able to sort similar positive effects in terms of
total public income generation (approx. +8,000
NOK per taxpayer).
On the other hand, MORAL is the configuration
that is (i) more cost effective -no need to set up a
detection system- (ii) more accepted by the
electorate -no punishment measures/fines put in
place by the policy makers-.
-----------------------------------------------------------
While DETECTION generates a positive effect also
on the long-run (subsequent year), that’s not the
case for the MORAL intervention [result #5].
Why does MORAL work?
Possible explanations:
i) Morality increases the salience of the call for contribution to the
public good.
ii) Morality changes the mood of the tax payer inducing pro-sociality
(this could explain why MORAL generate effects only in the short
run and not in the long run).
Real world Experiment