The Welfare Eﬀects of Social Media
Hunt Allcott, Luca Braghieri, Sarah Eichmeyer, and Matthew Gentzkow∗
November 8, 2019
Abstract
The rise of social media has provoked both optimism about potential societal beneﬁts and
concern about harms such as addiction, depression, and political polarization. In a randomized
experiment, we ﬁnd that deactivating Facebook for the four weeks before the 2018 US midterm
election (i) reduced online activity, while increasing oﬄine activities such as watching TV alone
and socializing with family and friends; (ii) reduced both factual news knowledge and political
polarization; (iii) increased subjective well-being; and (iv) caused a large persistent reduction in
post-experiment Facebook use. Deactivation reduced post-experiment valuations of Facebook,
suggesting that traditional metrics may overstate consumer surplus.
JEL Codes: D12, D90, I31, L86, O33.
Keywords: Social media, political polarization, subjective well-being, consumer surplus
from digital technologies.
———————————————————————————
∗
Allcott: New York University and NBER. hunt.allcott@nyu.edu. Braghieri: Stanford University. lucabrag@stanford.edu.
Eichmeyer: Stanford University. saraeich@stanford.edu. Gentzkow: Stanford University and
NBER. gentzkow@stanford.edu. We thank Nancy Baym, Moira Burke, Annie Franco, Alex Leavitt, Todd Rogers, Joel
Waldfogel, and seminar participants at Berkeley, Centro de Investigaci´on y Docencia Econ´omicas, Chicago, Columbia,
Instituto Tecnol´ogico Aut´onomo de M´exico, Microsoft, NBER Digitization, NYU, Stanford, the Technology Policy
Institute, UCLA, UC Santa Cruz, and University of Washington for helpful comments. We thank Raj Bhargava,
Zong Huang, and Kelly B. Wagman for exceptional research assistance. We are grateful to the Sloan Foundation and
the Knight Foundation for generous support. The study was approved by Institutional Review Boards at Stanford
(eProtocol #45403) and NYU (IRB-FY2018-2139). This RCT was registered in the American Economic Association
Registry for randomized control trials under trial number AEARCTR-0003409. Replication ﬁles and survey instruments
are available from https://sites.google.com/site/allcott/research. Disclosures: Allcott is a paid employee of
Microsoft Research. Gentzkow does paid consulting work for Amazon and is a member of the Toulouse Network for
Information Technology, a research group funded by Microsoft. Braghieri and Eichmeyer have no relevant or material
disclosures.
1
1 Introduction
Social media have had profound impacts on the modern world. Facebook, which remains by far
the largest social media company, has 2.3 billion monthly active users worldwide (Facebook 2018).
As of 2016, the average user was spending 50 minutes per day on Facebook and its sister platforms
Instagram and Messenger (Facebook 2016). There may be no technology since television that has
so dramatically reshaped the way people get information and spend their time.
Speculation about social media’s welfare impact has followed a familiar trajectory, with early
optimism about potential beneﬁts giving way to widespread concern about possible harms. At a
basic level, social media dramatically reduce the cost of connecting, communicating, and sharing
information with others. Given that interpersonal connections are among the most important
drivers of happiness and well-being (Myers 2000; Reis, Collins, and Berscheid 2000; Argyle 2001;
Chopik 2017), this could be expected to bring widespread improvements to individual welfare.
Many have also pointed to wider social beneﬁts, from facilitating protest and resistance in autocratic
countries, to encouraging activism and political participation in established democracies (Howard
et al. 2011; Kirkpatrick 2011).
More recent discussion has focused on an array of possible negative impacts. At the individual
level, many have pointed to negative correlations between intensive social media use and both
subjective well-being and mental health.1 Adverse outcomes such as suicide and depression appear
to have risen sharply over the same period that the use of smartphones and social media has
expanded.2 Alter (2018) and Newport (2019), along with other academics and prominent Silicon
Valley executives in the “time well-spent” movement, argue that digital media devices and social
media apps are harmful and addictive. At the broader social level, concern has focused particularly
on a range of negative political externalities. Social media may create ideological “echo chambers”
among like-minded friend groups, thereby increasing political polarization (Sunstein 2001, 2017;
Settle 2018). Furthermore, social media are the primary channel through which misinformation
spreads online (Allcott and Gentzkow 2017), and there is concern that coordinated disinformation
campaigns can aﬀect elections in the US and abroad.
In this paper, we report on a large-scale randomized evaluation of the welfare impacts of Facebook,
focusing on US users in the run-up to the November 2018 midterm elections. We recruited a
sample of 2,743 users through Facebook display ads, and elicited their willingness-to-accept (WTA)
to deactivate their Facebook accounts for a period of four weeks ending just after the election. We
then randomly assigned the 61 percent of these subjects with WTA less than $102 to either a Treatment
group that was paid to deactivate, or a Control group that was not. We veriﬁed compliance
1
See, for example, Abeele et al. (2018), Burke and Kraut (2016), Ellison, Steinﬁeld, and Lampe (2007), Frison and
Eggermont (2015), Kross et al. (2013), Satici and Uysal (2015), Shakya and Christakis (2017), and Tandoc, Ferrucci,
and Duﬀy (2015). See Appel, Gerlach, and Crusius (2016) and Baker and Algorta (2016) for reviews.
2
See, for example, Twenge, Sherman, and Lyubomirsky (2016), Twenge and Park (2017), Twenge, Martin, and
Campbell (2018), and Twenge et al. (2018).
2
with deactivation by regularly checking participants’ public proﬁle pages. We measured a suite
of outcomes using text messages, surveys, emails, direct measurement of Facebook and Twitter
activity, and administrative voting records. Less than two percent of the sample failed to complete
the endline survey, and the Treatment group’s compliance with deactivation exceeded 90 percent.
Our study oﬀers the largest-scale experimental evidence available to date on the way Facebook
aﬀects a range of individual and social welfare measures. We evaluate the extent to which time on
Facebook substitutes for alternative online and oﬄine activities, with particular attention to crowd
out of news consumption and face-to-face social interactions. We study Facebook’s broader political
externalities via measures of news knowledge, awareness of misinformation, political engagement,
and political polarization. We study the impact on individual utility via measures of subjective wellbeing,
captured through both surveys and text messages. Finally, we analyze the extent to which
forces like addiction, learning, and projection bias may cause sub-optimal consumption choices, by
looking at how usage and valuation of Facebook change after the experiment.
Our ﬁrst set of results focuses on substitution patterns. A key mechanism for eﬀects on individual
well-being would be if social media use crowds out face-to-face social interactions and thus
deepens loneliness and depression (Twenge 2017). A key mechanism for political externalities would
be if social media crowds out consumption of higher-quality news and information sources. We ﬁnd
evidence consistent with the ﬁrst of these but not the second. Deactivating Facebook freed up 60
minutes per day for the average person in our Treatment group. The Treatment group actually
spent less time on both non-Facebook social media and other online activities, while devoting more
time to a range of oﬄine activities such as watching television alone and spending time with friends
and family. The Treatment group did not change its consumption of any other online or oﬄine
news sources and reported spending 15 percent less time consuming news.
Our second set of results focuses on political externalities, proxied by news knowledge, political
engagement, and political polarization. Consistent with the reported reduction in news consumption,
we ﬁnd that Facebook deactivation signiﬁcantly reduced news knowledge and attention to
politics. The Treatment group was less likely to say they follow news about politics or the President,
and less able to correctly answer factual questions about recent news events. Our overall
index of news knowledge fell by 0.19 standard deviations. There is no detectable eﬀect on political
engagement, as measured by voter turnout in the midterm election and the likelihood of clicking
on email links to support political causes. Deactivation signiﬁcantly reduced polarization of views
on policy issues and a measure of exposure to polarizing news. Deactivation did not statistically
signiﬁcantly reduce aﬀective polarization (i.e. negative feelings about the other political party) or
polarization in factual beliefs about current events, although the coeﬃcient estimates also point in
that direction. Our overall index of political polarization fell by 0.16 standard deviations. As a
point of comparison, prior work has found that a diﬀerent index of political polarization rose by
0.38 standard deviations between 1996 and 2018 (Boxell 2018).
3
Our third set of results looks at subjective well-being. Deactivation caused small but signiﬁcant
improvements in well-being, and in particular in self-reported happiness, life satisfaction, depression,
and anxiety. Eﬀects on subjective well-being as measured by responses to brief daily text
messages are positive but not signiﬁcant. Our overall index of subjective well-being improved by
0.09 standard deviations. As a point of comparison, this is about 25-40 percent of the eﬀect of
psychological interventions including self-help therapy, group training, and individual therapy, as
reported in a meta-analysis by Bolier et al. (2013). These results are consistent with prior studies
suggesting that Facebook may have adverse eﬀects on mental health. However, we also show that
the magnitudes of our causal eﬀects are far smaller than those we would have estimated using the
correlational approach of much prior literature. We ﬁnd little evidence to support the hypothesis
suggested by prior work that Facebook might be more beneﬁcial for “active” users—for example,
users who regularly comment on pictures and posts from friends and family instead of just scrolling
through their news feeds.3
Our fourth set of results considers whether deactivation aﬀected people’s demand for Facebook
after the study was over, as well as their opinions about Facebook’s role in society. As the experiment
ended, participants reported planning to use Facebook much less in the future. Several weeks
later, the Treatment group’s reported usage of the Facebook mobile app was about 11 minutes (22
percent) lower than in Control. The Treatment group was more likely to click on a post-experiment
email providing information about tools to limit social media usage, and ﬁve percent of the Treatment
group still had their accounts deactivated nine weeks after the experiment ended. Our overall
index of post-experiment Facebook use is 0.61 standard deviations lower in Treatment than in Control.
In response to open-answer questions several weeks after the experiment ended, the Treatment
group was more likely to report that they were using Facebook less, had uninstalled the Facebook
app from their phones, and were using the platform more judiciously. Reduced post-experiment use
aligns with our ﬁnding that deactivation improved subjective well-being, and it is also consistent
with the hypotheses that Facebook is habit forming in the sense of Becker and Murphy (1988) or
that people learned that they enjoy life without Facebook more than they had anticipated.
Deactivation caused people to appreciate Facebook’s both positive and negative impacts on their
lives. Consistent with our results on news knowledge, the Treatment group was more likely to agree
that Facebook helps people to follow the news. About 80 percent of the Treatment group agreed
that deactivation was good for them, but they were also more likely to think that people would
miss Facebook if they used it less. In free response questions, the Treatment group wrote more text
about how Facebook has both positive and negative impacts on their lives. The opposing eﬀects
on these speciﬁc metrics cancel out, so our overall index of opinions about Facebook is unaﬀected.
Our work also speaks to an adjacent set of questions around how to measure the economic
3
Correlation studies on active vs. passive Facebook use include Burke, Marlow, and Lento (2010), Burke, Kraut,
and Marlow (2011), Burke and Kraut (2014), and Krasnova et al. (2013), and randomized experiments include Deters
and Mehl (2012) and Verduyn et al. (2015).
4
gains from free online services such as search and media.4 In standard models with consumers who
correctly optimize their allocation of time and money, researchers can approximate the consumer
surplus from these services by measuring time use or monetary valuations, as in Brynjolfsson and Oh
(2012), Brynjolfsson, Eggers, and Gannamaneni (2018), Corrigan et al. (2018), and others. But if
users do not understand the ways in which social media could be addictive or make them unhappy,
these standard approaches could overstate consumer surplus gains. Sagioglu and Greitemeyer
(2014) provide suggestive evidence: while their participants predicted that spending 20 minutes on
Facebook would make them feel better, it actually caused them to feel worse. Organizations such
as Time to Log Oﬀ argue that a 30-day “digital detox” would help people align their social media
usage with their own best interest.
To quantify the possibility that deactivation might help the Treatment group to understand
ways in which their use had made them unhappy, we elicited willingness-to-accept at three separate
points, using incentive-compatible Becker-DeGroot-Marschak (1964, “BDM”) mechanisms. First,
on October 11th, we elicited WTA to deactivate Facebook for weeks 1-4 of the experiment, between
October 12th and November 8th. We immediately told participants the amount that they had
been oﬀered to deactivate ($102 for the Treatment group, $0 for Control), and thus whether they
were expected to deactivate over that period. We then immediately elicited WTA to deactivate
Facebook for the next four weeks after November 8th, i.e. weeks 5-8. When November 8th arrived,
we then re-elicited WTA to deactivate for weeks 5-8. The Treatment group’s change in valuation
for weeks 5-8 reﬂects a time eﬀect plus the eﬀect of deactivating Facebook. The Control group’s
parallel valuation change reﬂects only a time eﬀect. Thus, the diﬀerence between how Treatment
vs. Control change their WTAs for deactivation for weeks 5-8 reﬂects projection bias, learning, or
other unanticipated experience eﬀects from deactivation.5
After weighting our sample to match the average US Facebook user on observables, the median
and mean willingness-to-accept to deactivate Facebook for weeks 1-4 were $100 and $180, respectively.
These valuations are larger than most estimates in related work by Brynjolfsson, Eggers,
and Gannamaneni (2018), Corrigan et al. (2018), Mosquera et al. (2018), and Sunstein (2019). A
standard consumer surplus calculation would aggregate the mean valuation across the estimated
172 million US Facebook users, giving $31 billion in consumer surplus from four weeks of Facebook.
However, consistent with our other results that deactivation reduced demand for Facebook,
deactivation caused WTA for weeks 5-8 to drop by up to 14 percent. This suggests that traditional
consumer surplus metrics overstate the true welfare gains from social media, though a calculation
that adjusts for the downward WTA revision would still imply that Facebook generates enormous
4
See, for example, Brynjolfsson and Saunders (2009), Byrne, Fernald, and Reinsdorf (2016), Nakamura, Samuels,
and Soloveichik (2016), Brynjolfsson, Rock, and Syverson (2018), and Syverson (2017).
5
This measurement connects to the literature on habit formation and projection bias, including Acland and Levy
(2015), Becker and Murphy (1988), Becker, Grossman, and Murphy (1991), Busse et al. (2015), Charness and Gneezy
(2009), Conlin, O’Donoghue, and Vogelsang (2007), Fujiwara, Meng, and Vogl (2016), Gruber and K¨oszegi (2001),
Hussam et al. (2016), Loewenstein, O’Donoghue, and Rabin (2003), and Simonsohn (2010).
5
ﬂows of consumer surplus.
What do our results imply about the overall net welfare impact of Facebook? On the one hand,
Facebook deactivation increased subjective well-being, and 80 percent of the Treatment group
reported that deactivation was good for them. On the other hand, participants were unwilling to
give up Facebook unless oﬀered fairly large amounts of money—even after they had deactivated
for four weeks, which should have allowed at least some learning or “detox” from addiction. It
is not entirely clear whether one should prioritize the survey measures or monetary valuations as
normative measures of consumer welfare. Benjamin et al. (2012) suggest that subjective well-being
measures like ours are not a complete measure of what people are trying to maximize when they
make decisions, but Bohm, Lind´en, and Sonneg˚ard (1997), Mazar, Koszegi, and Ariely (2014),
and other studies make clear that monetary valuations are not closely held and can be easily
manipulated. We think of these tensions as fodder for future research.
Our results should be interpreted with caution, for several reasons. First, eﬀects could diﬀer
with the duration, time period, or scale of deactivation. A longer period without Facebook might
have less impact on news knowledge as people ﬁnd alternative news sources, and either more or less
impact on subjective well-being. Eﬀects might be diﬀerent for our pre-election deactivation than
for deactivation in other periods. Furthermore, the eﬀects of deactivating a large share of Facebook
users would likely be diﬀerent due to network eﬀects, so our parameters are most relevant for
individuals independently determining their own Facebook use. Second, our sample is not fully
representative. Our participants are relatively young, well-educated, and left-leaning compared to
the average Facebook user; we included only people who reported using Facebook more than 15
minutes per day; and people willing to participate in our experiment may also diﬀer in unobservable
ways. Third, many of our outcome variables are self-reported, adding scope for both measurement
error and experimenter demand eﬀects. However, Section 5.6 ﬁnds no evidence of demand eﬀects,
and our non-self-reported outcomes paint a similar picture to the survey responses.
The causal impacts of social media have been of great interest to researchers in economics,
psychology, and other ﬁelds. We are aware of 12 existing randomized impact evaluations of Facebook.6
The most closely related is the important paper by Mosquera et al. (2018), which was made
public the month before ours. They also use Facebook deactivation to study news knowledge and
well-being, ﬁnding results broadly consistent with those reported here. Appendix Table A1 details
these experiments in comparison to ours. Our deactivation period is substantially longer and our
sample size an order of magnitude larger than most prior experimental work, including Mosquera
6
These studies sit within a broader media eﬀects literature that uses experimental and quasi-experimental methods
to quantify the eﬀects of media technologies such as television, media providers such as Fox News, and content such
as political advertising (Bartels 1993; Besley and Burgess 2001; DellaVigna and Kaplan 2007; Enikolopov, Petrova,
and Zhuravskaya 2011; Gentzkow 2006; Gerber and Green 2000; Gerber et al. 2011; Gerber, Karlan, and Bergan
2009; Huber and Arceneaux 2007; Martin and Yurukoglu 2017; Olken 2009; and Spenkuch and Toniatti 2016); for
reviews, see DellaVigna and Gentzkow (2010), Napoli (2014), Str¨omberg (2015), Enikolopov and Petrova (2015), and
DellaVigna and La Ferrara (2015).
6
et al. (2018). We measure impacts on a relatively comprehensive range of outcomes, and we are
the only one of these randomized trials to have submitted a pre-analysis plan. Given the eﬀect
sizes and residual variance in our sample, we would have been unlikely to have suﬃcient power to
detect any eﬀects if limited to the sample sizes in previous experiments. Our work also relates to
quasi-experimental estimates of social media eﬀects by Muller and Schwarz (2018) and Enikolopov,
Makarin, and Petrova (2018).
Sections 2 through 4 present the experimental design, descriptive statistics, and empirical strategy.
Section 5 presents the impact evaluation, and Section 6 discusses measurement of the consumer
surplus generated by Facebook.
2 Experimental Design
2.1 Experiment Overview
Figure 1 summarizes our experimental design and timeline. We timed the experiment so that the
main period of Facebook deactivation would end shortly after the 2018 US midterm elections, which
took place on November 6th. The experiment has eight parts: recruitment, pre-screen, baseline
survey, midline survey, endline survey, post-endline survey, post-endline emails, and daily text
messages.
Between September 24th and October 3rd, we recruited participants using Facebook ads. Our
ad said, “Participate in an online research study about internet browsing and earn an easy $30 in
electronic gift cards.” Appendix Figure A1 presents the ad. To minimize sample selection bias, the
ad did not hint at our research questions or suggest that the study was related to social media or
Facebook deactivation. We targeted the ads by demographic cells in an attempt to gather an initial
sample that was approximately representative of Facebook users on gender, age, college completion,
and political ideology. 1,892,191 unique users were shown the ad, of whom 32,201 clicked on it.
This 1.7 percent click-through rate is about twice the average click-through rate on Facebook ads
across all industries (Irvine 2018).
Clicking on the ad took the participant to a brief pre-screen survey, which included several
background demographic questions and the consent form. 17,335 people passed the pre-screen, by
reporting being a US resident born between the years 1900 and 2000 who uses Facebook more than
15 minutes and no more than 600 minutes per day. Of those people, 7,455 consented to participate
in the study.
After completing the consent form, participants began the baseline survey. The baseline
recorded email addresses, additional demographics, and a range of outcome variables. We also
asked for each participant’s name, zip code, Twitter handle, and phone number (“in order for us
to send you text messages during the study”), as well as the URL of their Facebook proﬁle page
(which we would use “solely to observe whether your Facebook account is active”). Finally, we
7
informed people that we would later ask them to deactivate their accounts for two 24-hour periods,
and conﬁrmed their willingness to do so. (We required all participants regardless of treatment
status to deactivate for these 24-hour periods to minimize selective attrition and to ensure that the
valuations described below reﬂect value of Facebook access, not the ﬁxed cost of the deactivation
process.)
In all, 3,910 people ﬁnished the baseline survey and were willing to deactivate. Of those, 1,013
were dropped from the experiment because of invalid data (for example, invalid Facebook proﬁle
URLs) or low-quality baseline responses (for example, discrepancies between average daily Facebook
usage reported in the pre-screen vs. baseline survey, completing the survey in less than ten minutes,
no text in short-answer boxes, and other patterns suggesting careless responses). The remaining
2,897 participants had valid baseline data, were included in our stratiﬁed randomization, and were
invited to take the midline survey.
On October 11th, we sent an email invitation to the midline survey. The survey ﬁrst asked participants
to deactivate their Facebook accounts for 24 hours and guided them through the process.
The survey clearly explained what deactivation entailed and how we would monitor deactivation.
Facebook allows users to deactivate and reactivate their accounts at any time. We informed participants
that they could continue to use Facebook Messenger while deactivated, and that their proﬁle
and friend network would be unchanged when they reactivated. We emphasized that Facebook
would automatically reactivate their account if they logged into the Facebook website or app, or if
they actively logged into any other app using their Facebook login credentials.7 We informed participants
that “We will verify whether or not you deactivated your account by pinging the Facebook
URL” that they had provided in the baseline survey.
The midline survey then used a Becker-DeGroot-Marschak (BDM) mechanism to elicit willingnessto-accept
(WTA) to stay deactivated for four weeks rather than 24 hours.8 We then revealed the
BDM price oﬀer. An additional 154 participants had dropped out before this point of the midline
survey, leaving 2,743 who received their price oﬀer. Participants whose WTA was strictly less than
the price draw were informed that they should deactivate for the full four weeks after midline. Finally,
the midline survey reminded people that we would again ask them to deactivate for 24 hours
7
A user’s Facebook account automatically reactivates whenever the user actively logs into any other app using their
Facebook login credentials. However, this does not fully preclude people from using other apps for which they had
used Facebook to log in. People can continue using other apps if they are already logged in, can set up non-Facebook
logins, or can log in with Facebook and then again deactivate their Facebook account.
8
The survey explained, “The computer has randomly generated an amount of money to oﬀer you to deactivate
your Facebook account for the next 4 weeks. Before we tell you what the oﬀer is, we will ask you the smallest oﬀer
you would be willing to accept. If the oﬀer the computer generated is above the amount you give, we will ask you
to deactivate for 4 weeks and pay you the oﬀered amount if you do. If the oﬀer is below that amount, we will not
ask you to deactivate.” We then asked several comprehension questions to make sure that participants understood
the mechanism. We did not tell participants the distribution or support of the oﬀer prices, both because we did not
want to artiﬁcially truncate the distribution of elicited WTA and because prior studies have found that providing
information on the bounds of the oﬀer price distribution can aﬀect BDM valuations (Bohm, Lind´en, and Sonneg˚ard
1997; Mazar, Koszegi, and Ariely 2014).
8
after the endline survey, and used a second BDM mechanism to elicit WTA to stay deactivated for
the four weeks after endline instead of just 24 hours. We refer to the four weeks after midline as
“weeks 1-4,” and the four weeks after endline as “weeks 5-8.”
On November 8th, two days after the midterm election, we sent an email invitation to the endline
survey. The endline survey ﬁrst measured the same outcome variables as the baseline survey. All
questions were identical, with the exception of cases discussed in Section 2.3 below, such as using
updated news knowledge questions and rephrasing questions about the midterm election to be in
the past tense. We then asked all participants to again deactivate their Facebook accounts for the
next 24 hours, and again elicited WTA to stay deactivated for the next four weeks (i.e., weeks 5-8)
instead of the next 24 hours. Participants were told, “With a 50% chance we will require you to
abide by the decision you made 4 weeks ago; with 50% chance we will ignore the decision you made
4 weeks ago and we will require you to abide by the decision you make today.”
We gathered data from two post-endline emails. On November 20th, we sent an email with
links to information on ways to limit smartphone social media use, and on November 25th, we sent
an email with links to donate to, volunteer for, or sign petitions related to political causes. Clicks
on these emails provide additional non-self-reported measures of interest in reducing social media
use and political engagement. Appendix Figures A2 and A3 present the two emails.
On December 3rd, we invited participants to a short post-endline survey in which we asked how
many minutes per day they had used the Facebook app on their smartphones in the past seven days.
We asked participants with iPhones to report the Facebook app time reported by their phone’s
Settings app, and we asked other participants to estimate. We also asked several open-answer
questions, such as “How has the way you use Facebook changed, if at all, since participating in this
study?”
For the approximately six weeks between baseline and endline, we sent daily text message
surveys to measure several aspects of subjective well-being in real time rather than retrospectively.
We rotated three types of questions, measuring happiness, the primary emotion felt over the past
ten minutes, and loneliness. Appendix Figure A4 presents the three questions.
We veriﬁed deactivation by checking each participant’s Facebook proﬁle page URL regularly
at random times. While a user can limit how much content other people can see in their proﬁles,
they cannot hide their public proﬁle page, and the public proﬁle URL returns a valid response if
and only if their account is active.9 This is thus our measure of deactivation. For all participants,
9
By default, Facebook proﬁle URLs end in a unique number, which is the numeric ID for that person in the
Facebook system. Users can update their default URL to be something customized, and they can change their
customized URL as often as they want. In the baseline survey, participants reported their proﬁle URLs, which could
have been either the default or customized version. Shortly after the baseline survey, we checked if each participant’s
Facebook proﬁle URL was valid by pinging it and looking in the page source for the string containing the person’s
numeric ID. If the numeric ID existed, we knew that the URL was valid. After that point, we used participants’
numeric IDs to construct their default numeric URLs, which allowed us to correctly measure deactivation even if they
changed their customized URL.
9
we veriﬁed deactivation approximately once per day for the seven days before midline and all
days between endline and the end of January 2019. Between midline and endline, we veriﬁed
deactivation approximately four times per day for people who were supposed to be deactivated
(i.e. the Treatment group) and once every four days for everyone else. During the post-midline
and post-endline 24-hour deactivation periods, we generally veriﬁed deactivation within about six
hours of when each participant completed the survey. If participants were not deactivated when
they were supposed to be, our program immediately sent an automated email informing them that
they should again deactivate as soon as possible, along with a survey asking them to explain why
they were not deactivated.
All participants received $5 per completed survey, paid via gift card immediately upon completion.
All participants were told that they would receive a $15 “completion payment” if they
completed all surveys, responded to 75 percent of text messages, kept their accounts deactivated
for the 24 hours after midline and endline, and, if the deactivation oﬀer price was above their
reported WTA, kept their accounts deactivated for the full period between midline and endline.
The latter requirement (making the completion payment contingent on complying with the BDM’s
deactivation assignment) makes it a strictly dominant (instead of weakly dominant) strategy to
truthfully report valuations in the BDM.10 These payments were in addition to the $102 that the
Treatment group received in exchange for deactivation.
2.2 Randomization
We used the BDM mechanism described above to randomly assign participants to Facebook deactivation.
Figure 1 illustrates the randomization. Participants with valid baseline data were
randomized into three groups that determined the BDM oﬀer price p for deactivation in weeks 1-4
(i.e., the weeks between midline and endline): p = $102 (approximately 33 percent of the sample),
p = $0 (approximately 67 percent), and p drawn from a uniform distribution on [$0, $170] (approximately
0.2 percent).11 We balanced the p = $102 and p = $0 group assignments within 48
strata deﬁned by age, average daily Facebook use, heavy vs. light news use (those who get news
from Facebook fairly often or very often vs. never, hardly ever, or sometimes), active vs. passive
Facebook use, and Democrat, Republican, or independent party aﬃliation.
The eﬀects of Facebook deactivation in weeks 1-4 are identiﬁed in the sample of participants
who were allocated to p = $102 or p = $0 and were willing to accept less than $102 to deactivate
10
As discussed above, we did not inform participants of the BDM oﬀer price distribution. Thus, more precisely,
truthfully reporting valuations is a strictly dominant strategy only within the support of the oﬀer price distribution
that participants expected us to use.
11
We chose $102 because our pilot data correctly suggested that there would be a point mass of WTAs at $100
and that it would maximize statistical power per dollar of cost to set an oﬀer price just high enough to induce those
participants to deactivate. We chose $170 as the top of the uniform distribution because it was the maximum that
we could pay participants without requiring tax-related paperwork.
10
in weeks 1-4. We call this the “impact evaluation sample.” Within the impact evaluation sample,
we call p = $102 the “Treatment” group, and p = $0 the “Control” group.
For deactivation in weeks 5-8 (i.e., the four weeks after endline), 0.2 percent of participants were
randomly selected to a BDM oﬀer price drawn randomly from p ∈ [0, 170], while the remaining
99.8 percent received oﬀer p = 0. We balanced this weeks 5-8 oﬀer price p between the weeks 1-4
oﬀer price groups, so two participants who were oﬀered p = $102 and four participants who were
oﬀered p = $0 were assigned to positive weeks 5-8 oﬀers p ∈ [0, 170].
This approach allows us to maintain incentive compatibility in the BDM mechanism, have
balance between Treatment and Control groups, and use a straightforward regression to estimate
treatment eﬀects of post-midline deactivation.
2.3 Outcome Variables
For the impact evaluation, we consider the outcome variables in the nine families described below.
Appendix B presents survey question text and descriptive statistics for each outcome variable and
moderator, grouped by family. We also construct indices that combine the outcome variables within
each family, weighting by the inverse of the covariance between variables at endline, as described
in Anderson (2008). In constructing these indices, we orient the variables so that more positive
values have the same meaning—for example, more positive means “more polarized” in all cases.
Outcomes to be multiplied by -1 are followed by “× (-1)” in Appendix B.1.
Substitute time uses
At baseline and endline, we asked participants how many minutes per day they spent on Facebook
on the average day in the past four weeks. At baseline, we also asked participants to report how
much of their free time on the average day in the past four weeks they spent on various activities,
ranging from using social media apps other than Facebook to spending time with friends and family
in person. At endline, we asked how much time they spent on the same activities, “relative to what
is typical for you.” We phrased the questions in this way in order to more precisely detect changes
in self-reported time use caused by the deactivation.
Social interaction
We have three measures of social interaction. The friends met in person variable is the natural
log of one plus the number of friends seen in person in the last week, as measured by a survey
question that asked participants to “list the ﬁrst names of as many friends you met in person last
week that you can think of in 1 minute.” Oﬄine activities is the number of oﬄine activities (such
as going out to dinner, spending time with your kids, etc.) that the person did at least once last
week. Diverse interactions is an indicator for whether the respondent interacted with someone who
11
voted the opposite way in the last presidential election plus an indicator for whether the respondent
interacted with someone from another country in the last week.
Substitute news sources
At baseline, we asked participants how often they got news from diﬀerent sources over the past four
weeks, including Facebook, cable TV, print, and radio news, borrowing a standard survey question
from the Pew Research Center (2018a). At endline, we again asked how often they got news from
those same sources, “relative to what is typical for you.” For the participants who reported having
a Twitter handle, we gathered data on number of tweets in the four weeks before baseline began
and in the four weeks between midline and endline. This allows a non-self-reported measure of one
kind of potential substitution away from Facebook.12
News knowledge
In order to detect broad changes in news exposure, we asked participants how closely they followed
politics, how closely they followed news about President Trump, and how many minutes per day
they spent watching, reading, or listening to the news (including on social media) over the past
four weeks.
In order to measure speciﬁc news knowledge, we included a 15-question news knowledge quiz.
For each question, we gave a statement from the news in the past four weeks and asked participants
to indicate if they thought the statement was true or false, or whether they were unsure. The order
of the 15 statements was randomized. Seven of the statements were from news stories covered in
the past four weeks in six news websites: New York Times, Wall Street Journal, Fox News, CNN,
MSNBC, and US News & World Report, such as “The Trump administration set the maximum
number of refugees that can enter the country in 2019 to 30,000.” Three of the headlines were false
modiﬁcations of articles from those same six news websites, such as “President Trump spoke at the
funeral of former Arizona Senator John McCain, honoring the late McCain’s wish.” (In reality, it
had been reported that President Trump was not invited to McCain’s funeral.) The news knowledge
variable is the count of true statements rated as true plus the count of false statements rated as
false, plus one-half for every statement about which the respondent was “unsure.” The ﬁnal ﬁve
statements were from fake news stories—rated false by third-party fact-checkers snopes.com and
factcheck.org—that circulated heavily within a four-week period before the survey. The fake news
knowledge variable is the count of fake statements correctly rated as “false” plus one-half for every
statement about which the respondent was unsure. Appendix B.1 presents the full news knowledge
quizzes from both baseline and endline.
12
In our pre-analysis plan, we grouped this number of tweets variables in the substitute news sources family, but
one might also think of it as a “substitute time use” because Twitter is not only used to read news.
12
Political engagement
We have two measures of political engagement. First, we measure whether participants voted
in the 2018 midterm election, by matching participants on name, birth year, and zip code to a
voting database supplied to Stanford by L2, a voting data provider. See Appendix C for details
on the match process. Second, we measure whether participants clicked on any of the links in the
post-endline politics email.
Political polarization
There are a variety of ways to measure political polarization (see, for example, Gentzkow 2016),
and we use both standard and novel measures. First, we included standard “feeling thermometer”
questions capturing how “warm or cold” participants felt toward the Demoratic and Republican
Parties and President Trump over the past four weeks. The party aﬀective polarization variable is
the respondent’s thermometer warmth toward her own party minus her warmth toward the other
party. For this and all other polarization variables, we include independents who lean toward a
party, and we drop independents who do not lean toward either party.
Second, the Trump aﬀective polarization variable is the thermometer warmth toward President
Trump for Republicans, and minus one times the thermometer warmth toward President Trump
for Democrats. Third, we asked respondents to list recent news events that made them angry at
the Republican or Democratic Party. Party anger is the natural log of one plus the length (in
characters of text) of her response about the other party minus the natural log of one plus the
length of her response about her own party. Fourth, we asked people how often they saw news that
made them better understand the point of view of the Republican Party, and a parallel question
for news about the Democratic Party. Congenial news exposure is the respondent’s answer about
her own political party minus her answer for the other party.
Fifth, we asked opinions about nine current political issues, such as “To what extent do you think
that free trade agreements between the US and other countries have been a good thing or a bad
thing for the United States?” These nine questions were all adapted from recent Pew Center and
Gallup opinion polls. The issue polarization variable reﬂects the extent to which the respondent’s
issue opinions align with the average opinion in her own party instead of the other party. Sixth,
belief polarization reﬂects the extent to which the respondent’s beliefs about current news events
(from the news knowledge quiz described above) align with the average belief in her own party
instead of the other party.13 Finally, vote polarization measures the strength of preferences for the
13
Speciﬁcally, for each issue or belief question q, we normalize responses by the standard deviation in the Control
group, determine Democrats’ and Republicans’ average responses µD
q and µR
q , re-center so that µD
q + µR
q = 0, and
re-sign so that µR
> 0. Deﬁne ˜yiq as individual i’s normalized, re-centered, and re-signed response to question q,
multiplied by -1 if i is a Democrat. ˜yiq thus reﬂects the strength of individual i’s agreement with the average view
of her party instead of the other party. For issue polarization, further deﬁne σq as the Control group within-person
standard deviation of ˜yiq for question q. This measures how much people’s views change between baseline and endline,
13
congressional candidate of the respondent’s party in the midterm election.14
Subjective well-being
There is a vast literature on measuring subjective well-being (see, for example, Kahneman et al.
2006), and we use standard measures from the literature. We modiﬁed existing scales in two ways.
First, we asked questions in reference to the past four weeks, so as to increase our ability to detect
changes as a result of Facebook deactivation. Second, in some cases we chose a subset of questions
from standard multi-question scales in order to focus on areas of subjective well-being that might
be most aﬀected by Facebook.
The happiness variable is the average response to two questions from the Subjective Happiness
Scale (Lyubomirsky and Lepper 1999), asking how happy participants were over the past four
weeks and how happy they were compared to their peers. Life satisfaction is the sum of responses
to three questions from the Satisfaction with Life Scale (Diener et al. 1985), such as the level of
agreement with the statement, “During the past 4 weeks, I was satisﬁed with my life.” Loneliness
is the Three-Item Loneliness Scale (Hughes et al. 2004). Finally, depressed, anxious, absorbed, and
bored reﬂect how much of the time during the past four weeks respondents felt each emotion, using
questions from the European Social Survey well-being module (Huppert et al. 2009).
The daily text messages allowed us to measure the aspects of subjective well-being that are
most important to record in the moment instead of retrospectively. This approach builds on
the Experience Sampling Method of Csikszentmihalyi and Larson (2014) and Stone and Shiﬀman
(1994). The variable SMS happiness is the answer to the question, “Overall, how happy do you
feel right now on a scale from 1 (not at all happy) to 10 (completely happy)?” The variable SMS
positive emotion is an indicator variable for whether the participant reports a positive emotion
when asked, “What best describes how you felt over the last ten minutes?”, with possible responses
such as “angry,” “worried,” “loving/tender,” etc. Finally, SMS not lonely uses the answer to the
question, “How lonely are you feeling right now on a scale from 1 (not at all lonely) to 10 (very
lonely)?”
and allows us to place higher weight on issues about which views are malleable over the deactivation period. For
belief polarization, let σq = 1. The issue and belief polarization measures are Yi = q ˜yiqσq. Appendix Table A15
shows that the issue polarization results are nearly identical if we set σq = 1.
14
Speciﬁcally, we asked “In the recent midterm elections, did you vote for the Republican Party’s or for the
Democratic Party’s candidate for Congress in your district? (If you did not vote, please tell us whom you would
have voted for.)” We code vote polarization as 0 for “other/don’t know.” For people who responded that they
had (or would have) voted for the Republican or Democratic candidate, we then asked, “How convinced were you
about whether to vote for the Republican candidate or the Democratic candidate?” In these cases, we code vote
polarization on a scale from -1 (very convinced to vote for the Democratic candidate) to +1 (very convinced to vote
for the Republican candidate), and then multiply by -1 for Democrats.
14
Post-experiment Facebook use
We have four measures of planned and actual post-experiment Facebook use. First, planned poststudy
use change is the extent to which participants plan to use Facebook more or less than they
had before they started the study. (This was included only in the endline survey.) Second, clicked
time limit email is an indicator for whether the respondent clicked any of the links in the postendline
social media time limit email. Third, speed of reactivation is minus one times the natural
log of one plus the number of days that the participant’s account remained deactivated between
the post-endline 24-hour deactivation period and our most recent measurement on December 17th.
Fourth, Facebook mobile app use is the natural log of one plus the number of minutes per day that
the participant reported using Facebook on their phone in the post-endline survey.
Opinions about Facebook
We asked eight questions eliciting people’s opinions about Facebook, such as “To what extent do
you think Facebook is good or bad for society?” and “To what extent do you think Facebook
makes people more or less politically polarized?” Each of these eight responses was on a ten-point
scale. In the endline survey only, we also asked deactivation bad: “As part of this study, you were
asked to deactivate your Facebook account for [24 hours/4 weeks]. To what extent do you think
that deactivating your account was good or bad for you?” Finally, we also included two open
answer text boxes in which we asked people to write out the most important positive and negative
impacts that Facebook has on their lives. The positive impacts and negative impacts variables are
the natural log of one plus the count of characters in the respective text box.
Secondary outcomes
We also consider the following two outcomes, which we labeled as “secondary” in our pre-analysis
plan. First, we consider the standard generic ballot question. At baseline, we asked “If the elections
for US Congress were being held today, would you vote for the Republican Party’s candidate or the
Democratic Party’s candidate for Congress in your district?” To increase precision, we then asked,
“How convinced are you about whether to vote for the Republican or Democratic candidate?” At
endline, we asked these questions in past tense, about whom the respondent did vote for in the 2018
midterm (or whom the respondent would have voted for had she voted, to avoid potentially selective
non-response). The voted Republican variable is the strength of preferences for the Republican
candidate. We labeled this outcome as secondary because we expected the estimates to be too
imprecise to be of interest.
Second, we asked people to report whether they had voted (at endline) and planned to vote
(at baseline) in the 2018 midterm. We labeled this as secondary because it is superseded by the
administrative voting data from L2.
15
We also gathered contributions to political campaigns from the Federal Election Commission
database. In our pre-analysis plan, we labeled this as secondary because very few Americans
contribute to political campaigns, and we did not expect to be able to detect eﬀects from four
weeks of deactivation. Indeed, only one person in the impact evaluation sample donated to a
political party between the October 2018 midline survey and July 2019. As a result, we deviate
from the pre-analysis plan by dropping this from our analysis.
3 Descriptive Statistics
Table 1 shows sample sizes at each step of our experiment, from the 1.9 million Facebook users
who were shown our ads, to the 1,661 subjects in the impact evaluation sample. Table 2 quantiﬁes
the representativeness of our sample on observables, by comparing the demographics of our impact
evaluation sample to our estimate of the average demographics of adult Facebook users and to the
US adult population. Comparing column 1 to columns 2 and 3, we see that our sample is relatively
high-income, well-educated, female, young, and Democratic, and uses Facebook relatively heavily.15
Appendix Table A14 shows that Treatment and Control are balanced on observables.
Table 3 documents very high response rates to the endline and post-endline surveys and subjective
well-being text messages. Of the 580 people in the Treatment group, only seven failed to
complete the endline survey. Of the 1,081 people in the Control Group, only 17 failed to complete
endline. The average participant responded to 92 percent of daily text messages, well above the
75 percent required in order to receive the completion payment.16 Treatment and Control have
statistically equal response rates to the endline survey and subjective well-being text messages. A
marginally signiﬁcantly larger share of the Treatment group responded to the post-endline survey;
this is less worrisome because Facebook mobile app use is the only variable from that survey for
which we calculate treatment eﬀects, and we show in Appendix Table A13 that using Lee (2009)
bounds to account for attrition does not change the conclusions. Finally, Table 3 also reports the
high level of compliance with our deactivation treatment: Treatment group participants were deactivated
on 90 percent of checks between October 13th (the ﬁrst day after the 24-hour post-midline
deactivation period) and November 7th (the day before endline), against two percent for Control.
As described above, if Treatment group members were found to have active accounts, we sent an
email informing them of this and asking them to promptly deactivate, along with a survey asking
15
In Appendix Figures A17, A18, A19, and A20, we ﬁnd that the two demographic variables that we pre-speciﬁed
as moderators, age and political party, do not appear to systematically moderate treatment eﬀects. Furthermore,
Figure 9 provides no systematic evidence that the eﬀects vary for people who use Facebook more vs. less heavily
before baseline. This suggests that re-weighting the sample for representativeness on these observables would not
substantively change the estimated eﬀects, although it would increase the standard errors.
16
Appendix Figure A26 shows the text message response rate by day (response rates declined slightly over the
course of the experiment) and shows that Treatment and Control response rates are statistically balanced in all days
of the deactivation period.
16
why they were not deactivated. From these surveys, along with email interactions and formal
qualitative interviews following our summer 2018 pilot study, we conclude that most Treatment
group members who did reactivate fall into one of two groups. The ﬁrst group consists of a small
number of users who changed their mind about participating in the experiment and reactivated
intentionally. The second group consists of users who brieﬂy reactivated by accident, for example
because they logged in to another app or online service using their Facebook account credentials.
Appendix Figure A27 shows the cumulative distribution of the share of time deactivated for
the Treatment group, and Appendix Figure A28 shows the distribution of reasons for deactivation
among those for whom this share was less than one. Together, these ﬁgures suggest that the small
group of intentional reactivators accounts for the vast majority of Treatment group non-compliance.
Given this, combined with the fact that the Control group was also found to be deactivated for a
small share of weeks 1-4, we will analyze the experiment as a randomized encouragement design.
4 Empirical Strategy
4.1 Pre-Analysis Plan
We submitted our pre-analysis plan on October 12, as this was the ﬁnal day before the Treatment
and Control groups could have begun to diﬀer. We submitted a slightly updated pre-analysis-plan
on November 7, the day before endline, with only one substantive change: on the basis of data
on reasons for non-compliance described above, we speciﬁed that our primary speciﬁcations would
use IV estimates instead of intent-to-treat estimates. The pre-analysis plan speciﬁed three things.
First, it speciﬁed the outcome variables and families of outcome variables as described above,
including which speciﬁc variables are included in the index for each family and which outcomes are
“secondary.” Versions of Figures 2, 3, 5, 6, 7, and 12 appear as ﬁgure shells in the pre-analysis
plan, although we changed some variable labels as well as the order in which we present the
families of outcome variables for expositional purposes. Second, the pre-analysis plan speciﬁed the
moderators we use when testing for heterogeneous treatment eﬀects, including which moderators
are “secondary.” Third, it speciﬁed the two regression speciﬁcations and the estimation sample as
described below.
4.2 Empirical Strategy
To estimate the local average treatment eﬀect (LATE) of Facebook deactivation, deﬁne Yi as some
outcome measured at endline, and Y b
i as a vector including the baseline value of the outcome and
the baseline value of the index that includes the outcome.17 Deﬁne Di as the percent of deactivation
17
Y b
i excludes the baseline value of the outcome for outcomes such as clicks on post-endline emails that do not
have a baseline value. Y b
i excludes the baseline index when Yi is not included in an index. When Yi is an index, Y b
i
is simply the baseline value of the index.
17
checks between October 13th and November 7th that person i is observed to be deactivated. Deﬁne
Ti ∈ {1, 0} as a Treatment group indicator, and νs as the vector of the 48 stratum dummies. We
estimate local average treatment eﬀects of deactivation using the following regression:
Yi = τDi + ρY b
i + νs + εi, (1)
instrumenting for Di with Ti. In Equation (1), τ measures the local average treatment eﬀect of
deactivation for people induced to deactivate by the promised $102 payment.18
The base sample for all regressions is the “impact evaluation sample”—again, participants who
were willing to accept less than $102 to deactivate in weeks 1-4 (the four weeks after midline) and
were oﬀered p = $102 or p = $0 to do so. For the political polarization outcomes, the sample
includes only Democrats and Republicans, as well as independents who lean toward one party or
the other. Sample sizes sometimes diﬀer across outcomes due to missing data: for example, the
post-endline survey has higher non-response than the endline survey, and many participants do not
have Twitter accounts.
We use robust standard errors in all regressions.
5 Impact Evaluation
This section presents treatment eﬀects of Facebook deactivation. The following subsections present
estimates for four groups of outcomes: substitution, news and political outcomes, subjective wellbeing,
and post-experiment Facebook use and opinions. We then present heterogeneous treatment
eﬀects. Finally, we provide evidence on experimenter demand eﬀects.
In the body of the paper, we present ﬁgures with local average treatment eﬀects and 95 percent
conﬁdence intervals from estimates of Equation (1), with outcome variables Yi normalized so that
the Control group standard deviation equals one. Appendix Tables A10 and A11 provide numerical
regression results for all individual outcome variables in both normalized (standard deviation) units,
as in the ﬁgures, and un-normalized (original) units. Appendix Table A12 provides numerical
regression results for all nine summary indices. These appendix tables also provide unadjusted
18
Facebook deactivation might have a larger impact for people who use Facebook more. Deﬁne Hi as person i’s
average daily hours of Facebook use reported at baseline, winsorized at 120 minutes. We can also estimate the local
average treatment eﬀect of deactivation per hour of daily Facebook use avoided using the following regression:
Yi = τDiHi + βHi + ρY b
i + νs + εi, (2)
analogously instrumenting for DiHi with TiHi.
If eﬀects of deactivation are indeed linear in avoided hours of Facebook use, then Equation (2) could provide more
statistical power than Equation (1). On the other hand, if eﬀects are closer to constant in baseline usage and/or Hi
is measured with error, then Equation (1) will oﬀer more power. In our pre-analysis plan, we speciﬁed that we would
make either Equation (1) or Equation (2) our primary speciﬁcation, depending on which delivered more power. In
reality, the results are very similar. Therefore, we focus on Equation (1) because it is simpler. Appendix E presents
results using Equation (2).
18
p-values and “sharpened” False Discovery Rate (FDR)-adjusted p-values following the procedure
of Benjamini, Krieger, and Yekutieli (2006), as outlined by Anderson (2008). The unadjusted
p-values are appropriate for readers with a priori interest in one speciﬁc outcome. The FDRadjusted
p-values for the individual outcomes limit the expected proportion of false rejections of
null hypotheses across all individual outcomes reported in the paper, while the FDR-adjusted pvalues
for the indices limit the expected proportion of false rejections of null hypotheses across
the nine indices. The sharpened FDR-adjusted p-values are less conservative than the unadjusted
p-values for p-values greater than about 0.15, and more conservative for unadjusted p-values less
than that.
5.1 Substitutes for Facebook
Figure 2 presents treatment eﬀects on substitutes for Facebook: substitute time uses, social interactions,
and substitute news sources. Substitution is of interest for two reasons. First, our treatment
entails deactivating Facebook and also re-allocating that time to other activities. Understanding
that re-allocation is thus crucial for conceptually understanding the “treatment.” Second, this substitution
helps to understand mechanisms for key eﬀects. One central mechanism through which
Facebook might aﬀect psychological well-being is by crowding out face-to-face interactions. However,
it’s also possible that when people deactivate, they primarily devote their newly available time
to other solitary pursuits. Furthermore, a central mechanism for possible political externalities is
that social media use crowds out consumption of higher-quality news. However, it’s also possible
that when people deactivate, they simply get less news overall instead of substituting to other news
sources.
The top group of outcomes in Figure 2 measures self-reported time use. Facebook usage was
reported in minutes. For all other activities, the endline survey asked respondents how much time
they spent on the activity in the last four weeks relative to what is typical for them, on a ﬁve-point
scale from “A lot less” to “A lot more.” For all time use outcomes, “Same” is the average answer
in the Control group.
The ﬁrst row conﬁrms that the treatment indeed reduced Facebook use as intended. At endline,
the Control group reported that they had used Facebook for an average of 59.53 minutes per day
over the past four weeks, and the local average treatment eﬀect of deactivation is 59.58 minutes
per day.19 As shown on Figure 2, this corresponds to a reduction of 1.59 standard deviations.
We ﬁnd that Facebook deactivation reduced time devoted to other online activities. Time using
non-Facebook social media falls by a quarter point on our ﬁve-point scale (0.27 SD), and time
on non-social online activities falls by 0.12 points (0.14 SD). Thus, Facebook appears to be a
19
Appendix Table A5 reports baseline means of our time use variables. The mean of self-reported Facebook minutes
at baseline is 74.5 minutes per day, and the mean of reported minutes using the Facebook mobile app at baseline is
60 minutes per day.
19
complement rather than a substitute for other online activities. This makes sense to the extent
that deactivating Facebook makes people less likely to be using their phones or computers in the
ﬁrst place, and less likely to follow Facebook links that direct to non-Facebook sites (e.g., a news
website or Twitter post). Furthermore, the Treatment group may have avoided logging into other
apps such as Spotify and Tinder because we had informed participants that using Facebook to
actively log into other apps would reactivate Facebook.
Rows 4-7 of Figure 2 suggest that the 60 minutes freed up by not using Facebook, as well as the
additional minutes from reductions in other online activities, were allocated to both solitary and
social activities oﬄine. Solitary television watching increases by 0.17 points on our scale (0.17 SD),
other solitary oﬄine activities increase by 0.23 points (0.25 SD), and time devoted to spending time
with friends and family increases by 0.14 points (0.16 SD). The substitute time uses index, which
does not include Facebook minutes, shows an increase in overall non-Facebook activities. All of the
online and oﬄine time use eﬀects are highly signiﬁcant with and without adjustment for multiple
hypothesis testing.
The middle group of outcomes in Figure 2 contains measures of social interaction. Deactivation
increased the count of oﬄine activities that people reported doing at least once last week by about
0.18 (0.12 SD). Appendix Figure A29 shows that the speciﬁc activities with the largest point
estimates are going out to dinner, getting together with friends, and spending time with parents.
The point estimates for the other oﬄine activities we measure (going to the cinema, talking to
friends on the phone, going to a party, going shopping, and spending time with your kids) are all
very close to zero. Notwithstanding the positive eﬀects on oﬄine activities, there are no statistically
signiﬁcant eﬀects on the number of friends that participants listed as having met in person last week,
or on diverse interactions (whether or not they interacted with someone who voted diﬀerently in
the last presidential election or interacted with someone from another country). We ﬁnd no eﬀects
on the social interaction index, although the point estimate is positive.
The bottom group of outcomes in Figure 2 measures news consumption. As with the substitute
time uses, the endline survey asked participants how much time they spent getting news from each
source in the last four weeks relative to what is typical for them; “Same” is again the average
answer in the Control group. As expected, Facebook deactivation substantially reduced the extent
to which people said they relied on Facebook as a news source. Consistent with the time use
results, the Treatment group also got substantially less news from non-Facebook social media sites
(0.36 SD). The point estimates for print, radio, and TV news are all positive but statistically
insigniﬁcant. Facebook deactivation has a positive but insigniﬁcant eﬀect on Twitter use. As we
discuss below in the news knowledge results, deactivation reduced the total time subjects report
spending consuming news by eight minutes per day, or 15 percent of the Control group mean of 52
minutes.
Overall, these results suggest that Facebook is a substitute for oﬄine activities but a complement
20
to other online activities. This suggests the possibility that Facebook could reduce subjective
well-being by reducing in-person interactions, but also impose positive political externalities by
increasing news knowledge. Below, we test these possibilities more directly.
5.2 Eﬀects on News and Political Outcomes
Figure 3 presents treatment eﬀects on news and political outcomes: news knowledge, political
engagement, and political polarization. News knowledge and political engagement are of interest
because well-functioning democratic societies fundamentally rely on well-informed voters who actually
show up to the polls to vote. Political polarization is of interest because it is may make
democratic decision making less eﬃcient, and may lead citizens to perceive democratic outcomes
as less legitimate (Iyengar, Sood, and Lelkes 2012; Iyengar and Westwood 2015).
Deactivation caused substantial reductions in both self-reported attention to news and directly
measured news knowledge. The top three rows show that deactivation reduced how much people
reported they followed news about politics and about President Trump (by 0.14 and 0.11 SD,
respectively), as well as the average minutes per day spent consuming news (a drop of eight minutes
per day, or 15 percent of the control group mean). Accuracy on our news knowledge quiz fell by
0.12 standard deviations.20 Tangibly, the Control group answered an average of 7.26 out of the 10
news knowledge questions correctly (counting “unsure” as 1/2 correct), and deactivation reduced
this average by 0.14. There is no detectable eﬀect on fake news knowledge, possibly reﬂecting the
limited reach of even the highly shared fake news items included in our survey. Overall, deactivation
reduced the news knowledge index by about 0.19 standard deviations.
There are no statistically detectable eﬀects on political engagement. As reported in Appendix
Tables A10 and A11, the point estimates suggest that deactivation increased turnout by three
percentage points according to the administrative data and decreased turnout by three percentage
points according to the self-reported data, and neither estimate is statistically diﬀerent from zero.
Similarly, the Treatment and Control groups are statistically equally likely to have clicked on any
link in the post-endline politics email. Appendix Figure A35 does show a marginally signiﬁcant
negative eﬀect on voted Republican, suggesting that deactivation may have reduced support for
Republican congressional candidates. The unadjusted p-value is 0.06, the sharpened FDR-adjusted
p-value is 0.08, and we had labeled this as a “secondary outcome” in our pre-analysis plan.
Prior research has shown that people tend to be exposed to ideologically congenial news content
in general (Gentzkow and Shapiro 2011) and on Facebook in particular (Bakshy, Messing, and
20
Appendix G presents more analysis of the eﬀects on news knowledge, including eﬀects on each individual news
knowledge and fake news knowledge question. All but one of the point estimates for the 10 news knowledge questions is
negative. The news knowledge questions with the largest eﬀects involve correctly responding that Elizabeth Warren’s
DNA test had revealed Native American ancestry and that Jeﬀ Sessions had resigned at President Trump’s request.
There was also a statistically signiﬁcant diﬀerence in knowledge about one fake news story: the Treatment group was
less likely to correctly respond that Cesar Sayoc, the suspect in an act of domestic terrorism directed at critics of
President Trump, was not a registered Democrat.
21
Adamic 2015). Thus, the above ﬁnding that deactivation reduced news exposure naturally suggests
that deactivation might have also reduced political polarization.
Indeed, deactivation did reduce political polarization. Point estimates are negative for all polarization
measures. The largest and most signiﬁcant individual eﬀect is on congenial news exposure:
deactivation decreased the number of times that people reportedly saw news that made them better
understand the point of view of their own political party relative to the other party. Deactivation
also decreased issue polarization, which Fiorina and Abrams (2008) single out as the “most direct”
way of measuring polarization.21 Appendix Table A10 shows that both of these eﬀects are highly
signiﬁcant after adjusting for multiple hypothesis testing. The other measures with the largest
point estimates are party anger and party aﬀective polarization, although these individual eﬀects
are not statistically signiﬁcant. Overall, deactivation reduced the political polarization index by
about 0.16 standard deviations.22
Figure 4 illustrates how deactivation reduced issue polarization, by plotting the distribution of
“issue opinions” for Democrats and Republicans in Treatment and Control at endline. Our issue
opinions measure exactly parallels the issue polarization variable used in the regressions, except
that we keep opinions on a left-to-right scale, with more negative indicating more agreement with
the average Democratic opinion, and more positive indicating more agreement with the average
Republican opinion. (By contrast, the issue polarization variable multiplies Democrats’ responses
by -1, so that a more positive value reﬂects more agreement with the average opinion in one’s
political party.) We then normalize issue opinions to have a standard deviation of one in the
Control group. The ﬁgure shows that deactivation moves both Democrats and Republicans visibly
toward the center. In the Control group, the issue opinions of the average Democrat and the
average Republican diﬀer by 1.47 standard deviations. In the Treatment group, this diﬀerence is
1.35 standard deviations—about eight percent less.
Are these polarization eﬀects large or small? As one benchmark, we can compare these eﬀects
to the increase in political polarization in the US since 1996, well before the advent of social media.
Using data from the American National Election Studies, Boxell (2018) calculates that the change
in a diﬀerent index of polarization measures increased by 0.38 standard deviations between 1996
and 2016. The 0.16 standard deviation eﬀect of Facebook deactivation on political polarization in
our sample is about 42 percent as large as this increase.23
21
Appendix Figure A30 presents results for each of the issue polarization questions. The issues for which deactivation
caused the largest decrease in polarization were the direction of racial bias in policing and whether the Mueller
investigation is biased.
22
Like all of our outcome families, the polarization index includes a range of diﬀerent outcomes with diﬀerent
interpretations. Exposure to congenial news is conceptually diﬀerent from aﬀective polarization and issue polarization.
Appendix Table A16 shows that the eﬀect on the political polarization index is robust to excluding each of the seven
individual component variables in turn, although the point estimate moves toward zero and the unadjusted p-value
rises to 0.09 when omitting congenial news exposure.
23
Speciﬁcally, Boxell’s polarization index increased by 0.269 units from 1996-2016, and the standard deviation of
Boxell’s polarization index across people in 2016 is 0.710 units, so political polarization increased by 0.269/0.71 ≈
22
Overall, these results suggest that Facebook plays a role in helping people stay informed about
current events, but also increases polarization, particularly of views on political issues.
5.3 Eﬀects on Subjective Well-Being
Figure 5 presents estimates of eﬀects on subjective well-being (SWB). These outcomes are of interest
because, as discussed in the introduction, many studies show cross-sectional or time-series
correlations between social media use and well-being, and on this basis researchers have speculated
that social media may have serious adverse eﬀects on mental health. The outcomes are re-signed
so that more positive represents better SWB—for example, the “depressed” variable is multiplied
by (-1).
We ﬁnd that deactivation indeed signiﬁcantly increases SWB. All but one of the ten point
estimates are positive. The magnitudes are relatively small overall, with the largest and most
signiﬁcant eﬀects on life satisfaction (0.12 SD), anxiety (0.10 SD), depression (0.09 SD), and
happiness (0.08 SD).24 All of these eﬀects remain signiﬁcant after adjusting for multiple hypothesis
testing. The text message based measures of happiness are not signiﬁcantly diﬀerent from zero,
with positive point estimates ranging from 0.01 SD to 0.06 SD. Deactivation improved our overall
SWB index by 0.09 standard deviations.
Are these subjective well-being eﬀects large or small? As one benchmark, we can consider the
eﬀect sizes in their original units, focusing on the measures with the largest eﬀects. Happiness is the
average response to two questions (for example, “Over the last 4 weeks, I think I was ...”) on a scale
from 1 (not a very happy person) to 7 (a very happy person). The Control group endline average
is 4.47 out of a possible 7, and deactivation caused an average increase of 0.12. Life satisfaction
is the extent of agreement with three questions (for example, “During the past four weeks, I was
satisﬁed with my life”) on seven-point Likert scales from “strongly disagree” “Strongly agree.” The
Control group endline average is 12.26 out of a possible 21, and deactivation caused an average
increase of 0.56. Depressed and anxious are responses to the question, “Please tell us how much
of the time during the past four weeks you felt [depressed/anxious],” where 1 is “None or almost
none of the time” and 4 is “All or almost all of the time.” The average responses are 2.99 and 2.60,
respectively, and deactivation caused average increases of 0.08 and 0.09.
As a second benchmark, a meta-analysis of 39 randomized evaluations ﬁnds that positive psy-
0.379 standard deviations over that period. Of course, this benchmarking exercise does not imply that political
polarization in the US would have increased by one-third less in the absence of Facebook, for many reasons. For
example, the treatment eﬀects in our sample from a four-week deactivation are unlikely to generalize to the US
population over Facebook’s 15-year life. Furthermore, some of our polarization measures are unique to our study.
The one measure that appears in both Boxell’s index and our index, party aﬀective polarization, rose by 0.18 standard
deviations between 1996 and 2016. Our point estimate of -0.06 standard deviations is about one-third of this amount,
although this estimate is not statistically diﬀerent from zero.
24
Appendix Figure A34 presents results for the individual questions within the happiness, life satisfaction, and
loneliness scales.
23
chology interventions (i.e. self-help therapy, group training, and individual therapy) improve subjective
well-being (excluding depression) by 0.34 standard deviations and reduce depression by 0.23
standard deviations (Bolier et al. 2013). Thus, deactivating Facebook increased our subjective
well-being index by about 25-40 percent as much as standard psychological interventions.
As a third benchmark, Appendix Table A17 presents a regression of our baseline SWB index
on key demographics (income, college completion, gender, race, age, and political party). College
completion is conditionally associated with 0.23 standard deviations higher SWB. Thus, the eﬀect
of deactivating Facebook is just over one-third of the conditional diﬀerence in subjective wellbeing
between college grads and everyone else. The table also shows that a $10,000 increase in
income is conditionally associated with a 0.027 standard deviation increase in SWB. Thus, the
eﬀect of deactivating Facebook is equal to the conditional diﬀerence in subjective well-being from
about $30,000 additional income. This income equivalent is large because “money doesn’t buy
happiness”: although income is correlated with SWB, the slope of that relationship is not very
steep.
Appendix Figure A31 presents eﬀects on the SMS outcomes by week of the experiment, to
test whether the eﬀects might have some trend over time. None of the eﬀects on any of the
three outcomes is statistically signiﬁcant in any of the four weeks. The point estimates do not
systematically increase or decrease over time, and if anything, the point estimates are largest in
the ﬁrst week. This suggests that the eﬀects of a longer deactivation might not be diﬀerent.
We can also compare our SWB eﬀects to what we would have estimated using the kind of
correlational approach taken by many previous non-experimental studies. These studies often have
speciﬁc designs and outcomes that don’t map closely to our paper, so it is diﬃcult to directly
compare eﬀect sizes with other papers. We can, however, replicate the empirical strategy of simple
correlation studies in our data, and compare our cross-sectional correlations to the experimental
results. To do this, we regress SWB outcomes at baseline on daily average Facebook use over the
past four weeks as of baseline, divided by the local average treatment eﬀect of deactivation on daily
average Facebook use between midline and endline, so that the coeﬃcients are both in units of
average use per day over the past four weeks.25
The baseline correlation between our SWB index and Facebook use is about three times larger
than the experimental estimate of the treatment eﬀect of deactivation (about 0.23 SD compared to
0.09 SD), and the point estimates are highly statistically signiﬁcantly diﬀerent. Controlling for basic
demographics brings down the non-experimental estimate somewhat, but it remains economically
25
Speciﬁcally, the non-experimental estimates are from the following regression:
Y b
i = τ ˜Hi + βXi + i, (3)
where Y b
i is participant i’s value of some outcome measured in the baseline survey, Xi is a vector of basic demographic
variables (household income, age, and college, male, white, Republican, and Democrat indicators), and ˜Hi
is baseline average daily Facebook use over the past four weeks (winsorized at 120 minutes per day) divided by the
local average treatment eﬀect on average daily Facebook use between midline and endline.
24
and statistically larger than our experimental estimate. Appendix Figure A32 presents the full
results for all SWB outcomes.26 These ﬁndings are consistent with reverse causality, for example if
people who are lonely or depressed spending more time on Facebook, or with omitted variables, for
example if lower socio-economic status is associated with both heavy use and lower well-being. They
could also reﬂect a diﬀerence between the relatively short-term eﬀects measured in our experiment
and the longer-term eﬀects picked up in the cross-section. However, the lack of a detectable trend
in treatment eﬀects on the text message outcomes over the course of our experiment (as noted
above and seen in Appendix Figure A31) points away from this hypothesis.
Subjects’ own descriptions in follow-up interviews and free-response questions are consistent
with these quantitative ﬁndings, while also highlighting substantial heterogeneity in the eﬀects.
Many participants described deactivation as an unambiguously positive experience. One said in an
interview,
I was way less stressed. I wasn’t attached to my phone as much as I was before. And I
found I didn’t really care so much about things that were happening [online] because I
was more focused on my own life... I felt more content. I think I was in a better mood
generally. I thought I would miss seeing everyone’s day-to-day activities... I really didn’t
miss it at all.
A second wrote, “I realized how much time I was wasting. I now have time for other things. I’ve
been reading books and playing the piano, which I used to do daily until the phone took over.”
A third wrote, “I realized I was using it too much and it wasn’t making me happy. I hate all of
the interactions I had with people in comment sections.”
Many others highlighted ways in which deactivation was diﬃcult. One said in an interview,
I was shut oﬀ from those [online] conversations, or just from being an observer of what
people are doing or thinking. . . I didn’t like it at ﬁrst at all, I felt very cut oﬀ from
people that I like. . . I didn’t like it because I spend a lot of time by myself anyway, I’m
kind of an introvert, so I use Facebook in a social aspect in a very big way.
Others described the diﬃculty of not being able to post for special events such as family birthdays
and not being able to participate in online groups.
Overall, our data suggest that Facebook does indeed have adverse eﬀects on SWB. However, the
magnitude of these eﬀects is moderate and may be smaller than correlation studies would suggest,
and our qualitative interviews suggest that the average eﬀect likely masks substantial heterogeneity.
26
One could also do similar experimental vs. non-experimental comparisons for other outcomes, but we have done
this only for SWB because SWB is the focus of the non-experimental literature in this area.
25
5.4 Post-Experiment Facebook Use and Opinions
Figure 6 presents eﬀects of deactivation on post-experiment demand for Facebook as well as participants’
subjective opinions about Facebook. These results are closely related to the ﬁndings
on subjective well-being, as we might expect participants who found deactivation increased their
happiness would choose to use Facebook less in the future. They also speak more directly to the
popular debate over whether social media are addictive and harmful. If deactivation reduces postexperiment
Facebook use, this is consistent with standard habit formation models such as Becker
and Murphy (1988), or with learning models in which experiencing deactivation caused people to
learn that they would be better oﬀ if they used Facebook less.27
Deactivation clearly reduced post-experiment demand for Facebook. These eﬀects are very
stark, with by far the largest magnitude of any of our main ﬁndings. The eﬀect on reported
intentions to use Facebook as of the endline survey is a reduction of 0.78 standard deviations:
while the average Control group participant planned to reduce future Facebook use by 22 percent,
deactivation caused the Treatment group to plan to reduce Facebook use by an additional 21 percent
relative to Control. In our post-endline survey a month after the experiment ended, we measured
whether people actually followed through on these intentions, by asking people how much time they
had spent on the Facebook mobile app on the average day in the past week. Deactivation reduces
this post-endline Facebook mobile app use by 12 minutes per day, or 0.31 standard deviations.
This is a 23 percent reduction relative to the Control group mean of 53 minutes per day, lining
up almost exactly with the planned reductions reported at endline. However, Appendix Table A13
shows that the reduction is less than half as large (8 percent of the Control group mean) and
not statistically signiﬁcant (with a t-statistic of -1.16) if we limit the sample to iPhone users who
reported their usage as recorded by their Settings app, thereby excluding participants who were
reporting personal estimates of their usage.
As a diﬀerent (and non-self-reported) measure of post-experiment use, we can look at the speed
with which people reactivated their Facebook accounts following the 24-hour post-endline period in
which both Control and Treatment were deactivated. Figure 7 presents the share of our deactivation
checks in which the Treatment and Control groups were deactivated, by day of the experiment.28
By day 35, one week after the end of the experiment, 11 percent of the Treatment group was still
deactivated, compared to three percent of the Control group. By day 91, nine weeks after the end
27
Appendix Figure A33 presents histograms of participants’ opinions about Facebook at baseline. People are evenly
divided on whether Facebook is good or bad for themselves and for society and whether Facebook makes people more
or less happy. Consistent with our results, people tend to think that Facebook helps people to follow the news better
and makes people more politically polarized.
28
There is a slight dip in deactivation rates for the Treatment group seven days after the deactivation period began.
This was caused by the fact that some participants failed to turn oﬀ a default setting in which Facebook reactivates
users’ proﬁles after seven days of deactivation. For technical reasons, our deactivation checking algorithm checked
the entire Control group once every few days between midline and endline in order to check the Treatment group
four times per day. After endline, we returned to checking all participants approximately once per day.
26
of the experiment, ﬁve percent of the Treatment group was still deactivated, against 2.5 percent
of Control. As Figure 6 shows, the local average treatment eﬀect on the speed of reactivation is a
highly signiﬁcant 0.59 standard deviations. Overall, deactivation clearly decreased post-experiment
use, reducing the index by 0.61 standard deviations. As introduced above, is consistent with models
of habit formation or learning.
The bottom group of outcomes in Figure 6 supplement the post-experiment use outcomes by
measuring participants’ qualitative opinions about Facebook. These are re-signed so that more
positive means more positive opinions, so agreement with the statement that “Facebook exposes
people to clickbait or false news stories” and the length of text about Facebook’s negative impacts
are both multiplied by (-1). The results are mixed. Deactivation increases the extent to which
participants think Facebook helps them follow the news better, and it also makes participants agree
more that people would miss Facebook if they stopped using it. On the other hand, participants who
deactivated for four weeks instead of 24 hours were more likely to say that their deactivation was
good for them.29 Deactivation increases both the positive impacts and negative impacts variables,
i.e. it makes people write more about both positive and negative aspects of Facebook. Overall,
deactivation had no statistically signiﬁcant eﬀect on the Facebook opinions index.
Figure 8 presents the distributions of Treatment and Control responses to two key questions
reﬂecting opinions about Facebook. Both Treatment and Control tended to agree that “if people
spent less time on Facebook, they would soon realize that they don’t miss it,” but deactivation
weakened that view. On this ﬁgure, the Treatment group’s average response on the scale from -5
to +5 was -1.8, while the Control group’s average response is -2.0. The bottom right panel shows
that both Treatment and Control tended to think that deactivation was good for them, but the
Treatment group is more likely to think that their (longer) deactivation was good for them. On this
ﬁgure, the Treatment group’s average response on the scale from -5 to +5 is -2.3, while the Control
group’s average response is -1.9. Remarkably, about 80 percent of the Treatment group thought
that deactivation was at least somewhat good for them, and the modal response was the strongest
possible agreement that deactivation was good (the left-most bar on the histogram). In both panels,
the Treatment group has a wider dispersion of responses, with more people strongly agreeing and
more people strongly disagreeing. This highlights the importance of testing for treatment eﬀect
heterogeneity, which we will do in the next section.
To give a richer sense of how deactivation aﬀected Facebook use, the post-endline survey included
a free-response question in which we asked people to write how they had changed their
Facebook use since participating in the study. We then use standard text analysis tools to determine
how the Treatment and Control groups responded diﬀerently. Speciﬁcally, we processed
29
One should be cautious in interpreting this eﬀect, as it could result both from a change of opinion about Facebook
and from the diﬀerence in length of the deactivation they were evaluating. As we shall see below, the Control group
also tends to believe that deactivation was good for them, but the modal answer was 0 (i.e., neither good nor bad),
suggesting that many people were indiﬀerent to such a short deactivation.
27
the text by stemming words to their linguistic roots (for example, “changes,” “changing,” and
“changed” all become “chang”), removing common “stop words” (such as “the” and “that”), and
making lists of all one, two, three, and four word phrases that appeared ﬁve or more times in the
sample. We then constructed Pearson’s χ2 statistic, which measures the extent of diﬀerential usage
rates between Treatment and Control; the phrases with the highest χ2 are especially unbalanced
between the two groups. This parallels Gentzkow and Shapiro’s (2010) approach to determining
which phrases are used more by Republicans vs. Democrats, except we determine which phrases
are used more by Treatment vs. Control.
The two panels of Table 4 present the 20 highest-χ2 phrases that were more common in Treatment
and in Control. The Treatment group was relatively likely to write that they were using
Facebook less or not at all (“use much less,” “not use facebook anymor,” “stop use facebook”) or
more judiciously—the phrase “use news app” is mostly from people saying that they have switched
to getting news from their phone’s news app instead of Facebook. By contrast, while a few of
the Control group’s most common phrases indicate lower use (variants of “more aware much time
spend” and “use facebook slightli less”), the great majority of their relatively common phrases
indicate that their Facebook use has not changed.
To more deeply understand the ways in which deactivation changed people’s relationship to
Facebook, we partnered with a team of qualitative researchers who analyzed our survey data and
additional participant interviews (Baym, Wagman, and Persaud 2019). They ﬁnd that many participants
emphasized that their time oﬀ of Facebook led them to use the platform more “consciously,”
aligning their behavior with their desired use. For example, some participants discussed avoiding
their news feed and only looking at their Facebook groups, while others removed the Facebook app
from their phones and only accessed the site using their computers.
5.5 Heterogeneous Treatment Eﬀects
5.5.1 Individual Moderators
In our pre-analysis plan, we speciﬁed that we would present separate estimates for subgroups deﬁned
by four primary moderators. Figure 9 presents those estimates. The top panel presents estimates
for heavy users vs. light users—that is, people whose baseline reported Facebook use was above vs.
below median. There is no consistent evidence that the eﬀects are diﬀerent for people who report
being heavier users, perhaps because Facebook use is measured with noise.
The second panel presents estimates for heavy news users vs. light news users—that is, those
who get news from Facebook fairly often or very often vs. never, hardly ever, or sometimes. As one
might expect, the estimated eﬀects for news knowledge are larger for people who get more news
from Facebook, but this diﬀerence is not statistically signiﬁcant. The pre-analysis plan speciﬁed
that we would limit these tests to only the news and political outcomes in Section 5.2,
28
The third panel presents separate estimates for active users vs. passive users. We measure this
using two questions: share of active vs. passive browsing using a question based on the Passive and
Active Facebook Use Measure (Gerson, Plagnol, and Corr 2017), and “what share of your time on
Facebook do you spend interacting one-on-one with people you care about.” Active vs. passive
users are deﬁned as having above- vs. below-median sum of their two responses to these questions.
This moderator is of interest because of a set of papers cited in the introduction suggesting that
passive Facebook use can be harmful to subjective well-being, while active use might be neutral or
beneﬁcial. Perhaps surprisingly, we see no diﬀerences in the eﬀects of deactivation on the subjective
well-being index. The pre-analysis plan speciﬁed that we would limit these tests to the four families
reported in the ﬁgure.
Finally, the fourth panel presents separate estimates of eﬀects on subjective well-being text
message surveys for text messages sent during the time of day when the respondent reported using
Facebook the most. We see no clear diﬀerences in the eﬀects on subjective well-being.
The pre-analysis plan also speciﬁed two secondary moderators: age (for all outcomes) and
political party (limited to the news and political outcomes). We considered these secondary because
we did not have a strong prior that we would be able to detect heterogeneous eﬀects. Appendix
Figure A9 presents estimates of eﬀects on these outcomes. There are no systematic patterns.
Appendix Figure A9 also includes heterogeneity by above vs. below-median valuation of Facebook.
While we added this moderator only after the pre-analysis plan was submitted, it is important
because our impact evaluation sample only includes participants with WTA less than $102.
Under the assumption that marginal treatment eﬀects are monotonic in WTA, treatment eﬀect
heterogeneity within our impact evaluation sample would be informative about treatment eﬀects
for the full population. The eﬀects for above- vs. below-median WTA diﬀer statistically for only
one index: the eﬀects on political polarization are driven by above-median WTA participants.
The above-median WTA point estimate is larger and statistically indistinguishable for two indices,
smaller and statistically indistinguishable for four indices, and opposite-signed for the ﬁnal index.
This provides some support for the view that eﬀect sizes would not be systematically diﬀerent in
the full Facebook user population including users with higher valuations.
Appendix Figure A9 presents one additional test of external validity that was suggested by a
referee after the pre-analysis plan was submitted. We construct sample weights that match the
impact evaluation sample to the observable characteristics of Facebook users in Table 2. Appendix
Figure A9 shows that participants with below- vs. above-median sample weights—that is, the
types of people who were especially likely vs. unlikely to participate in the study—do not have
systematically diﬀerent treatment eﬀects. This provides some further support for the view that
eﬀect sizes would be similar in the full Facebook user population.
Appendix F presents heterogeneous treatment eﬀects on each individual outcome.
29
5.5.2 All Possible Moderators
Many factors other than the speciﬁc variables we speciﬁed above might moderate treatment eﬀects
of Facebook deactivation. To search for additional possible moderators, we test whether any of the
demographics or outcome variable indices collected at baseline might moderate treatment eﬀects
on the key outcomes of interest. We consider six outcomes: the latter ﬁve indices (news knowledge,
political polarization, subjective well-being, post-experiment use, and Facebook opinions) plus the
variable deactivation bad, which we add because of the heterogeneity displayed in Figure 8. We
consider 13 potential moderators: all six demographic variables listed in Table 2 (income, years
of education, gender, race, age, and political party aﬃliation, which is on a seven-point scale
from strongly Democratic to strongly Republican) and the baseline values of all seven relevant
indices.30 We normalize each potential moderator to have a standard deviation of one, and we
denote normalized moderator k by Xk
i .
For all outcomes other than deactivation bad, we estimate the following modiﬁed version of
Equation (1):
Yi = τDi + αk
DiXk
i + ζXk
i + ρY b
i + νs + εi, (4)
instrumenting for Di and DiXk
i with Ti and TiXk
i . For deactivation bad, we simply estimate
Yi = αkXk
i + εi in the Treatment group only; this identiﬁes what types of people in the Treatment
group thought that deactivation was particularly good or bad. In total, we carry out 78 tests in 78
separate regressions: 13 potential moderators for each of the six outcomes.
There are many ways to estimate heterogeneous treatment eﬀects, including causal forests
Athey, Tibshirani, and Wager (2019) and lasso procedures. We chose this approach because it
delivers easily interpretable estimates.
Figure 10 presents the interaction coeﬃcients ˆαk and 95 percent conﬁdence intervals for each
of the six outcomes. To keep the ﬁgures concise, we plot only the ﬁve moderators with the largest
absolute values of ˆαk, so there are another eight smaller unreported ˆαk coeﬃcients for each outcome.
We highlight three key results. First, deactivation may reduce polarization more (i.e., Facebook
use may increase polarization more) for older people, white people, and men. Second, Facebook
deactivation has less positive eﬀect on subjective well-being for people who have more oﬄine social
interactions and are already more happy at baseline. This suggests that Facebook use may have
the unfortunate eﬀect of reducing SWB more for people with greater social and psychological need.
In our sample, these “higher-need” people also use Facebook more heavily. Third, people may
have some intuition about whether they will like deactivation: people with more positive baseline
opinions about Facebook are less likely to decrease their post-experiment use and less likely to
30
There are originally nine indices. We exclude the baseline substitute time uses index because it is not easily
interpretable, and we exclude the baseline post-experiment use index because this only includes Facebook mobile app
use.
30
think that deactivation was good for them.
5.6 Experimenter Demand Eﬀects
Most of our outcomes are self-reported, and it would have been diﬃcult to further conceal the
intent of the randomized experiment. This raises the possibility of experimenter demand eﬀects,
i.e. that survey responses depend on what participants think the researchers want them to say. To
test for demand eﬀects, the endline survey asked, “Do you think the researchers in this study had
an agenda?” Table 5 presents the possible responses and shares by treatment group.
For demand eﬀects to arise, participants must believe that the researchers want a particular
pattern of responses. Table 5 shows that 62 percent of both Treatment and Control groups thought
we had no particular agenda or were not sure. This suggests that demand eﬀects would not arise for
a solid majority of our sample. However, demand eﬀects could arise for the remaining 38 percent.
For experimenter demand eﬀects to bias our treatment eﬀects, either (i) the Treatment and
Control groups must have diﬀerent beliefs about what the researchers want, or (ii) participants
must sense what treatment group they are in and change their answers to generate the treatment
eﬀect that they think the researchers want (or don’t want). Table 5 shows that possibility (i) is
not true: perceived researcher agenda is closely balanced between Treatment and Control. To test
for possibility (ii), we can estimate treatment eﬀects separately for the subsample that thought
that we “wanted to show that Facebook is bad for people” vs. all other participants. If (ii) is
true, then our results should be diﬀerent in these two subsamples. Appendix Figure A36 shows
that this is not the case: the eﬀects on outcome indices that look “good” or “bad” for Facebook
(e.g. news knowledge, political polarization, subjective well-being, and post-experiment use) are
not statistically diﬀerent, and there is no pattern of point estimates to suggest that the results are
generally more “good” or “bad” in one of the two subsamples.
Of course, these tests are only suggestive. But combined with the fact that the non-self-reported
outcomes paint a similar picture to the self-reports, these tests suggest that demand eﬀects are
unlikely to be a major source of bias in our results.
6 Measuring the Consumer Surplus from Facebook
Quantifying the economic gains from free online services such as search and media is particularly
important given that these services represent an increasingly large share of the global economy. This
measurement has been particularly challenging because the lack of price variation (or any price at
all) makes it impossible to use standard demand estimation to measure consumer surplus.31 In
this section, we present two back-of-the-envelope consumer surplus calculations. First, we employ
31
As mentioned in the introduction, see Brynjolfsson and Saunders (2009), Byrne, Fernald, and Reinsdorf (2016),
Nakamura, Samuels, and Soloveichik (2016), Brynjolfsson, Rock, and Syverson (2018), and Syverson (2017).
31
the standard assumption that willingness-to-accept identiﬁes consumer surplus. Second, we adjust
consumer surplus to account for the possibility that deactivation might help people learn their true
valuation of Facebook. This adjustment highlights the challenges in using willingness-to-accept as
a measure of consumer welfare.
6.1 Standard Consumer Surplus Estimate
In a standard model, willingness-to-accept to abstain from Facebook equals consumer surplus.
Figure 11 presents the histogram of WTA to deactivate Facebook for the four weeks after midline
instead of only the 24 hours after midline. The median is $100, and almost 20 percent had valuations
greater than $500. After winsorizing valuations at $1000, the mean is $203. After re-weighting the
sample to match the observable characteristics of Facebook users in Table 2, the median is still
$100, and the winsorized mean is $180. Multiplying the mean by the estimated 172 million US
Facebook users would imply that 27 days of Facebook generates $31 billion of consumer surplus.
Our sample’s WTA for Facebook abstention is larger than in most other studies, but not all. In
an online panel weighted for national representativeness, Brynjolfsson, Eggers, and Gannamaneni
(2018) estimate that the mean WTA to not use Facebook for one month is $48, and that the median
WTA to hypothetically stop using social media for one year was $205 in 2016 and $322 in 2017.
In their sample of European college students, Brynjolfsson, Eggers, and Gannamaneni (2018) ﬁnd
a median WTA of $175 for one month.32 In samples of college students, residents of a college
town, and Amazon MTurk workers, Corrigan et al. (2018) estimate that the mean annualized
WTA to deactivate Facebook ranges from $1,139 to $1,921, depending on the sample and the
length of deactivation. In a sample of college students, Mosquera et al. (2018) estimate that the
median (mean) WTA to not use Facebook for one week is $15 ($25). In an unincentivized (stated
preference) survey of MTurk workers, Sunstein (2019) found a $1 per month median willingnessto-pay
for Facebook and a $59 per month median willingness-to-accept to not use Facebook.
There are many caveats to using this type of stylized calculation to approximate the consumer
surplus from Facebook. First, we (and Corrigan et al.) required participants to deactivate their
Facebook accounts instead of simply abstaining from logging in. For people who planned to avoid
using other apps with Facebook logins in order to avoid reactivating their Facebook accounts,
WTA overstates the value of Facebook access. Second, participants must believe the experimenter
will in fact enforce deactivation; WTA could naturally be lower for a partially enforced or unenforced
deactivation compared to an enforced deactivation. In some other studies, the method of
enforcement was either not made clear ex ante, or enforcement was not fully carried out ex post.33
32
Appendix Figure A37 compares our demand curve to the Brynjolfsson, Eggers, and Gannamaneni (2018) demand
curves.
33
Mosquera et al. told participants that they would “require” that they “not use their Facebook accounts” but did
not give additional details. Brynjolfsson et al.’s WTA elicitation stated that the experimenters “will randomly pick
1 out of every 200 respondents and her/his selection will be fulﬁlled,” and that they could enforce deactivation by
32
Third, any survey sample is unlikely to be representative of the Facebook user population on both
observable and unobservable characteristics. For example, we screened out people who reported
using Facebook 15 minutes or less per day, and while we re-weight the average WTAs to match
the average observables of Facebook users (including average daily usage), this re-weighting may
implicitly overstate the WTA of people who don’t use Facebook very much. Fourth, we (and all
other existing studies) estimate people’s Facebook valuations holding their networks ﬁxed. Due
to network externalities, valuations could be quite diﬀerent if participants’ friends and family also
deactivated. Fifth, one should be careful in annualizing these estimates or comparing WTAs for
diﬀerent durations of abstention, as our study and several others ﬁnd that the average per-day
valuation varies with the duration. Sixth, as we will see below, in practice people’s WTA may not
be closely held and could be easily anchored or manipulated, even in incentive compatible elicitations
such as ours. Finally, this calculation fails to speak to the possibility that people misperceive
Facebook’s value. We turn to that issue now.
6.2 How Deactivation Aﬀects Valuations
It is often argued that social media users do not correctly perceive the ways in which social media
could be addictive or make them unhappy. If this is the case, people’s willingness-to-accept to abstain
from Facebook would overstate “true” consumer surplus. For example, Alter (2018), Newport
(2019), many popular media articles (e.g. Ciaccia 2017; Oremus 2017), and organizations such as
the Center for Humane Technology and Time to Log Oﬀ argue that Facebook and other digital
technologies can be harmful and addictive. The Time to Log Oﬀ website argues that “everyone
is spending too much time on their screens” and runs “digital detox campaigns.” Sagioglu and
Greitemeyer (2014) document an “aﬀective forecasting error”: people predicted that spending 20
minutes on Facebook would make them feel better, but a treatment group randomly assigned to
20 minutes of Facebook browsing actually reported feeling worse.
Some of our results are also consistent with this argument. In the baseline survey, two-thirds
of people agreed at least somewhat that “if people spent less time on Facebook, they would soon
realize that they don’t miss it.” As reported earlier, about 80 percent of the Treatment group
thought that deactivation was good for them, and both qualitative and quantitative data suggest
that deactivation caused people to re-think and re-optimize their use.
The core of this argument is that people’s social media use does not maximize their utility, and
a “digital detox” might help them align social media demand with their own best interests. This
idea is related to several existing economic models. In a model of projection bias (Loewenstein,
O’Donoghue, and Rabin 2003), people might not correctly perceive that social media are habit
forming or that their preferences might otherwise change after a “digital detox.” In an experience
observing subjects’ time of last login, “given your permission.” In practice, the deactivation was mostly not enforced:
of the ten subjects randomly selected for enforcement, one gave permission.
33
good model, a “digital detox” might help consumers to learn their valuation of social media relative
to other uses of time. Of course, both of these mechanisms could also aﬀect demand after a period
of deactivation, so it is not clear whether the WTA before deactivation or after deactivation is more
normatively relevant.
To provide evidence on these issues, we elicited WTA at three points, as described earlier. First,
on the midline survey, we elicited WTA to deactivate Facebook in “weeks 1-4” (the four weeks after
midline). We call this WTA w1. Second, just after telling people their BDM oﬀer price on the
midline survey, and thus whether they were expected to deactivate in weeks 1-4, we elicited WTA
to deactivate in “weeks 5-8” (the four weeks after endline). We call this w2,1. Third, on the endline
survey, we elicited WTA to deactivate in weeks 5-8, after the Treatment group had experienced
deactivation in weeks 1-4, but the Control group had not. We call this w2,2.
The Control group’s change in WTA for weeks 5-8, ∆w2 := w2,2 − w2,1, captures any unpredicted
time eﬀect. The Treatment group’s WTA change ∆w2 reﬂects both the time eﬀect and
the unexpected change in valuation caused by deactivation. If the time eﬀect is the same in both
groups, then the diﬀerence in diﬀerences measures the eﬀect of deactivation on valuations due to
projection bias, learning, and similar mechanisms.
Figure 12 presents the average of WTA in Treatment and Control of w1, w2,1, and w2,2. Recall
that the impact evaluation sample includes only people with w1 < $102, so these averages are less
than the unconditional means discussed above and presented in Figure 11. Because of outliers in
the WTAs for weeks 5-8, we must winsorize WTA. We winsorize at $170 for this ﬁgure and our
primary regression estimates, as this is the upper bound of the distribution of BDM oﬀers that we
actually made for deactivation.
The Treatment group’s valuation for weeks 5-8 jumps substantially relative to its valuation
for weeks 1-4, while the Control group’s valuation for weeks 5-8 does not. We used open-answer
questions in the post-endline survey and qualitative interviews to understand this change. Some
of the large gap may be due to costs of deactivation being convex in the length of deactivation:
some people in the Treatment group wrote that they were much less comfortable deactivating for
eight weeks instead of four, as they would have to make much more extensive arrangements to
communicate with friends, co-workers, and schoolmates during a longer deactivation. However,
participants’ open-answer responses suggest that that the Treatment group’s WTA increase is
also aﬀected by anchoring on the $102 BDM oﬀer that was revealed after the elicitation of w1 but
before the elicitation of w2,1. Such anchoring is consistent with prior results showing that valuations
elicited using the BDM method can be aﬀected by suggested prices or other anchors (Bohm, Lind´en,
and Sonneg˚ard (1997), Mazar, Koszegi, and Ariely (2014)). Thus, we do not believe this increase
is relevant for a consumer welfare calculation, and we do not draw any substantive conclusion from
it.
Figure 12 also illustrates ∆w2, the change in valuation of weeks 5-8 between midline and endline.
34
The Control group’s valuation increases, reﬂecting an unpredicted time eﬀect. In open-answer
questions, some people wrote that they were less willing to deactivate during the Thanksgiving
holiday, and they may not have foreseen this as of the midline survey on October 11th. By contrast,
the Treatment group’s valuation for weeks 5-8 decreases. Thus, the diﬀerence in diﬀerences ∆w2
is negative.
We can estimate the diﬀerence in diﬀerences using the following regression:
∆w2,i = γDi + ρw1,i + νs + εi, (5)
instrumenting for Di with Ti. Table 6 presents results, winsorizing all WTAs at $170 in column
1 and at $1,000 in column 2. Relative to the Control group, the Treatment group reduced its
post-endline valuation by by $14 to $18, or about 14 percent of the Treatment group’s average
w2,1. This suggests that deactivation eliminated projection bias or facilitated learning that reduced
demand for Facebook by 14 percent. In turn, this suggests that the traditional estimates might
somewhat overstate consumer surplus.
This result is consistent with our ﬁnding in Section 5.4 that deactivation reduced post-experiment
Facebook use. However, because the WTA update ∆w2 is unexpected, it suggests that the results
from Section 5.4 may not be entirely explained by a “rational” habit formation model such as
Becker and Murphy (1988), in which people foresee how consumption aﬀects future marginal utility.
Instead, these results suggest that at least some of the reduced Facebook demand caused by
deactivation is driven by unexpected factors such as projection bias and learning.
One caveat is that the anchoring eﬀect described above could aﬀect our estimate of γ. If
anchoring has the same eﬀects on w2,1 and w2,2 in the Treatment group, then ∆w2 is unaﬀected,
and our estimate of γ is unbiased. If the anchoring eﬀects decay between midline and endline, this
would bias ˆγ away from zero, meaning that the true γ would be less than our estimate.34 This
would further strengthen our result that the valuation update caused by deactivation equals only
a small share of valuations.
One interpretation of these results is that they reinforce the standard model calculation that
Facebook generates many billions of dollars in consumer surplus. Another interpretation is that
they further highlight why standard consumer surplus calculations based on elicited valuations can
be problematic.
34
An alternative experimental design choice we considered was to elicit w2,1 before revealing the weeks 1-4 oﬀer
price, separately for the case in which the participant would be paid to deactivate for weeks 1-4 and the case in which
the participant would not be paid to deactivate. In this case, however, any anchoring eﬀect would have appeared on
w2,2 but not w2,1, generating an unambiguous spurious treatment eﬀect on ∆w2.
35
7 Conclusion
Our results leave little doubt that Facebook provides large beneﬁts for its users. Even after a four
week “detox,” our participants spent substantial time on Facebook every day and needed to be paid
large amounts of money to give up Facebook. Our results on news consumption and knowledge
suggest that Facebook is an important source of news and information. Our participants’ answers
in free response questions and follow-up interviews make clear the diverse ways in which Facebook
can improve people’s lives, whether as a source of entertainment, a means to organize a charity
or an activist group, or a vital social lifeline for those who are otherwise isolated. Any discussion
of social media’s downsides should not obscure the basic fact that it fulﬁlls deep and widespread
needs.
Notwithstanding, our results also make clear that the downsides are real. We ﬁnd that four
weeks without Facebook improves subjective well-being and substantially reduces post-experiment
demand, suggesting that forces such as addiction and projection bias may cause people to use
Facebook more than they otherwise would. We ﬁnd that while deactivation makes people less
informed, it also makes them less polarized by at least some measures, consistent with the concern
that social media have played some role in the recent rise of polarization in the US. The estimated
magnitudes imply that these negative eﬀects are large enough to be real concerns, but also smaller
in many cases than what one might have expected given prior research and popular discussion.
The trajectory of views on social media—with early optimism about great beneﬁts giving way
to alarm about possible harms—is a familiar one. Innovations from novels to TV to nuclear energy
have had similar trajectories. Along with the important existing work by other researchers, we
hope that our analysis can help move the discussion from simplistic caricatures to hard evidence,
and provide a sober assessment of the way a new technology aﬀects both individual people and
larger social institutions.
36
References
Abeele, Mariek M. P. Vanden, Marjolijn L. Antheunis, Monique M. H. Pollmann, Alexander P.
Schouten, Christine C. Liebrecht, Per J. van der Wijst, Marije A. A. van Amelsvoort, Jos Bartels,
Emiel J. Krahmer, and Fons A. Maes. 2018. “Does Facebook Use Predict College Students’ Social
Capital? A Replication of Ellison, Steinﬁeld, and Lampe’s (2007) Study Using the Original and
More Recent Measures of Facebook Use and Social Capital.” Communication Studies 69 (3):272–
282.
Acland, Dan and Matthew Levy. 2015. “Naivete, Projection Bias, and Habit Formation in Gym
Attendance.” Management Science 61 (1):146–160.
Allcott, Hunt and Matthew Gentzkow. 2017. “Social Media and Fake News in the 2016 Election.”
Journal of Economic Perspectives 31 (2):211–236.
Alter, Adam. 2018. Irresistible: The Rise of Addictive Technology and the Business of Keeping Us
Hooked. Penguin Random House.
American National Election Studies. 2016. “2016 Time Series Study.”
https://electionstudies.org/data-center/2016-time-series-study/.
Anderson, Michael L. 2008. “Multiple Inference and Gender Diﬀerences in the Eﬀects of Early
Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects.”
Journal of the American Statistical Association 103 (484):1481–1495.
Appel, Helmut, Alexander L. Gerlach, and Jan Crusius. 2016. “The Interplay Between Facebook
Use, Social Comparison, Envy, and Depression.” Current Opinion in Psychology 9:44–49.
Argyle, Michael. 2001. The Psychology of Happiness. Routledge.
Athey, Susan, Julie Tibshirani, and Stefan Wager. 2019. “Generalized Random Forests.” Annals
of Statistics 47 (2):1148–1178.
Baker, David A. and Guillermo Perez Algorta. 2016. “The Relationship Between Online Social
Networking and Depression: A Systematic Review of Quantitative Studies.” Cyberpsychology,
Behavior, and Social Networking 19 (11):638–648.
Bakshy, Eytan, Solomon Messing, and Lada Adamic. 2015. “Exposure to Ideologically Diverse
News and Opinion on Facebook.” Science 348 (6239):1130–1132.
Bartels, Larry M. 1993. “Messages Received: The Political Impact of Media Exposure.” American
Political Science Review 87 (2):267–285.
37
Baym, Nancy K., Kelly B. Wagman, and Christopher J. Persaud. 2019. “Mindfully Scrolling:
Rethinking Facebook after Time Deactivated.” In Media in Transition 10: Democracy and
Digital Media. Forthcoming May 2019.
Becker, Gary S., Michael Grossman, and Kevin M. Murphy. 1991. “Rational Addiction and the
Eﬀect of Price on Consumption.” The American Economic Review 81 (2):237–241.
Becker, Gary S. and Kevin M. Murphy. 1988. “A Theory of Rational Addiction.” Journal of
Political Economy 96 (4):675–700.
Becker, Gordon M., Morris H. DeGroot, and Jacob Marschak. 1964. “Measuring Utility by a SingleResponse
Sequential Method.” Economic Information, Decision, and Prediction 9 (3):226–232.
Belli, Robert F., Michael W. Traugott, Margaret Young, and Katherine A. McGonagle. 1999. “Reducing
Vote Overreporting in Surveys: Social Desirability, Memory Failure, and Source Monitoring.”
Public Opinion Quarterly 63.
Benjamin, Daniel J., Ori Heﬀetz, Miles S. Kimball, and Alex Rees-Jones. 2012. “What Do You
Think Would Make You Happier? What Do You Think You Would Choose?” American Economic
Review 102 (5):2083–2110.
Benjamini, Yoav, Abba M. Krieger, and Daniel Yekutieli. 2006. “Adaptive linear step-up procedures
that control the false discovery rate.” Biometrika 93 (3):491–507.
Besley, Timothy and Robin Burgess. 2001. “Political Agency, Government Responsiveness and the
Role of the Media.” European Economic Review 45 (4-6):629–640.
Bohm, Peter, Johan Lind´en, and Joakim Sonneg˚ard. 1997. “Eliciting Reservation Prices: Becker–
DeGroot–Marschak Mechanisms vs. Markets.” The Economic Journal 107 (443):1079–1089.
Bolier, Linda, Merel Haverman, Gerben J. Westerhof, Heleen Riper, Filip Smit, and Ernst Bohlmeijer.
2013. “Positive Psychology Interventions: A Meta-Analysis of Randomized Controlled Studies.”
BMC Public Health 13 (1):119.
Boxell, Levi. 2018. “Demographic Change and Political Polarization in the United States.” Available
at SSRN 3148805.
Brynjolfsson, Erik, Felix Eggers, and Avinash Gannamaneni. 2018. “Using Massive Online Choice
Experiments to Measure Changes in Well-being.” NBER Working Paper.
Brynjolfsson, Erik and Joo Hee Oh. 2012. “The Attention Economy: Measuring the Value of
Free Digital Services on the Internet.” In Proceedings of the 33rd International Conference on
Information Systems.
38
Brynjolfsson, Erik, Daniel Rock, and Chad Syverson. 2018. “Artiﬁcial intelligence and the modern
productivity paradox: A clash of expectations and statistics.” In The Economics of Artiﬁcial
Intelligence: An Agenda. University of Chicago Press.
Brynjolfsson, Erik and Adam Saunders. 2009. “What the GDP Gets Wrong (Why Managers Should
Care).” MIT Sloan Management Review 51 (1):95.
Burke, Moira and Robert E. Kraut. 2014. “Growing Closer on Facebook: Changes in Tie Strength
Through Social Network Site Use.” In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems. 4187–4196.
———. 2016. “The Relationship Between Facebook Use and Well-Being Depends on Communication
Type and Tie Strength.” Journal of Computer-Mediated Communication 21 (4):265–281.
Burke, Moira, Robert E. Kraut, and Cameron Marlow. 2011. “Social Capital on Facebook: Differentiating
Uses and Users.” In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems. 571–580.
Burke, Moira, Cameron Marlow, and Thomas Lento. 2010. “Social Network Activity and Social
Well-Being.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
1909–1912.
Busse, Meghan R., Devin G. Pope, Jaren C. Pope, and Jorge Silva-Risso. 2015. “The Psychological
Eﬀect of Weather on Car Purchases.” Quarterly Journal of Economics 130 (1):371–414.
Byrne, David M., John G. Fernald, and Marshall B. Reinsdorf. 2016. “Does the United States have
a productivity slowdown or a measurement problem?” Brookings Papers on Economic Activity
2016 (1):109–182.
Charness, Gary and Uri Gneezy. 2009. “Incentives to Exercise.” Econometrica 77 (3):909–931.
Chopik, William J. 2017. “Associations among relational values, support, health, and well-being
across the adult lifespan.” Personal Relationships 24 (2):408–422.
Ciaccia, Chris. 2017. “Facebook, cocaine, opioids: How addictive is the social network?” Fox News.
https://www.foxnews.com/tech/facebook-cocaine-opioids-how-addictive-is-the-social-network.
Conlin, Michael, Ted O’Donoghue, and Timothy J. Vogelsang. 2007. “Projection Bias in Catalog
Orders.” American Economic Review 97 (4):1217–1249.
Corrigan, Jay R., Saleem Alhabash, Matthew Rousu, and Sean B. Cash. 2018. “How much is social
media worth? Estimating the value of Facebook by paying users to stop using it.” PloS one
13 (12):e0207101.
39
Csikszentmihalyi, Mihaly and Reed Larson. 2014. “Validity and Reliability of the ExperienceSampling
Method.” In Flow and the Foundations of Positive Psychology. Springer, 35–54.
DellaVigna, Stefano and Matthew Gentzkow. 2010. “Persuasion: Empirical Evidence.” Annual
Review of Economics 2 (1):643–669.
DellaVigna, Stefano and Ethan Kaplan. 2007. “The Fox News Eﬀect: Media Bias and Voting.”
Quarterly Journal of Economics 122 (3):1187–1234.
DellaVigna, Stefano and Eliana La Ferrara. 2015. “Economic and Social Impacts of the Media.”
In Handbook of Media Economics, vol. 1. Elsevier, 723–768.
DellaVigna, Stefano, John A. List, Ulrike Malmendier, and Gautam Rao. 2017. “Voting to Tell
Others.” Review of Economic Studies 84.
Deters, Fenne Große and Matthias R. Mehl. 2012. “Does Posting Facebook Status Updates Increase
or Decrease Loneliness? An Online Social Networking Experiment.” Social Psychological and
Personality Science 4 (5):579–586.
Diener, Ed, Robert A. Emmons, Randy J. Larsen, and Sharon Griﬃn. 1985. “The Satisfaction
With Life Scale.” Journal of Personality Assessment 49 (1):71–75.
Duﬀ, Brian, Michael J. Hanmer, Won-Ho Park, and Ismail K. White. 2007. “Good Excuses:
Understanding who Votes with an Improved Turnout Question.” Public Opinion Quarterly 71 (1).
Ellison, Nicole B., Charles Steinﬁeld, and Cliﬀ Lampe. 2007. “The Beneﬁts of Facebook “Friends:”
Social Capital and College Students’ Use of Online Social Network Sites.” Journal of ComputerMediated
Communication 12 (4):1143–1168.
Enikolopov, Ruben, Alexey Makarin, and Maria Petrova. 2018. “Social Media and Protest Participation:
Evidence from Russia.” SSRN Working Paper.
Enikolopov, Ruben and Maria Petrova. 2015. “Media Capture: Empirical Evidence.” In Handbook
of Media Economics, vol. 1. Elsevier, 687–700.
Enikolopov, Ruben, Maria Petrova, and Ekaterina Zhuravskaya. 2011. “Media and Political Persuasion:
Evidence from Russia.” American Economic Review 101 (7):3253–85.
Facebook. 2016. “Facebook Q1 2016 Results - Earnings Call Transcript.”
https://seekingalpha.com/article/3968783-facebook-fb-mark-elliot-zuckerberg-q1-2016-results-
earnings-call-transcript.
40
———. 2018. “Facebook Reports Third Quarter 2018 Results.” Press Release.
https://investor.fb.com/investor-news/press-release-details/2018/Facebook-Reports-Third-
Quarter-2018-Results/default.aspx.
Fardouly, Jasmine and Lenny R. Vartanian. 2015. “Negative Comparisons About One’s Appearance
Mediate the Relationship Between Facebook Usage and Body Image Concerns.” Body Image
12:82–88.
Fiorina, Morris P. and Samuel J. Abrams. 2008. “Political Polarization in the American Public.”
Annual Review Political Science 11:563–588.
Frison, Eline and Steven Eggermont. 2015. “Toward an Integrated and Diﬀerential Approach to the
Relationships Between Loneliness, Diﬀerent Types of Facebook Use, and Adolescents’ Depressed
Mood.” Communication Research. doi:10.1177/0093650215617506.
Fujiwara, Thomas, Kyle Meng, and Tom Vogl. 2016. “Habit Formation in Voting: Evidence from
Rainy Elections.” American Economic Journal: Applied Economics 8 (4):160–188.
Gallup. 2018a. “Americans’ Views of Donald Trump’s Personal Characteristics (Trends).”
https://news.gallup.com/poll/235916/americans-views-donald-trump-personal-characteristics-
trends.aspx.
———. 2018b. “Opposition to Kavanaugh Had Been Rising Before Accusation.”
https://news.gallup.com/poll/242300/opposition-kavanaugh-rising-accusation.aspx.
———. 2018c. “Ten Takeaways About Americans’ Views of Guns.”
https://news.gallup.com/opinion/polling-matters/233627/ten-takeaways-americans-views-
guns.aspx.
Gentzkow, Matthew. 2006. “Television and Voter Turnout.” Quarterly Journal of Economics
121 (3):931–972.
———. 2016. “Polarization in 2016.” Toulouse Network for Information Technology Whitepaper.
Gentzkow, Matthew and Jesse Shapiro. 2011. “Ideological Segregation Online and Oﬄine.” Quarterly
Journal of Economics 126 (4):1799–1839.
Gerber, Alan S., James G. Gimpel, Donald P. Green, and Daron R. Shaw. 2011. “How Large and
Long-lasting Are the Persuasive Eﬀects of Televised Campaign Ads? Results from a Randomized
Field Experiment.” American Political Science Review 105 (1):135–150.
Gerber, Alan S. and Donald P. Green. 2000. “The Eﬀects of Canvassing, Telephone Calls, and Direct
Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94 (3):653–663.
41
Gerber, Alan S., Dean Karlan, and Daniel Bergan. 2009. “Does the Media Matter? A Field
Experiment Measuring the Eﬀect of Newspapers on Voting Behavior and Political Opinions.”
American Economic Journal: Applied Economics 1 (2):35–52.
Gerson, Jennifer, Anke C. Plagnol, and J. Corr, Philip. 2017. “Passive and Active Facebook
Use Measure (PAUM): Validation and Relationship to the Reinforcement Sensitivity Theory.”
Personality and Individual Diﬀerences 117:81–90.
Gonzales, Amy L. and Jeﬀrey T. Hancock. 2011. “Mirror, Mirror on my Facebook Wall: Eﬀects
of Exposure to Facebook on Self-Esteem.” Cyberpsychology, Behavior, and Social Networking
14 (1-2):79–83.
Gruber, Jonathan and Botond K¨oszegi. 2001. “Is Addiction “Rational”? Theory and Evidence.”
Quarterly Journal of Economics 116 (4):1261–1303.
Howard, Philip N., Aiden Duﬀy, Deen Freelon, Muzammil M. Hussain, Will Mari, and Marwa
Maziad. 2011. “Opening Closed Regimes: What Was the Role of Social Media During the Arab
Spring?” Available at SSRN 2595096.
Huber, Gregory A. and Kevin Arceneaux. 2007. “Identifying the Persuasive Eﬀects of Presidential
Advertising.” American Journal of Political Science 51 (4):957–977.
Hughes, Mary Elizabeth, Linda J. Waite, Louise C. Hawkley, and John T. Cacioppo. 2004. “A
Short Scale for Measuring Loneliness in Large Surveys: Results From Two Population-Based
Studies.” Research on Aging 26 (6):655–672.
Hunt, Melissa G., Rachel Marx, Courtney Lipson, and Jordyn Young. 2018. “No More FOMO:
Limiting Social Media Decreases Loneliness and Depression.” Journal of Social and Clinical
Psychology 37 (10):751–768.
Huppert, Felicia A., Nic Marks, Andrew Clark, Johannes Siegrist, Alois Stutzer, Joar Vittersø,
and Morten Wahrendorf. 2009. “Measuring Well-being Across Europe: Description of the ESS
Well-being Module and Preliminary Findings.” Social Indicators Research 91:301–315.
Hussam, Reshmaan, Atonu Rabbani, Giovanni Reggiani, and Natalia Rigol. 2016. “Handwashing
and Habit Formation.” Yale University, Economic Growth Center.
Irvine, Mark. 2018. “Facebook Ad Benchmarks for YOUR Industry.”
https://www.wordstream.com/blog/ws/2017/02/28/facebook-advertising-benchmarks.
Iyengar, Shanto, Gaurav Sood, and Yphtach Lelkes. 2012. “Aﬀect, Not Ideology: Social Identity
Perspective on Polarization.” Public Opinion Quarterly 76 (3):405–431.
42
Iyengar, Shanto and Sean J. Westwood. 2015. “Fear and Loathing across Party Lines: New Evidence
on Group Polarization.” American Journal of Political Science 59 (3):690–707.
Kahneman, Daniel, Alan B. Krueger, David Schkade, Norbert Schwarz, and Arthur A. Stone. 2006.
“Would You Be Happier If You Were Richer? A Focusing Illusion.” Science 312:1908–1910.
Kirkpatrick, David. 2011. The Facebook Eﬀect: The Inside Story of the Company That Is Connecting
the World. Simon & Schuster.
Krasnova, Hannah, H. Wenninger, T. Widjaja, and Peter Buxmann. 2013. “Envy on Facebook: A
Hidden Threat to Users’ Life Satisfaction.” In 11th International Conference on Wirtschaftsin-
formatik.
Kross, Ethan, Philippe Verduyn, Emre Demiralp, Jiyoung Park, David Seungjae Lee, Natalie Lin,
Holly Shablack, John Jonides, and Oscar Ybarra. 2013. “Facebook Use Predicts Declines in
Subjective Well-being in Young Adults.” PloS one 8 (8):e69841.
Lee, David Seungjae. 2009. “Training, Wages, and Sample Selection: Estimating Sharp Bounds on
Treatment Eﬀects.” The Review of Economic Studies 76 (3):1071–1102.
Loewenstein, George, Ted O’Donoghue, and Matthew Rabin. 2003. “Projection Bias in Predicting
Future Utility.” Quarterly Journal of Economics 118 (4):1209–1248.
Lyubomirsky, Sonja and Heidi S. Lepper. 1999. “A Measure of Subjective Happiness: Preliminary
Reliability and Construct Validation.” Social Indicators Research 46:137–155.
Mabe, Annalise G., K. Jean Forney, and Pamela K. Keel. 2014. “Do You “Like” My Photo? Facebook
Use Maintains Eating Disorder Risk.” International Journal of Eating Disorders 47 (5):516–
523.
Marotta, Veronica and Alessandro Acquisti. 2017. “Online Distractions, Website Blockers, and
Economic Productivity: A Randomized Field Experiment.” Preliminary Draft.
Martin, Gregory J. and Ali Yurukoglu. 2017. “Bias in Cable News: Persuasion and Polarization.”
American Economic Review 107 (9):2565–99.
Mazar, Nina, Botond Koszegi, and Dan Ariely. 2014. “True Context-dependent Preferences? The
Causes of Market-dependent Valuations.” Journal of Behavioral Decision Making 27 (3):200–208.
Molla, Rani and Kurt Wagner. 2018. “People spend almost as much time on Instagram as they
do on Facebook.” Recode. https://www.recode.net/2018/6/25/17501224/instagram-facebook-
snapchat-time-spent-growth-data.
43
Mosquera, Roberto, Moﬁoluwasademi Odunowo, Trent McNamara, Xiongfei Guo, and Ragan
Petrie. 2018. “The Economic Eﬀects of Facebook.” Available at SSRN 3312462.
Muller, Karsten and Carlo Schwarz. 2018. “Fanning the Flames of Hate: Social Media and Hate
Crime.” SSRN Working Paper.
Myers, David G. 2000. “The Funds, Friends, and Faith of Happy People.” American Psychologist
55 (1):56.
Nakamura, Leonard I., Jon Samuels, and Rachel H. Soloveichik. 2016. “Valuing “Free” Media in
GDP: An Experimental Approach.” FRB of Philadelphia Working Paper No. 16-24.
Napoli, Philip M. 2014. “Measuring Media Impact.” The Norman Lear Center.
http://www.learcenter.org/pdf/measuringmedia.pdf.
Newport, Cal. 2019. Digital Minimalism: Choosing a Focused Life in a Noisy World. Penguin
Random House.
Olken, Benjamin A. 2009. “Do Television and Radio Destroy Social Capital? Evidence from
Indonesian Villages.” American Economic Journal: Applied Economics 1 (4):1–33.
Oremus, Will. 2017. “Addiction for Fun and Proﬁt.” Slate.
https://slate.com/technology/2017/11/facebook-was-designed-to-be-addictive-does-that-make-
it-evil.html.
Pew Research Center. 2016. “On Views of Race and Inequality, Blacks and Whites Are
Worlds Apart.” http://www.pewsocialtrends.org/2016/06/27/on-views-of-race-and-inequality-
blacks-and-whites-are-worlds-apart/.
———. 2017. “American Trends Panel, Wave 25 March, Final Topline, March 13 - March 27,
2017.” http://www.journalism.org/wp-content/uploads/sites/8/2017/05/PJ 2017.05.10 mediaattitudes
topline.pdf.
———. 2018a. “News Use Across Social Media Platforms 2018.”
http://www.journalism.org/2018/09/10/news-use-across-social-media-platforms-2018/.
———. 2018b. “Political Survey, Final Topline, April 25 - May 1, 2018.”
http://assets.pewresearch.org/wp-content/uploads/sites/5/2018/05/10094104/05-10-18-foreign-
policy-topline-for-release.pdf.
———. 2018c. “Public Is Skeptical of the Iran Agreement - and Trump’s Handling of the
Issue.” http://www.people-press.org/2018/05/08/public-is-skeptical-of-the-iran-agreement-and-
trumps-handling-of-the-issue/.
44
———. 2018d. “Sexual Harassment at Work in the Era of #MeToo.”
http://www.pewsocialtrends.org/2018/04/04/sexual-harassment-at-work-in-the-era-of-metoo/.
———. 2018e. “Shifting Public Views on Legal Immigration Into the U.S.” http://www.people-
press.org/2018/06/28/shifting-public-views-on-legal-immigration-into-the-u-s/.
———. 2018f. “Social Media Use in 2018.” http://www.pewinternet.org/2018/03/01/social-media-
use-in-2018/.
Reis, Harry T., W. Andrew Collins, and Ellen Berscheid. 2000. “The Relationship Context of
Human Behavior and Development.” Psychological Bulletin 126 (6):844.
Sagioglu, Christina and Tobias Greitemeyer. 2014. “Facebook’s emotional consequences: Why
Facebook causes a decrease in mood and why people still use it.” Computers in Human Behavior
35:359–363.
Satici, Seydi Ahmet and Recep Uysal. 2015. “Well-being and problematic Facebook use.” Computers
in Human Behavior 49:185–190.
Settle, Jamie E. 2018. Frenemies: How Social Media Polarizes America. Cambridge University
Press.
Shakya, Holly B. and Nicholas A. Christakis. 2017. “Association of Facebook Use With Compromised
Well-being: A Longitudinal Study.” American Journal of Epidemiology 185 (3):203–211.
Silver, Brian D., Barbara A. Anderson, and Paul R. Abramson. 1986. “Who Overreports Voting?”
American Political Science Review 80 (2).
Simonsohn, Uri. 2010. “Weather to go to college.” The Economic Journal 120 (543):270–280.
Spenkuch, J¨org L and David Toniatti. 2016. “Political advertising and election outcomes.” CESifo
Working Paper Series 5780.
Stone, Arthur A. and Saul Shiﬀman. 1994. “Ecological Momentary Assessment (EMA) in Behavorial
Medicine.” Annals of Behavioral Medicine 16 (3):199–202.
Str¨omberg, David. 2015. “Media and Politics.” Annual Review of Economics 7 (1):173–205.
Sunstein, Cass. 2001. Republic.com. Princeton University Press.
———. 2017. Republic: Divided Democracy in the Age of Social Media. Princeton University Press.
———. 2019. “Valuing Facebook.” Behavioural Public Policy Forthcoming.
45
Syverson, Chad. 2017. “Challenges to Mismeasurement Explanations for the US Productivity
Slowdown.” Journal of Economic Perspectives 31 (2):165–86.
Tandoc, Edson C., Patrick Ferrucci, and Margaret Duﬀy. 2015. “Facebook use, envy, and depression
among college students: Is facebooking depressing?” Computers in Human Behavior 43:139–146.
Theocharis, Yannis and Will Lowe. 2016. “Does Facebook increase political participation? Evidence
from a ﬁeld experiment.” Information, Communication, and Society 19 (10):1465–1486.
Tromholt, Morten. 2016. “The Facebook Experiment: Quitting Facebook Leads to Higher Levels
of Well-Being.” Cyberpsychology, Behavior, and Social Networking 19 (11):661–666.
Twenge, Jean M. 2017. iGen: Why Today’s Super-Connected Kids Are Growing Up Less Rebellious,
More Tolerant, Less Happy–and Completely Unprepared for Adulthood–and What That Means for
the Rest of Us. Simon and Schuster.
Twenge, Jean M., Thomas E. Joiner, Megan L. Rogers, and Gabrielle N. Martin. 2018. “Increases
in Depressive Symptoms, Suicide-Related Outcomes, and Suicide Rates Among U.S. Adolescents
After 2010 and Links to Increased New Media Screen Time.” Clinical Psychological Science
6 (1):3–17.
Twenge, Jean M., Gabrielle N. Martin, and W. Keith Campbell. 2018. “Decreases in psychological
well-being among American adolescents after 2012 and links to screen time during the rise of
smartphone technology.” Emotion 18 (6):765–780.
Twenge, Jean M. and Heejung Park. 2017. “The Decline in Adult Activities Among U.S. Adolescents,
1976–2016.” Child Development. Advance online publication. doi:10.1111/cdev.12930.
Twenge, Jean M., Ryne A. Sherman, and Sonja Lyubomirsky. 2016. “More Happiness for Young
People and Less for Mature Adults: Time Period Diﬀerences in Subjective Well-Being in the
United States, 1972–2014.” Social Psychological and Personality Science 7 (2):131–141.
United States Census Bureau. 2017. “2017 American Community Survey 1-Year Estimates.”
https://www.census.gov/newsroom/press-kits/2018/acs-1year.html.
———. 2018. “Voting and Registration in the Election of November 2018.”
https://www.census.gov/data/tables/time-series/demo/voting-and-registration/p20-583.html.
Vanman, Eric J., Rosemary Baker, and Stephanie J. Tobin. 2018. “The Burden of Online Friends:
The Eﬀects of Giving Up Facebook on Stress and Well-Being.” The Journal of Social Psychology
158 (4):496–507.
46
Verduyn, Phuluppe, David Seungjae Lee, Jiyoung Park, Holly Shablack, Ariana Orvell, Joseph
Bayer, Oscar Ybarra, John Jonides, and Ethan Kross. 2015. “Passive Facebook Usage Undermines
Aﬀective Well-Being: Experimental and Longitudinal Evidence.” Journal of Experimental
Psychology 144 (2):480–488.
47
Table 1: Sample Sizes
Phase Sample size
Recruitment N=1,892,191 were shown ads
and baseline N=32,201 clicked on ads
N=22,324 completed pre-screen survey
N=20,959 were from US and born between 1900 and 2000
N=17,335 had 15 < daily Facebook minutes ≤ 600
N=7,455 consented to participate
N=3,910 ﬁnished baseline
N=2,897 had valid baseline and were randomized, of which:
Midline N=2,897 began midline
N=2,743 received a price oﬀer, of which:
N=1,661 were in impact evaluation sample
Endline N=2,710 began endline
N=2,684 ﬁnished endline, of which:
N=1,637 were in impact evaluation sample
Post-endline N=2,067 reported Facebook mobile app use, of which:
N=1,219 were in impact evaluation sample
Table 2: Sample Demographics
(1) (2) (3)
Impact
evaluation
sample
Facebook
users
US
population
Income under $50,000 0.40 0.41 0.42
College 0.51 0.33 0.29
Male 0.43 0.44 0.49
White 0.68 0.73 0.74
Age under 30 0.52 0.26 0.21
Republican 0.13 0.26
Democrat 0.42 0.20
Facebook minutes 74.52 45.00
Notes: Column 1 presents average demographics for the impact evaluation sample: participants who were
willing to accept less than $102 to deactivate Facebook for the four weeks after midline and were oﬀered
p = $102 or p = $0 to do so. Column 2 presents our estimate of average demographics of American adults
with a Facebook account. The top ﬁve numbers in Column 2 are inferred from a Pew Research Center
(2018f) survey of social media use by demographic group. The bottom number in Column 2 (the average
of 45 minutes of Facebook use per day) is approximated on those basis of sources such as Facebook (2016)
and Molla and Wagner (2018). Column 3 presents average demographics of American adults. The top ﬁve
numbers are from the 2017 American Community Survey (United States Census Bureau 2017), and the
Republican and Democrat shares are from the 2016 American National Election Study (American National
Election Studies 2016).
48
Table 3: Survey Response and Treatment Compliance Rates
(1) (2) T-test
Treatment Control P-value
Variable Mean/SD Mean/SD (1)-(2)
Completed endline survey 0.99
(0.11)
0.98
(0.12)
0.54
Share of text messages completed 0.92
(0.20)
0.93
(0.18)
0.45
Completed post-endline survey 0.95
(0.23)
0.92
(0.26)
0.07*
Share days deactivated 0.90
(0.29)
0.02
(0.13)
0.00***
N 580 1081
Notes: Columns 1 and 2 present survey response and treatment compliance rates for the Treatment and
Control groups in the impact evaluation sample: participants who were willing to accept less than $102 to
deactivate Facebook for the four weeks after midline and were oﬀered p = $102 or p = $0 to do so. Column
3 presents p-values of tests of diﬀerences in response rates between the two groups.
49
Table 4: Most Common Descriptions of Facebook Use Changes
Phrases Used More Often by Treatment
Phrase % Treatment % Control
not use facebook anymor 0.90 0
not spend much time 1.08 0.36
spend less time facebook 0.90 0.27
have not use facebook 0.72 0.18
not use facebook much 0.72 0.18
spend lot less time 0.72 0.27
use much less 2.87 0.63
deﬁnit use facebook less 0.54 0.18
use facebook lot less 0.54 0.18
use facebook much less 0.54 0.18
not use facebook 3.05 1.17
use littl bit less 0.54 0.27
have not use 1.25 0.18
ha not chang use 0.72 0.45
use facebook anymor 0.90 0.09
think use less 1.61 0.45
no ha not chang 0.54 0.36
use news app 0.72 0.09
still have not 0.90 0.18
much less 4.84 1.17
Phrases Used More Often by Control
Phrase % Treatment % Control
ha not chang 6.63 16.76
not chang sinc particip 0 0.99
ha not chang sinc 0.18 1.53
chang sinc particip studi 0 0.81
way use facebook ha 0.18 1.35
usag ha not chang 0 0.72
chang way use facebook 0.18 1.26
not chang 7.17 18.65
awar much time spend 0 0.63
ha not 8.24 19.64
not much ha chang 0 0.54
way use facebook 0.54 2.70
not think chang much 0 0.45
not chang much use 0 0.45
use facebook slightli less 0 0.45
more awar much time 0.18 0.99
chang sinc particip 0 1.08
much time spend 0.18 1.53
facebook ha not chang 0.72 2.07
use slightli less 0 0.90
Notes: The post-endline survey included the following question with an open response text box: “How
has the way you use Facebook changed, if at all, since participating in this study?” For all responses, we
stemmed words, ﬁltered out stop words, then constructed all phrases of length l = {1, 2, 3, 4} words. For
each phrase p of length l, we calculated the number of occurrences of that phrase in Treatment and Control
group responses (fplT and fplC) and the number of occurrences of length-l phrases that are not phrase p in
Treatment and Control responses (f∼plT and f∼plC). We then constructed Pearson’s χ2
statistic:
χ2
=
(fplT f∼plC − fplCf∼plT )
2
(fplT + fplC) (fplT + f∼plT ) (fplC + f∼plC) (f∼plT + f∼plC)
.
This table presents the 20 phrases with the highest χ2
that were most commonly written by the Treatment
and Control groups. The % Treatment and % Control columns present the share of people in the respective
group whose responses included each phrase.
50
Table 5: Perceived Researcher Agenda in Treatment and Control
(1) (2) T-test
Treatment Control P-value
Variable Mean/SD Mean/SD (1)-(2)
I don’t think they had a particular agenda 0.43
(0.49)
0.44
(0.50)
0.59
Yes, wanted to show that Facebook is good for people 0.03
(0.18)
0.04
(0.19)
0.79
Yes, wanted to show that Facebook is bad for people 0.35
(0.48)
0.35
(0.48)
0.79
I am not sure 0.19
(0.39)
0.18
(0.38)
0.62
N 573 1064
Notes: The endline survey asked, “Do you think the researchers in this study had an agenda?” Columns 1
and 2 present the share of the Treatment and Control groups who gave each possible response. Column 3
presents p-values of tests of diﬀerences in means between the two groups.
Table 6: Change in Facebook Valuation after Deactivation
(1) (2)
Share of time deactivated -14.36 -18.22
(2.60) (7.73)
Observations 1,634 1,634
Winsorized maximum WTA 170 1,000
Treatment mean weeks 5-8 WTA at midline 103 135
Notes: This table presents estimates of Equation (5). The dependent variable is the change in WTA for
post-endline deactivation measured at endline versus midline. Standard errors are in parentheses.
51
Figure 1: Experimental Design
Recruitment, pre-screen, and baseline
September 24 – October 3
Midline
October 11
Endline
November 8
Post-endline
December 3
Dailytextmessages
September27–November8
𝑝′ = 0 𝑝′ ∈ [0,170]
~ 0.2%
𝑝 = 102
“Treatment”
if WTA<$102
𝑝 = 0
“Control”
if WTA<$102
𝑝 ∈ [0,170]
~ 99.8%
~ 0.2%
~67%
~33%
52
Figure 2: Substitutes for Facebook
Substitute
timeuses
Social
interaction
Substitute
newssources Facebook minutes
Non-FB social media time
Non-social online time
TV alone time
Non-screen alone time
Friends and family time
Substitute time uses index
Friends met in person
Ofﬂine activities
Diverse interactions
Social interaction index
Facebook news
Number of tweets
Non-FB social media news
Non-social online news
Local TV news
Network TV news
Cable TV news
Print news
Radio news
Substitute news sources index
-2 -1.5 -1 -.5 0 .5
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation
of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions. Facebook
minutes is not included in the substitute time uses index, and Facebook news is not included in the substitute
news sources index, so we visually separate these two variables from the other variables in their respective
families. We also visually separate online and oﬄine time uses and news sources, although all online and
oﬄine substitutes enter their respective indexes.
53
Figure 3: Eﬀects on News and Political Outcomes
News
knowledge
Political
engagement
Political
polarization Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.3 -.2 -.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
54
Figure 4: Issue Opinions by Party at Endline
0.2.4.6
Density
-4 -2 0 2 4
Issue opinions (in units of Control group standard deviations)
Treatment Democrat Treatment Republican
Control Democrat Control Republican
kernel = epanechnikov, bandwidth = 0.2231
Notes: This ﬁgure presents kernel density plots of issue opinions for Democrats and Republicans in Treatment
and Control at endline. Issue opinions are attitudes about nine current political issues on a scale from -5 to
+5, such as “To what extent do you think that free trade agreements between the US and other countries
have been a good thing or a bad thing for the United States.” See Appendix B.1 for a list of all nine issue
questions. To construct the issue opinions measure, for each issue question q, we normalize responses by
the standard deviation in the Control group, determine Democrats’ and Republicans’ average responses µD
q
and µR
q , re-center so that µD
q + µR
q = 0, and re-sign so that µR
> 0. Deﬁne ˜yiq as individual i’s normalized,
re-centered, and re-signed response to question q. ˜yiq thus reﬂects the strength of individual i’s agreement
with the average Republican. Deﬁne σq as the Control group within-person standard deviation of ˜yiq for
question q. This measures how much people’s views change between baseline and endline, and allows us to
place higher weight on issues about which views are malleable over the deactivation period. The preliminary
issue opinion measure is Yi = q ˜yiqσq, and the ﬁnal issue opinion measure plotted in the ﬁgure is Yi
divided by the Control group standard deviation.
55
Figure 5: Eﬀects on Subjective Well-Being
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
56
Figure 6: Eﬀects on Post-Experiment Facebook Use and Opinions
Post-experiment
use
Facebook
opinions
Planned post-study use change
Clicked time limit email × (-1)
Speed of reactivation
Facebook mobile app use
Post-experiment use index
Improves social life
Good for you
Good for society
Makes people happy
Less polarized
Helps follow news
Clickbait, fake news × (-1)
People would miss Facebook
Deactivation bad
Positive impacts
Negative impacts × (-1)
Facebook opinions index
-1 -.5 0 .5
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
57
Figure 7: Probability of Being Deactivated
0.2.4.6.81
Sharedeactivated
0 50 100
Days after beginning of deactivation period
Midline/Endline 24hr periods Control Treatment
Notes: This ﬁgure shows the share of the Treatment and Control groups that had their Facebook accounts
deactivated, by day of the experiment, for the impact evaluation sample: participants who were willing to
accept less than $102 to deactivate Facebook for the four weeks after midline and were oﬀered p = $102 or
p = $0 to do so. The vertical gray areas reﬂect the 24 hour periods after midline and endline during which
both Treatment and Control were instructed to deactivate.
58
Figure 8: Key Opinions about Facebook in Treatment and Control
0.05.1.15.2.25
Density
-5 0 5
People would miss Facebook
Control Treatment
0.05.1.15.2.25
Density
-5 0 5
Deactivation bad
Control Treatment
Notes: This ﬁgure presents the distribution of responses in Treatment and Control for two key measures of
opinions about Facebook. See Section 2.3 for variable deﬁnitions.
59
Figure 9: Heterogeneous Treatment Eﬀects
Substitute time uses index
Social interaction index
Substitute news sources index
News knowledge index
Political engagement index
Political polarization index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-1 -.5 0 .5
Treatment effect (standard deviations)
Light users Heavy users
News knowledge index
Political engagement index
Political polarization index
-.4 -.2 0 .2 .4
Treatment effect (standard deviations)
Light news users Heavy news users
Social interaction index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-.8 -.6 -.4 -.2 0 .2
Treatment effect (standard deviations)
Passive users Active users
SMS happiness
SMS positive emotion
SMS not lonely
-.2 -.1 0 .1 .2 .3
Treatment effect (standard deviations)
Off-peak times Peak use times
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1), for subgroups deﬁned by the primary moderators in our pre-analysis plan. All variables are normalized
so that the Control group endline distribution has a standard deviation of one. Error bars reﬂect 95 percent
conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
60
Figure 10: Heterogeneous Treatment Eﬀects for All Moderators
Education
White
Party afﬁliation (Rep +, Dem -)
Subjective well-being index
Facebook opinions index
News
knowledge
-.2 -.1 0 .1 .2
Income
Education
Male
White
Age
Political
polarization
index
-.2 -.1 0 .1 .2
White
Party afﬁliation (Rep +, Dem -)
Social interaction index
Subjective well-being index
Facebook opinions index
Subjective
well-being
index
-.2 -.1 0 .1 .2
Education
Party afﬁliation (Rep +, Dem -)
Social interaction index
Substitute news sources index
Facebook opinions index
Post-experiment
use
index
-.2 -.1 0 .1 .2 .3
Male
Age
Party afﬁliation (Rep +, Dem -)
Social interaction index
Substitute news sources index
Facebook
opinions
index
-.2 -.1 0 .1 .2
Age
Social interaction index
News knowledge index
Political polarization index
Facebook opinions index
Deactivation
bad
-.4 -.2 0 .2 .4 .6
Moderation coefﬁcient
(standard deviation of outcome
per standard deviation of moderator)
Notes: This ﬁgure presents the moderators of local average treatment eﬀects of Facebook deactivation
estimated using Equation (4). For each of the six outcomes, we present the ﬁve moderators with the
largest moderation coeﬃcients ˆαk
. All outcome variables are normalized so that the Control group endline
distribution has a standard deviation of one, and all moderators are also normalized to have a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
61
Figure 11: Distribution of Willingness-to-Accept to Deactivate Facebook After Midline
0.1.2.3
Percentofsample
0 100 200 300 400 500
Willingness-to-accept for four-week deactivation ($)
Notes: This ﬁgure presents the distribution of willingness-to-accept to deactivate Facebook between midline
and endline. All responses above $525 are plotted at $525.
62
Figure 12: Average Valuation of Facebook in Treatment and Control
020406080100
MeanWTA
Weeks 1-4 Weeks 5-8 at midline Weeks 5-8 at endline
Control Treatment
Notes: This ﬁgure presents the mean willingness-to-accept (WTA) to deactivate Facebook in Treatment
and Control, for the impact evaluation sample: participants who were willing to accept less than $102 to
deactivate Facebook for the four weeks after midline and were oﬀered p = $102 or p = $0 to do so. The ﬁrst
pair of bars is the mean WTA for deactivation in weeks 1-4, the four weeks after the midline survey. The
second pair of bars is mean WTA for deactivation in weeks 5-8, the four weeks after the endline survey, as
elicited in the midline survey. The third pair of bars is mean WTA for deactivation in weeks 5-8, as elicited
in the endline survey.
63
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Online Appendix: Not for Publication
The Welfare Eﬀects of Social Media
Hunt Allcott, Luca Braghieri, Sarah Eichmeyer, and Matthew Gentzkow
1
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A1: Literature: Randomized Impact Evaluations of Facebook
Paper N Population Intervention Length Enforcement Outcomes PAP
Gonzales and
Hancock (2011)
63 College Look at proﬁle
vs. mirror
3 minutes None Self-esteem No
Deters and Mehl
(2012)
86 College Post more
status updates
1 week Scrape
proﬁle*
SWB No
Mabe, Forney,
and Keel (2014)
84 College
women
Browse
Facebook vs.
research ocelots
20 minutes None Eating disorder
risk
No
Sagioglu and
Greitemeyer
(2014)
263 MTurk Browse
Facebook
20 minutes None SWB No
Fardouly and
Vartanian
(2015)
112 College
women
Browse
Facebook vs.
other website
10 minutes None Body image,
mood
No
Verduyn et al.
(2015)
84 College Active vs.
passive use
10 minutes Screen
monitoring*
SWB No
Theocharis and
Lowe (2016)
197 Greek,
without
accounts
Sign up 6 months Payment
sent to
Facebook
account*
Voting, civic
engagement
No
Tromholt (2016) 886** Danish Not log in 1 week Self-report SWB No
Marotta and
Acquisti (2017)
455 MTurk Block Facebook
and YouTube
during work
hours
2 weeks Install
blocking
software
Work
productivity
No
Hunt et al.
(2018)
111 College Limit social
media to 10
minutes/day
4 weeks Weekly time
use screen
shots
SWB No
Vanman, Baker,
and Tobin
(2018)
123 Australian Not use
Facebook
5 days None Stress, SWB No
Mosquera et al.
(2018)
151†
College Not log in 1 week Check “last
active”
News, SWB,
WTA***
No
Allcott,
Braghieri,
Eichmeyer, and
Gentzkow (2018)
1,637 US
Facebook
ads
Deactivate 4 weeks Check URLs News, voting,
polarization,
SWB, WTA,
WTA changes
Yes
Notes: “N” is the number of people in the main empirical analysis, after attrition. †
1,765 people began
this study, but 151 people were randomized and completed the endline survey. *Instead of analyzing as a
randomized encouragement design, these studies dropped participants who did not comply with the treatment
conditions. **This study had an 12 percent attrition rate in treatment and a 26 percent attrition rate in
control. ***This study elicited WTA to participate in the experiment, which involved a 50 percent chance
of Facebook deactivation plus completing a survey, and a 50 percent chance of only completing a survey.
2
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
A Experimental Design Appendix
Figure A1: Facebook Advertisement Used for Recruitment
3
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A2: Post-Endline Social Media Time Limit Email
4
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A3: Post-Endline Politics Email
5
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A4: Subjective Well-Being Text Messages
(a) Happiness
(b) Primary Emotion
(c) Loneliness
6
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
B Variable Deﬁnitions and Descriptive Statistics
B.1 Variable Deﬁnitions by Family
Variable name Question text
Substitute time uses
Facebook minutes On an average day in the past 4 weeks, how many minutes would you say you
spent on Facebook, including through the Facebook app on your phone? (not
included in substitute time uses index)
(At baseline) On an average day in the last 4 weeks, how much free time (i.e. excluding
work) did you spend... [0 minutes, Between 1 and 30 minutes, Between 31
minutes and 1 hour, Between 1 and 2 hours, Between 2 and 3 hours, More
than 3 hours]
(At endline) In the last 4 weeks, relative to what is typical for you, would you say you
spent more or less of your free time (i.e. excluding work)... [A lot less, A little
less, Same, A little more, A lot more]
Non-FB social
media time
...using social media apps other than Facebook?
Non-social online
time
...online (on your computer, tablet, smartphone, etc.) for things other than
social media?
TV alone time ...watching TV or movies by yourself?
Non-screen alone
time
...on non-screen activities (e.g. cooking, reading books, exercising – anything
without an electronic screen in front of you) by yourself?
Friends and family
time
...doing anything with friends and family (in person)?
Social interaction
Friends met in
person
List the ﬁrst names of as many of the friends you met in person last week
that you can think of in 1 minute (if none, enter "none"). Separate the names
using commas (",").
Oﬄine activities Which of the following activities did you do at least once last week? Check all
that apply
Go out for dinner
Go to the cinema
Talk to friends on the phone
Go to a party
Get together with friends
Go to a shopping mall
7
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Spend time with your parents
Spend time with your kids
Diverse
interactions
Interact with someone who voted the opposite way as you in the last
presidential election
Interact with someone from another country
Substitute news sources
(At baseline) Over the past four weeks, how often did you... [Never, Hardly Ever,
Sometimes, Fairly Often, Very Often]
(At endline) In the last 4 weeks, relative to what is typical for you, would you say you
spent more or less time... [A lot less, A little less, Same, A little more, A lot
more]
Facebook news ...get news from Facebook (not included in substitute news sources index)
Print news ...read any newspapers in print?
Radio news ...listen to the news on the radio?
Local TV news ...watch local television news?
Network TV news ...watch national evening network television news (such as ABC World News,
CBS Evening News, or NBC Nightly News)?
Cable TV news ...watch cable television news (such as CNN, the Fox News cable channel, or
MSNBC)?
Non-FB social
media news
...get news from social media sites other than Facebook (e.g. Twitter or
Snapchat)?
Non-social online
news
...get news from news websites or apps other than social media?
Number of tweets ln(1+number of tweets in past four weeks)
News knowledge
Follow politics Thinking back over the last 4 weeks, how closely did you follow US politics?
[Not at all closely, somewhat closely, rather closely, very closely]
Follow Trump Thinking back over the last 4 weeks, how closely did you follow news about
President Trump? [Not at all closely, somewhat closely, rather closely, very
closely]
News minutes On an average day of the last 4 weeks, how many minutes did you spend
watching, reading or listening to the news (including news via social media)?
[text box]
News knowledge Of the following news events, which ones do you think are true, and which
ones do you think are false? [True, False, Unsure]
8
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
(At baseline)
True statements
Tension in trade negotiations escalated between the United States and China,
with President Trump announcing tariﬀs on $200 billion worth of goods.
An oﬀ-duty Dallas police oﬃcer entered the apartment of an
African-American neighbor and shot and killed the unarmed neighbor.
Deputy Attorney General Rod Rosenstein early in his tenure suggested
secretly recording President Trump and recruiting cabinet members to
remove him from oﬃce.
The Trump administration set the maximum number of refugees that can
enter the country in 2019 to 30,000.
Michael Cohen, President Donald Trump’s former personal attorney, agreed
to cooperate with the Mueller investigation team and discuss Trump’s
business dealings with Russia.
President Trump blasted Attorney General Jeﬀ Sessions for the indictments
of two lawmakers who supported Trump during the 2016 election.
CBS chief executive Les Moonves resigned after multiple sexual misconduct
allegations.
False statements President Trump’s former campaign chairman Paul Manafort refused deal to
cooperate with the Mueller investigation team in exchange for legal charges
against him being dropped.
President Trump spoke at the funeral of former Arizona Senator John
McCain, honoring the late McCain’s wish.
Hurricane Florence caused more than 300 deaths.
(At endline)
True statements
A prominent Saudi Arabian journalist who was critical of the country’s
government was killed inside the Saudi Arabian consulate in Istanbul.
In the weeks preceding the midterm elections, several high-proﬁle Democrats,
including Barack Obama and Hillary Clinton, were sent packages containing
explosive devices.
A mass shooting fueled by anti-Semitic sentiment took place in a synagogue
in Pittsburgh.
President Trump announced he plans to sign an executive order to prevent
second-generation immigrants born in the United States from automatically
being granted US citizenship.
The Department of Justice charged a Russian national allegedly involved in a
wide-ranging online disinformation campaign aimed at inﬂuencing the
Midterm elections.
One of the women who made allegations against Supreme Court Justice Brett
Kavanaugh has admitted to investigators that the allegations were fabricated.
9
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Attorney General Jeﬀ Sessions resigned at President Trump’s request.
False statements Harvard University recently stood trial for allegedly discriminating against
African-American applicants in its admission process.
Far-right candidate Jair Bolsonaro recently won an election to become the
President of Argentina.
Senator Elizabeth Warren’s DNA test results show that she has no native
American ancestry.
Fake news
knowledge
(At baseline)
After researcher Dr. Christine Blasey Ford accused Supreme Court nominee
Brett Kavanaugh of sexual assault, it is revealed that Kavanaugh’s mother
once ruled against Dr. Blasey Ford’s parents in a foreclosure case.
CNN’s Anderson Cooper reported deceptively on Hurricane Florence,
standing in a ditch to create the misleading impression that he was ﬁlming
amidst waist-deep ﬂoodwaters.
Mayor Carmen Yul´ın Cruz of San Juan was arrested for misappropriating $3
million in disaster relief funds intended for the victims of Hurricane Maria in
Puerto Rico.
Clerk refused to sell gas to a man ﬂeeing hurricane Florence over a Trump
bumper sticker.
WikiLeaks released an email showing that Hillary Clinton’s presidential
campaign bribed prominent Republicans to oppose Donald Trump during the
2016 election.
(At endline) Billionaire George Soros was revealed to be one of the funders of a caravan of
Central American emigrants traveling through Mexico to the US border.
A Russian feminist activist poured bleach on men who were “manspreading”
on the train ("manspreading" refers to men sitting in public transport with
legs wide apart, thereby covering more than one seat).
In a recent vote, all Democrats in Congress voted against a 2.8% cost of living
allowance in Social Security beneﬁts.
Cesar Sayoc, suspect in an act of domestic terrorism directed at vocal critics
of President Trump, was a registered Democrat.
None of the 154 mass shootings in 2018 was committed by a black man,
illegal alien, or woman.
Political engagement
Voted Takes value 1 if recorded as having voted in 2018 midterm, and 0 otherwise
10
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Clicked politics
email
Takes value 1 if clicked on any link in the post-endline politics email, and 0
otherwise
Political polarization
Party aﬀective
polarization
Thinking back over the last 4 weeks, how warm or cold did you feel towards
the parties and the president on the feeling thermometer?
Trump aﬀective
polarization
Thinking back over the last 4 weeks, how warm or cold did you feel towards
the parties and the president on the feeling thermometer?
Party anger List as many recent (last 4 weeks) news events you can think of that made
you angry at the [Republican/Democratic] Party. (If more than 5, just list
those 5 that left you most angry. If less than 5, list less. If none, enter "none"
in the ﬁrst textbox.)
Congenial news
exposure
Thinking back over the last 4 weeks, how often did you see news that made
you better understand the point of view of the [Republican/Democratic]
Party? [Never, Once, Two or three times, Four times or more]
Issue polarization To what extent do you think that free trade agreements between the US and
other countries have been a good thing or a bad thing for the United States?
(Pew Research Center 2018b)
Overall, would you say that blacks or whites are treated more fairly in dealing
with the police? (Pew Research Center 2016)
Do you think that employers ﬁring men who have been accused of sexual
harassment or assault before ﬁnding out all the facts is a major or a minor
problem? (Pew Research Center 2018d)
As you may know, Brett Kavanaugh is a federal judge who has been
nominated to serve on the Supreme Court. Would you like to see the Senate
vote in favor of Kavanaugh serving on the Supreme Court, or not? (Gallup
2018b)
On the whole, do you think immigration is a good thing or a bad thing for
this country today? (Pew Research Center 2018e)
How conﬁdent, if at all, are you that the Justice Department special counsel
Robert Mueller will conduct a fair investigation into Russian involvement in
the 2016 election? (Pew Research Center 2018c)
In general, do you feel that the laws covering the sale of ﬁrearms should be
made less strict, more strict, or kept as they are now? (Gallup 2018c)
In presenting the news dealing with political and social issues, do you think
that news organizations deal fairly with all sides, or do they tend to favor one
side? (Pew Research Center 2017)
11
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
To what extent do you think President Trump is honest and trustworthy?
(Gallup 2018a)
Belief polarization Level of agreement with co-partisans on beliefs questions
Vote polarization Strength of generic ballot preference for co-partisan candidate (see Voted
Republican question)
Subjective well-being
Happiness Over the last 4 weeks, I think I was [1 (not a very happy person) ... 7 (a very
happy person)]
Over the last 4 weeks, compared to most of my peers, I think I was [1 (less
happy) ... 7 (more happy)]
Life satisfaction Below are three statements that you may agree or disagree with. Indicate
your agreement with each item and please be open and honest in your
responding. [Strongly disagree, Disagree, Slightly disagree, Neither agree nor
disagree, Slightly agree, Agree, Strongly agree]
In most ways my life during the past 4 weeks was close to ideal.
The conditions of my life during the past 4 weeks were excellent.
During the past 4 weeks, I was satisﬁed with my life.
Loneliness × (-1) How often did you feel that you lacked companionship over the past four
weeks [Hardly ever, Some of the time, Often]
How often did you feel left out over the past four weeks [Hardly ever, Some of
the time, Often]
How often did you feel isolated from others over the past four weeks [Hardly
ever, Some of the time, Often]
Below are some ways you might have felt or behaved in the past 4 weeks.
Please tell us how much of the time during the past 4 weeks: [1 None or
almost none of the time, 2, 3, 4 All or almost all of the time]
Depressed × (-1) ... you felt depressed.
Anxious × (-1) ... you felt anxious.
Absorbed ... you were absorbed in doing something worthwhile.
Bored × (-1) ... you felt bored.
SMS happiness Overall, how happy do you feel right now on a scale from 1 (not at all happy)
to 10 (completely happy)?
12
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
SMS positive
emotion
What best describes how you felt over the last 10 minutes? Please text back
the corresponding number. [1: Lonely/left out 2: Shameful/guilty 3:
Absorbed in doing something worthwhile 4: Sad 5: Loving/tender 6: Bored 7:
Happy 8: Angry 9: Worried 10: Other positive feeling 11: Other negative
feeling 12: Other neutral feeling
SMS not lonely How lonely are you feeling right now on a scale from 1 (not at all lonely) to
10 (very lonely)?
Post-experiment use
Planned post-study
use change
After going through this study, how much more or less time do you plan to
spend on Facebook compared to before you started the study?
Clicked time limit
email × (-1)
Takes value 1 if clicked on any link in the post-endline social media time limit
email, and 0 otherwise
Speed of
reactivation
(-1) × ln(1+number of days deactivated after 24-hour post-endline
deactivation period)
Facebook mobile
app use
[if have an iPhone] Please write down the amount of screen time you spent on
the Facebook app according to your battery report.
[if do not have an iPhone] How many hours would you say you spent on the
Facebook app on your phone in the past seven days, in total?
Facebook opinions
Improves social life To what extent do you think Facebook improves or worsens people’s social
lives?
Good for you To what extent do you think Facebook is good or bad for you?
Good for society To what extent do you think Facebook is good or bad for society?
Makes people
happy
To what extent do you think using Facebook makes people more or less
happy?
People would miss
Facebook
To what extent do you agree or disagree with the following statement: “If
people spent less time on Facebook, they would soon realize that they don’t
miss it.”? (We multiply responses by -1, so more agreement with the
statement is more negative.)
Helps follow news To what extent do you think Facebook helps people follow the news better?
Clickbait, fake
news × (-1)
To what extent do you think Facebook exposes people to clickbait or false
news stories?
Less polarized To what extent do you think Facebook makes people more or less politically
polarized?
13
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Deactivation bad As part of this study, you were asked to deactivate your Facebook account for
[24 hours/4 weeks]. To what extent do you think that deactivating your
account was good or bad for you? (We multiply responses by -1, so
responding that deactivation was good is more negative.)
Positive impacts What are the most important positive impact(s) that Facebook has on your
life? [text box]
Negative impacts
× (-1)
What are the most important negative impact(s) that Facebook has on your
life? [text box]
Secondary outcomes
Voted Republican If the elections for US Congress were being held today, would you vote for the
Republican Party’s candidate or the Democratic Party’s candidate for
Congress in your district? [Republican candidate, Democratic candidate,
Other/don’t know]
[If would vote for Republican or Democratic candidate] How convinced are
you about whether to vote for the Republican candidate or the Democratic
candidate? [slider from 0 to 100]
Voted (self-report) Did you [midline: Do you plan to] vote in the midterm elections on November
6th, 2018?
Moderators
Time of day At what times of day do you usually use Facebook the most? [Morning
(6AM-12 noon), Afternoon (12 noon-5PM), Evening (5-9PM), Night
(9PM-midnight), Late night/early morning (midnight-6AM)
Active browsing People talk about two diﬀerent ways to use Facebook:
“Active” users often post status updates, comment on other people’s walls
and pictures, post photos, etc.
“Passive” users mostly check out their news feeds and/or other people’s
photos and proﬁles but don’t comment or interact much with others on the
site.
Which would you say describes your Facebook use best?
What share of your time on Facebook do you spend interacting one-on-one
with people you care about (for example, commenting on their posts or
sending them private messages)?
Get news from
Facebook
Over the past four weeks, how often did you ... get news from Facebook
[Never, Hardly Ever, Sometimes, Fairly Often, Very Often]
14
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Facebook minutes On an average day in the past 4 weeks, how many minutes would you say you
spent on Facebook, including through the Facebook app on your phone?
15
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
B.2 Descriptive Statistics
Table A3: Descriptive Statistics: Substitutes for Facebook and News and Political
Outcomes
Mean
Standard
deviation
Minimum
value
Maximum
value
N in
regression
Facebook minutes 59.53 37.38 0 120 1,639
Non-FB social media time 2.97 0.93 1 5 1,639
Non-social online time 3.28 0.88 1 5 1,639
TV alone time 3.10 1.02 1 5 1,639
Non-screen alone time 3.23 0.92 1 5 1,639
Friends and family time 3.24 0.91 1 5 1,639
Friends met in person 1.44 0.74 0 3 1,639
Oﬄine activities 3.06 1.53 0 8 1,639
Diverse interactions 0.99 0.79 0 2 1,639
Facebook news 2.98 1.05 1 5 1,639
Number of tweets 1.18 1.48 0 6 433
Non-FB social media news 3.04 1.03 1 5 1,639
Non-social online news 3.40 1.01 1 5 1,639
Local TV news 3.00 0.95 1 5 1,639
Network TV news 2.93 0.98 1 5 1,639
Cable TV news 2.93 1.01 1 5 1,639
Print news 2.72 0.95 1 5 1,639
Radio news 2.86 1.00 1 5 1,639
Follow politics 2.32 0.98 1 4 1,639
Follow Trump 2.09 0.92 1 4 1,639
News minutes 52.10 38.72 0 120 1,639
News knowledge 7.26 1.19 3 10 1,639
Fake news knowledge 2.72 0.74 0 5 1,639
Voted 0.71 0.45 0 1 1,341
Clicked politics email 0.02 0.15 0 1 1,651
Party aﬀective polarization 53.21 34.37 -86 100 1,455
Trump aﬀective polarization 32.73 26.72 -50 50 1,455
Party anger 1.48 1.81 -5 6 1,450
Congenial news exposure 1.00 1.54 -4 4 1,450
Issue polarization 2.89 2.97 -8 15 1,450
Belief polarization 2.16 5.21 -15 17 1,450
Vote polarization 0.63 0.48 -1 1 1,450
Notes: This table presents descriptive statistics for the dependent variables used in Equations (1) and (2).
Survey outcomes were recorded in the endline or post-endline surveys. The mean, standard deviation,
minimum, and maximum are for the prepared variables as used in the regressions, before normalizing to
standard deviation of one, for the Control group: participants who were willing to accept less than $102 to
deactivate Facebook for the four weeks after midline and were oﬀered p = $0 to do so. See Section 2.3 for
variable deﬁnitions. Facebook minutes and news minutes are winsorized at 120. Number of tweets is the
natural log of one plus the number of tweets.
16
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A4: Descriptive Statistics: Subjective Well-Being, Post-Experiment Facebook
Use and Opinions, and Secondary Outcomes
Mean
Standard
deviation
Minimum
value
Maximum
value
N in
regression
Happiness 4.47 1.41 1 7 1,639
Life satisfaction 12.26 4.78 3 21 1,639
Loneliness × (-1) -5.19 1.89 -9 -3 1,639
Depressed × (-1) 2.99 0.97 1 4 1,639
Anxious × (-1) 2.60 0.94 1 4 1,639
Absorbed 2.82 0.80 1 4 1,639
Bored × (-1) 2.93 0.88 1 4 1,639
SMS happiness 6.48 1.52 1 10 1,603
SMS positive emotion 0.53 0.25 0 1 1,606
SMS not lonely 7.60 1.70 1 10 1,604
Planned post-study use change -0.22 0.28 -1 1 1,637
Clicked time limit email × (-1) -0.09 0.28 -1 0 1,660
Speed of reactivation -0.41 0.69 -4 0 1,661
Facebook mobile app use 52.80 38.76 0 120 1,219
Improves social life -0.39 1.93 -5 5 1,639
Good for you -0.28 1.76 -5 5 1,639
Good for society -0.53 1.86 -5 5 1,639
Makes people happy -0.82 1.81 -5 5 1,639
Less polarized -2.48 1.76 -5 5 1,639
Helps follow news 0.31 2.41 -5 5 1,639
Clickbait, fake news × (-1) -2.71 2.06 -5 5 1,639
People would miss Facebook -1.97 1.99 -5 5 1,639
Deactivation bad -1.91 1.93 -5 5 1,639
Positive impacts 3.74 0.75 0 8 1,639
Negative impacts × (-1) -3.48 0.92 -7 0 1,639
Voted Republican -0.36 0.68 -1 1 1,639
Voted (self-report) 0.77 0.42 0 1 1,639
Notes: This table presents descriptive statistics for the dependent variables used in Equations (1) and (2).
Survey outcomes were recorded in the endline or post-endline surveys. The mean, standard deviation,
minimum, and maximum are for the prepared variables as used in the regressions, before normalizing to
standard deviation of one, for the Control group: participants who were willing to accept less than $102 to
deactivate Facebook for the four weeks after midline and were oﬀered p = $0 to do so. See Section 2.3 for
variable deﬁnitions. Facebook mobile app use is winsorized at 120. Positive impacts and negative impacts are
the natural log of one plus number of characters the participant wrote in the text box. Speed of reactivation
is negative one times the natural log of one plus the number of days that the participant remained deactivated
after 24-hour post-endline deactivation period), top-coded at the last day of measurement. Contributions is
the natural log of one plus the dollar amount of FEC contributions made between October 12 and November
10, 2018.
17
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A5: Descriptive Statistics: Pre-Experiment Time Use
Mean
Standard
deviation
Minimum
value
Maximum
value
Facebook minutes 74.5 35.5 20 120
News minutes 53.0 37.9 0 120
Non-FB social media time 75.7 76.3 0 240
Non-social online time 135.9 83.7 0 240
TV alone time 95.5 82.8 0 240
Non-screen alone time 105.9 79.2 0 240
Friends and family time 130.4 83.4 0 240
Facebook mobile app use 60.0 38.9 0 120
Notes: This table presents descriptive statistics for pre-experiment time use, for the impact evaluation
sample: participants who were willing to accept less than $102 to deactivate Facebook for the four weeks
after midline and were oﬀered p = $102 or p = $0 to do so. These survey outcomes were recorded in the
baseline and midline surveys. See Section 2.3 for variable deﬁnitions. Facebook minutes, news minutes, and
Facebook mobile app use are winsorized at 120.
18
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
C Voting Behavior
In order to study the eﬀect of our treatment on voting behavior in the 2018 midterm elections,
we matched the participants in our experiment to a voting database supplied to Stanford by L2, a
voting data provider.
We performed multiple rounds of merging, relaxing the merging criteria in each round in order
to increase the number of participants matched. Each round of merging was based on a diﬀerent
combination of variables that included ﬁrst name, middle initial, last name, birth year, zip code, and
state. Table A6 describes the criteria used in each round of merging and the number of participants
in the impact evaluation sample who were matched for the ﬁrst time in that round.
Approximately 93 percent of the matches are one-to-one: a participant in our experiment was
matched to one and only one individual in the L2 database. The remaining seven percent of the
matches are one-to-many: a participant in our experiment was matched to more than one individual
in the L2 database. Whenever the match was one-to-many, we constructed the voting variables by
taking an average of the voting behavior of the multiple individuals in the L2 database that the
participant in our experiment was matched to.
The overall match rate was 81 percent. Since the L2 database only contains records of registered
voters and since, according to the Census Bureau, the fraction of the total citizen population over
18 who reports not being registered to vote is around 15 percent, an 81 percent match rate is close
to the maximum that might be expected (United States Census Bureau 2018). Match rates are not
statistically diﬀerent between Treatment and Control, as shown in Table A7.
Table A8 shows local average treatment eﬀects estimated from the standard regression model
described in Equation (1), estimated oﬀ of three diﬀerent samples. Column 1 includes the subset of
participants in the impact evaluation sample who could be matched to entries in the L2 database.
This is our primary speciﬁcation reported in Appendix Table A10. Column 2 includes the entire
impact evaluation sample, assuming that participants who could not be matched to entries in the
L2 database did not vote. Finally, column 3 includes the subset of participants in the impact
evaluation sample who could be matched to a unique entry in the L2 database based on the most
strict match criterion—merge round 1 as described in Appendix Table A7. In all three columns,
we cannot reject the null hypothesis that deactivation does not aﬀect voting.
We also examined the relationship between self-reported voting behavior and voting behavior
according to the L2 database. Appendix Table A9 presents statistics for participants in the impact
evaluation sample who had unique matches in the L2 database. Almost everyone (98 percent) who
voted in the 2018 midterm elections according to the L2 database also reported on the endline
survey that they had voted. Conversely, only 57 percent of the people who did not vote in the 2018
midterm elections according to the L2 database elections reported not having voted. The results
are in line with prior literature showing that around 25-50 percent of non-voters tend to incorrectly
19
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
report on surveys that they voted (Belli et al. 1999; DellaVigna et al. 2017; Duﬀ et al. 2007; Silver,
Anderson, and Abramson 1986).
Table A6: Criteria for Matching Participants to the L2 Voting Database
Merge
round
(most to
least
strict)
First
name
Middle
initial
Last
name
Birth
year
Zip
code
State Number of
participants
matched (out of
1,661)
Cumulative
fraction (out
of 1,661)
1 x x x x x 818 0.49
2 x x x x x 88 0.55
3 x x x x 50 0.58
4 x x x x 150 0.67
5 x x x 16 0.68
6 x x x x 64 0.71
7 x x x x 66 0.75
8 x x x 89 0.81
Notes: This table describes the criteria used to match the participants in our experiment to entries in the
L2 voting database. It also shows the number of participants in the impact evaluation sample (N=1,661)
who were matched to an L2 record for the ﬁrst time in that round.
Table A7: Test of Diﬀerential Match Rates in Treatment vs. Control
Found a match
Treatment -0.02
(0.02)
Constant 0.81
(0.01)
Observations 1,661
Notes: This table presents the results of a regression of a binary variable indicating whether a participant
in the impact evaluation sample was matched to at least one record in the L2 database on a Treatment
indicator. Standard errors are in parentheses.
20
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A8: Treatment Eﬀect of Deactivation on Voting Behavior
(1) (2) (3)
Those with
match Full sample
Those with high
quality match
Share of time deactivated 0.03 0.02 0.02
(0.03) (0.02) (0.03)
Observations 1,341 1,661 818
Control group mean 0.71 0.57 0.79
Notes: This table presents the local average treatment eﬀects of deactivation on voter turnout estimated
using Equation (1), for three diﬀerent samples. Column 1 includes the subset of participants in the impact
evaluation sample who could be matched to entries in the L2 database. Column 2 includes the entire impact
evaluation sample; the turnout outcome variable is set to zero for participants who could not be matched to
any entry in the L2 database. Column 3 includes the subset of participants in the impact evaluation sample
who could be matched to the L2 database using the strictest merge criteria (based on ﬁrst name, middle
initial, last name, birth year, and zip code). Standard errors are in parentheses.
Table A9: Voting Behavior According to Self-Reports and the L2 Database
Mean self-reported voting
Administrative records:
Didn’t vote 0.43
Administrative records:
Did vote 0.98
Notes: This table presents statistics for participants in the impact evaluation sample who had unique matches
in the L2 database. It gives the fraction of participants who reported on the endline survey that they had
voted, separately for people who voted and who didn’t vote according to the administrative voting records.
21
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
D Tables of Treatment Eﬀect Estimates
Table A10: Treatment Eﬀects: Substitutes for Facebook and News and Political Out-
comes
Treatment
eﬀect
(original
units)
Standard
error
(original
units)
Treatment
eﬀect
(SD units)
Standard
error
(SD units) P-value
Sharpened
FDR-
adjusted
q-value
Facebook minutes -59.58 1.43 -1.59 0.04 0.00 0.00
Non-FB social media time -0.25 0.07 -0.27 0.07 0.00 0.00
Non-social online time -0.12 0.06 -0.14 0.06 0.03 0.05
TV alone time 0.17 0.05 0.17 0.05 0.00 0.00
Non-screen alone time 0.23 0.05 0.25 0.05 0.00 0.00
Friends and family time 0.14 0.05 0.16 0.06 0.00 0.01
Friends met in person 0.04 0.03 0.05 0.04 0.25 0.23
Oﬄine activities 0.18 0.08 0.12 0.05 0.02 0.03
Diverse interactions -0.04 0.04 -0.05 0.05 0.32 0.28
Facebook news -1.90 0.05 -1.81 0.04 0.00 0.00
Number of tweets 0.23 0.13 0.16 0.09 0.08 0.09
Non-FB social media news -0.37 0.07 -0.36 0.07 0.00 0.00
Non-social online news -0.02 0.06 -0.02 0.06 0.79 0.49
Local TV news 0.04 0.05 0.04 0.06 0.42 0.35
Network TV news 0.06 0.05 0.06 0.05 0.23 0.21
Cable TV news 0.02 0.05 0.02 0.05 0.70 0.45
Print news 0.02 0.05 0.02 0.05 0.72 0.45
Radio news 0.08 0.05 0.08 0.05 0.16 0.17
Follow politics -0.14 0.04 -0.14 0.04 0.00 0.00
Follow Trump -0.10 0.04 -0.11 0.04 0.01 0.02
News minutes -7.92 1.83 -0.20 0.05 0.00 0.00
News knowledge -0.14 0.06 -0.12 0.05 0.02 0.04
Fake news knowledge -0.04 0.04 -0.06 0.05 0.26 0.23
Voted 0.03 0.03 0.06 0.06 0.32 0.28
Clicked politics email 0.01 0.01 0.06 0.06 0.36 0.31
Party aﬀective polarization -1.98 1.40 -0.06 0.04 0.16 0.17
Trump aﬀective polarization -0.04 0.71 -0.00 0.03 0.96 0.56
Party anger -0.13 0.10 -0.07 0.05 0.18 0.17
Congenial news exposure -0.31 0.08 -0.20 0.05 0.00 0.00
Issue polarization -0.29 0.11 -0.10 0.04 0.01 0.02
Belief polarization -0.22 0.27 -0.04 0.05 0.43 0.35
Vote polarization -0.00 0.02 -0.01 0.05 0.91 0.56
Notes: This table presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). Column 1 and Column 2 present the eﬀect and standard error on un-normalized outcomes. Columns
3 and 4 present the eﬀect and standard error on normalized outcomes, where outcomes are normalized so
that the Control group endline distribution has a standard deviation of one. Columns 5 and 6 present the
unadjusted p-value and sharpened False Discovery Rate-adjusted two-stage q-value, respectively.
22
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A11: Treatment Eﬀects: Subjective Well-Being, Post-Experiment Facebook Use
and Opinions, and Secondary Outcomes
Treatment
eﬀect
(original
units)
Standard
error
(original
units)
Treatment
eﬀect
(SD units)
Standard
error
(SD units) P-value
Sharpened
FDR-
adjusted
q-value
Happiness 0.12 0.06 0.08 0.04 0.04 0.06
Life satisfaction 0.56 0.20 0.12 0.04 0.00 0.01
Loneliness × (-1) 0.05 0.08 0.03 0.04 0.54 0.40
Depressed × (-1) 0.08 0.04 0.09 0.04 0.03 0.05
Anxious × (-1) 0.09 0.04 0.10 0.05 0.03 0.05
Absorbed -0.01 0.04 -0.01 0.05 0.82 0.50
Bored × (-1) 0.06 0.04 0.07 0.05 0.17 0.17
SMS happiness 0.09 0.07 0.06 0.04 0.18 0.17
SMS positive emotion 0.01 0.01 0.05 0.05 0.31 0.28
SMS not lonely 0.01 0.09 0.01 0.05 0.88 0.54
Planned post-study use change -0.21 0.02 -0.78 0.07 0.00 0.00
Clicked time limit email × (-1) -0.04 0.02 -0.15 0.06 0.02 0.04
Speed of reactivation -0.41 0.06 -0.59 0.08 0.00 0.00
Facebook mobile app use -12.15 2.19 -0.31 0.06 0.00 0.00
Improves social life -0.00 0.09 -0.00 0.05 0.98 0.56
Good for you -0.01 0.09 -0.00 0.05 0.93 0.56
Good for society -0.04 0.09 -0.02 0.05 0.62 0.40
Makes people happy 0.14 0.09 0.08 0.05 0.13 0.15
Less polarized -0.06 0.09 -0.03 0.05 0.53 0.40
Helps follow news 0.31 0.11 0.13 0.05 0.01 0.02
Clickbait, fake news × (-1) -0.03 0.11 -0.01 0.05 0.79 0.49
People would miss Facebook 0.26 0.12 0.13 0.06 0.03 0.05
Deactivation bad -0.45 0.12 -0.23 0.06 0.00 0.00
Positive impacts 0.21 0.04 0.28 0.05 0.00 0.00
Negative impacts × (-1) -0.21 0.05 -0.23 0.05 0.00 0.00
Voted Republican -0.04 0.02 -0.07 0.04 0.06 0.08
Voted (self-report) -0.03 0.02 -0.06 0.05 0.18 0.17
Notes: This table presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). Column 1 and Column 2 present the eﬀect and standard error on un-normalized outcomes. Columns
3 and 4 present the eﬀect and standard error on normalized outcomes, where outcomes are normalized so
that the Control group endline distribution has a standard deviation of one. Columns 5 and 6 present the
unadjusted p-value and sharpened False Discovery Rate-adjusted two-stage q-value, respectively.
23
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A12: Treatment Eﬀects: Indices
Treatment
eﬀect
Standard
error P-value
Sharpened
FDR-adjusted
q-value
Substitute time uses index 0.14 0.06 0.03 0.03
Social interaction index 0.05 0.04 0.28 0.17
Substitute news sources index 0.03 0.06 0.63 0.39
News knowledge index -0.19 0.04 0.00 0.00
Political engagement index 0.07 0.06 0.27 0.17
Political polarization index -0.16 0.04 0.00 0.00
Subjective well-being index 0.09 0.04 0.02 0.03
Post-experiment use index -0.61 0.06 0.00 0.00
Facebook opinions index 0.07 0.06 0.21 0.17
Notes: This table presents local average treatment eﬀects of Facebook deactivation on index outcomes
estimated using Equation (1). Columns 1 and 2 present the eﬀect and standard error, with indices normalized
so that the Control group endline distribution has a standard deviation of one. Columns 3 and 4 present
the unadjusted p-value and sharpened False Discovery Rate-adjusted two-stage q-value, respectively.
Table A13: Treatment Eﬀects: Post-Experiment Facebook Mobile App Usage
(1) (2) (3) (4)
Full
sample
LATE
Full
sample
ITT
iPhone
only
LATE
iPhone
only
ITT
Share of time deactivated -11.46 -3.40
(2.26) (2.92)
Treatment -10.13 -3.05
(2.02) (2.63)
Observations 1,219 1,219 526 526
Control group endline mean 52.8 52.8 42.3 42.3
Lee (2009) treatment eﬀect lower bound -8.73 -2.04
Lee (2009) treatment eﬀect upper bound -7.76 -1.63
Lee (2009) 95% conﬁdence interval lower bound -13.77 -10.31
Lee (2009) 95% conﬁdence interval upper bound -3.18 5.16
Notes: This table presents treatment eﬀects of Facebook deactivation on post-experiment Facebook mobile
app use in units of minutes per day, as measured in the December 3rd post-endline survey. Columns 1
and 2 include all observations, while columns 3 and 4 limit the sample to iPhone users who reported their
Facebook mobile app usage as recorded by their System app, excluding participants who had reported
personal estimates. Columns 1 and 3 present local average treatment eﬀects estimated using Equation (1),
while columns 2 and 4 present intent-to-treat eﬀects and Lee (2009) bounds that account for attrition.
24
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
E Treatment Eﬀect Estimates Using Equation (2)
Figure A5: Substitutes for Facebook Using Equation (2)
Substitute
timeuses
Social
interaction
Substitute
newssources
Facebook minutes
Non-FB social media time
Non-social online time
TV alone time
Non-screen alone time
Friends and family time
Substitute time uses index
Friends met in person
Ofﬂine activities
Diverse interactions
Social interaction index
Facebook news
Number of tweets
Non-FB social media news
Non-social online news
Local TV news
Network TV news
Cable TV news
Print news
Radio news
Substitute news sources index
-1.5 -1 -.5 0 .5
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(2). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
25
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A6: Eﬀects on News and Political Outcomes Using Equation (2)
News
knowledge
Political
engagement
Political
polarization Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.2 -.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(2). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
26
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A7: Eﬀects on Subjective Well-Being Using Equation (2)
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.1 -.05 0 .05 .1 .15
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(2). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
27
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A8: Eﬀects on Post-Experiment Facebook Use and Opinions Using Equation (2)
Post-experiment
use
Facebook
opinions
Planned post-study use change
Clicked time limit email × (-1)
Speed of reactivation
Facebook mobile app use
Post-experiment use index
Improves social life
Good for you
Good for society
Makes people happy
Less polarized
Helps follow news
Clickbait, fake news × (-1)
People would miss Facebook
Deactivation bad
Positive impacts
Negative impacts × (-1)
Facebook opinions index
-.6 -.4 -.2 0 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(2). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
28
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F Heterogeneous Treatment Eﬀects
F.1 Secondary Moderators
Figure A9: Heterogeneous Treatment Eﬀects for Secondary and Ex-Post Moderators
Substitute time uses index
Social interaction index
Substitute news sources index
News knowledge index
Political engagement index
Political polarization index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-1 -.5 0 .5
Treatment effect (standard deviations)
Below median age
Above median age
Substitute time uses index
Social interaction index
Substitute news sources index
News knowledge index
Political engagement index
Political polarization index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-1 -.5 0 .5
Treatment effect (standard deviations)
Below median WTA
Above median WTA
News knowledge index
Political engagement index
Political polarization index
-1 -.5 0 .5
Treatment effect (standard deviations)
Democratic users
Republican users
Substitute time uses index
Social interaction index
Substitute news sources index
News knowledge index
Political engagement index
Political polarization index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-1 -.5 0 .5
Treatment effect (standard deviations)
Below median sample weight
Above median sample weight
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). Age and political party were the “secondary” moderators in our pre-analysis plan. Willingness-to-accept
and sample weight were not deﬁned as moderators of interest in our pre-analysis plan. All variables are
normalized so that the Control group endline distribution has a standard deviation of one. Error bars reﬂect
95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
29
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F.2 Light and Heavy Users
Figure A10: Substitutes for Facebook for Light and Heavy Users
Substitute
timeuses
Social
interaction
Substitute
newssources
Facebook minutes
Non-FB social media time
Non-social online time
TV alone time
Non-screen alone time
Friends and family time
Substitute time uses index
Friends met in person
Ofﬂine activities
Diverse interactions
Social interaction index
Facebook news
Number of tweets
Non-FB social media news
Non-social online news
Local TV news
Network TV news
Cable TV news
Print news
Radio news
Substitute news sources index
-3 -2 -1 0 1
Treatment effect
(standard deviations)
Light users Heavy users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 75 daily minutes, the median amount of Facebook use in the impact
evaluation sample. All variables are normalized so that the Control group endline distribution has a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
30
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A11: Eﬀects on News and Political Outcomes for Light and Heavy Users
News
knowledge
Political
engagement
Political
polarization
Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Light users Heavy users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 75 daily minutes, the median amount of Facebook use in the impact
evaluation sample. All variables are normalized so that the Control group endline distribution has a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
31
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A12: Eﬀects on Subjective Well-Being for Light and Heavy Users
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Light users Heavy users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 75 daily minutes, the median amount of Facebook use in the impact
evaluation sample. All variables are normalized so that the Control group endline distribution has a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
32
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A13: Eﬀects on Post-Experiment Facebook Use and Opinions for Light and
Heavy Users
Post-experiment
use
Facebook
opinions
Planned post-study use change
Clicked time limit email × (-1)
Speed of reactivation
Facebook mobile app use
Post-experiment use index
Improves social life
Good for you
Good for society
Makes people happy
Less polarized
Helps follow news
Clickbait, fake news × (-1)
People would miss Facebook
Deactivation bad
Positive impacts
Negative impacts × (-1)
Facebook opinions index
-1 -.5 0 .5
Treatment effect
(standard deviations)
Light users Heavy users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 75 daily minutes, the median amount of Facebook use in the impact
evaluation sample. All variables are normalized so that the Control group endline distribution has a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
33
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F.3 Light and Heavy News Users
Figure A14: Eﬀects on News and Political Outcomes for Light and Heavy News Users
News
knowledge
Political
engagement
Political
polarization
Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Light news users Heavy news users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for heavy news users vs. light news users (those who get news from Facebook fairly often or very often
vs. never, hardly ever, or sometimes). All variables are normalized so that the Control group endline
distribution has a standard deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section
2.3 for variable deﬁnitions.
34
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F.4 Active and Passive Users
Figure A15: Eﬀects on Subjective Well-Being and Social Interactions for Active and
Passive Users
Subjective
well-being
Social
interaction
Friends met in person
Ofﬂine activities
Diverse interactions
Social interaction index
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.2 -.1 0 .1 .2 .3
Treatment effect
(standard deviations)
Passive users Active users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for active users vs. passive users. We measure this using two questions: share of active vs. passive
browsing using a question based on the Passive and Active Facebook Use Measure (Gerson, Plagnol, and
Corr 2017), and “what share of your time on Facebook do you spend interacting one-on-one with people
you care about.” Active vs. passive users are deﬁned as having above- vs. below-median sum of their two
responses to these questions. All variables are normalized so that the Control group endline distribution has
a standard deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable
deﬁnitions.
35
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A16: Eﬀects on Post-Experiment Use and Opinions about Facebook for Active
and Passive Users
Facebook
opinions
Post-experiment
use
Planned post-study use change
Clicked time limit email × (-1)
Speed of reactivation
Facebook mobile app use
Post-experiment use index
Improves social life
Good for you
Good for society
Makes people happy
Less polarized
Helps follow news
Clickbait, fake news × (-1)
People would miss Facebook
Deactivation bad
Positive impacts
Negative impacts × (-1)
Facebook opinions index
-1 -.5 0 .5
Treatment effect
(standard deviations)
Passive users Active users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for active users vs. passive users. We measure this using two questions: share of active vs. passive
browsing using a question based on the Passive and Active Facebook Use Measure (Gerson, Plagnol, and
Corr 2017), and “what share of your time on Facebook do you spend interacting one-on-one with people
you care about.” Active vs. passive users are deﬁned as having above- vs. below-median sum of their two
responses to these questions. All variables are normalized so that the Control group endline distribution has
a standard deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable
deﬁnitions.
36
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F.5 Democrats and Republicans
Figure A17: Eﬀects on News and Political Outcomes for Democrats and Republicans
News
knowledge
Political
engagement
Political
polarization
Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.6 -.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Democratic users Republican users
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for Democrats vs. Republicans. All variables are normalized so that the Control group endline
distribution has a standard deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section
2.3 for variable deﬁnitions.
37
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
F.6 Younger and Older Users
Figure A18: Substitutes for Facebook for Younger and Older Users
Substitute
timeuses
Social
interaction
Substitute
newssources
Facebook minutes
Non-FB social media time
Non-social online time
TV alone time
Non-screen alone time
Friends and family time
Substitute time uses index
Friends met in person
Ofﬂine activities
Diverse interactions
Social interaction index
Facebook news
Number of tweets
Non-FB social media news
Non-social online news
Local TV news
Network TV news
Cable TV news
Print news
Radio news
Substitute news sources index
-2 -1.5 -1 -.5 0 .5
Treatment effect
(standard deviations)
Below median age Above median age
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 31.5 years, the median age in the impact evaluation sample. All variables
are normalized so that the Control group endline distribution has a standard deviation of one. Error bars
reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
38
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A19: Eﬀects on News and Political Outcomes for Younger and Older Users
News
knowledge
Political
engagement
Political
polarization Follow politics
Follow Trump
News minutes
News knowledge
Fake news knowledge
News knowledge index
Voted
Clicked politics email
Political engagement index
Party affective polarization
Trump affective polarization
Party anger
Congenial news exposure
Issue polarization
Belief polarization
Vote polarization
Political polarization index
-.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Below median age Above median age
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 31.5 years, the median age in the impact evaluation sample. All variables
are normalized so that the Control group endline distribution has a standard deviation of one. Error bars
reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
39
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A20: Eﬀects on Subjective Well-Being for Younger and Older Users
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.2 -.1 0 .1 .2 .3
Treatment effect
(standard deviations)
Below median age Above median age
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 31.5 years, the median age in the impact evaluation sample. All variables
are normalized so that the Control group endline distribution has a standard deviation of one. Error bars
reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
40
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A21: Eﬀects on Post-Experiment Facebook Use and Opinions for Younger and
Older Users
Post-experiment
use
Facebook
opinions
Planned post-study use change
Clicked time limit email × (-1)
Speed of reactivation
Facebook mobile app use
Post-experiment use index
Improves social life
Good for you
Good for society
Makes people happy
Less polarized
Helps follow news
Clickbait, fake news × (-1)
People would miss Facebook
Deactivation bad
Positive impacts
Negative impacts × (-1)
Facebook opinions index
-1 -.5 0 .5
Treatment effect
(standard deviations)
Below median age Above median age
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants above vs. below 31.5 years, the median age in the impact evaluation sample. All variables
are normalized so that the Control group endline distribution has a standard deviation of one. Error bars
reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
41
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
G News Knowledge
Appendix Figure A22 presents treatment eﬀects on the probability of correct answers for each
individual news knowledge question. Recall that we code a value of 1 for true statements correctly
rated as true or incorrect statements correctly rated as false, 0.5 for any statement rated as “unsure,”
and 0 for true statements incorrectly rated as false or incorrect statements incorrectly rated as true.
To unpack these results, Appendix Figures A23, A24, and A25 present local average treatment
eﬀects of Facebook deactivation on indicators for answering true, false or unsure to our sets of
true news, false news, and fake news questions respectively. By true news, we refer to the seven
statements about news events reported by major outlets in which we did not insert factual inaccuracies.
By false news, we refer to the three statements about news events reported by major news
outlets in which we did insert substantial factual inaccuracies. By fake news, we refer to the ﬁve
statements summarizing news articles that were deemed false on fact-checking websites and that
circulated heavily within the four-week period before the survey. At the bottom of each block of
news questions, we present treatment eﬀects on the average across the questions in that block.
Most of the estimates are not statistically signiﬁcant at any conventional level. Notwithstanding,
the pattern of point estimates for true and false news statements is cohesive: in eight out
of ten questions, deactivation induced people to move away from the correct answer and towards
either the incorrect answer or “unsure” (or both). This paints a richer picture of how Facebook
deactivation might reduce news knowledge: Treatment group participants are more likely to answer
“unsure” and, if they do not answer “unsure” and take a guess as to whether the news event is true
or false, they are more likely to answer incorrectly.
For the fake news questions, Facebook deactivation appears to have made people more likely
to answer “unsure” instead of “false.” This explains the negative point estimate of the eﬀect
of deactivation on fake news knowledge presented in Figure 3. Although not nearly statistically
signiﬁcant, one explanation for these point estimates is that Facebook circulates fake news but, at
least for the major fake news stores in our survey, provides corrective information that helps users
to correctly identify these stories as fake.
42
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A22: Eﬀects on News Knowledge and Fake News KnowledgeNews
knowledge
Fakenews
knowledge
Jeff Sessions resigns at Trump request
Several Democrats sent explosives
Kavanaugh accuser admits fabricating story
Justice Department charges Russian national
Shooting in Pittsburgh synagogue
Journalist killed in Saudi consulate
Trump plans to end birthright citizenship
Elizabeth Warren reveals Native ancestry
Harvard tried for anti-Asian bias
Bolsonaro becomes Brazilian president
Terrorist Cesar Sayoc was registered Democrat
George Soros funding immigrant caravan
Feminist poured bleach on manspreaders
No mass shootings committed by minorities
Democrats vote down cost of living beneﬁts
-.3 -.2 -.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
Figure A23: Eﬀects on Knowledge of True News Items
Believes
true
Believes
false
Is
unsure
Jeff Sessions resigns at Trump request
Kavanaugh accuser admits fabricating story
Justice Department charges Russian national
Journalist killed in Saudi consulate
Several Democrats sent explosives
Shooting in Pittsburgh synagogue
Trump plans to end birthright citizenship
Average across these questions
Jeff Sessions resigns at Trump request
Kavanaugh accuser admits fabricating story
Justice Department charges Russian national
Journalist killed in Saudi consulate
Several Democrats sent explosives
Shooting in Pittsburgh synagogue
Trump plans to end birthright citizenship
Average across these questions
Jeff Sessions resigns at Trump request
Kavanaugh accuser admits fabricating story
Justice Department charges Russian national
Journalist killed in Saudi consulate
Several Democrats sent explosives
Shooting in Pittsburgh synagogue
Trump plans to end birthright citizenship
Average across these questions
-.4 -.2 0 .2 .4
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). The left-hand side variables are indicators for answering true, false or unsure to each of our true news
items. All variables are normalized so that the Control group endline distribution has a standard deviation
of one. Error bars reﬂect 95 percent conﬁdence intervals.
43
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A24: Eﬀects on Knowledge of False News ItemsBelieves
true
Believes
false
Is
unsure
Bolsonaro becomes Brazilian president
Harvard tried for anti-Asian bias
Elizabeth Warren reveals Native ancestry
Average across these questions
Bolsonaro becomes Brazilian president
Harvard tried for anti-Asian bias
Elizabeth Warren reveals Native ancestry
Average across these questions
Bolsonaro becomes Brazilian president
Harvard tried for anti-Asian bias
Elizabeth Warren reveals Native ancestry
Average across these questions
-.4 -.2 0 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). The left-hand side variables are indicators for answering true, false or unsure to each of our false news
items. All variables are normalized so that the Control group endline distribution has a standard deviation
of one. Error bars reﬂect 95 percent conﬁdence intervals.
Figure A25: Eﬀects on Knowledge of Fake News Items
Believes
true
Believes
false
Is
unsure
Democrats vote down cost of living beneﬁts
No mass shootings committed by minorities
Feminist poured bleach on manspreaders
George Soros funding immigrant caravan
Terrorist Cesar Sayoc was registered Democrat
Average across these questions
Democrats vote down cost of living beneﬁts
No mass shootings committed by minorities
Feminist poured bleach on manspreaders
George Soros funding immigrant caravan
Terrorist Cesar Sayoc was registered Democrat
Average across these questions
Democrats vote down cost of living beneﬁts
No mass shootings committed by minorities
Feminist poured bleach on manspreaders
George Soros funding immigrant caravan
Terrorist Cesar Sayoc was registered Democrat
Average across these questions
-.3 -.2 -.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). The left-hand side variables are indicators for answering true, false or unsure to each of our fake news
items. All variables are normalized so that the Control group endline distribution has a standard deviation
of one. Error bars reﬂect 95 percent conﬁdence intervals.
44
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
H Additional Empirical Results
Table A14: Balance
(1) (2) T-test
Treatment Control P-value
Variable Mean/SD Mean/SD (1)-(2)
Income ($000s) 71.27
(50.22)
72.69
(51.80)
0.59
College 0.52
(0.50)
0.50
(0.50)
0.61
Male 0.44
(0.50)
0.42
(0.49)
0.60
White 0.68
(0.47)
0.68
(0.46)
0.77
Age 33.04
(12.54)
32.34
(11.71)
0.27
Republican 0.13
(0.34)
0.14
(0.34)
0.85
Democrat 0.41
(0.49)
0.42
(0.49)
0.53
Facebook minutes 75.20
(35.58)
74.15
(35.49)
0.57
Get news from Facebook 3.47
(1.12)
3.43
(1.06)
0.45
Active browsing 0.14
(0.98)
0.16
(0.97)
0.73
N 580 1081
F-test of joint signiﬁcance (p-value) 0.95
F-test, number of observations 1661
Notes: Columns 1 and 2 present demographics for the Treatment and Control groups in the impact evaluation
sample: participants who were willing to accept less than $102 to deactivate Facebook for the four weeks
after midline and were oﬀered p = $102 or p = $0 to do so. Column 3 presents p-values of tests of diﬀerences
in means between the two groups.
45
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A26: Response Rates to Daily Text Messages
.5.6.7.8.91
Overallresponserate
-10 0 10 20 30
-.06-.030.03.06
T-Cdifferenceinresponserate
-10 0 10 20 30
Days after beginning of deactivation period
T-C difference 95% CI
Notes: The ﬁgure shows response rates to the SMS survey and the diﬀerence in response rates between
Treatment and Control, for the impact evaluation sample: participants who were willing to accept less than
$102 to deactivate Facebook for the four weeks after midline and were oﬀered p = $102 or p = $0 to do so.
The vertical red line reﬂects the date of the midline survey.
46
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A27: Treatment Group Distribution of Share of Time Deactivated
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Cumulativeprobability
.1 .2 .3 .4 .5 .6 .7 .8 .9 1
Share of time deactivated
Notes: For each individual in the Treatment group who was willing to accept less than $102 to deactivate
Facebook for the four weeks after midline, we calculate the share of the deactivation checks in which that
person was deactivated. This ﬁgure presents the cumulative distribution of the share of the time deactivated
across people.
47
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A28: Reasons for Failure to Deactivate
0.2.4.6.8
Percentofparticipants
Virtually always active Active 4-27 days Active 1-3 days
Ever reactivated on purpose Ever reactivated accidentally
Notes: This ﬁgure presents reasons for failure to deactivate for Treatment group participants. Data were
gathered from an optional survey that we emailed to participants who were not deactivated when they were
supposed to be under the experiment protocols. The survey asked, “Why did your Facebook account get
reactivated? Your answer won’t aﬀect your payment – we’re just trying to ﬁgure out what problems people
are having.” Possible responses were, “I logged into my account using the Facebook website or the Facebook
app,” “somebody else logged into my account,” “I used an app (other than the Facebook app or the Facebook
messenger app) that uses my Facebook credentials to log in,” “Other (please specify),” and “I don’t know.”
We coded an individual as having reactivated “on purpose” if they ever clicked the ﬁrst answer (“I logged
into my account”). We coded an individual as having reactivated “accidentally” if they ever clicked on the
second, third, or ﬁfth answers. We also manually coded text that respondents wrote in the “Other (please
specify)” box as either “on purpose” or “accidental.” The bars display the share of all participants in the
subgroup (including participants who never responded to a survey) who ever responded that they reactivated
on purpose or accidentally.
48
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A29: Eﬀects on Oﬄine Activities and Diverse Interactions
Go out for dinner
Go to the cinema
Talk to friends on the phone
Go to a party
Get together with friends
Go shopping
Spend time with your parents
Spend time with your kids
Interact with someone who voted opposite way
Interact with someone from another country
-.1 0 .1 .2 .3
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
Figure A30: Eﬀects on Issue Polarization
Free trade good/bad
Police do/do not display racial bias
Response to alleged sexual harassment appropriate/hasty
Support for/against Brett Kavanaugh
Immigration good/bad
Robert Mueller is fair/biased
Gun laws too strong/weak
Media is fair/biased
Donald Trump is honest/dishonest
-.2 -.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
49
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A15: Eﬀects on Issue Polarization Using Unweighted Index
(1) (2)
Primary speciﬁcation
(standard deviation
weighted)
Robustness check
(equally weighted)
Share of time deactivated -0.10 -0.09
(0.04) (0.03)
Observations 1,450 1,450
Notes: This table presents local average treatment eﬀects of Facebook deactivation on issue polarization
estimated using Equation (1). Column 1 presents the speciﬁcation described in footnote 13 and presented
in the body of the paper. In this primary speciﬁcation, issue polarization is constructed by weighting each
of the nine issues by σq, the standard deviation of within-person changes on issue q, which allows us to
place higher weight on issues about which views are malleable over the deactivation period. This is how
we had originally analyzed the data. Column 2 presents a robustness check in which issue polarization is
constructed by weighting each issue equally. In both columns, issue polarization is normalized so that the
Control group endline distribution has a standard deviation of one.
Table A16: Robustness to Omitting Each Individual Variable from the Political Polarization
Index
Treatment
eﬀect
Standard
error P-value
Party aﬀective polarization -0.15 0.04 0.00
Trump aﬀective polarization -0.14 0.04 0.00
Party anger -0.15 0.04 0.00
Congenial news exposure -0.07 0.04 0.09
Issue polarization -0.14 0.04 0.00
Belief polarization -0.14 0.04 0.00
Vote polarization -0.16 0.04 0.00
Observations 1455
Notes: This table presents local average treatment eﬀects of Facebook deactivation on the political polarization
index estimated using Equation (1). All variables are normalized so that the Control group endline
distribution has a standard deviation of one. Each row omits the variable listed from the index. See Section
2.3 for variable deﬁnitions.
50
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Table A17: Correlation Between Subjective Well-Being Index and Demographics at
Baseline
(1)
Income ($000s) 0.0027
(0.0005)
College 0.2335
(0.0488)
Male 0.2033
(0.0482)
White -0.0066
(0.0531)
Age 0.0154
(0.0021)
Republican 0.2136
(0.0723)
Democrat -0.0492
(0.0507)
Observations 1,661
Notes: This table presents estimates of a regression of the baseline subjective well-being index on demographic
variables. The subjective well-being index is normalized to have a standard deviation of one.
51
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A31: Eﬀects on Subjective Well-Being Measured in Text Messages, By Week
-.10.1.2
Effect(stddev)
1 2 3 4
SMS happiness-.10.1.2
Effect(stddev)
1 2 3 4
SMS positive emotion
-.2-.10.1.2
Effect(stddev)
1 2 3 4
Weeks after beginning of deactivation period
SMS not lonely
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
52
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A32: Comparing Experimental and Non-Experimental Estimates of Eﬀects on
Subjective Well-Being
Happiness
Life satisfaction
Loneliness × (-1)
Depressed × (-1)
Anxious × (-1)
Absorbed
Bored × (-1)
SMS happiness
SMS positive emotion
SMS not lonely
Subjective well-being index
-.2 -.1 0 .1 .2 .3
Treatment effect (standard deviations)
Experimental
estimates
Unconditional
correlations
Conditional
correlations
Notes: The solid markers present local average treatment eﬀects of Facebook deactivation estimated using
Equation (1). The empty markers present non-experimental estimates from the following regression:
Y b
i = τ ˜Hi + βXi + i,
where Y b
i is participant i’s value of some outcome measured in the baseline survey, Xi is a vector of controls
(household income, age, and college, male, white, Republican, and Democrat indicators), and ˜Hi is baseline
average daily Facebook use over the past four weeks (winsorized at 120 minutes per day) divided by the
local average treatment eﬀect on average daily Facebook use between midline and endline. This division
makes experimental and non-experimental estimates comparable in the sense that they are both in units of
average use per day over the past four weeks. The empty diamond markers present unconditional correlations
(excluding Xi from the regressions), while the empty square markers present estimates conditional on Xi.
All variables are normalized so that the Control group endline distribution has a standard deviation of one.
Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
53
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A33: Baseline Opinions about Facebook
0.05.1.15.2
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
improves or worsens people's social lives
0.05.1.15.2
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
is good or bad for you
0.05.1.15.2
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
is good or bad for society
0.05.1.15.2
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
makes people more or less happy
Note: Long-dashed line is mean, short-dashed line is median
0.05.1.15.2.25
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
makes people more or less politically polarized
0.05.1.15.2.25
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
helps people follow the news better
0.05.1.15.2.25
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
gives more or less exposure to false news/clickbait
0.05.1.15.2.25
Fraction
-5 -4 -3 -2 -1 0 1 2 3 4 5
people wouldn't miss Facebook
Note: Long-dashed line is mean, short-dashed line is median
Notes: These ﬁgures present histograms of Facebook opinions from the baseline survey. Variables are resigned
so that “positive” views about Facebook are positive, “negative” views about Facebook are negative,
and zero is neutral. See Section 2.3 for variable deﬁnitions.
54
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A34: Eﬀects on Subjective Well-Being Components
Happiness
Happiness relative to peers
Life was ideal
Conditions of life were excellent
Satisﬁed with life
Lacked companionship × (-1)
Felt left out × (-1)
Isolated from others × (-1)
-.1 0 .1 .2
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. Each variable is one of the components that comprise
the outcomes Happiness, Life satisfaction, and Loneliness × (-1) respectively.
55
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A35: Eﬀects on Secondary Outcomes
Voted Republican
Voted (self-report)
-.15 -.1 -.05 0 .05
Treatment effect
(standard deviations)
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1). All variables are normalized so that the Control group endline distribution has a standard deviation of
one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
56
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A36: Eﬀects on Outcome Indices by Perceived Researcher Agenda
Substitute time uses index
Social interaction index
Substitute news sources index
News knowledge index
Political engagement index
Political polarization index
Subjective well-being index
Post-experiment use index
Facebook opinions index
-1 -.5 0 .5
Treatment effect (standard deviations)
Show Facebook good/no agenda/not sure
Show Facebook bad
Notes: This ﬁgure presents local average treatment eﬀects of Facebook deactivation estimated using Equation
(1) for participants who did vs. did not think that the researchers had an “agenda” to “show that Facebook
is bad for people.”All variables are normalized so that the Control group endline distribution has a standard
deviation of one. Error bars reﬂect 95 percent conﬁdence intervals. See Section 2.3 for variable deﬁnitions.
57
Online Appendix Allcott, Braghieri, Eichmeyer, and Gentzkow
Figure A37: Comparison to Demand Curves from Brynjolﬀson et al. (2018)
0
50
100
150
200
250
300
350
400
450
500
550
600
Price($)
0 10 20 30 40 50 60 70 80 90 100
Percent who keep Facebook
Our sample BEG (2018) US online panel, TIOLI
BEG (2018) European students, BDM BEG (2018) European students, TIOLI
Notes: This ﬁgure compares our demand curve (based on the distribution of willingness-to-accept to deactivate
for the four weeks after midline) to demand curves for one month of Facebook use from Brynjolfsson,
Eggers, and Gannamaneni (2018). “TIOLI” refers to their “take it or leave it” elicitation, whereas “BDM”
refers to their BDM elicitation. For their European student sample, valuations were elicited in Euros; we
transform these to dollars using the exchange rate when the elicitation was carried out in July 2017.
58