RESEARCH ARTICLE
The Democracy of Dating: How Political
Affiliations Shape Relationship Formation
Matthew J. Easton1
and John B. Holbein2
1
Department of Political Science, Brigham Young University, 745 Kimball Tower, Provo, UT 84602, USA,
matteaston@byu.edu, Twitter: @easton_matty and 2
Frank Batten School of Leadership and Public Policy,
University of Virginia, 111 Garrett Hall, Charlottesville, VA 22903, USA, jh5ak@virginia.edu, Twitter:
@johnholbein1
Abstract
How much does politics affect relationship building? Previous experimental studies have
come to vastly different conclusions – ranging from null to truly transformative effects. To
explore these differences, this study replicates and extends previous research by conducting
five survey experiments meant to expand our understanding of how politics does/does not
shape the formation of romantic relationships. We find that people, indeed, are influenced
by the politics of prospective partners; respondents evaluate those in the political outgroup
as being less attractive, less dateable, and less worthy of matchmaking efforts.
However, these effects are modest in size – falling almost exactly in between previous study
estimates. Our results shine light on a literature that has, up until this point, produced a
chasm in study results – a vital task given concerns over growing levels of partisan animus
in the USA and the rapidly expanding body of research on affective polarization.
Keywords: affective polarization; relationship formation; polarization; partisan attachments; replication;
survey experiment
Introduction
Affective polarization is the soup du jour in contemporary discussions of American
politics. Scholars define affective polarization as animosity between partisans
wherein “Democrats and Republicans both say that the other party’s members
are hypocritical, selfish, and closed-minded, and they are unwilling to socialize
across party lines, or even to partner with opponents in a variety of : : : activities”
Author order based on alphabetization; the authors contributed equally to this paper. We wish to thank
the National Science Foundation (SES-1657821) for funding support. We are grateful to Matthew Baldwin,
Michael Barber, David Broockman, Adam Dynes, Samantha Frazier, Jay Goodliffe, Chris Karpowitz,
Jeremy Pope, Julia Stamper and audiences at Brigham Young University, the Mary Lou Fulton
Undergraduate Research Conference, the 2018 MPSA, and the 2019 MPSA for their help and feedback
on this paper. The studies in this paper were approved by the Brigham Young University (E18118) and
University of Virginia (3698) Institutional Review Boards. The data, code, and any additional materials
required to replicate all analyses in this article are available at the Journal of Experimental Political
Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/24CXA7.
The authors have no conflicts of interest to report.
© The Experimental Research Section of the American Political Science Association 2020
Journal of Experimental Political Science (2020) 1–13
doi:10.1017/XPS.2020.21
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
(Iyengar and Krupenkin 2018). Researchers, journalists, and pundits alike are
increasingly concerned about rising levels of partisan animus (e.g. Iyengar et al.
2019; Mason 2018; Iyengar and Krupenkin 2018), lamenting that “Democrats
and Republicans alike are far more likely today than they were only a few decades
ago to say their rivals are not just wrong but stupid, selfish, and close-minded"1
and
that “politics seems more nasty, divided and polarized than ever.”2
According to
some, politics now influences everything from (perhaps) where we live (though
see Brown and Enos 2018; Gimpel and Hui 2015; Mummolo and Nall 2017), to
how we conduct economic transactions (Engelhardt and Utych 2018), to even
who cleans our teeth (Ladd 2018). But, just how much of a role does politics play
in the relationships we choose to (not) build with others?
To answer this question, we place our focus on estimating the causal effect of
political identities (i.e. partisanship, ideology, and candidate support) on the formation
of romantic relationships. This venue allows us to see whether political attachments
bleed into one of the foundational human behaviors (relationship building)
that is seemingly apolitical. Work on the effect of partisan animus on relationship
building is still in its nascent stages and leaves important questions unanswered. For
instance, from previous work it remains unclear 1.) just how much politics shapes
the crucial early stages of relationship building and 2.) whether any biases against
political out-groups are really simply dislike of politics in general (as some have
argued recently; e.g. Klar and Krupnikov 2016, C.4; Klar, Krupnikov, and Ryan
2018; Klofstad, McDermott, Hatemi 2012; Klofstad, McDermott, and Hatemi
2013). While a few previous studies have used romantic relationships to study
the role of partisan affect, the experimental literature on this topic has produced
vastly different results that range from small and not significant to large – observing
effects as large as a full standard deviation change in outcome measures – and statistically
significant. In short, this literature is definitively not settled. That previous
experiments have yielded effects as disparate as they have necessitates further exploration.
Before we can begin to understand why political biases exist and/or how to
address these biases (if at all), we have to come to a clearer understanding of the
extent and nature of these biases.
Our objective is to further explore these divergent findings by, first, attempting to
replicate previous results in the current political climate and, second, by attempting
to explain the differences in results obtained by previous work. To preview our findings
from the five survey experiments we run, we find clear evidence that political
bias in dating is almost exactly in between previous work – substantially larger than
what Huber and Malhotra (2017) show in their 2012 study but simultaneously substantially
smaller than what Nicholson et al. (2016) found in their own 2012 study.
Our effect estimates (from our studies pooled together) are much larger than the
effects identified by Huber and Malhotra (2017), and even when we make the treatments
as parallel as possible – by also manipulating the ideology of the prospective
partner – our effects are still much larger.3
At the same time, Nicholson et al. (2016)’s
1
See “We need political parties. But their rabid partisanship could destroy American democracy.” Vox,
September 5, 2017.
2
See “Politics Is More Partisan Now, But It’s Not More Divisive” FiveThirtyEight, January 19, 2018.
3
This is for Huber and Malhotra’s effect on attractiveness; we benchmark the other outcomes below.
2 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
effect is much larger than the partisan effect we estimate and the most similar treatment
we estimate – that of sharing one’s preferred candidate with a potential partner.
Going one step further, we are able to precisely rule out six different likely
explanations for the difference in previous findings, including the timing of the
study, the outcome measures used, the sampling framework employed, demand
effects, how the treatment is signaled, and the nuances of the treatment signal itself
(be it a party, ideology, or candidate support signal). Ruling out these explanations is
vitally important. Though we do not land on one definitive reason for the differences
in previous studies, we can say confidently that our results are remarkably
consistent across various design decisions – always being in between Huber and
Malhotra (2017) and Nicholson et al. (2016) – suggesting a modest, rather than null
or transformative, effect of politics on relationship building. Regardless of the exact
reasons driving the differences in previous research, our results suggest that while
political biases in relationship building are real (and are much larger today than
when Huber and Malhotra conducted their original study), politics plays a much
smaller role than Nicholson et al. (2016) have argued. This is true despite the fact
that direct survey-based measures of affective polarization have indicated an
increase in partisan animus in recent years. At best we can tell that it appears that
partisan bias has grown substantially since Huber and Malhotra did their original
study but that Nicholson et al. overestimated levels of political bias in relationship
formation.
Our results make several important conceptual and methodological contributions
to the experimental study of political animus in contemporary democracies.
Our work shows that while studies of political bias in nonpolitical domains are
robust to quite reasonable methodological changes in experimental design, this literature
is still susceptible to (1) the inherent problems associated with an overreliance
on individual studies to come to definitive conclusions about a general
phenomena and (2) the difficulties attached with studying an area that appears
to be (based on direct survey measures) rapidly changing over time.
Background and Conceptual Framework
Political animosity has increasingly taken center-stage in US politics (Mason 2018).
In recent years, media reports have shown people on both sides of the political
divide screaming at one another during protests, rallies, and on online platforms.
Indeed, recent research has shown that partisans often blatantly dehumanize members
of the opposing party (Cassese 2019). This gulf is especially pernicious as it
occurs even when Democrats and Republicans agree on policy (Mason 2018).
This heated political rhetoric has real-life implications for everyday Americans.
Scholars and pundits argue that levels of affective polarization are only going to
grow and continue to shape the behavior of elected officials (Diermeier and
Li 2019).4
Scholars have often used direct survey measures to measure political animus and
have argued that political hatred is growing and is at an all-time high (Iyengar et al.
4
We do not have space to review the full literature on affective polarization here. Those interested in this
larger literature should see the excellent recent review by Iyengar et al. (2019);
The Democracy of Dating 3
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
2019; Mason 2018; Iyengar and Krupenkin 2018; Iyengar, Sood, and Lelkes 2012).5
While directly asking people how much they dislike the out-party is one approach to
measuring partisan animus, it is certainly not the only – nor the best – way to do so.
Indeed, social desirability may mask individuals’ willingness to report their true feelings
toward the out-party (Iyengar and Westwood 2015; Iyengar et al. 2018). In
short, there are reasons to move beyond direct questioning to randomized control
trials.6
Experimental Work on Political Animus in Relationship Building
Two studies have experimentally tested how politics shapes relationship formation –
Huber and Malhotra (2017) and Nicholson et al. (2016). While important in their
own way, these two studies come to strikingly different conclusions about how
much politics influences relationship building.
Huber and Malhotra (2017) use a (2012) sample of individuals from Survey
Sampling International (SSI) and test whether randomly assigned ideological positions
influence respondents’ subjective evaluations of prospective candidate profiles.
They find that respondents evaluate potential dating partners more favorably and
are more likely to reach out to them when they (randomly) have similar political
characteristics. However, these effects are fairly small to modest substantively –
ranging from 0.9% to 9.4% of a standard deviation depending on the outcome
explored.
A similar experiment conducted by Nicholson et al. (2016) uses a sample from a
module of the 2012 Cooperative Congressional Election Study (CCES). They randomly
assign respondents to view a profile of an Obama supporter, a Romney supporter,
or one that had no stated candidate preference. In contrast to Huber and
Malhotra (2017), Nicholson et al. (2016) find large differences in how individuals
evaluate the attractiveness of prospective dating partners. Individuals randomly
assigned to view a dating profile from a supporter of the out-party’s candidate
for president assess that person as being a full 1.05 standard deviation
(p < 0.001) less attractive than individuals who support the in-party candidate.
(Attractiveness is the only outcome they examine.)
Perhaps it goes without saying, but when a literature that seeks to answer a question
comes to vastly different conclusions running another study is very valuable.
Seeing these two divergent results prevents us from coming to clear conclusions of
just how much politics shapes relationship building. This literature is decidedly not
settled.
5
For non-experimental studies of how politics shapes/forms relationships, see Alford et al. 2005; Alford
et al. 2011; Funk et al. 2013; Hatemi et al. 2009; Hatemi et al. 2010; Hatemi et al. 2011; Martin et al. 1986;
McDermott et al. 2014; Smith et al. 2011; Smith et al. 2012; Kandler et al. 2012; Luo and Klohnen 2005;
Petter 2018; Iyengar, Konitzer, and Tedin 2018; Hersh and Ghitza 2018; Klofstad, McDermott, Hatemi 2012;
Klofstad, McDermott, and Hatemi 2013; Frisbie 2016; Iyengar et al. 2019; Kiefer 2017; and Watson et al.
2004.
6
While experiments designed to detect partisan animus are not easy or straightforward, scholars have
shown their promise in this area (e.g. Iyengar et al. 2019; Iyengar and Westwood 2015; Gift and Gift
2015; McConnell et al. 2018; Michelitch 2015; Shafranek Forthcoming; Mason 2016).
4 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
Beyond simply replicating these studies, however, an important task is to try to
explain why the original two studies came to different conclusions. Doing so helps
push the literature forward. However, knowing exactly why these similar designs
yielded such vastly different results is tricky given that some results from individual
studies are just outliers, through no fault of the authors (hence the need for replication).
Though there are almost an infinite number of design choices that could
have produced differences in estimated effects, we argue that some of the most likely
are (1) the timing of the study (given the ever-changing self-reported levels of partisan
animus), (2) the outcome measures used, (3) the sampling framework
employed, (4) demand effects, (5) how the treatment is signaled, and (6) the nuances
of the treatment signal itself (be it a party, ideology, or candidate signal). We explore
all of these possibilities in this paper.
Data and Methods
To help increase our understanding of just how big of a role politics plays in relationship
formation, we ran five survey experiments. The first experiment (fielded on
MTurk) was meant to act as a baseline for our other results.7
Experiments 2–5
changed various aspects of the experimental design to further unpack the effects
(Easton and Holbein 2020).8
In the first experiment, we asked MTurk workers to view a hypothetical dating
profile in which we randomized (blocking by the respondent’s own political party
and gender preferences in dating) the party of the person in the profile (the profiles
are provided in the Online Appendix). The profiles were designed based on an
extensive review of current dating sites and were meant to mimic online dating profiles
in their design, amount of information provided, and type of information provided.
The experiment intentionally held constant various characteristics that
individuals might conflate with party (race, facial features, age, etc.). We then asked
survey-takers questions about the person in the profile’s attractiveness and the likelihood
they would respond to a message from, go on a date with, see themselves in a
relationship with, and set up that person listed in the profile. (Though the survey
block led with an encouragement for people to imagine that they were single and
looking for someone to date, this last outcome was included for individuals in steady
relationships that had a difficult time doing so. As it turns out, our results don’t vary
by relationship status of the individual taking the survey.) For the full block of survey
ordering and questions, see the Online Appendix.
Our second experiment exactly replicated the design from the first experiment on
the CCES. This allowed us to see whether the sample used drives any differences in
observed effects. It allowed us to improve on the external validity dimension and
also provided us with the exact same sampling framework as Nicholson et al. (2016).
7
Our design choices for our baseline experiments (1 and 2) were preregistered at the OSF (see https://osf.
io/vufxe/). Experiments 3–5 built on these and tested complementary channels, while still following the
protocols of the original pre-analysis plan.
8
Research on the generalizability of effects on Mturk has found that “more representative of the U.S.
population than in-person convenience samples—the modal sample in published experimental political science—but
less representative than subjects in Internet-based panels or national probability samples”
(Berinsky, Huber, and Lenz 2012; see also Boas, Christensen, and Glick 2020).
The Democracy of Dating 5
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
Our third experiment had the exact same design as Experiments 1 and 2 with one
additional wrinkle that allowed us to test for demand effects, using the exact same
format as Mummolo and Peterson (2019; see the Online Appendix for the wording
we used).
In our fourth experiment, we manipulated how the treatment was delivered.
Instead of having the (fictional) people in the dating profiles self-signal their politics,
we had the surveyors (us) deliver the signal on an earlier screen. This variation in
treatment serves more than just an attempt to see if results are robust to treatment
delivery modes. It also helps address a potential concern in the study of political
animus that the signaling party simultaneously signals other things about the person
providing the signal. Some may worry that self-signaling politics conflates two treatment
effects into one – the first being the effect of political affiliation (what we are
after) and the second being an additional penalty that might be given to individuals
who are especially vocal/opinionated about their political preferences (e.g. Klar and
Krupnikov 2016; Klar et al. 2018). To mute this potential concern, experiment four’s
design was meant to mimic (as best as possible) an environment where a person
finds out information about a potential partner through indirect means. While this
approach doesn’t completely eliminate the possibility that the person in the dating
profile is over-zealous of their political values, it does tone down this potential
mechanism.
In our fifth experiment, we explored two similar reasons that the effects might
vary – timing and type of treatment being administered – be it a signal about one’s
ideology, candidate support, or partisanship. Previous work – including the two
experimental studies we are building upon – often bounces around between these
types of treatments. But they need not have the same effect, especially given that
people may hold different attachments to their belief systems, political groups,
and/or candidates they choose to support. So, we tested all three manipulations
at the same time.
Before turning to our results, we pause here to (briefly) note that one of the inherent
and unavoidable challenges of replications/extensions is that the sample and
design being used can never be exactly the same as previous studies. For example,
in manipulating candidate support it does not make sense to use the same candidates
that Nicholson et al. (2016) used – i.e. Romney and Obama – given that these
two are no longer candidates for higher office. That being said, below we do our best
to make our samples and designs parallel with the most-related studies to ours
(Huber and Malhotra 2017 and Nicholson et al. 2016). Like all replications, ours
is a conceptual, not an exact, replication – though our replications are very close
to the original studies. Moreover, it’s important to note that if the effects previously
observed are so sensitive to design choices, the effect being estimated may not be as
robust or generalizable as previously thought.
Results
Figure 1 provides an overview of our results. (It focuses on the attractiveness outcome
– the only one contained in all three studies.) As was mentioned earlier,
Figure 1 shows the massive gap between the findings from Huber and Malhotra
6 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
(2017) and Nicholson et al. (2016). The difference between the two is a staggering
whole standard deviation.
We start first by examining the effects of our party manipulation. Figure 1 shows
that our partisanship treatments elicit a 0.408 standard deviation (95% CI: 0.33,
0.49) change when we pool our five experiments together. That is to say, being
Figure 1
The Effect Political Similarity on Evaluations of Attractiveness. [A] Comparison Across Studies. [B]
Study Effects Relative to Permutation Estimates. Note: the effect of being in a similar group to the
dating profile on perceived levels of attractiveness. Effects are plotted as points, with corresponding
90% (narrow line) and 95% (wider line) confidence intervals. Figure benchmarks our four estimates
(middle) and a pooled estimate from a meta-analysis of these (highlighted with a box) to an ideology
treatment from Huber and Malhotra (2017) (on the top) and a candidate treatment from Nicholson
et al. (2016) (on the bottom). In our study 3, we use only those in the no-demand effects condition to
parallel across the treatments. N’s from top to bottom: 9790 (unit of analysis: individual profile);
513 (individuals); 457 (individuals); 387 (individuals); 725 (individuals); 1786 (individuals); 3868
(individuals); 570 (individuals); and 1786 (individuals). We exactly replicate the models used in Huber
and Malhotra (2017) and Nicholson et al. (2016), with the one exception of standardizing
the outcome measures.
The Democracy of Dating 7
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
randomly assigned to a treatment of seeing a profile of someone who is the same
political party as you has a modest effect on how attractive you rate them. These
estimates are statistically significant at the 0.1% level and also distinct from random
shuffles in permutation tests (see panel B). They are also quite different from what
previous research has shown. Our pooled party effect is much smaller than
Nicholson et al. (2016)’s effect and much larger than that shown by Huber and
Malhotra (2017). Simply put, our results suggest that politics does not have a transformational
effect on dating relationships but also does not have null effects.
This conclusion holds across reasonable design changes that could be explaining
the chasm between previous results. First, Figure 1 shows that even when we use the
exact same sampling frame as used by Nicholson et al (2016) – i.e. the CCES (our
Study 2) – their results are statistically (p < 0.05) and substantively (39%) larger
than ours. Also, we should note that the attractiveness effect is uniquely large.
Second, our conclusion of a modest, not a null or transformational, effect holds
when we make the treatments as similar as possible – testing simultaneously the
effect of party, ideological, and candidate alignment (our Study 5). As can be seen,
the reason for the differences between the Huber and Malhotra (2017) and
Nicholson et al. (2016) studies does not appear to be because of a different individual
identity being manipulated. In both cases, our treatments are substantively and
significantly distinct from previous research. When we manipulate ideology, our
effects are much larger than Huber and Malhotra (2017)’s. When we manipulate
candidate affiliation, Nicholson et al. (2016)’s are still much larger. In fact, our ideological
and candidate manipulations are much more similar in size – and not statistically
distinct from – to our party treatments. Regardless of how you administer
the treatment, politics has a modest – not a null or transformational – effect. (For all
effects measured in Study 5, see Figure A4 in the Online Appendix.)
Third, our results suggest that the differences seen by Huber and Malhotra (2017)
and Nicholson et al. (2016) are not due to timing. This may seem intuitive given that
both of the earlier studies were conducted in 2012 (though perhaps in different
months; the replication materials in the two papers are unclear on this point).
However, our Experiment #5 affords us direct evidence that even when we equalize
timing down to the exact same study fielded at the exact same time, we do not see
effects similar to those observed by Huber and Malhotra (2017) and Nicholson et al.
(2016). Why is this? Though it’s tough to know for sure, one explanation is that our
results are consistent with partisan animus having grown since 2012 and Nicholson
et al. (2016)’s estimates being an outlier.
Fourth, the differences are not explained by how the treatment was administered –
be it by the person in the profile or someone else (i.e. Study 4). Figure 1 also shows the
effect of the potential partner being in an out-party on evaluations on measures of
attractiveness. As can be seen, the treatment effect is still large in size, statistically significant,
and not statistically or substantively distinct from our two other MTurk experiments.
Simply changing the delivery mechanism still keeps us in the moderate effect
domain.
Fifth, our results of a modest, not null or transformational, effect holds when we
look across different outcomes of dateability. As Figures 2 and 3 illustrate, the same
part effect persists on our other measures that we explore. In addition to showing
that our results hold across various outcome measures, our results suggest in total
8 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
that partisan affect may be more than just disliking partisanship generally as some have
suggested (e.g. Klar and Krupnikov 2016, C.4; Klar et al. 2018; Klofstad et al. 2012,
2013). If this were the case, we would expect to see dating profiles with any partisan
information being penalized regardless of whether a person identifies with the same
or a different party. Instead, we see a partisan asymmetry: individuals’ party plays a
strong role in influencing how they evaluate people from the same or different parties.
Finally, the differences in previous estimates are not explained by demand effects.
Should demand effects exist, we would expect significant differences between those
who were randomly assigned to be told the research question and those that were
not. However, we find that there are no such differences. Our treatment effects are
statistically indistinguishable and substantively the same, regardless of whether we
tell the respondent our research question or not.
In short, regardless of the sampling framework used, the timing of the survey, the
treatment used, how the treatment is administered, the outcomes one looks at, and
demand effects are minimized, the results are the same – politics has modest, but not
null or transformational, effects on relationship building.
Conclusion
This manuscript provides evidence that romantic relationships are influenced by
politics, albeit simultaneously less and more than previous research has suggested.
Figure 2
Benchmarking to Previous Studies (Respond to Message). Note: figure displays the effect of being in a
similar group to the dating profile on the respondent’s willingness to respond to a message. Effects are
plotted as points, with corresponding 90% (narrow line) and 95% (wider line) confidence intervals.
Figure benchmarks our three estimates (middle) from Studies 1, 2, 3, 4, and 5 and a pooled estimate
from these to an ideology treatment from Huber and Malhotra (2017) (on the top) and a same
candidate treatment (from study 5). In our study 3, we use only those in the no-demand effects
condition to parallel across the treatments. (Nicholson et al. 2016 is removed from this comparison as
they only look at attractiveness.) N’s from top to bottom: 9790 (unit of analysis: individual profile – the
rest are individual level); 1786, 513, 457, 387, 725, 1786, 3868, 1786.
The Democracy of Dating 9
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
We have documented that this modest – but not null or transformational – effect is
robust to many different experimental designs. These show that previous studies
have given us an incomplete picture of how politics shapes the vitally important
formation of relationships.
We note that there are important limitations to our analysis. First, we acknowledge
that our experiment (like the experiments that preceded it) uses hypothetical
profiles on a survey experiment and may not fully capture the effect of political
affiliations on actual dating platforms such as Tinder or OKCupid. However, we
(and our Institutional Review Boards) felt ethically restrained from creating false
dating profiles on these platforms (what comes dangerously close to what some people
call “catfishing”), as (among other reasons) under this design respondents would
be unable to give their consent before engaging in the experiment. Further, our goal
was to explore previous studies which also used hypothetical dating profiles. Given
all this, using a survey experiment was the best available option. Second, like other
experiments on politics and relationship formation, we have not tackled the extent
to which partisan penalties are actually reflective of other intuited characteristics.
While some experimental research on roommate selection suggests that when other
individual characteristics are randomized politics that trumps other individual characteristics
(Shafranek Forthcoming), we cannot be certain this holds in relationship
building. Future work would do well to explore the extent to which citizens use
Figure 3
Benchmarking to Previous Studies (Be in a Relationship). Note: figure displays the effect of being in a
similar group to the dating profile on their willingness to see themselves in a long-term relationship
with the person. Effects are plotted as points, with corresponding 90% (narrow line) and 95% (wider
line) confidence intervals. Figure benchmarks our three estimates (middle) from Studies 1, 2, 3, 4, 5
and a pooled estimate from these to an ideology treatment from Huber and Malhotra (2017) (on the
top). In our study 3, we use only those in the no-demand effects condition to parallel across the
treatments. (Nicholson et al. 2016 is removed from this comparison as they only look at attractiveness.)
N’s from top to bottom: 9790 (unit of analysis: individual profile – the rest are individual level);
1786, 513, 457, 387, 725, 1786, 3868, 1786.
10 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
partisanship to intuit other characteristics of individuals and whether providing
additional information affects the effects we have explored. Third, future work
would do well to further consider the effect of the sample on estimates of partisanship
on relationship building. While we have done so in some here (e.g. using
MTurk and the CCES), future research would do well to consider effects in particular
in online opt-in panels (e.g. Lucid). The extent to which our findings will differ
from these samples will depend on the extent of treatment effect heterogeneity,
where recent scholarship has shown to be fairly minimal across many survey experiments
(e.g. Coppock, Leeper, and Mullinix 2018; Coppock and McClellan 2019).
Finally, our outcomes don’t fully measure all aspects of a relationship, especially
preferences that develop later in the dating process. While later-stage relationship
building is important to measure, we purposefully designed our experiment to capture
earlier inclinations that online daters are more likely to experience. The early
part of relationship formation is especially important, given the potential for transformation
and convergence once relationships have been well established (Iyengar
et al. 2019).
There is still much to be explored regarding affective polarization and romantic
relationships. For example, there remains an important unanswered normative
question in the literature on affective polarization about whether people should
or should not be sorting by politics. While this question will inevitably come down
to the values that one holds, future work would still do well to study the broader
implications of partisan sorting for the well-being of individuals and society as a
whole. While we have focused on the nature and the extent of bias, future work
would do well to consider the consequences of such bias on the overall health of
individuals, their relationships, and the communities in which they live.
Supplementary material. To view supplementary material for this article, please visit https://doi.org/10.
1017/XPS.2020.21
References
Alford, John R., Carolyn L. Funk, and John R. Hibbing. 2005. Are Political Orientations Genetically
Transmitted? American Political Science Review 99(2): 153–67.
Alford, John R., Peter K. Hatemi, John R. Hibbing, Nicholas G. Martin, and Lindon J. Eaves. 2011. The
Politics of Mate Choice. Journal of Politics 73(2): 362–379.
Berinsky, Adam J., Gregory A. Huber, and Gabriel S. Lenz. 2012. Evaluating Online Labor Markets for
Experimental Research: Amazon. com’s Mechanical Turk. Political Analysis 20(3): 351–68.
Boas, Taylor C., Dino P. Christenson, and David M. Glick. 2020. Recruiting Large Online Samples in the
United States and India: Facebook, Mechanical Turk, and Qualtrics. Political Science Research and
Methods 8(2): 232–50.
Brown, Jacob R., and Ryan D. Enos. 2018. Partisan Segregation. Working Paper. Retrieved from https://
scholar.harvard.edu/files/renos/files/brownenos. pdf
Cassese, Erin C. 2019. Partisan Dehumanization in American Politics. Political Behavior 1–22, forthcoming.
Coppock, Alexander and Oliver A. McClellan. 2019. Validating the Demographic, Political, Psychological,
and Experimental Results Obtained from a New Source of Online Survey Respondents. Research &
Politics 6(1): 2053168018822174.
Coppock, Alexander, Thomas J. Leeper, and Kevin J. Mullinix. 2018. Generalizability of Heterogeneous
Treatment Effect Estimates Across Samples. Proceedings of the National Academy of Sciences 115(49):
12441–12446.
The Democracy of Dating 11
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
Diermeier, Daniel and Christopher Li. 2019. Partisan Affect and Elite Polarization. American Political
Science Review 113(1): 277–281.
Easton, Matthew J. and John B. Holbein. 2020 Replication Data for: The Democracy of Dating: How Political
Affiliations Shape Relationship Formation. Harvard Dataverse. https://doi.org/10.7910/DVN/24CXA7
Engelhardt, Andrew M. and Stephen M. Utych. 2018. Grand Old (Tailgate) Party? Partisan Discrimination
in Apolitical Settings. Political Behavior: 1–21, forthcoming.
Funk, Carolyn L., Kevin B. Smith, John R. Alford, Matthew V. Hibbing, Nicholas R. Eaton, Robert F.
Krueger, Lindon J. Eaves, and John R. Hibbing. 2013. Genetic and Environmental Transmission of
Political Orientations. Political Psychology 34(6): 805–819.
Garrett, R. Kelly, Shira Dvir Gvirsman, Benjamin K. Johnson, Yariv Tsfati, Rachel Neo, and Aysenur
Dal. 2014. Implications of pro-And Counterattitudinal Information Exposure for Affective Polarization.
Human Communication Research 40(3): 309–332.
Gelman, Andrew. 2015. The Connection Between Varying Treatment Effects and The Crisis of
Unreplicable Research: A Bayesian Perspective. Journal of Management 41(2): 632–643.
Gift, Karen and Thomas Gift. 2015. Does Politics Influence Hiring? Evidence from A Randomized
Experiment. Political Behavior 37(3): 653–675.
Gimpel, James G. and Iris S. Hui. 2015. Seeking Politically Compatible Neighbors? The Role of
Neighborhood Partisan Composition in Residential Sorting. Political Geography 48: 130–142.
Hatemi, Peter K., John R. Alford, John R. Hibbing, Nicholas G. Martin, and Lindon J. Eaves. 2009. Is
There a “Party” in Your Genes? Political Research Quarterly 62(3): 584–600.
Hatemi, Peter K., John R. Hibbing, Sarah E. Medland, Matthew C. Keller, John R. Alford, Kevin B.
Smith, Nicholas G. Martin, and Lindon J. Eaves. 2010. Not By Twins Alone: Using the Extended
Family Design to Investigate Genetic Influence on Political Beliefs. American Journal of Political
Science 54(3): 798–814.
Hatemi, Peter K., Nathan A. Gillespie, Lindon J. Eaves, Brion S. Maher, Bradley T. Webb, Andrew C.
Heath, and Sarah E. Medland et al. 2011. A Genome-Wide Analysis Of Liberal And Conservative
Political Attitudes. The Journal of Politics 73(1): 271–285.
Hersh, Eitan and Yair Ghitza. 2018. Mixed Partisan Households and Electoral Participation in the United
States. PloS One 13(10): e0203997.
Huber, Gregory A. and Neil Malhotra. 2017. Political Homophily in Social Relationships: Evidence from
Online Dating Behavior. The Journal of Politics 79(1): 269–283.
Imai, Kosuke and Marc Ratkovic. 2013. Estimating Treatment Effect Heterogeneity in Randomized
Program Evaluation. The Annals of Applied Statistics 7(1): 443–470.
Iyengar, Shanto, Gaurav Sood, and Yphtach Lelkes. 2012. Affect, Not Ideology: A Social Identity
Perspective on Polarization. Public Opinion Quarterly 76(3): 405–431.
Iyengar, Shanto and Masha Krupenkin. 2018. The Strengthening of Partisan Affect. Political Psychology
39(2): 201–218.
Iyengar, Shanto and Sean J. Westwood. 2015. Fear and Loathing Across Party Lines: New Evidence on
Group Polarization. American Journal of Political Science 59(3): 690–707.
Iyengar, Shanto, Tobias Konitzer, and Kent Tedin. 2018. The Home as a Political Fortress: Family
Agreement in an Era of Polarization. The Journal of Politics 80(4): 1326–1338.
Iyengar, Shanto, Yphtach Lelkes, Matthew Levendusky, Neil Malhotra, and Sean J. Westwood. 2019. The
Origins and Consequences of Affective Polarization in the United States. Annual Review of Political
Science 22: 633–645, forthcoming.
Kandler, Christian, Wiebke Bleidorn, and Rainer Riemann. 2012. Left or Right? Sources of Political
Orientation: The Roles of Genetic Factors, Cultural Transmission, Assortative Mating, and
Personality. Journal of Personality and Social Psychology 102(3): 633.
Klar, Samara and Yanna Krupnikov. 2016. Independent Politics: How American Disdain for Parties Leads
to Political Inaction. New York, NY: Cambridge University Press.
Klar, Samara, Yanna Krupnikov, and John Barry Ryan. 2018. Affective Polarization or Partisan Disdain?
Untangling a Dislike for the Opposing Party from a Dislike of Partisanship. Public Opinion Quarterly
82(2): 379–390.
Klofstad, Casey A., Rose McDermott, and Peter K. Hatemi. 2012. Do Bedroom Eyes Wear Political
Glasses? The Role of Politics in Human Mate Attraction. Evolution and Human Behavior 33(2): 100–108.
12 Matthew J. Easton and John B. Holbein
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of
Klofstad, Casey A., Rose McDermott, and Peter K. Hatemi. 2013. The Dating Preferences of Liberals and
Conservatives. Political Behavior 35(3): 519–538.
Ladd, Jonathan 2018. Affective and Perceptual Polarization Among Party Activists. Working Paper.
Retrieved from https://www.jonathanmladd.com/uploads/5/3/6/6/5366295/polarizationactivists_
ladd_jan11_.pdf
Levendusky, Matthew S. 2018. Americans, Not Partisans: Can Priming American National Identity Reduce
Affective Polarization? The Journal of Politics 80(1): 59–70.
Levendusky, Matthew and Neil Malhotra. 2016. Does Media Coverage of Partisan Polarization Affect
Political Attitudes? Political Communication 33(2): 283–301.
Luo, Shanhong and Eva C. Klohnen. 2005. Assortative Mating and Marital Quality in Newlyweds: A
Couple-Centered Approach. Journal of Personality and Social Psychology 88(2): 304.
Martin, Nicholas G., Lindon J. Eaves, Andrew C. Heath, Rosemary Jardine, Lynn M. Feingold, and Hans
J. Eysenck. 1986. Transmission of Social Attitudes. Proceedings of the National Academy of Sciences
83(12): 4364–4368.
Mason, Lilliana. 2015. “I Disrespectfully Agree”: The Differential Effects of Partisan Sorting on Social and
Issue Polarization. American Journal of Political Science 59(1): 128–145.
Mason, Lilliana. 2016. A Cross-Cutting Calm: How Social Sorting Drives Affective Polarization. Public
Opinion Quarterly 80(S1): 351–377.
Mason, Lilliana. 2018. Uncivil Agreement: How Politics Became Our Identity. Chicago, IL: University of
Chicago Press.
McConnell, Christopher, Yotam Margalit, Neil Malhotra, and Matthew Levendusky. 2018. The
Economic Consequences of Partisanship in a Polarized Era. American Journal of Political Science
62(1): 5–18.
McDermott, Rose, Dustin Tingley, and Peter K. Hatemi. 2014. Assortative Mating on Ideology Could
Operate through Olfactory Cues. American Journal of Political Science 58(4): 997–1005.
Michelitch, Kristin. 2015. Does Electoral Competition Exacerbate Interethnic or Interpartisan Economic
Discrimination? Evidence from a Field Experiment in Market Price Bargaining. American Political
Science Review 109(1): 43–61.
Mummolo, Jonathan, and Erik Peterson. 2019. Demand Effects in Survey Experiments: An Empirical
Assessment. American Political Science Review 113(2): 517–529.
Mummolo, Jonathan, and Clayton Nall. 2017. Why Partisans Do Not Sort: The Constraints On Political
Segregation. The Journal of Politics 79(1): 45–59.
Nicholson, Stephen P., Chelsea M. Coe, Jason Emory, and Anna V. Song. 2016. The Politics of Beauty:
The Effects of Partisan Bias on Physical Attractiveness. Political Behavior 38(4): 883–898.
Shafranek, Richard M. Forthcoming. Political Considerations in Nonpolitical Decisions: A Conjoint
Analysis of Roommate Choice. Political Behavior: 1–30.
Smith, Kevin B., Douglas R. Oxley, Matthew V. Hibbing, John R. Alford, and John R. Hibbing. 2011.
Linking Genetics and Political Attitudes: Reconceptualizing Political Ideology. Political Psychology 32(3):
369–397.
Smith, Kevin, John R. Alford, Peter K. Hatemi, Lindon J. Eaves, Carolyn Funk, and John R. Hibbing.
2012. Biology, Ideology, and Epistemology: How Do We Know Political Attitudes Are Inherited and Why
Should We Care? American Journal of Political Science 56(1): 17–33.
Watson, David, Eva C. Klohnen, Alex Casillas, Ericka Nus Simms, Jeffrey Haig, and Diane S. Berry.
2004. Match Makers and Deal Breakers: Analyses of Assortative Mating in Newlywed Couples.
Journal of Personality 72(5): 1029–1068.
Westwood, Sean and Erik Peterson. 2019. Compound Political Identity: How Partisan and Racial Identities
Overlap and Reinforce. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3417476
(July 4, 2020).
Cite this article: Easton MJ and Holbein JB. The Democracy of Dating: How Political Affiliations Shape
Relationship Formation. Journal of Experimental Political Science. https://doi.org/10.1017/XPS.2020.21
The Democracy of Dating 13
use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/XPS.2020.21
Downloaded from https://www.cambridge.org/core. King's College London, on 29 Jul 2020 at 11:46:25, subject to the Cambridge Core terms of