13
Media Psychophysiology and Neuroscience
Bringing Brain Science into Media Processes
and Effects Research
Paul D. Bolls, René Weber, Annie Lang, and Robert F. Potter
Welcome to a new age of studying the human brain “on” media! Media researchers now
enthusiastically embrace theories and methods from brain science to examine how media
effects emerge from embodied mental processes. This enthusiasm is seen in recent special
editions of journals (Afifi & Floyd, 2015; Weber, 2015), handbooks from the International
Communication Association (ICA; e.g., Floyd & Weber, in press), the significant presence
of related research at major communication conferences (e.g., the ICA’s Communication
Science and Biology Interest Group), as well as advertised tenure-track faculty openings.
We are now past the stage where the use of physiological indicators of brain activity in
media research was viewed as novel by some and with suspicion by others. Various forms
of brain science are now seen as some of the most promising approaches to shedding
light on the complex and dynamic ways that media effects occur. Physiological indicators
of brain activity are not only a permanent fixture in the media researcher’s toolkit (Potter
& Bolls, 2012), but neuroscience—the formal discipline of brain science—is becoming
firmly established as one of the disciplines providing the intellectual roots for communication
science. It is an exciting time for new and more experienced researchers to learn how brain
science can be applied to studying media processes and effects.
This chapter overviews how media psychophysiology and neuroscience offer researchers
a valuable framework for investigating media processes and effects. We avoid a detailed technical
discussion of psychophysiology and neuroscience in favor of a more conceptual overview
of how the framework presented here can be applied. We also attempt to introduce a vast
amount of available resources to help interested scholars.
The chapter begins with a discussion of how a framework for the application of brain science
in media research emerged from principles of psychophysiology and neuroscience. Next, we
review common physiological indicators used in media research labs and recent studies using
these measures. We conclude with a look to the future and consider specific ways that this
framework can advance knowledge of media processes and effects.
195
196 • Paul D. Bolls et al.
The Media Psychophysiology and Neuroscience Paradigm
The previous edition of this book (Bryant & Oliver, 2009) included a chapter authored by
three of us (Lang, Potter, & Bolls, 2009). That chapter introduced psychophysiology as
a new paradigm for studying how individuals cognitively and emotionally process media.
Psychophysiology, at that time, was familiar to a relatively small group of researchers studying
media processes and effects. That has changed. Familiarity with psychophysiology has
grown, but we must avoid forgetting the important role an historical paradigm change
played in this growth to avoid repeating past mistakes. This change occurred when media
researchers embraced the psychophysiological paradigm. Our review of this history focuses
on four major phases.
Phase 1: the Beginning of Psychology and Early Inspiration for Brain Science
The first era of formal social science research in psychology included a theoretical focus on
brain physiology and mental processes. For example, a consideration of brain activity as fundamental
to human nature can be seen in the development of the earliest theories of emotion.
Those earliest theories wrestled with the relationship between physiological responses and
mental experiences, with three predominant views emerging: physiological responses as primary
drivers of emotion (James, 1894), conscious experiences of emotion eliciting specific physiological
responses (Cannon, 1931), and emotion unfolding from the interaction of physiological
activity and mental experiences (Schachter & Singer, 1962). During this time, Elizabeth Duffy
(1957) also identified arousal as a drive fueling human behavior, and Donald Hebb (1955) identified
brain systems underlying arousal. However, this work lost prominence in psychology
when the field enthusiastically embraced behaviorism.
Nevertheless, interest in physiological responses as the drivers of emotion trickled into media
effects research in the early 20th century, by way of a relatively unknown experiment that was
part of the famous Payne Fund studies (Dysinger & Ruckmick, 1933). The researchers studied
arousal in response to movies by recording the skin conductance of viewers. This study is
unfamiliar to most because it was overshadowed by the work of many media effects researchers
who joined psychology in embracing behaviorism.
Phase 2: the Age of Behaviorism in Psychology and Media Effects Research
Three of the most famous researchers in psychology—John Watson, B. F. Skinner, and Ivan
Pavlov—are known for ushering in the age of behaviorism (Hergenhahn & Henley, 2013). Watson’s
behaviorist view of psychological research left no doubt concerning the legitimate focus of
psychology (Watson, 1913). His view completely dismissed the notion that mental processes can
be validly studied. Pavlov and Skinner are known for their pioneering research on learning and
conditioning grounded in behaviorism (Skinner, 1938). Their studies represent the purest form
of behavioristic research rooted in what has been termed the stimulus-response paradigm
(Potter & Bolls, 2012).
Media effects research emerged under the stimulus-response paradigm; most histories identify
behaviorism as the scientific foundation for the field (Delia, 1987; Dennis & Wartella,
1996). It was during the age of behaviorism that Albert Bandura conducted his famous Bobo
doll studies on the effects of exposure to media violence (e.g., Bandura, Ross, & Ross, 1963;
Media Psychophysiology and Neuroscience • 197
see also Chapters 7 and 14 in this volume). Behavioral effects of media are still as crucial to
understand today as they were at that time. But behaviorism left researchers with a significant
blind spot: a lack of scientific understanding of what creates behavioral responses to meaningful
stimuli like media content. This limitation led to the first of two major paradigm shifts
discussed herein. The first paradigm change is associated with the “cognitive revolution” in
psychology and resulted in the adoption of the information processing paradigm (Potter &
Bolls, 2012).
Phase 3: the Cognitive Revolution Spreads through Psychology and Media Effects Research
Thanks to the cognitive revolution in psychology, it is now commonly accepted that understanding
behavioral responses to stimuli (including those tied to media exposure) requires
observing what occurs in the minds and bodies of individuals between exposure to a stimulus
and the response. Potter and Bolls (2012) labeled this mental activity intervening processes.
Thus, the paradigm shift from behaviorism to information processing can better be thought of
as the shift from a stimulus-response paradigm to a stimulus-intervening processes-response
paradigm.
Under behaviorism, intervening processes were thought to be hidden within the “black box”
of the human mind, which could not be validly observed. The cognitive revolution in psychology
focused on bursting open this supposed “black box” and systematically studying how cognitive
processes unfold as individuals take in, process, and respond to information in the
environment. This approach came to be known as information processing (for an excellent
review of this intellectual history, see Lachman, Lachman, & Butterfield, 1979).
Media researchers working in the late 1970s and 1980s adopted the information-processing
paradigm, calling for research to investigate related mental processes and effects (Chaffee, 1977;
Geiger & Newhagen, 1993). It was during this phase that psychophysiological measurement
reentered media effects research (e.g., Zillmann & Bryant, 1974), with physiological activity
examined as a response to media content. However, research conducted through this lens produced
inconsistent results (Potter & Bolls, 2012). We have since come to understand that
physiological systems do not respond in a unitary correlated linear fashion (Stern, Ray, & Quigley,
2001). Our increasing knowledge of these systems led to a second significant paradigm
shift, when the psychophysiological paradigm was adopted by the researchers in the late 1980s
(Lang et al., 2009).
Phase 4: the Psychophysiological Paradigm Emerges
The emergence of psychophysiology in psychology began before the cognitive revolution with
the work of researchers in the 1960s who renewed interest in physiological activity as an
explanation of human nature and experience (Berntson & Cacioppo, 2000). This work laid the
foundation for physiological activity to remain a permanent focus of psychological science. It
also enabled the birth of the Society for Psychophysiological Research and the formation of the
psychophysiological paradigm (Cacioppo, Tassinery, & Berntson, 2007). Development of this
new paradigm led to the third phase of psychophysiological measurement in media processes
and effects research (Potter & Bolls, 2012). With the shift, media researchers moved away from
viewing physiological activity as a response to media content. Instead, they came to view specific
patterns of physiological activity as indicators of embodied mental processes.
198 • Paul D. Bolls et al.
The psychophysiological paradigm is the foundation for the media psychophysiology and
neuroscience we reference in the title of this chapter. The term media psychophysiology was
introduced by Potter and Bolls (2012) and generally refers to the application of psychophysiological
science in media processes and effects research. The discipline of neuroscience is one of
the disciplines psychophysiology draws from. Neuroscience, however, is more focused on directly
recording brain activity through physiological indicators of central nervous system activity.
Neuroscience also requires different analytical tools and techniques (for more on the application
of neuroscience in media research, see Weber, Sherry, & Mathiak, 2008). The assumptions of
the psychophysiological paradigm apply to both areas. We discuss three of the most fundamental
assumptions below (for more detail, see Cacioppo et al., 2007, an essential resource for those
wishing to adopt these approaches).
The Human Mind Is Embodied
First, the psychophysiological paradigm assumes that all forms of human mental activity exist
in the brain and are observable through specific patterns of nervous system activity that echo
through the brain and body. These patterns can be complex, can unfold on various temporal
and spatial scales, and may require further refined measurement techniques, but these patterns
as an index of mental activity do exist.
Embodied Mental Activity Unfolds across Time
Second, psychophysiological measures indicate variation in mental processes across time as an
individual perceives and responds to a stimulus. Instant-by-instant changes in physiological
activity that reflect mental processes have extremely important psychological meaning. In other
words, dependent on how a physiological response varies over time, the same physiological
indicator can have different meanings and interpretations. For instance, different patterns of
heart rate deceleration and acceleration across time can indicate either an orienting or startle
response to a stimulus.
The Purpose of Physiology Is to Sustain Life, Not Index Mental Processes
Third, physiological indicators of mental processes generate a very small signal. This makes the
application of physiological indicators of mental activity in research a challenging proposition
that requires detailed training in the approach. Researchers who deploy psychophysiological
indicators to study mental activity must validly record small levels of variation in nervous
system signals that actually reflect variation in mental processes and are distinct from noise.
Measurement Science in Media Psychophysiology and Neuroscience
Measurement science in psychophysiology focuses on validating how specific patterns of nervous
system activity can be mapped onto specific mental processes. This effort is consistent
with the formal definition of psychophysiology, which can be summarized as the scientific study
of psychological phenomena as revealed through physiological activity of the functioning nervous
system (Cacioppo et al., 2007). Two domains of measurement science in psychophysiology
exist: the physiological domain consisting of nervous system activity, and the psychological
Media Psychophysiology and Neuroscience • 199
domain consisting of mental processes. Researchers face the difficult challenge of validating
complex patterns of activity in the physiological domain as psychophysiological indicators of
specific mental processes. This challenge is compounded by the fact that physiological responses
can often be mapped to different mental processes. The introduction of neuroscience further
complicates the picture, as researchers attempt to draw conclusions about mental processes
from specific patterns of brain activity while recognizing the complexity of the human brain
(Weber, Mangus, & Huskey, 2015; Weber et al., 2008).
The brain is made up of a mind-boggling collection of networks consisting of neurons,
one of the most basic biological units of the nervous system. Most areas of the brain are
interconnected, meaning that most networks of neurons are connected to many other
networks of neurons. All of this can make the task of mapping specific patterns of brain
activity to specific mental processes an incredibly difficult task for measurement science in
psychophysiology. A healthy respect for this challenge, as well as for the complexity of mapping
relationships between nervous system activity and specific mental processes, is crucial
for valid and valuable media psychophysiology and neuroscience research. The availability
of resources and technological development has made it easier for interested researchers to
conduct experiments using physiological measures. Researchers must be vigilant to
avoid oversimplifying the process and inadvertently sacrificing the validity of the experiments
they conduct.
A final point about measurement science in psychophysiology, especially as it applies to
media research, is that researchers should maintain a proper perspective on the relationships
between psychophysiological indicators and other measures of responses to stimuli.
A balanced, nuanced perspective on the value of psychophysiological indicators in media
research is required. The entirety of individuals’ experience of interacting with media is not
reflected in validated psychophysiological indicators. There are important concepts in media
research for which there is no validated physiological indicator. The most valuable insights into
media processes and effects will be based on triangulation of data obtained from physiological
responses, self-reports, and behavioral observations. Further, all of these measures may or may
not be highly correlated because they measure the output of processes that do not reflect the
same thing and may not respond in a uniform fashion. For example, a self-report measure of
arousal does not measure the same thing as skin conductance, a physiological indicator of
arousal. We expand on this point below with a brief overview of the human nervous system,
which produces the physiological activity this work is based on.
Overview of the Human Nervous System
The human nervous system is made up of billions of neurons. These nerve cells produce bioelectrical
activity that gets transmitted throughout the entire nervous system. Neurons generate small
voltage electrical signals called action potentials. Action potentials travel between neurons
throughout the nervous system, which connects the brain, spinal column, organs, glands, and
muscles. Bioelectrical activity generated by the billions of neurons throughout the nervous system
not only keeps us alive but also produces our mental experiences. Psychophysiological measures
record these bioelectrical signals, which researchers take as reflecting embodied mental processes.
The human nervous system is organized into two major divisions, which are completely
interconnected: the central nervous system and the peripheral nervous system. Psychophysiological
indicators are primarily categorized according to whether the central or the peripheral
200 • Paul D. Bolls et al.
nervous system is being recorded. The central nervous system consists of the brain and
spinal column. The peripheral nervous system consists of connections between the spinal
column and the peripheral organs, glands, and muscles (e.g., heart, sweat glands, facial
muscles). Motor neurons in the peripheral nervous system generate muscular activity
throughout the body and are referred to as somatic nerves. Peripheral nervous system connections
that control organs and glands are referred to as the autonomic nervous system.
The autonomic nervous system is further divided into the parasympathetic and sympathetic
nervous systems.
This is admittedly a brief description of a highly complex set of systems. The information
offered is only intended to give the reader a basic understanding of how the nervous system
produces the physiological activity that underlies psychophysiological indicators. For a more
detailed discussion of the anatomy and functioning of the human nervous system,
we strongly recommend the writings of Stern and colleagues (2001) and Cacioppo and colleagues
(2007).
With that stated, we now explore measures of physiological activity. The bulk of media
research in the area has relied on measures of the peripheral nervous system. Thus, we begin
with a discussion of those approaches, followed by a discussion of central nervous system measures
used in media processes and effects research.
Peripheral Nervous System Measures in Media Processes and Effects Research
Peripheral nervous system measures have the most established history in psychophysiology. As
noted above, the first media effects experiment to record physiological activity measured skin
conductance, a peripheral nervous system measure based on the activity of sweat glands. The
early work of psychophysiologists focused on mapping variation in peripheral nervous system
activity reflected in heart rate and skin conductance to variation in mental processes related to
attention and arousal respectively (Graham, 1979; Lacey & Lacey, 1974; Sokolov, 1963). The
early work also included studies of emotion, in which precise variation in facial expressions
based on peripheral nervous system activity consisting of facial muscle responses were recorded
(facial electromyography [EMG]; Fridlund & Cacioppo, 1986).
Skin conductance, heart rate, and facial EMG have emerged as the most commonly used
psychophysiological indicators in media processes and effects research in large part due to
early validation connecting these measures with variation in attention, arousal, and positive/
negative emotional responses. Because the brain is connected to the spinal column (i.e., central
nervous system) which contains nerves that run to all organs, glands, and muscles of the
peripheral nervous system, brain activity generated through mental processes expands into the
peripheral nervous system influencing the activity of organs, glands, and muscles. That is why
heart rate, skin conductance, and facial EMG are valid indicators of embodied mental processes
occurring in the brain.
Heart Rate in Media Processes and Effects Research
Heart rate is based on variation in the cardiac cycle that occurs as the heart beats and circulates
blood throughout the body. Heart rate can be recorded in two ways. First, it can be recorded
through pulse rate, produced by blood flowing through the arteries and veins. This method typically
involves the application of a photoplethysmograph (often placed on a finger or earlobe)
Media Psychophysiology and Neuroscience • 201
that uses a photo sensor to measure blood flow and quantify pulse rate. Second, heart rate can
be recorded by an electrocardiogram, which requires placing surface electrodes on the wrist and
forearms. The electrocardiogram produces a visual representation of the bioelectrical activity
generated during the cardiac cycle. Heart rate is quantified by measuring the amount of time
between successive instances of the strongest positive peak in the electrocardiogram, which is
referred to as the inter-beat interval.
The heart is innervated by both the parasympathetic and sympathetic branches of the peripheral
nervous system, which complicates interpretation of changes in heart rate (Koruth, Lang,
Potter, & Bailey, 2015). That being said, heart rate is most often used as a psychophysiological
indicator of cognitive resources allocated to encoding information into a short term memory
representation (Potter & Bolls, 2012), based on the assumption that a primary task involved in
using media is taking in information from the content and encoding it in short term memory.
This task increases parasympathetic nervous system activity, which decreases heart rate. It is
important to remember that this is an assumption, and recent work by media processes and
effects researchers has demonstrated the tremendous value of assessing heart rate variability,
a technique for teasing apart parasympathetic and sympathetic influences on heart rate in interpreting
heart rate data (Koruth et al., 2015).
Heart rate as a psychophysiological indicator of cognitive-resource allocation has been used
in various media contexts, including, for example, how mobile media users cognitively process
advertisements embedded in mobile content (Clark, Leslie, Garcia-Garcia, & Tullman, 2018).
The researchers found that users allocated more cognitive resources (as evidenced through
heart rate change) to encoding the ads when (1) the user had more control over interacting
with the ad, (2) the ad was placed within the content in the least disruptive manner, and (3)
the user was offered incentives for viewing the ad.
Heart rate has also been used to study how emotional content in anti-tobacco advertisements
can influence the way smokers process the messages. In a recent study, heart rate data showed
that combining smoking cues with disgusting images resulted in defensive message processing
in smokers (i.e., fewer cognitive resources allocated to encoding the message; Clayton, Leshner,
Bolls, & Thorson, 2017). This study is in line with other research (e.g., Yegiyan, 2015) using
heart rate data to show how the emotional valence of video content can impact cognitive
resources allocated to encoding, with pleasant (or appetitive) emotional content increasing (as
evidenced by heart rate deceleration) and unpleasant (or aversive) content decreasing (as evidenced
by heart rate acceleration) cognitive resources allocated to encoding.
Skin Conductance in Media Processes and Effects Research
Skin conductance involves recording changes in electrical conductance of the small current
between two recording electrodes. Electrical conductance across the surface of the skin changes
due to the activity of eccrine sweat glands. These sweat glands are associated with what has
been termed psychological sweating (e.g., sweating palms due to nervousness). Accordingly, skin
conductance is most frequently measured by placing electrodes on the surface of the palm or
midsections of the fingers. Skin conductance is based on the activity of sweat glands below the
surface of the skin, not the actual appearance of sweat on the skin’s surface. The level of sweat
in the bulb of the gland changes skin conductance. Electrical resistance of the skin surface
decreases and conductance increases as the level of sweat rises in the gland. The eccrine sweat
glands are solely innervated by the sympathetic branch of the peripheral nervous system. This
202 • Paul D. Bolls et al.
makes skin conductance a highly valid and reliable psychophysiological indicator of sympathetic
arousal, taken to reflect the intensity of emotional responses evoked by a stimulus (Potter &
Bolls, 2012).
Skin conductance has been used to study both general sympathetic arousal as well as anxiety
specifically, in response to different media contexts (e.g., computer-mediated versus face-to-face
communication; Shalom, Isreali, Markovitzky, & Lipsitz, 2015). One particularly interesting line
of research examines arousal responses to video game content. In one study of a racing game,
researchers observed that the intensity level of the game led to increased skin conductance,
which also had a positive correlation with risk-taking inclination and resulted in a greater likelihood
of risky driving behaviors (Deng, Chan, Wu, & Wang, 2015). Skin conductance has also
been used to study sympathetic arousal while parents view media content with their children
(i.e., parental co-viewing; e.g., Rasmussen, Keene, Berke, Densley, & Loof, 2017). The mere presence
of a co-viewing parent increased the child’s level of sympathetic arousal experienced when
viewing educational television content.
Facial EMG in Media Processes and Effects Research
Facial electromyography (EMG) involves the recording of facial muscle activity through the
placement of surface electrodes over specific muscle regions on the face. The electrodes record
the bioelectrical signal that is generated by the firing of action potentials in the neurons of
muscle tissue that occur when muscles contract. This makes facial EMG a highly sensitive measure
of facial expressions because it records potentially meaningful activity that might not be
visible on a person’s face.
Facial EMG is one of the more challenging peripheral nervous system measures to validly
record because the signal is especially susceptible to noise. The human face contains many
muscles located closely together; thus, valid recording of the facial EMG signal is highly
dependent on precise electrode placement. Noise can enter the recorded signal due to the
activity of nearby muscles, as well as electrical frequencies generated by other equipment in
the environment that may overlap with the frequency of the facial EMG signal.
Facial EMG is a validated psychophysiological indicator of emotional valence to a stimulus
(Potter & Bolls, 2012). Emotional responses recorded through facial EMG can be positive, negative,
or a mixture of positive and negative valence. Three different facial muscle regions are of
particular interest to media processes and effects researchers: corrugator (located along the
brow), orbicularis oculi (outer corner of the eye), and the zygomatic (along the cheek). Corrugator
activity reflects variance in negative emotional responding, whereas orbicularis oculi and
zygomatic activity reflects variance in positive emotional responding.
Facial EMG has proven to be a useful indicator of emotional valence in a wide range of
media research. Researchers have, for example, used the measure to examine how positive and
negative online news about corporations interact with existing corporate reputations in influencing
how readers emotionally respond to the stories (Ravaja et al., 2015). Facial EMG activity
reflecting a negative emotional response was higher for negative news stories about corporations
with good reputations, whereas a positive response was higher for negative stories about corporations
with bad reputations. Facial EMG has also been used to show that more calloused individuals
have weaker negative emotional responses to violent films (Fanti, Kyranides, &
Panayiotou, 2017) and that variation in the emotional tone of traffic safety videos can meaningfully
impact viewer responses (Howell, Ekman, Almond, & Bolls, 2018).
Media Psychophysiology and Neuroscience • 203
Central Nervous System Measures in Media Processes and Effects Research
An exciting development in media research has been the resurgence in the use of central nervous
system measures. Researchers working in the late 1970s and early 1980s used electroencephalogram
(EEG; the electrical recording of cortical activity from the surface of the scalp) to study
attention paid to television advertisements (Appel, Weinstein, & Weinstein, 1979; Krugman,
1971; Rothschild, Hyun, Reeves, Thorson, & Goldstein, 1988). But the latest age of central
nervous system measures use has introduced a whole new ballgame: media neuroscience (Mangus,
Adams, & Weber, 2015).
Media neuroscience and the use of brain imaging are exciting developments because the
work provides a view of the brain “on” media. Central nervous system measures give
researchers a peek at the functioning of mental processes engaged when individuals interact
with media that emerge directly from brain activity. The use of these measures is referred to as
brain imaging because the data recorded literally produces images of brain activity. It is critical
to remember that images of brain activity can be as meaningless as looking at pictures of
“blobs” on brains without proper training and expertise. In no way does this chapter substitute
for that; specific training in brain anatomy and physiology, as well as measurement and data
analysis techniques used with these measures, is required (Hopp & Weber, in press; Turner,
Huskey, & Weber, in press; Weber, Mangus, et al., 2015).
This section highlights two common brain imaging techniques used in the application of
central nervous system measures in media neuroscience: EEG and fMRI (functional magnetic
resonance imaging). The valid application of both measures in media processes and effects
research requires a basic understanding of brain anatomy. Here we provide a general summary
that we hope is useful for a basic understanding of EEG and fMRI.
One major division in brain anatomy is between cortical areas (like the cerebral cortex,
the outer folds of brain matter) and sub-cortical structures (like the limbic system). EEG
only records cortical activity. fMRI is required for imaging activity occurring in subcortical
structures. The brain is further organized into left and right hemispheres and four
lobes: the frontal, temporal, occipital, and parietal that are contained in each hemisphere.
Areas of the brain can play a dominant function in a mental process, but a highly localized
view of brain function that tries to pin specific mental processes to specific brain
locations is misguided. Higher cognitive functions are rooted in the activity of the cerebral
cortex, whereas sub-cortical structures in the limbic system (e.g., amygdala, hippocampus)
play an important role in emotion and memory. It would be a complete misunderstanding,
however, to assume, for instance, that television advertisements that elicit higher activation
in the amygdala are loved more than advertisements that elicit less activation in
this sub-cortical structure.
EEG in Media Processes and Effects Research
Through the application of surface electrodes, EEG records small electrical signals that can be
detected from the surface of the scalp. This electrical signal is generated by the firing of neurons
in activated cortical areas. Cortical activity can be recorded from both hemispheres and all four
of the brain lobes. This allows inferences to be made about mental processes engaged during
various tasks—including interacting with media—emerging from cortical networks throughout
the hemispheres and brain lobes.
204 • Paul D. Bolls et al.
Researchers have used EEG to study multiple forms of cognitive and emotional processes
engaged through media use (for a recent review, see Morey, 2018). Herein we briefly review
a few examples of recent experiments. Attention, one of the earliest cognitive processes studied
in media research using EEG, continues to be of interest in recent work. Variation among brain
waves recorded through EEG (specifically beta, theta, and alpha) continues to be used to reliably
indicate how much attention, for instance, different types of video advertising elicits in viewers
(Daugherty, Hoffman, Kennedy, & Nolan, 2018). Morey (2017) used EEG to study specific
memory processes involved in cognitive processing of ads. Specifically, the gamma frequency
band of the EEG signal was analyzed in response to exposure to positive and negative political
television spots; this brainwave frequency is thought to be related to memory processes related
to semantic processing and binding of information in memory. Results indicated that Republicans
remembered more content from positive ads, whereas Democrats remembered more content
from negative ads. Further, increased activity in the gamma frequency band of the EEG
signal was related to increased semantic processing and memory.
EEG has also been used to investigate cognitive and emotional processes associated with
newer, interactive media technologies. For example, individuals who report a preference for
computer-mediated communication have shown a heightened predisposition to pay attention
to emotional stimuli and decreased capacity to regulate emotional responses through their
recorded EEG activity (Babkirk, Luehring-Jones, & Dennis-Tiwary, 2016). Researchers have
also used EEG to identify neurobiological correlates that make up a brain network underlying
the psychological phenomenon known as fear of missing out (FOMO), which is often
associated with social media use (Lai, Altavilla, Ronconi, & Aceto, 2016). Finally, one of the
most extensive applications of EEG in media processes and effects research has been to
examine violent video game effects. In a recent study, frequent players of graphically violent
games were found to experience lower levels of empathy, as evidenced by less attention paid
to positive and negative facial expressions of others (Stockdale, Morrison, Palumbo,
Garbarino, & Silton, 2017).
fMRI in Media Processes and Effects Research
fMRI measures differences in magnetic properties of fluids and tissues in the brain. Hemoglobin,
a protein that transports oxygen in the blood, has different magnetic properties depending
on the level of oxygen in blood cells. Neural activity consumes energy in the form of oxygen
and glucose. Consequently, neural processing leads to changes of oxygen concentration in blood
cells, which changes the magnetic properties in blood, and then leads to small disturbances in
an MRI scanner’s strong magnetic field (Logothetis & Pfeuffer, 2004). These disturbances can
be detected with highly sensitive receivers in the fMRI scanner. This effect is known as the
blood oxygen level dependent (BOLD) effect and is one of the foundational principles of functional
MRI. Taking advantage of this effect, latest generation fMRI scanners allow for the generation
of brain images with a maximum spatial resolution of 0.5–1.0 cubic millimeter and
a maximum temporal resolution of 0.5–1.0 second.
As with all psychophysiological measures, the BOLD effect generates a very weak signal compared
to the numerous additional noise sources that cannot be avoided in fMRI scanning.
Therefore, the generation and analysis of fMRI data requires sophisticated data pre-processing
in order to obtain reliable and valid measures of brain activity at specified locations and at specified
time points. For readers interested in the details of fMRI data analysis, we recommend
Media Psychophysiology and Neuroscience • 205
the well-written fMRI textbook from Huettel, Song, and McCarthy (2014). Easy-to-process
introductions into fMRI tailored for communication and media scholars are also available (see
Weber, Mangus, et al., 2015; Weber, Fisher, Hopp, & Lonergan, 2017). For communication
scholars interested in advanced fMRI analyses, we recommend Bassett and Sporns (2017),
Sporns (2010), and Turner et al. (in press).
The availability of fMRI facilities for communication scholars has significantly increased
within the past ten years. This has led to decreases in costs for fMRI research. As a result, communication
and media scholars have started studying a diverse set of topics with fMRI, including
the use of brain imaging data as a predictor for persuasive message effectiveness (e.g., Falk,
Berkman, Mann, Harrison, & Lieberman, 2010; Huskey, Mangus, Turner, & Weber, 2017;
Weber, Huskey, Mangus, Westcott-Baker, & Turner, 2015), attentional processes and flow
during media exposure (e.g., Huskey, Craighead, Miller, & Weber, 2018; Weber, Alicea, Huskey,
& Mathiak, 2018), narrative engagement and neurocinematics (e.g., Hasson et al., 2008; Schmälzle,
Häcker, Honey, & Hasson, 2015), message virality (e.g., Scholz et al., 2017), and affectionate
communication in close relationships (Hesse et al., 2013). For recent and more detailed summaries
of such studies, we recommend Falk, Cascio, and Coronel (2015), Floyd and Weber (in
press), Weber, Mangus, et al. (2015), and Weber et al. (2017).
A Bright Future for Media Processes and Effects Research
This chapter has presented media psychophysiology and neuroscience as a new framework for
media processes and effects research. The adoption of the psychophysiological paradigm by
media researchers in the late 1980s generated exciting growth in the field. The infusion of
neuroscience and other related scientific disciplines into the ways that psychophysiology is
applied in media research has further brightened the future of media processes and effects
research.
The media psychophysiology and neuroscience framework described here offers a new perspective
on media effects. But this new perspective should not lead to intellectual arguments
over the importance of studying media effects, as some might conclude. Instead, it should further
elevate the importance of media effects research in communication and across all scientific
disciplines. But this framework does assume different answers to some of the most basic questions
confronting researchers in this area.
• What are media effects? According to the media psychophysiology and neuroscience framework,
media effects consist of changes that constantly occur when individuals interact with
media of any form, as opposed to changes in a static outcome measure obtained after media
exposure.
• Where do media effects occur? According to the media psychophysiology and neuroscience
framework, media effects occur in and emerge from the context-dependent functioning of the
embodied human mind.
• When do media effects occur? According to the media psychophysiology and neuroscience
framework, media effects unfold across time through the functioning of dynamic embodied
mental processes evoked through media use.
These perspectives shape a recommended path forward to discovering unique knowledge
about media processes and effects.
206 • Paul D. Bolls et al.
One area of research where the potential of this new perspective can be clearly illustrated
is newer interactive and immersive media technology and content. Although media technology
and content change constantly, the basic functions of the embodied human mind do not.
This truth gives the framework described in this chapter an advantage over traditional
approaches to media research in advancing knowledge of newer forms of media. Traditional
approaches tend to focus on effects tied to a specific media form. In contrast, the media psychophysiology
and neuroscience framework focuses on dynamic embodied mental processes
occurring in the human brain and nervous system, which do not fundamentally change with
each new form of media content and technology. This makes the media psychophysiology
and neuroscience framework useful for exploring traditional, newer, and yet-to-be-invented
forms of media.
An additional reason this approach has tremendous value is that media technologies are
becoming increasingly immersive by becoming more embodied. Many newer technologies (e.g.,
virtual reality; see Chapter 26 in this volume) are designed to engage a wider form and degree
of bodily responses through larger and more intense sensory experiences and physical bodily
interactions. Such changes make the embodied experience of interacting with newer forms of
media technology more critical to understanding the related processes and effects. The media
psychophysiology and neuroscience framework offers researchers a valid approach to conceptualize
all media use as an embodied experience.
A final point has to do with the unique opportunity that researchers who adopt the
framework described here have to build bridges between academia and industry. Academic
researchers are (often rightly) more focused on the theoretical and social importance of
their work rather than the practical. Mutual enthusiasm for the application of brain science
in media research, however, could help break through some of the barriers that have prevented
academic-industry research collaboration in the past. This is an opportunity that
does not exist within other research areas. For example, neuromarketing has sparked great
enthusiasm for brain science in the media industries. Neuromarketing research utilizes all
the measures discussed in this chapter (along with several additional ones) in an attempt to
gain insight into consumers’ brains as a means for optimizing the effectiveness of persuasive
messages. This new area of marketing research was formally established around 2006 and
now includes formal academic programs, mostly located in European business schools.
Numerous research companies offer neuromarketing research worldwide. All of the major
US television networks either conduct their own neuromarketing research or have
a neuromarketing research provider to test content. One example of a successful academicindustry
neuromarketing collaboration involved the Advertising Research Foundation and
neuromarketing companies to examine the validity and value of neuromarketing research.
That effort resulted in a scientific peer-reviewed publication on the validity of measures
used in neuromarketing research (Varan, Lang, Barwise, Weber, & Bellman, 2015). Opportunities
for similar collaborations should only increase in the future.
In conclusion, we hope that this chapter is a valuable resource for beginning and more
experienced researchers, whether you plan to adopt this framework or not. As we have tried
to convey, many additional resources are also available for understanding how psychophysiology
and neuroscience is applied in media research. The new age of psychophysiology and
brain science in media effects research is exciting, with the potential for new knowledge
never greater.
Media Psychophysiology and Neuroscience • 207
References
Afifi, T. D., & Floyd, K. (Eds.). (2015). Communication, biology, and physiology [Special issue].
Communication Monographs, 82(1). doi:10.1080/03637751.2015.1005964
Appel, V., Weinstein, S., & Weinstein, C. (1979). Brain activity and recall TV advertising. Journal
of Advertising Research, 19, 7–15.
Babkirk, S., Luehring-Jones, P., & Dennis-Tiwary, T. A. (2016). Computer-mediated communication
preferences predict biobehavioral measures of social-emotional functioning. Social Neuroscience,
11, 637–651. doi:10.1080/17470919.2015.1123181
Bandura, A., Ross, D., & Ross, S. A. (1963). Imitation of film-mediated aggressive models. The
Journal of Abnormal and Social Psychology, 66, 3–11. doi:10.1037/h0048687
Bassett, D. S., & Sporns, O. (2017). Network neuroscience. Nature Neuroscience, 20(3), 353–364.
doi:10.1038/nn.4502
Berntson, G. G., & Cacioppo, J. T. (2000). Psychobiology and social psychology: Past, present, and
future. Personality and Social Psychology Review, 4, 3–15. doi:10.1207/S15327957PSPR0401_2
Bryant, J., & Oliver, M. B. (Eds.). (2009). Media effects: Advances in theory and research.
New York, NY: Routledge. doi:10.4324/9780203877111
Cacioppo, J. T., Tassinary, L. G., & Berntson, G. (Eds.). (2007). Handbook of psychophysiology.
New York, NY: Cambridge University Press. doi:10.1017/cbo9780511546396
Cannon, W. B. (1931). The James-Lange theory of emotion: A critical examination and an alternative
theory. America Journal of Psychology, 39, 106–124. doi:10.2307/1422695
Chaffee, S. H. (1977). Mass media effects: New research perspectives. In D. Lerner & L. M. Nelson
(Eds.), Communication research: A half-century appraisal (pp. 210–241). Honolulu, HI: University
of Hawaii Press.
Clark, K. R., Leslie, K. R., Garcia-Garcia, M., & Tullman, M. L. (2018). How advertisers can keep
mobile users engaged and reduce video-ad blocking: Best practices for video-ad placement and
delivery based on consumer neuroscience measures. Journal of Advertising Research, 58, 311–
325. doi:10.2501/jar-2018-036
Clayton, R. B., Leshner, G., Bolls, P. D., & Thorson, E. (2017). Discard the smoking cues—Keep
the disgust: An investigation of tobacco smokers’ motivated processing of anti-tobacco commercials.
Health Communication, 32, 1319–1330. doi:10.1080/10410236.2016.1220042
Daugherty, T., Hoffman, E., Kennedy, K., & Nolan, M. (2018). Measuring consumer neural activation
to differentiate cognitive processing of advertising: Revisiting Krugman. European Journal
of Marketing, 52(1/2), 182–198. doi:10.1108/EJM-10-2017-0657
Delia, J. G. (1987). Communication research: A history. In C. R. Berger & S. H. Chaffee (Eds.),
Handbook of communication science (pp. 20–98). Newbury Park, CA: Sage.
Deng, M., Chan, A. H., Wu, F., & Wang, J. (2015). Effects of racing games on risky driving behaviour,
and the significance of personality and physiological data. Injury Prevention, 21, 238–244.
doi:10.1136/injuryprev-2014-041328
Dennis, E. E., & Wartella, E. (Eds.). (1996). American communication research: The remembered
history. Mahwah, NJ: Lawrence Erlbaum Associates.
Duffy, E. (1957). The psychological significance of the concept of “arousal” or “activation”. Psychological
Review, 64, 265–275. doi:10.1037/h0048837
Dysinger, W. S., & Ruckmick, C. A. (1933). The emotional responses of children to the motion picture
situation. Oxford, UK: Macmillan.
Falk, E. B., Berkman, E., Mann, T., Harrison, B., & Lieberman, M. D. (2010). Predicting
persuasion-induced behavior change from the brain. Journal of Neuroscience, 30, 8421–8424.
doi:10.1523/JNEUROSCI.0063-10.2010
Falk, E. B., Cascio, C. N., & Coronel, J. C. (2015). Neural prediction of communication-relevant
outcomes. Communication Methods and Measures, 9, 30–54. doi:10.1080/19312458.2014.999750
208 • Paul D. Bolls et al.
Fanti, K. A., Kyranides, M. N., & Panayiotou, G. (2017). Facial reactions to violent and comedy
films: Association with callous–unemotional traits and impulsive aggression. Cognition and
Emotion, 31, 209–224. doi:10.1080/02699931.2015.1090958
Floyd, K., & Weber, R. (Eds.). (in press). Communication science and biology. New York, NY:
Routledge.
Fridlund, A. J., & Cacioppo, J. T. (1986). Guidelines for human electromyographic research. Psychophysiology,
23, 567–589. doi:10.1111/j.1469-8986.1986.tb00676.x
Geiger, S., & Newhagen, J. (1993). Revealing the black box: Information processing and media
effects. Journal of Communication, 43, 42–50. doi:10.1111/j.1460-2466.1993.tb01303.x
Graham, F. K. (1979). Distinguishing among orienting defense, and startle reflexes. In
H. D. Kimmel, E. H. Van Olst, & J. F. Orlebeke (Eds.), The orienting reflex in humans (pp.
137–167). Hillsdale, NJ: Lawrence Erlbaum Associates.
Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., & Heeger, D. J. (2008). Neurocinematics:
The neuroscience of film. Projections, 2, 1–26. doi:10.3167/proj.2008.020102
Hebb, D. O. (1955). Drives and the CNS (conceptual nervous system). Psychological Review, 62,
243–254. doi:10.1037/h0041823
Hergenhahn, B. R., & Henley, T. (2013). An introduction to the history of psychology (7th ed.).
Belmont, CA: Cengage Learning.
Hesse, C., Floyd, K., Rauscher, E. A., Frye-Cox, N., Hegarty, J., & Peng, H. (2013). Alexithymia
and impairment of decoding positive affect: An fMRI study. Journal of Communication, 63,
786–806. doi:10.1111/jcom.12039
Hopp, F., & Weber, R. (in press). The state-of-the-art and the future of brain imaging methodology
in communication research. In K. Floyd & R. Weber (Eds.), Communication Science and
Biology. New York, NY: Routledge.
Howell, M., Ekman, D. S., Almond, A., & Bolls, P. (2018). Switched on: How the timing of aversive
content in traffic safety videos impact psychophysiological indicators of message processing.
Health Communication, Advanced online publication. doi:10.1080/10410236.2018.1517706
Huettel, S. A., Song, A. W., & McCarthy, G. (2014). Functional magnetic resonance imaging (3rd
ed.). Sunderland, MA: Sinauer.
Huskey, R., Craighead, B., Miller, B. M., & Weber, R. (2018). Does intrinsic reward motivate cognitive
control? A naturalistic fMRI study based on the synchronization theory of flow. Cognitive
Affective and Behavioral Neuroscience, 18, 902–924. doi:10.3758/s13415-018-0612-6
Huskey, R., Mangus, J. M., Turner, B., & Weber, R. (2017). The persuasion network is modulated
by drug-use risk and predicts anti-drug message effectiveness. Social Cognitive and Affective
Neuroscience, 12, 1902–1915. doi:10.1093/scan/nsx126
James, W. (1894). Discussion: The physical basis of emotion. Psychological Review, 1, 516–529.
doi:10.1037/h0065078
Koruth, K. J., Lang, A., Potter, R. F., & Bailey, R. L. (2015). A comparative analysis of dynamic
and static indicators of parasympathetic and sympathetic nervous system activation during TV
viewing. Communication Methods and Measures, 9, 78–100. doi:10.1080/19312458.2014.999752
Krugman, H. E. (1971). Brain wave measures of media involvement. Journal of Advertising
Research, 11, 3–9. doi:10.4135/9781452231501.n13
Lacey, J. I., & Lacey, B. C. (1974). On heart rate responses and behavior: A reply to Elliott. Journal
of Personality and Social Psychology, 30, 1–18. doi:10.1037/h0036559
Lachman, R., Lachman, J. L., & Butterfield, E. C. (1979). Cognitive psychology and information processing:
An introduction. London, UK: Psychology Press.
Lai, C., Altavilla, D., Ronconi, A., & Aceto, P. (2016). Fear of missing out (FOMO) is associated
with activation of the right middle temporal gyrus during inclusion social cue. Computers in
Human Behavior, 61, 516–521. doi:10.1016/j.chb.2016.03.072
Media Psychophysiology and Neuroscience • 209
Lang, A., Potter, R., & Bolls, P. (2009). Where psychophysiology meets the media: Taking the
effects out of mass media research. In J. Bryant & M. B. Oliver (Eds.), Media effects: Advances
in theory and research (3rd ed., pp. 185–206). New York, NY: Routledge.
Logothetis, N. K., & Pfeuffer, J. (2004). On the nature of the BOLD fMRI contrast mechanism.
Magnetic Resonance Imaging, 22, 1527–1531. doi:10.1016/j.mri.2004.10.018
Mangus, J. M., Adams, A., & Weber, R. (2015). Media neuroscience. In R. A. Scott &
S. M. Kosslyn (Eds.), Emerging trends in the behavioral and social sciences (pp. 1–14). Hoboken,
NJ: Wiley-Blackwell.
Morey, A. C. (2017). Memory for positive and negative political TV ads: The role of partisanship
and gamma power. Political Communication, 34, 404–423. doi:10.1080/10584609.2016.1266534
Morey, A. C. (2018). Electroencephalography in communication research: A review of the past
and a glimpse of future possibilities. Annals of the International Communication Association,
42, 243–269. doi:10.1080/23808985.2018.1537723
Potter, R. F., & Bolls, P. (2012). Psychophysiological measurement and meaning: Cognitive and
emotional processing of media. New York, NY: Routledge.
Rasmussen, E. E., Keene, J. R., Berke, C. K., Densley, R. L., & Loof, T. (2017). Explaining parental
coviewing: The role of social facilitation and arousal. Communication Monographs, 84, 365–
384. doi:10.1080/03637751.2016.1259532
Ravaja, N., Aula, P., Falco, A., Laaksonen, S., Salminen, M., & Ainamo, A. (2015). Online news
and corporate reputation: A neurophysiological investigation. Journal of Media Psychology, 27,
118–133. doi:10.1027/1864-1105/a000149
Rothschild, M. L., Hyun, Y. J., Reeves, B., Thorson, E., & Goldstein, R. (1988). Hemispherically
lateralized EEG as a response to television commercials. Journal of Consumer Research, 15(2),
185–198. doi:10.1086/209156
Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of emotional
state. Psychological Review, 69, 379–399. doi:10.1037/h0046234
Schmälzle, R., Häcker, F. E. K., Honey, C. J., & Hasson, U. (2015). Engaged listeners: Shared
neural processing of powerful political speeches. Social Cognitive and Affective Neuroscience,
10, 1137–1143. doi:10.1093/scan/nsu168
Scholz, C., Baek, E. C., O’Donnell, M. B., Kim, H. S., Cappella, J. N., & Falk, E. B. (2017). A neural
model of valuation and information virality. Proceedings of the National Academy of Sciences,
114, 2881–2886. doi:10.1073/pnas.1615259114
Shalom, J. G., Israeli, H., Markovitzky, O., & Lipsitz, J. D. (2015). Social anxiety and physiological
arousal during computer mediated vs. face to face communication. Computers in Human
Behavior, 44, 202–208. doi:10.1016/j.chb.2014.11.056
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Oxford, England:
Appleton-Century.
Sokolov, E. N. (1963). Perception and the conditioned reflex. Oxford, UK: Pergamon Press.
Sporns, O. (2010). Networks of the brain. Boston, MA: MIT press.
Stern, R. M., Ray, W. J., & Quigley, K. S. (2001). Psychophysiological recording (2nd ed.). Oxford,
UK: Oxford University Press.
Stockdale, L., Morrison, R. G., Palumbo, R., Garbarino, J., & Silton, R. L. (2017). Cool, callous and
in control: Superior inhibitory control in frequent players of video games with violent content.
Social Cognitive and Affective Neuroscience, 12, 1869–1880. doi:10.1093/scan/nsx115
Turner, B., Huskey, R., & Weber, R. (in press). Charting a future for fMRI in communication
science. Communication Methods and Measures. doi:10.1080/19312458.2018.1520823
Varan, D., Lang, A., Barwise, P., Weber, R., & Bellman, S. (2015). How reliable are neuromarketers’
measures of advertising effectiveness? Data from ongoing research holds no common
210 • Paul D. Bolls et al.
truth among vendors. Journal of Advertising Research, 55, 176–191. doi:10.2501/JAR-55-
2-176-191
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177.
doi:10.1037/h0074428
Weber, R. (Ed.). (2015). Brain, mind, and media: Neuroscience meets media psychology [Special
issue]. Journal of Media Psychology, 27(3). doi:10.1027/1864-1105/a000108
Weber, R., Alicea, B., Huskey, R., & Mathiak, K. (2018). Network dynamics of attention during
a naturalistic behavioral paradigm. Frontiers in Human Neuroscience, 22(182). doi:10.3389/
fnhum.2018.00182
Weber, R., Eden, A., Huskey, R., Mangus, J. M., & Falk, E. (2015). Bridging media psychology and
cognitive neuroscience: Challenges and opportunities. Journal of Media Psychology, 27, 146–
156. doi:10.1027/1864-1105/a000163
Weber, R., Fisher, J. T., Hopp, F. R., & Lonergan, C. (2017). Taking messages into the magnet:
Method-theory synergy in communication neuroscience. Communication Monographs, 85, 81–
102. doi:10.1080/03637751.2017.1395059
Weber, R., Huskey, R., Mangus, J. M., Westcott-Baker, A., & Turner, B. (2015). Neural predictors
of message effectiveness during counterarguing in antidrug campaigns. Communication Monographs,
82, 4–30. doi:10.1080/03637751.2014.971414
Weber, R., Mangus, J. M., & Huskey, R. (2015). Brain imaging in communication research:
A practical guide to understanding and evaluating fMRI studies. Communication Methods and
Measures, 9, 5–29. doi:10.1080/19312458.2014.999754
Weber, R., Sherry, J., & Mathiak, K. (2008). The neurophysiological perspective in mass communication
research. Theoretical rationale, methods, and applications. In M. J. Beatty,
J. C. McCroskey, & K. Floyd (Eds.), Biological dimensions of communication: Perspectives,
methods, and research (pp. 41–71). Cresskill, NJ: Hampton Press.
Yegiyan, N. S. (2015). Explicating the emotion spillover effect. Journal of Media Psychology, 27,
134–145. doi:10.1027/1864-1105/a000164
Zillmann, D., & Bryant, J. (1974). Effect of residual excitation on the emotional response to provocation
and delayed aggressive behavior. Journal of Personality and Social Psychology, 30, 782–791.
doi:10.1037/h0037541