13
Media Psychophysiology and Neuroscience
Bringing Brain Science into Media Processes
and Effects Research
Paul D. Bolls, René Weber, Annie Lang, and Robert F. Potter
Welcome to a new age of studying the human brain "on" media! Media researchers now enthusiastically embrace theories and methods from brain science to examine how media effects emerge from embodied mental processes. This enthusiasm is seen in recent special editions of journals (Ann & Floyd, 2015; Weber, 2015), handbooks from the International Communication Association (ICA; e.g., Floyd & Weber, in press), the significant presence of related research at major communication conferences (e.g., the ICA's Communication Science and Biology Interest Group), as well as advertised tenure-track faculty openings. We are now past the stage where the use of physiological indicators of brain activity in media research was viewed as novel by some and with suspicion by others. Various forms of brain science are now seen as some of the most promising approaches to shedding light on the complex and dynamic ways that media effects occur. Physiological indicators of brain activity are not only a permanent fixture in the media researcher's toolkit (Potter & Bolls, 2012), but neuroscience—the formal discipline of brain science—is becoming firmly established as one of the disciplines providing the intellectual roots for communication science. It is an exciting time for new and more experienced researchers to learn how brain science can be applied to studying media processes and effects.
This chapter overviews how media psychophysiology and neuroscience offer researchers a valuable framework for investigating media processes and effects. We avoid a detailed technical discussion of psychophysiology and neuroscience in favor of a more conceptual overview of how the framework presented here can be applied. We also attempt to introduce a vast amount of available resources to help interested scholars.
The chapter begins with a discussion of how a framework for the application of brain science in media research emerged from principles of psychophysiology and neuroscience. Next, we review common physiological indicators used in media research labs and recent studies using these measures. We conclude with a look to the future and consider specific ways that this framework can advance knowledge of media processes and effects.
195
196 . Paul D. Bolls et al.
The Media Psychophysiology and Neuroscience Paradigm
The previous edition of this book (Bryant & Oliver, 2009) included a chapter authored by three of us (Lang, Potter, & Bolls, 2009). That chapter introduced psychophysiology as a new paradigm for studying how individuals cognitively and emotionally process media. Psychophysiology, at that time, was familiar to a relatively small group of researchers studying media processes and effects. That has changed. Familiarity with psychophysiology has grown, but we must avoid forgetting the important role an historical paradigm change played in this growth to avoid repeating past mistakes. This change occurred when media researchers embraced the psychophysiological paradigm. Our review of this history focuses on four major phases.
Phase 1: the Beginning of Psychology and Early Inspiration for Brain Science
The first era of formal social science research in psychology included a theoretical focus on brain physiology and mental processes. For example, a consideration of brain activity as fundamental to human nature can be seen in the development of the earliest theories of emotion. Those earliest theories wrestled with the relationship between physiological responses and mental experiences, with three predominant views emerging: physiological responses as primary drivers of emotion (James, 1894), conscious experiences of emotion eliciting specific physiological responses (Cannon, 1931), and emotion unfolding from the interaction of physiological activity and mental experiences (Schachter & Singer, 1962). During this time, Elizabeth Duffy (1957) also identified arousal as a drive fueling human behavior, and Donald Hebb (1955) identified brain systems underlying arousal. However, this work lost prominence in psychology when the field enthusiastically embraced behaviorism.
Nevertheless, interest in physiological responses as the drivers of emotion trickled into media effects research in the early 20th century, by way of a relatively unknown experiment that was part of the famous Payne Fund studies (Dysinger & Ruckmick, 1933). The researchers studied arousal in response to movies by recording the skin conductance of viewers. This study is unfamiliar to most because it was overshadowed by the work of many media effects researchers who joined psychology in embracing behaviorism.
Phase 2: the Age of Behaviorism in Psychology and Media Effects Research
Three of the most famous researchers in psychology—John Watson, B. F. Skinner, and Ivan Pavlov—are known for ushering in the age of behaviorism (Hergenhahn & Henley, 2013). Watson's behaviorist view of psychological research left no doubt concerning the legitimate focus of psychology (Watson, 1913). His view completely dismissed the notion that mental processes can be validly studied. Pavlov and Skinner are known for their pioneering research on learning and conditioning grounded in behaviorism (Skinner, 1938). Their studies represent the purest form of behavioristic research rooted in what has been termed the stimulus-response paradigm (Potter & Bolls, 2012).
Media effects research emerged under the stimulus-response paradigm; most histories identify behaviorism as the scientific foundation for the field (Delia, 1987; Dennis & Wartella, 1996). It was during the age of behaviorism that Albert Ban dura conducted his famous Bobo doll studies on the effects of exposure to media violence (e.g., Bandura, Ross, & Ross, 1963;
Media Psychophysiology and Neuroscience • 197
see also Chapters 7 and 14 in this volume). Behavioral effects of media are still as crucial to understand today as they were at that time. But behaviorism left researchers with a significant blind spot: a lack of scientific understanding of what creates behavioral responses to meaningful stimuli like media content. This limitation led to the first of two major paradigm shifts discussed herein. The first paradigm change is associated with the "cognitive revolution" in psychology and resulted in the adoption of the information processing paradigm (Potter & Bolls, 2012).
Phase 3: the Cognitive Revolution Spreads through Psychology and Media Effects Research
Thanks to the cognitive revolution in psychology, it is now commonly accepted that understanding behavioral responses to stimuli (including those tied to media exposure) requires observing what occurs in the minds and bodies of individuals between exposure to a stimulus and the response. Potter and Bolls (2012) labeled this mental activity intervening processes. Thus, the paradigm shift from behaviorism to information processing can better be thought of as the shift from a stimulus-response paradigm to a stimulus-intervening processes-response paradigm.
Under behaviorism, intervening processes were thought to be hidden within the "black box" of the human mind, which could not be validly observed. The cognitive revolution in psychology focused on bursting open this supposed "black box" and systematically studying how cognitive processes unfold as individuals take in, process, and respond to information in the environment. This approach came to be known as information processing (for an excellent review of this intellectual history, see Lachman, Lachman, & Butterfield, 1979).
Media researchers working in the late 1970s and 1980s adopted the information-processing paradigm, calling for research to investigate related mental processes and effects (Chaffee, 1977; Geiger & Newhagen, 1993). It was during this phase that psychophysiological measurement reentered media effects research (e.g., Zillmann & Bryant, 1974), with physiological activity examined as a response to media content. However, research conducted through this lens produced inconsistent results (Potter & Bolls, 2012). We have since come to understand that physiological systems do not respond in a unitary correlated linear fashion (Stern, Ray, & Quig-ley, 2001). Our increasing knowledge of these systems led to a second significant paradigm shift, when the psychophysiological paradigm was adopted by the researchers in the late 1980s (Lang et al, 2009).
Phase 4: the Psychophysiological Paradigm Emerges
The emergence of psychophysiology in psychology began before the cognitive revolution with the work of researchers in the 1960s who renewed interest in physiological activity as an explanation of human nature and experience (Berntson & Cacioppo, 2000). This work laid the foundation for physiological activity to remain a permanent focus of psychological science. It also enabled the birth of the Society for Psychophysiological Research and the formation of the psychophysiological paradigm (Cacioppo, Tassinery, & Berntson, 2007). Development of this new paradigm led to the third phase of psychophysiological measurement in media processes and effects research (Potter & Bolls, 2012). With the shift, media researchers moved away from viewing physiological activity as a response to media content. Instead, they came to view specific patterns of physiological activity as indicators of embodied mental processes.
198 . Paul D. Bolls et al.
The psychophysiological paradigm is the foundation for the media psychophysiology and neuroscience we reference in the title of this chapter. The term media psychophysiology was introduced by Potter and Bolls (2012) and generally refers to the application of psychophysiological science in media processes and effects research. The discipline of neuroscience is one of the disciplines psychophysiology draws from. Neuroscience, however, is more focused on directly recording brain activity through physiological indicators of central nervous system activity. Neuroscience also requires different analytical tools and techniques (for more on the application of neuroscience in media research, see Weber, Sherry, & Mathiak, 2008). The assumptions of the psychophysiological paradigm apply to both areas. We discuss three of the most fundamental assumptions below (for more detail, see Cacioppo et al, 2007, an essential resource for those wishing to adopt these approaches).
The Human Mind Is Embodied
First, the psychophysiological paradigm assumes that all forms of human mental activity exist in the brain and are observable through specific patterns of nervous system activity that echo through the brain and body. These patterns can be complex, can unfold on various temporal and spatial scales, and may require further refined measurement techniques, but these patterns as an index of mental activity do exist.
Embodied Mental Activity Unfolds across Time
Second, psychophysiological measures indicate variation in mental processes across time as an individual perceives and responds to a stimulus. Instant-by-instant changes in physiological activity that reflect mental processes have extremely important psychological meaning. In other words, dependent on how a physiological response varies over time, the same physiological indicator can have different meanings and interpretations. For instance, different patterns of heart rate deceleration and acceleration across time can indicate either an orienting or startle response to a stimulus.
The Purpose of Physiology Is to Sustain Life, Not Index Mental Processes
Third, physiological indicators of mental processes generate a very small signal. This makes the application of physiological indicators of mental activity in research a challenging proposition that requires detailed training in the approach. Researchers who deploy psychophysiological indicators to study mental activity must validly record small levels of variation in nervous system signals that actually reflect variation in mental processes and are distinct from noise.
Measurement Science in Media Psychophysiology and Neuroscience
Measurement science in psychophysiology focuses on validating how specific patterns of nervous system activity can be mapped onto specific mental processes. This effort is consistent with the formal definition of psychophysiology, which can be summarized as the scientific study of psychological phenomena as revealed through physiological activity of the functioning nervous system (Cacioppo et al, 2007). Two domains of measurement science in psychophysiology exist: the physiological domain consisting of nervous system activity, and the psychological
Media Psychophysiology and Neuroscience • 199
domain consisting of mental processes. Researchers face the difficult challenge of validating complex patterns of activity in the physiological domain as psychophysiological indicators of specific mental processes. This challenge is compounded by the fact that physiological responses can often be mapped to different mental processes. The introduction of neuroscience further complicates the picture, as researchers attempt to draw conclusions about mental processes from specific patterns of brain activity while recognizing the complexity of the human brain (Weber, Mangus, & Huskey, 2015; Weber et al, 2008).
The brain is made up of a mind-boggling collection of networks consisting of neurons, one of the most basic biological units of the nervous system. Most areas of the brain are interconnected, meaning that most networks of neurons are connected to many other networks of neurons. All of this can make the task of mapping specific patterns of brain activity to specific mental processes an incredibly difficult task for measurement science in psychophysiology. A healthy respect for this challenge, as well as for the complexity of mapping relationships between nervous system activity and specific mental processes, is crucial for valid and valuable media psychophysiology and neuroscience research. The availability of resources and technological development has made it easier for interested researchers to conduct experiments using physiological measures. Researchers must be vigilant to avoid oversimplifying the process and inadvertently sacrificing the validity of the experiments they conduct.
A final point about measurement science in psychophysiology, especially as it applies to media research, is that researchers should maintain a proper perspective on the relationships between psychophysiological indicators and other measures of responses to stimuli.
A balanced, nuanced perspective on the value of psychophysiological indicators in media research is required. The entirety of individuals' experience of interacting with media is not reflected in validated psychophysiological indicators. There are important concepts in media research for which there is no validated physiological indicator. The most valuable insights into media processes and effects will be based on triangulation of data obtained from physiological responses, self-reports, and behavioral observations. Further, all of these measures may or may not be highly correlated because they measure the output of processes that do not reflect the same thing and may not respond in a uniform fashion. For example, a self-report measure of arousal does not measure the same thing as skin conductance, a physiological indicator of arousal. We expand on this point below with a brief overview of the human nervous system, which produces the physiological activity this work is based on.
Overview of the Human Nervous System
The human nervous system is made up of billions of neurons. These nerve cells produce bioelec-trical activity that gets transmitted throughout the entire nervous system. Neurons generate small voltage electrical signals called action potentials. Action potentials travel between neurons throughout the nervous system, which connects the brain, spinal column, organs, glands, and muscles. Bioelectrical activity generated by the billions of neurons throughout the nervous system not only keeps us alive but also produces our mental experiences. Psychophysiological measures record these bioelectrical signals, which researchers take as reflecting embodied mental processes.
The human nervous system is organized into two major divisions, which are completely interconnected: the central nervous system and the peripheral nervous system. Psychophysiological indicators are primarily categorized according to whether the central or the peripheral
200 . Paul D. Bolls et al.
nervous system is being recorded. The central nervous system consists of the brain and spinal column. The peripheral nervous system consists of connections between the spinal column and the peripheral organs, glands, and muscles (e.g., heart, sweat glands, facial muscles). Motor neurons in the peripheral nervous system generate muscular activity throughout the body and are referred to as somatic nerves. Peripheral nervous system connections that control organs and glands are referred to as the autonomic nervous system. The autonomic nervous system is further divided into the parasympathetic and sympathetic nervous systems.
This is admittedly a brief description of a highly complex set of systems. The information offered is only intended to give the reader a basic understanding of how the nervous system produces the physiological activity that underlies psychophysiological indicators. For a more detailed discussion of the anatomy and functioning of the human nervous system, we strongly recommend the writings of Stern and colleagues (2001) and Cacioppo and colleagues (2007).
With that stated, we now explore measures of physiological activity. The bulk of media research in the area has relied on measures of the peripheral nervous system. Thus, we begin with a discussion of those approaches, followed by a discussion of central nervous system measures used in media processes and effects research.
Peripheral Nervous System Measures in Media Processes and Effects Research
Peripheral nervous system measures have the most established history in psychophysiology. As noted above, the first media effects experiment to record physiological activity measured skin conductance, a peripheral nervous system measure based on the activity of sweat glands. The early work of psychophysiologists focused on mapping variation in peripheral nervous system activity reflected in heart rate and skin conductance to variation in mental processes related to attention and arousal respectively (Graham, 1979; Lacey & Lacey, 1974; Sokolov, 1963). The early work also included studies of emotion, in which precise variation in facial expressions based on peripheral nervous system activity consisting of facial muscle responses were recorded (facial electromyography [EMG]; Fridlund & Cacioppo, 1986).
Skin conductance, heart rate, and facial EMG have emerged as the most commonly used psychophysiological indicators in media processes and effects research in large part due to early validation connecting these measures with variation in attention, arousal, and positive/ negative emotional responses. Because the brain is connected to the spinal column (i.e., central nervous system) which contains nerves that run to all organs, glands, and muscles of the peripheral nervous system, brain activity generated through mental processes expands into the peripheral nervous system influencing the activity of organs, glands, and muscles. That is why heart rate, skin conductance, and facial EMG are valid indicators of embodied mental processes occurring in the brain.
Heart Rate in Media Processes and Effects Research
Heart rate is based on variation in the cardiac cycle that occurs as the heart beats and circulates blood throughout the body. Heart rate can be recorded in two ways. First, it can be recorded through pulse rate, produced by blood flowing through the arteries and veins. This method typically involves the application of a photoplethysmograph (often placed on a finger or earlobe)
Media Psychophysiology and Neuroscience • 201
that uses a photo sensor to measure blood flow and quantify pulse rate. Second, heart rate can be recorded by an electrocardiogram, which requires placing surface electrodes on the wrist and forearms. The electrocardiogram produces a visual representation of the bioelectrical activity generated during the cardiac cycle. Heart rate is quantified by measuring the amount of time between successive instances of the strongest positive peak in the electrocardiogram, which is referred to as the inter-beat interval.
The heart is innervated by both the parasympathetic and sympathetic branches of the peripheral nervous system, which complicates interpretation of changes in heart rate (Koruth, Lang, Potter, & Bailey, 2015). That being said, heart rate is most often used as a psychophysiological indicator of cognitive resources allocated to encoding information into a short term memory representation (Potter & Bolls, 2012), based on the assumption that a primary task involved in using media is taking in information from the content and encoding it in short term memory. This task increases parasympathetic nervous system activity, which decreases heart rate. It is important to remember that this is an assumption, and recent work by media processes and effects researchers has demonstrated the tremendous value of assessing heart rate variability, a technique for teasing apart parasympathetic and sympathetic influences on heart rate in interpreting heart rate data (Koruth et al, 2015).
Heart rate as a psychophysiological indicator of cognitive-resource allocation has been used in various media contexts, including, for example, how mobile media users cognitively process advertisements embedded in mobile content (Clark, Leslie, Garcia-Garcia, & Tullman, 2018). The researchers found that users allocated more cognitive resources (as evidenced through heart rate change) to encoding the ads when (1) the user had more control over interacting with the ad, (2) the ad was placed within the content in the least disruptive manner, and (3) the user was offered incentives for viewing the ad.
Heart rate has also been used to study how emotional content in anti-tobacco advertisements can influence the way smokers process the messages. In a recent study, heart rate data showed that combining smoking cues with disgusting images resulted in defensive message processing in smokers (i.e., fewer cognitive resources allocated to encoding the message; Clayton, Leshner, Bolls, & Thorson, 2017). This study is in line with other research (e.g., Yegiyan, 2015) using heart rate data to show how the emotional valence of video content can impact cognitive resources allocated to encoding, with pleasant (or appetitive) emotional content increasing (as evidenced by heart rate deceleration) and unpleasant (or aversive) content decreasing (as evidenced by heart rate acceleration) cognitive resources allocated to encoding.
Skin Conductance in Media Processes and Effects Research
Skin conductance involves recording changes in electrical conductance of the small current between two recording electrodes. Electrical conductance across the surface of the skin changes due to the activity of eccrine sweat glands. These sweat glands are associated with what has been termed psychological sweating (e.g., sweating palms due to nervousness). Accordingly, skin conductance is most frequently measured by placing electrodes on the surface of the palm or midsections of the fingers. Skin conductance is based on the activity of sweat glands below the surface of the skin, not the actual appearance of sweat on the skin's surface. The level of sweat in the bulb of the gland changes skin conductance. Electrical resistance of the skin surface decreases and conductance increases as the level of sweat rises in the gland. The eccrine sweat glands are solely innervated by the sympathetic branch of the peripheral nervous system. This
202 . Paul D. Bolls et al.
makes skin conductance a highly valid and reliable psychophysiological indicator of sympathetic arousal, taken to reflect the intensity of emotional responses evoked by a stimulus (Potter & Bolls, 2012).
Skin conductance has been used to study both general sympathetic arousal as well as anxiety specifically, in response to different media contexts (e.g., computer-mediated versus face-to-face communication; Shalom, Isreali, Markovitzky, & Lipsitz, 2015). One particularly interesting line of research examines arousal responses to video game content. In one study of a racing game, researchers observed that the intensity level of the game led to increased skin conductance, which also had a positive correlation with risk-taking inclination and resulted in a greater likelihood of risky driving behaviors (Deng, Chan, Wu, & Wang, 2015). Skin conductance has also been used to study sympathetic arousal while parents view media content with their children (i.e., parental co-viewing; e.g., Rasmussen, Keene, Berke, Densley, & Loof, 2017). The mere presence of a co-viewing parent increased the child's level of sympathetic arousal experienced when viewing educational television content.
Facial EMG in Media Processes and Effects Research
Facial electromyography (EMG) involves the recording of facial muscle activity through the placement of surface electrodes over specific muscle regions on the face. The electrodes record the bioelectrical signal that is generated by the firing of action potentials in the neurons of muscle tissue that occur when muscles contract. This makes facial EMG a highly sensitive measure of facial expressions because it records potentially meaningful activity that might not be visible on a person's face.
Facial EMG is one of the more challenging peripheral nervous system measures to validly record because the signal is especially susceptible to noise. The human face contains many muscles located closely together; thus, valid recording of the facial EMG signal is highly dependent on precise electrode placement. Noise can enter the recorded signal due to the activity of nearby muscles, as well as electrical frequencies generated by other equipment in the environment that may overlap with the frequency of the facial EMG signal.
Facial EMG is a validated psychophysiological indicator of emotional valence to a stimulus (Potter & Bolls, 2012). Emotional responses recorded through facial EMG can be positive, negative, or a mixture of positive and negative valence. Three different facial muscle regions are of particular interest to media processes and effects researchers: corrugator (located along the brow), orbicularis oculi (outer corner of the eye), and the zygomatic (along the cheek). Corrugator activity reflects variance in negative emotional responding, whereas orbicularis oculi and zygomatic activity reflects variance in positive emotional responding.
Facial EMG has proven to be a useful indicator of emotional valence in a wide range of media research. Researchers have, for example, used the measure to examine how positive and negative online news about corporations interact with existing corporate reputations in influencing how readers emotionally respond to the stories (Ravaja et al, 2015). Facial EMG activity reflecting a negative emotional response was higher for negative news stories about corporations with good reputations, whereas a positive response was higher for negative stories about corporations with bad reputations. Facial EMG has also been used to show that more calloused individuals have weaker negative emotional responses to violent films (Fanti, Kyranides, & Panayiotou, 2017) and that variation in the emotional tone of traffic safety videos can meaningfully impact viewer responses (Howell, Ekman, Almond, & Bolls, 2018).
Media Psychophysiology and Neuroscience • 203
Central Nervous System Measures in Media Processes and Effects Research
An exciting development in media research has been the resurgence in the use of central nervous system measures. Researchers working in the late 1970s and early 1980s used electroencephalogram (EEG; the electrical recording of cortical activity from the surface of the scalp) to study attention paid to television advertisements (Appel, Weinstein, & Weinstein, 1979; Krugman, 1971; Rothschild, Hyun, Reeves, Thorson, & Goldstein, 1988). But the latest age of central nervous system measures use has introduced a whole new ballgame: media neuroscience (Mangus, Adams, & Weber, 2015).
Media neuroscience and the use of brain imaging are exciting developments because the work provides a view of the brain "on" media. Central nervous system measures give researchers a peek at the functioning of mental processes engaged when individuals interact with media that emerge directly from brain activity. The use of these measures is referred to as brain imaging because the data recorded literally produces images of brain activity. It is critical to remember that images of brain activity can be as meaningless as looking at pictures of "blobs" on brains without proper training and expertise. In no way does this chapter substitute for that; specific training in brain anatomy and physiology, as well as measurement and data analysis techniques used with these measures, is required (Hopp & Weber, in press; Turner, Huskey, & Weber, in press; Weber, Mangus, et al, 2015).
This section highlights two common brain imaging techniques used in the application of central nervous system measures in media neuroscience: EEG and fMRI (functional magnetic resonance imaging). The valid application of both measures in media processes and effects research requires a basic understanding of brain anatomy. Here we provide a general summary that we hope is useful for a basic understanding of EEG and fMRI.
One major division in brain anatomy is between cortical areas (like the cerebral cortex, the outer folds of brain matter) and sub-cortical structures (like the limbic system). EEG only records cortical activity. fMRI is required for imaging activity occurring in subcortical structures. The brain is further organized into left and right hemispheres and four lobes: the frontal, temporal, occipital, and parietal that are contained in each hemisphere. Areas of the brain can play a dominant function in a mental process, but a highly localized view of brain function that tries to pin specific mental processes to specific brain locations is misguided. Higher cognitive functions are rooted in the activity of the cerebral cortex, whereas sub-cortical structures in the limbic system (e.g., amygdala, hippocampus) play an important role in emotion and memory. It would be a complete misunderstanding, however, to assume, for instance, that television advertisements that elicit higher activation in the amygdala are loved more than advertisements that elicit less activation in this sub-cortical structure.
EEG in Media Processes and Effects Research
Through the application of surface electrodes, EEG records small electrical signals that can be detected from the surface of the scalp. This electrical signal is generated by the firing of neurons in activated cortical areas. Cortical activity can be recorded from both hemispheres and all four of the brain lobes. This allows inferences to be made about mental processes engaged during various tasks—including interacting with media—emerging from cortical networks throughout the hemispheres and brain lobes.
204 . Paul D. Bolls et al.
Researchers have used EEG to study multiple forms of cognitive and emotional processes engaged through media use (for a recent review, see Morey, 2018). Herein we briefly review a few examples of recent experiments. Attention, one of the earliest cognitive processes studied in media research using EEG, continues to be of interest in recent work. Variation among brain waves recorded through EEG (specifically beta, theta, and alpha) continues to be used to reliably indicate how much attention, for instance, different types of video advertising elicits in viewers (Daugherty, Hoffman, Kennedy, & Nolan, 2018). Morey (2017) used EEG to study specific memory processes involved in cognitive processing of ads. Specifically, the gamma frequency band of the EEG signal was analyzed in response to exposure to positive and negative political television spots; this brainwave frequency is thought to be related to memory processes related to semantic processing and binding of information in memory. Results indicated that Republicans remembered more content from positive ads, whereas Democrats remembered more content from negative ads. Further, increased activity in the gamma frequency band of the EEG signal was related to increased semantic processing and memory.
EEG has also been used to investigate cognitive and emotional processes associated with newer, interactive media technologies. For example, individuals who report a preference for computer-mediated communication have shown a heightened predisposition to pay attention to emotional stimuli and decreased capacity to regulate emotional responses through their recorded EEG activity (Babkirk, Luehring-Jones, & Dennis-Tiwary, 2016). Researchers have also used EEG to identify neurobiological correlates that make up a brain network underlying the psychological phenomenon known as fear of missing out (FOMO), which is often associated with social media use (Lai, Altavilla, Ronconi, & Aceto, 2016). Finally, one of the most extensive applications of EEG in media processes and effects research has been to examine violent video game effects. In a recent study, frequent players of graphically violent games were found to experience lower levels of empathy, as evidenced by less attention paid to positive and negative facial expressions of others (Stockdale, Morrison, Palumbo, Garbarino, & Silton, 2017).
fMRI in Media Processes and Effects Research
fMRI measures differences in magnetic properties of fluids and tissues in the brain. Hemoglobin, a protein that transports oxygen in the blood, has different magnetic properties depending on the level of oxygen in blood cells. Neural activity consumes energy in the form of oxygen and glucose. Consequently, neural processing leads to changes of oxygen concentration in blood cells, which changes the magnetic properties in blood, and then leads to small disturbances in an MRI scanner's strong magnetic field (Logothetis & Pfeuffer, 2004). These disturbances can be detected with highly sensitive receivers in the fMRI scanner. This effect is known as the blood oxygen level dependent (BOLD) effect and is one of the foundational principles of functional MRI. Taking advantage of this effect, latest generation fMRI scanners allow for the generation of brain images with a maximum spatial resolution of 0.5-1.0 cubic millimeter and a maximum temporal resolution of 0.5-1.0 second.
As with all psychophysiological measures, the BOLD effect generates a very weak signal compared to the numerous additional noise sources that cannot be avoided in fMRI scanning. Therefore, the generation and analysis of fMRI data requires sophisticated data pre-processing in order to obtain reliable and valid measures of brain activity at specified locations and at specified time points. For readers interested in the details of fMRI data analysis, we recommend
Media Psychophysiology and Neuroscience • 205
the well-written fMPJ textbook from Huettel, Song, and McCarthy (2014). Easy-to-process introductions into fMRI tailored for communication and media scholars are also available (see Weber, Mangus, et al, 2015; Weber, Fisher, Hopp, & Lonergan, 2017). For communication scholars interested in advanced fMRI analyses, we recommend Bassett and Sporns (2017), Sporns (2010), and Turner et al. (in press).
The availability of fMRI facilities for communication scholars has significantly increased within the past ten years. This has led to decreases in costs for fMRI research. As a result, communication and media scholars have started studying a diverse set of topics with fMRI, including the use of brain imaging data as a predictor for persuasive message effectiveness (e.g., Falk, Berkman, Mann, Harrison, & Lieberman, 2010; Huskey, Mangus, Turner, & Weber, 2017; Weber, Huskey, Mangus, Westcott-Baker, & Turner, 2015), attentional processes and flow during media exposure (e.g., Huskey, Craighead, Miller, & Weber, 2018; Weber, Alicea, Huskey, & Mathiak, 2018), narrative engagement and neurocinematics (e.g., Hasson et al, 2008; Schmal-zle, Hacker, Honey, & Hasson, 2015), message virality (e.g., Scholz et al, 2017), and affectionate communication in close relationships (Hesse et al, 2013). For recent and more detailed summaries of such studies, we recommend Falk, Cascio, and Coronel (2015), Floyd and Weber (in press), Weber, Mangus, et al. (2015), and Weber et al. (2017).
A Bright Future for Media Processes and Effects Research
This chapter has presented media psychophysiology and neuroscience as a new framework for media processes and effects research. The adoption of the psychophysiological paradigm by media researchers in the late 1980s generated exciting growth in the field. The infusion of neuroscience and other related scientific disciplines into the ways that psychophysiology is applied in media research has further brightened the future of media processes and effects research.
The media psychophysiology and neuroscience framework described here offers a new perspective on media effects. But this new perspective should not lead to intellectual arguments over the importance of studying media effects, as some might conclude. Instead, it should further elevate the importance of media effects research in communication and across all scientific disciplines. But this framework does assume different answers to some of the most basic questions confronting researchers in this area.
• What are media effects? According to the media psychophysiology and neuroscience framework, media effects consist of changes that constantly occur when individuals interact with media of any form, as opposed to changes in a static outcome measure obtained after media exposure.
• Where do media effects occur? According to the media psychophysiology and neuroscience framework, media effects occur in and emerge from the context-dependent functioning of the embodied human mind.
• When do media effects occur? According to the media psychophysiology and neuroscience framework, media effects unfold across time through the functioning of dynamic embodied mental processes evoked through media use.
These perspectives shape a recommended path forward to discovering unique knowledge about media processes and effects.
206 . Paul D. Bolls et al.
One area of research where the potential of this new perspective can be clearly illustrated is newer interactive and immersive media technology and content. Although media technology and content change constantly, the basic functions of the embodied human mind do not. This truth gives the framework described in this chapter an advantage over traditional approaches to media research in advancing knowledge of newer forms of media. Traditional approaches tend to focus on effects tied to a specific media form. In contrast, the media psychophysiology and neuroscience framework focuses on dynamic embodied mental processes occurring in the human brain and nervous system, which do not fundamentally change with each new form of media content and technology. This makes the media psychophysiology and neuroscience framework useful for exploring traditional, newer, and yet-to-be-invented forms of media.
An additional reason this approach has tremendous value is that media technologies are becoming increasingly immersive by becoming more embodied. Many newer technologies (e.g., virtual reality; see Chapter 26 in this volume) are designed to engage a wider form and degree of bodily responses through larger and more intense sensory experiences and physical bodily interactions. Such changes make the embodied experience of interacting with newer forms of media technology more critical to understanding the related processes and effects. The media psychophysiology and neuroscience framework offers researchers a valid approach to conceptualize all media use as an embodied experience.
A final point has to do with the unique opportunity that researchers who adopt the framework described here have to build bridges between academia and industry. Academic researchers are (often rightly) more focused on the theoretical and social importance of their work rather than the practical. Mutual enthusiasm for the application of brain science in media research, however, could help break through some of the barriers that have prevented academic-industry research collaboration in the past. This is an opportunity that does not exist within other research areas. For example, neuromarketing has sparked great enthusiasm for brain science in the media industries. Neuromarketing research utilizes all the measures discussed in this chapter (along with several additional ones) in an attempt to gain insight into consumers' brains as a means for optimizing the effectiveness of persuasive messages. This new area of marketing research was formally established around 2006 and now includes formal academic programs, mostly located in European business schools. Numerous research companies offer neuromarketing research worldwide. All of the major US television networks either conduct their own neuromarketing research or have a neuromarketing research provider to test content. One example of a successful academic-industry neuromarketing collaboration involved the Advertising Research Foundation and neuromarketing companies to examine the validity and value of neuromarketing research. That effort resulted in a scientific peer-reviewed publication on the validity of measures used in neuromarketing research (Varan, Lang, Barwise, Weber, & Bellman, 2015). Opportunities for similar collaborations should only increase in the future.
In conclusion, we hope that this chapter is a valuable resource for beginning and more experienced researchers, whether you plan to adopt this framework or not. As we have tried to convey, many additional resources are also available for understanding how psychophysiology and neuroscience is applied in media research. The new age of psychophysiology and brain science in media effects research is exciting, with the potential for new knowledge never greater.
Media Psychophysiology and Neuroscience • 207
References
Ann, T. D., & Floyd, K. (Eds.). (2015). Communication, biology, and physiology [Special issue]. Communication Monographs, 82(1). doi:10.1080/03637751.2015.1005964
Appel, V., Weinstein, S., & Weinstein, C. (1979). Brain activity and recall TV advertising. Journal of Advertising Research, 19, 7-15.
Babkirk, S., Luehring-Jones, P., & Dennis-Tiwary, T. A. (2016). Computer-mediated communication preferences predict biobehavioral measures of social-emotional functioning. Social Neuroscience, 11, 637-651. doi:10.1080/17470919.2015.1123181
Bandura, A., Ross, D., & Ross, S. A. (1963). Imitation of film-mediated aggressive models. The Journal of Abnormal and Social Psychology, 66, 3-11. doi:10.1037/h0048687
Bassett, D. S., & Sporns, O. (2017). Network neuroscience. Nature Neuroscience, 20(3), 353-364. doi:10.1038/nn.4502
Berntson, G. G., & Cacioppo, J. T. (2000). Psychobiology and social psychology: Past, present, and future. Personality and Social Psychology Review, 4, 3-15. doi:10.1207/S15327957PSPR0401_2
Bryant, J., & Oliver, M. B. (Eds.). (2009). Media effects: Advances in theory and research. New York, NY: Routledge. doi:10.4324/9780203877111
Cacioppo, J. T., Tassinary, L. G., & Berntson, G. (Eds.). (2007). Handbook of psychophysiology. New York, NY: Cambridge University Press. doi:10.1017/cbo9780511546396
Cannon, W. B. (1931). The James-Lange theory of emotion: A critical examination and an alternative theory. America Journal of Psychology, 39, 106-124. doi:10.2307/1422695
Chaffee, S. H. (1977). Mass media effects: New research perspectives. In D. Lerner & L. M. Nelson (Eds.), Communication research: A half-century appraisal (pp. 210-241). Honolulu, HI: University of Hawaii Press.
Clark, K. R., Leslie, K. R., Garcia-Garcia, M., & Tullman, M. L. (2018). How advertisers can keep mobile users engaged and reduce video-ad blocking: Best practices for video-ad placement and delivery based on consumer neuroscience measures. Journal of Advertising Research, 58, 311— 325. doi:10.2501/jar-2018-036
Clayton, R. B., Leshner, G., Bolls, P. D., & Thorson, E. (2017). Discard the smoking cues—Keep the disgust: An investigation of tobacco smokers' motivated processing of anti-tobacco commercials. Health Communication, 32, 1319-1330. doi:10.1080/10410236.2016.1220042
Daugherty, T., Hoffman, E., Kennedy, K., & Nolan, M. (2018). Measuring consumer neural activation to differentiate cognitive processing of advertising: Revisiting Krugman. European Journal of Marketing, 52(1/2), 182-198. doi:10.1108/EJM-10-2017-0657
Delia, J. G. (1987). Communication research: A history. In C. R. Berger & S. H. Chaffee (Eds.), Handbook of communication science (pp. 20-98). Newbury Park, CA: Sage.
Deng, M., Chan, A. H., Wu, F., & Wang, J. (2015). Effects of racing games on risky driving behaviour, and the significance of personality and physiological data. Injury Prevention, 21, 238-244. doi:10.1136/injuryprev-2014-041328
Dennis, E. E., & Wartella, E. (Eds.). (1996). American communication research: The remembered history. Mahwah, NJ: Lawrence Erlbaum Associates.
Duffy, E. (1957). The psychological significance of the concept of "arousal" or "activation". Psychological Review, 64, 265-275. doi:10.1037/h0048837
Dysinger, W. S., & Ruckmick, C. A. (1933). The emotional responses of children to the motion picture situation. Oxford, UK: Macmillan.
Falk, E. B., Berkman, E., Mann, T., Harrison, B., & Lieberman, M. D. (2010). Predicting persuasion-induced behavior change from the brain. Journal of Neuroscience, 30, 8421-8424. doi: 10.1523/JNEUROSCI.0063-10.2010
Falk, E. B., Cascio, C. N., & Coronel, J. C. (2015). Neural prediction of communication-relevant outcomes. Communication Methods and Measures, 9, 30-54. doi:10.1080/19312458.2014.999750
208 . Paul D. Bolls et al.
Fanti, K. A., Kyranides, M. N., & Panayiotou, G. (2017). Facial reactions to violent and comedy films: Association with callous-unemotional traits and impulsive aggression. Cognition and Emotion, 31, 209-224. doi:10.1080/02699931.2015.1090958
Floyd, K., & Weber, R. (Eds.), (in press). Communication science and biology. New York, NY: Routledge.
Fridlund, A. J., & Cacioppo, J. T. (1986). Guidelines for human electromyographic research. Psychophysiology, 23, 567-589. doi:10.1111/j.l469-8986.1986.tb00676.x
Geiger, S., & Newhagen, J. (1993). Revealing the black box: Information processing and media effects. Journal of Communication, 43, 42-50. doi:10.1111/j.l460-2466.1993.tb01303.x
Graham, F. K. (1979). Distinguishing among orienting defense, and startle reflexes. In H. D. Kimmel, E. H. Van Olst, & J. F. Orlebeke (Eds.), The orienting reflex in humans (pp. 137-167). Hillsdale, NJ: Lawrence Erlbaum Associates.
Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., & Heeger, D. J. (2008). Neu-rocinematics: The neuroscience of film. Projections, 2, 1-26. doi:10.3167/proj.2008.020102
Hebb, D. O. (1955). Drives and the CNS (conceptual nervous system). Psychological Review, 62, 243-254. doi:10.1037/h0041823
Hergenhahn, B. R., & Henley, T. (2013). An introduction to the history of psychology (7th ed.). Belmont, CA: Cengage Learning.
Hesse, C, Floyd, K., Rauscher, E. A., Frye-Cox, N., Hegarty, J., & Peng, H. (2013). Alexithymia and impairment of decoding positive affect: An fMRI study. Journal of Communication, 63, 786-806. doi:10.1111/jcom.l2039
Hopp, F., & Weber, R. (in press). The state-of-the-art and the future of brain imaging methodology in communication research. In K. Floyd & R. Weber (Eds.), Communication Science and Biology. New York, NY: Routledge.
Howell, M., Ekman, D. S., Almond, A., & Bolls, P. (2018). Switched on: How the timing of aversive content in traffic safety videos impact psychophysiological indicators of message processing. Health Communication, Advanced online publication. doi:10.1080/10410236.2018.1517706
Huettel, S. A., Song, A. W., & McCarthy, G. (2014). Functional magnetic resonance imaging (3rd ed.). Sunderland, MA: Sinauer.
Huskey, R., Craighead, B., Miller, B. M., & Weber, R. (2018). Does intrinsic reward motivate cognitive control? A naturalistic fMRI study based on the synchronization theory of flow. Cognitive Affective and Behavioral Neuroscience, 18, 902-924. doi:10.3758/sl3415-018-0612-6
Huskey, R., Mangus, J. M., Turner, B., & Weber, R. (2017). The persuasion network is modulated by drug-use risk and predicts anti-drug message effectiveness. Social Cognitive and Affective Neuroscience, 12, 1902-1915. doi:10.1093/scan/nsxl26
James, W. (1894). Discussion: The physical basis of emotion. Psychological Review, 1, 516-529. doi:10.1037/h0065078
Koruth, K. J., Lang, A., Potter, R. F., & Bailey, R. L. (2015). A comparative analysis of dynamic and static indicators of parasympathetic and sympathetic nervous system activation during TV viewing. Communication Methods and Measures, 9, 78-100. doi:10.1080/19312458.2014.999752
Krugman, H. E. (1971). Brain wave measures of media involvement. Journal of Advertising Research, 11, 3-9. doi:10.4135/9781452231501.nl3
Lacey, J. I., & Lacey, B. C. (1974). On heart rate responses and behavior: A reply to Elliott. Journal of Personality and Social Psychology, 30, 1-18. doi:10.1037/h0036559
Lachman, R., Lachman, J. L., & Butterfield, E. C. (1979). Cognitive psychology and information processing: An introduction. London, UK: Psychology Press.
Lai, C, Altavilla, D., Ronconi, A., & Aceto, P. (2016). Fear of missing out (FOMO) is associated with activation of the right middle temporal gyrus during inclusion social cue. Computers in Human Behavior, 61, 516-521. doi:10.1016/j.chb.2016.03.072
Media Psychophysiology and Neuroscience • 209
Lang, A., Potter, R., & Bolls, P. (2009). Where psychophysiology meets the media: Taking the effects out of mass media research. In J. Bryant & M. B. Oliver (Eds.), Media effects: Advances in theory and research (3rd ed., pp. 185-206). New York, NY: Routledge.
Logothetis, N. K., & Pfeuffer, J. (2004). On the nature of the BOLD fMRI contrast mechanism. Magnetic Resonance Imaging, 22, 1527-1531. doi:10.1016/j.mri.2004.10.018
Mangus, J. M., Adams, A., & Weber, R. (2015). Media neuroscience. In R. A. Scott & S. M. Kosslyn (Eds.), Emerging trends in the behavioral and social sciences (pp. 1-14). Hoboken, NJ: Wiley-Blackwell.
Morey, A. C. (2017). Memory for positive and negative political TV ads: The role of partisanship
and gamma power. Political Communication, 34, 404-423. doi:10.1080/10584609.2016.1266534 Morey, A. C. (2018). Electroencephalography in communication research: A review of the past
and a glimpse of future possibilities. Annals of the International Communication Association,
42, 243-269. doi:10.1080/23808985.2018.1537723 Potter, R. F., & Bolls, P. (2012). Psychophysiological measurement and meaning: Cognitive and
emotional processing of media. New York, NY: Routledge. Rasmussen, E. E., Keene, J. R., Berke, C. K., Densley, R. L., & Loof, T. (2017). Explaining parental
coviewing: The role of social facilitation and arousal. Communication Monographs, 84, 365-
384. doi:10.1080/03637751.2016.1259532 Ravaja, N., Aula, P., Falco, A., Laaksonen, S., Salminen, M., & Ainamo, A. (2015). Online news
and corporate reputation: A neurophysiological investigation. Journal of Media Psychology, 27,
118-133. doi:10.1027/1864-1105/a000149 Rothschild, M. L., Hyun, Y. J., Reeves, B., Thorson, E., & Goldstein, R. (1988). Hemispherically
lateralized EEG as a response to television commercials. Journal of Consumer Research, 15(2),
185-198. doi:10.1086/209156 Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of emotional
state. Psychological Review, 69, 379-399. doi:10.1037/h0046234 Schmälzle, R., Hacker, F. E. K., Honey, C. J., & Hasson, U. (2015). Engaged listeners: Shared
neural processing of powerful political speeches. Social Cognitive and Affective Neuroscience,
10, 1137-1143. doi:10.1093/scan/nsul68 Scholz, C, Baek, E. C, O'Donnell, M. B., Kim, H. S., Cappella, J. N., & Falk, E. B. (2017). A neural
model of valuation and information virality. Proceedings of the National Academy of Sciences,
114, 2881-2886. doi:10.1073/pnas.l615259114 Shalom, J. G., Israeli, H., Markovitzky, O., & Lipsitz, J. D. (2015). Social anxiety and physiological
arousal during computer mediated vs. face to face communication. Computers in Human
Behavior, 44, 202-208. doi:10.1016/j.chb.2014.11.056 Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Oxford, England:
Appleton - Century.
Sokolov, E. N. (1963). Perception and the conditioned reflex. Oxford, UK: Pergamon Press. Sporns, O. (2010). Networks of the brain. Boston, MA: MIT press.
Stern, R. M., Ray, W. J., & Quigley, K. S. (2001). Psychophysiological recording (2nd ed.). Oxford,
UK: Oxford University Press. Stockdale, L., Morrison, R. G., Palumbo, R., Garbarino, J., & Silton, R. L. (2017). Cool, callous and
in control: Superior inhibitory control in frequent players of video games with violent content.
Social Cognitive and Affective Neuroscience, 12, 1869-1880. doi:10.1093/scan/nsxll5 Turner, B., Huskey, R., & Weber, R. (in press). Charting a future for fMRI in communication
science. Communication Methods and Measures. doi:10.1080/19312458.2018.1520823 Varan, D., Lang, A., Barwise, P., Weber, R., & Bellman, S. (2015). How reliable are neuromark-
eters' measures of advertising effectiveness? Data from ongoing research holds no common
210 . Paul D. Bolls et al.
truth among vendors. Journal of Advertising Research, 55, 176-191. doi:10.2501/JAR-55-2-176-191
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158-177. doi:10.1037/h0074428
Weber, R. (Ed.). (2015). Brain, mind, and media: Neuroscience meets media psychology [Special issue]. Journal of Media Psychology, 27(3). doi:10.1027/1864-1105/a000108
Weber, R., Alicea, B., Huskey, R., & Mathiak, K. (2018). Network dynamics of attention during a naturalistic behavioral paradigm. Frontiers in Human Neuroscience, 22(182). doi:10.3389/ fnhum.2018.00182
Weber, R., Eden, A., Huskey, R., Mangus, J. M., & Falk, E. (2015). Bridging media psychology and cognitive neuroscience: Challenges and opportunities. Journal of Media Psychology, 27, 146-156. doi:10.1027/1864-1105/a000163
Weber, R., Fisher, J. T., Hopp, F. R., & Lonergan, C. (2017). Taking messages into the magnet: Method-theory synergy in communication neuroscience. Communication Monographs, 85, 81-102. doi:10.1080/03637751.2017.1395059
Weber, R., Huskey, R., Mangus, J. M., Westcott-Baker, A., & Turner, B. (2015). Neural predictors of message effectiveness during counterarguing in antidrug campaigns. Communication Monographs, 82, 4-30. doi:10.1080/03637751.2014.971414
Weber, R., Mangus, J. M., & Huskey, R. (2015). Brain imaging in communication research: A practical guide to understanding and evaluating fMRI studies. Communication Methods and Measures, 9, 5-29. doi:10.1080/19312458.2014.999754
Weber, R., Sherry, J., & Mathiak, K. (2008). The neurophysiological perspective in mass communication research. Theoretical rationale, methods, and applications. In M. J. Beatty, J. C. McCroskey, & K. Floyd (Eds.), Biological dimensions of communication: Perspectives, methods, and research (pp. 41-71). Cresskill, NJ: Hampton Press.
Yegiyan, N. S. (2015). Explicating the emotion spillover effect. Journal of Media Psychology, 27, 134-145. doi:10.1027/1864-1105/a000164
Zillmann, D., & Bryant, J. (1974). Effect of residual excitation on the emotional response to provocation and delayed aggressive behavior. Journal of Personality and Social Psychology, 30, 782-791. doi:10.1037/h0037541