Adobe Systems 1 Content analysis and expert surveys Radicalization of politics in CEE Adobe Systems 2 Outline How to analyze populist content? a)Content analysis (various forms) – the logic, possibilities, the data b)Expert surveys – advantages and disadvantages, the data c) Goal: to get the basics of selected methods of analysis of political contents, introduction of some data sources Adobe Systems 3 Based on the ideational approach to populism Populism is “a thin-centered ideology that considers society to be ultimately separated into two homogeneous and antagonistic camps, “the pure people” versus “the corrupt elite”, and which argues that politics should be an expression of the volonté générale (general will) of the people.” (Mudde 2004: 543). ̶ ̶ Adobe Systems 4 Ideational approach to populism Three core components of populism ̶ Anti-elitisim Sovereignty People-centrism Adobe Systems 5 Content analysis Content/ textual analysis: ̶ideas of political actors are measured through texts that they produce (speeches, party manifestos, etc.) ̶Texts as „true“ representatives of what politicians say are plan to do Adobe Systems 6 Content analysis Types of content analysis: ̶Mancoded/handmade ̶Computerised/automatized ̶ Note: Classical content analysis can be combined with computerised content analysis (semi-automated content analysis). - Usually follow the deductive, concept/theory rooted approach There are two main types… Adobe Systems 7 Content analysis Classical content analysis: ̶a codebook is defined ̶coders systematically analyse text by means of a codebook (human-coded approach) In a classical content analysis researchers first create a codebook, in which they specify what they mean by populism, how they want to measure/ code it. This codebook is then used by coders to analyse populism in texts of political actors – speeches of presidents, party manifestos etc. People do the analysis – referred to as human-coded approach. Adobe Systems 8 Content analysis Computerised content analysis: ̶dictionary of populist terms defined (indicators of populism) ̶computer counts the proportion of words that we consider to be indicators of populism ̶Mostly doable in English (some tools in other languages, possibility to use Google Translate) In computerised content analysis researchers first create dictionary containing words/ phrases which are indicators of populism. Computer is then given both the texts of political actors in which populism should be measured and the dictionary. Output is the proportions of populist words in those texts. Adobe Systems 9 Content analysis: units of analysis Ukip Pledge 600 Výsledek obrázku pro trump speech …election manifestos, speeches… Adobe Systems 10 Content analysis: units of analysis Související obrázek Výsledek obrázku pro trump tweet …debates, newspaper articles, but also Tweets or Facebook posts. Adobe Systems 11 Content analysis units of analysis ≠ units of measurement Chunks of these texts are then units of our analysis. Adobe Systems 12 Content analysis: units of measurement Donald Trump inauguration speech transcript 1 2 3 4 These can be paragraphs… Adobe Systems 13 Content analysis: units of measurement Donald Trump inauguration speech transcript …words… Adobe Systems 14 Content analysis: units of measurement Donald Trump inauguration speech transcript 1 …entire texts (common in case of speeches). In such case we talk about holistic grading approach in content analysis. Adobe Systems 15 Content analysis in practise Rooduijn , M. & Pauwels, T. 2011. “Measuring Populism: Comparing Two Methods of Content Analysis.” West European Politics 34(6), 1272–1283, online https://www.tandfonline.com/doi/abs/10.1080/01402382.2011.616665 Lets have a look at the article on content analysis you have been asked to read before the class Adobe Systems ̶authors use both classical and computerised content analysis to measure and compare the degree of populism of political parties in the UK, the Netherlands, Germany and Italy ̶ ̶units of analysis: election manifestos ̶core components of populism: people-centrism, anti-elitism ̶ 16 Rooduijn & Pauwels 2011 Content analysis in practise In their research,… They measure populism in election manifestos of political parties in these countries. And they try to identify people-centrism and anti-elitism in these party manifestos. Adobe Systems Classical content analysis ̶units of measurement: paragraph ̶à comparison of the degree of populism among parties based on % of populist paragraphs in their manifestos ̶paragraphs in which both people-centrism and anti-elitism were present selected as populist ̶ 17 Rooduijn & Pauwels 2011 Content analysis in practise In classical content analysis they look at individual paragraphs in election manifestos. Only if both people-centrism and anti-elitism are identified in a paragraph, that paragraph is coded as populist. Therefore, the comparison of the degree of populism…. Adobe Systems Classical content analysis: operationalisation of the core components of populism ̶people-centrism: “Do the authors of the manifesto refer to the people?” ̶examination of every possible reference to the people (citizens, our country, society, we the people, etc.) list of words and combinations of words that could refer to the people: people, citizen(s), community, society, public, population, nation(al), all of us, each of us, everyone, our, we, voter(s), electorate, referenda, direct democracy, public opinion, country, specific country (depending of course on the country under analysis ̶interpretation of the broader context 18 Rooduijn & Pauwels 2011 Content analysis in practise By people-centrism authors mean reference to the people. Coders provided with the list of words and combinations of words that may refer to the people, such as…. Coders also asked to take into account the context – word we may not refer to the people but to the party. Adobe Systems Classical content analysis: operationalisation of the core components of populism ̶anti-elitism: “Do the authors of the manifesto criticise elites?” ̶focus on criticism concerning the elite in general (critique on a specific party/ politician not coded) ̶interpretation of the broader context 19 Rooduijn & Pauwels 2011 Content analysis in practise By anti-elitism authors mean ciriticsm concerning the elite in general, not ciriticism of specific party or polician. Again, coders asked to take into account the context. Adobe Systems Computerised content analysis ̶units of measurement: words ̶à comparison of the degree of populism among parties based on % of populist words (words considered populism) ̶ 20 Rooduijn & Pauwels 2011 Content analysis in practise In computerised content analysis authors look at individual words in party manifestos. Therefore, comparison of the degree of populism is based… in those manifestos. Adobe Systems Computerised content analysis: operationalisation of the core components of populism ̶focus only on words that refer to anti-elitism ̶measurement of people-centrism by means of individual words in nearly impossible (words we and our often mentioned not in reference to the people, but to the political party) ̶selection of words into dictionary based on both empirical and theoretical reasoning ̶election manifestos of populist parties not analysed in this study used to make list of words that such parties have used to express their negativity towards elites ̶ 21 Rooduijn & Pauwels 2011 Content analysis in practise Authors developed their own dictionary. Contains only words that refer to anti-elitism. They used party manifestos of populist parties not included in the analysis to identify those words. Adobe Systems 22 Rooduijn & Pauwels 2011 Content analysis in practise Here you can see what a dictionary used in a computerised content analysis can look like. Star symbols next to each chunk. Means that such chunk can be followed or preceded by any letters. Why is it useful? Adobe Systems 23 Rooduijn & Pauwels 2011 Content analysis in practise Example: chunk corrupt* -> not only word corrupt but also corruption, corrupted counted as indicator of populism in a text Adobe Systems 24 Content analysis The Manifesto Project: https://manifesto-project.wzb.eu/ ̶ ̶content analysis of parties’ election manifestos in order to study parties’ policy preferences ̶for each dataset: ̶codebook ̶coding instructions ̶list of coded documents (titles of parties’ election manifestos) ̶list of political parties If interested in PM project – look at it at home. In the available datasets you can find informaiton about parties‘ policy preferences. Does not measure populism but you can look at policy preferences of populist political parties or compare them to other parties. Adobe Systems 25 Jungar, Jupskas (2014) ̶Populist Radical Right Parties in the Nordic Region: A New and Distinct Party Family? ̶Comparing policy positions of PRR parties in Scandinavia ̶Rooted in the concepts of party families and PRR ̶Using CMP data (l-r and authoritarian/liberal indices) Adobe Systems 26 Content analysis Harvard Dataverse – Global Populism Database: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/LFTQEZ ̶ ̶measurement of the level of populist discourse in the speeches of 215 chief executives (presidents and prime ministers) from 66 countries across all continents, mostly between 2000 and 2018 ̶holistic grading For this course, Global populism Database more usufl Dataset contains data on measurement ….. Holistic grading used – entire text – speech – one unit of analysis. Example of codebook. Adobe Systems 27 Advantages & Drawbacks of different types of content analysis Adobe Systems 28 Expert surveys Who are respondents in expert surveys? ̶ Adobe Systems 29 Expert surveys Expert surveys ̶respondents are experts with a high degree of specialized knowledge in their own national contexts ̶ ̶experts are presented with predefined questions or scales and asked to use their best judgements when answering the questions/ placing given actors on given scales ̶ Note: Each scale needs to be given a clear title and anchored at each end with two precise substantive definitions of the scale endpoints based on the theoretical understanding of the issue we are interested in. Adobe Systems 30 Expert surveys in practise Wiesehomeier, N. (2018). Expert Surveys In Hawkins, K., Carlin, R., Littvay, L. & Rovira Kaltwasser, C. (eds.) The Ideational Approach to Populism: Concept, Theory, and Analysis. London: Routledge, pp. 90–111. Chapter on ES Adobe Systems ̶expert survey used to measure populism in Latin America (two waves: 2011/2012 and 2015) ̶ ̶location of parties and presidents along a 20-point scale of Political communication; 1 indicates the populist endpoint, 20 the pluralist endpoint ̶pluralism acknowledges diversity in society ̶pluralist actors consider different groups that constitute the social fabric of a country as legitimate, and favour the diffusion of power ̶ 31 Wiesehomeier 2018 Expert surveys in practise The author used ES to meaure…. Survey was conducted in two waves In a survey, experts were asked to loacte parties along a Unlike populism, pluralism…. Adobe Systems Second wave (2015): Scale of Political communication Dimension: People-centrism ̶1: “Identifies with the common people and celebrates their authenticity.” ̶20: “Refers more generally to citizens and their unique interests.” 32 Wiesehomeier 2018 Expert surveys in practise Let‘s have a look at the scales used in the 2nd wave, on whhich experrts were asked to allocate of political actors - Two scales – one for people-centrism, other for anti-elitism - endpoints of the scale on people-centrism – populist endpoint describe as: -Pluralist endpoint described as: Adobe Systems Second wave (2015): Scale of Political communication Dimension: Anti-elite: morality ̶1: “Demonizes and vilifies opponents.” ̶20: “Treats opponents with respect.” 33 Wiesehomeier 2018 Expert surveys in practise - endpoints of the scale on anti-elitism– populist endpoint describe as: -Pluralist endpoint described as: Adobe Systems 34 Expert surveys Useful datasets: ̶2017 Chapel Hill Expert FLASH Survey (CHES) ̶Populism and Political Parties Expert Survey (POPPA) ̶ Two sources of useful datasets Adobe Systems 35 Expert surveys 2017 CHES: https://www.chesdata.eu/1999-2014-chapel-hill-expert-survey-ches-trend-file-1 ̶ ̶carried out in January and February 2018, asked about party positions in 2017 in14 EU member states: Czech Republic, Estonia, France, Germany, Greece, Hungary, Italy, Netherlands, Poland, Portugal, Slovakia, Spain, Sweden, UK ̶questions on EU placement, economic and social party positioning, policy issues - EU economic coordination, populism, and migration ̶data including sample questionnaire and codebook available First is the Chappel hill expert flash survey whcih was carried out in…. Expertes were asked about….. Survey included questions on Websites – you can find sample questiaonnere, codebook Adobe Systems 36 Expert surveys 2017 CHES: items on populism PEOPLE_VS_ELITE: “Some political parties take the position that 'the people' should have the final say on the most important issues, for example, by voting directly in referendums. At the opposite pole are political parties that believe that elected representatives should make the most important political decisions. Where do the parties fall on this dimension?” ̶ ̶0: “Elected office holders should make the most important decisions.” ̶10: “Extremely important.” ̶ Adobe Systems 37 Expert surveys 2017 CHES: items on populism ANTIELITE_SALIENCE: “Salience of anti-establishment and anti-elite rhetoric.” ̶ ̶0: “Not important at all.” ̶10: “Extremely important.” Adobe Systems 38 Expert surveys 2017 CHES: items on populism ̶mean scores on items used to compare how populist are the parties within a given country ̶ Adobe Systems 39 Engler et al. (2019) ̶Assessing the diversity of anti-establishment and populist politics in Central and Eastern Europe ̶Using expert surveys to discover a) policy positions of AEPs and b) compare them with other parties Adobe Systems 40 Expert surveys POPPA: http://poppa-data.eu/ ̶party positions and attitudes on key attributes related to party ideology, party attitudes toward political representation and party organisation ̶carried out in Spring 2018 ̶data available in late 2019 GLOBAL PARTY SURVEY - https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/WMGTNS ̶ ̶ Interesting source of data is Populism and Political Parties Expert surveys was carried out in… Epxerts asked about Adobe Systems 41 Conclusion ̶You can meaure level of populism by collecting new data or by using existing data sources ̶An economic way how to assess policy positions of parties or compare them accross countries or time ̶In same cases not necessary to know the language, althouhg the numbers needs to be explained