RESEARCHING COMMUNICATIONS A Practical Guide to Methods in Media and Cultural Analysis David Deacon, Michael Pickering, Peter Golding and Graham Murdock Department of Social Sciences, Loughborough University Hodder Arnold A MEMBER OF THE HODDEll HEA111.1NE CROUI' Distributed in the United States of America by Oxford University Press Inc., New York Samples, populations and types of sampling SELECTING AND SAMPLING Sampling is a central part of everyday life. Dipping a toe in the water, flicking through a magazine, 'zapping' across television channels, sipping a glass of wine are all examples of the kinds of routine sampling activity we constantly engage in. We sample from our environment for a range of reasons: to save time, to anticipate events, to minimise discomfort, to decide on future actions, to expand our horizons, and so on. Just as sampling is an integral element of social life, so it is at the heart of all 'scientific' activity, whether in the human or natural sciences. Although researchers sample for broadly similar reasons to everybody else, issues concerning sample validity are inevitably more crucial and complicated, because of the more complex and challenging questions being investigated. For example, you do not need to drink a litre of milk to decide whether it has gone sour. One rancid mouthful should be sufficient for you to pour the rest down the sink or take the carton back for a refund. But if you were trying to assess the presence of an infective agent in a nation's milk supply, testing one randomly selected milk carton off a supermarket shelf would be useless. You would need to cast your net far more widely and systematically. To give you some idea of what might be required, a study of the presence of Listeria monocytogenes in milk in Denmark took samples from 1 132 958 cows from 36 199 herds over a 23-year period (Jensen et al. 1996). In communication and cultural studies, sampling issues involve all kinds of areas, most commonly people, social groups, events, activities, institutions and texts. In this chapter we ignore the last area, as textual sampling is dealt with in detail later (see Chapters 6-10, 12, 13). But it is worth noting in passing the clear parallels between debates surrounding the sampling of texts and sampling issues in other areas. For instance, the rationales for theoretical sampling explained below resonate strongly with the arguments used to support many forms of selective qualitative textual analysis. Similarly, the concerns and strategies for achieving representative samples of large numbers of people match those that arise when using quantitative content analysis to map the macro-dimensions of media discourse. Samples are taken from populations. In research, the term 'population' does not necessarily mean people; it can refer to aggregates of texts, institutions, or anything else being investigated. Furthermore, research populations are defined by specific research objectives. A population can be very small or very large - it depends on who or what you are investigating. For example, if a wine taster sampled a bottle of Chassagne Montrachet Blanc 1996, to deduce the quality of the vintage, his 'population' would be every bottle produced by the vineyard in that year. However, if a forensic chemist was called to examine the contents of a half-finished bottle of wine discovered beside the poisoned corpse of a wine taster, her 'population' would be the specific contents of the bottle. Sampling techniques used in analysing people and institutions can be broadly divided into two categories: random sampling (or 'probability sampling'} and non-random sampling (or 'non-probability sampling'). The key distinctions between these approaches are set out in Table 3.1. These are the foundation for many other variations in sampling styles, which we review later in this chapter. However, there are three broad matters that concern all forms of sampling: sample error, sample size and non-response. We begin by considering these general issues. Sample error: random errors and constant errors Sample error is where the values from a sample differ from the 'true' or actual values of a population. This issue is most evident with samples that attempt to assert claims to general representativeness, but it does hold implications fcitable 3.1 Distinctions between random and non-random sampling Random sampling Non-random sampling Selection of sample units Random - units are selected by chance Non-random - researchers purposively select sample units Estimating chances of inclusion Equality of selection The chance of each unit of a population being selected for a sample (the 'sample fraction') can be calculated Every unit of a population has an equal chance of being selected Selection-chance is unknown St cannot be guaranteed that every unit of a population has an equal chance of being selected 41 researching communications all forms of sampling to some degree. (NB: As we discuss in the next section, not every sampling technique has representational aspirations.) It is accepted that some degree of sample error is unavoidable, and statisticians have developed various statistical tests to estimate the impact it is likely to have had upon the accuracy of sample results (see Chapter 5). The crucial assumption made in these tests is that these errors are random errors - in other words, that sample error is due to the random variation that occurs when you select a smaller number of cases to represent a larger population. However, there is always the risk that samples may contain constant errors - structural biases that systematically distort their representative qualities. Constant errors do not occur randomly, but rather have a consistent pattern that marginalises or over-emphasises certain sections of the population. For example, suppose a student wanted to estimate the average age of all journalists working in the national press, and she compiled her sample by noting down the journalists' by-lines that appeared in six national newspaper titles over six months, and then sent each journalist thus listed a brief questionnaire. The problem with this sampling method is that it would build an elite bias into her sample, because not every journalist gets her or his name listed on a story (a 'by-line'). It is an honour that tends to be conferred on the better-paid, more experienced members of editorial teams, who also tend to be older than those who do not get their work attributed. For this reason, the average age of the sample would probably exceed the average age of her population, because it would marginalise the younger and far more numerous minions. Detecting constant errors in sampling is not always an easy thing to do. It is a matter of scrutinising a study's sampling and research procedures for significant skews. Sample size What size do samples need to be to be credible? Common sense suggests that the larger a sample is, the more confidence you can have in its representativeness. Although this logic applies to a large degree with all sampling, it does not apply as completely as you might initially suppose. For example, in Chapter 5 we show how the size of a random sample has a direct impact upon statistical estimations of its accuracy: the bigger the sample, the smaller the estimation of the standard error of the sample. However, a point is eventually reached where substantial increases in sample size begin to have only small effects on the calculated precision of sample measures (Henry 1990: 118). Once this starts to happen, the benefits of increasing the statistical accuracy of sample measures by sampling more extensively may be seen to be outweighed by the cost and inconvenience of greatly increasing sample size. selecting and sampling 43 It is in qualitative research that the automatic assumption that 'big is beautiful' is most directly challenged. This is because a lot of qualitative studies are less concerned with generating an extensive perspective (producing findings that can be generalised more widely) than in providing intensive insights into complex human and social phenomena in highly specific circumstances (Maykut and Morehouse 1994: 56). This means that qualitative research tends to use comparatively small samples which are generated more informally and organically than those most typically used in quantitative research. Moreover, these 'emergent and sequential' samples (ibid.: 63) do not aim to build up large numbers of similar cases for the purposes of making broader inferences, but rather stop gathering information once the research reaches 'saturation point' (where the data collection stops revealing new things and the evidence starts to repeat itself). According to Lincoln and : Guba (1985) this point can be reached quite quickly, after even as few as 12 interviews. Some qualitative researchers do not even seek this saturation point. For instance, in a fascinating study of the limits to audience power in decoding texts, Condit (1989) sampled just two students, selected on the basis of their strong and contrasting views on abortion (pro-choice and pro-life). Condit shows that despite the vehemence of their beliefs, both participants made very similar readings of the intended message of an episode of Cagriey and Lacey that dealt with the abortion issue. We would not want to overstate this distinction between qualitative and quantitative sampling. Although there are examples where qualitative researchers have no concerns whatsoever about drawing wider inferences from their research subjects, in many instances interpretive studies are interested in drawing wider conclusions, a process described by Carey (1975:190) as 'gingerly reaching out to the full relations within a culture or a ■total way of life'. The key difference is that within the qualitative tradition, samples tend to be seen as illustrative of broader social and cultural ■ processes, rather than strictly and generally representative. If there is one thing that does unite qualitative and quantitative research on the issue of sample size, it is that there are no definitive guidelines. In most cases the final decision will be a compromise between the minimal theoretical and empirical requirements of the study and other external considerations (such as the time and resources available to the researcher). Non-response Non-response is a term that covers a variety of scenarios. Sometimes it relates to the refusal of respondents to co-operate with research because of their hostility, suspicion, apathy or confusion. Where non-response is deliberate, non-cooperation can either be overt (e.g. completely refusing to participate) or covert (e.g. choosing a 'don't know' category to answer a 1*1—ji /-\f\\-n11"4vj >i_wi'ij'JU1NI^/\ 1 jljfw^ question to avoid revealing their real views). The term can also apply to those occasions where a researcher has failed to record responses accurately. Non-response can obviously seriously undermine the representativeness or illustrative value of a sample. The main concern is that respondents and non-respondents may differ from each other in some important respect. This issue is particularly visible with random sampling because these methods give a precise statistical indication of non-response, and once a sample has been selected the researcher has no choice but to stick with it and do their best to achieve a high response rate (through call-backs, re~mailings, etc.). Eventually they are expected to indicate their success or failure in gaining a high level of response, which could be the basis for congratulation or embarrassment. In contrast, non-random sampling techniques permit researchers to look elsewhere if any of their original selection refuse to co-operate, which speeds things up and normally ensures an adequate quantity of response. However, this can present its own dangers, as there is no way of knowing whether those who refused to co-operate differed in some important respect from those who co-operated. At the very least, response rates can alert the researcher to potential deficiencies in the sample composition. Because of the dangers of there being significant differences between respondents and non-respondents, it is essential to try to maximise response levels. Where non-response is due to researcher error, this can be controlled by taking time and care when either recording, coding or entering findings to minimise data loss. Where non-response relates to the omissions or recalcitrance of respondents, as we see in later chapters, there are several important ways in which these absences can be limited by effective research design and administration. Even so, researchers are not in ultimate control of this matter, and consequently need to be sensitive and honest when high levels of non-response threaten the validity of their sample. Random sampling There are several forms of random sampling, but all of them involve consideration of two issues: defining a population and identifying a sampling frame. Defining a population As we have noted, the population of a piece of research is never constant; it is defined by research objectives. Defining a population provides a basis for deciding upon an adequate and appropriate sampling strategy and signals how broadly the findings can be extrapolated. This last point is important in that it helps those reading the research to appraise the validity of research conclusions. Say an Australian market researcher interviewed a random sample of 1000 adults about their newspaper reading, all from an affluent selecting and sampling 4j3 suburb of Melbourne. If the researcher defined his population as being 'the adult population of Australia' and drew inferences about national press readership on the basis of his results, the validity of his sample could be criticised on two grounds. First, Australia's federal political structure is mirrored in its highly regionalised press, which means the research would greatly underestimate the readership of titles produced outside of the state of Victoria. Second, the targeting of one affluent, suburban region would lead to the under-representation of certain sections of the Australian population (e.g. ■working-class and certain ethnic-minority communities). This in turn would distort the patterns of readership for particular titles. However, if the researcher more modestly defined his research population as being 'middle-class adults in Melbourne', the first criticism would disappear, and concerns regarding the second would reduce considerably. Sampling frames A sampling frame is a list that should contain all (or most) of the 'elements' of the population you wish to sample. The identification of a sampling frame is an obvious area where constant errors can intrude into the sampling process and compromise the representativeness of the research. This is because there may be a discrepancy between the working population of a study (the sampling frame) and the general population (Smith 1975: 107). For example, it is widely accepted that telephone directories make very unsatisfactory sampling frames for surveying adult populations, as not every household has a telephone, and not every household that does, consents to be listed in the directory (see Traugott and Lavrakas 1996: 59-60). Consequently, the directories tend to under-represent people at the top and bottom ends of the socio-economic scale. Even electoral registers are known to contain significant areas of under-representation, particularly among younger age groups and certain ethnic-minority communities (see Arber 1993: 81). Sometimes you have to accept that your sampling frame may not completely capture your research population, but you should always be alert to the implications of any obvious and serious discrepancies. In many cases, you may have to construct your own sampling frame from a range of sources because a suitable list for your population does not conveniently exist or available lists may be insufficiently comprehensive. ■For example, in a survey investigating the information and communication needs of British charities and voluntary organisations, two of the authors collated and cross-referenced 18 separate directories and local-authority grant lists (Deacon and Golding 1991) because the existing purpose-specific directories of voluntary organisations tended to under-represent ethnic-minority groups, more informal community-based groups and recently established groups. When compiling a sampling frame in this way, it is vital to remember to remove duplications in entries, as random-sampling procedures assume that every element in the sampling frame has an equal 46 researching communications selecting and sampling 47 chance of being selected. Once you have identified or compiled an adequate and appropriate sampling frame, you are in a position to start selecting your sample from it. Let us now review these procedures. Simple random sampling Simple random sampling is where each sample element is selected on a completely random basis from the sampling frame. This involves assigning each element on the sampling frame a unique number and then randomly selecting numbers between the top and bottom value, until you have the requisite number of elements for your sample. Traditionally, tables of random numbers provided in most statistics textbooks are used to guarantee a truly random selection. More recently, computer packages have been developed that can provide a randomised selection more quickly. However, despite these technological innovations and the general simplicity of the procedure, this method can prove very time-consuming when selecting a large sample. Systematic sampling Systematic sampling provides a less laborious method for random selection of sample units. You start by numbering the elements in your sampling frame, and then decide how many elements you need for your sample. Next you divide your required sample number into the sampling-frame total, which gives you a 'sampling interval'. A random number is then selected between 1 and this value, which gives the first element of your sample and the starting point for the selection of the rest. From this point you select every nth entry on the sampling frame (using the sampling interval) until you have completed your selection. A worked example of this process is provided in Box 3.1. One point you need to ensure when applying this strategy is that your selection procedure does not inadvertently tie in with patterns in the sampling frame. To give a simple illustration: if a sampling frame alternately Box 3.1 An example of systematic sampling In a survey of young children's attitudes to children's television programmes, a research team obtains the class registers from 25 junior schools. In total, these list the names of 2500 children, from which the researchers want to draw a sample of 500. Step I: Divide 500 into 2500. This produces a sampling interval of 5. Step 2: Select a random number between I and 5 (3). Step 3: Take the 3rd entry on the sampling frame as the first unit of the sample, then select the 8th, 13th, 18th, 23rd, 28th, 33rd, and so on, until 500 individuals have been selected. ■ listed females and males and the sampling interval was an even number, ■ the resulting sample would be made up solely of either females or males. Stratified random sampling " , Stratified random sampling involves separating the research population into distinct, non-overlapping groups (or 'strata'), each containing subjects ;■' that share similar characteristics. Sample elements are then randomly, and separately, selected from each stratum using systematic sampling tech- f niques. The main advantage of this method compared with simple random sampling and systematic sampling is that it allows you to ensure that the ;. ■ sample composition is representative in relation to important variables ; related to the research. For example, if you were investigating gender differ- ences in soap-opera viewing, you would probably want to ensure an equal divide of female and male respondents for the purposes of comparison. If you employed either of the basic random-sampling techniques, you might ;' not achieve such parity, particularly if your sample was small. However, if ; you stratified your sample selection by gender, sample equivalence in terms ; of this important variable would be guaranteed. ; ■ Most stratified samples are organised in such a way that the proportion .' of sample elements in each stratum matches known distributions in the i' population as a whole (known as 'proportionate' stratified random sam- ;; pling - see Box 3.2 for an example). But stratified samples may be deliber- t ately 'non-proportionate' in their composition (i.e. the proportions of the '<■- strata do not directly correspond to known distributions in the population) J if a researcher has a particular interest in strata that would contain very few sample elements if proportionality were strictly observed. For example, if f' somebody conducted a sample survey of 1000 UK adults to examine differ-.; ences in leisure pursuits among people from different ethnic communities, '. the total number of participants not defined as 'White' would not exceed 50 ;: if the sample were proportionate, because people from ethnic-minority ' communities constitute only 5 per cent of the UK population. Such a low proportional presence of representatives from ethnic-minority communities would undermine the key objectives of the research, so the researcher would probably seek to boost the presence of these communities in the > research sample. Of course, when a sample is deliberately distorted in this way, any projections made regarding the population as a whole require arithmetical corrections, with data being appropriately re-weighted in line with known population distributions. '-\ Stratified random sampling is a popular sampling technique because of 1 its cost-effectiveness and the control it provides to the researcher. But it is not always possible to apply. On some occasions the information contained I in the sampling frame is insufficiently detailed to permit the accurate sort- ing of its contents into different strata. For instance, you may not be able to ascertain the gender of people listed on a sampling frame because only sur-' name and initials are provided. 48 researching communications Box 3.2 Producing a proportionate stratified random sampling A research student in the United States wants to investigate the media strategies and relations of locally elected public officials. As part of this study, she wants to send a questionnaire to a random sample of 500 officials, stratified by gender and type of government (County, Municipal, Town/Township). To produce a proportionate stratified sample, she first needs to identify the known distribution of this population in relation to these variables: Distribution of locally elected officials by gender and type of government County Municipal Town/ Township Male IS percent 34 percent 27 percent Female 4 percent 10 percent 10 percent (Source: US Bureau of the Census (1997) Statistical Abstract for the US: p. 218. Notes: percentages add up to 100. Total number of cases: 281 636.) She now needs to distribute the 500 sample units in proportions that directly replicate these population distributions. For example, she needs to include 75 male County officials in her survey, which represents 15 per cent of a sample of 500. County Municipal Town/ Township Male 75 officials 170 officials 135 officials Female 20 officials 50 officials 50 officials Cluster sampling One of the major drawbacks of both random-sampling strategies discussed so far is that they present difficulties when researchers are investigating geographically dispersed populations. Say a student wanted to conduct a personal interview survey with a random selection of national and local journalists in India. The first major problem she would confront would be to produce a comprehensive nationwide sampling frame. No centralised register of these professionals exists, and according to one recent estimate India's print media alone exceed 24 800 newspapers and magazines (Chapman et al. 1997: 19). Even assuming she had the time and patience to compile an adequate sampling frame, she would face a considerable amount of travelling to complete all the interviews if she randomly selected her sample on a nationwide basis. She could reduce her workload by randomly selecting several regions of India and focusing her sampling on these areas. This would considerably reduce the logistics involved in creating a sampling frame, and would mean a lot less travelling. It is an example of what is known as cluster sampling. Although the 'clusters' in cluster sampling are most typically institutions or other physical locations, 'time' is occasionally used as an additional form ;' of clustering. For example, a sample of cinema-goers might be compiled by i randomly selecting people attending a random selection of cinemas at ran- domly selected times. But it is important to emphasise that a principle of I genuine randomness must be retained in sample selection. As Schofield 1' (1996: 34) explains: For a genuine probability sample, both the time periods, or any other form of cluster, and the individuals surveyed should be chosen at random. Simply accepting all the individuals who turn up or pass by at some specified time or times until the required number has been obtained would not constitute cluster sampling which is a probability method. Although the main advantages of cluster sampling are that it saves time and can be used when a sampling frame listing population elements is not available and would not be feasible to create, it does have deficiencies. The main one is that it reduces the precision of the sampling and increases the calculated standard error of the sample (for an explanation of how the calculation of sample error differs for cluster samples in comparison with other random samples, see Henry 1990:107-9). In broad terms, this is because ele-ments within particular clusters often tend to be alike and consequently there is a greater risk that the sample may be less truly representative of the population as a whole. For this reason, the greater the clustering in a sample, the less confidence we can have in its general representativeness. To illustrate this point, let us imagine an international survey of trade unionists' attitudes towards the mainstream news media, based on a cluster sample of two unions, one from the US and one from the UK. As Manning (1998) demonstrates, there are considerable national and international variations in the disposition of unions towards the media, from those who see journalists as class enemies to those who are generally optimistic about their union's chances of getting a good press. It is questionable whether sampling <:< two clusters would adequately capture this diversity of opinion, even if the survey sent questionnaires to hundreds of members from each union. It is distinctly possible that the political environment and history of each union (which are the 'clusters' of the sample) would produce very distinctive atti-tudinal cultures within them that are atypical of the union movement as a whole. A more reliable strategy would be to sample the same number of respondents but from a wider range of trade unions. As Moser and Kalton explain, 'a large number of small clusters is better - other things being equal - than a small number of large clusters' (1971: 105). This example also highlights how it can be useful to introduce formal stratification into your cluster sampling: considering at the outset how your clusters may vary and building these differences into the sample selection process. For example, a recent survey of social scientists about their media contact was based on a combination of cluster and stratified sampling 50 researching communications selecting and sampling 57 (Fenton el al. 1998: 93). The clusters in this sample were the specific organisational units within which social scientists are employed, which were stratified in three ways: by type of institution (university department, independent research institute, government department); by social-science discipline ('sociology', 'psychology", 'economies', 'political science', 'business and management', 'social policy' and 'other social science orientated'); and, for the university departments, by externally accredited research performance (high, medium, low). This complex stratification was deemed necessary to capture the varied contexts within which social scientists work in the UK, that may have significant implications for their media relations. This combination of a range of stratification variables is an example of what is known as multi-stage cluster sampling. Non-random sampling The one element that all non-random sampling methods share is that sample selection is not determined by chance. It is important to emphasise that 'non-randomness' in this context is not meant negatively - i.e. that the researcher tried but failed to achieve true randomness. For this reason, this type of sampling is sometimes referred to as 'judgemental' or 'purposive' sampling, terms that stress the conscious and deliberate intentions of those who apply the procedures. Although non-random sampling is most commonly a feature of qualitative research, it is also sometimes used in quantitative research. This most commonly occurs with 'quota-sampling' methods. Quota sampling Quota sampling shares some similarities with stratified random sampling and multi-stage random sampling, in that researchers first need to clearly define their population and gain detailed information about it. However, quota sampling does not require a sampling frame. Instead, the researcher decides on a range of criteria that are likely to be important to the study and then sets a series of 'quotas' in relation to them that are filled to produce a representative sample. As with proportionate stratified random sampling, the size of each quota should be weighted to match known distributions in the population. The more selection criteria that are identified, the greater the number of quotas will be (see the example given in Box 3.3). This increases the logistical problems in filling each. However, the more sophisticated and multi-layered the quota categories are, the greater confidence you can have in a sample's representativeness. Quota samples are widely favoured in research where speed is essential, for example in opinion-poll research about developing events and in market research. Apart from the fact that they do not need a sampling frame, they do not require call-backs to locate people who were not initially contacted, and the samples are not compromised by low response (you keep going until your quotas are full). But this presents various ways in which 'constant errors' can creep into the sample. The technique can produce bunching in quota categories rather than an even spread, because interviewers approach people who most evidently fit into them and neglect people at the margins (e.g. in looking for respondents between 21 and 30, the interviewers may produce a sample with a high proportion of people in their mid-twenties, because they are the most readily identified as fitting into the category). Furthermore, the time and location at which the sampling takes place can affect the sample's representativeness. If you quota sampled in a city centre in the mid-afternoon you might marginalise people who work in certain professions or who are resident outside the city.1 Many researchers who employ quota-sampling techniques also conduct the kinds of statistical tests and population estimates that, strictly speaking, should be the preserve of randomly selected samples (see Chapter 5, pp. 98-9). Their rationale for doing so is that a well-designed quota sample will be al least as representative as a randomised sample, and it is therefore legitimate to use them for making statistical inferences. This pragmatic reasoning, which is most frequently advanced by market and opinion researchers who value the cost-effectiveness and ease of administration of the method, is not accepted by statistical purists. They argue that the non-randomness of the sample selection means it is inappropriate to make statistical projections that ■are based on theories of probability and chance (for a discussion of the controversy see Moser and Kalton 1971:127-37). Despite these disagreements about the true 'scientific' status of quota sampling, we can see that this method shares the motivations of all forms of random sampling: to produce a representative sample from which to make broader inferences. Advocates of the method claim it is merely a different means to the same end, and reject the argument that the intervention of human subjectivity in the selection process inevitably compro-■ mises sample accuracy. These formal, representative concerns make quota : sampling atypical of most non-random sampling. With most other judgemental sampling methods the intentions of the researcher are transparent, unapologetic and of pivotal significance, which reflect the different theoretical and empirical concerns of the mainly qualitative studies that use them as their basis. :. 1. Quota sampling was identified as one of the reasons behind the spectacular : failure of British opinion pollsters to predict the victory of the Conservative Party in the 1992 British general election. First, because most samples were collated during the day, when large numbers of people are at work, certain sorts of professional people were under-represented who had a greater propensity to support the ■' Conservatives than Labour. Second, because quota sampling does not quantify non-responses the method obscured 'the disproportionate probability for Conservative voters to refuse interviews to pollsters' (Noble 1992:18). selecting and sampling 53 Box 3.3 Designing a quota sample A quota sample of schoolchildren incorporates three variables: • gender (female/male) • age group (5-10 years, 11-15 years, 16+ years) • parental occupation (professional/intermediate, skilled manual/non-manual, partly skilled/unskilled, unemployed) This means the researcher ha s to find respondents to fit into 24 quota categories: (Gender) (Age) (Parental profession) Female 5-10 years 1 Professional/Intermediate 2 Skilled Manual/Non-Manual 3 Partly Skilled/Unskilled 4 Unemployed 11-15 years 5 6 Professional/Intermediate Skilled Manual/Non-Manual 7 8 Partly Skilled/Unskilled Unemployed 1 6+ years 9 Professional/Intermediate 10 1 1 Skilled Manual/Non-Manual Partly Skilled/Unskilled 12 Unemployed Male 5-10 years 13 Professional/Intermediate 14 Skilled Manual/Non-Manual 15 Partly Skilled/Unskilled 1 6 Unemployed I 1-15 years 17 Professional/Intermediate 18 Skilled Manual/Non-Manual 19 Partly Skilled/Unskilled 20 Unemployed 1 6+ years 21 Professional/Intermediate 22 Skilled Manual/Non-Manual 23 Partly Skilled/Unskilled 24 Unemployed Theoretical sampling Theoretical sampling is a method that abandons concerns about representativeness (Glaser and Strauss 1967). Instead of looking for typical cases, the researcher deliberately seeks out respondents who are most likely to aid theoretical development by extending and even confounding emerging hypotheses. This search continues until respondents start to reiterate issues that have already emerged (known as 'saturation point'). To give a hypothetical example, say you wanted to use theoretical sampling as the basis for an exploration of journalistic attitudes towards the British royal family. To ■do so would involve first theorising the main points of diversity across the British media and then compiling a sample that captures all elements of these differences (e.g. press/broadcast, 'high-brow'/Tow-brow', generalist correspondents/specialist correspondents, news gatherers/news processors, entertainment oriented/news oriented, etc.). These distinctions may be added to or elaborated as the research progresses and new issues emerge. Snowball sampling ■Snowball sampling is not completely distinct from theoretical sampling, as theoretical samples are often derived from snowball-sampling techniques. ■Nevertheless, there is a value in retaining a distinction as snowball sampling is often adopted for practical reasons rather than because of clearly identified theoretical objectives. Snowball sampling is mainly used where no list or institution exists that could be used as the basis for sampling. Like a snowball rolling down a hill, a snowball sample grows through momentum: initial contacts suggest further people for the researcher to approach, who in turn may provide further contacts. This method is consistently used in research into either very closed or informal social groupings, where the social knowledge and personal recommendations of the initial contacts are invaluable in opening up and map- : ping tight social networks. Typical-case sampling With typical-case sampling the researcher seeks to identify a case that exemplifies the key features of a phenomenon being investigated. The method needs to be supported by other, more generalised sampling evidence to support the claims to typicality. For example, a researcher might want to contrast the media usage of a typical middle-class Swedish family .with that of a typical middle-class Norwegian family. To do so in a credible way would involve consulting formal demographic data (details about average family size, ages, occupations, education, ethnicity, etc.) to establish what typicality might mean in each context. Critical-case sampling Lindlof describes critical-case sampling as 'a person, event, activity, setting, or (less often) time period that displays the credible, dramatic properties of a "test case" ... [A] critical case should demonstrate a claim so strikingly that it will have implications for other, less unusual, cases' (1995: 130). Critical-case sampling is more widespread than you might suppose, although it is not always formally conceptualised as such. For example, many studies of relations between journalists and the state during military conflict could be described as 'critical-case samples', as they often use the overt tensions during these periods to identify nascent aspects of political and professional culture. Witness the concluding remarks from two separate studies of the media's role during recent conflicts involving western military forces: The Falklands crisis had one unique and beneficial side effect. Its limited time-scale and crowded succession of incidents made it an experience of great intensity. It briefly illuminated aspects of British society normally hidden from view. It exposed habitual abuses by the armed forces, . Government, Whitehall and the media; it did not create them. (Harris 1983: 152) The Gulf war case . . . reveals the clash between the mythologies of jour- \ nalists and politicians in American culture, mythologies that establish norms and roles that are more or less carried out in practice. (Paletz 1994; 291) Convenience sampling Despite the differences between the qualitative-sampling procedures listed above, one aspect shared by them all is that selection of sample units is consciously shaped by the research agenda. Convenience sampling differs in that sample selection is less preconceived and directed, more the product of expediency, chance and opportunity than of deliberate intent. It is useful to think of there being two types of convenience sampling: a weak version and a strong version. 'Weak' convenience sampling is the least desirable form and is where sample units or clusters are selected simply because they are nearest to hand. An example would be the university professor who uses her students as research subjects, or the undergraduate student who dragoons friends, neighbours and family into participating in his final-year project. The 'strong' version of convenience sampling a is where sampling focuses around natural clusters of social groups and | individuals, who seem to present unexpected but potentially interesting opportunities for research. For example, a researcher might suddenly find she can gain access to members of a religious sect who make extensive use | of the Internet to promote their beliefs. On these occasions, it is the chance I availability of these 'natural outcroppings of data' that initiates the research process. 1 Focus-group sampling Focus-group research involves bringing small groups of people together to discuss issues identified by researchers. It may seem strange to include a section dedicated to focus-group sampling in a general discussion of non-random sampling methods, first because there is no consistency in sampling procedures used in focus-group research and second, because the various sampling methods used are often hybrids of existing sampling strategies reviewed above. Nevertheless, we believe a'dedicated section is required for several reasons. In the first place, focus-group research is becoming an ever more popular qualitative research method within communication and cultural studies (a popularity mirrored in its growing salience in market research and politics (Wring 1998)). Furthermore, examining specific sampling strategies used in focus-group research demonstrates how qualitative sampling strategies are rarely straightforward matters involving well-established sampling protocols. Rather, they often depend upon the creativity and resourcefulness of the researcher. Although the use of focus groups in communication research has a long history (e.g. Merton 1956), it is since the early 1980s that they have become one of the most popular means for analysing media audiences. In particular, focus groups have become closely associated with the reception analysis paradigm, described by McQuail as 'effectively the audience research arm of cultural studies' (1997: 19). This diffuse body of work has sought to introduce an 'ethnography of reading' (Morley 1980) into audience research; that highlights the social context of media consumption and the agency and discernment of audience members in the decoding process (for a review see Moores 1993). Focus groups have proved popular in this area because they are seen to produce rich qualitative material well suited to detailed interpretive analysis (transcripts of people discussing their views and actions in their own words and, to some degree, on their own terms). Furthermore, their group basis is claimed to provide insight into the interactional dynamics of small groups (May 1993: 95) and to mimic the way that everyday media interpretations tend to be 'collectively constructed' (Richardson and Corner 1986) by people in social, familial and professional networks.2 So, how do you go about designing a focus-group sample that is sufficiently varied to enable you to capture and compare the social and individual constructions of meaning? As we show in the examples below, there is no consensus in the methods adopted in the myriad studies published over recent years. The first question you need to deal with is which groups should you select? In some research the selection is directed by the research topic, and 2. It should be noted in passing that neither of these claims made for focus-group research is uncontested. Some have disputed whether the material generated "through group discussions can claim to be truly 'ethnographic' (Nightingale 1989; Murdock 1997) and others reject the assertion that individual-based interviews treat ■people as social atoms divorced from social context (Wren-Lewis 1983; Jordin and Brunt 1988). How group interviews relate to broader social relations and dynamics remains, empirically and theoretically, a complex issue. We should also note that not : .all reception analysis studies depend on focus groups to gather their data. For example, Ang's (1985) study of Dutch viewers of Dallas used letters sent to her by fans of the show. 56 researching communications the researchers focus on groups that are assumed to have strong and contrasting interests on the issue. Let us offer a concrete example. If you were concerned with analysing the 'gendered' reception and evaluation of media texts, and if you wanted to focus on cases where the reception process is itself generically associated with a specific gender of consumer, the focus groups convened are likely to involve either exclusively male or exclusively female members. This is because research has shown that men tend to dominate conversations and have different conceptions of the public-private divide from women (see Kramarae 1981; Fishman 1990; Tannen 1990; Cameron 1995). However, mixed-gender groups could be chosen if you wanted to explore the ways in which the actual co-presence of people of the opposite gender affects media reception and response. On other occasions, the researcher may simply seek to select a widely stratified range of groups according to a range of social, cultural and economic factors. In many instances, group selection combines both of these considerations. Selection criteria have proved controversial among some reception analysts because of concerns that the design of the selection process may inadvertently shape the nature of the conclusions reached. According to Wren-Lewis (1983), this whole process involves prejudging what the pertinent variables are behind decoding, which puts the cart before the horse. In his view a more appropriate strategy would be to deduce the salient social variables 'after the fact', once you have looked carefully at how individuals have responded of their own volition. However, such a strategy effectively rules out the use of focus groups and requires a complete reliance on individualised interviewing. Another sampling issue with focus groups is whether you should use social and professional groups that already exist (preconstituted groups), or create your own for research purposes (researcher-constituted groups). The advantage of preconstituted groups is that they are more natural and participants may be comfortable in each other's company (Philo 1990: 223). The main advantage of researcher-constituted groups is that they give you greater control over the composition of the sample. Size is another issue that often arises in relation to focus-group research. First, how many groups should you include in your study? The answer normally depends upon your ultimate aims in conducting the research. If you are interested in 'going "wider" in analysis, embracing a broader range of variables and attempting to engage with these as far as possible as;they occur in the settings of "everyday life"' (Corner 1996: 299), you are likely to need quite a few. However, if you are interested in focusing closely around a particular issue or social group, 'to engage quite tightly with the interface of signification and comprehension' (ibid.), then the numbers required will normally be less. Additionally, you need to consider how many participants there should be in each group. In most cases, you would want to keep the selecting and sampling 57 numbers down, particularly when the groups are researcher-constituted and you need to minimise nervousness. However, you also want a sufficient number of people to stimulate exchanges and debate. To strike a balance between these factors, the most common number of participants per group is between 5 and 10. In Table 3.2 we summarise the sample strategies and design used in three recent reception studies, to illustrate how these and other issues related to sampling have been tackled in focus-group research. The first is Schlesinger et al.'s (1992) Women Viewing Violence, which examined women's reactions to the representation of violent acts against women in selected films and programmes. The second is Corner et al.'s (1990) Nuclear Reactions, which analysed the responses of people from different political and social 'interest groups' to documentary, PR and campaign material concerning: the issue of nuclear power. The final study is Philo's (1996) Media and Mental Distress, which explored public perceptions of mental illness and its: coverage in the media. There are several things worth noting from this comparison. The first is the different ways in which the research agenda of each affected the selection of groups. In Women Viewing Violence and Nuclear Reactions the groups were selected at least in part because of their proximity to the topic being ■ investigated (women who had direct experience of sexual or domestic violence and people who had either worked for or campaigned against the nuclear industry). In contrast, the selection of focus groups in Media and Mental Distress was more independent of the research topic. Instead, the groups were selected to match broadly socio-economic variation across the West of Scotland region. Of course, social and economic stratification are also present in the first two studies' samples, but these factors co-exist with research-driven criteria selection. We can also note the different ways each study built social stratification into its sampling. The Women Viewing Violence study closely approximates the procedures involved in quota sampling (Schlesinger et al. 1992: 26), albeit not precisely. Not every variation that would occur when linking four sampling factors was covered3 and the number of participants in each group was not weighted to mirror actual distributions in the popula-Uon. Significantly, this is also the only study that was more or less solely 3. Among the groups representing 'women with experience of violence' no distinction was drawn in relation to social class. Social-class distinctions were also not made for the ethnic-minority groups selected to represent 'women with no experience of violence'. Finally, there were no 'Afro-Caribbean women' groups selected for Scotland. These omissions were due to the extreme, possibly insurmountable, logistical difficulties that would have been created in attempting to cover all 32 possible quota-categories. As it was, '[ejfforts to form the fourteen viewing groups proved to be one of the most time-consuming and difficult aspects of the research' (Schlesinger etal 1992: 25). Project Authors Research issue Number of focus groups/participants Average group size {figure rounded) Focus-group origins Media texts focused upon Women Viewing Violence Schlesinger, P„ Dobash, R.E. Dobash, R.P. and Weaver, C. 'Man/ women live lives in which they are subjected to physical and sexual abuse by their male partners or face the risk of such abuse by strangers, and most women watch members of their sex being similarly abused, at times, on television. What do they think about this? And are those reactions different for women who have actually lived through the real experience of violence than for those who have not?' (p. I) • 14 groups • 91 participants (all women) Researcher-constituted • Crimewatch UK (BBC I documentaiy programme that includes dramatised reconstructions of actual crimes, The episode chosen included a section on the murder of a woman.) • EostEnders (BBC I TV soap opera, The episode chosen included scenes of a man being violent towards one of the central female characters.) • Closing Ranks (ITV TV police drama. A dramatised account of the covering up of domestic violence committed by a male ■ police officer) • The Accused (Hollywood film. Examines the group rape of a woman and the trial and and prosecution of the men responsible) Nudear Reactions Comer, J., Richardson, K. and Fenton, N. 'jTjo explore some of the ways in which television, and then viewers, 'made sense' of the nuclear energy issue during a period when public awareness of the topic had dramatically increased' (p, I) 12 groups" 65 participants* (gender mix) i Preconstituted':t/4 Researcher-constituted « Uncertain Legacy (BBC2 documentary exploring the health and waste disposal issues related to the nuclear industiy.) i from Our Own Correspondent (dramatisation in documentary form produced by anti-nuclear activists highlighting the consequences of a radiation leak in the UK) ■ Energy - the Nudear Option (promotional video produced by the nuclear industry) ■ A Life or a Living? (BBC documentary examining incidences of child leukaemias near nuclear power stations) Media and Mental Distress Philo, G., Crepaz-Keay, D„ Henderson, L McLaughlin, G., Piatt, S. and Seeker, J. [To] trace the processes by which key messages [about mental distress] are received, and focus specifically on the conditions under which they are believed, rejected or reinterpreted. We will examine the role of key variables such as persona! experience or cultural history and show how these can condition different responses across a variety of audience groups' (p. 82) 6 groups 64 participants (gender mix) Preconstituted Group participants were asked to 're-script' media content (dialogue and editorial details) from: • A scene from Coronation Street (ITV soap opera) depicting the stalking of two central characters by a mentally ill person. • A series of newspaper reports dealing with the violence and instability of mentally distressed people. Project Factors considered in the stratification of groups Nature of stratification Other issues Women Viewing Violence • Experience of sexual/domestic violence (yes/no) • Geographic location (Scotland/ England) • Ethnicity (White, Asian, Afro-Caribbean); • Class (middle-class, working-class). Highly structured- The organisation of the groups closely resembled the procedures used in designing quota sample categories, However, sample recruitment difficulties in gaining access to a sufficient number of women who had experienced violence and from some ethnic communities, meant that quota-sampling logic' could not be completely applied, (If it had been, the inclusion of the four stratification variables would have generated 32 groups.) A market-research company was used to recruit the groups of women who had not experienced violence. The groups of women who had experienced violence were recruited via the researchers' personal contacts with women's aid organisations. Nuclear Reactions • Party-political orientation (Labour/ Conservative/SLD) • Proximity to the nuclear power issue (through campaigning or employment) » Social class • Professional status • Gender. Relatively unstructured. The sample selection blends the identification of 'interest groups' likely to have very firm opinions about the nuclear industry (e.g. workers employed in the industiy and Friends of the Earth campaigner's) with preconstituted groups that broadly and collectively arraign across the stratification variables identified above (e.g. groups from the local Rotary Club, a women's discussion group, unemployed people from a trade-union resource centre). The inclusion of four 'researcher-constituted' groups was due to the screening of the A Life or a Living programme as the research was under way. The team felt it was such an interesting example of the mediation of the nuclear energy issue that they extended their sarnpling of members of the public to explore people's responses to it. Media and Mental Distress • Class • Income levels • Gender - - ■- • Occupation Fairly structured. Although the study sought to analyse people in 'naturally occurring units' (p. 83), these groups were selected from 'randomly chosen' (ibid.) areas in the West of Scotland, stratified by income, occupation and housing types. The intention was to produce a sample that was 'broadly representative' of the West of Scotland region. However, the authors warn, 'Such a sample is not large enough to make generalisations about the whole population' (p. 82). These figures are estimates based on the limited sampling details made available in the book. Table 3.2 Comparison of sampling strategies in three focus-group audience studies 60 researching communications selecting and sampling 67 dependent on 'researcher-constituted' discussion groups,4 which allowed researchers to balance the composition of their groups precisely. There are also aspects of the sampling strategy that closely approximate 'snowball sampling' procedures: the groups of women who had experienced sexual or domestic violence were all approached via women's aid groups. In contrast, the Nuclear Reactions study stratified groups more informally, by sampling mainly preconstituted 'interest groups' defined in relation to several factors (known involvement in the nuclear energy debate, party political stance, professional status, etc.). In this respect the sample selection bears quite a resemblance to the strategies used in theoretical sampling that, as we have seen, strive to maximise variation. Such a comparison is further supported by the fact that the research team tagged on four further groups at the end, to explore responses to a programme that was broadcast as the research was in process. In further contrast, Media and Mental Distress used a sampling strategy that bears some resemblance to multi-stage cluster sampling, as group members were from 'naturally occurring' units living in randomly selected neighbourhood areas stratified by income indicators. • Examples of sampling strategies used in qualitative research were provided (theoretical sampling, snowball sampling, typical-case sampling, critical-case and convenience sampling). • The discussion of these other non-random sampling methods highlighted how sampling issues in more intensive, interpretive research tend to depart from formal concerns about sample representativeness, and are more concerned with the illustration of social processes and dynamics. • Finally, sampling issues involved in focus-group research were examined. Summary: key points • The distinction between samples and populations was explained and three general issues were identified that apply to all forms of sampling, (sample error, sample size and non-response). • The key differences between random sampling and non-random sampling were identified. The issues involved in defining a population and identifying a sampling frame were discussed. • The main forms of random sampling were set out, with examples given for each (simple random sampling, systematic random sampling, stratified random sampling, cluster sampling and multi-stage cluster sampling). • The main forms of non-random sampling were discussed, beginning with quota sampling. It was explained that this method was atypical of other forms of non-random sampling, in that it shared similar concerns to random-sampling procedures regarding sample representativeness. 4. The main exception to this came with the group of Scottish Asian women who had no experience of sexual or domestic violence. These participants effectively became a preconstituted group because they 'were only willing to participate alongside other members of their families, considering themselves safe if they were with other women with whom they felt familiar' (ibid.: 207).