I?h BíraooycnoN to research dľ*.;n CHAPTER 5 IDKNnFYING POPULATiOrCJ AND ťAMFUB 127 field i» a point of discussion, but il can lead to biun in (he sample thai may influence Ihc results. Samí1; ;\'c frrpks_________________________________________________________ Errors in (his domain are not mistakes, but inaccuracies in resulting data. Sampling errant occur simply because datu arc being collected on a sample and not on the population. The first four sampling techniques considered in the previous section were devised to enhance the probability that any sample would be representative of a population, thus endeavouring to minimize these errors. To illustrate this, consider the distribution of IQ scores for a small population of 3000 shown in curve (a) of Figure 5.3. Beneath this is the distribution for a single sample of 40 subjects, curve (b). Note that ils mean, .vA, is not exactly that of the population, u, nor would wc necessarily expect II to be. due to natural random variability. One way to quantify the collective errors attributable to sampling for interval or ratio data is to consider what would happen if wc were to take sample after sample from a population. For each sample, the" mean could be found and plotted on a graph, providing us with a distribution of sampl* imam. It is expected that for representative samples, though, ihcso' means would be very close to that of the population menn. It is also assumed (hat since this sampling error would be random, the distribution of sampling means would be normal with the mean of the «mple means being the population mean, as in the narrow peak shown in curve (c> of Figure 5.3. This natural variability in a set of sample means of scores/measures around the population mean provides an opportunity to describe the outcome mathematically. Hays (1994) provides a rigorous discussion of the calculation of sampling errors, probabilities and the variance of this distribution, with respect to sampling for survey«, for those who are interested. The derivations given, as well as justification for them, do generate a commonly used and conceptually useful estimation. Wc use an estimate of the distribution standard deviation simply because U is not reasonable to take all the possible samples from a popu^ lalion ami plot a graph of tlie means. Usually described us the standard error of ilie mean the ■distribution of i . ■ 11 :■> n means for IHM 40 *| 7! [5 where iris the standard deviation of the population and« is the size of the sam-, pic;. Its dependence on the size of the samples can he seen from the equation note that the larger the samples, the smaller the variability of the set of. sample means. This estimate is based on the assumption that these are simple, rudoB unplM Otiw mm—hum prodm jnitorinoi tod consequent!) a higher value for the standard error of the mean than the one provided by equation (5.1). Using this value, wc can consider some interesting Issues. In situations where, the population mean is known, it would be possible to determine whether, on. the basis of the mean of a sample of size n, it is likely that a particular sample is representative of the population, in other word*, whether its mean is within (, = 102 :: - acceptable limits. Obviously, what is 'acceptable' could bo an arguable point, but usually il is wwumed (as with many statistical tests) that a mean which falls within the range of 95% of all possible means in the normal distribution is representative. When we look at a normal distribution, il U necessary to remember that (he area under the curve represents (he number of samples. Assuming thai the toul area represents 100% of all pouthle samples, then we can identify various percentages. Tbc easiest way to do this is to use the number of standard deviations away from the mean as the .v-axis, referred to as a í-score. Thus for a given mean score for o sample and the population mean, wc can find a corresponding r-scorc, _ *-q "t where X Is the sample mean and as is standard error of the mean. Looking at the tahle showing the areas beyond any given z-score (Tabic H, I in Appendix B), wc find that 2.5% of I he area is beyond 1.96 standard deviations from the mean. Thus, any mean producing a j-score greater than 1.96, is unlikely to be pjrt of the population, as shown in Figure 5.4. This would allow us to reject any null hypothesis that (here was no significant difference between the population and sample means. Therefore, ihcrc is a diUciencc between the sample and population menus greater than what could be attributed (O natural variability. The natural expectation is that not all samples will be exactly the same (some possible sources of the extra difference will be considered in the next section). If the difference U too great, it is said to be statistically significantly ditferent. 128' üNTKODUCTiON TO RESEARCH DESIGN because the probability dial it belongs lo (he population is so small. Even if the probability is unite (less (hun 5%), it is considered to be so unlikely lhal the sample is labelled unrepresentative. HOW does one determine litis mathematically lor a given sample? 1-cl us con sidcr ihc example of Harry Teacher, who wished to conduct u study on the elfte« livcncss of a new reading programme in primary school. lie wauled two' equivalent classes of children; be randomly selected two local schools and rim- domly selected a class in each. In order to try to justify some general iza hi I it y of Ihc study based on these two classes of Year 4 children, he decided to see what their mean IQs were and whether they were typical. He found from the teacher lhal one class of 30 had a mam IQ of řA ~ 104, and the other iD = 99. First he: had to find the standard error of the mean. Knowing that the population mean, for IQ scores was 100 with a standard deviation of 15, a os=—ř V" 15 as = 2.74 Tins provides a value for the standard error of the mean for simple random i pies of 30 children. Thus the 2-scorc for the two means would bo 99- 100 ZK -n 2.74 UM 100 ä""0"36 •7.1 ,M '■■ FIGURE S J Normal distribution of sample means with 5% significance levels, whfM m Is the population mean .tmi lliľ i-scores nie the means expressed ay number of standard deviations above und below the mean Ke)ecl i r U^sS^É i = -\M i*l.96 O tAPTEB 5 IDENTIFYING POľUl.ATIONS AND SAMn.ES 129 Since neither of these means was more (han 1.96 standard deviations from the population mean, he could assume that the samples were representative of the population, at least for IQ scores. This docs not protect him against other traits and experiences for which the two classes might not he typical, but it docs provide some support. Fowler (1993) points out that (his estimate of the standard error of the mean is only vnlid if one has taken a simple random sample. St ratified random samples tend to produce lower values for the standard error of the mean, while cluster samples lend lo have higher values than lor simple random samples. Thus, Harry Tcuchcr's analysis may produce an underestimate of the standard error of the mean, since his selection of clusters of students may produce a more homogeneous grouping for the trail than that for the population. In some situations, this may be an important factor and may actually influence the results, but here the aim is to test the relative representativeness of the sample. ■ If population parameters are not available, the standard error of the mean can be estimated using sample statistics, J A. where ,vA is the standard deviation of sample group A and nA is the sample size of sample group A. It does give one some indication of Ihc error, though strictly it also applies just to simple random sampling. Tins situation allows lis to make a different type of statement about the results: is the sample mean, A'A. close enough to the population mean, /tl Using (he estimate of the standard error of the mean in equation (5.2), it is possible to establish a confidence interval. an interval of scores in which we can be reasonably confident that the population mean will fall. For example, if we wish lo establish an interval in which we arc 95% confident (hat the population mean occurs, then it will be *A :1:1.96s, (5.3) For example, if a simple random sample of 25 childreu had a mean score on a school-based lest in mathematics, xA = 71.60, with a standard deviation, JA = 12.20, then 12.20 and therefore the 95% confidence interval for the population mean, u> would be 71.60 ± 1.96x2.44 or 71.60 ±4.78 Alternatively, it could be expressed as 66.82 < /* < 76.38 Tin's means tbat for 19 out of 20 samples, we could expect the true population mean to be within the interval. This is illustrated in Figure 5.5 as mean scores with their corresponding confidence iutcrvuls for 20 samples with W of (hem overlapping with the population mean, fi, while one does not. Obviously it would also be possible to calculate 99% confidence intervals as well, if needed. Examining ihc equation, it becomes apparent that the best way ,rt = ^ř = 2.44 IM INTV-XHXľnON TO NSMAUTH DBCN M------ , . ľ -f-■ . Ä, 1 1 . ř i —•■ *i Jr. *j ------a- • i -: : ■i —a- a-— *»- ►- "í.. ■ •*■ •*------ »II-*11 ; FKiUKU 5 J Twenty tumples * .'.1 : -; *» t 91- ■•* wilh 95% confidence Lni-iv.il1. and true doiit nul Include íb? : ------*" 10 reduce llic confidence interval would Iv l........m*.c ihe sample ii/e (in i thť ilennmitiutor und Um ovci Jill value ine leases) This is not I lie only con vidcia :ishIc[iI'Ic in Hoc i tec on pcpul.iln>ti (mute M«u,- ,»< male i mflios of (he standurd ci i in of die mean miiy I« lower for si i milled samples a higjier for cluster samples, Ihus altering llic coiilidcncc interval The importar of t hi», though, is dependent on the nature of the nifercncei to be made the populations. This is nil satisfactory us long as we are considering interval dulu, I hear yoffl uy> So kt us consider another example, one where the trait under consideration n nominal (a set of calcgonci) rather than interval or ratio. In a recent study í Ugandám which the itnlhor participated (Black nah, 1998), il wn> necessary tjj employ a sampling technique lhal had to ho seen as representing secondary^ ■dwell (ton noh Ol UM \9 districts in rl«<- lounti) ľhe saiupliin* li.niie coiV] leil isl of 550(OVef>nwnt'SUpptiHed Mcondiiry schools Rtoii|H-.l hydil tni i A simple random mím plo of all scl.....Is could have left Home sparscl populmed districts out of the study, so it was decided theic would he at ■ one achool randomly selected from each dislncl (stratum) and dislikts wil ovor (luce schools would he proportionally more randomly selected. Thus i 01AITER5 roBWnFYIKG POPULATIONS AND SAMPLES HI each district was represented by at least one school and largu districts had I larger representation, providing a sample of 77 schools (|4H of tbc Mttl) Having agreed on llm, it was necessary toimtify that ihe sample was slill trpir M'nlalivc ut schools in llic country for oilier impoilnnl trails: day versus board ing. separate sex versus mixed, and Inundation body (parents, religious group, etc ) To test the hypotheses that the sample was iqncscnlalivc of the whole population of schools for each of these Ihrce trails required (he use of a non-paiauielric test: chi-square. x • '"hi* involves compaiing the observed frequencies of occurrence in each category of the sample with what one would expect from national frequencies. Analogous to normal distributions of interval data for samples which did not match exactly with the population, we would not expect samples to have exactly the .same proportions of categories as the population. Again, the question must be asked, how much is loo much of a deviation'' Table 5.2 provides llic data, wilh the observed fieq neum«, *;,, (oi each type ul M'lmol foundation body in Ihe tirst column llic second column has the national percentages ami the third contains Ihe frequencies based on whit would l>e expected if the 77 schools were tXúfífy like the niilional diHtribution, the expected frequency, if,, Visual mspe.li......f the data doe* not show much dillW nice, hut the x'-lcsl provided an indication of the probability that the sample was one that was rcpicienlative of the whole population (We will come back to this test later In Chapter 19, and investigate it in gicalei depth ihcn.) Roughly speaking, the test sums the absolute differences in frequencies between tbe two groups for all the characteristics, then determining whether the total is more, or less, than what would he expected by chance. Since the differences between observed and expected frequencies could he either positive or negative, the differences are squared heroic being added together. Otherwise. the lotal could approach zero even when there were large differences, the negative cancelling the positive values. The formula for the x'-Hlatistic is quite simple, the sum of all the difference, amjared, each divided by tlie appnmiiaie expected frcqueacy. V V'V^ (M, where the O, arc the observed frequencies m the sample. Ihe K, arc the expected frequencies based upon population |>ci ccnlagcs, and í represents the categories I'umdjtlftii boil) ObMrnd Nalliiiiul Batpeelad 0,<»mrk) p«rrnitag* l\ Church of Uganda 13 42.1 it A Catholic Church ľ II | 16 i I'aieiils M 11.4 114 Malta School! (-"ini. il l 1 1 M < lovcrnt iicnl '■ 77 1.9 United Mudini School« i w 4.0 (Uher 4 n r, 1 4 v Totals IOQ.0% 77 " 84 9 INTRODUCTION TO RFSEAKQH DESIGN ranging from 1 to m. Note, when consulting tables for this statistic (sec, for example. Table B.9 in Appendix B), that it has df = m — 1 degrees of freedom. The easiest way to process such data is to add another column to the table, in which the differences squared divided by the expected frequency are placed. As we will see later, this is conveniently done on a spreadsheet, but for small" amounts of data, the calculation is easily carried out on a calculator. The ■ column values arc added as shown in Table 5.3, and the sum compared to the standard table for x'> Which value one uses depends on the number of categories minus one: in the example, there are seven foundation bodies, thus six degrees of freedom, since six frequencies could vary, but then the seventh would be fixed. Table B.9 in Appendix B provides a value of 14.07, thus a x"--> ratio greater than this would indicate a significant difference between thej sample and the population. In the example io Table 5.3, the final value is'" 0.74, nowhere near significant (p > 0.05): thus, at least for this trait, the-' sample of secondary schools could be said to reflect (he national pattern. Sampling errors are unavoidable, though hopefully minimized through sound' sampling procedures. On the other hand, non-sampling errors can be attributed. 10 such processes as incorrect sampling frame, poor measuring instruments,', incorrect data processing and non-response by subjects in the sample. The defi-nitron of sampling frames was considered earlier, and incorrect ones are cquiva-lent to defining the population to be all voters and using a telephone number list" as the sampling frame. The skills related to the design of measuring instruments-, and data collection are the subjects of Chapters 8-11, and the choice of statistical test is covered in Chapters 13-22. This leaves non-response, which is to be' considered in the next section with suggestions as how to minimize it and anyv effects thai it may have on the validity of the results. Before going on, carry out Activity 5.3.