NORUSlS. Marija J. 1998. Guido to Data Analysis. Upoer Saddle River. Prenttce Hall. Evaluating Results from Samples 165 Sjjgg What's a random sample? A random sample gives every member aiia of the population (animal, vegetable, mineral, oť whatever) the same chance oř being included in the sample. No particular type of creature or thing is systematica I ly excluded from the sample, and no particular type is more likely to be included than any other. Lach member is also selected independently; including one particular member doesn't alter the chance of including another. A sample is biased if, for example, rich people have a better chance of being included than poor people, or healthier people are more likely to be selected than sick people. You can't draw correct conclusions about: the population based on the results from such a sample. ■ a h Let's use the computer to solve the following problem. A Noted Physician claims that she has a better treatment for the Disease or Interest. Or 10 patients who received her new treatment, 70% were cured. Extensive literature on the topic indicates that nationwide, only 50% of patients with this disease are cured. Based on the results of her experiment, can you tell if the physician has really made inroads into the treatment of this disease? Are the Observed Results Unlikely? To evaluate the physician's claim, you have to ask yourself the question, Are the results she observed (7 out of 10 cures) unlikely if the true population cure rate is 50% ř You know that if half of all people with a disease can 1« cured, that doesn't mean chat any tune you select 10 patients, exactly 5 will be cured by the treatment. Consider a coin-tossing analogy. You know thai if a coin is fair, heads and tails arc equally likely. If you flip a fair coin 10 times, however, you don'r expect to see exactly 5 heads every 10 flips. Sometimes you get more heads and sometimes, more tails. {Try flipping a coin 10 times and see how many heads—cures—you get. Record your results. Repeat this as many times as you have the patience for and then make a stcm-and-leaf plot of the results. You can compare your results with those you'll see in this chapter.) To evaluate the physician's claim, instead of spending the afternoon flipping a coin, you can use the computer to construct a population in which half of the patients are cured and half are not. That's the situation if die physician's claim is not true. Then you can have the computer take a random sample of 10 patients and record the percentage that are cured. Have it repeat this procedure 500 times. 166 Chapter 9 The reason you're doing this is to see what kind of sample results ate possible if the new treatment is not different from the standard one. You can then determine whether rinding 70% cured in a sample of 10 patients is an unusual finding when the true cure rate is 50%. A stem-and-leaf plot of the results of the 500 experiments is shown in Figure 9.1. Figure 9.1 Stem-and-leaf plot of percentage cured for sample size 10 You can obtain stem-ono-teaf plo is using the €xpk>te ptQCQllUil), 9$ described in Chapter G Select the vatiablu cu S:cn « L«»t J.ďO Bit I«!-» H. 00 2 70.00 3 S8.00 < in-oo a 95.00 6 66.00 i n m-, u 5-00 Sŕ-;íľľO I••101 ■MO JO 000 ooo ooooooe moc joqoo oooo ;i...i; :i'i: .i! HC HC IC HC GO C i! y 'X'C 30U-'Ji;:|i .( ooo aooococoooo 00 0 00 oôc socoo ooc jo o sec; 000 OOOOOOCOOOOOOflOO oocjocooooooo 00 0 0000 0O0O0O0 JC 30C ooo ."!■ i: :,■: ■: :■■: 1 »901 Stpn width: ľ*>h loa:, 10.00 ) casalsi Far most samples, the euro rate is closa té 50% From this plot, you can tell approximately how often you would expect to see various outcomes in samples of size 10. The distribution of all possible sample outcomes for a statistic (such as the percentage cured) is called the sampling distribution of the statistic. í$pj Exactly what is a statistic anyhow? A statistic is some characteristic ■&xM of a sample. The sample mean and variance arc both examples of statistics. The term parameter is used to describe the characteristics of rhe population. For example, the average height of people in your sample is a statistic. If you measured the heights of all people in the population of interest, that would be called a parameter of the population. Parameters are usually designated (by statisticians, at least) with Greek symbols. For example, the mean of a population is called u (mu), while the mean of a sample is called X. Similarly, the standard deviation of the population is called o (sigma), while the value for a sample is called s, Mosr of the time, population values, or parameters, are not known. You must estimate them based on statistics calculated from samples. ■ ■ * The sampling distribution is usually calculated mathematically. In this case, you're using a computer to give you some idea of what it looks like. In Figure 9.1, you see that for most samples, the percentage of cures is close to 50%. In fact, 307 out of the 500 experiments resulted NORUŠ1S, Marija J. 1998. Guide to Date Analysis, Upper Saddle River, Prentice Hall. Ill I iff Iff Hi ! I f!|ř •a - 2 3 «1111 iti ifiriiijfř H ill C 2' 6ŕ Ü? O 3-5 ^i £ «J vs vs ui = =- Ci T' "ttttíji; Iflilíí Híí «! ~ -. = < o E* 3 -• : o U) Ol 7 3 s f s s í Dl O s s r CO M I' O "lj C C g S I! -1 ft =«- r, - — a. d 14 r i S = = It! « P 2 » sí-« p o ,t[! ■íŕflftä t* «O ~ o a « rt O ■ . « X 5 " 3 3 r S -a ■» p. -— rŕfrBí* S-O B llllíll O" -i: s.-a ■n = =■ S =.3.« § 1 = 9 §"8 Uli II!!!!! suiir-? — i, — H tIPífl 3- -*■ (6 _ " ET o *" . m C c w 5 ■ i ** O F1 &-0 = S -f b. A« -S s g S B n * E II s pi äfflfl I« í ilS-l 3" H 3 ES-Mfs c o a- -ľ H--? rsaó= s- S = r i» vr --I Evaluating Results Írom Samples \ 69 The Effect of Sample Size As you saw above, when the true cure rate is 50%, there's a good chance ihat anywhere from 3 to 7 patients could be cured in a sample of 10. Most of the outcomes that can occur would not be considered unusual, because they could reasonably occur if the true cure rate is 50%. What's more, if the new treatment results in a cure rate of 60% or 70%, you probably would not detect the improvement, since many sample rates that are compatible with true rates or 60 or 70% are also compatible with the 50% rate. That means that based on a sample of only 10 patients, it's very difficult to evaluate a new treatment. IM Can you ever tell from a sample of just 10 patients that a new Ireat-■91 ment is better? Yes. Since the existence of one little green man could convince you that there's life on Mars, similarly, 10 cures of a previously incurable disease could convince you that it's worth pursuing your treatment. It all depends on how unlikely your results are. ■ ■ ■ To sec what effect sample size has on your ability to evaluate the physician's claim, consider what happens if you take samples of 40 patients, instead of just 10, from the same population with a cute rate of 50%. The results of this computer experiment are shown in Figure 9.3. (Note that each stem in the plot is now divided into two rows.) When you compare Figure 9.3 with Figure 9-1, you see that the values arc much closer to 50% than before. Values greater than 60% or less than 40% arc now noticeably less likely. These rates were nor particularly unusual when you had samples of 10 patients. Based on Figure 9.3, you would estimate your chance of finding a sample rate of 70% or more or 30% or less when the true rate is 50% io be about 3 in 500, 0.6%. That means that only about 1 in 200 times would such a cure rate occur if the new treatment doesn't differ from the standard treatment. In summary, when you have samples of 40 cases, an observed rare of 70% or more, or 30% or less, is possible, but not very likely when the true population rate is 50%. If the physician sees the same cure rate of 70% based on a sample of 40 patients, you would be more likely to believe that perhaps she's onto something. Her results really would be unusual when die true cure rate is 50%. 170 Chapter 9 ffijgg Jhsí bow unusual does "unusual" need to bei The rule of thumb ^■»fj that is usually used to characterize results as unusual is a probability of 5% or less. That is, if results as extreme or more extreme than those observed arc expected to occur in 5 ior fewer) samples out of 100, the results are considered unusual, or statistically significant. ■ ■ ■ Figure 9.3 Stem-and-leaf plot of percentage cured for sample size 40 To COÍ3Ä7 tlus steioandJeef plot, select the variable cwstMO in rtie Stplcte c&it'og l>v* I r-.'-.i. m . 7.00 29.03 71.00 114.00 128.00 n ní 17.DO ; : Sien uldch: Each Icof ■ stan fa !.£ i 12í mrrmn 0ODOĎQOOI32IM »T 23 2? 2ií « . 555S55SSS5SS55555177777777777771777777 S . 00000000000000000002222222222222222ÍÍ2J22Í? 5 . SÍSSSSSS5S55S5SSS7777777T777777 L. . SeSCÍíCDJODJíJi? 6 . MS 7 , ů 10. oo L Jímms ťrnctiijr.íl I«**)«. Cure ŕäwš cosier (Töte tigfuly around 50% "an i" Figure 9, Larger samples improve your chances of detecting a difference in the cure rates (if in facr there is one) because there is less variability in the possible outcomes. Consider Figure 9.4, which contains descriptive statistics for the distribution shown in Figure 9.3. The mean value is again close to 50%. nie standard deviation, however, is much smaller than for samples of size 10. It is now 7.29%, compared to the standard deviation of 16.22% in Figure 9.2. There's a pattern in the way that sample size affects the variance or the sampling distribution of means. If you increase the sample size by a factor of four, the variance decreases by a factor of four. Since the standard deviation is the square root of the variance, it decreases by a factor of two. Figure 9.4 Descriptive statistics for samples of size 40 N Minimum Maximum Mean Sid. Deviation Valid N Itisiwisôl 500 ECO 30.CO 70.CO •1&.ÜW 7.2664 / Ale istandard error of <"* maan is much smaller than Ic*:šámWoí:š6ôiq -. ■:■-.. s mni O O Ď. Ö. O g. ... r i II o — IS 5 S" I 3 - r = 3 3- ~ 3 ■1 a z lííilíií: i&lii'ü I u D "O _. iílis E ř 2? "U š S 'g 8 g -J B g h, Ilrlll flfUSHHH! 3 - n »HUi, »mnnmiiuiwii mim H ITS. !.?*-£ Hlíi Ifíffllff» ttföffil I Jilílíti ^?3 !íí »Mi« UÍM" — mssi 5 c 3 íttiiH I £2«s2 -j ■ - 8 ui ■j -• o — | Z e u -m •I _ \ 883 Ä 1 1 \ « \ ..■ e: í b 27 NORUŠ1S, Marija J. 1998. Guide to Data Analysis. Upper Saddle River, Prentice Hall. fffrS ~£ o- g % -■ ' a': :. 5 =? -j w - :; T-. \ 1 ! í G 2 01 c 3 3 -: ■< ■ '■