204 BUILDING AND APPLYING LOGIT AND LOGLINEAR MODELS c. Using conditional logistic regression, (i) conduct an exact test of the hypothesis of no effect of lymphocytic infiltration, controlling for the other variables; (ii) compute a 95% confidence interval for the effect. Interpret results. Table 7.10 Lymphocytic Osteoblastic Disease-Free Infiltration Sex Pathology Yes No 3 0 2 0 4 0 1 0 5 0 3 2 5 4 6 11 High Female No Yes Male No Yes Low Female No Yes Male No Yes Source: LogXact-Turbo User Manual, Cambridge, MA: Cytel Software (1993), p. 5-23. 7.33. For the model logit(ir) = a + ßx, let (xi,yi) denote the x and y values for subject i,i= I,... ,N. Suppose yr = 0 for all x below some point and y i = 1 for all x above that point. Explain intuitively why ß = oo. (Note: In technical terms, the sufficient statistics are £];y, for a and ^^o*; for ß. For a given ÜC y<< Sxiyi ta^es its maximum possible value for this assignment of {y;} to fixed {#,•}.) 7.34. Refer to {p,ijk = ni+kn+jk/n++k} for model (XZ, YZ). Show that these fitted values have the same X-Z and Y-Z marginal totals as the observed data. For 2X2 X K tables, show that dxY(k) ~ 1- Illustrate these results for the fit of model (AM, CM) to Table 6.3. 7.35. Refer to formula (7.5.3) applied to the independence model for a 2 X 2 table. Show that for the constraints \x = k\ = 0, for which/3 = (A, Af.Aj), Xhas rows (1,0,0), (1,0,1), (1,1,0), (1,1,1). 7.36. For Table 6.3, show the matrix representation of loglinear model (AC, AM, CM). Specify the parameter constraints used in your model matrix. Show the matrix representation of the corresponding logit model, when M is a response. 7.37. A GLM has form g(fi) = X/3, for a monotone function g. Explain what each symbol in this formula represents for fitting the ordinary linear regression model to n observations on a normally distributed response and a single predictor. CHAPTER 8 Multicategory Logit Models This chapter presents generalizations of logistic regression models that handle multicategory responses. Section 8.1 presents models for nominal response variables, and Sections 8.2 and 8.3 present models for ordinal response variables. As in ordinary logistic regression modeling, explanatory variables can be continuous and/or categorical. At each combination of levels of the explanatory variables, the models assume that the response counts for the categories of Y have a multinomial distribution. This generalization of the binomial applies when the number of response categories exceeds two. Logistic regression models are a special case of these models for binary responses. Like logistic regression models and unlike loglinear models, multicategory logit models treat one classification as a response and the other variables as explanatory. Nevertheless, when explanatory variables are categorical, some of these models are equivalent to loglinear models for contingency tables. 8.1 LOGIT MODELS FOR NOMINAL RESPONSES Suppose ľ" is a nominal variable with / categories. The order of listing the categories is irrelevant. Let {tt\ ,..., irj} denote the response probabilities, satisfying J2; Tj = 1. When one takes n independent observations based on these probabilities, the probability distribution for the number of outcomes that occur of each of the J types is the multinomial. It specifies the probability for each possible way of allocating the n observations to the / categories. Here, we simply mention this as a sampling model for the counts, and we will not need to calculate such probabilities. For the case 7 = 2, this is the binomial distribution (1.2.2). Multicategory (also called polychotomous) logit models simultaneously refer to all pairs of categories, and describe the odds of response in one category instead of another. Once the model specifies logits for a certain (/ — 1) pairs of categories, the rest are redundant. 205 206 MULTICATEGORY LOGIT MODELS 8.1.1 Baseline-Category Logits Logit models for nominal responses pair each response category with a baseline category, the choice of which is arbitrary. When the last category (/) is the baseline, the baseline-category logits are Given that the response falls in category j or category /, this is the log odds that the response is j. For / = 3, for instance, the logit model uses log(771/773) and log(7r2/7r3). The logit model using the baseline-category logits with a predictor x has form Io§ (~) = aJ+ ßj*' j=l,....J-l. (8.1.1) The model consists of J - 1 logit equations, with separate parameters for each. That is, the effects vary according to the response category paired with the baseline. When / = 2, this model simplifies to a single equation for log(iri/n^) = logit(iri), resulting in the ordinary logistic regression model for binary responses. Parameters in the / - 1 equations (8.1.1) determine parameters for logits using all other pairs of response categories. For instance, for an arbitrary pair of categories aandb, *S)M&)Mš)Mš)' = (aa + ßax) - (ab + ßbx) = (aa ~ ab) + (ßa - ßb)x. (8.1.2) Thus, the logit equation for categories a and b has intercept parameter (aa — ab) and slope parameter (ßa — ßb). For optimal efficiency, one should use software that fits the / — 1 logit equations (8.1.1) simultaneously. Estimates of the model parameters then have smaller standard errors than when binary logistic regression software fits each component equation in (8.1.1) separately. For simultaneous fitting, the same parameter estimates occur for a pair of categories no matter which category is the baseline. This logit model is an important tool in marketing research for modeling how subjects choose one of a discrete set of options. A generalization of model (8.1.1) allows the explanatory variables to take different values for different response categories. For instance, the choice of a brand of car would likely depend on price, which would vary among the brand options. This generalized model, often called a multinomial logit model, is beyond the scope of this text (see Agresti (1990), Sec. 9.2). The following example deals with discrete choice in a somewhat different context: for alligators rather than humans. LOGIT MODELS FOR NOMINAL RESPONSES 207 Table 8.1 Alligator Size (Meters) and Primary Food Choice," for 59 Florida Alligators 1.241 1.301 1.301 1.32 F 1.32 F 1.40 F 1.421 1.42 F 1.451 1.45 0 1.471 1.47 F 1.501 1.521 1.551 1.601 1.631 1.65 0 1.651 1.65 F 1.65 F 1.68 F 1.701 1.73 0 1.781 1.781 1.78 0 1.801 1.80 F 1.85 F 1.881 1.931 1.981 2.03 F 2.03 F 2.16 F 2.26 F 2.31 F 2.31 F 2.36 F 2.36 F 2.39 F 2.41 F 2.44 F 2.46 F 2.56 0 2.67 F 2.721 2.79 F 2.84 F 3.25 0 3.28 0 3.33 F 3.56 F 3.58 F 3.66 F 3.68 0 3.71 F 3.89 F "F - Fish, / = Invertebrates, O = Other, Source: Thanks to M. F. Delany and Clint T. Moore for providing these data. 8.1.2 Alligator Food Choice Example Table 8.1 is taken from a study by the Florida Game and Fresh Water Fish Commission of factors influencing the primary food choice of alligators. For 59 alligators sampled in Lake George, Florida, Table 8.1 shows the alligator length (in meters) and the primary food type, in volume, found in the alligator's stomach. Primary food type has three categories: Fish, Invertebrate, and Other. The invertebrates were primarily apple snails, aquatic insects, and crayfish. The "other" category includes reptiles (primarily turtles, though one stomach contained tags of 23 baby alligators that had been released in the lake during the previous year!), amphibean, mammal, plant material, and stones or other debris. We apply logit model (8.1.1) with / = 3 to these data, using Y — "primary food choice" as the response and X = "length of alligator" as predictor. Table 8.2 shows ML parameter estimates and standard errors for the two logits using "other" as the baseline category. From Table 8.2, logi^/^) = 1.61S — 0.11 Ojc, and log^/^) -5.697 - 2.465x. By (8.1.2), the estimated log odds that the response is "fish" rather than "invertebrate" equals Iog(n-,/#2) = (1-618 - 5-697) + [-0.310 - (-2.465)]* = -4.080 + 2.355i. For each logit, one interprets the estimates just as in ordinary binary logistic regression models, conditional on the event that the response outcome was one of those two categories. For instance, given that the primary food type is fish or invertebrate, the estimated probability that it is fish increases in length x according to an S-shaped curve. For alligators of length x + 1 meters, the estimated odds that Table 8.2 Parameter Estimates and Standard Errors (in parentheses) for Generalized Logit Model Fitted to Table 8.1 Food Choice Categories for Logit Parameter (Fish/Other) (Invertebrate/Other) Intercept 1.618 5.697 Length -0.110(,517) -2.465(.900) 208 MULTICATEGORY LOGIT MODELS primary food type is "fish" rather than "invertebrate" equal exp(2.355) = 10.5 times the estimated odds for aligators of length x meters. To test the hypothesis that primary food choice is independent of alligator length, we test//o : ßj ~ Ofory = 1,2 in model (8.1.1). The likelihood-ratio test takes twice the difference in maximized log likelihoods between this model and the simpler one having response independent of length. The test statistic equals 16.8 with df= 2, giving a P-value of .01 and strong evidence of a length effect. 8.1.3 Estimating Response Probabilities One can alternatively express the multicategory logit model directly in terms of the response probabilities, as _ exp(a/ + ßjx) TTj - =-----------r-^~T- ; - 1,...,/- 1, (a.1.3) The denominator is the same for each probability, and the numerators for various j sum to the denominator, so J2j ^j ~ 1- The parameters equal zero in (8.1.3) for whichever category is the baseline in the logit expressions. For instance, the model that defines parameters with category J as the baseline for the denominator of each logit (as in (8.1.1)) has aj - ßj = 0. For category j, (ctj, ßj) then refers to the logit equation for logOy/V/]. The estimates in Table 8.2 contrast "fish" and "invertebrate" to "other" as the baseline category. The estimated probabilities (8.1.3) of the outcomes (Fish, Invertebrate, Other) equal ____________exp(I.62-0.1hr)__________ 771 ~ 1 + exp( 1.62 - 0.1 \x) + exp(5.70 - 2A7x) exp(5.70 - 2.47x) 7T2 = I + exp(1.62 - 0.1 It) + exp(5.70 - 2.47x) 1 773 ~ 7+ exp(l .62 - 0.1 Ix) + exp(5.70 ~ 2.47x) The 1 term in each denominator and in the numerator of 7r3 represents exp(á3 + fox) using ôj = & = 0. The three probabilities sum to 1, since the numerators sum to the common denominator. The logs of the ratios of [tti/ít^] and [fti/^ů produce the results in Table 8.2. For an alligator of length x = 3.9 meters, for instance, the estimated probability that primary food choice is "other" equals ____________________1__________________ _ 773~ í + exp[i.62-0.10(3.9)]+exp[5.70-2.47(3.9)] ~ °'23' Figure 8.1 displays the three response probabilities as a function of length. LOGIT MODELS FOR NOMINAL RESPONSES 209 Predicted Probability 1 2 3 Length of Alligator Figure 8.1 Predicted probabilities for primary food choice. 8.1.4 Belief in Afterlife Example Logit models for nominal responses also apply when explanatory variables are categorical, or a mixture of categorical and continuous. When predictors are entirely categorical, one can display the data as a contingency table. If the data are not sparse, one can test model goodness of fit using X2 or G2 statistics. In addition, the models then have equivalent loglinear models. To illustrate, Table 8.3, taken from the 1991 General Social Survey, has response variable Y = belief in life after death, with categories (Yes, Undecided, No), and explanatory variables X\ = gender and X2 = race. We use dummy variables for the predictors, with X{ = 1 for females and 0 for males, and x-i = 1 for whites and 0 for blacks. Using "no" as the baseline category for belief in life after death, we form the model lofi(^) =aj + ßfXl+ßXx2, y-1.2, (8.1.4) where G and R superscripts identify the gender and race parameters. The model assumes a lack of interaction between gender and race in their effects on belief in 210 MULTICATEGORY LOGIT MODELS Table 8.3 Belief in Afterlife, by Race and Gender, with Fitted Values for Generalized Logit Model Gender Belief in Afterlife Race Yes Undecided No White Female 371 49 74 (372.8) (49.2) (72.1) Male 250 45 71 (248.2) (44.8) (72.9) Black Female 64 9 15 (62.2) (8.8) (16.9) Male 25 5 13 (26.8) (5.2) (H.n Source: 1991 General Social Survey. life after death. The effect parameters represent log odds ratios with the baseline category. For instance, ßf is the conditional log odds ratio between gender and response categories 1 and 3 (yes and no), given race; ß^ is the conditional log odds ratio between gender and response categories 2 and 3 (undecided and no). Table 8.3 also shows ML fitted values for this model. The goodness-of-fit statistics are G2 = 0.8 and X2 = 0.9. The sample has two non-redundant logits at each of four gender-race combinations, for a total of eight logits. Model (8.1.4), considered for j = 1 and 2, contains six parameters. Thus, the model has residual df = 8 — 6 = 2. The model fits well. Table 8.4 shows the parameter estimates and their standard errors. The estimated odds of a "yes" rather than a "no" response for females are exp(0.419) = 1.5 times those for males, controlling for race; for whites, they are exp(0.342) = 1.4 times those for blacks, controlling for gender. To test the effect of gender, we test H0 : ßf = 0 for j = 1,2. The likelihood-ratio test compares G2 = 0.8 (df = 2)toG2 = 8.0 (df — 4) obtained by dropping gender from the model. The difference of 7.2, based on df = 2, has a P-value of .03 and shows evidence of a gender effect. By contrast, the effect of race is not significant, the model deleting race having G1 = 2.8 (df = 4). This partly reflects the larger standard errors that the effects of race have, due to a much greater imbalance between sample sizes in the two race categories than occurs with gender. Table 8.4 Parameter Estimates and Standard Errors (in parentheses) for Generalized Logit Model Fitted to Table 8.3 Parameter Belief Categories for Logit (Yes/No) (Undecided/No) Intercept Gender (F = 1) Race (W = 1) 0.883 (.243) 0.419 (.171) 0.342 (.237) -0.758 (.361) 0.105 (.246) 0.271 (.354) CUMULATIVE LOGIT MODELS FOR ORDINAL RESPONSES 211 Table 8.5 Predicted Probabilities for Belief in Afterlife Gender Belief in Afterlife Race Yes Undecided No White Black Female Male Female Male 0.76 0.68 0.71 0.62 0.10 0.12 0.10 0.12 0.15 0.20 0.19 0.26 Table 8.5 displays predicted probabilities for the three response categories. To illustrate, for white females (xi = x2 = 1), the estimated probability of response 1 ("yes") equals ________________exp[0.883 + 0,419(1) + 0.342(1)]________________ 1 + exp[0.883 + 0.419(1) + 0.342(1)] + exp[-0.758 + 0.105(1) + 0.271(1)] ~ ' ' This also follows directly from the fitted values, as 372.8/(372.8 + 72.1 + 49.2) = .76. 8.1.5 Connection with Loglinear Models When all explanatory variables are categorical, logit models have corresponding loglinear models. The loglinear model has the most general interaction among the explanatory variables from the logit model. It has the same association and interaction structure relating the explanatory variables to the response. To illustrate, the model fitted to Table 8.3 assumes main effects of gender (G) and race (R) on belief (B) in afterlife, with no interaction. It corresponds to the loglinear model (GR, BG, BR) of homogeneous association. The same estimated effect of G on B results from logit or loglinear parameters. For instance, model (GR,BG,BR) has estimated conditional odds ratio relating B categories 1 = "yes" and 3 = "no" to G equal to expfjtfip + Xf2G - Äff - Äff) = 1.5, as obtained above for the logit model. The simpler logit model that deletes the race effect on belief corresponds to loglinear model (GR,BG). For this logit model and for the one having both G and R as predictors of belief, the corresponding loglinear model contains the G-R term. This term represents the interaction among variables that are explanatory in the logit model, 8.2 CUMULATIVE LOGIT MODELS FOR ORDINAL RESPONSES When response categories are ordered, logits can directly incorporate the ordering. This results in models having simpler interpretations and potentially greater power than ordinary multicategory logit models. The cumulative probabilities are the probabilities that the response Y falls in category j or below, for each possible j. The jth cumulative probability is 212 MULTICATEGORY LOGľľ MODELS The cumulative probabilities reflect the ordering, with P(Y ^ 1) ^ P(Y < 2) s ■ - - < P (Y ^ J) = 1. Models for cumulative probabilities do not use the final one, P(Y < /), since it necessarily equals 1. The logits of the first / — 1 cumulative probabilities are Iogit[P(7 ^ j)] = log P(Y ^ j) l~P(Y <;) y-n-j+l + -•• + TTj J These are called cumulative logits. Each cumulative Iogit uses all / response categories. A model for the jth cumulative logit looks like an ordinary logit model for a binary response in which categories 1 to ;' combine to form a single category, and categories j + 1 to J form a second category. In other words, the response collapses into two categories. Ordinal models simultaneously provide a structure for all / — 1 cumulative logits. For 7 = 3, for instance, models refer both to log[iri/(7r2 + 7^)3 and loglX^ + -t^í/Ví]- 8.2.1 Proportional Odds Model For a predictor X, the model logit[P(r < j)] = aj-hßx, ./= 1,...,/- 1, (8.2.1) has parameter ß describing the effect of X on the log odds of response in category ;' or below. In this formula, ß does not have a j subscript, so the model assumes an identical effect of X for all J — 1 collapsings of the response into binary outcomes. When this model fits well, it requires a single parameter rather than J —\ parameters to describe the effect of X. Interpretations for this model refer to odds ratios for the collapsed response scale, for any fixed j. For two values *i and x2 of X, the odds ratio utilizes cumulative probabilities and their complements, P(Y^j\X = x2)/P(Y>j\X = x2) p(Y sy|jr = xO/P(Y >j\x = xiY The log of this odds ratio is the difference between the cumulative logits at those two values of x. This equals ß(x2 ~ x{), proportional to the distance between the x values. The same proportionality constant (ß) applies for each possible point j for the collapsing. Because of this property, model (8.2.1) is called a proportional odds model. In particular, for x% - x\ = 1, the odds of response below any given category multiply by eß for each unit increase in X. When the model holds with ß = 0, X and Y are statistically independent. Explanatory variables in cumulative logit models can be continuous, categorical, or of both types. The ML fitting process uses an iterative algorithm simultaneously CUMULATIVE LOGIT MODELS FOR ORDINAL EESPONSES 213 P(Y J). The common effect ß for each j implies that the three response curves have the same shape. Any one curve is identical to any of the others simply shifted to the right or shifted to the left. As in logistic regression, the size of \ß\ determines how quickly the curves climb or drop. At any fixed x value, the curves have the same ordering as the cumulative probabilities, the one for P(J ^ 1) being lowest. Figure 8.2 has ß > 0. Figure 8.3 shows corresponding curves for the category probabilities, P(Y = j) = P(Y < J) - P(Y áj- 1). As x increases, the response on Y is more likely to fall at the low end of the ordinal scale. When jß < 0, the 1.0 - P(Y=j) 0.8 - 0.6 - 0.4 0.2 - 0.0 X Figure 8.3 Depiction of category probabilities in proportional odds model. 214 MULTICATEGORY LOGTT MODELS curves in Figure 8.2 descend rather than ascend, and the labels in Figure 8.3 reverse order. Then, as x increases, Y is more likely to fall at the high end of the scale. (The model is sometimes written instead as logit[P(Y ^ /)] = aj — ßx, so that ß > 0 corresponds to Y being more likely to fall at the high end of the scale as x increases.) 8.2.2 Political Ideology Example Table 8.6, from the 1991 General Social Survey, relates political ideology to political party affiliation. Political ideology uses a five-point ordinal scale, ranging from very liberal to very conservative. Let* be a dummy variable for political party, with* = 1 for Democrats and x = 0 for Republicans. The ML fit of the proportional odds model (8.2.1) has estimated effect ß = 0.975 (ASE = 0.129). For any fixed j, the estimated odds that a Democrat's response is in the liberal direction rather than the conservative direction (i.e., Y ^ j rather than Y > j) equal exp(0.975) = 2.65 times the estimated odds for Republicans. A 95% confidence interval for this odds ratio equals exp(0.975 ± 1.96 X 0.129), or (2.1,3.4). A fairly substantial association exists, Democrats tending to be more liberal than Republicans. Table 8.6 also displays the fitted values for the model. These have an odds ratio of exp()3) = 2.65 for each of the four collapsings of the data to a 2 X 2 table. For instance, for the estimated odds of a very liberal response, (78.4X44.0 + 151.7 + 75.5 + 104.0) (83.2+168.2 + 49.1+49.1X31.8) ' ' One can use the fitted values or parameter estimates to calculate estimated probabilities. The cumulative probabilities equal pry £ /) = «P(«j + frO V J) l + exp(a; + ßx)" For instance, &i = —2.469, so the first estimated cumulative probability for Democrats (jc = 1) equals exp[-2.469 + .975(1)] = 1 + exp[-2.469 + .975(1)] ' ' From the fitted values for the 428 Democrats, this also equals 78.4/428. Table 8.6 Political Ideology by Party Affiliation, with Fitted Values for Cumulative Logit Model Political Ideology Slightly Moderate Conservative Party Affiliation Democratic Republican Very Liberal 80 (78.4) 30 (31.8) Slightly Liberal 81 (83.2) 46 (44.0) 171 (168.2) 148 (151.7) 41 (49.1) 84 (75.5) Very Conservative Total 428 55 (49.1) 99 (104.0) (49.1) 99 407 (104.0)___ Source: 1991 General Social Survey. CUMULATIVE LOGIT MODELS FOR ORDINAL RESPONSES 215 The model fits Table 8.6 well, with G2 = 3.7andX2 = 3.7, based on df = 3. The model with ß = 0 specifies independence between ideology and party affiliation, and is equivalent to the loglinear model of independence. The G2 test of fit of that special case is simply the G2 test of independence (2.4.3) for two-way contingency tables, which equals G2 = 62.3 based on df = 4. The model permitting an effect fits considerably better than the independence model. The likelihood-ratio statistic foran ordinal test of independence (#o : ß = 0) is the difference between G2 values for the independence model and the proportional odds model. The difference in G2 values of 62.3- 3.7 = 58.6, based on df = 4-3 = 1, gives extremely strong evidence of an association (P < .0001). Like the test for the ordinal loglinear model presented in Section 7.2.3, this test uses the ordering of the response categories. When the model fits well, it is more powerful than the tests of Section 2.4 based on df = (I — l)(J — 1), since it focuses on a restricted alternative and has only a single degree of freedom. Similar strong evidence results from the Wald test, using z2 = (ß/ASE)2 = (0.975/0.129)2 = 57.1. 8.2.3 Invariance to Choice of Response Categories When the proportional odds model holds for a given response scale, it also holds with the same effects for any collapsing of the response categories. For instance, if a model for categories (Very liberal, Slightly liberal, Moderate, Slightly conservative, Very conservative) fits well, approximately the same estimated effects result from collapsing the response scale to (Liberal, Moderate, Conservative). This invariance to the choice of response categories is a nice feature of the model. Two researchers who use different response categories in studying an association should reach similar conclusions. To illustrate, we collapse Table 8.6 to a three-category response, combining the two liberal categories and combining the two conservative categories. Then, the ML estimated effect of party affiliation changes only from 0.975 (ASE = .129) to 1.006 (ASE = 0.132). Interpretations are unchanged. Some loss of efficiency occurs in collapsing ordinal scales, resulting in larger standard errors. In practice, when observations are spread fairly evenly among the categories, the efficiency loss is minor unless one collapses to a binary response. It is usually inadvisable to collapse ordinal data to binary. The proportional odds model implies trends upward or downward among distributions of Y at different values of explanatory variables. When X refers to two groups, as in Table 8.6, the model fits well when subjects in one group tend to make higher responses on the ordinal scale than subjects in the other group. The model does not fit well when the response distributions differ in their dispersion rather than their average. If Democrats tended to be primarily moderate in ideology, while Republicans tended to be both very conservative and very liberal, then the Republicans' responses would show greater dispersion than the Democrats'. The two ideology distributions would be quite different, but the proportional odds model would not detect this if the average responses were similar. 216 MULTICATEGORY LOGIT MODELS When a proportional odds model does not fit well, an improved fit may result from a more complex model having quadratic or interaction terms, or from a different link that allows P(Y ^ j) to approach 1 at a different rate than it approaches 0 and permits a nonsymmetric appearance for the category probability curves of Figure 8.3 (Agresti (1990), Section 9.5). When simplified ordinal models do not fit well, one can fit ordinary multicategory logit models of form (8.1.1) and use the ordinality in an informal way in interpreting the associations. 8.3 PAIRED-CATEGORY LOGITS FOR ORDINAL RESPONSES* Cumulative logit models for ordinal responses use the entire response scale in forming each logit. This section presents alternative logits for ordered categories that, like baseline-category logits for nominal responses, use pairs of categories. 8.3.1 Adjacent-Categories Logits One alternative forms J-~\ logits using all pairs of adjacent categories. The adjacent-categories logits are '-(^f)' '-'.....'-■ For J = 3, for instance, the logits are log(ttz/^i) and log(773/^2). A model using these logits with a predictor x has form log (^) = «j + ßj*. j=l,...J-l. (8.3.1) These logits, like the baseline-category logits, determine logits for all pairs of response categories. Model (8.3.1) is equivalent to logit model (8.1.1), with ßj in (8.3.1) being equivalent to /3/+i — ßj in (8.1.1). A simpler version of model (8.3.1), loS (^) = <*J + ßx- j=h...,J- 1. (8-3.2) has identical effects {ßj = ß} for each pair of adjacent categories. For this model, the effect of X on the odds of making the higher instead of the lower response is the same for all pairs of adjacent categories. This model and proportional odds model (8.2.1) use a single parameter, rather than / — 1 parameters, for the effect of X. When the model holds, independence is equivalent to ß = 0. The simpler adjacent-categories logit model (8.3.2) implies that the coefficient of x for the logit based on arbitrary response categories a and b equals ß(a — b). The effect depends on the distance between categories, so this model recognizes the ordering of the response scale. One can check whether this simplification is justified by comparing the model fit to that of (8.3.1), or equivalently to logit model (8.1.1). PAIRED-CATEGORY LOGITS FOR ORDINAL RESPONSES 217 8.3.2 Political Ideology Example Revisited To illustrate the adjacent-categories logit model, we return to Table 8.6 and model political ideology using the simple model (8.3.2) having a common effect for each logit. Let x = 0 for Democrats and x = 1 for Republicans. The ML estimate of the party affiliation effect is ß = 0.435. The estimated odds that a Republican's ideology classification is in category ;' + 1 instead of j are exp(ß) = 1.54 times the estimated odds for Democrats. This is the estimated odds ratio for each of the four 2X2 tables consisting of a pair of adjacent columns of Table 8.6. For instance, the estimated odds of "slightly conservative" instead of "moderate" ideology are 54% higher for Republicans than for Democrats. The estimated odds ratio for an arbitrary pair of columns a > b equals exp[j8(a - b)]. The estimated odds that a Republican's ideology is "very conservative" (category 5) instead of "very Uberal" (category 1) are exp[0.435(5 - 1)] = (1.54)4 = 5.7 times those for Democrats. Republicans tend to be much more conservative than Democrats. The model fit has G2 = 5.5 with df = 3, a reasonably good fit. The special case of the model with ß = 0 specifies independence of ideology and party affiliation and is equivalent to the loglinear model of independence. The G2 fit of that model is simply the G2 statistic (2.4.3) for testing independence, which equals G2 = 62.3 with df = 4. The model permitting a party affiliation effect fits much better than the independence model. The likelihood-ratio test statistic for the hypothesis that party affiliation has no effect on ideology (Hq : ß = 0) is based on the difference between the G2 values for the two models. It equals 62.3 - 5.5 = 56.8 with df = 4 - 3 = 1. There is very strong evidence of an association (P < .0001). Results are similar to those for the cumulative-logit analysis in Section 8.2.2. For two-way contingency tables, the more general model (8.3.1) and the equivalent logit model (8.1.1) are saturated. An advantage of using the single effect parameter is that the simpler model is unsaturated. 8.3.3 Connection with Loglinear Models Many loglinear models for ordinal variables have simple representations as adjacent-category logit models. For instance, Section 7.2.1 introduced the linear-by-linear association model, log fuj = A + A* + Aj + ßmvj, (8.3.3) for which the association term uses monotone scores for the rows and columns. For this model with column scores {vj = j}, the adjacent-category logits within row i are = logíflfj+i) - l0g(jUi/) = (A + A? + kj+1 + ßuiVj+1) - (A + Af + Aj + ßmvj) = (Aj+1 - kj) + ßut. 218 MULTICATEGORY LOGIT MODELS This has form (8.3.2), identifying a,- with (Aj+1 - Aj) and the row scores {«;} with the levels of*. In fact, the two models are equivalent. The logit representation (8.3.2) provides an interpretion for model (8.3.3). The conclusions presented above for fitting logit model (8.3.2) to Table 8.6 also result from fitting the loglinear model (8.3.3) with equally-spaced column scores. Analogous remarks apply to multi-way tables. For instance, consider Table 7.5 on job satisfaction (S), income (/), and gender (G), analyzed in Section 7.3. Unlike the loglinear models described there, logit models treat job satisfaction as a response variable and income and gender as explanatory variables. Let g denote a dummy variable for gender, and let {x,-} denote scores assigned to the levels of income. Consider the model for job satisfaction response probabilities, log C—1) = <*j + ßlXi + ß2g, j = 1,2,3. (8.3.4) The model assumes identical effects of explanatory variables for each adjacent pair of response categories. This model is equivalent to a special case of loglinear model (Gl, GS, IS) in which the G-S and I-S associations each have linear-by-linear (Ĺ X L) form, with equally-spaced scores for S and the scores {*,} for / and g for G. That special case is also a special case of the homogeneous L X L model (7.3.1) (which assumed L X Ĺ structure for the I-S association) that also provides linear-by-linear structure for the G-S association. Using income categories {1,2,3,4} as scores, model (8.3.4) fits well, with G2 = 12.6 having df = 19. It provides estimates ft = 0.389 (ASE = 0.155) and ß2 = 0.045 (ASE = 0.314). Satisfaction is positively associated with income, given gender, but shows no evidence of association with gender, given income. 8.3.4 Continuation-Ratio Logits Another approach forms logits for ordered response categories in a sequential manner. One constructs models simultaneously for These are called continuation-ratio logits. They refer to a binary response that contrasts each category with a grouping of categories from lower levels of the response scale. A second type of continuation-ratio logit contrasts each category with a grouping of categories from higher levels of the response scale; that is, log (--------S--------) ,iog (--------^--------) „...log (2=±) . Models using these logits have different parameter estimates and goodness-of-fit statistics than models using the other continuation-ratio logits. PAIRED-CATEGORY LOGITS FOR ORDINAL RESPONSES 219 Table 8.7 Outcomes for Pregnant Mice in Developmental Toxicity Study Response (mg/kg per day) Dead Malformation Normal 0 (controls) 15 1 281 62.5 17 0 225 125 22 7 283 250 38 59 202 500 144 132 9 Source: Based on results in C. J. Price et al., Fund. Appl, Toxicol., 8:115-126 (1987). I thank Dr. Louise Ryan for showing me these data. We illustrate continuation-ratio logits using Table 8.7, which refers to a developmental toxicity study. Rodent studies are commonly used to test and regulate substances posing potential danger to developing fetuses. This study administered diethylene glycol dimethyl ether, an industrial solvent used in the manufacture of protective coatings, to pregnant mice. Each mouse was exposed to one of five concentration levels for ten days early in the pregnancy. Two days later, the uterine contents of the pregnant mice were examined for defects. Each fetus had the three possible outcomes (Dead, Malformation, Normal). The outcomes are ordered, normal being the most desirable result. We use continuation-ratio logits to model the probability of a dead fetus, using log[7ri/(iT2 + ir3y], and the conditional probability of a malformed fetus, given that the fetus was live, using log^/n^). We used scores {0,62.5,125,250,500} for concentration level. The two models are ordinary logistic regression models in which the responses are column 1 and columns 2-3 combined for one fit and column 2 and column 3 for the second fit. The estimated effect of concentration level is 0.0064 (ASE = 0.0004)formefirstlogit,and0.0174(AS£ = 0.0012) for the second logit. In each case, the less desirable outcome is more likely as concentration level increases. For instance, given that a fetus was live, the estimated odds that it was malformed rather than normal changes by a multiplicative factor of exp(100 X 0.0174) = 5.7 for every 100-unit increase in concentration level. When models for different continuation-ratio logits have separate parameters, as in mis example, separate fitting of models for different logits gives the same results as simultaneous fitting. The sum of the separate G2 statistics is an overall goodness-of-fit statistic pertaining to the simultaneous fitting of the models. For Table 8.7, the G2 values are5.8forthe first logit and 6.1 for the second, each based on df = 3. We summarize the fit by their sum, G2 = 11.8, based on d f = 6. This analysis treats pregnancy outcomes for different fetuses as independent observations. In fact, each pregnant mouse had a litter of fetuses, and statistical dependence may exist among different fetuses in the same litter. The model also treats different fetuses at a given concentration level as having the same response probabilities. Heterogeneity of various types among the litters (for instance, due to different physical conditions of different pregnant mice) would usually cause these probabilities to vary somewhat among litters. Either statistical dependence or heterogeneous probabilities 220 MULTTCATEGORY LOGIT MODELS violates the binomial assumption and typically causes overdispersion—greater variation than the binomial model predicts. For example, at a fixed concentration level, the number of mice that die in a litter may vary among pregnant mice to a greater degree than if the counts were independent and identical binomial variates. The total G2 for testing the continuation-ratio model shows evidence of lack of fit. The structural form chosen for the model may be incorrect. The lack of fit may partly or entirely, however, reflect overdispersion caused by dependence within litters or heterogeneity among litters. Both factors are common in developmental toxicity studies. See Collett ((1992), Ch. 6) and Morgan ((1992), Ch. 6) for ways of handling overdispersion. PROBLEMS 8.1. Refer to the example in Section 8.1.2. a. Using the model fit, calculate an odds ratio that describes the estimated effect of length on primary food choice being either "invertebrate" or "other." b. Estimate the probability that food choice is "invertebrate," for an alligator of length 3.9 meters. c. Find the length at which the outcomes "invertebrate" and "other" are equally likely. 8.2. T*«:}.? ?.? displays primary food choice for a sample of alligators, classified by length ^'" 2.3 meters, > 2.3 meters) and by the lake in Florida in which they were caught. Table 8.8 Size Primary Food Choice Lake Fish Invertebrate Reptile Bird Other Hancock S 2.3 23 4 2 2 8 >2.3 7 0 1 3 5 Oklawaha <2.3 5 11 1 0 3 >2.3 13 8 6 1 0 Trafford á 2.3 5 11 2 1 5 >2.3 8 7 6 3 5 George £2.3 16 19 1 2 3 >2.3 17 1 0 1 3 Source: Wildlife Research Laboratory, Florida Game and Fresh Water Fish Commission. a. Use a logit model to describe effects of length and lake on primary food choice. b. Using your model, estimate the probability that the primary food choice is "fish," for the various length and lake combinations. Interpret. c. Which loglinear model is equivalent to this logit model? PEOBLEMS 221 8.3. Refer to the example in Section 8.1.4, a. Using the fitted model, estimate the probability of response "no" for black females, (i) using parameter estimates, (ii) using fitted values. b. Fit the logit model for which race has no effect on belief in an afterlife, controlling for gender. Test the goodness of fit, and interpret. Which loglinear model is equivalent to this model? c. Describe the gender effect for the model in (b) by reporting the estimated odds ratio for each pair of response categories. Interpret, and explain how these relate to the marginal gender-belief odds ratios. d. Does the model in (b) imply that race and belief in an afterlife are marginally independent, collapsing over gender? (Hint: Consider collapsibility for the corresponding loglinear model.) 8.4. Refer to Problem 6.7. Treating vote for President (P) as the response variable and B and D as explanatory variables, find a logit model that fits these data well. Interpret parameter estimates for the effects of B on P. Which loglinear model is equivalent to this model? Test the significance of the B-P conditional association, and interpret. 8.5. Refer to Problem 2.21 with Table 2.15. Using modeling methods, describe and make inferences about the effect of mammography experience on women's attitudes. 8.6. Table 8.9 shows results of a survey of women, relating frequency of breast self-examination and age. Fit a proportional odds model. Analyze goodness of fit, and conduct descriptive and inferential analyses about the association. Table 8.9 Frequency of Breast Self-Examination Age Monthly Occasionally Never <45 45-59 60+ 91 150 109 90 200 198 51 155 172 Source: R. T. Senie et al., Am. J. Public Health, 71:583-590 (1981). Reprinted with permission of the American Public Health Association. 8.7. For a sample of 40 subjects, Table 8.10 relates Y = mental impairment (1 = None , 2 = Mild, 3 = Moderate, 4 = Impaired) to socioeconomic status (SES = 1, high; SES = 0, low) and a life events index, which is a composite measure of both the number and severity of important life events (such as loss of job, divorce, death in family) that the subject experienced within the past three years. a. Fit a proportional odds model, and interpret effects. b. Add an interaction term between the predictors. Interpret this model by describing the impact of life events on mental impairment separately at each level of SES. Check whether this model provides an improved fit.