Chapter 2 THE EARLY HISTORY OF PRICE INDEX RESEARCH* W.E. Diewert 1. Introduction The purpose of this paper is to present a brief overview of the ancient history of price measurement. At least five distinct approaches to price and quantity measurement (or index number theory) can be distinguished in the early literature on the subject: (i) the tabular standard (or the commodity standard or the fixed basket approach); (ii) the statistical approach; (iii) the test approach; (iv) the Divisia index approach and (v) the economic approach. We shall discuss the history of each approach in turn in Sections 2-6 below. In Sections 7 to 11 below, we shall discuss issues that are perhaps somewhat controversial. Section 7 briefly discusses the merits of the test approach to index number theory while Section 8 presents an extended discussion of the chain principle. Sections 9 and 10 discuss the possible magnitudes of the substitution bias and the new good bias respectively while Section 11 asks whether the theory of the cost of living index has been exhausted. Section 12 concludes with a list of recommendations directed towards statistical agencies. Essays in Index Number Theory, Volume I W.E. Diewert and A.O. Nakamura (Editors) ©1993 Elsevier Science Publishers B.V. All rights reserved. *This paper was originally a comment on "Price Index Research and its Influence on Data: A Historical Review," a paper presented by Jack E. Triplett in Washington, D.C. on May 12, 1988 at the 50th Anniversary Conference of the Conference on Research in Income and Wealth. The author is indebted to R.C. Allen, B.M. Balk and J.E. Triplett for valuable historical references and discussions over the years. This research was supported by a grant from the Social Sciences and Humanities Research Council of Canada and by the Summer Institute of the National Bureau of Economic Research on Productivity. Neither institution is to be held responsible for any opinions expressed by the author. 34 Essays in Index Number Theory 2. The Early History 35 2. The Fixed Basket Approach The essence of the fixed basket approach or the tabular standard may be explained as follows. Suppose that there are N goods that consumers in a location can purchase during two periods. In periods 1 and 2, the relevant price vectors are p1 = (p\,... ,p]^) and p2 = (p2,... ,p%), respectively. Suppose further (unrealistically) that the quantities purchased of the N goods are constant during the two periods, with the constant vector of purchases being defined as q = (gi,..., (jat). Then a natural measure of the average level of prices in period 2 relative to period 1 is p2 ■ q/p1 ■ q where pl ■ q = J^^Li PnQn is the inner product of the vectors pf and q. The above approach to price measurement has been independently proposed by many people. The earliest known proposer of the method was William Fleetwood, the Bishop of Ely, who wrote the book Chronicon Preciosum in 1707.1 The constant basket of goods he used to compare the value of money (or, conversely, the level of prices) for an Oxford student of 1707 compared to an Oxford student of 1460 was 5 quarters of wheat, 4 hogsheads of beer and 6 yards of cloth. Perhaps the next independent discovery of the tabular standard was made by the Legislature of Massachusetts in 1780. An account of this discovery is given by Willard Fisher [1913]. A tabular standard was used to index the pay of soldiers fighting in the Revolutionary War (a massive inflation had drastically reduced the real value of the fixed nominal pay of the soldiers). The constant quantity basket was 5 bushels of corn, 68 and 4/7 pounds of beef, 10 pounds of sheep's wool and 16 pounds of sole leather.2 Joseph Lowe [1823; 316] was not an independent discoverer of the constant basket index number formula (1) p2 ■ q/p1 ■ q, since he explicitly refers to Fleetwood's book. However, he developed the concept in such detail that he should be considered the father of the consumer price index. Lowe was well aware that the constant basket of commodities q could vary across demographic groups; on page 332, he presented some representative family budgets for cottagers and for the middle class. On page 97 of the Appendix, he noted that price indexes may be required for other classes of consumers or producers such as farmers and miners while on page 336 of the main text, he advocated the construction of separate "standards" for the laboring class, decomposed into unmarried laborers and married laborers with 2, 1A good account of Fleetwood's contributions with extensive quotations can be found in Ferger [1946]. 2See W. Fisher [1913; 437]. 3 or 4 children. Finally, Lowe [1823; 33] also envisaged a national "table of reference" which would price out a constant national consumption vector at the prices of each year t and on pages 94 and 95 of the Appendix, he constructed two such hypothetical tables. How would the constant vector of commodities q in (1) be determined? Lowe answered this question as follows: As to quantity, a variation can take place only with increase of population or change of habits, and any alteration of that kind must be so gradual, that we run very little hazard in assuming a similarity of amount during a given period, which for the sake of precision, we shall suppose to be five years. Lowe [1823; Appendix 95] Lowe [1823; 334] also proposed that the national government should fund the collection of the relevant price and quantity statistics, but if this was not done, then Lowe felt that government agencies should at least provide what data they had at their disposal "on the demand of any respectable association." Lowe [1823; 335-343] listed a host of applications for his proposed tables of reference, including the following: (i) wages, salaries and rents could be indexed to eliminate the anomalies arising out of unforeseen fluctuations in the value of the country's currency, (ii) they would facilitate salary negotiations, (iii) they could be used to index long term agricultural leases and (iv) bond holders could be paid in real terms if they wanted that option. Lowe [1823; 346] concluded with some pertinent observations on why his proposal had not been implemented up to his time: This has, we believe, been owing to two causes; the unfortunate neglect of political economy in the education of our public men; and the interest of government, the greatest of all debtors, to prevent the public from fixing its attention on the gradual depreciation of money that went on during the half century to the late peace. Lowe [1823; 346] Scrope [1833; 406-407] followed in Lowe's footsteps but was the first to use the term tabular standard to describe the price index defined by (1). However, his treatment was not nearly as detailed as that of Lowe,3 so we will pass on to list others who have endorsed the tabular standard. 3Scrope does not refer Lowe (or anybody else for that matter) since, as a Member of Parliament, he seemed to be writing an extended political speech. However, it seems likely that he knew of Lowe's work since his proposals and terminology were so similar to those of Lowe. Scrope [1833; 320] also proposed a comprehensive system of social security for workers "when rendered incapable of labor by illness, age or accident." Another early proposer of the 36 Essays in Index Number Theory If quantities were to remain constant during the two periods under consideration, a whole host of authors endorsed formula (1) to measure price change, including Jevons [1865; 122] [1884; 122], Sidgwick [1883; 67-68], Edgeworth [1925; 212]4 (originally published in 1887), Marshall [1887; 363], Bowley [1899] [1901; 227] [1928; 223], Walsh [1901; 540] [1921; 543] [1924; 544] and Pigou [1912; 38]. During this period, the precise specification of the constant quantity vector q was a problem which was addressed. Thus Laspeyres [1871] proposed that q should equal q1 = (q\ ..., gjy), the base period quantity vector, while Paasche [1874] proposed that q should equal q2 = (q\,..., qfj), the current period quantity vector. Thus (1) can be specialized to yield the famous Laspeyres and Paasche price indexes, Pp and Pp: (2) PL{p\p2,q\q2)=p2-q1/p1-q1- (3) PP{p\p2,q\q2)=p2 ■q2/p1 ■ q2. Given that quantities would not be exactly equal during the two periods under consideration, various authors started to argue that averages of (2) and (3) should be used to measure price change. Thus Sidgwick [1883; 68] and Bowley [1901; 227] proposed the use of (1/2)PL + (1/2)PP, while Edgeworth [1925; 214] (originally published in 1887) proposed that the q in (1) be set equal to the arithmetic average of the two quantity vectors, (l/2)g1 + (l/2)q2, (Edgeworth states that this variant was also independently proposed by Alfred Marshall). Bowley [1899] suggested the geometric mean of Pl and Pp, which later came to be known as Irving Fisher's [1922] ideal price index Pp defined as (4) Pp{p\p2, q\ q2) = [p2 ■ qV ■ f/p1 ■ iV ■ ^ ■ Walsh [1901; 398] proposed that the components qi of the quantity vector q in (1) should be set equal to the geometric means of the quantities in the two periods. Thus the Walsh price index is n n (5) PW{p\p2,q\q2) ee J2(lll")1/2p"/J2(lU")1/2Pr i=l j=l Finally, Pigou [1912; 46] suggested PpPp as a measure of price change. Since this price index has rather poor homogeneity properties, Pigou later tabular standard who does not refer to Fleetwood or Lowe is the American economist Willard Phillips who made his proposal in 1828; see Jastram [1951] for a description of the contributions of Phillips. 4Edgeworth [1925; 331] called the method the consumption standard rather than the tabular standard. 2. The Early History 37 modified his measure by taking the square root which yields Pp defined by (4); see Pigou [1932; 69]. At this stage, the fixed basket approach to index number theory merged into the test and economic approaches. 3. The Statistical Approach This approach, which originated with Jevons [1865] [1884], assumed that increases in the supply of money increased all prices proportionately except for random fluctuations. Thus with additive errors and a sufficient number of independent observations, an appropriate price index could be obtained by taking the arithmetic mean of the price ratios p\jp\ while with multiplicative errors, an appropriate price index could be obtained by taking the geometric mean of the price ratios. This second alternative was advocated by Jevons, and thus we obtain the Jevons price index Pj: n (6) Pj{p\p2)^\[{p2/p]Y'N. i=l In addition to Jevons, two other prominent economists who advocated the statistical approach to index numbers were Bowley [1901; 223-226] [1921; 202] [1928; 217-223] and Edgeworth [1888] [1896] [1901] [1923] [1925].5 Edgeworth mainly advocated the median of the price ratios pf/p} as the best estimator of price change. The statistical approach was criticized by Irving Fisher [1911; 194-196] who explained in an absolutely convincing manner why all prices cannot move proportionately (due to the existence of fixed price contracts, for example). Fisher's criticisms were ignored by the profession as were those of Walsh [1924]. However, Keynes [1930; 71-81] effectively demolished the naive statistical approach by constructing various tables of index numbers which showed systematic differences over time and hence the hypothesis of approximate proportional change in all prices could not be maintained empirically.6 Bowley [1928; 221] 5Bowley was not exclusively an advocate of the statistical approach as we shall see later. Also, initially Edgeworth took a broader view of the index number theory (recall his endorsement of the tabular standard if quantities remained constant), but after this initial broadness, he became a very strident defender of the statistical approach; in particular, Edgeworth's [1923] criticisms of Walsh's test approach became quite heated. 6Keynes [1930; 72] also used some rather colorful language to criticize the 38 Essays in Index Number Theory 2. The Early History 39 also criticized the approach on narrower statistical grounds by indicating that the price movements were not statistically independent. Although Jevons' naive statistical approach is no longer advocated, statistical sampling of the prices of the various components of a price index is still done today. A problem with many of these sampling procedures is that prices are sampled independently of quantities. Pigou [1932; 77] was perhaps the first to propose that values should be sampled in the two periods under consideration, along with the corresponding prices and quantities, and then the Fisher ideal index Pp defined by (4) should be used to construct a measure of price change over the commodities in the sample of values. This sample price index could then be used to deflate the population value ratio over the two periods. Pigou's proposal deserves serious consideration by statistical agencies even today. 4. The Test Approach The origins of the test approach are rooted in the more or less casual observations of the early workers in the index number field on their favorite index number formulae or those of their competitors. Thus Jevons [1884; 152] (originally published in 1865) recognized that his unweighted geometric mean formula (6) gave index number comparisons between any two years that were independent of the base year. Edgeworth [1896; 137] gave a clear general treatment of this base invariance test7 which we can phrase as follows. Let P(p°,pl, q°, ql) be a generic index number formula of the type defined by (2) to (5) above which compares the level of prices in period t to the level of prices in period 0, the base year. Let p* and g* be the price and quantity vectors pertaining to year t for t — 0,1,... ,T. Let i, s and t denote arbitrary years. With the base year equal to 0, the level of prices in year t relative to s is taken to be P(p° ,pf ,q° ,qf)/P(p° ,ps ,q° ,qs). If we change the base to year i, then the level of prices in period t relative to s is P(pl,pt, q1, ql)/P{pl, ps, q1, qs). The base invariance test demands that these statistical approach as the following quotation indicates: "I have long believed that this is a will-o'-the-wisp, a circle-squaring expedition which has given an elusive taint, difficult to touch or catch, to the theory of price index numbers traditional in England. This is not equally true of America. Nevertheless, whilst the Americans have not worshipped the mythical creature, they have not (with the exception, perhaps, of Mr. Walsh) actively combatted him or dragged him out of the twilit cave where Edgeworth judiciously kept him." 7Edgeworth mistakenly attributed this test to Westergaard [1890], but West-ergaard's circular test is slightly different as we shall see. two numbers be equal; i.e., that (7) P(pV, q°, q^/Pip0,?3, q°, qs) = P{p\p\ q\ q')/P(p\ps, q\ qs)■ Our next test was first proposed by Laspeyres [1871; 308], and has come to be known as the strong identity test: if prices in the two periods under consideration remain constant, then even if the quantities change,8 the level of prices should remain unchanged; i.e., we should have (8) P(p,p,q1,c(i) = l where P denotes the index number formula or function, p = (pi,... ,pn) denotes the common price vector in both periods and /2)(4 + Sj2)ln(Pj2M) where the shares s\ are defined as s\ = pjqj/p* ■ qf, t — 1, 2 and i — 1, Since the indexes Pl , Pp and Pt can differ considerably, the Divisia approach does not lead to a practical resolution of the price measurement problem.18 To conclude this section on the Bennet-Divisia approach, we note that Bennet [1920; 457] suggested the following discrete approximations to measure differences (rather than the ratios of Divisia) in the aggregate price and quantity levels: n (20) AP = P(2) P(l) = £(l/2)(«£ + g?)(p? - p}); i=l n (21) AQ = Q(2) - 0(1) = £(1/2)(PJ - q\). Bennet also showed that the difference in expenditures for the two periods, ^2i=iPiQi ~ Si=iPiliJ was exactly equal to AP + AQ, where AP and AQ are defined by the right hand sides of (20) and (21).19 17Hofsten [1952] appears to be the first researcher who noticed this difficulty with the Divisia price index. 18See also Samuelson and Swamy [1974; 579] for alternative suggestions on how to approximate the continuous time Divisia indexes with discrete data. 19 The right hand side of (21) can be regarded as an approximation to the arithmetic average of the compensating and equivalent variations defined by Hicks [1941-42]. Thus the measurement of differences in aggregate quantities led to consumer surplus theory while the measurement of ratios led to the economic theory of index numbers. 44 Essays in Index Number Theory 2. The Early History 45 6. The Economic Approach The economic approach to index number theory20 relies on the assumption of optimizing behavior on the part of economic agents: utility maximizing or expenditure minimizing behavior on the part of consumers and profit maximizing or cost minimizing behavior on the part of producers. The first two papers to use an explicit utility maximizing framework appear to be by Bennet [1920] and Koniis [1924]. Bennet's paper drew on an earlier paper by Bowley [1919] (he used Bowley's notation and data) and may be regarded as an attempt to determine the approximate magnitude of the substitution bias using the assumption of a quadratic utility function. Bowley [1928; 226] [1938] was in turn influenced by Bennet and developed his own quadratic approximations. Bennet's paper was very short and sketchy and did not have the impact that the Koniis paper eventually had. Koniis [1924; 16-18] not only presented a very clear definition of the true cost of living for an individual optimizing consumer, he also developed the now well known Paasche and Laspeyres bounds.21 Koniis [1924; 20-21] also showed that the Paasche and Laspeyres price indexes, (3) and (2) above, bound the true cost of living index even in the general nonhomothetic preferences case, provided that we evaluate the true index at a suitable utility level that is between the base and current period levels. To complete our brief survey of the early history of the economic approach to index number theory, we shall review the economic approach under four subheadings: (i) basic theoretical definitions, (ii) the theory of bounds, (iii) exact index numbers and (iv) econometric approaches. 6.1 Basic theoretical definitions There are three main branches of the economic approach, (i) For the true cost of living index, see Koniis [1924], Samuelson [1947; 156] and Pollak [1971a]. For related quantity indexes, see Bowley [1928; 230], Allen [1949], Malmquist [1953] and Pollak [1971a]. (ii) For theoretical definitions of the output price index, see Hicks [1940], Fisher and Shell [1972b], Samuelson and Swamy [1974; 588-592], Archibald [1977] and Diewert [1983b]. For related quantity indexes, see Bowley [1921; 203], Bergson [1961; 31-34], Moorsteen [1961], Fisher and Shell [1972b; 53], Samuelson and Swamy [1974, 588-591], Sato [1976b; 438] and Hicks [1981; 256]. (iii) The input cost index was defined by Court and 20For surveys, see Frisch [1936], Samuelson [1947; 146-163], Ruggles [1967], Pollak [1971a], Samuelson and Swamy [1974], Afriat [1977], Deaton and Muell-bauer [1980; 167-184] and Diewert [1980] [1981a] [1983a] [1983b] [1987]. 21See, for example, Diewert [1983a; 171]. Lewis [1942-43], Triplett [1983; 274] and Diewert [1980; 459] and corresponding quantity indexes were defined in Diewert [1980; 456-460]. There is a fourth branch of the economic approach that has received less attention: (iv) constant utility income deflators. On this last branch of theoretical index number theory, see Diewert and Bossons [1992]. 6.2 The theory of bounds Observable bounds to the generally unobservable economic price and quantity indexes were first worked out by Pigou [1912; 44-46] [1932; 62-63] and Haberler [1927; 78-92] independently of Koniis [1924; 17-19], who established the Paasche and Laspeyres bounds for the true cost of living. For a generalization of these bounds to nonlinear budget constraints, see Frisch [1936; 18].22 It is clear that a large portion of revealed preference theory that is often attributed to Hicks [1940] and Samuelson [1947; 157] had already been developed by Pigou, Koniis, Haberler and Frisch. Other researchers who established bounds on true indexes in the two observation situation include Leontief [1936; 49], Friedman [1938; 125], Allen [1949], Malmquist [1953], Moorsteen [1961; 464], Pollak [1971a], Fisher and Shell [1972b; 57-62], Samuelson and Swamy [1974; 581-591], Archibald [1977] and Diewert [1981a; 167-179] [1983a; 173-210] [1983b; 1056-1090]. The above theory of bounds all pertains to the two observation situation. Afriat [1967] [1977] generalized the two observation theory to cover the many observation case. 6.3 Exact index numbers Let an aggregator function23 i.e., q = (qi,..., (Jat). The cost function C which is generated by / may be defined as (22) C(u, p) = min{p • q : f(q) > u}; i.e., C{u,p) is the solution to the problem of minimizing the cost p ■ q = ~!2i=iPi1i °f achieving at least the utility (or output) level u, where p = (pi,..., pn) is an exogenous vector of prices facing the consumer (or producer). 22 The English translation of the original Russian article by Koniis [1924] did not become available until 1939. Thus Frisch [1936; 25] did not have access to the original Koniis article and he mistakenly attributed the Koniis limits or bounds on the true cost of living to Haberler [1927]. 23Diewert [1976a; 115] introduced this term to cover both the production and utility function context. f(q) be given where q is an N dimensional quantity vector; 46 Essays in Index Number Theory 2. The Early History 47 An index number formula or function P(p1 ,p2, g1,g2) of the type we considered in Section 4 is defined to be exact24 for an aggregator function / if (23) P(p\p2, q\ q2) = C(u, p2)/C(u, p1) for some utility or output level u where g* solves (22) when p — pf for t — 1,2; i.e., P is exact for / (or its dual cost function C) if P equals the relevant economic index under the assumption of optimizing behavior on the part of an economic agent using the aggregator function /. The right hand side of (23) is Pxip1 ,p2, u), the Koniis price index or true cost of living index for a consumer that has the utility function / and faces the vector of prices pf in period t for t = 1,2. The English language literature on exact index numbers has its roots in the theory of quadratic approximations. As we indicated earlier, Bennet [1920; 460] attempted to determine an appropriate index number formula for the true cost of living of a single "satisfaction" maximizing consumer under the hypothesis that the underlying utility function /(g) was a general quadratic function. Bowley [1928; 226] [1938] followed up on Bennet's approach and provided his own second order approximation. Frisch [1936; 27-29] criticized Bowley's index number formula and developed an alternative formula which he called the double expenditure method. Wald [1939; 329] and Balk [1981; 1556] correctly pointed out that Frisch's index number formula was not exact for a general quadratic utility function. However, Frisch [1936; 29-30] did correctly show25 that his index number formula collapsed to the Fisher ideal price index Pp defined by (4) if one assumed homothetic quadratic preferences so that /(g) = Sj=i 5^j=i aij1i1j = 1 ' Aq where A = [a^] is a symmetric N x N matrix of parameters that characterize tastes. This is an early example of an exact index number formula. Another early example was given by Wald [1939; 325] who assumed the following general quadratic aggregator function: (24) f(q) = a0 + a-q+(l/2)q-Aq, A = A\ where ao, a = [d\,..., aV] and A = [a^] are respectively a parameter, a vector of parameters and a symmetric N x N matrix of parameters. Unfortunately, in order to evaluate Wald's general index number formula that is exact for (24), information on income elasticities is required (we shall not write out his general index number formula since it is rather complex). However, if we assume homothetic preferences again (i.e., the aggregator function is a monotonically increasing function of a linearly homogeneous function) 24The first mention of the word "exact" in this index number context appears to be by Samuelson [1947; 155]. Afriat [1972b; 45] also uses the term exact. 25See Balk [1981; 1556] for a clear proof of Frisch's result. so that ao — 0 and a — On in (24)26 then all of the consumer's income elasticities equal unity and Wald's general index number formula collapses down to the Fisher price index (4) and again we obtain the exact index number result of Frisch.27 Unknown to the above authors, the Frisch-Wald exact index number result had already been obtained by Koniis and Byushgens [1926; 167-172] a decade earlier.28 In this remarkable paper, they introduced duality theory into the economics literature; i.e., they expressed consumer preferences not only by the direct utility function /(g) but also by the corresponding indirect utility function g defined as follows: (25) g(p, y) = max{/(g) : p ■ q < y}; i.e., the indirect utility function g(p, y) gives the maximum utility attainable as a function of the prices faced by the consumer p = (pi,... ,pn) and the income or expenditure y > 0 to be spent on the N goods during the period under consideration. Koniis and Byushgens assumed that the direct utility function / was linearly homogeneous in which case the indirect utility function g can be expressed as follows in terms of the unit cost function c{p) = C(l,p) where C(u,p) was defined by (22) above: (26) g{p,y) = y/c(p). Koniis and Byushgens considered three classes of homothetic preferences which were defined via the indirect utility function g(p,y) or equivalently, using (26), via the unit cost function c{p). The first case they considered had the following unit cost function: n (27) c(p)=^2alpl, at > 0, i — l,...,N. i=l As is well known,29 the dual direct utility function is the fixed coefficient or no substitution / defined as follows: (28) f(qi, ...,qn)= mm{qt/at : i = 1,. .., N}. i 26With these restrictions, f(q) — q ■ Aq — h[g(q)] where h(x) = x2 and g(q) = (q ■ Aq)1/2. Thus in this case, / is a monotonically increasing function of the linearly homogeneous function g(q) over the set of q such that q ■ Aq > 0. 27For a nice exposition and evaluation of the Frisch and Wald quadratic approximation results as well as of some proposals by Samuelson [1974a; 16], see Balk [1981]. 28 The Koniis and Byushgens [1926] paper was published in Russian and non Russian speaking economists seemed to be unaware of the paper although it was indirectly mentioned by Schultz [1939; 8]. 29See Pollak [1971a; 105]. 48 Essays in Index Number Theory 2. The Early History 49 Under these conditions, Koniis and Byushgens [1926; 162] showed that the Laspeyres and Paasche indexes, Pp and Pp defined by (2) and (3) above, will exactly equal the true cost of living (defined by the right hand side of (23) for any positive utility level u), provided that the consumer's direct utility function is defined by (26) and (27). The second case they considered was the case of Cobb-Douglas preferences30 which can be characterized by the following unit cost function: n n (29) c(p) — aY\.Pi^ «i>o, at — i. i=l i=l Koniis and Byushgens [1926; 165] showed that the generalized Jevons index defined by (6) (except that 1/N is replaced by o^) is exact for these Wicksell-Cobb-Douglas preferences, where the unknown parameters a\ can be determined as follows: a\ — p\q\/pf ■ qf, the ith expenditure share, i — 1,..., N, for any period t. In the final case considered by Koniis and Byushgens [1926; 168], the consumer's preferences were characterized by the following unit cost function: n n (30) c(p) ^ (p ■ Bp)1'2 = (J2J2b jP%Pj i=l i=l 1/2 where B = [bij] is a symmetric N x N matrix of unknown parameters that characterize preferences. They showed that the Fisher ideal index Pp defined by (4) was exact for the preferences characterized by (26) and (30). Koniis and Byushgens [1926; 171] also showed that if the inverse of the matrix B existed, say A — B~x, then the direct utility function corresponding to (26) and (30) (31) f(q) ^{q-Aq)1'2 which is a monotonie transformation of the homogeneous quadratic utility function considered by Frisch and Wald. Finally, Koniis and Byushgens [1926; 171] exhibited both the system of inverse demand functions, p/y — Aq/q-Aq, and the system of ordinary demand functions, q — yBp/p ■ Bp, that correspond to the homogeneous quadratic preferences defined by (31). They also suggested that the unknown parameters appearing in the A or B matrices could be determined given a sufficient number of price and quantity observations. However, note that a knowledge of A or B is not required in order to evaluate the Fisher price index Pp. 30The Cobb-Douglas functional form was perhaps first used in the economics literature by Wicksell [1958; 98] in the nineteenth century. After the contributions of Bowley, Frisch and Wald to the theory of exact index numbers, the subject remained dormant until Afriat [1972b; 44-47], Pollak [1971a; 117-132] and Samuelson and Swamy [1974; 573-574] reexamined the subject. All of these authors examined the three cases considered by Koniis and Byushgens and some other cases as well. Diewert [1976a; 134] defined a price index function P(px,p2, q1, q2) to be superlative if P was exact for preferences which had a cost function C(u,p) — uc{p) where c{p) is a unit cost function that could provide a second order approximation to an arbitrary twice continuously differentiable linearly homogeneous function. The idea was that a superlative index number formula P(p1 ,p2, q1, q2), which could be evaluated using only observable price and quantity data for the two periods under consideration, would correspond to a flexible functional form for a unit cost function c{p). For example, Pp defined by (4) is a superlative price index since it is exact for the c defined by (30) and this c has the required second order approximation property. Another example of a superlative index is the Walsh index P\y defined by (5) since it is exact for the unit cost function which is dual to the aggregator function f(q) = J]fc=i SjLi o,ijqi-l2qjXl2; see Diewert [1976a; 132]. A third example of a superlative price index is Pp defined by (19) which is exact for a translog unit cost function;31 see Diewert [1976a; 121]. Unfortunately, Diewert [1976a] defined two (infinite) families of superlative index number formulae and this raised the question as to which formula should be used in empirical applications. However, Diewert [1978b] showed that all choices of a superlative formulae gave the same answer to the second order and hence the choice was usually immaterial. More precisely, Diewert showed that every known superlative index number formula P(p1 ,p2jq1 ,q2) had the same first and second derivatives when evaluated at equal prices (i.e., P1 — P2) and equal quantities (i.e., q1 — q2).32 As an interesting footnote to the history of economic thought, it should be noted that Diewert was not the first to use the above second order approximation technique when evaluating index number formulas and their derivatives; Edgeworth [1901; 410-411] used a variant of it to show that the Walsh index Pw defined by (5) approximated to the second order the Edgeworth [1925; 213] - Marshall [1887; 372] index defined as follows: (32) PEM{p,pZ,q,qZ) n J2(im(ql+q2)p2 i=l I n i=i (l/2)(q) + q2)Pj 31 The translog unit cost function was introduced by Christensen, Jorgenson and Lau [1971]. 32Vartia [1978] provided an alternative derivation of this result. 50 Essays in Index Number Theory 2. The Early History 51 The Taylor series expansion technique around an equal price and quantity point was used again by Edgeworth [1923; 347] to show that the Laspeyres index Pp defined by (2) and the Fisher index Pp defined by (4) satisfy the circularity test (9) to the first order. Since this proposition seems to have been forgotten, we sketch a proof of it. Let the index number function P be either Pl or Pp and define the functions / and g as follows: (33) f{p\p2,p3,q\q2,q3)=P{p\p2,q\q2)P{p2,p3,q2,q3)-g(p1,p2,p3,q1,q2,q3)=P(p1,p3,q\q3). For P — Pp or for P — Pp, it can be verified that the following equalities hold: (34) f{p\p2,p3,q\q2,q3)=g{p\p2,p3,q\q2,q3) =1; Vpi/b1,^2,^3,?1,?2,?3) = ^Pig(p\p2,p3,q\q2,q3) = -q/p ■ q; Vp2 /(p1, p2, p3, q1, q2, q3) = Vp2g(p1, p2, p3, q1, q2, q3) = Oat5 ^ Ps f{px ,v2 ,P* ,qx ,q2 = ^ p^giv1 ,v2 ,q1 ,q2 = g/p- g; Vgi/b1,?2,?3,?1,?2,?3) = VgifffrSp2,?3,?1,?2,?3) = 0*, 2 = 1,2,3, provided that the above functions are evaluated at equal prices (i.e., p1 — p2 — p3 = p) and equal quantities (i.e., q1 — q2 — q3 = q) where Vp./ = (df/dpi,... ,df/dplN) is the vector of first order partial derivatives of / with respect to the components of p1, etc. The meaning of (34) is that the functions / and g approximate each other to the first order when evaluated at an equal price and quantity point. Thus the Laspeyres and Fisher price indexes satisfy the circularity test to the first order. Using the results in Diewert [1978b; 898] and the results in the above paragraph, it can be shown that the Paasche, Laspeyres and all superlative index number formulae will similarly satisfy the circularity test to the first order. Note that Edgeworth's proposition helps to explain Fisher's [1922; 280] empirical finding that Pp satisfied the circularity test to a very high degree of approximation. 6.4 Econometric estimation of preferences In this variant of the economic approach to index numbers, the parameters that characterize consumer preferences are estimated. Preferences may be represented by: (i) the direct utility function; (ii) the indirect utility function; (iii) the distance function33 or (iv) the cost or expenditure function. Once any 33Malmquist [1953] was the first to use the distance function in index number theory. one of these functions is known, the other three functions can be calculated, at least in principle. In particular, the cost function C{u,p) can be calculated and hence the Koniis price index defined by the right hand side of (23) can be calculated. Additional references to the recent literature on the econometric approach to the estimation of preferences can be found in Chapter 7 of Deaton and Muellbauer [1980]. It seems appropriate to add a few references to the early history of this approach to index number theory. The earliest effort at a strategy for determining the parameters which characterize preferences was made by Bennet [1920; 462]. He assumed a quadratic direct utility function and showed how the consumer's system of demand functions could be obtained in the three good case (although he did not quite exhibit a closed form solution). He then made some comments on how many observations would be required in the general N good case in order to determine all 1 + N + (l/2)N(N + 1) of the parameters of the quadratic utility function f(q) defined by (24).34 We have already seen that Koniis and Byushgens [1926] were contributors to the econometric approach, since they derived the demand functions corresponding to Cobb-Douglas preferences and homogeneous quadratic preferences. Koniis and Byushgens [1926; 172] noted that one price-quantity observation would suffice to determine the parameters of Cobb-Douglas preferences while (N + l)/2 observations would be required to determine all N(N + l)/2 parameters for the homogeneous quadratic functional forms, (30) or (31). They also noted that statistical determination of the homogeneous quadratic preferences would be difficult. Another early contributor to the econometric approach to index numbers was Wald [1937] [1939; 325], who assumed quadratic preferences; recall (24). In addition, he assumed that the demand functions regarded as functions of income (or expenditure) were known functions for the two periods under consideration. (Alternatively, just a knowledge of the income elasticities of demand at the two observed price and quantity points would suffice.) With these assumptions, Wald was able to derive an exact index number formula, so his approach is actually a blend of the exact and econometric approaches. Wald's blended approach seems worthy of further study. However, the pure econometric approach has severe limitations. The problem with the latter approach is that in order to provide a second order approximation to general preferences, we require approximately N2/2 parameters in the N good case. Since N is perhaps equal to 50,000 for a typical consumer (a supermarket alone has 15,000 to 20,000 separate items), the required number of parameters to be 34Bennet [1920; 462] recognized that not all of the parameters of a quadratic utility function could be identified (due to the unobservable nature of utility) since he wrote of determining the parameters only up to a ratio. 52 Essays in Index Number Theory estimated is approximately 1.25 billion, which would require price and quantity observations for about 25,000 periods.35 This concludes our survey of the ancient history of index numbers. We turn now to a discussion of some more controversial issues in index number theory. 7. On the Test Approach to Index Number Theory Although the economic approach to index number theory is perhaps the most compelling approach, it should be mentioned that the test approach has some advantages. In particular, the test approach does not suffer from the following limitations of the economic approach: (i) the economic approach is based on optimizing behavior, an assumption which may not be warranted in general; (ii) the economic approach generally relies on separability assumptions36 about the underlying aggregator functions, assumptions which are unlikely to be true in general and (iii) in deriving capital rental prices, the economic approach is usually based on ex ante expectations about future prices, expectations which cannot be observed, whereas the test approach can be based on ex post accounting data, which can be observed.37 8. On the Chain Principle and Multilateral Indexes The chain principle can be contrasted with the fixed base principle for constructing a series of index numbers which extends over three or more periods. Given price and quantity data, pl, ql, i — 1, 2, 3 for three periods and a bilateral price index function P{p1,p2, q1 ,q2) that depends only on the data for two periods, the fixed base sequence of aggregate price levels for the three periods 35These perhaps overly pessimistic calculations lead to the following impossibility theorem: economists will never know the truth, even to the second order. However, recently Diewert and Wales [1988] have devised techniques for obtaining good approximations to completely flexible preferences using only a minimal number of parameters. 36For a definitive treatment of separability concepts and duality theory, see Blackorby, Primont and Russell [1978]. 37See the theoretical discussion in Diewert [1980; 475-476] and the empirical results in Harper, Berndt and Wood [1989] on alternative rental price formulae. 2. The Early History 53 would be (35) 1, P{p\p2,q\q2), P{p\p3,q\q3) while the chain sequence of price levels would be (36) 1, P{p\p2,q\q2), P{p\p2,q\q2)P{p2,p3,q2,q3). Historically, the fixed base principle was the first to be used empirically. In the English language literature, the chain principle was first proposed by Alfred Marshall [1887; 373],38 basically as a method for overcoming the difficulties in comparing prices over two distant periods, due to the invention of new commodities.39 Irving Fisher [1911; 203], who gave the chain system its name, noted that the chain system was invariant to changes in the base period and he also saw the advantage of the method in dealing with the new good problem as the following quotation indicates:40 It may be said that the cardinal virtue of the successive base or chain system is the facility it affords for the introduction of new commodities, the dropping out of obsolete commodities, and the continued readjustment of the system of weighting to new commodities. Fisher [1911; 204] While it is true that the use of the chain principle has an advantage in dealing with the introduction of new commodities, it has the following severe disadvantage: it does not satisfy Walsh's multiperiod identity test, (13) above.41 Thus, as Szulc [1983] and Hill [1988] show, if prices and quantities 38Walsh [1901; 207] attributed the chain principle to Julius Lehr [1885; 45-46], who was motivated to introduce the principle in order to deal with new goods. 39Divisia [1926; 44-47] saw his method as being a variant of the chain method, with the basic discrete period being one year (which would minimize seasonal fluctuations). Divisia [1926; 45] also thought that the chain method was the only logical way to make price comparisons over long periods due to the introduction of new goods and the discovery of new inventions. As new goods in his time, he mentioned machine guns (to replace the bow and arrow), submarines, aircraft, the potato and the steam engine. 40However, later Fisher [1922; 308-309] preferred the fixed base system. Pigou [1932; 71] criticized Fisher's later position and endorsed the chain principle for the usual reason: with the introduction of new commodities, the chain principle is the only way to make comparisons between distant periods. 41Walsh [1901; 401] [1924; 506] made this criticism of the chain system. Szulc [1983; 540] also uses Walsh's test in his evaluation of fixed base versus chain index numbers. 54 Essays in Index Number Theory 2. The Early History 55 systematically oscillate around constant values, the use of the chain method will give biased results. The above difficulty with the chain method was not adequately appreciated by Diewert [1978b; 895] who argued for the use of the chain principle on the grounds that it would reduce the spread between the Laspeyres and Paasche indexes, (2) and (3) above, and between all known superlative indexes, since price and quantity changes will generally be smaller between adjacent periods than between distant periods. He argued that the spread between the Paasche and Laspeyres indexes would be greater than between the superlative indexes because Pp and Pp only approximate each other to the first order, while superlative indexes approximate each other to the second order (recall our discussion of the Edgeworth second order approximation technique in Section 6.3 above). Diewert [1978b; 894] also presented the results of some numerical experiments using Canadian per capital consumption data for 13 commodity classes over the years 1947-1971. These results showed that the chain method did in fact lead to a smaller spread between Pp defined by (2), Pp defined by (3), Pp defined by (4) and Pp defined by (19) than when a fixed base year, 1947, was used. Although the chain method will give poor results with oscillating data, Szulc [1983] and Hill [1988] show theoretically that chaining will tend to reduce the spread between the Laspeyres and Paasche price indexes, provided that prices and quantities trend monotonically over the time periods in question. Thus Diewert's [1978b; 894] empirical results could be rationalized by the hypothesis that monotonie trends in the data outweighed oscillatory movements. While the chain system fails to satisfy Walsh's identity test (13), Hill [1988] showed that the fixed base system fails to satisfy an analogue to the strong identity test (8). Consider the base period to be period 0 and suppose that the bilateral price and quantity indexes, P and Q respectively, are given and they satisfy the product test (14). Then under the fixed base system, price P* and quantity Q* comparisons between periods t and t + 1 are made as follows: (37) P*(p°,pW+1, q°, q\ 0, in place of (39). 43Fisher [1922; 280-283] also found that PF satisfied Walsh's multiperiod identity test (13) to a high degree of approximation. 44Actually, Fisher [1922; 301] used a slightly different normalization. 56 Essays in Index Number Theory 2. The Early History 57 Fisher then suggested that the price level for period t, P* say, should be an arithmetic average of the period t price level in each of the T series in (40); i.e., we have P1 — 1 and for t — 2, 3,..., T: t (41) p* = (i/T) P(pk,pt,vk,qt)/P(pk,p1,qk, fe=i Note that the multiperiod or multilateral period t aggregate price level P* defined by (41) is a function of the data for all T periods; i.e., P* — Pt(p1,... ,pT, q1, ..., qT). The corresponding multiperiod or multilateral period t aggregate quantity level Qf may be defined as (42) QV, ■■■,pT,q\...,qT)=pt-qllP\p\ . . . ,pT, q1, . . . , qT). Fisher's [1922; 307] second multilateral method was the broadened base system. In this method, the period t price level P* was defined as follows: t (43) Pt(p\...,pT,q\...,qT)-,(l/T)(KJ2lk) -P\ t=l,...,T. k=l The corresponding quantity levels Q1 can be defined numerically using the multilateral product test relations (42). This method was used by the Economic Commission for Latin America in the early 1960's. It is described in Ruggles [1967; 185] and is referred to as the market basket method. Note that (43) is a T period generalization of the two period Edgeworth-Marshall bilateral formula, Pem, defined by (32) above. Fisher was not the first to suggest the market basket method: Walsh [1901; 431] suggested the same method but called it Scrope's method with arithmetic weights. Walsh [1901; 399] also suggested the following system of multilateral price levels P*, which he called Scrope's method with geometric weights: (44) pt^E(LUfe) Pt t=l,...,T. i=l k=l The corresponding Q* can be defined by (42) as usual. It is obvious that (44) is the multilateral generalization of Walsh's bilateral index P\y, defined by (5) above. The above multilateral methods construct price and quantity levels for each period. It is also possible to devise multilateral methods where the multilateral price index Pst gives the level of prices in period t relative to the level of prices in period s and is a function of all of the price and quantity information for the T periods; i.e., Pst = P^ip1,... ,pT, q1,..., qT). Thus Walsh [1924; 509] defined the following multiperiod generalization of the Fisher ideal index (4): (45) Pst = {pl ■ q1pt ■ q2 . . .33* • qT/ps ■ q1ps ■ q2 . . .ps ■ qT)1/T, s, t = 1, . . . , T. The corresponding quantity indexes can be defined as follows: (46) Qst(p\ ...,pT,q\...,qT)=pt ■qllps ■ qs Pst (p1, . . . , pT, q1, . . . , qT). Gini [1931; 10] also defined the multilateral price indexes (45) and called the method the successive weights system. Both Walsh and Gini noted that (45) collapsed to Pp defined by (4) if T — 2 and s — 1 and t — 2. Finally, Gini [1931; 12] did propose a new multilateral method which he called the circular weight system, because the resulting system of price indexes Pst satisfied a multilateral analogue to the circular test (9).45 Given any bilateral price index function P(p1 ,p2, g1 ,q2), define Gini's multilateral level of prices in period t relative to the period s level as follows: (47) pst = 'p(p1>pt>ql>qt) p(p2>pt>y2>yt) P(pt,pt,qT,qt)'1/T for s, t — 1,..., T. The corresponding Qst can be defined by (46). This method was later proposed (using Pp as the P) by Elteto and Koves [1964] and Szulc [1964] in the multiregional context and is known as the EKS system. Gini [1931; 13-24] tested out his circular weight system (as well as some other alternatives) using the Fisher ideal Pp defined by (4) as his bilateral P, for eight time period observations on five Italian cities. Thus Gini's computations were both multiperiod (between time periods) and multilateral (between locations). The ratio type price indexes, (45) proposed by Walsh and (47) proposed by Gini, can be converted into price levels Pt(p1,... ,pT, q1,..., qT) as follows: corresponding to (45), define the period t price level P* as (48) Pt = (pt-q1pt-q2...pt-qT)1'T, t=l,...,T, and corresponding to (47), define P* as (49) P* ee [P{p1,pt,q1,qt)P{p2,pt, q2,q1)... P(pT,pt, qT, g*)]1/T In each case, it can be verified that Pst — P*/Ps. 45In fact, all of the multilateral indexes defined in this section satisfy the multilateral circularity test. 58 Essays in Index Number Theory 2. The Early History 59 If we take the Gini-EKS price levels defined by (49) and divide each of them through by P1, it can be seen that the resulting normalized price levels P*/P1 are closely related to Fisher's (normalized) blended price levels defined by (41): for the Fisher price levels, we take the arithmetic means of the numbers P(pk,pf, qk, qf) /P(pk,px, qk, q1), k — 1, .. . ,T, while for the Gini-EKS price levels, we take the geometric mean of the same T numbers. Walsh [1901; 399] [1924; 509] noted the primary disadvantage of using the multiperiod full information price level functions Pt(p1, .. . ,pT, q1, ■ ■ ■ , qT): if the number of periods increases, all of the indexes have to be recomputed.46 This is not necessarily a fatal objection since it is normal practice for statistical agencies to periodically issue historical revisions and there is no reason why the revisions could not be accomplished using multiperiod indexes. However, at present, it does not seem prudent to enthusiastically endorse a multiperiod system of index numbers since not enough research has been done on the axiomatic properties of the various multilateral or multiperiod alternatives.47 Moreover, it would be desirable to develop multiperiod exact and superlative index number formulae and then examine the axiomatic properties of the resulting indexes. In the bilateral case, the Fisher ideal price index Pp emerges as the natural choice of a functional form since it seems to satisfy more reasonable tests than any other known formula and it is superlative as well. We need a multilateral counterpart to this "ideal" bilateral functional form. To sum up: a comparison of the fixed base, chain, all binary comparisons and multiperiod systems leads to no clear choice at this stage. However, if a definite choice has to be made, I would vote for the chain system used with the bilateral Fisher ideal index Pp. 9. Is the Substitution Bias Small? The substitution bias in the consumer price index is the discrepancy between the Laspeyres or Paasche price indexes, Pl and Pp defined by (2) and (3) above, and the consumer's true cost of living index, defined by the right hand side of (23). There is an analogous substitution bias in the output price index. Some researchers argue that these substitution biases can be ignored. For instance, Triplett writes: "Though it has long been a staple of economists' educations, the substitution bias in a fixed-weight price index for consumption is just not very large" (Triplett [1988; 26]). To support the above opinion, 46Walsh [1901; 399] also made the following theoretical objection: "Besides, how is a past variation between two years several years ago to be affected by present variations?" 47For a start on this topic, see Diewert [1987] [1988]. Triplett cites the relatively close agreement between the Laspeyres and Paasche price indexes for U.S. aggregate consumption data; see Manser and McDonald [1988]. Triplett's judgment would be correct if in fact these indexes were true microeconomic Paasche and Laspeyres indexes, but they are not: microeco-nomic samples of price ratios for various goods i are combined with base period expenditure shares that are obtained from periodic consumer expenditure surveys. The resulting aggregate indexes are not quite Pl and Pp defined by (2) and (3) above. It may well be that Triplett is correct in his judgment, but the evidence to support his position has not yet been presented. Due to the computer revolution, it is now possible to undertake some experiments which could help to determine the extent of the substitution bias. Retail outlets that have computerized price and quantity information on their sales could be sampled. Detailed microeconomic price and quantity vectors pf and ql could be constructed and the Laspeyres, Paasche and Fisher indexes defined by (2)-(4) above could be calculated and compared with corresponding official consumer or producer price indexes that covered the same range of goods. Such firm oriented experiments could provide useful information on the size of the substitution bias.48 10. Is the New Good Bias Small? Changes in quality and the introduction of new goods are the source of another bias problem. We first briefly review the ancient literature on methods for quality adjustment.49 Some of the early researchers on price measurement were aware of the problem of quality change but the pace and direction of the change did not seem large enough to warrant an explicit treatment.50 48 Such firm based experiments would yield information on the substitution bias in output price indexes. Perhaps some day in the future when consumers use credit or banking cards to pay for all of their purchases, we could obtain an accurate paper trail that could be used to construct true microeconomic Laspeyres, Paasche and Fisher price indexes. 49For reviews of the modern literature on quality adjustment using hedonic regression techniques, see Griliches [1990] and Triplett [1990b]. 50Thus Lowe [1823; Appendix 87] states: "In regard to the quality of our manufactures, we must speak with more hesitation, and can hardly decide whether the balance be in favour of the present or of a former age; for if our fabrics are now much more neat and convenient, they are in a considerable degree less durable." 60 Essays in Index Number Theory 2. The Early History 61 However, by the latter part of the nineteenth century, Sidgwick realized that not only were improvements in the quality of goods leading to a bias in price comparisons, but also the growth of international and interregional trade (due primarily to transportation improvements) led to the systematic introduction of "entirely new kinds of things" and this too led to a bias in price comparisons. As the following quotation indicates, Sidgwick thought that utility theory would play a role in eliminating these biases: Here again there seems to be no means of attaining more than a rough and approximate solution of the problem proposed; and to reach even this we have to abandon the prima facie exact method of comparing prices, and to substitute the essentially looser procedure of comparing amounts of utility or satisfaction. Sidgwick [1883;68] Unfortunately, the mathematical apparatus of consumer theory was not sufficiently developed at that time to enable Sidgwick to make any specific progress on the new good problem. In a brilliant paper, Marshall not only proposed the tabular standard, the chain system and the Edgeworth-Marshall index number formula (32), he also made the first real progress on the appropriate treatment of new goods, as the following quotation indicates: This brings us to consider the great problem of how to modify our unit so as to allow for the invention of new commodities. The difficulty is insuperable, if we compare two distant periods without access to the detailed statistics of intermediate times, but it can be got over fairly well by systematic statistics. A new commodity almost always appears at first at something like a scarcity price, and its gradual fall in price can be made to enter year by year into readjustments of the unit of purchasing power, and to represent fairly well the increased power of satisfying our wants which we derive from the new commodity. Marshall [1887; 373] As the above quotation indicates, Marshall was well aware of the product cycle and he felt that the early introduction of new commodities into the consumer price index in the context of the chain system would capture most of the benefits due to the introduction of new commodities. As we shall see later, not quite all of the benefits are captured using Marshall's suggested method, since his method incorrectly ignores the new good in the first period that it makes its appearance. Marshall [1887; 373-374] also realized that improvements in transportation led to the general availability of location specific goods, such as fish at the seaside or strawberries at a farm. Marshall correctly felt that these "old" goods that suddenly became available at many locations should be regarded as "new" goods and treated in the same way as a genuinely new good. His words on this important observation are worth quoting: This class of consideration is of much more importance than at first sight appears; for a great part of modern agriculture and transport industries are devoted to increasing the periods of time during which different kinds of food are available. Neglect of this has, in my opinion, vitiated the statistics of the purchasing power of many in medieval times with regard to nearly all kinds of foods except corn; even the well-to-do would hardly get so simple a thing as fresh meat in winter. Marshall [1887; 374] Marshall's suggested treatment of the new good problem (i.e., use the chain system) was acknowledged and adopted by many authors including Irving Fisher [1911; 204] (temporarily) and Pigou [1912; 47]. As we saw earlier in Section 8, Divisia [1926; 45] working from his independent perspective also suggested the use of the chain method as a means of dealing with the new good problem. The next important contributor to the discussion of new goods in price measurement was Keynes. Keynes [1930; 94] described in some detail one of the most common methods for dealing with the new good problem: simply ignore any new or disappearing goods in the two time periods under consideration and calculate the price index on the basis of the goods that are common to the two situations. The corresponding quantity index was to be obtained residually by deflating the relevant value ratio by this narrowly based price index. Keynes called this method the highest common factor method. This method would be identical to Marshall's chain method if the two time periods were chosen to be adjacent ones. However Keynes [1930; 105-106] advocated his method in the context of a fixed base system of index numbers and he specifically rejected the chain method for three reasons: (i) each time a new product is introduced, a chain index does not take into account the benefits of the expanded choice set, and thus over long periods of time, the chain price index will be biased upwards and the corresponding quantity index will be biased downwards; (ii) the chain index fails Walsh's multiperiod identity test (13) above, and (iii) the chain method was statistically laborious. Keynes' last objection to the chain method is no longer relevant in this age of computers. Moreover, Keynes was unable to offer any positive alternative to the chain method for comparing situations separated by long periods of time as the following quotation indicates: We cannot hope to find a ratio of equivalent substitution for gladiators against cinemas, or for the conveniences of being able to buy 62 Essays in Index Number Theory 2. The Early History 63 motor cars against the conveniences of being able to buy slaves. Keynes [1930; 96] However, Keynes' first objection to the chain method (which was later echoed by Pigou [1932; 72]51) was certainly valid (as was his second objection). A satisfactory theoretical solution to Keynes' first objection did not occur until Hicks adapted the analytical apparatus of consumer theory to the problem. When new consumer goods make their appearance for the first time, say in period 2, their prices and quantities can be observed. In period 1, the quantities of the new goods are all obviously zero but what are the corresponding prices? Hicks [1940; 114] provided a theoretical solution: They are those prices which, in the 1 situation, would just make the demands for these commodities (from the whole community) equal to zero. These prices cannot be estimated, but we can observe that between the two situations the demands for these commodities will have increased from zero to certain positive quantities; and hence it is reasonable to suppose that the 'prices' of these commodities will usually have fallen relatively to other prices. This principle is sufficient to give us a fairly good way of dealing with the case of new goods. Hicks [1940; 114] Of course, in the context of the producer price index, the appropriate period 1 shadow prices for the new goods are those prices which just induce each period 2 producer of the new goods to produce zero quantities in period 1. Hicks' basic idea was used extensively by Hofsten [1952; 95-97] who dealt not only with new goods, but also adapted the Hicksian methodology to deal with disappearing goods as well. Hofsten [1952; 47-50] also presents a nice discussion of various methods that have been used to adjust for quality change. Frank Fisher and Karl Shell [1972b; 22-26] laid out the formal algebra for constructing the period 1 Hicksian "demand reservation prices" defined in the above quotation by Hicks. Diewert [1980; 498-501] used the Hicksian framework to look at the bias in the Fisher price index Pp defined by (4) when the reservation prices were incorrectly set equal to zero and compared this index to the Fisher price index that simply ignored the existence of the new goods in the two periods under consideration (which is Marshall's method).52 Diewert [1980; 501-503] also made some suggestions for estimating the appropriate Hicksian reservation prices in an econometric framework. 51Pigou [1932; 71] also had a nice criticism of Keynes' highest common factor method which was later repeated by Hofsten [1952; 59]. Pigou also criticized Fisher's [1922; 308-312] later preference for the fixed base method. 52The second index has a smaller bias than the first index. Is the new good bias large or small? One can only answer this question in the context of the price measurement procedures used by individual statistical agencies. In Diewert [1987; 779], some simple hypothetical examples were given which showed that traditional fixed base procedures could generate much higher measures of price increase than would be generated using the chain method.53 However, what is needed is empirical evidence. Numerical computation of alternative methods based on detailed firm data on individual prices and quantities where new goods are carefully distinguished would cast light on the size of the new good bias. Thus the firm oriented experiments suggested at the end of the previous section to cast light on the size of the substitution bias could also be used to study the size of the new good bias. Another line of empirical work which would be of interest would be to collect industry price and quantity data on various major new goods (e.g., microwave ovens, video recorders, home computers, satellite dishes, etc.) and then attempt to rework the relevant price indexes in the light of this extra data. 11. Has the Theory of the Cost of Living Index Been Exhausted? Triplett appears to answer the above question in the affirmative as the following quotation indicates: The COL index has been subjected to far more research, both theoretical and empirical, than any other price index topic in the history of index numbers. It seems to me that much of the fruit has been picked from this tree. Triplett [1988; 25] It seems to me that the harvest is not yet over. A large gap in our current statistical system is in the area of the consumer's allocation of time. Many years ago, Becker [1965] showed how the consumer's time constraint could be integrated into traditional consumer theory and he applied his new framework to cast light on a wide variety of applied economic problems. Additional applications can be found in a more recent book edited by Juster and Stafford [1985]. In order to implement Becker's theory, information on the consumer's allocation of time is required, broken up 53Since 1978, the U.S. Bureau of Labor Statistics has used a probability sampling approach in the consumer price index which probably reduces some of this fixed weight bias, but the bias is not eliminated. 64 Essays in Index Number Theory 2. The Early History 65 into: (i) time at work,54 (ii) time commuting to work, (iii) time spent shopping, (iv) time spent at housework, and (v) time spent at various leisure activities. Since many productivity improvements involve efficiencies in the consumer's use of time (e.g., a new subway line, an automated banking machine, electronic scanning of prices at the supermarket, etc.), it seems appropriate for statistical agencies to consider the implementation of a version of Becker's framework. Another area of household statistics which requires further theoretical development and empirical implementation is the area of income statistics: labor income should be decomposed into price and quantity components, income taxes should be taken into account in an appropriate manner and capital gains should be recognized as components of income. The point here is that most of the household measurement theory has concentrated on the commodity demand side and there has not been enough emphasis on the household factor supply and income sides.55 12. Conclusion In Sections 2 to 6 above, we provided an overview of the early literature on price measurement and index number theory in general. In Sections 7 to 11, we discussed various topics that are more controversial. In some cases, we also provided a historical survey of these topics. In Section 8, we reviewed the early literature on the chain principle and various alternatives to it, and in Section 10 we reviewed the early literature on the new good problem. It seems appropriate to conclude by listing seven recommendations for statistical agencies (the first four agree with those made by Triplett [1988]): (1) Statistical agencies should be encouraged to provide users with adequate printed documentation. (2) The decomposition of labor income (on the household side) and labor payments (on the firm side) into price and quantity components needs improvement: weighted index numbers should be used for quantities rather than unweighted personhours, and personhours should be disaggregated into various occupational, educational and demographically homogeneous categories. (3) In the context of the cost of living index, the flow of services concept should be extended to other classes of consumer durables in addition to housing. (4) My preferred method for decomposing a value ratio into price and quantity components is the use of a superlative index number formula in the 54Riddell [1983] discusses the issues involved in incorporating leisure time (defined to be time spent not at work) in measures of the standard of living and he calculates some of his new measures of economic welfare using Canadian data. 55See Diewert and Bossens [1992] for an introduction to these topics. context of the chain method. (5) Since it is usually impossible to collect complete price and quantity information for each value cell in the relevant accounting framework, it will be necessary to resort to some sort of sampling principle. The appropriate objects to sample are values within the relevant cell in the first period. These sampled values would then be broken up into detailed prices and quantities which would then be observed in the following period as well. Finally, Fisher ideal price indexes should be constructed using these sampled values for the two periods and the corresponding quantity indexes should be constructed by deflating the relevant population value ratios by these (sample) price indexes. The entire procedure is explained in some detail by Pigou [1932; 75-77]. (6) The sizes of the substitution bias and the new good bias are still in doubt. The empirical experiments described in Sections 9 and 10 above would be useful in determining the size of these biases. (7) More empirical and theoretical work needs to be done on the household supply side; see the suggestions on incorporating Becker's theory of the allocation of time and on the construction of household real income indexes made in Section 11 above. References for Chapter 2 Afriat, S.N., 1967. "The Construction of Utility Functions from Expenditure Data," International Economic Review 8, 67-77. Afriat, S.N., 1972b. "The Theory of International Comparisons of Real Income and Prices." In International Comparisons of Prices and Outputs, D.J. Daly (ed.), National Bureau of Economic Research, New York: Columbia University Press, 13-69. Afriat, S.N., 1977. The Price Index, London: Cambridge University Press. Allen, R.G.D., 1949. "The Economic Theory of Index Numbers," Economica N.S.I&, 197-203. Archibald, R.B., 1977. "On the Theory of Industrial Price Measurement: Output Price Indexes," Annals of Economic and Social Measurement 6, 57-72. Balk, B.M., 1981. "Second Thoughts on Wald's Cost-of-Living Index and Frisch's Double Expenditure Method," Econometrica 49, 1553-1558. Becker, G.S., 1965. "A Theory of the Allocation of Time," Economic Journal 75, 493-517. Bennet, T.L., 1920. "The Theory of Measurement of Changes in Cost of Living," Journal of the Royal Statistical Society 83, 455-462. Bergson (Burk), A., 1961. National Income of the Soviet Union since 1928, Cambridge, Mass.: Harvard University Press. 66 Essays in Index Number Theory 2. The Early History 67 Blackorby, C, D. Primont and R.R. Russell, 1978. Duality, Separability and Functional Structure: Theory and Economic Applications, New York: North-Holland. Bowley, A.L., 1899. "Wages, Nominal and Real," In Dictionary of Political Economy, Vol. 3, R.H.I. Palgrave (ed.), London: Macmillan, 640-651. Bowley, A.L., 1901. Elements of Statistics, Westminster England: P.S. King and Son. Bowley, A.L., 1919. "The Measurement of Changes in the Cost of Living," Journal of the Royal Statistical Society 82, 343-372. Bowley, A.L., 1921. "An Index of the Physical Volume of Production," Economic Journals, 196-205. Bowley, A.L., 1928. "Notes on Index Numbers," The Economic Journal 38, 216-237. Bowley, A.L., 1938. "Notes on Professor Frisch's 'The Problem of Index Numbers'," Econometrica 6, 83-84. Christensen, L.R., D.W. Jorgenson and L.J. Lau, 1971. "Conjugate Duality and the Transcendental Logarithmic Production Function," Econometrica 39, 255-256. Court, L.M. and H.E. Lewis, 1942-43. "Production Cost Indices," Review of Economic Studies 10, 28-42. Deaton, A. and J. Muellbauer, 1980. Economics and Consumer Behavior, Cambridge, England: Cambridge University Press. Diewert, W.E., 1976a. "Exact and Superlative Index Numbers," Journal of Econometrics 4, 114-145, and reprinted as Ch. 8 in THIS VOLUME, 223-252. Diewert, W.E., 1978b. "Superlative Index Numbers and Consistency in Aggregation," Econometrica 46, 883-900, and reprinted as Ch. 9 in THIS VOLUME, 253-373. Diewert, W.E., 1980. "Aggregation Problems in the Measurement of Capital." In The Measurement of Capital, D. Usher (ed.), Studies in Income and Wealth, Vol. 45, National Bureau of Economic Research, Chicago: University of Chicago Press, 433-528, and reprinted in Diewert and Naka-mura [1993]. Diewert, W.E., 1981a. "The Economic Theory of Index Numbers: A Survey." In Essays in the Theory and Measurement of Consumer Behavior in Honour of Sir Richard Stone, A. Deaton (ed.), London: Cambridge University Press, 163-208, and reprinted as Ch. 7 in THIS VOLUME, 177-221. Diewert, W.E., 1983a. "The Theory of the Cost-of-Living Index and the Measurement of Welfare Change." In Diewert and Montmarquette [1983; 163-233], reprinted in Diewert [1990; 79-147] and in Diewert and Naka-mura [1993]. Diewert, W.E., 1983b. "The Theory of the Output Price Index and the Mea- surement of Real Output Change." In Diewert and Montmarquette [1983; 1049-1113], and reprinted in Diewert and Nakamura [1993]. Diewert, W.E., 1987. "Index Numbers." In The New Palgrave: A Dictionary of Economics, Vol. 2, J. Eatwell, M. Milgate and P. Newman (eds.), London: Macmillan, 767-780, and reprinted as Ch. 5 in THIS VOLUME, 71-104. Diewert, W.E., 1988. "Test Approaches to International Comparisons." In Measurement in Economics, W. Eichhorn (ed.), Heidelberg: Physica-Verlag 67-86, and reprinted as Ch. 12 in THIS VOLUME, 305-316. Diewert, W.E. and J. Bossons, 1992. "Adjusting the Consumer Price Index for Changes in Taxes," In Diewert and Nakamura [1993]. Diewert, W.E. and C. Montmarquette, 1983. Price Level Measurement: Proceedings from a conference sponsored by Statistics Canada, Ottawa: Statistics Canada. Diewert, W.E. and A.O. Nakamura, 1993. Essays in Index Number Theory, Vol. II, Amsterdam: North-Holland, forthcoming. Diewert, W.E. and T.J. Wales, 1988. "A Normalized Quadratic Semifiexible Functional Form," Journal of Economics 37, 327-342. Divisia, F., 1926. "L'indice monétaire et la théorie de la monnaie," Paris: Societě anonyme du Recueil Sirey. Published as Divisia [1925]. Edgeworth, F.Y., 1888. "Some New Methods of Measuring Variation in General Prices," Journal of the Royal Statistical Society 51, 346-368. Edgeworth, F.Y., 1896. "A Defence of Index-Numbers," The Economic Journal 6, 132-142. Edgeworth, F.Y., 1901. "Mr. Walsh on the Measurement of General Exchange Value," The Economic Journal 11, 404-416. Edgeworth, F.Y., 1923. "The Doctrine of Index-Numbers According to Mr. Correa Walsh," The Economic Journals, 343-351. Edgeworth, F.Y., 1925. Papers Relating to Political Economy, Vol. 1, New York: Burt Franklin. Eichhorn, W., 1973. "Zur Axiomatischen Theorie des Preisindex," Demostratio Matematica 6, 561-573. Eichhorn, W., 1976. "Fisher's Tests Revisited," Econometrica 44, 247-256. Eichhorn, W. and J. Voeller, 1976. Theory of the Price Index: Fisher's Test Approach and Generalizations, Lecture Notes in Economics and Mathematical Systems, Vol. 140, Berlin: Springer-Verlag. Eltetö, O. and P. Köves, 1964. "On a Problem of Index Number Computation Relating to International Comparison," Statisztikai Szemle 42, 507-518. Ferger, W.F., 1946. "Historical Note on the Purchasing Power Concept and Index Numbers," Journal of the American Statistical Association 41, 53- 57. Fisher, F.M. and K. Shell, 1972b. The Economic Theory of Price Indexes, New York: Academic Press. 68 Essays in Index Number Theory 2. The Early History 69 Fisher, I., 1911. The Purchasing Power of Money, London: Macmillan. Fisher, I., 1921. "The Best Form of Index Number," Journal of the American Statistical Association 17, 535-537. Fisher, I., 1922. The Making of Index Numbers, Boston: Houghton Mifflin. Fisher, W.C., 1913. "The Tabular Standard in Massachusetts History," Quarterly Journal of Economics 27, 417-451. Friedman, M., 1938. "Discussion." In Studies in Income and Wealth, Vol. 2, Conference on Research in National Income and Wealth, National Bureau of Economic Research, M. Friedman (ed.), Baltimore: Waverly Press, 123-130. Frisch, R., 1930. "Necessary and Sufficient Conditions Regarding the Form of an Index Number which shall Meet Certain of Fisher's Tests," American Statistical Association Journal 25, 397-406. Frisch, R., 1936. "Annual Survey of General Economic Theory: The Problem of Index Numbers," Econometrica 4, 1-39. Gini, C, 1931. "On the Circular Test of Index Numbers," Metron 9 (2), 3-24. Griliches, Z., 1990. "Hedonic Price Indexes and the Measurement of Capital and Productivity: Some Historical Reflections." In Fifty Years of Economic Measurement 54, E.R. Berndt and J.E. Triplett (eds.), Chicago: University of Chicago Press, 185-202. Harberler, G. von, 1927. Der Sinn der Indexzahlen, Tubingen: J.C.B. Mohr. Harper, M.J., E.R. Berndt and D.O. Wood, 1989. "Rates of Return and Capital Aggregation Using Alternative Rental Prices." In Technology and Capital Formation, D.W. Jorgenson and R. Landau (eds), Cambridge, Mass.: MIT Press, 331-372. Hicks, J.R., 1940. "The Valuation of the Social Income," Econometrica 7, 105-124. Hicks, J.R., 1941-42. "Consumers' Surplus and Index-Numbers," Review of Economic Studies 9, 126-137. Hicks, J.R., 1981. Wealth and Welfare, Cambridge, Mass.: Harvard University Press. Hill, T.P., 1988. "Recent Developments in Index Number Theory and Practice," OECD Economic Studies No. 10, Spring, 123-148. Hofsten, E. von, 1952. Price Indexes and Quality Change, London: George Allen and Unwin. Hulten, C.R., 1973. "Divisia Index Numbers," Econometrica 41, 1017-1026. Jastram, R.W., 1951. "Willard Phillips, A Predecessor of Paasche in Index Number Formulation," Journal of the American Statistical Association 46, 124-126. Jevons, W.S., 1865. "Variations of Prices and the Value of Currency since 1762," Journal of the Royal Statistical Society 28 (June), 294-325. Jevons, W.S., 1884. Investigations in Currency and Finance, London: Macmillan. Juster, F.T. and F.P. Stafford (eds.), 1985. Time, Goods and Weil-Being, Ann Arbor: Institute for Social Research, The University of Michigan. Keynes, J.M., 1930. Treatise on Money, Vol. 1, London: Macmillan. Konus A.A., 1924. English translation, titled "The Problem of the True Index of the Cost of Living," published in 1939 in Econometrica 7, 10-29. Konus A.A. and S.S. Byushgens, 1926. "K Probleme pokupatelnoi cili deneg" (English translation of Russian title: "On the Problem of the Purchasing Power of Money"), Voprosi Konyunkturi 11(1) (supplement to the Economic Bulletin of the Conjuncure Institute), 151-172. Laspeyres, E., 1871. "Die Berechnung einer mittleren Waarenpreissteigerung," Jahrbücher für Nationalökonomie und Statistik 16, 296-315. Lehr, J., 1885. Beiträge zur Statistik der Preise, Frankfurt: J.D. Sauerländer. Leontief, W., 1936. "Composite Commodities and the Problem of Index Numbers," Econometrica 4, 39-59. Lowe, J., 1823. The Present State of England in Regard to Agriculture, Trade and Finance, second edition, London: Longman, Hurst, Rees, Orme and Brown. Malmquist, S., 1953. "Index Numbers and Indifference Surfaces," Trabajos de Estatistica 4, 209-242. Manser, M.E. and R.J. McDonald, 1988. "An Analysis of Substitution Bias in Measuring Inflation, 1959-85," Econometrica 56, 909-930. Marshall, A., 1887. "Remedies for Fluctuations of General Prices," Contemporary Review 51, 355-375. Reprinted as Ch. 8 in Memorials of Alfred Marshall, A.C. Pigou (ed.), London: Macmillan, 1925. Moorsteen, R.H., 1961. "On Measuring Productive Potential and Relative Efficiency," Quarterly Journal of Economics 75, 451-467. Paasche, H., 1874. "Uber die Preisentwicklung der letzten Jahre nach den Hamburger Börsennotirungen," Jahrbücher für Nationalökonomie und Statistik 23, 168-178. Pierson, N.G., 1896. "Further Considerations on Index-Numbers," Economic Journals, 127-131. Pigou, A.C., 1912. Wealth and Welfare, London: Macmillan. Pigou, A.C., 1932. The Economics of Welfare, fourth edition, London: Macmillan. Pollak, R.A., 1971a. "The Theory of the Cost of Living Index," Research Discussion Paper 11, Office of Prices and Living Conditions, Bureau of Labor Statistics, Washington, D.C. In Diewert and Montmarquette [1983; 87-161], and reprinted in Diewert [1990; 5-77] and Pollak [1989; 3-52]. Pollak, R.A., 1989. The Theory of the Cost-of-Living Index, Oxford: Oxford University Press. 70 Essays in Index Number Theory Riddell, W.C., 1983. "Leisure Time and the Measurement of Economic Welfare." In Diewert and Montmarquette [1983; 337-367], and reprinted in Diewert [1990; 249-277]. Ruggles, R., 1967. "Price Indexes and International Price Comparisons." In Ten Economic Studies in the Tradition of Irving Fisher, W. Fellner et al. (ed.), New York: John Wiley, 171-205. Samuelson, P.A., 1947. Foundations of Economic Analysis, Cambridge, Mass.: Harvard University Press. Samuelson, P.A., 1974a. "Remembrances of Frisch," European Economic Review 5, 7-23. Samuelson, P.A. and S. Swamy, 1974. "Invariant Economic Index Number and Canonical Duality: Survey and Synthesis," American Economic Review 64, 566-593. Sato, K., 1976b. "The Meaning and Measurement of the Real Value Added Index," Review of Economics and Statistics 58, 434-442. Schultz, H., 1939. "A Misunderstanding in Index-Number Theory: The True Konus Condition on Cost-of-Living Index Numbers and Its Limitations," Econometrica 7, 1-9. Scrope, CP., 1833. Principles of Political Economy, London: Longman, Rees, Orme, Brown, Green and Longman. Sidgwick, H., 1883. The Principles of Political Economy, London: Macmillan. Stone, R., 1956. Quantity and Price Indexes in National Accounts, Paris: The Organization for European Cooperation. Szulc (Schultz), B.J., 1964. "Indices for Multiregional Comparisons," Przeglad Statystycany (Statistical Review) 3, 239-254. Szulc (Schultz), B.J., 1983. "Linking Price Index Numbers." In Diewert and Montmarquette [1983: 537-566]. Triplett, J.E., 1983. "Concepts of Quality in Input and Output Price Measures: A Resolution of the User-Value Resource — Cost Debate." In The U.S. National Income and Product Accounts: Selected Topics, Murray F. Foss (ed.), Chicago: University of Chicago Press. Triplett, J.E., 1988. "Price Index Research and its Influence on Data: A Historical Review." Paper presented at the 50th Anniversary Conference of the Conference on Income and Wealth, Washington, D.C., May 12. Triplett, J.E., 1990b. "Hedonic Methods in Statistical Agency Environments: An Intellectual Biopsy." In Fifty Years of Economic Measurement, Studies in Income and Wealth 54, E.R. Berndt and J.E. Triplett (eds.), Chicago: University of Chicago Press, 207-233. Vartia, Y.O., 1978. "Fisher's Five-Tined Fork and Other Quantum Theories of Index Numbers." In Theory and Applications of Economic Indices, W. Eichhorn, R. Henn, O. Opitz and R.W. Shephard (eds.), Würzburg: Physica-Verlag, 271-295. 2. The Early History 71 Ville, J., 1951-52. "The Existence-Conditions of a Total Utility Function," The Review of Economic Studies 19, 123-128. This is an English translation of Ville [1946]. Wald, A., 1937. "Zur Theorie der Preisindexziffern," Zeitschrift fur Nationalökonomie 8, 181-219. Wald, A., 1939. "A New Formula for the Index of Cost of Living," Econometrica 7, 319-331. Walsh, CM., 1901. The Measurement of General Exchange Value, New York: Macmillan. Walsh, CM., 1921. "The Best Form of Index Number: Discussion," Quarterly Publication of the American Statistical Association 17 (March), 537-544. Walsh, CM., 1924. "Professor Edgeworth's Views on Index-Numbers," Quarterly Journal of Economics 38, 500-519. Westergaard, H., 1890. Die Grundzüge der Theorie der Statistik, Jena: Fischer. Wickseil, K., 1958. Selected Papers on Economic Theory, Cambridge, Mass.: Harvard University Press.