Lessons from the U.S. Unemployment Insurance Experiments Bruce D. Meyer Journal of Economic Literature, Vol. 33, No. 1. (Mar., 1995), pp. 91-131. Stable URL: http://links.jstor.org/sici?sici=0022-0515%28199503%2933%3A1%3C91%3ALFTUUI%3E2.0.CO%3B2-%23 Journal of Economic Literature is currently published by American Economic Association. Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/journals/aea.html. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. The JSTOR Archive is a trusted digital repository providing for long-term preservation and access to leading academic journals and scholarly literature from around the world. The Archive is supported by libraries, scholarly societies, publishers, and foundations. It is an initiative of JSTOR, a not-for-profit organization with a mission to help the scholarly community take advantage of advances in technology. For more information regarding JSTOR, please contact support@jstor.org. http://www.jstor.org Mon Dec 3 14:48:38 2007 Journal of Economic Literature Vol.XXXIII (March1995), pp. 91-131 Lessons from the US. Unemployment Insurance Experiments BY BRUCED. MEYER Northwestern University This paper was originally prepared for Employment and Immigration Canada. I would like to thank Rebecca Blank, Robert LaLonde, Arthur Stinchcombe, Ging Wong, Paula Worthington, three anonymous referees, participants at the Canadian Employment Research Forum, The EIC Seminar on the Use of Reemployment Cash Bonus Incentives in the United States, the University of Chicago, and Harvard University for their com- ments. R 1. Introduction can be learned about the labor market and UI policy from the recent UI experiECENTLY, there has been extensive ments. evaluation of unemployment insur- The UI experiments have taken two ance (UI) reforms in the United States. main forms: cash bonuses and iob searchJ The proposed reforms generally have programs. Four cash bonus experiments sought to improve the reemployment made payments to UI recipients who prospects of UI claimants and reduce the found jobs quickly and kept them for a budgetary costs of UI. A novel aspect of specified period of time.1 Six job search many of these recent evaluations is that ex~erimentsevaluated combinations ofI they have been true social experiments services including additional information where individuals were randomly as- on job openings, more job placements, signed into either a group subject to spe- and more extensive checks of UI eligibilcial treatment and incentives, or a group itye2 Experiments offering self-employthat received existing services and incentives. Despite the large number of ex- lThe four bonus experiments are the Illinois periments, there is no comprehensive UI Incentive Experiments, New iersey UI Reemployment Demonstration, Pennsy vania Reemployanalysis of their results to date. Their re- ment Bonus Demonstration, and Washington sults are found largely in government re- Reem loyment Bonus Experiments. 2 T e SIX 'ob search experiments are the Nevada ports and have not been systematically Claimant . /P acement Program, Charleston Claimcompared or interpreted. This contrasts ant Placement and Work Test Demonstration, sharply with the extensive literature on New Jersey UI Reemployment Demonstration, Nevada Claimant Employment Program, Washingthe Negative Income Tax and welfare ex- ton Alternative Work Search Experiment, and the periments. In this paper, I describe what Wisconsin Eligibility Review Pilot Project. 92 Journal of Economic Literature, Vol. XXXIII (March 1995) ment assistance or extensive job training have been tried or are underway.3 A. The Reasons for the Experiments Several forces seem to have caused this outburst of experimentation. Recently, there has been increased concern about both the long-term unemployed and displaced workers. Between 1975 and 1985 the unemployment rate was persistently higher than it had been in the first 30 years after World War 11. Much of this change was attributed to increases in unemployment among people with very long periods of joblessness. Structural changes in the economy seemed to be eliminating a large number of jobs. The Bureau of Labor Statistics estimates that 5.1 million workers with at least three years of job tenure became separated from their employer during the 1981-85 period. These increases in joblessness seemed to call for new measures to provide for the unemployed. Identifying displaced workers and targeting services to them was one goal of the experiments (Patricia Anderson, Walter Corson, and Paul Decker 1991, p. 1). On the other hand, the large increases in unemployment during this period drained state UI trust funds forcing many states to borrow to pay their benefits (Wayne Vroman 1986).By the begin- 3 Self-employment demonstrations are currently underway in Massachusetts and Washington State. These experiments waive the work test which requires UI recipients to look for work actively. Lump sum payments to start a business are also part of the experiment in Washington State. These demonstrations are not evaluated here because their goals differ from the other demonstrations and they have not been completed. It is also clear from the interim reports that self-employment interests only a small percent of UI recipients (see Terry Johnson and Janice Leonard 1992; and Jacob Benus et al. 1992). Training programs for UI recipients such as the Texas program described in Howard Bloom (1990) are also omitted because they are much higher cost programs and are better analyzed in com arison to other training prorams. See Judi& Gueron and Edward Pauly h991) or LaLonde (1992). ning of 1983, 28 states owed $13.7 billion to the federal government. The sorry financial state of the UI system pushed many policy makers to look for ways to save money on UI programs. Reducing the costs of the UI program was a goal of all of the experiments. There has been recently a growing desire to get the unemployed back to work which the experiments also emphasize. These forces seem to have been a particularly important impetus in the case of the bonus ex- periments. The job search experiments were spurred by these forces as well as ideas about reforming UI operations. Some of the early experiments pointed to abuses of the UI system such as overpayments. A prime cause of overpayments that is mentioned is the failure of claimants actively to seek work.4 These experiments also mentioned a conflict between the goals of the UI offices which must monitor eligibility and the state Employment Service (ES) which refers people to jobs. Often these two functions are provided in different locations and they are generally provided by different individuals. While the conflict is often described in vague terms, it seems that the ES has an incentive to be more selective in whom it refers to employers than might be desirable for the UI system in the short run. Several of the experiments sought to increase UI/ES cooperation (John Steinman 1978, pp. 1-6; and Corson, David Long, and Walter Nicholson 1985, pp. 2-4). These forces for reform have influenced both state and federal administrators to try changes to existing UI programs. The first bonus experiment, which took place in Illinois, was a state effort. The Illinois experiment appeared to be very successful. This success 4 For an overview of the nonexperimental literature on this topic see Paul Burgess and Jerry Kingston (1990). 93Meyer: Unemployment Insurance Experiments spurred further experimentation, much of it hoping to fine tune the bonus idea rather than to firmly establish whether the broad idea of bonuses makes sense (Robert Spiegelman, Christopher O'Leary, and Kenneth Kline 1991, pp. xii-xiv; and Corson et al. 1992, p. 1).The three bonus experiments that followed were partly supported by the U.S. Department of Labor. The job search experiments were a mix of state and federal initiatives. The state sponsored experiments have tended to be more ambitious, but have also been less carefully designed and evaluated. B. The Use of Random Assignment The UI experiments examined here all used random assignment. Claimants were assigned to a treatment group or the control group usually using the last two digits of a person's social security number. This focus of the paper on true social experiments is motivated by two factors. First, experimental evidence, if properly conducted and interpreted, is likely to be the most convincing to social scientists and policy makers alike. Experimental evidence is easy to explain to policy makers, and the assumptions necessary for its validity are often weaker than those required in nonexperimental empirical work.5 Randomization should provide treatment and control groups with identical characteristics except for sampling error. Estimates of the effect of treatments can then be made by comparing outcome measures such as benefits paid or earnings for those who received the special treatment to the same measures for those who did not. The second reason to focus on experiments is purely practical; the focus narrows the scope of the paper to a small group of reforms that can be adequately analyzed. See Jerry Hausman and David Wise (1985) for a discussion of these issues. Of particular interest are the papers and comments in Chapters 3 and 5. One should bear in mind, though, that experiments are not a panacea. In many situations it is not politically or morally feasible to randomize. Results from experiments may not be well suited to testing theory or generalizing beyond specific treatments. Randomization may not be done perfectly. We will see below that there were deviations (usually minor) from true randomization in the UI experiments. There are also threats to the validity of results from experiments such as spillover effects.6 C. Evaluating the Evaluations The degree of success of the experiments can be measured in a number of dimensions. In all cases I report whether the experiments achieved their objective of reducing weeks of benefits paid and costs. Because a speedier exit from the UI rolls might result in a worse job placement, I also focus on claimants' subsequent earnings when it is possible. Some other outcome measures are analyzed below when available. I also discuss analyses of the overall costs and benefits of the reforms tested in most of the experiments. In many parts of the paper it is necessary to calculate summary statistics other than those presented in the original reports in order to describe the experiments in a uniform way. I do not discuss the effects of the experiments on subgroups of the sample populations, both for brevity and because the imprecision of the estimates leads to a morass of insignificant results. D. A Brief Summay The bonus experiments show that economic incentives do affect the speed with which people leave the unemployGThorough discussions of the potential weaknesses of experiments can be found in Thomas Cook and Donald Campbell (1979) and James Heckman (1992). Journal of Economic Literature, Vol.XXXlZl (March 1995) TABLE 1 MAINDESIGNCHARACTERISTICS EXPERIMENTSOF THE FOURBONUS Enrollment period Bonus amount Relative bonus amount When bonus offer made Qualificationperiod Reemployment period Population examined Eligibilityfor bonus Sample sizes Sources: Illinois UI Incentive Experiments" July 1984to November 1984 $500 for all claimants About 4 times average weeMy benefit At Job Service registration 11weeks for all claimants 4 months 2 0 5 5 year olds without definite recall date, or membership in union using hiring halls No exlusions 3,952 in control group, 8,149 in 2 treatment groups New Jersey UI Reemployment Demonstrationh July 1986to June 1987 1/2of remaining entitlement, but declining 10%per week About 10times weekly benefit at beginning, $1,644 on average Approximately5 weeks after claim filed 10weeks, but beginning about 7 weeks after claim 60% of bonus after 4 weeks, rest after 12 Excluded those under 25, those with definite recall date, members of unions using hiring halls and those with tenure less than 3 years. Those returning to Pre-UI employer ineligible 2,385 in control group, 8,675 in 6 treatment groups a Stephen A. Woodbury and Spiegelman (1987);Spiegelman and Woodbury (1987). '1 Corson et al. (1989);Anderson, Corson, and Decker (1991). Corson et al. (1992). d Spiegelman, O'Leary, and Kline (1991). Meyer: Unemployment Insurance Experiments TABLE 1(Cont.) Pennsylvania Washington Reemployment Bonus Reemployment Bonus Demonstratione Experimentsd July 1988to October 1989 February 1988to November 1988 2 levels: 3 and 6 times the 3 levels: 2,4 and 6 times the weekly benefit amount weekly benefit amount 3 times the weekly benefit Offer varied from $110averaged about $500 $1,254; average offer of $562 After first payment or When claim filed waiting week 2 level: 6 and 12weeks 2 levels: .2times potential duration + 1week, and .4 times potential duration t 1week (3-13 weeks) 16weeks 4 months Excluded those with definite No important restrictions recall date of less than 60 days, and those who obtained jobs exclusively through their union Those recalled by pre-layoff Those who returned to same employer ineligible job or hired through union hiring hall ineligible 3,392 in control group, 3,082in control group, 10,694in 6 12,452in 6 treattreatment groups ment groups Notes: (1)In Illinois a second treatment paid the $500 bonus to the employer. (2) In New Jersey three treatments were tried, mandatory job-search assistance (JSA) only, JSA plus relocation assistance and training, and JSA plus bonus. The bonus became available only about seven weeks after filing for UI benefits. (3) In Pennsylvania five of six treatments were combined with an optional job search workshop. A fifth treatment group had a bonus amount which declined from six times the weeklv UI benefit over a 12-week period. A sixth treatment was a bonus amount of six times the weekly UI benefit with a 12-week qualification peroid, but no job search workshop. 96 Journal of Economic Literature, Vol. XXXIII (March 1995) ment insurance rolls. UI is not a completely benign transfer; it affects claimants' behavior. This is shown by the declines in weeks of UI receipt found for all of the bonus treatments, several of which are statistically significant. The experiments also tend to show that speeding claimants' return to work does not decrease total or quarterly earnings following the claim, but the evidence is less strong because the estimates are imprecise. The cost-benefit analyses indicate that the bonus experiments usually lead to small net losses for the UI program, and are about break-even for society as a whole. I further argue that the permanent adoption of a reemployment bonus could have unintended negative consequences. The key drawback of the experiments is that they cannot account for the effect of a reemployment bonus on the size of the claimant population. A reemployment bonus makes the first trip to the UI office much more valuable for claimants as they become eligible for a large payment if they find a job quickly. Such a payment is most valuable to someone who plans to start a job soon and may not currently believe filing for UI is worth the trouble. Using estimates of the effects of benefits on filing rates, I show that changes in initial filing could eliminate or reverse the positive effects of a bonus. The job search experiments test several alternative reforms which appear more promising. The five experiments try several different combinations of services to improve job search and increase enforcement of work search rules. Nearly all of these combinations reduce UI receipt, increase earnings (when collected), and have benefits that exceed costs. Section 2 describes the design and results of the bonus experiments, while Section 3 does the same for the job search experiments. These sections include cost-benefit analyses and problems with' extrapolating the results to permanent programs. The implications of the results for theories of unemployment and program participation are discussed in Section 4. Section 5 describes implication for policy, while Section 6 discusses the value of the experiments and a further role for experimentation. Section 7 concludes. 2. The Bonus Experiments The cash bonus experiments made payments to UI recipients who found jobs quickly and kept them for a specified period of time. This section describes in turn the design of the experiments, the outcomes of the experiments, and then whether the results suggest that permanent adoption of a bonus program would be sensible. A. The Experimental Designs Table 1 summarizes the main design characteristics of the four bonus experiments. The experiments differed along a large number of dimensions. These dimensions included the amount of the bonus payment, the period during which a person had to find a job to qualify for the bonus (the qualification period), and the period the individual had to remain at the new job to receive the bonus (the reemployment period). In addition to these elements, the point at which the offer of a bonus was made known to the claimants differed, as did the population of claimants offered the bonus, and the eligibility of people returning to their previous employer or hired through a union hiring hall. The first bonus experiment in Illinois offered all claimants, regardless of their previous earnings or weekly benefit amount, a $500 bonus. To put this dollar amount in perspective, it was about four times the average weekly unemployment 97Meyer: Unemployment Insurance Experiments insurance benefit. The qualification period was eleven weeks for all claimants and the reemployment period was four months. In other words, to receive a bonus a person had to start a job within eleven weeks of filing for benefits and keep that job for four months. The other experiments had different benefit amounts, qualification periods, and reemployment periods. The later experiments, particularly those in Pennsylvania and Washington, tried several different combinations of benefit amounts and qualification periods. Washington tried three different formulas for the bonus amount combined with two different formulas for the qualification period, for six different treatments in all. Pennsylvania also had six different treatments. Random assignment was also used to assign individuals to these alternative treatments. While the bonus amount was the same $500 for all claimants in Illinois, in the three other experiments it was a multiple of a person's weekly benefit amount (which depends on their previous earnings). In Washington, the qualification period also varied across individuals within a treatment group, depending on their potential duration (which in turn depends on the stability of a person's earnings over the previous year). These formulas which determined the benefit amount and qualification period were the same within a treatment group, but differed across treatment groups. One of the treatments in the original Illinois experiment offered the $500 bonus to employers who hired claimants. This treatment had a much smaller impact than the one which paid the $500 to employees, and this approach was not tried in any of the later experiments. Each of the experiments randomly assigned eligible UI claimants to either a control group which received existing benefits and services, or one of the treatment groups. However, three of the four experiments excluded several groups prior to this randomization. Generally, those with a definite recall date and those typically hired through a union hiring hall arrangement were excluded. The unemployment durations of these individuals were thought to be unlikely to be affected by a bonus. The experiments following the one in Illinois all excluded individuals recalled to a previous job. This exclusion eliminated the subsidy to temporary layoffs that would have been present if the Illinois bonus design had been permanently adopted. There were some other aspects of the treatments that somewhat complicate comparisons across treatments and states. Several of the experiments combined the bonus offer with the provision of other services. The New Jersey experiment was a hybrid between a reemployment bonus experiment and a job search experiment. New Jersey had a mandatory job-search assistance (JSA) program which was part of all treatment groups (but not available to the controls). I discuss the bonus elements of the New Jersey demonstration here, while the job search treatments are discussed in Section 3. However, to assess the effects of the bonus alone, it seems most natural to compare the bonus with JSA treatment to the JSA only treatment. For some of the published statistics one can make such comparisons, but for other outcome measures one must make the comparison informally. Pennsylvania also tried a job search workshop, but it was optional and the participation rate was under three percent. Therefore, I follow Corson et al. (1992)and assume that we can ignore the effects of the workshop. Before examining the results, it is worth mentioning whether random assignment was properly carried out in the experiments. From the descriptions in the reports it appears that it was, with one exception. In Pennsylvania, the Sam- 98 Journal of Economic Literature, Vol. XXXIII (March 1995) TABLE 2 EFFECTSOF THE BONUS ON WEEKSOF BENEFITS EARNINGS AS EXPERIMENTS AND REEMPLOYMENT MEASURED TREATMENTMINUSCONTROL (standard errors in parentheses) Weeks of Benefits in Benefit Year Earnings ($) Without With 3rd Quarter Total Regressors Regressors After Claim Earnings State and Treatment (1) (2) (3) (4) Illinois UI Incentive Experimentsa 1. Claimant Experiment -1.15 3 366 (0.29) (58) (190) 2. Employer Experiment -0.36 -96 -78 (0.27) (58) (164) New Jersey UI Reemploymentb Demonstration la. JSA Plus Bonus -0.90 4 . 9 7 (0.25) (0.24) lb. JSA only -0.50 4 . 4 7 (0.2.) (0.24 lc. JSA Plus Bonus -0.40 Minus JSA only (0.25) Pennsylvania Reemployment Bonus Demonstrationc 1. Low Bonus, Short Qualification Period 2. Low Bonus, Long Qualification Period 3. High Bonus, Short Qualification Period 4. High Bonus, Long Qualification Period 5. Initially High but Declining Bonus, Long Qualification Period pling rate for the treatment groups was as for the others also suggest that ran- changed twice relative to the controls domization was done successfully (see (Corson et al. 1992, pp. 38-39). There- Woodbury and Spiegelman 1987, pp. fore, the average data of entry for the 517, 522 for Illinois; Corson et al. 1989, treatment groups was different from that p. 213 for New Jersey; Corson et al. of the controls. Given the similarity of 1992, p. 46 for Pennsylvania; and the results below with and without con- Spiegelman, O'Leary, and Kline 1991, p. trols for quarter of entry, this deviation 85 for Washington). from random assignment does not ap- B. Effects on Weeks of Benefits and pear to be especially damaging. Com- Earnings parisons of pre-experiment means of various variables for the treatment and Table 2 reports estimates of the efcontrol groups in this experiment as well fects of the four experiments on weeks of Meyer: Unemployment Insurance Experiments 99 TABLE 2 (Cont.) Weeks of Benefits in Benefit Year Earnings ($) Without With 3rd Quarter Total Regressors Regressors After Claim Earnings State and Treatment (1) (2) (3) (4) Washington Reemployment Bonus Experiments<' 1. Low Bonus, Short -0.06 -0.04 1 -226 Qualification Period (0.30) (0.29) (84) (236) 2. Medium Bonus, Short -0.18 -0.27 100 -134 Qualification Period (0.30) (0.29) (83) (232) 3. High Bonus, Short -0.62 -0.70 176 196 Qualification Period (0.34) (0.33) (94) (262) 4. Low Bonus, Long -0.51 -0.62 -9 -186 Qualification Period (0.30) (0.29) (83) (232) 5. Medium Bonus, Long -0.14 -0.26 24 -186 Qualification Period (0.30) (0.29) (83) (232) 6. High Bonus, Long -0.73 -0.75 189 300 Qualification Period (0.34) (0.33) (95) (265) Sources: awoodbury and Spiegelman (1987,p. 521);Meyer (1988,p. 38);and author's calculations. bCorson et al. (1989,pp. 256,383);Anderson, Corson, and Decker (1991,p. 22);and author's calculations. Corson et al. (1992,p. 92);Decker and O'Leary (1993,Table 9);unpublished tabulations supplied by Paul Decker; and author's calculations. d Spiegelman, O'Lear): and Kline (1991,pp. 88,92);Decker and O'Leary (1993,Table 9); and author's calculations. Notes: (1)To aid interpretation of the results, the control group mean weeks of benefits were 20.1, 17.9, 14.9,15.2 and the 3rd quarter earnings were $2,070, 2,507, 2,605, 3,270 for Illinois, New Jersey, Pennsylvania, and Washington, in order. (2) All of the earnings numbers are not conditional on earnings being positive. The New Jersey, Pennsylvania, and Washington earnings numbers are the coefficients on dummy variables for the different treatments from regressions with individual level control variables. The total earnings measure in Illinois, Pennsylvania, and Washington is the 4 quarters beginning with the quarter of claim. In New Jersey the total earnings measure is the first 10 quarters after the quarter of claim.(3)The sixth Pennsylvaniatreatment group that did not receive the voluntary job search workshop has been combined with the fourth treatment group in this table. See the text for a discussion. unemployment benefits received and experiments. Furthermore, if there is a measures of reemployment earnings. Re- tendency for claimants to take a job ducing the weeks of UI received by quickly to receive the bonus, but then claimants was an explicit goal of all of become unemployed later, the change in the experiments. I report the weeks of first spell length would be a biased meabenefits received in the benefit year7 sure of the longer term effects of the borather than dollars of benefits or weeks nus. In all cases, I report the mean in the first spell of UI receipt because weeks in the benefit year for the treatbenefit year weeks are precisely mea- ment group in question minus the mean sured, and are easier to compare across for the control group. The mean for a treatment group is the mean for all those 7The benefit year is the 52 week period beginning with the week the individual filed for UI assigned to the treatment group not just benefits. those who participate or receive a bonus. 100 Journal of Economic Literature, Vol. XXXIII (March 1995) This method allows us to use the controls as a valid comparison group and provides an estimate of the effect of the bonus on the number of weeks of UI benefits received. When available, I also report the same statistic obtained after one accounts for the effects of individual characteristics in a regression analysis. These individual characteristics typically include age, sex, race, and other variables such as location in the state and recall expectations. The control group mean weeks of benefits and earnings are reported in the notes to Table 2. Several of the evaluation reports overstate the case for using regression controls for individual characteristics in the estimates of experimental effects. As long as the randomization is done properly, comparisons of means are unbiased and probably more convincing to policy makers because there are no ad hoc choices of control variables. The improved precision of the estimates with regression controls appears small in all cases in Table 2, which is expected given the small proportion of the variance in the dependent variable explained by the equations.8 Large differences between the estimates with and without regression controls should be taken as evidence that random assignment was not carried out properly. We should find the small differences of Table 2 below encourag- ing. The negative numbers reported in the first two columns of Table 2 indicate that the number of weeks of UI benefits received was lower for those who were offered a reemployment bonus. The bonus offers appear to have caused a decline in UI receipt in all of the experiments, though many of the declines are not statistically significant at conventional levels. The Illinois claimant experiment had 8For example, the New Jersey benefit year weeks equations has an R2 of ,064. by far the largest effect and the one that is statistically the strongest. Mean weeks of benefits fell by over one week when individuals were offered $500 for starting a job within eleven weeks of filing for benefits. The employer experiment in Illinois had a much smaller impact on weeks of benefits, and payments to employers were not tried in any of the later experiments. Some reasons for the differing results of the two experiments are mentioned in Section 5 below. The three subsequent experiments, in New Jersey, Pennsylvania, and Washington, all estimate much smaller effects of reemployment bonuses. In New Jersey, the job-search assistance (JSA) and bonus treatment has a large estimated effect of almost one week, and this difference from the control group is statistically significant. However, when one compares this treatment to the JSA only treatment, one sees that the estimated additional effect of the bonus is much smaller and only marginally statistically significant. In both Pennsylvania and Washington the estimated effects of the different bonuses on weeks of benefits are mostly small and of weak statistical significance. To put the magnitudes of the estimated effects in perspective, the average weeks of UI received in the benefit year by the control group ranged from 15 to 20 weeks in the different experiments. The effects of the bonuses cluster around one-half of a week, or about three percent of the average duration of receipt. There is some tendency for the larger bonuses and those with a longer qualification period to have a larger effect, but this relationship is not strong. In both Pennsylvania and Washington, regressions were estimated to determine if there was any statistical evidence that the effects of the bonuses rose with either the bonus amount or the qualification period. In both experiments a 101Meyer: Unemployment Insurance Experiments higher bonus and longer eligibility period were associated with greater impacts, but neither effect was statistically significant. Each thousand dollars of a bonus was estimated to reduce UI receipt one-third of a week in Pennsylvania and almost one-half of a week in Washington. Decker and O'Leary (1993) pooled the data from Pennsylvania and Washington though they noted many differences in the samples, UI programs, economic conditions, and estimated outcomes in the two states. Not surprisingly, the estimated effect of each thousand dollars of a bonus is about the average of the two estimates above at 4-tenths of a week. This relationship is found to be statistically significant, while the effect of the eligibility period is not. Other measures of UI receipt, such as dollars of benefits received (see column (1)of Table 3 below) or weeks in the first spell, show a similar pattern of decreases by those offered bonuses. There is also little evidence of a larger treatment effect for first spell weeks of UI compared to benefit year weeks as the first spell effect is larger in Illinois, identical in New Jersey and tends to be smaller in Pennsylvania and Washington. Overall, the experiments show the tendency of people to find a job more quickly if there are monetary incentives to do so. However, the evidence comes from the consistent negative values for the treatment minus control differences in weeks of UI receipt rather than from precisely measured effects of each treat- ment. A key measure of the success of the bonus offers is their effect on claimant earnings. I focus on two distinct measures of earnings: weekly or quarterly earnings which is an indicator of the quality of the job; and total earnings during a period of several quarters after filing the UI claim which is largely determined by the number of weeks worked. If claimants are induced to find a job more quickly, the job they find may be less desirable. I examine quarterly earnings because it is an easily quantifiable measure of the desirability of a job and it is available for all of the bonus experiments. Column (3) of Table 2 reports the estimated impact of the different bonus offers on earnings during the third quarter after filing for UI. In almost all cases, this earnings measure should capture earnings after leaving the UI rolls. As with the weeks of UI estimates above, I report the earnings of those who received the bonus offer minus the earnings of those in the control group. The estimated effects of the experiments on third quarter earnings are somewhat encouraging. The experiments show no statistically significant declines in earnings, and many of the bonuses offered to workers actually indicate no change in earnings or small increases. In Illinois the claimant experiment group has slightly higher earnings than the control group, but the difference is not significant. Treatment group earnings for the employer experiment are lower, but not significantly lower than the claimant experiment or control group. In New Jersey the difference in the change in earnings between the JSA only and the JSA plus bonus group is positive. In Pennsylvania there are slight increases in earnings for all treatment groups except for the "low bonus short qualification period" group where the hypothesis of no change in earning cannot be rejected.9 In Washington the estimates are that earnings generally rose, but the changes are statistically insignificant as they are for the other experiments. Even if quarterly earnings on the new 9The inclusion of the job search workshop as part of the treatments may have affected these results somewhat, but as less than three percent of claimants attended the workshop the bias is likely to be small. Journal of Economic Literature, Vol.XXXIII (March1995) TABLE 3 COST-BENEFITANALYSESOF THE BONUS FROM THREEPERSPECTIVES PARTSOF EXPERIMENTS AND COMPONENT THE CALCULATIONS (allnumbers per treatment group member) Change in Benefits Bonus Costs Administrative Costs State and Treatment (1) (2) (3) Illinois UI Incentive Experimentsa 1. Claimant experiment 2. Employer experiment New Jersey UI Reemploymentb Demonstration la. JSA plus bonus 1b. JSA only 1c. JSA plus bonus minus JSA only Pennsylvania Reemployment Bonus Demonstrationc 1. Low bonus, short qualificationperiod 2. Low bonus, long qualificationperiod 3. High bonus, short qualificationperiod 4. High bonus, long qualificationperiod 5. Initiallyhigh but declining bonus, long qualificationperiod Washington Reemployment bonus Experimentsd 1. Low bonus, short qualificationperiod 2. Medium bonus, short qualificationperiod 3. High bonus, short qualificationperiod 4. Low bonus, long qualification period 5. Medium bonus, long qualification period 6. High bonus, long qualificationperiod Sources: Woodbury and Spiegelman (1987,pp. 521-28); Meyer (1988,p. 38);and author's calculations. Only the UI system calculationsare in Spiegelman and Woodbury. No administrative costs are availablefor this experiment. bAnderson,Corson, and Decker (1991, pp. 22, 62-67); and author's calculations. Unlike the calculationspresented here, those in Anderson, Corson, and Decker include imputed fringe benefits and UI taxes. Meyer: Unemployment Insurance Experiments TABLE 3 (Cont.) Perspective (Benefits minus Costs) All UI System Government Society Change in Earnings (1)+(2)+(3) (5)+.2"(4) (5)+(4) (4) (5) (6) (7) Corson et al. (1992, pp. 14156); Decker and O'Leary (1993, Table 9); and author's calculations. The change in earnings number including fringe benefits had to be used for Treatment 5. dspiegelman, O'Leary, and Kline (1991, p. 196); Decker and O'Leary (1993, Table 9); and author's calculations. Notes: (1)These calculations use the total earnings changes from Table 2 where possible. The calculations ignore fringe benefits and UI payroll taxes. Journal of Economic Literature, Vol. XXXIII (March 1995) job were unchanged, an individual's total earnings after filing for UI might have increased if he or she worked more weeks. The bonus offers reduced weeks of UI received, but did this translate into an equivalent number of more weeks worked? Column (4) of Table 2 reports earnings over as long a period as is available in the reports and papers. The period is one year except for New Jersey where it is two and one half years. The expected relationship between fewer weeks of UI and more weeks worked is at best loosely true. The treatments with the largest reductions in weeks of UI are associated with positive earnings changes of about the right magnitude. But, there are many reductions in weeks of UI associated with total earnings drops. A few observations about the experimental evidence are in order here. There is correlation in the estimated treatment impacts across treatments. While the treatments are given to separate randomly chosen populations, the impact estimates all subtract out the outcomes for a common comparison group. This common component leads to the covariance between the different estimated treatment impacts generally equaling more than half their variance. An unusually high earnings draw for the control group will make all treatments look worse. Thus, when examining the weight of the evidence one should not count the five treatments of Pennsylvania as five times as much evidence as the one treatment of Illinois. Overall, there was not sufficient attention to obtaining precise earnings estimates in the experiments. Such estimates are crucial for any overall evaluation because they are the key component of the cost-benefit analyses be- low. C. Other Outcomes Several of the other objectives of the bonus experiments were not achieved. The New Jersey experiment attempted to target displaced workers who were implicitly defined as those at least 25 years old with three years of job tenure and without a definite recall date. These screens eliminated three-fourths of the claimants and selected a group that had 17.9 weeks of UI receipt on average compared to 15.1 for those not selected. While those selected are statistically different from those not selected, it is not clear these differences are large enough to merit special treatment. Furthermore, the differential effect of the treatments on different groups should also be taken into account when targeting groups. While all of the experiments examined whether the bonuses had larger UI benefit and/or earnings effects on certain groups, no clear patterns emerged as to which groups responded the most to the treatments. The Pennsylvania and Washington experiments sought to determine an optimal bonus amount and qualification period. Optimal is not precisely defined in the evaluations, but can be taken to mean a treatment which maximizes net benefits in a cost-benefit analysis. Each experiment tried six different treatments with different benefit amounts and qualification periods. The small number of observations per treatment caused the impacts to be imprecisely measured and makes comparisons among the treatments difficult. As mentioned above, regression models which assumed a linear relationship between the outcome measures and the benefit amount often yield insignificant estimates, though the results are somewhat mixed. Thus, the imprecision of the results makes the design of an optimal offer impractical. D. Analyses of Costs and Ben4its To assess whether a reemployment bonus is a desirable policy to adopt permanently, we need an overall accounting of 105Meyer: Unemployment Insurance Experiments the costs and benefits of such a program. I believe that the cost-benefit analyses in the published reports are easily misinterpreted, as they all ignore likely effects of a reemployment bonus on the inflow to unemployment and the UI claims pool as discussed in Section 2.E below. Nevertheless, for completeness I summarize their results. The reports on the experiments have cost-benefit analyses usually undertaken from a number of perspectives. These perspectives usually include that of the UI system, government as a whole, and society in general. The cost-benefit analyses from different perspectives differ as to what is included among the costs and benefits. From the perspective of the UI system, the costs of a bonus program are the bonus payments and administrative expenses. The benefits are the reduced UI payments.10 From the perspective of the government as a whole, the benefits also include tax revenues on any earnings increases. A societal perspective ignores bonuses and UI payments as they are just transfers between the government and individuals. The costs are the administrative expenses, and the benefits are any increases in claimants' earnings and fringe benefits. This is the procedure that is followed in the evaluation of most of the experiments. The evaluations do not try to quantify work costs such as childcare, or the leisure value of time out of work which would enter a complete accounting. One should also note that these are not true economic measures of changes in welfare such as compensating or equivalent variation. Thus, they ignore changes in dead-weight loss from tax distortions if payroll taxes change, for example. I should emphasize that the cost-benefit analyses make several strong assump- 10This perspective is that of a narrow administrator who wants to insure as large a UI budget surplus as possible without altering taxes. tions. They generally assume that workers who take a job more quickly do not displace other workers from those jobs. They assume that the short-run response by current UI claimants is a good estimate of the long-run response to a permanent bonus program. Lastly, they assume that there is no new entry into the pool of UI claimants after the financial rewards for a short UI spell have in- creased. The cost-benefit analyses I report in Table 3 differ from those in the published reports in several ways. Most of the differences are due to my attempt to provide statistics that are comparable across experiments. No analysis of UI system or societal costs, and benefits was included in the Illinois reports. The components of the net benefit calculations, i.e., the changes in benefits, bonus payments, administrative costs, and earnings, all come from the individual reports and papers. I have not imputed changes in fringe benefits due to the treatments, but I have imputed taxes at the rate of 20 cents on each dollar of earnings. In all cases, I have used actual earnings changes from administrative data. These numbers were used in the other experiments, but the Washington experiment report imputed earnings by assuming each week of reduced UI receipt translated into an additional week of work at the average earnings level. My changes from the reported analyses make little difference for the ultimate results except in the case of the Washington experiment where the imputed earnings numbers of the published report differ substantially from actual earnings. The Illinois claimant experiment sharply reduced UI payments without paying a large number of bonuses. Consequently, both the UI system and the total government cost-benefit analyses were strongly favorable. Total earnings 106 Journal of Economic Literature, Vol.XXXIII (March1995) also increased, so that a societal level cost-benefit analysis was also favorable.. In New Jersey, there were only small differences between the bonus with the JSA, and JSA only estimates for UI system and all government cost-benefit measures. Thus, the additional benefits of the bonus beyond that of the JSA were essentially zero. Due to worse earnings nu~nbersfor the JSA with bonus treatment than the JSA only treatment, the borlus itself had negative societal level net benefits. In Pennsylvania, most treatments had costs to the UI system and to all government that exceeded benefits, but society tended to benefit due to increases in worker earnings. In Washington, the UI system and all government analyses were similarly unfavorable. In addition, because earnings declined in most cases, four of six treatments had negative net societal benefits. These cost-benefit analyses should not be surprising given the UI benefit and earnings responses described above. The Illinois experiment was a success from all perspectives because it elicited a large reduction in weeks of UI benefits claimed. In the other experiments, there were small responses of weeks of benefits to the bonuses, leading to small and mostly negative effects on the UI system and total government. The earnings numbers are the key element of the societal level cost-benefit analyses, but they are measured impre- cisely. One could use other methods to estimate earnings impacts such as assuming that one week less of UI receipt translates directly into one week more of earnings. Unfortunately, the evidence for this assumption is weak as described earlier. Such an assumption would make treatment minus pre-treatment) for the treatment groups relative to the controls, rather than the post-treatment differences in levels that are used in the current reports. Given the positive correlation in earnings over time, this would provide a better point estimate of the treatment effects on earnings (though it would have even larger standard errors). For all of the New Jersey, Pennsylvania and Washington treatments (over a dozen treatments) the base-period earnings (pre-treatment)are greater than the control group earnings.ll Thus, such analyses would make the cost-benefit analyses of the experiments even less fa- vorable. E, Interpreting the Long-run Effects of Bonus Offers It is unclear whether the experimental impacts of bonuses just described can be directly applied to a permanent program. There are at least three sources of uncertainty.12First, if one group of individuals is encouraged to go back to work early, they may gain employment at the expense of others who are unable to get jobs. Second, with a permanent program a different fraction of eligible claimants might apply for the bonus causing a change in the costs of the bonus offer. Third, by increasing the financial reward for short UI spells, a permanent bonus would probably increase the number of l l F o r New Jersey see Corson et al. (1989, p. 213); for Pennsylvania see Corson et al. (1992, p. 46); for Washington see Spiegelman, O'Leary, and Kline (1991 ?, p. 85). The New Jersey JSA plus bonus treatment also had higher base-period earnings than the JSA only treatment. In Illinois, claimant experiment base-period earnings exceeded control group earnings, but em loyer exsperiment earnings were lower; see Woo bury and Spiegelman (1987, p. 522). 12These sources of uncertainty can be called the all government and societal level cost benefit analyses much more favorable. A better choice would probably be to examine true changes in earnings (postthreats to external validity following the terminology of Campbell and Julian Stanley (1966). For an alternative discussion of threats to the validity of several of the experiments see Spiegelman and Woodbury (1990). 107Meyer: Unemployment Insurance Experiments people unemployed between job changes and increase the number of UI filers. These three arguments imply that the simple cost benefit analyses in Section 2.D overstated the benefits of a reemployment bonus. I will discuss displacement, participation rates, and entry effects in turn. Displacement occurs if by taking jobs more quickly, those eligible for a bonus reduce the number of job vacancies available to other unemployed workers. This possibility has been mentioned by a large number of researchers.13 The easiest way to understand this possibility is to consider a situation where the number of jobs is fixed. If such displacement occurs, the cost-benefit analyses have overstated the net benefits of bonuses, because the gains to those eligible for a bonus come partly at the expense of longer unemployment for other workers. Theoretical examinations of displacement have led to ambiguous results: some theories suggest no displacement, while others suggest substantial displacement. In the end, the issue must be settled empirically. Unfortunately, we have little direct evidence from any source to determine the importance of displacement.14 However, some indirect evidence on the importance of displacement comes from the literature on immigration. While much of this literature suggests that large increases in the number of immigrants looking for jobs 13See George Johnson (1979), Robert Solow (1980) and Laurie Bassi and Orley Ashenfelter (1986) for discussions of displacement in training programs, and Anthony Atkinson (1987) for displacement and UI. Also see some of the reports on the experiments, in particular Spiegelman and Woodbury (1987, pp. 8.8-8.10), Corson et al. (1992, pp. 156-57) and S iegelman, O'Leary, and Kline (1991 ?, p p 230-31f 14 Bassi and Ashenfelter (1986) state that there have been no formal analyses of this issue. Phillip Levine (1993) has recently examined displacement for UI recipients, but additional research is needed before strong conclusions can be made. has little effect on low-wage natives, other papers find substantial effects.15 Other indirect evidence that suggests some displacement comes from work on cohort crowding effects. This literature finds that larger cohorts of workers tend to have lower employment rates and receive lower wages (David Bloom, Freeman, and Sanders Korenman 1987). One should also bear in mind that there are positive general equilibrium effects of decreases in wages as prices fall. This paper is not going to resolve the longstanding debate on the importance of displacement. As current evidence is inconclusive, the possibility of displacement remains a caveat to the results in the experiments. The issue of participation also affects the long run net benefits of a reemployment bonus. A substantial portion of those eligible for a bonus in each of the experiments did not fill out the required paperwork to receive the money. The 11linois claimant experiment estimated that 55 percent of eligible claimants received the bonus, in Washington the estimated takeup rate was 66 percent, and in Pennsylvania a lower bound was estimated to be 65 percent (for Illinois see Woodbury and Spiegelman 1987, p. 525; for Pennsylvania see Corson et al. 1992, pp. 83-85; for Washington see. Spiegelman, O'Leary, and Kline 1991, pp. 183- 85). If all of these additional claimants received a bonus, the UI system and total government cost benefit analyses would be much less favorable. However, it is unclear whether long-run participation would be any higher given the low participation rate for many permanent social programs.16 The issue of incom- 15See George Bor'as, Richard Freeman, and Lawrence Katz (19921 and its discussion of past work. 16See Robert Moffitt (1983) for participation rates in three programs that range from 38 to 69 percent. 108 Journal of Economic Literature, Vol.XXXIII (March1995) plete participation is discussed further in Section 5. The last reason to pause before extrapolating the experimental results to a permanent program is the possibility that a bonus would affect entry onto the UI rolls.l7 The experiments estimate one important behavioral response to financial incentives, but implicitly set several other behavioral responses to zero. A bonus offer substantially raises the reward for initially contacting the UI office for someone who knows he will begin a new job within the qualification period, and raises the reward by something less for all others.18Recall (from Table 1) that the bonus amounts ranged from three to ten times the weekly benefit. A permanent bonus program may lead to an increase in the claimant pool for three reasons. First, the currently unemployed who do not file could respond to the increased incentives to file for UI benefits. Second, those currently not unemployed but changing jobs could start work slightly later and file for UI benefits. And third, firm layoff policies could respond to the increased compensation for short UI spells. Using results from previous work as well as new evidence, this section documents that all of these responses are potentially large.lg First, those who become unemployed under existing incentives but do not file could respond to the new incentives. 17The following aragraphs are an updated version of a section o?Meyer (1991) A similar argument can be made about the delayed offer of a bonus in the New Jersey ex eriment where the bonus offer was concealed from claimants until five weeks after filin . In a permanent program, this concealment c o J d not occur and would induce claimants to stay on the UI rolls until offered the bonus. See Spiegelman and Woodbury (1990) and Meyer (1988) for a discussion of this problem. 18.4 claimant would also need to know that she would hold the new job until the end of the reemlo ment period. &See Moffitt (1992) for the similar argument for the possibility of entry responses in the context of welfare and training programs. Survey estimates indicate that takeup rates for UI are substantially below one, with the range of estimates in the literature for the fraction of eligibles receiving UI ranging from 0.55 to 0.83.20 This finding indicates that a large number of the currently unemployed could respond to additional incentives to file for UI. One can obtain a rough estimate of the quantitative effect of a bonus offer on the probability of filing for UI. Let P be the probability of filing for benefits. Then the percentage change in the probability of filing when a bonus offer is instituted can be written as P(~ivena bonus offer)-P (givenno bonus offer P(given no bonus offer) -- aln(P) aln(weeklybenefit) expected percentage change in UI dollars -- aln(P) aln(weeklybenefit) probability qualify for bonus bonus amount weekly benefit (weeksof receipt given qualify for bonus). The first equality indicates that the percentage increase in the probability of filing is equal to the effect of a percentage change in benefits times the expected percentage change in UI dollars received due to the bonus offer. The second equality replaces the expected percentage change in UI dollars with the product of the probability of receiving a bonus and the conditional percentage increase in UI dollars received. We can obtain estimates of the first expression from past research and estimates of the second and third from the results of the 20The 0.55 estimate comes from a special Current Population Survey supplement reported in Vroman (1991), while the 0.83 estimate is for Panel Study of Income Dynamics household heads reported in Blank and David Card (1991). 109Meyer: Unemployment Insurance Experiments experiments. Anderson and Meyer (1993) estimate that the first expression, the takeup elasticity, is between .46 and .78.21 In what follows, I use the Illinois experiment as an example because it was the most successful. If increased entry can outweigh the benefits there, it is unlikely that the other experiments would yield favorable results. The probability of receipt of a bonus in Illinois was .136, the bonus amount was $500, and the mean weekly benefit $119. The mean weeks of benefits conditional on qualifying for the bonus (finding a job within eleven weeks of filing) is taken to be the control group mean conditional on receiving ten or less weeks of UI in the first spell, or 3.65. Combining these estimates suggests that takeup would rise between 7 and 12 percent. For two reasons, this range probably underestimates the effect of a bonus offer on takeup. Remember that the .I36 bonus receipt rate is made up of a 25 percent qualification rate and a 54 percent takeup rate. First, it is likely that the marginal UI applicant induced to apply by a bonus would have a short spell on UI that previously would have made application unrewarding. Thus, it is unlikely that only 25 percent of the claimants would qualify for a bonus. Second, it is unlikely that only 54 percent of those induced to apply and who qualify for a bonus would fill out the paperwork to receive it. Those induced to apply because of the bonus would presumably be more willing to fill out the paperwork to receive the bonus. Even ignoring these factors, one obtains a substantial increase in the costs of 21This is the range of point estimates, which one should not interpret as a confidence interval. Anderson and Meyer (1993) is the only estimate this elasticity using individual cl%?zia; accurate information on benefit amounts and potential duration. Other papers using aggregate data have generally found ranges of point estimates that overlap with this range. a bonus offer. Costs per person with no takeup effect were $500".136= $68. With a takeup effect they would be $500".136"(1 + .07 to .12) + $119"3.65"(.07 to .12)= $103 to $128. In addition, once more people are induced to enter the UI system, they may be subject to its incentives to prolong unemployment, leading to increased joblessness as well as greater costs. Second, those who currently change jobs without an intervening spell of unemployment could become unemployed for a few weeks, start work slightly later, and file for UI benefits and the bonus. A large fraction of those changing jobs without unemployment would be ineligible because they quit their previous job (though the enforcement of this restriction is far from complete). However, estimates of the fraction of layoffs that do not result in unemployment range from 0.14 to over 0.30 in three sources.22 Third, a permanent bonus program would subsidize firm layoffs by increasing the compensation for short UI spells.23 Frank Brechling (1981), Kim Clark and Lawrence Summers (1982), and Robert Tope1 (1983) find large and statistically significant effects of higher UI benefits on the probability of entering layoff unemployment. The approaches of Brechling and Topel are particularly designed to capture the incentives firms face when considering layoffs. Their results suggest that higher compensation for short UI spells will make layoffs more attractive to firms. While much of the effect of benefits is 22Jacob Mincer (1991) ives the over 0.30 figure, Peter Gottschalk a n f Tim Maloney (1985) give 0.24 for involuntary se arations which includes fires, and Robert Gi%bons and L. Katz (1991) give 0.14 for dis laced workers. 23These effects woufd be softened somewhat by experience rating of the UI tax. But given the slack in experience rating in most states, most of the effect would likely remain. 110 Journal of Economic Literature, Vol. XXXIII (March 1995) probably on layoffs that are initially thought to be temporary, in about onethird of such layoffs the worker is not recalled. (See L. Katz and Meyer 1990.) Therefore, even if recalled individuals cannot receive a bonus, there will likely be an effect on layoffs as part of the compensation package in unstable industries increases. In addition, if we increase the compensation for laid off workers, it makes jobs with a high probability of layoff more attractive. The incentives will subsidize the expansion of unstable industries, also tending to increase layoffs and unemployment.24 One should note that all of the effects described in this section would tend to increase unemployment as well as increase UI payments. It is possible that part of the experimental impacts of the treatments were due to Hawthorne effects. The term Hawthorne effects refers to responses that come from the act of experimentation itself rather than the intended treatment.25 For example, claimants may have reduced their receipt of UI because they were under study per se, rather than because of the bonus offer. This explanation is not precise so, it is hard to rule it out. However, Hawthorne effects are a less plausible explanation if larger bonuses tended to produce larger responses and if groups that were expected to be more responsive to a bonus did respond more. Such evidence would sug- 24See Donald Deere (1991) for em irical evidence on UI induced industry cross-su%sidies,expansion of high layoff industries, and unemploy- ment. 25The term Hawthorne effects comes from experiments conducted at the Hawthorne plant of the Western Electric Company in Chicago between 1924 and 1933. The first of the experiments appeared to show that changes in the level of illumination resulted in increases in worker productivity and job satisfaction whether the lighting was increased or decreased. For a critical examination of these experiments see Richard Franke and James Kaul (1978). gest responses to the character of the treatment rather than the act of experimentation per se. There is some evidence of this form because there is a tendency for the larger bonuses to elicit greater responses. But, as mentioned above, this tendency is only significant in pooled data from Pennsylvania and Washington, not for either of the states separately. There is also little evidence of a larger response by low earnings individuals to the fixed $Ti00 bonus in 11linois (see Meyer 1991). On the other hand, it is encouraging that the claimant experiment in Illinois was successful, while the more complicated treatment of the employer experiment had little im- pact. F. Additional Comments While the initial reaction of policy makers and academics to the bonus experiment idea was very positive, I believe the initial optimism is fading. With time we can see more clearly the weaknesses of the experiments as a guide to the viability of a permanent bonus program. The initial results from Illinois were so striking that additional bonus experiments were tried. If the Illinois experiment had produced weak results, the idea probably would not have been re- peated. The large magnitude of the Illinois responses is still puzzling. Carl Davidson and Woodbury (1991a) have argued that the availability of extended benefits during the first part of the Illinois sample period interacted with the bonus to produce larger effects. They find larger differences between the treatments and control groups during the period that extended benefits were available. However, the Illinois claim duration effects during the period when extended benefits were not available were still larger than the most comparable treatments from Pennsylvania and TABLE 4A MAINDESIGNCHARACTERISTICS JOBSEARCHOF THE DOL EVALUATED EXPERIMENTS Charleston Claimant Placement New Jersey UI Reemployment Washington Alternative and Work Test Demonstrationa ~emonstrationb Work Search ExperimentC Enrollment period February 1983to December 1983 July 1986to June 1987 July 1986to August 1987 Treatments Three treatments, all 3 were required to Periodic contacts with Employment Three treatments: report to ES and were monitored: (1) 2 Service required along with job search (1) exception reporting based on the enchanced placement interviews and a workshop of 5 half-days plus one-on- honor system. UI payments sent 3-hour job search assistance session one session (JSA). unless office told to s t o ~ :(2) similar toL . , , required after 4 weeks of UI; (2) 2 control group, but group eligibility required interviews only; (3) 1 reviews and follow-ups had diffferent required interview only. The control emphasis; (3) intensive work search group was not subject to usual ES assistance with 2-day job search reporting requirement. workshop When first contacted After 1st UI payment Approximately 5 weeks after claim filed At interviewfollowingclaim filing Population examined Mass layoff claimantsexcluded Excluded those under 25, those with a All eligibleclaimants definite recall date, members of unions using hiring halls, and those with tenure less than 3 years Sample sizes 1,428 in control group, 4,247 in 3 2,385 in control group, 6,226 in 2 2,871 in control group, 6,763 in 3 treatment groups treatment groups treatment groups Sources: ,LCorson,Long, and Nicholson (1985). "orson, et al. (1989),and Anderson, Corson, and Decker (1991). Johnson and Daniel Klepinger (1991). Other Important Characteristics New Jersey UI Reemployment Demonstration: the job search assistance treatment were part of a larger demonstration which is also described in tlie bonus experiment section.An additionaltreatment composed ofjob search assitanceplus an offer of training and relocationassistance is not described here for reasons given in tlie text. 112 Journal of Economic Literature, Vol.XXXIII (March1995) TABLE 4B MAINDESIGNCHARACTERISTICS JOB SEARCHOF STATE EXPERIMENTS Nevada Claimant Placement Program (NCPP)a Wisconsin Eligibility Review Pilot Project ( E R P ) ~ Enrollment period February 1977to March 1978 March 1983 to August 1983 Treatments One treatment: more staff attention and more referrals, weekly interviews and eligibility checks, all services from same ES/UI team which coordinated their efforts. 6-hour job search workshop conducted by ES staff; also tried 3-hour job search workshop When first contacted After 3rd UI Payment 80% when first filed, 20% at first ES interview (approximately 6 to 9 weeks after claim) Population examined After eligible claimants Claimants indefinitely separated from most recent job Samplesizes 1,174 in control group, 2,371 in treat- 2,277 in control group, 2,587 in ment group Sources: a Steinman (1978). b Wisconsin Job Service (1984,undated). Washington. Other possible explanations include the state of the macroeconomy (the unemployment rate at the time of the experiments was highest in Illinois) or that Illinois allowed employees recalled by their former employer to receive the bonus. 3. TheJob Search Experiments The job search experiments tried several different combinations of improved provision of job finding services and increased enforcement of the job search requirements for the receipt of UI. This section describes in turn the design of the experiments and the effects of the experiments on weeks of benefits, earnings, enforcement of work search rules, job searches, referrals and placements. I then analyze whether the results suggest that permanent adoption of an expanded job search program would be sensible. This part is divided into cost-benefit analyses, and interpreting the long-run effects of the programs. treatment groups A. The Experimental Designs Tables 4A and 4B summarize the main design characteristics of the five job search experiments.26 I have split ;he job search experiments into two groups and given these groups the labels Department of Labor (DOL) evaluated job search experiments and state job search experiments.27 The reason behind this division is that the DOL experiments are more carefully designed, implemented and evaluated than the state experiments. Thus, their results should be accepted with greater confidence. Before 26Because of defects in the experimental design, I have dropped the Re-Employ Minnesota program and the Nevada Claimant Em loyment Program (NCEP) from this version of tRe paper. See Meyer (1992) for a description of these pro- grams. 27These labels are an oversimplification in that the DOL s u ~ ~ o r t e dex~eriments were fed-I I erallstate cooperative eff&ts and some federal support was provided for the Nevada Claimant Placement Project. However, the DOL experiments are described in DOL ublications, while the state efforts are describe% in state staff re- ports. 113Meyer: Unemployment Insurance Experiments describing the experiments, it is useful to describe the status quo that these experiments altered. UI claims procedures and the work test differs significantly across states, but there are several common elements. When UI claimants file for benefits, they are typically divided into those that are required to register with the employment service and those that are not. Those that are exempted typically are on layoff with a definite recall date or are members of a union that hires through a union hiring hall. Some states monitor whether a claimant registers with the employment service, while others do not. States require all claimants to submit evidence of their job search efforts. Typically, but not always, they require details of employer contacts made during a week. After a claimant has been unemployed for some time, she is typically asked to report for an eligibility review. Often this review takes the form of a group session followed by an individual review of eligibility. This is the procedure for a typical state, but states differ in many respects. Each of the experiments altered the existing state procedures in a different way and several of the experiments had more than one treatment. The main differences between the treatments were the job finding services provided, the additional reporting requirements, and whether or not a job search workshop was required. The experiments also differed according to when the new services or requirements were provided and whether some claimants were excluded from the demonstration. The job finding services provided differed substantially across the experiments. In the Charleston experiment claimants were interviewed and a job-development attempt or referral was made. They were placed in the state job matching system and taught how to use job listings. In New Jersey claimants were interviewed and tested. A job resource center was set up in each office and listings of job openings and telephones were made available. In Washington, claimants had an interview which emphasized job-finding techniques and phone rooms were made available. The Nevada experiment emphasized more personnel attention by ES/UI personnel and also insisted that all ES/UI contacts be provided by the same person or pair of individuals (one from UI, one from ES). A separate treatment group in the New Jersey experiment was provided training or relocation assistance. For the reasons mentioned earlier (its higher cost and similarity to training programs, not job search programs), I did not study this treatment. Four experiments, Charleston, New Jersey, Washington, and Wisconsin, required that at least one of their treatment groups attend a seminar on how to find a job. The Charleston workshop "lasted approximately three hours and provided a forum for discussing basic search and interviewing strategies, and for making relevant labor-market information available." The New Jersey workshop lasted five half-days and "included such topics as dealing with the loss of one's job, self-assessment, developing realistic job goals, organizing an effective job search strategy, and developing resumes and effective job application and interview techniques." The Washington workshop lasted two days and "included training on skills assessment, interview and marketing techniques, telephone canvassing, completing applications, and preparing resumes." The Wisconsin workshop lasted six hours and included how the job service works, job opportunities assessment, how people find jobs, application completion, and interviewing techniques (for Charleston see Corson, Long, and Nicholson 1985, p. 19; for New Jersey see Corson et al. 1989, p. 72; 114 Journal of Economic Literature, Vol. XXXIII (March 1995) for Washington see T. Johnson and Klepinger 1991, p. 5; for Wisconsin see Wisconsin DILHR, undated, p. 10). An important element of several of the experiments was more frequent checks of claimant eligibility, and additional required visits to the UI or ES offices to maintain UI eligibility. The Nevada Claimant Placement Program required weekly interviews and eligibility checks, Charleston required a different number of in depth interviews depending on the treatment group, and New Jersey required periodic contacts with the Employment Service. This combination of additional services and tightened eligibility checks makes it difficult to determine what aspects of the experiments induced the changes in outcomes below. One of the treatments in Washington tried the opposite approach of requiring fewer checks of claimant job search activities. This exception reporting treatment was based on the honor system, meaning that UI payments were sent to claimants until they indicated that they were no longer eligible, i.e., had returned to work. Claimants were instructed to make an active job search, but no job search services were provided and no reporting of work-search contacts was required. The experiments also differed as to who was included and when claimants were first contacted and told of their obligations or opportunities. Some experiments excluded from the study population certain groups such as those on temporary layoff.28 The experiments that delayed contacting claimants or excluded others generally did this to exclude certain individuals who usually did not need to report to the Employment Service or were thought to be unlikely to benefit from the job search services. 28111 all experiments the exclusions took place prior to random assignment or were applied to both treatment and control groups. Before turning to the results, it is again worth reporting whether random assignment was properly carried out in the experiments. From the descriptions in the reports it appears that there were two experiments with deviations from perfect random assignment. In Washington, the allocation rate for the treatment groups relative to the controls was changed in the middle of the experiment (T. Johnson and Klepinger 1991, p. 11). Therefore, the average data of entry for the treatment groups was different from that of the controls. Similarly, in Charleston the allocation of claimants among the treatment groups was different for one site, leading to the geographic representation of the treatment groups being different from the controls (Corson, Long, and Nicholson 1985, p. 26). Given that the Washington estimates control for the season and year in which claimants filed, the deviation from random assignment there is unlikely to appreciably bias the results. It appears that the Charleston regression estimates do not control for the sites, so the deviation from randomization is more of a concern there. However, the two groups that were shuffled had roughly similar impacts so that any bias is likely to be small. Comparisons of the preexperiment means of various variables for the treatment and control groups in these two experiments as well as for the others do suggest comparability of the groups (see Corson, Long, and Nicholson 1985, p. 27 for Charleston; Corson et al. 1989, p. 213 for New Jersey; Steinman 1978, p. 19 for Nevada; T. Johnson and Klepinger 1991, p. 16 for Washington; and Wisconsin Job Service 1984, p. 11for Wisconsin). B. Effects on Weeks of Benefits and Earnings Tables 5A and 5B report estimates of the effects of the six job search experi- 115Meyer: Unemployment Insurance Experiments TABLE 5A EFFECTSOF DOL SUPPORTED EXPERIMENTS OF BENEFITSAND REEMPLOYMENTJOBSEARCH ON WEEKS EARNINGS: AS TREATMENT MEASURED MINUSCONTROL (standard errors in parentheses) ,Weeks of Benefits Earnings ($) State and Treatment Without Regressors (1) With Regressors (2) lst, 2nd, or 3rd Quarter After Claim (3) Total Eanlings (4) Charleston Claimant Placement and Work Test Demonstration" 1. 2 Interviews and Job Search Session 2. 2 Interviews only 3. 1Interview only New Jersey UI Reemployment Demonstrationb 1. Job-Search Assistance (JSA) Washington Alternative Work Search Experimentc 1. Exception Reporting 2. Individually Tailored Work Search 3. Intensive Work Search Sources: Corson, Long, and Nicholson (1985,pp. 57,69, and 75) and author's calculations. "Corson, et al. (1989,pp. 256 and 383),and Anderson, Corson, and Decker (1991,p. 22) and author's calculations. Johnson and Klepinger (1991,pp. 27 and 44). Notes: (1)To aid interpretation of the results, the control group mean weeks of benefits were 15.5,17.9,and 14.5for the three experiments in the order listed above. The control group earnings corresponding to column (3) were $1,369, 2,507, and $2,019 in the 3 experiments in order. (2)In Charleston the UI benefits measure is for the first 6 months of the claim, and the two earnings measures are for the second quarter and the first two quarters after claim. (3) In New Jersey the UI weeks measure is for the Benefit Year, and the two earnings measures are for the third quarter and first 10 quarters after the quarter of claim (with regression controls). (4) In Washington the UI weeks measure is for the Benefit Year, and the two earnings measures are for the first quarter and first year following the claim. ments on weeks of unemployment bene- of the reasons for using these outcome fits received and measures of reemploy- measures. In the tables, the mean for ment earnings when available. Table 5A the treatment group in question minus reports the results for the higher quality the mean for the control group provides DOL evaluated experiments, while 5B an estimate of the effect of the treatreports the results for the state experi- ment on UI receipt. When available, I ments. See Section 2 B. for a discussion also report the same statistic obtained 116 Journal of Economic Literature, Vol. XXXIII (March 1995) TABLE 5B EFFECTSOF STATE EXPERIMENTSJOBSEARCH ON WEEKSOF BENEFITS EARNINGS:AND REEMPLOYMENT MEASURED MINUSCONTROLAS TREATMENT (standard errors in parentheses) Weeks of Benefits Without State and Treatment Regressors Nevada Claimant Placement -3.90 Program (NCPP)" (0.41)* Wisconsin Eligibility Review Pilot Project (ERPjb -0.62 (6-hourworkshop) (0.43)" Sources: BSteinman (1978, pp. 28 and 351, and author's calculations. bW7isconsin Job Service (1984,pp. 13and 15). The standard error for the NCPP has been estimated using the reported sample sizes and a standard deviation of weeks of benefits of 10. The Wisconsin standard error was approximated using the significance level reported in Wisconsin Department of Labor (1984). Notes: (1) To aid interpretation of the results, the control group mean weeks of benefits was 12.4and 16.4 for the two experiments in the order listed above. (2)In the NCPP the UI weeks measure is for the first spell, and for the sample excluding union members. (3) In Wisconsin the UI weeks measure is all weeks accumulated by a fixed date that was just under a year after the end of the intake for the experiment. after one accounts for the effects of individual characteristics in a regression analysis. The DOL evaluated job search experiments show moderate sized effects on weeks of UI received. dharleston, New Jersey, and Washington all show reductions of about one-half of a week in UI receipt from the treatments with more intensive services and oversight. Several of these effects are statistically significant or nearly statistically significant and the estimated impacts are about the same size as those of the bonus experiments. The Washington exception reporting treatment that did not provide any services and reduced the oversight of claimants was associated with a large and statistically significant 3.34 week increase in weeks of UI received. As was the case for the bonus experiments, regression controls have almost no effect on the point estimates or their standard errors. The Wisconsin job search workshop experiment showed a moderate decline in the duration of UI receipt that is similar in magnitude to those for the DOL evaluated experiments. The change is not statistically significant however. The Nevada experiment shows a very large and statistically significant decline of 3.9 weeks of UI benefits received by the treatment group.29 This experiment emphasized higher quality delivery of existing services and required claimants to return to the same ES/UI team for all contacts. The Nevada experiment seems especially promising, but its approach requires more study because its evaluation is one of the least complete. Earnings impacts are also available for the DOL evaluated experiments and are reported in Table 5A. I have less choice of earnings measures for these job search experiments than I had with the bonus experiments. The measure of quarterly earnings is the second, third, and first quarter after claim for Charleston, New Jersey, and Washington, respectively. The measure of total earnings is the first two, ten, and four quarters, for the three experiments in order. All of the quarterly earnings measures suggest slight increases in earnings for the more intensive treatments (all treatments except the first two in Washington). The total earnings numbers are also always positive and are consistent with a one-half week reduction in UI receipt being asso- 29Standard errors on the weeks of UI effects were not reported for the two state experiments, so I have approximated them using the reported sample sizes and a standard deviation of weeks of 10, which is slightly larger than the average from the other experiments. 117Meyer: Unemployment Insurance Experiments ciated with one-half week more of work. However, because of their large standard errors, the earnings numbers are consistent with a wide range of possible imp a c t ~ . ~ ~ There does not appear to be a symmetric decline in earnings for the Washington exception reporting treatment which removed almost all oversight by the Employment Service and increased UI receipt by 3.34 weeks. These additional weeks of UI receipt do not seem to have been translated into fewer weeks of work, as total earnings in the year were only slightly lower while hourly earnings over the year were insignificantly higher than the control group. While this Washington treatment did not affect earnings during the year after claim, it did significantly increase the probability of a worker returning to the same employer or industry. C. Enforcement of the Work Search Requirements One of the goals of the job search experiments was to reduce abuse of the UI system by claimants who do not search for work as required. This section reports the effects of the experiments on the enforcement of work search rules. In four of the five job search experiments, administrative measures of enforcement such as denials of benefits increased significantly.31 These four experiments required claimants to make additional trips to the Employment Service and failure to report could result in denial of bene- 30 While standard errors are unavailable for two of the experiments, the sample sizes are such their earnings estimates are also likely to be imprecise. 31For Charleston see Corson, Long, and Nicholson (1985, pp. 44-54); for New Jersey see Corson et al. (1989, pp. 273-77); for the Nevada NCPP see Steinman (1978, pp. 36-42); for Wisconsin see Wisconsin Job Service (1984, Washington there were no significant c,.anges Inin denials for the main treatment group; see T. Johnson and Klepinger (1991,pp. 39-43). fits. The experiments differ whether the increased denials were due to failure to report or to better enforcement of the work search rules. In Charleston it appears that better enforcement of work search was more important, while in New Jersey and Washington an increase in denials for failure to report was the only significant change.3Vt is possible that those failing to report would have been denied for work search reasons, but the results suggest that part of the reduction in UI receipt in these experiments came from the additional costs imposed on claimants. In all cases however, denials per se would account for a small fraction of the decline in weeks of UI received. For example, in Charleston there was a rise in denials of specific weeks for the treatment groups, but it would take ten weeks per denial to account for the declines in UI receipt these groups showed (Corson, Long, and Nicholson 1985, pp. 48-51). If increased enforcement was responsible for most of the decline in UI receipt, the decline must have come from indirect effects of tighter enforcement such as deterring violations of work-search rules. The Washington exception reporting treatment provides some evidence of the effects of changes in the opposite direction, i.e., reducing reporting requirements. UI duration increased over three weeks when this treatment had claimants notify the UI office when they had found a job and did not require the reporting of work-search contacts. It is unclear whether this increase indicates that claimants were slow in notifying the UI office that they had found a job, or that eliminating the work-search reporting 32A hazard model analysis in the Washington report analyzes the timing of the effects of the work search services. It suggests that the shorter durations of UI receipt are due to the costs of appearing at the UI office. See T. Johnson and Klepinger (1991,pp. 31-38). TABLE 6A EFFECTSOF DOL EVALUATED EXPERIMENTS OF BENEFITS AND REEMPLOYMENTJOBSEARCH ON WEEKS EARNINGS MEASURED MINUSAS TREATMENT CONTROL Perspective (Benefitsminus Costs) All UI System Government Society Change in Administrative Change in Benefits Costs Earnings (I)+@) (4)+.2*(3) (4)+(3) State and Treatment (1) (2) (3) (4) (5) (6) Charleston Claimant Placement and Work Test Demonstrationa 1. 2 interviewsandjob search session 73 -18 152 55 85 207 2. 2 interviews only 59 -13 264 46 99 310 3. 1 interview only 53 -5 110 48 70 158 New Jersey UI Reemployment Demonstrationb 1. Job-SearchAssistance (JSA) WashingtonAlternative Work Search Experiment! 1. Exceptionreporting -265 0 -23 -265 -270 -288 2. Individuallytailored work search -5 -2 -24 -7 -12 -3 1 3. Intensive work search 68 -14 292 54 112 346 Sources: a Corson, Long, and Nicholson (1985,pp. 57, 104)and author's calculations. b Anderson,Corson and Decker (1990,pp. 22,6%66) and author's calculations. Johnson and Klepinger (1991,pp. 27,44 and 50). 119Meyer: Unemployment Insurance Experiments TABLE 6B COST-BENEFIT OF THE STATE EXPERIMENTSANALYSES JOBSEARCH State and Treatment Nevada Claimant Placement Program (NCPP)a Wisconsin EligilibilityReview Pilot Project (ERP)b Sources: asteinman (1978,pp. 59-61), and author's calculations. bWisconsinJob Service (1984,p. 13). requirement actually reduced search effort, or just reduced disqualifications for weeks without search. As total earnings did not fall, it seems likely that the change in requirements did not reduce search effort appreciably. D. Job Searches, Referrals, and Placements Two of the five job search experiments generated large increases for the treatment groups in measures of job search assistance such as job searches, referrals, placements, and counseling (for Charleston see Corson, Long, and Nicholson 1985, pp. 37-44; for the Nevada NCPP see Steinman 1978, pp. 36-39; for New Jersey see Corson et al. 1989, pp. 246- 51; for Washington see T. Johnson and Klepinger 1991, pp. 19-21). The experiment with the largest reduction in weeks of UI was one of these two. The experiments generating large increases in services were in Nevada and Charleston. In both experiments referrals to new jobs or placements more than doubled. In New Jersey the treatment groups received more counseling but only moderate sized, but statistically significant increases in job referrals (from 6.1% to 8.5%).At the other extreme, la cements and referrals tended to be ~ b k e rfor the Change Administrative UI System in Benefits Costs Benefits-Costs (1) (2) (3) 318 4 9 82 -25 E. Analyses of Costs and Benefits To assess whether a job search program is a desirable policy to adopt permanently, we need an overall accounting of the costs and benefits of such a program, as we did for the bonus experiments in Section 2.D. Table 6A reports cost-benefit analyses and components of the calculations for the higher quality DOL evaluated experiments. Table 6B reports similar, but less complete, evaluations for the state experiments. Except for the New Jersey experiment, the costs of the job search treatments were always low compared to the savings in UI benefits. Therefore, the UI system and total government (in the absence of large negative earnings changes) benefitted from the programs.33 The benefits of the job search experiments tended to exceed costs by a wide margin. For example, in both Nevada and Wisconsin the costs of the additional services and oversight was much less than half of the savings in UI benefits. Societal level cost benefit analyses require earnings estimates as they compare the administrative expenses of the experiment to changes in claimant treatment groups in Washington. Infor- 33111 the New Jersey JSA only treatment, the UI system broke even and total government gained, mati0n on and placements was while the SA and training treatment had higher not available for Wisconsin. costs than benefits. 120 Journal of Economic Literature, Vol. XXXIII (March 1995) earnings. Unfortunately, earnings are only available for the DOL evaluated experiments. These experiments indicate earnings increases are associated with less UI receipt. However, I should emphasize that most of the estimates of earnings impacts are not precise. Overall, the job search experiments are very encouraging, but conclusive societal cost benefit analyses will require more precise earnings impacts. In addition, none of the cost-benefit analyses include the cost of any additional burdens placed on claimants as part of a strengthened work test. These costs are likely to be particularly important in the experiments that put less weight on job finding assistance such as Washington and Charles- ton. F. Interpreting the Effects of the Experiments A key question in the interpretation of the job search experiments is the relative importance of increased services and work search requirements in the determination of outcomes. It is clear that a wide range of treatments was successful because the Wisconsin experiment had very little additional enforcement while the Washington experiment treatments showed little effect on measures of services such as referrals and placements. The other experiments seem to lie somewhere in between, however it is difficult to discern the relative importance of services and requirements in a treatment where they are combined. In New Jersey the report authors argue that the direct effects of the increased compliance checks were small. Claimants were unlikely to be disqualified for failure to report for services. The Charleston authors indicate that reporting requirement coupled with the cessation of payments were the most important part of the treatments. While in Nevada, the author of the report argues that continuity, i.e., returning to same UI/ES team and the individualized services were the key (see Corson et al. 1989, pp. 275-77; Corson, Long, and Nicholson 1985, p. 108; and Steinman 1978, pp. 48-50). Thus, the bottom line is that a wide range of approaches seem to be successful. A key advantage of the job search approach to UI reform is that it is unlikely to increase entry into the pool of UI claimants. Unlike a reemployment bonus, job search assistance does not increase the incentives to file for UI if one already has a job lined up or expects to have a short spell. The experimental treatments that make continued receipt of UI more costly, such as some of those in Washington and those in Charleston, might reduce the pool of UI claimants. The possibility remains that those encouraged to find a job quickly will do so at the expense of others who are displaced by those workers. However, to the extent that job search assistance improves the matching of workers and jobs, displacement may be less important. In addition, several of the experimental treatments led claimants to use the employment service more intensively. It is an important unresolved question as to whether this will draw down the quality and quantity of available job listings for others. Alternatively, it might induce employers to list more jobs with the ES as the number (and possibly the quality) of potential applicants grows. One should not take the results of the experiments to indicate that the job service should haphazardly make more referrals. Such a strategy could sharply reduce employers' willingness to list jobs. It is possible that part of the experimental impacts of the job search treatments were due to Hawthorne effects if claimants responded to being in an experiment per se rather than the intended treatments. Hawthorne effects may be 121Meyer: Unemployment Insurance Experiments more plausible for the job search experiments than the bonus experiments because they involved repeated contact with claimants which the bonus experiments did not have. As mentioned above in Section 2 E., the imprecision of this hypothesis makes it hard to rule out completely. However, with the job search experiments there is stronger evidence that more intensive treatments produced larger responses, and treatments which were expected to increase UI receipt did so. The most intensive treatments were in the two Nevada experiments which also had the largest UI duration responses. The intensity of these treatments is indicated by the descriptions of the services, the large effects on referrals and placements, and the large administrative expenses per claimant. Other experiments such as New Jersey and Washington had small or no effect on service provision, and Charleston and Washington administrative expenses were extremely low because few services were provided. Washington also had two treatments which could have been expected to have no effect or increase UI receipt, and the results supported this. The exception reporting treatment lead to more weeks of UI and the second treatment that was similar to the control group had no change in weeks of UI. Thus, the pattern of the responses in the job search experiments is not consistent with Hawthorne effects being the main cause. A potentially more important problem with the Nevada experiment is that the job service personnel may not have been representative of the entire pool of service providers. In Nevada, the treatment group received assistance from four twoperson teams. If job service personnel differ substantially in their ability to assist claimants, a sample of four is not large enough for such differences to average out. The outcome differences between treatment and controls may be partly a measure of the personnel's ability rather than the effect of the services provided. 4. Implications of the Experiments for Theoy Two main approaches have been used by labor economists to analyze unemployment: labor supply models and search theory. These approaches can be used to understand the results of the bonus experiments and, to a lesser extent, the job search experiments. Labor supply theories such as Moffitt and Nicholson (1982) model unemployment in a static labor-leisure choice framework. An individual's utility is taken to be an increasing function of income and unemployment, where unemployment is valued because of its leisure component. It is assumed that an individual can become reemployed at any time and search behavior does not affect the reemployment wage. An individual maximizes utility subject to a budget constraint that is altered by UI. Further assume that the period for this maximization is a year and that any leisure during the period must be taken in the first spell of unemployment. A reemployment bonus then raises the budget constraint by the bonus amount over the range of unemployment less than the qualification period. Figure 1 displays the original and modified budget constraint created by a bonus program. This change in the budget constraint has different effects depending on a person's preferences and thus their location on the original budget constraint. The effect of the bonus on the combined population is uncertain. If initially a person was unemployed for less than the qualification period, then the income effect will cause her to lengthen her unemployment spell. On the other hand, if she 122 Journal of Economic Literature, Vol. XXXlIl (March 1995) Income o) Bonus offer budget segmentI /- Bonus amount Qualification UI Unemployment Period Potential (weeks) Duration Figure 1. was originally unemployed for longer than the qualification period, she has an incentive to reduce her unemployment spell to receive the bonus. While the theoretical effects of the labor supply model on unemployment are ambiguous, the likely effect of a bonus offer is to decrease mean unemployment duration. Certainly, the first effect is limited to small changes in duration which keep unemployment spells less than the qualification period. Search theory provides a reason other than the consumption of leisure for why an individual might choose some unemployment. Dale Mortensen (1987) analyzes the effects of a reemployment bonus using a simplified version of the models described in Mortensen (1986).34 In this partial equilibrium model, individuals are wealth maximizing and have a constant search intensity. Assume constant, but different, offer arrival rates when unemployed and employed, and a stationary wage distribution. With some additional assumptions, this model implies a lower reservation wage with a bonus than without, during the qualification period. This result translates into a 34 See Levine (1991) and Anderson (1992) for further extensions of this model. higher hazard of exit fronl unemployment during the qualification period and shorter mean unemployment duration with a bonus offer. The bonus experiment mean duration and earnings numbers can be interpreted using these theories. The reduction in mean UI receipt for most bonus treatments is consistent with both of these models. The lack of significant negative effects on earnings is consistent with labor supply models where the wage is fixed. But, because there are large standard errors around most of these earnings numbers, it would be inappropriate to reject search theory. The earnings numbers are also consistent with most of the adjustment to the bonus offer in a search model coming through search intensity rather than the reservation wage. While these models can explain the changes in UI receipt and earnings, they are less successful in explaining the second order question of the differences between the results of the different experiments. Levine (1991) tests whether a simple search theory model fits the data from both the Illinois and New Jersey bonus experiments. Given the large differences between the bonus responses in the two states (see Table 2), he finds that it is difficult to reconcile the results of the two experiments. Davidson and Woodbury (1991a) argue that the differing results of the Illinois and Washington experiments may be due to the availability of FSC in Illinois, but the results of their theoretical model are surprising and do not seem very gen- eral. Besides their implications for mean unemployment duration, there are several other important predictions of these models and their extensions. Because of the discontinuity in the budget set in the labor supply model at the end of the qualification period, many people will maximize their utility by receiving UI for 123Meyer: Unemployment Insurance Experiments exactly the qualification period.35 One must argue informally, as Moffitt and Nicholson do in their paper, that the random nature of job finding leads people to cluster around this discontinuity point. This modification would lead tothe prediction of a rising hazard just before the end of bonus eligibility. Mortensen's search model also im~liesthat theL hazard rises as an unemployment spell progresses until one reaches the point where bonus eligibility ends. Then, the hazard drops discretely to a constant lower There is onlv weak evi-J dence from the Illinois employee experiment for a higher hazard before the end of the qualification period. Meyer (1991) finds that the hazard rises eight percent in the last two weeks, but this change is not significantly different from zeri. A visual examination of the Pennsylvania hazards appears to show an increase in job taking just prior to the end of eligibility, but the changes are unlikely to be significant given the sample sizes (Corson et al. 1992, p. 103).Thus, the Illinois and Pennsylvania data only weakly show the increases in job finding predicted by labor supply and search theories.37 One might expect that formal search or labor supply models of unemployment would predict that those with lower earnings, or lower weekly UI benefit payments, would respond more to a bonus of a given dollar amount. These im- 35 Levine (1988) simulates the effects of the bonus using four different sets of preferences and finds that between one-quarter and one-half of the sample is located at the discontinuity in the budget set. His results might be softened if incomplete participation were assumed. 36 Most of the simulations performed by Mortensen using this model show a sharp increase in the reem lovinent hazard just before the end of F . . .the bonus e lgibllitYperiod. 370ne would not expect similar behavior in New Jersey since the bonus decreased to zero at the end of the eligibility period. An examination of this issue is not possible in the ~ublishedU7ashington data sincLthe length of tLe eligibility period varied across individuals. plications are shown in Meyer (1991) which also examines the evidence from the Illinois experiment. The effects generally point in the expected direction, but are small and insignificant. One cannot examine this issue in the other bonus experiments because the bonuses are proportional to weekly benefits. Interpreting the job search experiments in a formal way is more difficult than it is for the bonus experiments. The job search experiments provide a mix of useful information and increased requirements. The information, which includes referrals, suggestions of search areas, aid in writing resumes and interviewing techniques can be interpreted as increasing the productivity of search.38 Alternatively, the increased enforcement of work search rules can be interpreted as making the leisure component of unemployment more costly or inducing more search. Both interpretations are consistent with the experimental results of shorter spells of UI receipt and earnings that do not appreciably decline and may even increase. Some of the experimental data can be used to examine search models with the possibility of recall to a previous job. The data are also informative about the role of recall expectations on job finding. Models incorporating these characteristics can be found in L. Katz ( 1986). In one of his models an individual revises downward her expectation of being recalled as her unemployment spell continues. Anderson (1992) examines the New Jersey experiment with these models in mind. She finds that the bonus offer and the job search assistance workshop have little effect on the recall rate, as we would expect because the experiment did not allow recalled workers to 3Qven if the control grou investment in p. . .search productivity is privately e ficlent, it may be beneficial for the government to provide assistance if it reduces UI program costs. 124 Journal of Economic Literature, Vol. XXXIII (March 1995) receive a bonus. She further finds that the job search workshop seems to have much of its effect through convincing people who initially expect to be recalled to consider searching for a new job. She also finds a declining negative effect of initial recall expectations on new job finding, which agrees with L. Katz's model of worker updating of recall ex- pectations. Besides their implications for theories of unemployment, the bonus experiments and their low takeup rates are informative about theories of program participation. As in Moffitt (1983), economists typically explain incomplete participation by transaction costs or stigma. The results of the bonus experiments agree with past findings of low participation in many social insurance programs. As indicated above, the takeup rates for the bonuses ranged from 55 to 66 percent.39 These numbers provide further evidence of stigma or transaction costs associated with government grants. The low participation is surprising given that the receipt of a bonus typically required filling out only one or two forms. An important role for transaction costs rather than stigma is suggested by these results as potential bonus recipients had already overcome any stigma of initially applying for UI. Follow-up surveys for two of the bonus experiment also suggested that stigma was not as important as lack of information about and trust in the experiment, forgetfulness, and the complications of the procedure (Spiegelman and Woodbury 1987, pp. 7.5-7.7; and Spiegelman, O'Leary, and Kline 1991, pp. 183-85). The extremely low 12 percent takeup rate in the Illinois Employer Experiment (where the employer received the $500 bonus) also suggests 39This omits the employer experiment in Illinois where the takeup rate was a minuscule 12 percent. the difficulties that employees have with telling their employers that they are "on sale" and possibly negotiating a side pay- ment.4O 5. Implicationsfor Policy A number of conclusions can be drawn from the four reemployment bonus and five job search experiments. First, the bonus experiments show that economic incentives do affect the speed with which people leave the unemployment insurance rolls. UI is not a completely benign transfer; it affects claimants' behavior. This is shown by the declines in weeks of UI receipt found for all of the bonus treatments, several of which are statistically significant. While the halfweek impacts that were typically found may seem small, one should remember that typically only about 15 percent of claimants received a bonus in most treatments. Only about 30 to 40 percent of claimants found jobs during the qualification period, and typically only about half of these submitted the paperwork indicating they had found a job. The experiments also show that speeding claimants' return to work does not appear to significantly decrease quarterly earnings and may increase total earnings following the claim. With large standard errors for most treatments, we cannot say anything stronger. If further research supports a conclusion of no effect on quarterly earnings, it would be an important finding and would imply that small changes in the generosity of UI are unlikely to affect the reemployment wages of claimants. I have also argued that the results do not show the desirability of a permanent 40Also see John Donohue (1989) who argues that comparisons of the Em loyer and Claimant Experiments in Illinois provi1e a test of the Coase theorem. 125Meyer: Unemployment Insurance Experiments reemployment bonus program. The simple cost benefit analyses that do not account for entry effects show that most bonus treatments generate small losses for the UI system. Corresponding societal net benefits are positive about onehalf of the time and negative in the other cases. Furthermore, accounting for entry effects would make cost-benefit analyses of a bonus program less favorable for both the UI system and society as a whole. A reemployment bonus makes filing for UI much more valuable, as claimants become eligible for a large payment if they file and then find a job soon. Empirical estimates discussed above suggest that these effects might be large. An important caveat on the societal cost benefit analyses is that they are not definitive because they rely crucially on earnings estimates that are im- precise. These observations also suggest that the optimal UI literature, such as Steven Shave11 and Laurence Weiss (1979) and Hausman (1984), has overlooked a key element in designing a pattern of benefits. Front loading benefits is likely to have an effect on entry into UI, both through takeup by the unemployed and through firm temporary layoffs. One cannot alter one part of the UI payment structure to reduce moral hazard and expect all other behavior to remain un- changed. On the other hand, the job search experiments show that various combinations of increased enforcement of work search rules and additional job finding services can reduce UI receipt and unemployment in a cost effective way. Nearly all of the combinations tried by the five experiments reduce UI receipt, and the reductions in UI receipt are often statistically significant. The more intensive treatments tend to have bigger effects, but unfortunately the largest impact occurred for the least well designed and evaluated experiment. Nearly all of the treatments have benefits that exceed costs for the UI system. If one accounts for any reduction in the excess burden of taxation as government expenditures and taxes fall, the calculation would likely lead to more positive cost benefit analyses. Societal level cost benefit analyses do have some uncertainty because the earnings change estimates, when available, have large standard errors. These results suggest that there are benefits to UI claimants and society from strengthening the work test and the employment service. In thinking about the form changes should take a key issue is the weight put on enforcement versus services. It is clear from the Washington exception reporting treatment that some enforcement of the work test is necessary. This treatment increased UI receipt by over three weeks and led to little change in job quality. The social value of strengthening the work test as in Charleston and some of the Washington treatments depends on whether the increased costs fall disproportionately on claimants who are not actively seeking work. It seems likely that this is the case, but the experiments do not examine this issue directly. They do tend to find that UI officials are reluctant to disqualify claimants for lack of participation in treatments if that was their only work test violation. The experimental results, as well as work such as Burgess and Kingston (1990) indicate that there is a role for further search requirements as part of the bargain for UI receipt. On the services side we should consider making job search assistance universal. The exact combination of services we should include is not completely clear, but job search workshops and individual attention by the same personnel seem promising. The uncertainty about the right combination of services leads to the next section. 126 Journal of Economic Literature, Vol.X X X I I l (March1995) 6. The Valueof the Experiments and Future Experiments The bonus experiments should convince any hardened skeptics that monetary incentives have a substantial effect on job finding. Because there is already a wealth of nonexperimental evidence on the effects of unemployment insurance payments on unemployment duration (Theresa Devine and Nicholas Kiefer 1991, Ch. 5; and Atkinson and John Micklewright 1991), it is likely that the number of such skeptics is small. However, there are people both within and outside the economics profession that are only convinced by experimental evidence. We are also unlikely to learn more from future experimentation with reemployment bonuses as it is hard to generalize from bonus experiments to the effects of other UI reforms. Furthermore, experimental methods cannot easily examine takeup and other entry issues which are crucial to evaluating bonuses. Examining such effects would require randomization at the site or state level. Even this research design is unlikely to be successful because the magnitude of the component of variation in outcomes attributable to the site itself is likely to be large. Job search experiments have both been more successful to date and are more promising for future experimentation. Nonexperimental evaluation of the employment service or the UI worksearch requirement is very difficult. To evaluate UI search requirements, there are few quantifiable differences between state systems that one can use to estimate effects on unemployment durations or earnings. In contrast, the effects of monetary incentives can be examined using the many differences in benefits, across states, within states, and over time, that are easily quantifiable. The job search experiments provide evidence that is difficult, if not impossible, to obtain from nonexperimental sources. Similarly, to estimate the effects of the employment service one needs a group to compare to those who register. An appropriate comparison group is generally not available. The decision to use the employment service is typically made by a worker and cannot be considered to be independent of that worker's observable and unobservable characteristics. In addition, if one examines the effect of ES referrals, biases would result if ES employees select the most suitable registrants to refer based on characteristics that the researcher does not observe. Such selection is likely to occur as ES employees seek to satisfy employers that provide job listings. A further complication arises because the use of the service takes place over time. The unemployed typically use it only after they have unsuccessfully tried other means of finding work. Thus, simple comparisons between those that do and do not use the ES spuriously indicate that the ES lengthens their unemployment durations.41 Given these impediments to research, it is not surprising that the nonexperimental evidence contrasts sharply with the evidence from the job search experiments. Various authors have tried to overcome this lack of an appropriate comparison group in studies of the effectiveness of the employment service, but the results are not convincing.42 In fact, some of the papers find that UI job search requirements prolong unemployment as do visits to public employment 4lSee Arnold Katz (1980) who explains this argument in detail. 4Qee A. Katz (1980), T. Johnson, Katherine Dickenson, and Richard West (1985), Michael Keele and Phili Kobbins (1985), Carol Komero, Dona6 Cox, anRA Katz (1991) and Lars Osberg (1993). The identification strategy in Steven Director and Frederick Englander (1987) is more convincing, but it is difficult to estimate effects using aggregate data. 127Meyer: Unemployment Insurance Experiments service.43 These results clearly contrast with the results of the job seaich experiments. The Washington treatment which relaxed the job search requirement appeared to prolong unemployment, while those that tighten it seem to shorten spells. The provision of additional job search assistance seems to speed the return to work. If we were t d believe the nonex~erimentalevidence. it would im-I ply that not only are the employment service and UI search requirements a waste of money, but that