Abstract In the current political and economic climate, there are rapidly escalating demands from public funders and other stakeholders for indicators which can capture the achievements and impacts of the voluntary sector. Having clarified the context of these demands, the primary aim of this article is to develop criteria for measuring the 'performance' of voluntary organizations. This is undertaken by marrying relevant international theoretical literature suggestive of their roles and contributions with a 'production of welfare' approach, which has a proven track record in the evaluation of human services. Eight domains of performance are suggested (economy, effectiveness, efficiency; choice/pluralism; equity; participation; innovation and advocacy), embracing twenty-two separate indicator sets. MEASURING THE PERFORMANCE OF VOLUNTARY ORGANIZATIONS Jeremy Kendall and Martin Knapp Key words Nonprofit organizations, performance, third sector, voluntary organizations Jeremy Kendall and Martin Knapp Personal Social Services Research Unit (PSSRU) London School of Economics Houghton Street London WC2A 2AE, UK Vol. 2 Issue 1 2000 105-132 Public Management ISSN 1461-667X print/ISSN 1470-1065 online © 2000 Taylor & Francis Ltd 709 106 Public Management: an international Journal of research and theory INTRODUCTION The social, economic and political importance of voluntary (non-profit) organizations across the world has never really been in doubt, although it may have taken the formal mapping of their scope and scale to drive home the message (Salamon et al. 1999). It is clear that the governments of many countries rely heavily on indigenous and/or international voluntary organizations to resource or deliver basic goods and services. Indeed, often large (and, in many fields, growing) amounts of public money are routed through the sector, either directly or indirectly. These resource transfers might be made through grants, contracts for specific services, tax exemptions, secondments of staff, free places on training courses or in other ways. They may come from any number of tiers of domestic governments, as well as through transnational bodies such as the European Union and the United Nations. Growing attention is consequently being focused on the performance of voluntary organizations. Numerous forces have combined to increase the pressures of demand for such data. In fact, few voluntary organizations are strangers to outcome or performance assessment of one kind or another: their very existence often requires them to argue their effectiveness in tackling social problems, supporting vulnerable individuals or promoting particular causes. However, protected from the full rigours of the market, voluntary organizations have no indicator of net profit or stock market price to measure their performance. And unlike public authorities which are always, in principle, scrutinized periodically by the electorate, and increasingly now also inspected and audited by independent or other quasi-independent bodies, many voluntary organizations lack democratic governance and are unlikely to find themselves suddenly dumped out of office. For sure, often voluntary organizations do have some institutional arrangements designed to enable internal or external monitoring. But the absence of either a simple and widely acknowledged 'bottom line' or (often) the discipline of the ballot box is at the root of some of the uncertainty as to their roles and achievements. It also generates conceptual and empirical difficulty in relation to performance assessment. This latter observation is the prompt for this article, whose primary aim is to develop and elaborate theoretically relevant criteria for measuring voluntary organization performance. We start by examining the context within which such performance measures are sought. We then discuss what prompts voluntary activity, and those theoretical perspectives on roles, activities and achievements relevant to performance identification. Section 4 ('A framework for performance evaluation') sets out a conceptual framework for identifying the core dimensions of performance, and section S ('Developing performance indicators') turns to the measures which follow from the framework and the theoretical arguments on which it is built. The final section draws the article together and discusses the generalizability of the performance framework suggested here. Before we proceed we should clarify what we mean by a voluntary organization. Kendall & Knapp: Measuring the performance of voluntary organizations 107 Definitions of 'the voluntary sector' are often contested because the boundaries around the sector are fuzzy and poorly understood (Kendall and Knapp 199S; Salamon and Anheier 1997). For the purposes of this article we use Salamon and Anheier's (1997) structural-operational definition which conceptually employs the following criteria: nonprofit distribution; constitutional independence from the state; formal organization; and receipt of voluntary input. It is important to be explicit about this because other definitions have been used by other scholars and in public management, and will undoubtedly have relevance for how performance and its measurement might be conceptualized. PERFORMANCE DEMANDS Voluntary organizations attract a multiplicity of stakeholders, who may hold divergent views on the primary roles of voluntary activity, and hence different expectations of what organizations can or should achieve. There are also likely to be different perspectives on how those achievements are to be measured (Kanter and Summers 1987). As we noted at the outset, the public sector is often an important stakeholder in the sector. A study of the economic character of the voluntary sector in nineteen developed countries found that on average 42 per cent of total revenue originated from government, 47 per cent from private earnings and 11 per cent from private giving (Salamon et al. 1998: 11). Public accountability is thus a major source of demand for performance measurement. Another is accountability to other funders. The voluntary sector in some countries receives more of its revenue from non-state sources, and in some fields and regions these latter will be substantially more important. Many non-government funders will have their own accountability needs and demands. Larger charitable foundations, for example, may carry out checks on recipient bodies. Smaller funders — and especially individual donors — will not have the same opportunities to monitor performance but may be able to rely on publicly appointed regulators, such as the Charity Commission in England and Wales, or the Internal Revenue Service in the US. Furthermore, organizations themselves need information for their day-to-day operation, especially to monitor expenditure and whether they are performing in pursuit of their charitable (or other) objectives. Their goals may include an intent to employ paid staff or involve volunteers so as to improve the impact on their target client groups or communities (that is, in the terminology of later sections of this article, to improve their outputs or outcomes from a given set of inputs). They may want to monitor their effectiveness in tackling social problems (where they are service-providing organizations) or in bringing such problems to public attention (if they have a campaigning role). 108 Public Management: an international Journal of research and theory Individual people are important stakeholders in the voluntary sector, whether as volunteers, employees, donors, fee-paying clients or beneficiaries. Some individual stakeholders will be in a position to make explicit demands for performance data — at the organization's annual general meeting, on the management committee, through the media and so on — while others, and particularly those whose characteristics leave them vulnerable and without viable 'exit' or 'voice' options (Hirschman 1970), will need representation and support. Advocacy groups can thus be seen as another source of demand for performance data. Before we discuss the ways in which these various demands for performance monitoring can be responded to appropriately, we need to take a closer look at what prompts voluntary activity and makes it distinctive. This will provide theoretical foundations upon which to build a plausible conceptual framework, and will also suggest salient dimensions of performance. THEORETICAL UNDERPINNINGS AND THEIR IMPLICATIONS The international academic literature contains a number of theories, most of them built on empirical observation (and certainly which themselves have stimulated new empirical research), which hypothesize reasons for voluntary organization existence, roles, activities and achievements. We do not go through the literature theory by theory (see Kendall and Knapp 1999, for such a review in the discussion of 'evaluation'), but instead pick out five major themes, and identify their implications for performance and its measurement. Public goods and externalities In orthodox economics, market forces ensure the efficient allocation of resources in line with citizens' tastes and expressed preferences. But some markets 'fail', for example those for goods and services which are jointly consumed, non-excludable and non-rival. These so-called public (or quasi-public) goods will be produced at sub-optimal levels in a free market, suggesting a prima Jacie case for government provision. But governments may not be able or willing to meet all demands for certain quasi-public goods, or may not offer the right quality of output, creating a demand for services produced by voluntary agencies (Weisbrod 197S, 1988). Individuals may have sufficient incentive to join together to produce more or different outputs ('differentiated demand') to meet their otherwise unsatisfied demands, and free-riding and high transaction costs may make it more likely that voluntary rather than for-profit firms will be established. A parallel argument can be made with regard to distributional issues. It is well established that free market outcomes may be both inefficient Kendall & Knapp: Measuring the performance of voluntary organizations 109 and inequitable (Hockman and Rogers 1969). Concerns about the latter can prompt the founding of voluntary bodies when governments are unwilling or unable to act. The 'public good' or externality features of the goods, services and processes of voluntary organizations may leave their 'products' unpriced or with prices that diverge significantly from marginal social opportunity costs (Weisbrod 1996). Moreover, because of the limited ability of the for-profit sector to function in these contexts, voluntary organizations may accumulate considerable market power, even developing monopoly power (albeit often innocently). This creates difficulties for performance assessment. Competition cannot be used as a yardstick because the market does not send out reliable or interpretable signals. Thus prices will not reflect true social values, and the figures in annual expenditure accounts will not reflect the true social costs of activities (production). If there are no or few comparators, benchmark standards will not exist, and consumers may find it difficult to signal disapproval because of limited opportunities to take their business elsewhere. The public good properties will also make it inherently difficult to measure outputs, since the impacts are felt widely. Asymmetric information, transactions costs and multiple stakeholding A second theme in the theoretical literature builds on the observation that information is often scarce and not available equally to everyone. Again the free market is the metaphorical starting point, but because of information difficulties (particularly asymmetries), purchasers and donors may feel vulnerable to exploitation by for-profit producers (Hansmann 1980, 1987). Purchasers and donors are not able to monitor output quantity or quality to their complete satisfaction, and so prefer to purchase from producers in whom they have greater trust. And because voluntary organizations do not distribute profits to any owners — a fundamental principle of many legal systems, including English charity law — or because altruistically oriented individuals are assumed to 'self-select' into the sector, these organizations are seen as more trustworthy. More generally, voluntary organizations can be efficient responses to the (potentially high) transactions costs of monitoring (Krashinsky 1986). Ben-Ner and Van Hoomissen (1993) stress the role of actors who are simultaneously demanders and suppliers as 'coalitions' in pulling together the necessary resources to allow 'production' to take place. 'Social constructionists' have emphasized how these stakeholders, despite the evident incentives to work together, tend at the same time to hold different values and priorities. These conflicts of interest and the resultant ambiguities in organizations' operations that emerge to diffuse them, may occur in many types of organization, but arguably are particularly associated with the voluntary sector (Billis 1993; Billis and Glennerster 1997). These theories suggest that the voluntary sector could be disproportionately populated with organizations whose impacts are hard to measure, generating high 110 Public Management: an international Journal of research and theory transaction costs, involving multiple interests and being ambiguously interpreted. In sum, the consequences of an organization's activities have certain features. They are: • distributed over long periods of time; g Pe • physically remote from the funder or donor; • only reliably assessed by the individuals receiving services ('experience goods'), but when users are unable to judge quality or pass on their judgements; • understood or appreciated asymmetrically by producers; and contested by different stakeholders. • Many of these features characterize voluntary sector activities, and in combination, generate particular difficulties for performance assessment. Trust, social capital and voluntarism A third theme — trust — pervades almost all conceptual approaches to understanding the voluntary sector. It is central to Hansmann's theories, as we have just seen, and helps to explain how organizations fraught with divergent stakeholder interests and ambiguous goods can 'muddle through'. And it has been developed in Coleman's social capital approach in a broader and more dynamic vein. Its contribution is recognized in participation as well as in service provision, it is associated with multiple and not just dyadic transactions, it is seen as cutting across organizational boundaries and it is treated as a resource that increases with use and decays with disuse. The term capital, as part of the concept, implies a resource or factor input that facilitates production, but is not consumed or otherwise used up in production . . . Social refers in this context to aspects of social organisation, ordinarily informal relationships, established for non-economic purposes, yet with economic consequences. (Coleman 1993: 175) To this last phrase, we can add political and social consequences. Social capital is an intangible resource to be found in relations between people (networks) rather than in inanimate objects (physical capital) or individuals (human capital). These networks and the trust they can facilitate are clearly not uniquely associated with the voluntary sector (Foley and Edwards 1997; Dasgupta and Serageldin 1999). However, people's creation of, and subsequent involvement in participatory voluntary organizations may either represent social capital per se, or create it as a by-product (Putnam 1993). The net result of participation is argued to be citizens who are more trustworthy, community-aware and other-regarding Other political analysts have developed similar arguments around the benefits of voluntarism, without applying the 'social capital' label (Bellah et al. 198S; Verba et al. 1996). Some social psychologists have emphasized the importance of voluntary participation in building self-esteem and Kendall & Knapp: Measuring the performance of voluntary organizations 111 enhancing feelings of competence and self-worth (Deci and Ryan 198S). Finally and more generally the motives underpinning volunteering or membership activities include not only altruism and empathy (Schroeder et al. 199S), but also social adjustment aims (to fit normative expectations of behaviour, gain social prestige or expand social networks), search motives (to learn more about an organization or movement, and perhaps influence it) and instrumental aims (to acquire skills, experiences or contacts to improve career opportunities; Knapp et al. 1996). As we shall see, volunteering and membership can be measured, but nevertheless pose challenges for performance evaluation because participants are often more than just 'human resources' helping an organization to deliver services. The social capital literature (and its precursors and offshoots) emphasizes that the 'impact' of an organization may be inter-temporal, and that the processes associated with voluntary action can themselves have public good and trust attributes. Volunteering and membership activities might also generate utility by changing participants into fuller citizens. Participation can equip them with skills and orientations which empower them and make them more 'productive' in economic, social and political contexts within and beyond the confines of voluntary sector organizational life. The net result is that participants may be more 'virtuous' in the general philosophical sense, a disposition which can embrace both a greater 'capacity to discern the . . . public interest', because participation has heightened awareness of their fellow citizens, and the motivation may develop 'to act as the public interest, so discerned, requires' (Brennan 1996: 2S6). Advocacy and campaigning Advocacy and campaigning are activities closely associated with voluntary organizations. They seek to achieve social and political change by pushing otherwise neglected issues up the political agenda, questioning prevalent assumptions, and more generally proposing alternative technologies or assumptive worlds. This is most obviously the case for 'social movement organisations' (McAdam et al. 1988), but many organizations are 'polyvalent' (Evers 199S) combining service provision and participation with advocacy and campaigning. (In fact, empirical evidence from the US and UK suggests that deliberate advocacy for social change is actually only pursued by a small minority of voluntary organizations; Knight 1993; Salamon 1993; Shore et al. 1994.) Wolch (1990) developed a complex, 'state-centred' model drawing on the Marxian tradition of social analysis, new institutionalism and Giddensian social theory. She identified three continua along which voluntary organization outputs can be located: commodified-------------non-commodified direct service-------------advocacy elitist-------------participatory 112 Public Management: an international Journal of research and theory The first dimension is primarily concerned with public good aspects, and the third with voluntarism and social capital. The contrast between these and the second dimension is: 'that various types of output can generate social change, but some are more directly designed to produce social change than others. Advocacy output seeks to direct or create social change by influencing public policy or corporate practices' (Wolch 1990: 25). Evers' (1993, 199S) welfare mix approach portrays associations as bringing specific values into this arena — reflecting the imprint of, for example, churches, new social movements, or demographic or cultural minorities from which they grow — but emphasizes that these values influence rather than determine their goals and actual behaviour, since they must also respond to economic and political pressures. The sector is part of the 'tension field' of civil society, in which the logics of other sectors are played out. Perhaps the most likely place to find conclusions about what this means for performance is in the social movements literature. However, an exhaustive review concluded that: rarely . . . have movement scholars sought to assess how effective movements are in achieving their ends. Nor have researchers been any better about studying the impact of collective action on society as a whole or on those who participated in the movement. (McAdam era/. 1988: 727) The evidence rehearsed by McAdam is largely descriptive. The failure to advance comparative performance research on 'social movement organizations' or polyvalent organizations is no accident, but results from problems of attribution in complex political environments, and a variety of hazards of strategic misrepresentation of views by users (6 and Forder 1996: 229—30). Performance measurement may have to rely on indirect measures of actual effects, or subjective impressions of impact, or even simply (but uninformatively) measures of resources allocated to this activity. Innovation A final dimension of performance routinely rehearsed as a contribution of voluntary organizations is innovation, but this remains relatively undertheorized. Osborne's (1998) pioneering empirical analyses in the social welfare field pointed to the association between multiple dependencies in complex networks and 'institutional context' (including relations with government) and the presence of innovation. He suggests that 'institutional contexts' perform the role in the voluntary sector that profit does in the for-profit sector in inducing innovation, but as with other analysts his empirical evidence is confined to internal comparisons within the voluntary sector. Innovation can be defined as 'the introduction of change in production'. To distinguish it from service development, innovation involves the adoption of observ- Kendall & Knapp: Measuring the performance of voluntary organizations 113 able discontinuities in service design (6 1993; Osborne 1998). Three key varieties can be identified: • product innovation, involving new goods and services, which can be radical ('wholly new' outputs), product differentiating (adaptation of existing outputs) or market differentiating (new users); • process innovation, involving a new technology to produce a given set of outputs; and • organizational innovation, where agencies either adopt a new internal structure, or adopt new sets of external relationships. Ultimately the benefits of innovation could be of two varieties: new service configurations or technologies which are more effective, efficient or equitable for beneficiaries, and the accumulation of skills of the social capital type whose benefits extend further. A FRAMEWORK FOR PERFORMANCE EVALUATION This review of theories suggests the relevance of a variety of performance criteria, including those familiar from 'new managerialist' approaches to public administration — effectiveness, economy and efficiency — but also choice and diversity, social justice or equity, participation, advocacy and innovation. We now define these criteria with more precision and seek to locate them within a framework to structure the design, collection and interpretation of indicators. The very development of consolidated indicators implies that we seek to treat value-laden issues as 'objectively' as possible, and do not attempt to explain how these conflicts are socially constructed: we treat this as analytically a separate problem, logically prior or external to the approach taken here. (See Kendall and Knapp 1999, for a review of social constructionist approaches and applications.) We do so by moving through a number of stages, illustrating each diagrammatically and making links between the theoretical perspectives identified earlier and their implications for practical measurement. We will argue that taking into account the distinctive features of voluntary organizations entails going beyond for-profit sector or new public management models of resource—outcome relationships. The 'production of welfare' framework A starting point found to be useful in our own health and social welfare research draws analogies to economists' theories and empirical methods of cost and production 114 Public Management: an international Journal of research and theory relations. We should stress that we are arguing by analogy, and we will quickly broaden the argument from what may look 'too economistic' an approach. In fact the production of welfare (POW) framework, as it is known and has been developed in the Personal Social Services Research Unit, stems only in part from economics, and has been heavily influenced by other disciplines (Davies and Knapp 1981; Knapp 1984). In its previous applications, the POW framework has performed well by the criteria suggested by, for example, Deutsch (1963) for choosing between different models in the social (or physical) sciences. The framework is economical in its description of processes of service delivery and activity, and it focuses attention on factors, purposes and processes on primary significance in the contexts in which it has been developed. Third, the POW approach has explanatory or predictive capacity including in terms of Deutsch's sub-criteria of rigour (reliability), combinatorial richness (generating multiple hypotheses) and organizing power (generalizability to other contexts). The main elements of the POW framework are: • resource inputs — mainly staff, volunteers, members and capital; • the costs of those resource inputs, or alternatively the budget for an agency used to purchase resource inputs, plus recognition of opportunity costs; • non-resource inputs — influences on the achievement of outcomes (see below) which do not have an identifiable price or are not (currently) marketed. Examples are the social milieu or 'ethos' of the contextual environment; and the assumptive worlds (opinions, attitudes, ideologies) of individual workers, managers, volunteers or members which help to generate this environment; • intermediate outputs or outcomes — volumes of service output, probably with a quality dimension, perhaps weighted for user characteristics ('casemix'); • the final outcomes — most simply changes over time in the welfare, quality of life and field-specific status (e.g. educational attainment or health) of end users induced by the (productive) activity in question, as well as appropriate externality effects. Final outcomes are influenced ('produced') by the volume and quality of services provided (the intermediate-outputs), which in turn are made possible by employing combinations of resource and non-resource inputs. Resources are procured by spending money (budget costs) and/or by diverting them from alternative uses (opportunity costs). Figure 1 summarizes these hypothesized links. As a simple organizing framework, POW ought to be able to contain numerous theoretical concepts, evaluative approaches, professional objectives and stakeholder goals. In its original formulations, the POW framework helped to identify and define four criteria that have become common currency in public management discussions in the UK: economy, effectiveness, efficiency and equity (Figures 2 and 3). Kendall & Knapp: Measuring the performance of voluntary organizations 115 Costs or budget Resource inputs Non-resource inputs Intermediate outputs -> Final outcomes Figure 1: The production of welfare EFFICIENCY Costs or budget \ ECONOMY Resource ,, inputs Non-resource inputs Intermediate outputs \ -» Final outcomes EFFECTIVENESS (intermediate) EQUITY (intermediate) EFFECTIVENESS (final) EQUITY (final) Figure 2: The production of welfare and the 4 Es The 'production of welfare' in context The 'production' process is neither closed nor static. There are many influences on the main elements of the system, stemming from their embeddedness within broader social, political and economic contexts. For example, an organization's budget will be influenced by decisions taken by funding bodies as to the amounts they are prepared to allocate to an individual organization or field, and the constraints they put around how funds are used. Some of those decisions will be independent of recipient performance, such as when macro-economic management requires a reduction in government spending. Similarly, the non-resource inputs will be partly shaped externally. The opinions, attitudes and ideologies that motivate and shape stakeholder behaviour are not hatched in a vacuum, but reflect stakeholders' own cultural baggage and predispositions, which in turn are fostered and developed both by the external environment, and internally within the immediate 'production process' itself. On this theme, our earlier discussion of the social capital and innovation literatures should alert us particularly to the efficiency-enhancing properties of one very general 116 Public Management: an international Journal of research and theory Economy ■ resource savings ■ the pursuit of less costly treatments * fairness in access to or use of resources or services Efficiency ■ reducing the cost of achieving a particular level of outcomes ■ or improving the outcomes achieved from a particular level of expenditure Equity Effectiveness ■ improvements in patient or family health and quality of life fairness in achieving improvements in or levels of health and quality of life Figure 3: Evaluative criteria predisposition: the propensity to trust and cooperate with others. At its crudest, the general argument is that these dispositions are strongly influenced by context, where the context should be understood in terms of the 'network environment' — the web of relationships in proximate communities which are critical in shaping attitudes and expectations. For any given organization (or field) whose 'productivity' is being examined, the character and influence of non-resource inputs can be said to reflect the network within which they are found. Consequently, production that takes place in a network environment rich in social capital (high stocks of trust and pro-cooperative dispositions) should achieve better final outcomes than production in a situation with poor social capital. These networks are in a sense 'boundary spanning', facing both inwards and outwards. They are 'internal' in the sense that they affect the attitudes and behaviour of those directly involved in production, and also 'external' because they represent the assumptive worlds of those not directly involved in the organization who, because externalities are involved, experience and react to its 'production'. Putnam (1993) argued that it is the geographical density and character of networks of associations around the communities where production physically takes place that are of particular pertinence. But 'imaginary' communities, or communities of interest could arguably have the same effect, particularly as modern transport and increased economic affluence make it more feasible for people who otherwise live apart to gather together and interact regularly. Of course, sustained 'virtual' proximity for members of communities of interest is also possible without face-to-face contact through telecommunications and internet media. The trust thus generated may be Kendall & Knapp: Measuring the performance of voluntary organizations 117 -j Macro-level context (institutional/societal environment) ------[_ Meso-level context (network environment) Costs or budget Funding decisions by government and others Free resources Resource inputs Non-resource ^~ inputs Individual expectations, assumptive worlds and predispositions Intermediate outputs -> Final outcomes Figure 4: The production of welfare in broader context 'thinner' or more 'abstract' (see Newton 1997) and perhaps harder to 'police', but its importance is underscored by massive associated macro-sociological changes (Giddens 1990). In Figure 4, we use the expression meso (middle range) to capture this layer of 'proximate' social context — or 'networks' — in which production is embedded. However, the social movement literature (McAdam et al. 1988) and the views of some other scholars, particularly in response to Putnam's arguments, imply that political institutions at the broadest societal level also influence production. This macro level is distinguished from the meso both by its perceived remoteness from the production process (from the stakeholder perspective), and its greater degree of institutionalization. That this level cannot be ignored is fairly obvious in the case of (conventional) resource inputs (e.g. government budgeting decisions). But non-resource inputs may also be rooted at this macro level, as with nationally sanctioned professional guidelines and other hegemonic values which, on a day-to-day basis, are taken for granted, but nevertheless fix institutional parameters in which production occurs. Broader performance criteria The four core criteria in Figure 2 are likely to be necessary but not sufficient for assessing voluntary organization performance. Opportunities for individuals to participate as volunteers, members and donors may bring benefits which are not easily captured in the 3 Es or equity. Consequently, the simple ability of an organization to expand opportunities for meaningful participation may also be an indication of performance. Our discussion of relevant theories suggested the salience of choice over both the intermediate outputs of organizations (pluralism can be seen as desirable in 118 Public Management: an international Journal of research and theory EFFICIENCY CHOICE Costs or ' budget Resource inputs Intermediate outputs EFFECTIVENESS EQUITY CHOICE ECONOMY PARTICIPATION ECONOMY PARTICIPATION Non-resource inputs -> Final / outcomes EFFECTIVENESS EQUITY CHOICE CHOICE Figure 5: The production of welfare and performance criteria its own right) and over the production process itself. (Consumers may prefer goods produced in an environmentally friendly manner, or where all staff participate democratically in workplace decisions, or which do not exploit low-wage economies.) Figure S thus attaches a broader set of criteria to the POW framework. The dynamic picture Performance assessment by state decision makers usually assumes a constant macro context, and perhaps also meso context. That is, objectives are pursued within the framework of a given set of policies, with the network of actors taken as stable. Yet the advocacy activities of voluntary organizations are often undertaken precisely to challenge these parameters. An important strand of multiple stakeholder and welfare mix arguments (indeed social movement theory more generally) focuses on the dynamic sense in which voluntary organizations inhabit a 'tension field' in which potentially synergetic discourses are undertaken and potentially productive new opinions, ideologies and expectations are generated. With this in mind, lobbying activities can be interpreted as directing resources into various forms of intermediate outputs — such as meeting government officials, organizing conferences, arranging demonstrations — to bring about meso or macro change (arrows from the core to the two outer layers in Figure 6). On the face of it, such action is based on a belief that securing changes in attitudes, opinions and state policies helps in the pursuit of organizational objectives. Hence, the relationship is two-way, potentially feeding back into final outcomes. The arrow between the meso and macro environments in Figure 6 represents the efforts of alliances or networks of voluntary agencies attempting to engineer institutional-level ideological or policy changes. Other performance criteria are inherently dynamic. By definition, innovation Kendall & Knapp: Measuring the performance of voluntary organizations 119 Macro-level (societal context) Meso-level (network context) Costs or budget Resource inputs Non-resource inputs Intermediate outputs -» Final outcomes Figure 6: Feedback effects between levels involves a change in some aspects of the production process. Product innovation, now couched in the POW framework implies a change in the balance and composition of intermediate outputs, such as reaching a new client group, changing existing services or introducing new ones for an existing clientele. Process innovation potentially involves combining resource and non-resource inputs in new ways to create a new 'production function', either enabling the same levels of intermediate or final outcomes to be achieved with different combinations of inputs (presumably at lower cost), or securing a higher level or different configuration of outcomes with existing resources. Organizational innovation could be understood as an attempt to connect with the meso or macro environments in new ways. Finally, we have seen that one strand of the social capital approach amounts to the claim that participation in voluntary organizations actually changes those involved, particularly the capacities and motivations of volunteers and members. 'Feedback' arrows from intermediate and final outcomes back to non-resource inputs and the network environment (Figure 7) could thus represent induced changes 'internal' to participating members and volunteers, in contrast to the meso and macro effects associated with advocacy and campaigning already discussed above. While these changes are 'internal' to participants, many of these newly acquired skills or attributes will be of value in the wider 'external' contexts (a dynamic variant of the public goods argument). For example, an individual may emerge with a more 'civic' perspective that equips them to act more 'productively', not only in the context of this particular organization, but also outside it, particularly in the proximate network or community. (Coleman 1993, emphasizes that, unlike physical or human capital, social capital is neither tradable nor readily transferable to other contexts because it is defined by the very context in which it is nurtured.) 120 Public Management: an international Journal of research and theory Macro-level (societal context) Meso-level (network context) Costs or budget Resource inputs Non-resource inputs Intermediate outputs -> Final outcomes •R&edback effect from outputs ■tojion-resourjii inputs Figure 7: Feedback effects between levels Each of these dynamic elements — advocacy, innovation and participation might be seen as a performance criterion in its own right, as we do in the next section. But each can also be of value because, in the longer term (a timescale which theory suggests is likely to be relevant), it has the potential to improve efficiency or equity. DEVELOPING PERFORMANCE INDICATORS We now offer recommendations for performance indicators for voluntary activity which have their roots in the theories reviewed in section 3 ('Theoretical underpinnings and their implications'), and their interconnections via the framework set out in section 4 ('A framework for performance evaluation'). The practical problems that would arise when using such indicators are not discussed here. A programme to explore these implementation issues is currently underway in Northern Ireland. Moreover, we have previously undertaken and are currently engaged in empirical research which directly seeks to measure organizational performance along some of these dimensions (e.g. Knapp et al. 1999; Kendall 2000; Kendall and Knapp 2000). The purpose here is rather to demonstrate how particular indicators follow logically from the foregoing discussion, seeking to be as conceptually comprehensive as possible. The discussion is organized around eight performance domains, each with at least two indicator sets (Figure 8). The relative number of indicator sets in each domain does not reflect relative importance, but rather our ability to distinguish sufficiently distinct concepts which — from a theoretical standpoint at least — warrant measurement. The indicators might be employed to compare within the voluntary sector Kendall & Knapp: Measuring the performance of voluntary organizations 121 Economy Equity • Resource inputs • Redistributive policy consistency • Expenditures • Service targeting • Average costs • Benefit-burden ratios Effectiveness (service provision) • Final outcomes • Accessibility • Procedural equity • Recipient satisfaction Participation • Output volume • Membership/volunteers • Output quality • Attitudes Choice/pluralism Advocacy • Concentration • Advocacy resource inputs • Diversity • Advocacy intermediate outputs Efficiency Innovation • Intermediate output efficiency • Reported innovations • Final outcome efficiency • Barriers and opportunities Figure 8: Performance domains and indicator sets between organizations, localities or fields; or to assess changes over time (perhaps following some exogenous change, such as a funding cutback); or to make inter-sectoral comparisons. We do not, however, proceed to this level of specificity in this article. The performance domains (and indicators) are not necessarily of equal importance, nor indeed need they all be relevant to every voluntary organization or the context within which performance is being examined. Each has potential relevance on the theoretical and other grounds discussed earlier, but practical relevance may need to be determined or negotiated in specific cases and contexts. There are also inevitably tradeoffs to be made between domains: improvement in one may be achieved only at the expense of deterioration in another. Multiple criteria clearly complicate organizational comparisons, but reflect the reality of multiple objectives and multiple stakeholders. Economy The criterion of economy has generally limited relevance, for cost reduction without regard to its outcome effects is rarely helpful. Nevertheless, economy is an aim of some stakeholders, and it underpins performance measures in some other domains. The criterion is closely associated with cost measurement, of course, which either should be as comprehensive as possible (including the associated and knock-on effects of voluntary organization activities as well as internal expenditures), or should employ 122 Public Management: an international Journal of research and theory partial measures in full cognizance of their (potential) limitations. Cost-shifting has been worryingly prevalent in some areas of public policy, and can be encouraged (usually unintentionally) by narrowly applied performance criteria. All resource inputs to a production process and all other resource effects, proximate and distant, should be recognized, particularly in activities as inheritantly social and public as those pursued by many voluntary organizations. To assess economy, techniques are needed to record resource input levels and costs associated with a set of activities or group of recipients. Measures of input are straightforward to design, although aggregating them is difficult without unit cost or price weights. The economic cost of a resource is the value of the benefits forgone from not using the resource in its highest valued alternative use (the 'opportunity cost'). Because resources are finite, almost any activity will impose an opportunity cost even when apparently provided free of charge (for example, volunteers or 'free' office space). Most resources will have market values (prices) set by the forces of demand and supply, but market imperfections — including the externality, public good and monopoly power problems discussed — will distort them from the social opportunity costs of resources. Adjustments will therefore be needed. Three indicator sets are suggested to capture the resource/cost consequences of voluntary sector activity and to assess performance in the economy domain. The first would be resource inputs, focused perhaps on certain inputs, such as physical capital resources (e.g. how many vehicle miles, or how much community centre space). Most performance reviews are likely to want to look at sets of resource inputs, which then requires their aggregation, in turn requiring measures of expenditures or total (aggregate) costs. Imputed values will be needed for uncosted inputs, most obviously volunteer time (Foster 1997; Gaskin 1997). A third suggestion is to calculate the average cost for the typical user or beneficiary, or across all users. If unit costs need to be calculated for particular services, there are well-worked methodologies for the purpose (Beecham 199S) and compendia of existing figures (Netten et al. 1998). Effectiveness Earlier we distinguished two broad types of effectiveness: intermediate outputs (volumes of goods and services produced, their quality and perhaps some indication of 'casemix'); and final outcomes (actual impacts on service users or 'external' communities, depending on context). The latter would usually be preferred to the former, of course, but the complexities and costs of measuring final outcomes in the fields in which voluntary action is concentrated are legion. The principles of final outcome measurement can be briefly summarized here with an example (see Knapp 1984, 199S, for fuller account). Ideally, the final outcomes of Kendall & Knapp: Measuring the performance of voluntary organizations 123 (say) a child and family service — where voluntary organizations are often leading-edge or dominant providers — would be measured by comparing what happens with and without the service, or with two different types of service. Changes in child and family welfare could be assessed along a number of salient dimensions (social development, educational attainment, peer attachment or family stability) over some pre-specified period of time (which should usually be long-term) using well-validated, reliable scales. The views of different stakeholders — children and families themselves, service providers, the general public, and so on — might be sought (and perhaps examined separately in the light of multiple stakeholder theory's prediction of divergent perspectives prior to aggregation). Some dimensions might be more heavily weighted than others (either formally and quantitatively, or compared informally and qualitatively by decision makers), and there might be differential weighting to reflect the tendency for people to prefer imminent to distant benefits (the 'time preference' issue). Final outcome measures have been obtained in complex areas, including child and family support, but they need to be carefully designed and are often expensive to collect. They should usually be field-specific since the professions with relevant expertise tend to be so defined. Generic measures, such as quality of life indicators, are unlikely to be sensitive enough in specific fields, although they do have the advantage of currency across more than one field. If final outcomes are too difficult to obtain, can we rely on intermediate output measures? They are simpler to collect but harder to interpret. It is certainly valid to rely on service volume and quality measures as performance indicators, but it would be more valid and reassuring if they were accompanied by measures of their impact on the welfare of recipient individuals and communities (their links with final outcomes). Because this is rare, intermediate measures of effectiveness are often used only with some uncertainty about recipient-related consequences. These considerations lead us to recommend four indicator sets to measure effectiveness. Although normally field-specific, final outcome indicators could be sought to measure the relative (comparative) impact on the welfare and quality of life of recipient individuals or communities of voluntary organization activities. Notwithstanding the difficulties of interpretation and excessive generality, a second approach would be to seek ratings of satisfaction from users, recipients or beneficiaries. However, we must remember that the users of some voluntary sector services may not easily be able to express their satisfaction levels. Third, in most circumstances, and aggregating across recipients, it should be possible to decide whether more of a service is better than less of it. Output volume would then make a useful performance indicator. Finally, on the assumption that consensus can be reached as to the definition and measurement of output quality, indicators could be sought of this fourth effectiveness dimension. It is, however, important to be clear about the relationship between any quality measures and final outcomes. 124 Public Management: an international Journal of research and theory or Choice/pluralism Choice, diversity or pluralism arguments are variations on the theme of (intermediate output) effectiveness and the voluntary sector's (potential) ability to promote choice has been a recurrent theme in policy discussions. Some of the output volume and quality measures suggested above are relevant here too, but now attention is focused on variety across a field or market. Two additional measures are therefore suggested. Provided sensible boundaries can be put around an identified market, choice might be measured by (the inverse of) a market concentration index. A second indicator would measure diversity per se, for example, by variability in scores along some quality dimension. Efficiency Among the many interpretations and measures of efficiency, five can be distinguished Each could be discussed and measured with respect to either final outcomes intermediate outputs. Technical efficiency is the production of the maximum set of intermediate outputs or final outcomes from given inputs. Input mix efficiency or price efficiency refers to the achievement of maximum output or outcome from given expenditure: the mix of resource inputs is adjusted in cognizance of their prices and supply availabilities to achieve greater effectiveness. Output mix efficiency reflects the different values attached to an organization's various outputs and outcomes, and is achieved when the output/outcome mix is altered from a given budget or set of resources. Vertical target efficiency measures the extent to which target populations or objects are served and horizontal target efficiency is the extent to which those people in need (or other target groups) actually receive services. (We discuss measures of vertical and horizontal target efficiency in the section on equity below.) In comparative and experimental contexts, there are well-developed and widely employed tools of economic evaluation for assessing efficiency, notably cost-benefit analysis (CBA), cost-effectiveness analysis (CEA) and cost-utility analysis (CUA). These evaluative tools differ in that they measure outcomes differently, and consequently they address slightly different policy or practice questions. A CEA would rely on outcome measures of the kind discussed in the effectiveness section above, while a CBA converts these to monetary measures (for example using 'willingness-to-pay' techniques; Pearce et al. 1998) and a CUA employs a summary 'utility' measure of impact. Because the monetary valuation of outcomes is far from straightforward and because utility metrics have not been developed for many fields, cost-effectiveness analyses are the most likely to be employed in the foreseeable future. For example, Weisbrod (1996) notes that experience goods — which we have stressed are commonly what voluntary organizations deliver — pose difficulties for willingness-to-pay methods because of their hypothetical basis. Kendall & Knapp: Measuring the performance of voluntary organizations 125 In principle, efficiency can be assessed by combining any of the effectiveness measures suggested earlier, plus monetary or utility valuations of them if they can be constructed, with any of the economy measures. In practice this can be hard to achieve, but is usually informative. Without specifying which particular type of efficiency to address, we can suggest two broad indicator sets. The first would measure intermediate output efficiency, looking at intermediate outputs (such as service volume or even as crude a measure as number of clients served) produced from a particular budget. The other would assess final outcome efficiency, which should be more informative than looking at intermediate output efficiency for the reasons discussed, although as ever the practical difficulties of the more theoretically appropriate indicators are likely to be greater. Equity Equity is fairness or justice, and is usually discussed in relation to the enjoyment of effects or benefits (intermediate or final outcomes), and/or the burden of paying for them. Equity assessments thus pose questions such as: Do people with equivalent needs have access to, or use equivalent amounts of, certain goods or services? Do people with greater needs get more services? Do people with greater means (for example, in relation to income, wealth, innate abilities or opportunities) contribute disproportionately more to the costs of producing collective goods or services? Alternatively, is it right that better-off members of the community should have to pay more (through taxes, say) than worse-off members, even when the latter make greater use of services funded out of taxation? These grand questions can open up grand debates. Our aim here is much more modest and pragmatic. It should be possible to achieve consensus on some equity questions, but many will generate controversy. Also note that equity is often most usefully examined at a societal or community level and not (usually) in relation to individual service providers (such as voluntary organizations). Consequently, the practical task of performance assessment might check that activities are contributing to, or at least not undermining, broader government or societal strategies of redistribution or targeting. However, equity is a multi-faceted criterion, and not contributing in this way could be either good or bad depending on your ideology or position in society. Certainly it would be paradoxical to herald the voluntary sector as a champion of diversity and choice, while simultaneously criticizing it for being different from government. Many equity discussions concern whether or whose needs are met. Need is in fact itself already an equity concept: to define someone or some area as 'in need' is — implicitly or explicitly — to accord priority to them over other individuals or areas. If need is defined by government, then assessing whether an organization is 'meeting needs' is to assess its contribution to formal re distributive policy. (Voluntary 126 Public Management: an international Journal of research and theory organizations that campaign for changes to re distributive policies are themselves potentially part of the agenda-setting group.) If need is defined by the organization itself then it is legitimate to examine the (micro) re distributive aims of the organization or its main stakeholders. One equity measure could therefore be some indication of whether an organization's activities are consistent with government redistributive policies. The indicators of redistributive policy consistency would need to cover both benefits and burdens. Different tiers of government may have different redistributive policies, which complicates the construction and interpretation of such indicators. A second indicator set could directly assess an organization's ability to hit its own targets: do voluntary intermediate outputs reach their intended target beneficiaries (as stated in operational objectives or charitable instruments)? Whether or not these targets are consistent with state-level or other macro redistributive policies is no longer at issue. Vertical and horizontal target efficiency are two such measures of service targeting. Another equity measure — not necessarily distinct from previous indicators — could look at benefit—burden ratios: does one part of society contribute heavily to the resourcing of an organization or field (from taxes, user charges, donations or volunteering), and how are the resultant services or final outcomes distributed? Wolpert suggests measuring 'the income difference between donors and recipients, or the percentage of the lowest income group that has been assisted' (1996: 11), but there are numerous other options here. The accessibility of voluntary organization activities could also be examined. Do voluntary organizations restrict service access to a narrow range of people? For example, are high charges levied for admission to cultural activities which are supported heavily by taxpayers, or is access barred to certain religious groups, or is it age-specific? These barriers to access may be entirely legitimate and appropriate, but the point is to assess such equity of access. Indeed, given the challenges posed by final outcome measurement, examining access could assume greater importance in performance assessment. Finally, if crudely, it would be possible to seek proxy indicators for procedural equity which reflect processes or procedures, perhaps even configurations of resource and non-resource inputs. For example, does a voluntary organization have a democratic governance structure, does it charge lower fees to 'disadvantaged' users, or is it located in a deprived area? Participation Social capital arguments — encompassing participation, membership and volunteering — have attracted growing attention in discussions of the voluntary sector. Popular Kendall & Knapp: Measuring the performance of voluntary organizations 127 measures of participation are associational density in terms of per capita numbers of organizations (Knight 1993), active members (Putnam 1993; Hall 1997) or hours volunteered (Marshall 1997). The popularity of these measures stems from their simplicity. But three key assumptions are being made: more members or volunteers (headcounts) or more volunteering (hours) per capita imply more participation; more participation implies greater trust; and • • greater trust facilitates better social, economic, psychological and/or political outcomes. Other things being equal, by this logic voluntary organizations with relatively high membership or levels of volunteering, or communities with high density networks of member- or volunteering-involving organizations, are 'better performers'. There are obvious caveats to enter here. The quality of membership or volunteering varies, perhaps to the extent that it involves meaningful participation. Second, the relationship between membership, trust and 'final' outcome productivity is not known a priori, and needs to be verified. There is some evidence of correlations between membership, volunteering and propensities to trust, but these studies do not examine whether such trust leads to better outcomes (Onyx and Bullen 1997; Dekker and Van Den Broek 1998). Third, if made out of context, the argument is lopsided, neglecting the influence of what, in our terminology, are 'resource inputs' and 'macro environment' (Edwards and Foley 1997) . For example, the political benefits of participation arise not from the presence of 'civic skills' alone, but from their co-presence with financial resources and time (Verba et al. 1996). This all suggests the importance of examining this domain alongside data on the domains of economy, efficiency and effectiveness, themselves carefully tailored to reflect voluntary sector realities in terms of final outcomes and voluntary inputs. Counts of members anilor volunteers, and hours expended, preferably per capita, with a focus on geographical and/or virtual or interest communities would be a useful, if partial, starting point. Second, social capital development embraces a large number of potential changes in individuals' dispositions. Onyx and Bullen (1997) suggest a thirty-six-item scale of attitudes ani resultant actions to measure social capital. This could be applied to members or volunteers but the approach harbours potential biases. Participants may want, or feel pressured to exaggerate the beneficial effects of membership if they suspect that policy towards an organization is contingent on how they answer. An alternative would be to pose similar questions in absolute rather than relative terms to volunteers/members ani non-volunteers/members in the 'community' in question, and to test for systematic differences (after holding other potential influences constant). 128 Public Management: an international Journal of research and theory Advocacy Advocacy could be seen as an intermediate output valued in its own right, but is more likely to be appreciated because of its (perhaps very long-term) impact upon final outcome effectiveness, efficiency, equity or its generation of participative benefits. We noted earlier the enormous difficulty of directly measuring the final outcomes of advocacy, and so associated indicators would probably have to focus on inputs or intermediate outcomes. In relation to (a) citizen advocacy, and (b) public policy advocacy (campaigning or lobbying), one could assess the proportion of an organization's or field's budget allocated to advocacy, or the proportion of total paid staff, volunteer workforce and/ or membership base involved in the activity, or even the proportion of the workforce or membership base whose job descriptions or titles have advocacy as their primary purpose (e.g. NCVO 1990). These are all advocacy resource input measures. Other indicators might measure advocacy intermediate outputs, for example, by asking organizations about the frequency of various advocacy activities (NCVO 1990). Innovation Innovation is by definition a dynamic process involving significant discontinuities with previous practices. It could be instigated at various points in the 'production process'. Innovation is of relevance primarily to the extent that it influences intermediate outputs or final outcomes, or improves efficiency, equity or social capital. Yet the innovation in question may involve deliberate reconfigurations of activities, making 'like with like' comparisons very difficult. One approach to assessing innovative performance would simply be to ask whether organizations have innovated in each of the senses noted earlier. Definitions and concepts could build upon Osborne's (1998) research. There are obvious problems here, including the possibility of strategic behaviour. Respondents have an incentive to exaggerate such reported innovations if they think this will feed into an evaluation of their overall 'performance'. Seeking proof could quickly generate a large volume of case-material evidence. Another approach would be to canvass views on innovation from people 'external' to an organization, but again this would be costly and time-consuming An example of the latter would be to look for (say) efficiency-improving innovations (see Ferlie et al. 1989), but this takes us back to the problems of assessing efficiency, and anyway begins to look more like an efficiency indicator per se. Another approach would be to seek measures of innovative opportunities or barriers to innovation: what contextual factors encourage or discourage innovation? Again, Kendall & Knapp: Measuring the performance of voluntary organizations 129 however, the indicators could be highly subjective, open to bias and still quite difficult to interpret. CONCLUSION The voluntary sector's social, economic and political contributions, and the growing interdependence of the sector with the state, are drawing increasing attention to the need to understand what voluntary organizations achieve. It is not just accountability for public funds that is driving demands for better performance measurement, although this is certainly a crucial force. The sector's very existence and form, and thus also many of its activities, challenge the assessor to seek indicators that range beyond those conventionally employed in for-profit firms and public organizations. The primary purpose of this article has been to suggest indicators that not only begin to capture the potentially multiple contributions of voluntary organizations in a modern mixed economy, but in so doing reflect the inherent and defining characteristics of such organizations. These were shown to be clearly and closely linked to theoretical perspectives on the existence, roles, activities and achievements of voluntary organizations. Some of these indicators are identical or similar to measures currently in use; to begin to employ others would represent significant departures from current practice. Some might need regular monitoring; others could perhaps be left unchecked in a general climate of well-placed trust. Some might not be relevant to particular organizations or contexts, and their omission may need to be negotiated. Some performance domains might be pursued only at the expense of others, generating the need to make careful trade-offs between them. There will clearly be many such difficulties. Some other challenges will arise because voluntary organizations heavily populate human service fields where we have seen that outcome assessment is notoriously difficult. Another set of issues stems from the heavy emphasis placed by many voluntary organizations on the processes of service provision, particularly the emphasis on (meaningful) participation, pluralism, choice and innovation. These are inherently difficult objectives to pin down. Performance measures are demanded by a number of stakeholders, including voluntary organizations themselves, their funders, the public sector bodies charged with regulating them, and many individual employees, volunteers, donors and service users. Consequently, none of these challenges can be avoided if voluntary organization performance is to be examined in a valid and robust manner. There is no single criterion of performance upon which to rely, particularly in view of the multiple-stakeholder context within which most voluntary organizations find themselves, nor is there any simple or uncontroversial way to aggregate indicators across domains. What 130 Public Management: an international Journal of research and theory is crucial is to be explicit about these issues to ensure that the debate is as sensitive as possible to, and informed about, the character of voluntary action. REFERENCES Beecham, J. K. (1995) 'Collecting and Estimating Costs' in M. R. J. Knapp (ed.) The Economic Evaluation of Mental Health Care. Aldershot, Hants: Ashgate. Bellah, R. N., Madsen, R., Sullivan, W. M., Swidler, A. And Tipton, S. M. (1985) Habits of the Heart: Individualism and Commitment in American Life, New York: Harper & Row. Ben-Ner, A. And Van Hoomissen, J. (1993) 'Nonprofit Organizations in the Mixed Economy: A Demand and Supply Analysis' in A. Ben-Ner and B. Gui (eds) The Nonprofit Sector in the Mixed Economy. Michigan, MI: University of Michigan Press. Billis, D. (1993) Organising Public and Voluntary Agencies, London: Routledge. Billis, D. And Glennerster, H. (1997) 'Human Services and the Voluntary Sector: Towards a Theory of Comparative Advantage'. Journal of Social Policy, 27:1 pp79—98. Brennan, G. (1996) 'Selection and the Currency of Reward' in R. E. Goodin (ed.) The Theory of Institutional Design. Cambridge: Cambridge University Press. Coleman, J. (1993) 'A Rational Choice Perspective on Economic Sociology' in N. J. Smelser and R. Swedberg (eds) The Handbook of Economic Sociology. Princeton; NJ: Princeton University Press. Dasgupta, P. and Serageldin, I. (eds) (1999) Social Capital: A Multifaceted Perspective. Washington, DC: The World Bank. Davies, B. P. And Knapp, M. R. J. (1981) Old People's Homes and the Production ojWeljare, London: Routledge & Kegan Paul. Deci, E. And Ryan, R. (1985) Intrinsic Motivation in Human Behaviour, New York & London: Plenum Press. Dekker, P. And Van Den Broek, A. (1998) 'Civil Society in Comparative Perspective: Involvement in Voluntary Associations in North America and Europe'. Voluntas, 9:1 ppll—38. Deutsch, K. (1963) The Nerves of Government, New York: Free Press. Edwards, B. And Foley, M. W. (1997) 'Social Capital and the Political Economy of Our Discontent'. American Behavioral Scientist, 40:5 pp669—78. Evers, A. (1993) 'The Welfare Mix Approach: Understanding Welfare Pluralism Systems' in A. Evers and I. Svetlik (eds) Balancing Pluralism: New Welfare Mixes in Care for the Elderly. Aldershot, Hants: Avebury. --------(1995) 'Part of the Welfare Mix: The Third Sector as an Intermediate Area between Market Economy, State and Community'. Voluntas, 6:2 ppl59—82. Ferlie, E., Challis, D. J. And Davies, B. P. (1989) Efficiency-Improving Innovations in Social Care of the Elderly, Aldershot, Hants: Gower. Foley, M. W. And Edwards, B. (1997) 'Escape from Politics? Social Theory and the Social Capital Debate'. American Behavioral Scientist, 40:5 pp550—61. Foster, V. (1997) 'What Value Should Be Placed on Volunteering?' in C. Pharoah (ed.) Dimensions of the Voluntary Sector 1991. West Mailing: Charities Aid Foundation. Gaskin, K. (1997) 'Assessing the Economic Value and Cost of Volunteering' in C. Pharoah (ed.) Dimensions of the Voluntary Sector 1991. West Mailing: Charities Aid Foundation. Giddens, A. (1990) The Consequences of Modernity, Cambridge: Polity Press. Hall, P. A. (1997) 'Social capital in Britain'. Paper prepared for Bertelsmann Stiftung Workshop on Social Capital, Berlin, June. Hansmann, H. B. (1980) 'The Role of Nonprofit Enterprise'. Yale Law Journal, 89 pp835-98. Kendall & Knapp: Measuring the performance of voluntary organizations 131 -------- (1987) 'Economic Theories of Nonprofit Organisation' in W. W. Powell (ed.) The 'Non-profit Sector: A Research Handbook. New Haven, CT: Yale University Press. Hirschman, A. O. (1970) Exit, Voice and Loyalty, Cambridge: Harvard University Press. Hockman, H. And Rogers, J. (1969) 'Pareto-Optimal Redistribution'. American Economic Review, 59 pp542-57. Kanter, R. M. And Summers, D. V. (1987) 'Doing Well by Doing Good: Dilemmas of Performance Measurement in Nonprofit Organisations and the Need for a Multiple Constituency Approach' in W. W. Powell (ed.) The Nonprofit Sector: A Research Handbook. New Haven, CT: Yale University Press. Kendall, J. (2000) 'Services aux Personnes Agees en Angleterre: Vers un Explanation' in J. Defourny, J.-L. Laville and M. Nyssens (eds) Service, Lien Social et Emploi. Paris: Desclée de Brouwer. Kendall, J. And Knapp, M. R. J. (1995) 'A Loose and Baggy Monster: Boundaries, Definitions and Typologies' in J. Davis Smith and R. Hedley (eds) Introduction to the Voluntary Sector. London: Routledge. -------- (1999) 'Evaluation and the Voluntary (Nonprofit) Sector: Emerging Issues' in D. Lewis (ed.) International Perspectives in Voluntary Action: Reshaping the Third Sector. London: Earthscan Publications Ltd. -------- (2000) 'Voluntary Sector Providers of Care for Older People in Comparative Perspective' in M. Harris and C. Rochester (eds) Voluntary Organisations and Social Policy: Perspectives on Change and Choice. London: Macmillan. Knapp, M. R. J. (1984) The Economics of Social Care, London: Macmillan. -------- (1995) The Economic Evaluation of Mental Health Care, Aldershot, Hants: Arena. Knapp, M. R. J., Koutsogeorgopoulou, V. And Davis Smith, J. (1996) 'The Economics of Volunteering: Examining Participation Patterns and Levels in the UK'. Non-Profit Studies, 1 pp38-55. Knapp, M. R. J., Hallam, A., Beecham, J. And Baines, B. (1999) 'Private, Voluntary or Public? Comparative Co st-Effectiveness in Community Mental Health Care'. Policy and Politics, 27:1 pp25-41. Knight, B. (1993) Voluntary Action, GENTRIS: Ovingham and London. Krashinsky, M. (1986) 'Transaction Costs and a Theory of the Nonprofit Organisation' in S. Rose- Ackerman (ed.) The Economics of Nonprofit Institutions. New York: Oxford University Press. McAdam, D., McCarthy, J. D. And Zald, M. N. (1988) 'Social Movements' in N. J. Smelser (ed.) Handbook of Sociology. Newbury Park, CA: Sage Publications. Marshall, T. F. (1997) Comparing the Areas, LOVAS Paper 3, London: Home Office Research & Statistics Directorate. NCVO (1990) Cause and Effect: A Survey of Campaigning in the Voluntary Sector, London: National Council for Voluntary Organisations. Netten, A. P., Knight, J. And Dennett, J. H. (1998) Unit Costs of Health and Social Care 1998, University of Kent, Canterbury: Personal Social Services Research Unit. Newton, K. (1997) 'Social Capital and Democracy'. American Behavioral Scientist, 40:5 pp575—86. Onyx, J. and Bullen, P. (1997) 'Measuring social capital in five communities in NSW: An analysis'. Centre for Australian Community Organisations and Management (CACOM) Working Paper No. 41. Sydney: University of Technology Osborne, S. P. (1998) Voluntary Organisations and Innovation in Public Services, London: Routledge. Pearce, D., Foster, V, Mourato, S., Ozdemiroglu, E. And Dobson, S. (1998) 'What is the Economic Value of the Charitable Sector?' in C. Pharoah and M. Smerdon (eds) Dimensions of the Voluntary Sector 1998. West Mailing: Charities Aid Foundation. 132 Public Management: an international journal of research and theory Putnam, R. (1993) Making Democracy Work: Civic Traditions in Modern Italy, Princeton, NJ: Princeton University Press. Salamon, L. M. (1993) 'The nonprofit sector and democracy: Prerequisite, impediment or irrelevance?'. Unpublished paper prepared for Democracy and the Nonprofit Sector Symposium, Wye, Maryland, 14 December. Salamon, L. M. And Anheier, H. K. (1997) Defining the Nonprofit Sector: A Cross-National Analysis, Manchester: Manchester University Press. Salamon, L. M., Anheier, H. K. And Associates (1998) The Emerging Sector Revisited: A Summary, Baltimore, MD: Center for Civil Society Studies, Johns Hopkins University. Salamon, L. M., Anheier, H. K., List, R.,Toepler, S., Sokolowski, S. J. And Associates (1999) Global Civil Society: Dimensions of the Nonprofit Sector. Baltimore, MD: Centre for Civil Society Studies, Johns Hopkins University. Schroeder, D. A., Penner, L. A., Dovidio, J. F. And Piliavin, J. A. (1995) The Psychology of Helping and Altruism: Problems and Puzzles, New York & London: McGraw-Hill. Shore, P., Knapp, M. R. J., Kendall, J. And Carter, S. (1994) 'The Voluntary Sector in Liverpool' in S. Saxon-Harrold and J. Kendall (eds) Researching the Voluntary Sector, 2. London: Charities Aid Foundation. Verba, S., Sclozman, K. L., And Brady, H. E. (1996) Voice and Equality: Civic Voluntarism in American Politics, Cambridge, MA: Harvard University Press. Weisbrod, B. A. (1975) 'Towards a Theory of the Nonprofit Sector' in E. Phelps (ed.) Altruism, Morality and Economic Theory, New York: Russell Sage. -------- (1988) The Nonprofit Economy, Cambridge: Harvard University Press. -------- (1996) 'Evaluating the nonprofit sector: Need, obstacles, and approaches' in Measuring the Impact of the Independent, Not-for-Profit Sector in Society, Independent Sector, Working Papers from Carnegie Conference Center, Washington, DC, 5—6 September. Wolch, J. R. (1990) The Shadow State: Government and Voluntary Sector in Transition, New York: The Foundation Center. Wolpert, J. (1996) 'The distributional impacts of nonprofits and philanthropy' in Measuring the Impact of the Independent, Not-for-Profit Sector in Society, Independent Sector, Working Papers from Carnegie Conference Center, Washington, DC, 5—6 September. 6, P. (1993) 'Innovation by Nonprofit Organisations: Policy and Research Issues'. Nonprofit Management and Leadership, 3:4 pp397—413. 6, P. and Forder, J. (1996) 'Can Campaigning Be Evaluated?'. Nonprofit and Voluntary Sector Quarterly, 25:2 pp225-4-7.