ResearchGate
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228799426
State-Level Effects of Transitional Justice: What Do We Know?
Article in International JournalofTransitional Justice- November2010
301:10.1093/ijtj/ijq012
CITATIONS READS
125 540
3 authors, including:
^^^k   James Ron
^3jF   Independent Consultant
44 PUBLICATIONS 2,024 CITATIONS SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Project    Will Global South Publics Donate $$ to HROs? View project
All content following this page was uploaded by James Ron on 07 May 2014.
The user has requested enhancement of the downloaded file.
The International Journal of Transitional Justice, 2010,1-26, doi: 10.1093/ijtj/ijqOI 2
State-Level Effects of Transitional Justice: What Do We Know?
Oskar N.T. Thorns,* James Ron^ and Roland Paris'
Abstract1
At the core of policy debates on the state-level effects of transitional justice is a series of competing claims about the causal effects of various transitional justice mechanisms. A review of recent scholarship on transitional justice shows that empirical evidence of positive or negative effects is still insufficient to support strong claims. More systematic and comparative analysis of the transitional justice record is needed in order to move from 'faith-based' to 'fact-based' discussions of transitional justice impacts.
Introduction
The last two decades have witnessed a remarkable proliferation of transitional justice (TJ) processes, including trials and truth commissions in postconflict and postauthoritarian societies. Such processes have become central ingredients in the 'menu' of policy options recommended by international organizations and outside experts for societies in transition from war or authoritarianism.
At first glance, the case for TJ seems incontrovertible. Principles of fundamental justice require holding individuals accountable for the worst transgressions of universal human rights, including genocide, war crimes and crimes against humanity. Proponents also assert that TJ offers other benefits, including promoting reconciliation and psychological healing, fostering respect for human rights and the rule of law and helping establish conditions for a peaceful and democratically governed country.2
* PhD Student, Department of Politics, Princeton University, USA. Email: othoms@princeton.edu t Associate Professor, Norman Paterson School of International Affairs, Carleton University, Canada.
Email: jron@connect.carleton.ca ** Associate Professor, Graduate School of Public and International Affairs, and Director, Centre for
International Policy Studies, University of Ottawa, Canada. Email: rparis@uottawa.ca
1 This article is based on an earlier, more detailed report: Oskar N.T. Thorns, James Ron and Roland Paris, The Effects of Transitional Justice Mechanisms: A Summary of Empirical Research Findings and Implications for Analysts and Practitioners (Ottawa: Centre for International Policy Studies, University of Ottawa, 2008). The authors wish to thank Eric Brahm, Robert Brym, Suzanne Buckley-Zistel, Colleen Duggan, Don Hubert, David Mendeloff, Errol Patrick Mendes, Valerie Percival, Andrew Reiter, Harvey Weinstein and Christoph Zürcher, along with the editors of this journal and three anonymous reviewers, for commenting on earlier drafts. All errors of fact or interpretation remain our own. The earlier report was prepared with financial support from the Canadian Department of Foreign Affairs and International Trade's Global Peace and Security Fund.
2 See, for example, Jaime Malamud-Goti, 'Transitional Governments in the Breach: Why Punish State Criminals?' Human Rights Quarterly 12(1) (1990): 1-16; Richard J. Goldstone, 'Justice as a Tool for Peacemaking: Truth Commissions and International Criminal Tribunals,' New York
© The Author (2010). Published by Oxford University Press. All rights reserved. For Permissions, please email journals.permissions@oxfordjournals.org.
2   O.N.T. Thorns, J. Ron and R. Paris
Sceptics have challenged these claims, arguing that 'digging up the past' and identifying perpetrators can trigger renewed conflict by sharpening societal divisions or provoking backlash.3 Some also argue that the prospect of TJ may reduce the chances of negotiating peace settlements in the first instance, particularly in cases where powerful actors capable of blocking such settlements fear punishment for past actions.4
At the core of this debate is a series of empirical claims and counterclaims about the causal effects of TJ. But how much do we actually know about these effects? Does TJ strengthen or threaten societal peace in transitional countries? Does it lead to greater or less respect for human rights and the rule of law? Does it foster reconciliation or exacerbate divisions? Answers to these questions based on evidence and careful analysis are essential to building knowledge about the actual effects of TJ mechanisms. They are also crucial for policy makers in international organizations, donor governments and transitional countries.
In this article, we review the empirical literature on TJ's effects on institutions and policy processes at the state level, rather than on the substate, community or individual levels.5 In choosing to focus our discussion in this way, we do not mean to diminish the importance of TJ processes at the substate level, or to suggest that local- or individual-level studies are unimportant. Findings at these and other levels may be complementary to, or at odds with, those at the state level,6 and we welcome any similar reviews of microlevel findings.
Our review is based on an extensive survey of the empirical research on the effects of two prominent TJ mechanisms: trials and truth commissions. We chose these because they have been widely applied in transitional countries, they are at the centre of TJ policy debates and they have received the most attention and analysis in the relevant social science literature. For reasons of space, we do not review other mechanisms, including amnesties.7
University Journal of International Law and Politics 28(3) (1996): 485-503; Juan E. Méndez, 'In Defense of Transitional Justice,' in Transitional Justice and the Rule of Law in New Democracies, ed. A. James McAdams (Notre Dame, IN: University of Notre Dame Press, 1997); Payam Akhavan, 'Beyond Impunity: Can International Criminal Justice Prevent Future Atrocities?' American Journal of'International Law 95(1) (2001): 7-31.
3 Jack Snyder and Teslie Vinjamuri, 'Trials and Errors: Principle and Pragmatism in Strategies of International Justice,' International Security 28(3) (2003/04): 5-44; David Mendeloff, 'Truth-Seeking, Truth-Telling, and Postconflict Peacebuilding: Curb the Enthusiasm?' International Studies Review 6(3) (2004): 355-380.
4 Nick Grono and Adam O'Brien, 'Justice in Conflict? The ICC and Peace Processes,' in Courting Conflict? Justice, Peace and the ICC in Africa, ed. Nicholas Waddell and Phil Clark (London: Royal African Society, 2008).
5 We also do not consider the global effects of TJ, including the development of international norms or the global incidence of conflict. While some TJ processes may improve global and regional norms and accountability, they may also create distinct harms or be irrelevant in specific countries. Some countries, moreover, may never benefit from TJ, even though the world, on average, is better off.
6 David Backer, 'Cross-National Comparative Analysis,' in Assessing the Impact of Transitional Justice: Challenges for Empirical Research, ed. Hugo van der Merwe, Victoria Baxter and Audrey R. Chapman (Washington, DC: US Institute of Peace Press, 2009).
7 Other mechanisms not considered here include reparations (both material and symbolic), traditional and grassroots conflict resolution, memorialization and historical education and other transitional legal and institutional reforms, including vetting and lustration. On TJ mechanisms,
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 3
We scoured the social science literature for studies investigating the effects of trials and truth commissions on: (1) state respect for personal integrity rights, including the right to life and inviolability of the human person; (2) levels of political violence; (3) adherence to the rule of law; (4) democratization; (5) popular perceptions of regime legitimacy; and (6) a political culture of human rights and diversity.8 The literature on the state-level effects of TJ is still young and relatively small, and we found more relevant studies on some of these outcomes than on others.
We reviewed published books, articles, reports and several as-yet unpublished studies and scrutinized the methods, data, analysis and conclusions of these works. We limited our search to studies that explicitly sought to evaluate outcomes of specific mechanisms in a comparative fashion across space or time (including comparisons within a given country). We did not examine 'grey' literature, including practitioner accounts or programme evaluation studies, but instead focused on scholarly works that attempt to evaluate TJ outcomes through comparative analysis. Some of these works use qualitative methodologies, others use quantitative techniques, while still others use both. Of the more than 100 TJ-related studies consulted, we found 15 comparative works with a specific focus on state-level TJ effects of trials and commissions. Of these, seven were fully or partly qualitative.
Our principal finding is that reliable empirical knowledge on the state-level impacts of TJ is still limited. Systematic research is nascent. Many early findings are questionable or contradictory. Although most studies maintain that TJ processes have helped specific countries, others suggest TJ has been ineffectual or even harmful. Given the dearth of reliable evidence, strong claims about TJ, which continue to pepper academic and policy writing, appear to be based more on faith than on fact. The literature does not provide policy makers with sound empirical foundations for making informed decisions about when, where and
see, for instance, Neil J. Kritz, 'Coming to Terms with Atrocities: A Review of Accountability Mechanisms for Mass Violations of Human Rights,' Law and Contemporary Problems 59(4) (1996): 127-152; Martha Minow, Between Vengeance and Forgiveness: Facing History after Genocide and Mass Violence (Boston, MA: Beacon Press, 1998); Ruti G. Teitel, Transitional Justice (New York: Oxford University Press, 2000). Blanket or partial amnesties are viable policy responses, but many TJ advocates view them as denials of justice that encourage further impunity In recent years, however, some TJ scholars have begun to view amnesties more charitably, especially when used in a circumscribed, conditional and democratically supported manner. See, Louise Mallinder, 'Can Amnesties and International Justice Be Reconciled?' International Journal of Transitional Justice 1 (2) (2007): 208-230. For the increasing acceptability of amnesties, see also, Naomi Roht-Arriaza, 'The New Landscape of Transitional Justice,' in Transitional Justice in the Twenty-First Century: Beyond Truth Versus Justice, ed. Naomi Roht-Arriaza and Javier Mariezcurrena (Cambridge: Cambridge University Press, 2006); Mark Freeman, Necessary Evils: Amnesties and the Search for Justice (New York: Cambridge University Press, 2009). 8 These outcomes are commonly seen as areas in which TJ can help. This list is not exhaustive, however; for example, we did not evaluate TJ effects on reconciliation of various kinds because aspects of the notion are captured by some of our outcomes. For discussion on this topic, see, Laurel E. Fletcher and Harvey M. Weinstein, 'Violence and Social Repair: Rethinking the Contribution of Justice to Reconciliation,' Human Rights Quarterly 24(3) (2002): 573-639; Mendeloff, supra n 3; Siri Gloppen, 'Roads to Reconciliation: A Conceptual Framework,' in Roads to Reconciliation, ed. Elin Skaar, Siri Gloppen and Astri Suhrke (Lanham, MD: Lexington Books, 2005); Tristan Anne Borer, 'Truth Telling as a PeaceBuilding Activity: A Theoretical Overview,' in Telling the Truths: Truth Telling and Peace Building in Post-Conflict Societies, ed. Tristan Anne Borer (Notre Dame, IN: University of Notre Dame Press, 2006).
International Journal of Transitional Justice, 2010,1-26
4   O.N.T. Thorns, J. Ron and R. Paris
how to promote TJ. Moving towards more fact-based discussions will require careful, sustained and cross-national analyses of the TJ record.
This conclusion should not be cause for dismay. Like early contributions to other nascent research programmes, the existing empirical TJ literature has helped elucidate the analytical challenges to be tackled. Importantly, an emphasis is emerging on more systematic qualitative and quantitative analysis, including new efforts at cross-national data collection.9 Given the complexity of TJ processes, foolproof policy formulas cannot and never will be developed; still, the prevailing ambiguity surrounding TJ impacts can be significantly reduced through further and more careful research.
This Review's Scope
The contemporary TJ agenda emerged from debates over how best to deal with repressive legacies in Latin America, but now concerns itself more generally with democratic and postconflict transitions worldwide. TJ advocates argue for the moral, legal and practical importance of 'practices, mechanisms and concerns that arise following a period of conflict, civil strife, or repression, and that are aimed directly at confronting past violations of human rights and humanitarian law.'10 In one form or another, TJ is now included in most peace processes,11 and 'has come to dominate debates on the intersection between democratization, human rights protections, and state reconstruction after conflict.'12 Truth commissions, for example, have emerged as 'a staple of postconflict peacebuilding efforts.'13 According to some observers, this 'unprecedented spike' since the mid-1980s14 in efforts to address past human rights abuses is tantamount to a 'revolution in accountability' or a global 'justice cascade,' which has included widespread use of the two TJ mechanisms examined in this article.15
9 See, David Backer, 'The Human Face of Justice: Victims' Responses to South Africa's Truth and Reconciliation Commission Process' (PhD diss., University of Michigan, 2004); Helga Malmin Binningsbo, Jon Elster and Scott Gates, 'Civil War and Transitional Justice, 1946-2003: A Dataset' (paper presented at 'Transitional Justice and Civil War Settlements' workshop, Bogota, Colombia, 18-19 October 2005); Kathryn Sikkink and Carrie Booth Walling, 'The Impact of Human Rights Trials in Latin America,' Journal of Peace Research 44(4) (2007): 427-445; Geoff Dan cy, Hunjoon Kim and Eric Wiebelhaus-Brahm, 'The Turn to Truth: Trends in Truth Commission Experimentation,' Journal of Human Rights 9(1) (2010): 45-64; Tricia D. Olsen, Leigh A. Payne and Andrew G. Reiter, Transitional Justice in Balance: Comparing Processes, Weighing Efficacy (Washington, DC: US Institute of Peace Press, 2010); Hunjoon Kim and Kathryn Sikkink, 'Explaining the Deterrence Effect of Human Rights Prosecutions for Transitional Countries,' International Studies Quarterly (forthcoming).
10 Roht-Arriaza, supra n 7 at 2.
11 Robert Bates et al., Consolidating Peace and Mitigating Conflict in the Aftermath of Violence (Washington, DC: American Political Science Association Task Force on Political Violence and Terrorism, 2007).
12 Kieran McEvoy, 'Beyond Legalism: Towards a Thicker Understanding of Transitional Justice,' Journal of Law and Society 34(4) (2007): 412.
13 Eric Brahm, 'Uncovering the Truth: Examining Truth Commission Success and Impact,' International Studies Perspectives 8(1) (2007): 16.
14 Sikkink and Walling, supra n 9 at 430.
15 See, Ellen Lutz and Kathryn Sikkink, 'The Justice Cascade: The Evolution and Impact of Foreign Human Rights Trials in Latin America,' Chicago Journal of International Law 2(1) (2001): 1-34;
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 5
Trials
Trials have multiple goals, including truth, deterrence, punishment, reconciliation and promotion of the rule of law.16 Trial advocates believe criminal punishment serves the needs of victims, reinforces social norms, removes political threats to the new regime and deters future abusers.17 Special deterrence is achieved through the outright removal of perpetrators and implicated leaders, while generalized deterrence is effected by spurring changes in the cost-benefit calculations of potential perpetrators and by tacitly instilling inhibitions against abuse. Advocates also believe that credible threats of punishment boost political stability and encourage constructive political behaviour.18
Advocates say trials respond to victims' needs and provide psychologically therapeutic effects, offering victims a sense of justice and catharsis, as well as 'a sense that their grievances have been addressed and can hopefully be put to rest, rather than smouldering in anticipation of the next round of conflict.'19 They also hope that trials contribute to reconciliation by establishing criminal accountability - which stigmatizes and marginalizes the elites that perpetuate conflict - and that they cool desires for vengeance.20 Accountability, moreover, helps separate individual and collective guilt, arguably blocking cycles of resentment and violence.21 Finally, TJ advocates argue that trials promote the rule of law by obliging governments to conduct themselves according to publicly known and broadly applicable rules, and reinforce democratic transitions by publicly heralding the end of abuses, distinguishing between old and new orders and generating incentive to rebuild the judiciary.22
Critics of trials reject or question many of these claims. The main problem with prosecutions, critics say, is that they pay insufficient attention to political realities. The prospect of trials may present an obstacle to ending conflicts through negotiation.23 Containment of spoilers should be the priority, and unless international actors are ready to commit major on-the-ground resources, including armed forces, it is more useful to 'offer the potential spoilers a deal that will leave them weak but secure.'24 For trials to deter abuse, critics say, the domestic justice
Chandra Lekha Sriram, Globalising Justice for Mass Atrocities: A Revolution in Accountability (New York: Routledge, 2005); Olsen, Payne and Reiter, supra n 9.
16 See, for example, Jane E. Stromseth, 'Introduction: Goals and Challenges in the Pursuit of Accountability,' in Accountability for Atrocities: National and International Responses, ed. Jane E. Stromseth (Ardsley, NY: Transnational Publishers, 2003).
17 Kritz, supra n 7; Neil J. Kritz, 'The Rule of Law in the Postconflict Phase: Building a Stable Peace,' in Turbulent Peace: The Challenges of Managing International Conflict, ed. Chester A. Crocker, Fen Osier Hampson and Pamela Aall (Washington, DC: US Institute of Peace Press, 2001).
18 Akhavan, supra n 2.
19 Kritz, supra n 7 at 128.
20 Minow, supra n 7; Akhavan, supra n 2.
21 Mendez, supra n 2; Kritz, supra n 7.
22 Kritz, supra n 17; Malamud-Goti, supra n 2; Minow, supra n 7. See also, Fletcher and Weinstein, supra n 8; Mark A. Drumbl, Atrocity, Punishment, and International Law (New York: Cambridge University Press, 2007).
23 Grono and O'Brien, supra n 4.
24 Snyder and Vinjamuri, supra n 3 at 13.
International Journal of Transitional Justice, 2010,1-26
6   O.N.T. Thorns, J. Ron and R. Paris
infrastructure must be well established, but since such conditions are rare in post-conflict situations, bargains and amnesties are better ways to secure peace.25
Critics also take issue with the notion that future perpetrators will refrain from abuses, if the threat of prosecution is credible and high, noting that this assumption is problematic when perpetrators think they are acting for their group's greater good, particularly when they believe that their group's survival is at stake.26 In other cases, perpetrators seek to join violent groups for reasons of personal gain or survival and are likely more motivated by immediate peer approval than by the threat of international sanction.27 Moreover, the trial proponents' optimism about the potential for international deterrence does not take into account a broadly sceptical criminology literature.28
Truth Commissions
Truth commissions involve official investigations into past abuses. According to one prominent analyst:
(1) Truth commissions focus on the past; (2) they investigate a pattern of abuses over a period of time, rather than a specific event; (3) a truth commission is a temporary body, typically in operation for six months to two years, and completing its work with the submission of a report; and (4) these commissions are officially sanctioned, authorized or empowered by the state (and sometimes also by the armed oppositions, as in a peace accord).29
Truth commissions are useful because they make it difficult to deny gross violations of human rights and signal official determination to avoid the recurrence of violations.30
Although both truth commissions and trials seek to establish truth, commissions supply narrative, rather than forensic, accounts of the past.31 Trial truths have special credibility due to stricter rules governing the admissibility of evidence, the defendant's ability to cross-examine and the need to overcome presumption of innocence.32 Unlike trials, moreover, commissions can investigate broader contexts of abuses, including the institutional and structural factors that made them possible. Truth commission findings can also support other TJ measures, generating the background for prosecutions, establishing a basis for reparations and offering recommendations for institutional reform.33
25 Ibid.
26 David Wippman, 'Atrocities, Deterrence, and the Limits of International Justice,' Fordham International Law Journal 23(2) (1999): 473-488.
27 Drumbl, supra n 22. See also, Malamud-Goti, supra n 2.
28 McEvoy, supra n 12.
29 Priscilla B. Hayner, Unspeakable Truths: Facing the Challenges of Truth Commissions (New York: Routledge, 2001), 14. See also, Teitel, supra n 7.
30 Goldstone, supra n 2.
31 Borer, supra n 8.
32 Mendez, supra n 2.
33 See, Kritz, supra n 7; Mendez, supra n 2; Minow, supra n 7; Hayner, supra n 29. See also, Brahm, supra n 13.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 7
Proponents of truth commissions argue that they provide a forum for victims (or their relatives) to tell their stories and that this may help with individual psychological healing34 and social reconciliation, 'breaking the cycle of revenge and hatred between former enemies.'35 Some also say that truth commissions may advance democracy and promote the rule of law, even though they do not explicitly subscribe to judicial procedures, by encouraging the reform of judicial institutions and helping to establish accountability, build a human rights culture and restore social trust.36
It has been taken largely as a given that truth telling fosters reconciliation,37 but critics of truth commissions fear they may be dangerous because a commission's attempts at establishing a true record of past abuses may generate resentment among victims and perpetrators alike. The establishment of painful 'truths' in divided societies can provoke further tensions, inflaming volatile situations and providing new grievances to be exploited by cynical elites.38 Truth commissions may also provide perpetrators with 'public relations smoke screens' to deflect attention from continuing abuses.39
TJ Impact Studies
Surprisingly, explicit studies of TJ impact at the state level are still rare. Most TJ scholarship seems to have assumed rather than 'problematized' the effects of accountability processes. More precisely, scholars tend to assert the need for TJ and list the purported strengths and weaknesses of different TJ mechanisms while paying less attention to careful impact analysis.40 Among the minority of studies that do investigate TJ impacts, many are single-case studies of individual countries rather than structured cross-case comparisons, making general policy-relevant conclusions difficult to draw.41 Single-country studies that examine one instance of TJ in one context cannot support strong general assertions about cause and effect. Instead, they are better suited to generating hypotheses or to exploring case-specific details.
There also appears to be a 'selection bias' in much of the TJ literature. Early single-case studies have tended to focus on a few well-documented TJ cases, such as
34 Minow, supra n 7.
35 Hayner, supra n 29 at 154. See also, Andrew Rigby, Justice and Reconciliation: After the Violence (Boulder, CO: Lynne Rienner, 2001).
36 Hayner, supra n 29; Pablo de Greiff, 'Truth Telling and the Rule of Law,' in Telling the Truths: Truth Telling and Peace Building in Post-Conflict Societies, ed. Tristan Anne Borer (Notre Dame, IN: University of Notre Dame Press, 2006). For a review of such arguments, see, Brahm, supra n 13.
37 See Mendeloff, supra n 3; Borer, supra n 8.
38 Snyder and Vinjamuri, supra n 3; Mendeloff, supra n 3. See also, Gloppen, supra n 8; Brahm, supra n 13.
39 Snyder and Vinjamuri, supra n 3 at 20. See also, Eric Wiebelhaus-Brahm, Truth Commissions and Transitional Societies: The Impact on Human Rights and Democracy (New York: Routledge, 2010).
40 Fletcher and Weinstein, supra n 8; Mendeloff, supra n 3; Leslie Vinjamuri and Jack Snyder, Advocacy and Scholarship in the Study of International War Crimes Tribunals and Transitional Justice,' Annual Review of Political Science 7 (2004): 345-362; Borer, supra n 8; Brahm, supra n 13; Backer, supra n6.
41 Backer, supra n 6.
International Journal of Transitional Justice, 2010,1-26
8   O.N.T. Thorns, J. Ron and R. Paris
Argentina, Chile and South Africa. The existing knowledge base, in other words, seems to be built on a biased sample.42 Further, many cross-national comparisons have focused on Latin America. Although comparisons across countries within a single region may help to control for other factors influencing TJ outcomes - including culture, political legacy and socioeconomic development - the applicability of these findings to other regions may be limited. For example, Latin American countries have higher average levels of democracy and wealth than sub-Saharan African countries, where TJ has also been pursued. As a result, cross-national studies of Latin American countries may tell us more about the region's overall conditions than about TJ itself. Moreover, most TJ in Latin America took place in countries undergoing transitions from authoritarianism to democracy. As a result, studies of these countries may tell us little about TJ in societies transitioning away from, or still mired in, internal armed conflict, such as Uganda, Sierra Leone and Afghanistan. The latter are likely to face quite different challenges due to recent experiences of war43 and may not even have embarked on real democratic transitions.
Finally, the existing literature tends to analyse short-term TJ outcomes, often using overly narrow criteria for success. Most truth commission impact studies, for example, define success as the degree to which a commission completed its objectives within a designated period.44 These findings may reveal a great deal about the managerial talents of truth commission administrators, but they tell us little about the deeper social, political or institutional changes that such processes seek to achieve.45
For these reasons - and in spite of the voluminous writing on TJ in recent years - there is a limited body of literature devoted to the systematic and rigorous investigation of TJ outcomes. This sets the stage for the literature review below and anticipates some of our critiques of existing TJ scholarship. In what follows, we assess the state of knowledge on the effects of trials and truth commissions, organizing our review by the number of cases examined in each study. We thus move from studies analysing individual cases to those studying fewer than 20 cases, and end with studies with larger samples. Although social scientists debate the relative merits of smaller- and larger-sample studies,46 we take no position in this methodological debate other than to note that single-case studies offer little
42 Brahm, supra n 13; Backer, supra n 6. See also, Ellen Lutz, 'Transitional Justice: Lessons Learned and the Road Ahead,' in Transitional Justice in the Twenty-First Century: Beyond Truth Versus Justice, ed. Naomi Roht-Arriaza and Javier Mariezcurrena (Cambridge: Cambridge University Press, 2006).
43 Juan E. Mendez, 'Accountability for Past Abuses,' Human Rights Quarterly 19(2) (1997): 255-282; Olsen, Payne and Reiter, supra n 9; Roland Paris, At War's End: Building Peace after Civil Conflict (New York: Cambridge University Press, 2004).
44 Margaret L. Popkin and Naomi Roht-Arriaza, 'Truth as Justice: Investigatory Commissions in Latin America,' Law and Social Inquiry 20(1) (1995): 79-116; Audrey R. Chapman and Patrick Ball, 'The Truth of Truth Commissions: Comparative Lessons from Haiti, South Africa, and Guatemala,' Human Rights Quarterly 23(1) (2001): 1-43.
45 Brahm, supra n 13.
46 Small-sample cases score higher on validity, while larger-sample cases do better on reliability For discussion on this topic, see, James Mahoney and Gary Goertz, A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research,' Political Analysis 14(3) (2006): 227-249. See also,
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 9
in terms of general conclusions. Further, we do not assume that either qualitative or quantitative approaches are superior; each has advantages and disadvantages, depending on the matter under study. Our focus, rather, is on the logical coherence, appropriateness of method, quality of data and analysis and soundness of the conclusions within each study.
Trials
Single-Case Studies
Payam Akhavan offers a prime example of a qualitative, single-case study that has been widely cited but whose conclusions are questionable. He argues that efforts by the International Criminal Tribunal for the former Yugoslavia (ICTY) had a positive impact on peacebuilding in the Balkans, marginalizing Serb ultranationalism and encouraging the emergence of moderate leaders.47 Further, he finds that ICTY indictments had a moderating effect on ethnic politics in Croatia and set useful cooperation benchmarks for Serbian and Croatian integration into the European Union (EU). The study's chief strength is its process tracing of tribunal impacts through analysis of political reactions to major ICTY actions. Its major weaknesses are that the evidence is anecdotal and that the author does not sufficiently consider other factors or explanations.
James Meernik conducted another study of the same topic but arrived at very different findings. Although Meernik focuses on a single country - Bosnia - his sample is far larger than Akhavan's because he examines monthly time-series data from January 1996 to July 2003.48 Specifically, he tests the effects of international arrests of and verdicts on high-ranking political or military figures from each ethnic group on levels of interethnic conflict and cooperation, measured by events reported in the local press. After controlling for other variables, Meernik finds little impact on societal peace at one- and six-month intervals following relevant judicial actions. Instead, prior levels of conflict and cooperation, along with EU actions, were statistically significant and had stronger effects. He finds that the same was true, to some extent, for North Atlantic Treaty Organization and US actions.
Meernik's is a good example of a rigorous single-country study using statistical analysis. It does have shortcomings, however, as the author himself acknowledges, including problems of measurement and establishing causality. Meernik notes that the news-based indicator of conflict and cooperation focuses largely on the actions of prominent people and may undercount routine and peaceful interactions among
Henry E. Brady and David Collier, eds., Rethinking Social Inquiry: Diverse Tools, Shared Standards (Lanham, MD: Rowman and Littlefield, 2004).
47 Akhavan, supra n 2. He also analyses the impact of the International Criminal Tribunal for Rwanda in a separate case study with moderately positive but weak results. We do not review this part of the study here as it relies on a counterfactual that is hard to evaluate.
48 James Meernik, 'Justice and Peace? How the International Criminal Tribunal Affects Societal Peace in Bosnia,' Journal of Peace Research 42(3) (2005): 271-289.
International Journal of Transitional Justice, 2010,1-26
10   O.N.T. Thorns, J. Ron and R. Paris
ordinary Bosnians. These quotidian activities are, arguably, important indicators of intercommunal tension and social response to ICTY arrests and verdicts.
Small- to Medium-Sample Comparative Studies
Jane Stromseth, David Wippman and Rosa Brooks' study examines the impact of international tribunals and mixed or hybrid trials on the rule of law and the capacity of domestic justice systems in the former Yugoslavia, Rwanda, Timor Leste and Sierra Leone.49 It finds mixed, complex and unclear effects. Although international trials help marginalize perpetrators, they are less successful in local judicial capacity building and in demonstrating the importance of legal accountability to domestic audiences. Both the ICTY and the International Criminal Tribunal for Rwanda have done a poor job of boosting local support for accountability and justice, and Kosovo's mixed trials have garnered little local legitimacy. Although the study illuminates the challenges of international involvement in TJ, its heterogeneous sample and lack of controlled comparison complicate efforts to evaluate causal claims.
Another recent medium-sample study is Kathryn Sikkink and Carrie Booth Walling's analysis of the impact of human rights trials on democratic stability, human rights and conflict in 17 Latin American countries, from 1979 to 2004.50 Contrary to fears voiced by TJ sceptics, this study does not find that trials have harmful effects. In fact, the authors argue, arguments for trials' destabilizing impact are based exclusively on early evidence from Argentina, a country that has since had more TJ trials than any other, while also enjoying the longest democratic period in its history. Sikkink and Walling also discern no evidence that trials extend or exacerbate conflict. Although most of the 17 countries experienced one or more conflicts, TJ trials tended to follow, rather than precede, the fighting. Finally, the authors find little evidence that trials lead to more human rights violations; instead, trials are associated with equal or better human rights conditions.
The Sikking and Walling study uses unique and innovative data, and its regional focus on Latin America creates a useful homogenous sample. The authors' statistical methods are rudimentary, however, employing simple descriptive statistics with no controls. As a result, it is hard to know whether positive effects are in fact due to TJ trials or to other factors, including democracy. The authors readily acknowledge this limitation.
Large-Sample Comparative Studies
An unpublished study by Hunjoon Kim and Kathryn Sikkink seeks to build on Sikkink and Walling by using multivariate statistical methods and global trials data from 93 transitional countries, in the period from 1980 to 2004.51 It provides a
49 Jane E. Stromseth, David Wippman and Rosa Brooks, Can Might Make Rights? Building the Rule of Law after Military Interventions (New York: Cambridge University Press, 2006).
50 Sikkink and Walling, supra n 9.
51 Kim and Sikkink, supra n 9.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice   11
positive assessment of trials' impact on subsequent human rights conditions, measured by the Cingranelli-Richards (CIRI) Physical Integrity Rights Index, which is based on a scholarly coding of annual country reports by Amnesty International and the US Department of State. The analysis controls for many well-established determinants of repression and finds that transitional countries with trials are less repressive than those without trials. This finding holds in situations of civil war transitions, providing some information on scope conditions for the role of TJ. To see whether trials have effects separate from other TJ measures, the study controls for truth commissions, but finds no conclusive results. This study is the first statistical assessment to draw on a global and rigorously assembled trial dataset, and its findings are encouraging to trial advocates.
Truth Commissions Single-Case Studies
A sophisticated single-case study of truth commission impact is James Gibson's nationally representative survey of South African attitudes towards the country's postapartheid Truth and Reconciliation Commission (TRC), the rule of law and racial reconciliation.52 In general, Gibson finds that South Africans are suspicious of the rule of law, but that their sentiments resemble those of many European publics. Racial identity is found to be a significant factor in shaping individual attitudes, and suspicion of the law amongst blacks drives the national average. Moreover, while some 44 percent of the South African population is at least somewhat reconciled, the study indicates that this result is much lower for the black community.
On a more positive note, a majority of the study participants across all races accept the TRC's version of events. After controlling for other factors, Gibson's analysis finds that those individuals who accept the TRC's version are also more likely to support the rule of law. Support for the rule of law, in turn, is associated with conciliatory racial attitudes and support for a weaker form of majoritarian-ism. In a separate analysis of reconciliation, measured by support for interracial reconciliation, rule of law, political tolerance and institutional legitimacy, Gibson also finds that acceptance of the TRC's version of events is associated with reconciliation at the individual level. The TRC, Gibson concludes, may have had a positive influence through its exposure of abuses by all sides.
Still, Gibson readily acknowledges that his cross-sectional data cannot conclusively support causal claims because it does not trace changes over time. Since most attitudes co-vary, they may all be caused by an underlying and omitted factor.
52 James L. Gibson, 'Overcoming Apartheid: Can Truth Reconcile a Divided Nation?' Politikon 31(2) (2004): 129-155; James L. Gibson, 'Truth, Reconciliation, and the Creation of a Human Rights Culture in South Africa,' Law and Society Review 38(1) (2004): 5-40. The cross-sectional survey of 3,727 individuals was nationally representative and oversampled minorities for statistically meaningful results.
International Journal of Transitional Justice, 2010,1-26
12   O.N.T. Thorns, J. Ron and R. Paris
Thus, the TRC may have reflected, rather than caused, social reconciliation. A review of the study also raises some doubts about the logic and coherence of the evidence presented. First, the study does not observe real differences in individuals' exposure to the TRC that convincingly explain their acceptance of its truth. Second, the evidence only marginally supports the assertion that the TRC increased acceptance of its truth; for the majority of people, knowledge of - and confidence in - the TRC are not clearly associated with truth acceptance. Finally, the finding that reconciliation is lowest among Africans, but that the latter also exhibit the most confidence in the TRC and the highest acceptance of its truth, suggests that the TRC may have in fact impeded, rather than helped, reconciliation.54
Small- to Medium-Sample Comparative Studies
William Long and Peter Brecke's study of 11 civil wars in 10 countries that experienced 'reconciliation events' finds that extensive truth telling is the critical starting point for successful peace settlements, as the three settlements without truth telling did not last. Successful reconciliation, the authors find, is a protracted and complex process that includes truth telling, identity redefinition and 'partial justice short of revenge.'55 The study's strength stems from its careful tracing of the effects of reconciliation in multiple cases. Its research design, however, leads to ambiguity regarding whether truth telling is a cause of peace or is incidental, as other factors may cause both lasting peace and reconciliation.56 Any attempt at causal inference is further complicated by the study's lack of control cases.
Similarly, findings of a positive TJ effect appear in an unpublished paper by Charles Kenney and Dean Spears, who use multivariate regression to assess the effect of truth commissions on democratization in 16 postauthoritarian Latin American countries from 1979 to 2003.57 The study's regionally homogenous sample reinforces the notion that Latin America has specific qualities that encourage successful TJ and, as the authors note, it is unclear whether truth commissions are causally related to democracy. The study also does not account for later authoritarian reversions.
Large-Sample Comparative Studies
An unpublished PhD dissertation by Belinda Botha analyses the effects of truth commissions on regime legitimacy in all 56 nascent democracies in Eastern Europe, Africa and Latin America between 1984 and 1995, and its findings are generally
53 James L. Gibson, 'The Contribution of Truth to Reconciliation: Lessons from South Africa,' Journal of Conflict Resolution 50(3) (2006): 409-432.
54 David Backer, 'Book Review,' review of Overcoming Apartheid: Can Truth Reconcile a Divided Nation?by James L. Gibson, Comparative Political Studies 39(9) (2006): 1157-1161.
55 William J. Long and Peter Brecke, War and Reconciliation: Reason and Emotion in Conflict Resolution (Cambridge, MA: MIT Press, 2003), 65.
56 The study is criticized for this reason by Snyder and Vinjamuri, supra n 3.
57 Charles D. Kenney and Dean E. Spears, 'Truth and Consequences: Do Truth Commissions Promote Democratization?' (paper presented at Annual Meeting of the American Political Science Association, Washington, DC, 1-4 September 2005).
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice   13
positive. Whereas most other research measures regime legitimacy through individual attitudes, Botha uses collective protests as a proxy, arguing that more protests should signal lower legitimacy. Although the logic of this argument is open to debate,59 the study is a reasonable attempt to measure such an elusive and complex concept. Botha's analysis also controls statistically for level of repression and the presence of free and fair national elections, and all her indicators rely on news-based event counts. She finds that a truth commission's presence is associated with lower levels of protests and that this link endures over time. As the first large-sample statistical assessment of truth commissions, this study breaks new ground. Given its limited controls and a sample that is likely not homogeneous, however, it does not support a strong causal claim.
The final study of truth commission effects we review mixes methods to combine the advantages of large-sample statistical analysis with the detailed process-tracing of small-sample comparative research. Eric Wiebelhaus-Brahm's study examines the effects of truth commissions on human rights protection and democratization and offers less ground for optimism than other truth commission impact studies.60 His statistical analysis examines the impact of truth commissions in 78 countries within a global sample of more than 150 countries from 1981 to 2006, and finds that, on average, they had a negative effect on subsequent respect for human rights, but no statistically discernable and significant effect on the level of democracy. Wiebelhaus-Brahm's findings are not definitive, however, as his analysis does not consider variation in the implementation of truth commissions and his research design does not probe scope conditions. As a result, we cannot know from his study whether truth commissions have positive or negative effects in specific political or social environments.
Yet, Wiebelhaus-Brahm also conducts detailed qualitative case studies of truth commissions in El Salvador, Chile, South Africa and Uganda, along with comparisons to neighbouring control cases. By tracing the implementation of truth commission recommendations, he finds that commissions can in fact help by discrediting previously unaccountable institutions, such as the military. Otherwise, he finds little positive or negative impact on democratization. More broadly, he finds that commissions vary in their effect on institutional reforms. Thus, in El Salvador, the UN-sponsored truth commission was largely ignored, and individuals named in the report faced few repercussions. Wiebelhaus-Brahm finds, however, that El Salvador is now more respectful of human rights, which is due in large
58 Belinda Botha, 'Truth Commissions and Their Consequences for Legitimacy' (PhD diss., University of Houston, 1998).
59 Legitimate democratic regimes may, under some conditions, have far more protests than illegitimate authoritarian regimes because social movements and protestors of all kinds feel both empowered and enabled. For instance, South Africa has experienced a large number of protests in recent years, but the African National Congress consistently wins national elections with clear majorities. See, Peter Alexander, 'Rebellion of the Poor: South Africa's Service Delivery Protests - A Preliminary Analysis,' Review of African Political Economy 37(123) (2010): 25-40.
60 Wiebelhaus-Brahm, supra n 39. See also, Eric Brahm, 'Truth and Rights: Truth Commissions and Human Rights Protection in a Cross-National Context' (paper presented at Annual Meeting of the American Political Science Association, Washington, DC, 1-4 September 2005).
International Journal of Transitional Justice, 2010,1-26
14   O.N.T. Thorns, J. Ron and R. Paris
part to police and military reforms. The country has also experienced a massive postconflict crime wave, which has produced broad public support for draconian 'law and order' measures. This suggests that there may be little popular support for human rights, broadly conceived, and that a truth commission's discernable effects are limited to politically motivated abuses.61 Similar conditions, moreover, apply in South Africa, suggesting that popular attitudes towards human rights -including the rights of suspected criminals - are not tightly linked to reductions in politically inspired abuses. A general culture of human rights, in other words, is hard to inculcate, and postconflict truth commissions may not be either sufficient or necessary to create such a culture.
This study is a good example of using mixed methods to investigate TJ impacts, but its findings are preliminary, as Wiebelhaus-Brahm himself warns, because of data limitations and the difficulty of isolating the effects of truth commissions from other factors. Still, more mixed-method studies of this kind, combining high-quality comparative case studies with sophisticated statistical analysis, would greatly advance the field.
Research on Multiple TJ Mechanisms
A few studies do not focus on single TJ mechanisms but consider broader TJ processes, and all of the studies reviewed here include trials and truth commissions in their analyses. One important collection of qualitative studies edited and compared by Alexandra Barahona de Brito, Carmen Gonzalez-Enriquez and Paloma Aguilar examines the determinants of TJ policy choices, focusing on trials, truth commissions and purges in 19 transitional societies across Europe and Latin America and in South Africa. The study also analyses their impacts on democratization and rule of law while taking other factors into account, including transitional conditions and institutional reform. It finds no clear link between 'backward-looking' truth and justice efforts and the functioning of democracy. In Spain, Hungary and Uruguay, for example, democracy developed well without TJ, while in Portugal, El Salvador and Guatemala TJ made little or no contribution. In still other instances, including Argentina, South Africa and the Czech Republic, TJ seems to have contributed to democracy, though the causal links are complex. 'Forward-looking' institutional reforms, by contrast - such as wresting control of the police, military and judiciary from authoritarian forces and establishing their accountability to democratic institutions - are more consistently linked to democratization, although the study finds no indication that reform cannot proceed without purges.62
The study also finds that backward-looking TJ policies, including trials and truth commissions, tend to make positive contributions when they initiate dynamics that generate wider reforms. This depends, however, on starting conditions,
61 For a more detailed discussion of reform efforts and TJ polices in El Salvador, see, Margaret Popkin, Peace without Justice: Obstacles to Building the Rule of Law in El Salvador (University Park, PA: Pennsylvania State University Press, 2000).
62 Alexandra Barahona de Brito, Carmen Gonzalez-Enriquez and Paloma Aguilar, eds., The Politics of Memory: Transitional Justice in Democratizing Societies (Oxford: Oxford University Press, 2001).
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice   15
institutional and political legacies and styles of TJ implementation. When TJ was carried out in a democratic and fair manner in a country under study, it had positive impacts on public support for the new regime. In other words, Barahona de Brito et al. find that TJ is more likely to succeed in countries that are already well on their way to democracy. Correlations between democracy and successful TJ outcomes are endogenous,' meaning that TJ policies may have gotten off the ground in the first place because democratization was already underway. This also means that TJ on its own is unlikely to help where it is needed most. The study's policy implication is that countries with poor starting conditions cannot be pushed onto a positive political and democratic trajectory through TJ alone.
Another recent case study collection edited by Charles Call employs a similar methodology, combining eight country studies on security and justice reforms in postconflict societies with a cross-case assessment. The volume's focus is on reforms rather than TJ perse, but it does explore TJ impacts on the legitimacy and efficacy of justice systems. Interestingly, the study finds no clear link between 'justice for past abuses and the quality and accessibility of justice in the future.'63 Instead, it argues that deficiencies in postconflict justice systems are caused by factors unrelated to TJ, including institutional choices, political decisions and style of war termination. Still, volume contributors do find some grounds for TJ policy relevance, arguing that in Rwanda, Timor Leste and Bosnia popular perceptions of current justice systems were negatively affected by the way past abuses were addressed. Overall, the study's strength is its detailed comparative analysis of larger reform agendas, of which TJ is one part. The approach highlights the value of considering TJ in the context of wider peacebuilding policies, but this also means that its analysis of TJ impacts is limited and that its research design is not geared towards uncovering specific, causal TJ impacts.
A more recent study by Tricia Olsen, Leigh Payne and Andrew Reiter analyses the effect of TJ on democracy and human rights during 91 transitions in 74 countries from 1970 to 2004.64 The study analyses TJ in three ways: as an ordered scale ranging from doing nothing (de facto amnesties) through formal amnesties, truth commissions and trials; as individual mechanisms; and as various combinations of specific mechanisms. The human rights analyses use the Political Terror Scale (PTS) and CIRI measures, both of which are based on scholarly coding of annual Amnesty International and US Department of State reports. The democracy analyses use Polity IV and Freedom House indicators. Controlling for national wealth, world region, timing and prior level of democracy, the authors find some statistically significant and positive effects 10 years after the transitions began but are unable to draw strong conclusions. They find positive effects of TJ in their analyses of the ordered TJ scale and combinations of mechanisms involving trials and amnesties,
63 Charles T. Call, ed., Constructing Justice and Security after War (Washington, DC: US Institute of Peace Press, 2007), 398.
64 Olsen, Payne and Reiter, supra n 9; Tricia D. Olsen, Leigh A. Payne and Andrew G. Reiter, 'The Justice Balance: When Transitional Justice Improves Human Rights and Democracy,' Human Rights Quarterly (forthcoming).
International Journal of Transitional Justice, 2010,1-26
16   O.N.T. Thorns, J. Ron and R. Paris
but only for some of their democracy and human rights measures. Analysed on their own, none of the TJ mechanisms is statistically associated with improved human rights or democracy. Surprisingly, while truth commissions have a positive effect in combination with trials and amnesties, on their own truth commissions (and not amnesties) have a statistically significant and negative effect on human rights conditions.
In an earlier version of the study, the authors also found that the ordered TJ measure has a significant and positive effect on the rule of law after five years, as measured by the rule of law indicator in the World Bank's Governance Indicators dataset.65 Their supplementary descriptive statistics, however, show that only trials are associated with higher rule of law scores, and that all other TJ policy choices are negatively associated with the rule of law.
The Olsen et al. study thus finds some positive TJ effects and is a noteworthy contribution in that it is based on an important, new and comprehensive TJ dataset. Its research design, however, does not advance our understanding of scope conditions. Still, like the Kim and Sikkink study, it is an important and rigorous forthcoming work that is likely to be a key contribution to the field.
Jack Snyder and Leslie Vinjamuri's comparative analysis of 32 postconflict countries from 1989 to 2003 is far less sanguine about TJ's state-level impacts.66 The authors argue that properly designed and enforced amnesties have done a better job than trials and truth commissions of ending civil wars and encouraging human rights, democracy and the rule of law, as measured by Freedom House and Polity IV indicators. Snyder and Vinjamuri suggest that TJ has only worked under specific scope conditions. Trials have contributed to ending abuses only when peace spoilers are weak and the country's justice infrastructure is well established, while truth commissions have been irrelevant or harmful unless employed in countries well on the road to democracy. Truth commissions, moreover, often provide political cover for amnesties, masking the real facilitators of peace.
The Snyder and Vinjamuri study is an important sceptical voice in the TJ debate. The authors pointedly criticize TJ proponents for 'lacking empirical foundations.' Still, some of their own case studies are cursory and do not convincingly trace impacts from individual TJ mechanisms. The authors also do not examine the effects of amnesties as critically as those of other TJ processes. Finally, their study does not consider variation in the implementation of different TJ mechanisms. Their methods and controls are not methodologically sophisticated, suggesting that their work is exploratory rather than confirmatory. Still, their focus on the comparative strength of spoilers and democratic forces, along with their consideration of levels of institutional development, is a vital contribution to the debate over the necessary scope conditions for TJ success.
Leigh A. Payne, Tricia D. Olsen and Andrew G. Reiter, 'Does Transitional Justice Work?' (paper presented at 49th International Studies Association Convention, San Francisco, CA, 26 March 2008).
Snyder and Vinjamuri, supra n 3.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice   17
The sceptical or 'realpolitik' side of the TJ debate has recently been reinforced by Tove Grete Lie, Helga Malmin Binningsbo and Scott Gates, who assess the impact of trials, truth commissions, purges, reparations and amnesties on peace duration in 200 postconflict cases from 1946 to 20 03.67 Using multivariate regression, the authors control for type of conflict termination, conflict characteristics (duration and whether conflict is territorial), Gross Domestic Product (GDP) per capita and postconflict regime type. They find that hard-nosed political and military factors matter most to peace duration, as one-sided victories are the strongest predictor of peace duration. None of the TJ mechanisms they test is statistically significant, although most are positively associated with peace. In an important discovery, however, the authors have found that trials have a statistically significant and positive effect when one-sided victories are removed from the equation. TJ advocates, in other words, maybe mistakenly attributing peace to TJ when the real contributing factor is a one-sided military victory.
The Lie et al. study thus reinforces the notion that the military and the political balance of power have a greater impact than TJ on state-level outcomes in post-conflict situations. Their analysis of scope conditions, moreover, supports some conclusions made by Snyder and Vinjamuri, Baharahona de Brita and colleagues, as well as others. When Lie et al. restrict their analysis to a subset of 87 democracies, they find that reparations and truth commissions are positively associated with peace duration, and that these results are not sensitive to conflict termination patterns. Amnesties in democracies, moreover, appear to be associated with an increase in peace failure.
This research advances the debate in important ways. First, it finds that TJ overall does not have positive effects on peace duration, but that one-sided military victories do. More important, it shows very different results when countries are further disaggregated into democracies and nondemocracies. TJ does not have a clear impact across postconflict countries, but seems more successful in reasonably democratic countries. Thus, political regime type matters. Overall, the study makes a powerful contribution to the debate over scope conditions, suggesting that nonretributive TJ promotes peace when it occurs in countries already well on the path to democracy.
The study suffers from some important drawbacks, however. Like most TJ impact studies, it does not consider variation in the implementation of particular TJ mechanisms. But since TJ implementation may vary substantially, the lumping together of all TJ efforts masks the effects of implementation style. The study's sample selection criteria are also unclear, as its data includes considerably fewer postconflict cases than appear in the original dataset.68 We thus do not know whether the exclusion of other cases has biased the results.
Tove Grete Lie, Helga Malmin Binningsbo and Scott Gates, 'Postconflict Justice and Sustainable Peace,' World Bank Policy Research Working Paper 4191, Postconflict Transition Working Paper 5 (April 2007).
See, Binningsbo, Elster and Gates, supra n 9.
International Journal of Transitional Justice, 2010,1-26
18   O.N.T. Thorns, J. Ron and R. Paris
Data and Analytic Limitations
One of our general conclusions is that most studies find that TJ has either positive or no effects at all. Few studies find that TJ has strong negative impacts. Even this conclusion needs to be treated with caution, however, as few of the studies reviewed above are careful cross-case comparisons. Although more sophisticated comparative research is now emerging, this second-generation effort is still in its early stages.
Individual-case studies help us understand contexts and build theories, but, to provide broad guidance, they must be tested on other cases to determine their scope and generalizability. One way of doing this is systematic comparison of individual cases. The Barahona de Brito et al. and Call volumes are important examples of this approach, as the editors of both collections supplement individual country studies with comparative analysis. Interestingly, neither volume finds robust, cross-case TJ effects, underlining our concerns with the generalizability of TJ efforts. The broader point, however, is that, until recently, most TJ case studies were not embedded in well-designed comparative studies, which limited their policy utility. Good research design - including careful attention to case selection, standardized comparison across cases and interstudy comparability - is vitally important.69
Another key problem is the tendency of most single-country studies to concentrate on cases that have experienced TJ. Well-designed comparisons, by contrast, would also look at cases with similar starting conditions that did not experience TJ, since controlled comparisons of this sort can help determine whether an observed impact is due to TJ or, perhaps, to other factors. To be most effective, these comparisons should match cases based on other relevant variables - such as per capita GDP and level of state institutionalization, peace and democratization -so as better to gauge TJ's unique effects. Although no study can possibly control for every potential variable, carefully selected comparisons will reduce the risk of specious conclusions.
The large-sample cross-national studies we reviewed are encouraging, but they, too, face a range of problems, including a dearth of high-quality data. Only a few available cross-national indicators are relevant to the TJ agenda, including the heavily used but still problematic indicators of state-level human rights performance, PTS and CIRI. Complex social phenomena are hard to capture with quantitative indicators, and concepts such as state respect for human rights, social reconciliation, public culture or state legitimacy are notoriously difficult, and perhaps even impossible, to quantify. As a result, even the most careful dataset will be open to criticism. Social scientists refer to this problem as 'measurement validity,' and it is common to all quantitative social science research. Of course, research should not be driven by what can be more easily measured but by gaps in the knowledge. Thus, research into TJ effects would be greatly facilitated by improvements in the
69 For problems of research design in both qualitative and quantitative human rights research, see, Emilie M. Hafner-Burton and James Ron, 'Seeing Double: Human Rights Impact through Qualitative and Quantitative Eyes,' World Politics 61(2) (2009): 360-401.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 19
quality of cross-national data and the creation of new, innovative measures. By using news-based event data in new ways, the Botha and Meernick studies offer important examples of such an innovative approach.
To get a better sense of the data-related challenges facing contemporary TJ researchers, consider the two leading country-level indicators of respect for core human rights: PTS and CIRI.70 Although both offer comparable country-level data over many years and are widely used, including by some of the TJ impact studies reviewed above, they also come with distinct disadvantages. First, both suffer from missing data for some years and countries. Second, they are coded from Amnesty International and US Department of State reports, both of which have their own biases. Third, both datasets use limited scales of five and nine points, respectively, meaning that the gaps between scores may conceal more than they reveal. Countries with real differences are lumped together. Finally, limited scales and coding biases mean that most countries show little change over time, making abuses appear 'stickier' than they may in fact be. In many cases, changes may occur in increments too small to register, and, in some cases, report authors or data coders maybe reluctant to 'reward' countries with slightly better assessments.71 Leading measures of democracy, such as the Polity IV and Freedom House indicators, have also been subject to similar types of criticism.72 Further, there are few broadly accepted international indicators for state legitimacy or state capacity over time.
Other datasets used in TJ research are equally problematic. Although the Meernik study uses press-based events data to great effect, this has its drawbacks. One systematic study of state violence in Guatemala, for instance, found that press accounts of repression varied dramatically by source, and that newspaper coverage was inversely related to repression. Thus, more information on abuses was available when actual rates of abuse declined.73 Another study found that the volume of human rights press reporting was influenced by a host of factors other than actual abuse, and that rich countries received more attention than poor ones.74 News-based indicators, in other words, may be only partially accurate.
Finally, when any dataset is closely scrutinized by critics, shortcomings soon appear. For example, one researcher carefully studied Sikkink and Walling's dataset and source material and found a host of problems whose severity is difficult to
70 These measures provide annual scores indicating the level of respect for personal integrity rights in a given country
71 Hafner-Burton and Ron, supra n 69.
72 See, for instance, Joe Foweraker and Roman Krznaric, 'Measuring Liberal Democratic Performance: An Empirical and Conceptual Critique,' Political Studies 48(4) (2000): 759-787; Gerardo L. Munck and Jay Verkuilen, 'Conceptualizing and Measuring Democracy: Evaluating Alternative Indices,' Comparative Political Studies 35(1) (2002): 5-34.
73 Christian Davenport and Patrick Ball, 'Views to a Kill: Exploring the Implications of Source Selection in the Case of Guatemalan State Terror, 1977-1995,' Journal of Conflict Resolution 46(3) (2002): 427-450; Patrick Ball, 'On the Quantification of Horror: Field Notes on Statistical Analysis of Human Rights Violations,' in Repression and Mobilization, ed. Christian Davenport, Hank Johnston and Carol Mueller (Minneapolis, MN: University of Minnesota Press, 2005). Davenport and Ball also find that human rights organizations are better at reporting certain types of abuses than others.
74 Howard Ramos, James Ron and Oskar N.T. Thorns, 'Shaping the Northern Media's Human Rights Coverage, 1986-2000,' Journal of Peace Research 44(4) (2007): 385-406.
International Journal of Transitional Justice, 2010,1-26
20   O.N.T. Thorns, J. Ron and R. Paris
assess. Some cases were included in the data even when they did not seem to fit the study's stated TJ definition, and there were coding errors and inconsistencies.75 No data collection effort is perfect, and the Sikkink and Walling trials dataset is both innovative and promising. Still, findings derived from one dataset should always be tested - and then repeatedly retested - on other data before serving as the evidentiary basis for policy.
As a general rule, policy should be guided by a reasonably large and consistent body of high-quality research, and the TJ field is still years away from this maturity. There have been several impressive large-scale TJ data collection efforts in recent years, and analyses of these and other new datasets are bound to move the field forward.
A Research Agenda for Analysing TJ's Impact
In the following sections, we consider the utility of various research methodologies for TJ policy.
Cross-National Comparisons
The best way to establish country scope conditions for TJ success, irrelevance or failure is systematic comparison across multiple countries.76 We found only 10 cross-national studies focusing specifically on trials and/or truth commissions (and two more that covered broader postconflict policies). More of these are needed to strengthen early findings. They could either consist of statistical treatments of multiple countries over time or of qualitative 'controlled comparisons' of carefully chosen cases. Both research efforts would advance the TJ agenda by examining the impact of major variables, such as level of international intervention, type of transition (democratic or postconflict), state strength, institutional capacity, world region and level of economic development, and others. This, in turn, would help researchers and practitioners determine whether TJ is likely to be successful only in countries with certain levels of international engagement, democracy, economic development or state capacity. Cross-national studies do not automatically advance scope conditions but rather need to be specifically designed to do so through sampling or investigation of interaction effects.
Individual Country Studies
Our interest in comparative studies is not meant to obviate the importance of individual-case studies. Even when experts and policy makers have a better grasp of scope conditions, they still will not have ironclad guidelines for individual countries. Regardless of findings at the cross-national level, there maybe significant within-country variation across regions and over time, due to ethnicity, level of
75 This research is documented in an unpublished MA thesis. Its findings should be treated as tentative until the paper has undergone peer review. See, Theodore Blank, 'Measuring Transitional Justice: The Extent and Impact of Effective Prosecution in Latin America' (MA thesis, Carleton University, 2007).
76 See, Backer, supra n 6.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 21
economic and state development and historical experience. Policy making, like the practice of medicine, requires a delicate balance to be struck between general findings and individual cases. Policy makers and TJ practitioners should be aware of the best and most recent general scholarship, but actors in specific contexts need to work with researchers who have deep country knowledge. One promising way of furthering both individual-case analysis and general knowledge is rigorous within-country analyses, such as Meernik's rare study of time-series data. As for cross-national work, country-level studies can be quantitative, qualitative or a combination of both.
Opinion Polls, Focus Groups and Interviews
Conscientious country-level policy making requires extensive consultation with interest groups, populations and social sectors. These can draw on a range of accepted research methods, including qualitative interviews with samples of various kinds, ethnographies among perpetrators, victims or ordinary citizens, focus groups and population-based surveys.
Such consultations are also ethically vital. TJ interventions, like any other policy 'treatments,' should not be foisted on populations without informed consent. After all, it is the populations, not the donors, who will suffer if things go awry.77 The views of vulnerable groups, such as rural women, lower castes or oppressed minorities, should receive special consideration, as these groups are likely to suffer most if TJ prompts renewed instability. As the TJ effort moves forward, moreover, both the general population and specific social sectors should be regularly consulted to determine whether they still support the TJ campaign in its current form. The need for consultation and informed consent is particularly urgent given the role of powerful foreign actors in promoting TJ. In many poor or postconflict areas, locals are ill-equipped to reject or modify external policy interventions. Importantly, such studies should be carried out by evaluators who are independent of the TJ implementers themselves and who have no particular organizational or professional stake in specific TJ policy choices.
Population-Based Surveys
Population-based surveys represent a particularly promising avenue for future research. Surveys have long been a staple of social science and are useful because they can estimate individual attitudes and experiences in specific times and places. In sufficiently large samples, they can provide more specific information on subgroups within a broader population. Surveys are thus a powerful tool, but their true potential has still to be explored in analyses of conflict and human rights.78
77 See, Jack Goldsmith and Stephen D. Krasner, 'The Limits of Idealism,' Daedalus 132(47) (2003): 47-63; Thorns, Ron and Paris, supra n 1; Office of the UN High Commissioner for Human Rights, Rule-Of-Law Tools for Post-Conflict States: National Consultations on Transitional Justice (2009).
78 Oskar N.T. Thorns and James Ron, 'Public Health, Conflict and Human Rights: Toward a Collaborative Research Agenda,' Conflict and Health 1(11) (2007), http://www.conflictandhealth.com/ content/1/1/11.
International Journal of Transitional Justice, 2010,1-26
22   O.N.T. Thorns, J. Ron and R. Paris
The most important potential contribution is that surveys can help policy makers establish what the public considers to be the most pressing TJ issues and the most appropriate mechanisms. When properly designed, moreover, surveys could help evaluate the extent of pro- and anti-TJ constituencies and assess the impact of TJ by tracking attitudes towards ongoing or past initiatives, human rights and former adversaries. Such information is especially useful when gathered at regular intervals over time in panel studies.79
The field of survey-based TJ studies in postconflict settings is still in its infancy, but initial results are intriguing, including those of surveys conducted in northern Uganda, Rwanda and the former Yugoslavia.80 These studies demonstrate widespread support for TJ, although in at least one important case, respondents favour amnesty over prosecutions.81 Surveys may help map the extent of human rights abuses, name parties responsible for abuses, define local interpretations of justice, specify desirable TJ mechanisms and identify the authorities that local populations want to guide the TJ effort.
Surveys also suffer from some methodological limitations.82 Importantly, the existing survey scholarship suffers from a lack of over-time data on the same people, or even on the same general population.83 Yet, it is precisely this kind of data that facilitates causal arguments. Respondents' attitudes towards TJ after implementation, for example, may be shaped by their pre-TJ views rather than by the TJ process itself.84 Surveyors can ask respondents to try and remember how they viewed TJ in the past, but current opinions invariably influence perceptions
Gunnar Theissen, 'Public Opinion Research on Transitional Justice Mechanisms: Issues and Challenges,' in Assessing the Impact of Transitional Justice: Challenges for Empirical Research, ed. Hugo van der Merwe, Victoria Baxter and Audrey R. Chapman (Washington, DC: US Institute of Peace Press, 2009).
Notable examples include: International Center for Transitional Justice (ICTJ) and Human Rights Center, Forgotten Voices: A Population-Based Survey of Attitudes About Peace and Justice in Northern Uganda (July 2005); Berkeley-Tulane Initiative on Vulnerable Populations and ICTJ, When the War Ends: A Population-Based Survey on Attitudes about Peace, Justice, and Social Reconstruction in Northern Uganda (December 2007); the surveys in Eric Stover and Harvey M. Weinstein, eds., My Neighbor, My Enemy: Justice and Community in the Aftermath of Mass Atrocity (Cambridge: Cambridge University Press, 2004). See also, James Gibson and Jeffrey Sonis, 'Cambodians' Support for the Rule of Law on the Eve of the Khmer Rouge Trials,' International Journal of Transitional Justice, 4(3): XXX and Patrick Vinck and Phuong Pham, 'Outreach Evaluation: The International Criminal Court in the Central African Republic' International Journal of Transitional Justice, 4(3): XXX.
ICTJ and Human Rights Center, supra n 80; Berkeley-Tulane Initiative on Vulnerable Populations and ICTJ, supra n 80.
Surveys measure perceptions of institutional performance rather than performance itself. Perceptions are not always accurate or relevant. Some respondents, moreover, are likely to give answers that they think interviewers or powerful local figures want to hear, rather than their own opinions. For survey limitations, see, Theissen, supra n 79; James L. Gibson, 'Taking Stock of Truth and Reconciliation in South Africa: Assessing Citizen Attitudes through Surveys,' in Assessing the Impact of Transitional Justice: Challenges for Empirical Research, ed. Hugo van der Merwe, Victoria Baxter and Audrey R. Chapman (Washington, DC: US Institute of Peace Press, 2009).
See, David Backer, 'Watching a Bargain Unravel? A Panel Study of Victims' Attitudes about Transitional Justice in Cape Town, South Africa,' International Journal of Transitional Justice, 4(3): XXX for one of the few longitudinal studies conducted.
Backer, supra n 6. See also Backer's critique of Gibson's study, supra n 54.
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 23
of past attitudes. Surveys, moreover, provide snapshots in time, making it difficult to draw broad conclusions about dynamic events.85 Attitudes may change quickly, and support for or discomfort with TJ at specific points in time may not provide conclusive evidence of success or failure. Further, different regions, social sectors and groups are likely to have different views on TJ, and aggregate national surveys will miss this information if they are not designed to capture differences through stratification and oversampling.
To avoid simplistic or mistaken conclusions, surveys should always be supplemented with other research methods, including interviews and focus group discussions with key stakeholders and special interest groups. Otherwise, TJ advocates may mistake broad public support for TJ with feasibility. If warlords, political leaders or other key figures are virulently opposed to TJ, it may not matter if the broader public is in favour of it. Some groups have more leverage than others, and well-organized, funded or armed groups can derail peace deals and TJ efforts despite favourable public opinion. Still, surveys provide a wealth of context-specific knowledge that may be invaluable to TJ experts and policy makers. Just as we should not assume particular TJ impacts, we also must not assume that TJ enjoys similar support or opposition across constituencies, countries and transitions.
Conclusion
Most studies covered in this review find that TJ has either positive or no effects at all. Few of them find that TJ has negative impacts. Strong claims in either direction, however, are so far not supported by the existing scholarly literature. Individual case studies have unearthed contradictory findings, such as Akhavan's and Meernik's differing analyses of the ICTY's impact on peace in the former Yugoslavia. Similarly, cross-national studies have turned up contradictions and unclear results. Snyder and Vinjamuri find potentially negative effects of trials and truth commissions, but Sikkink and Walling find no harmful effects in the case of trials. In a classic example of mixed findings, Wiebelhaus-Brahm's statistical analysis indicates that, once other factors are controlled for, truth commissions have no effect on democracy or human rights protection. His qualitative case studies, however, suggest that truth commissions have limited and positive effects through their support of institutional reforms.
Scholars and practitioners have made strong causal claims for both positive and negative state-level TJ effects, but these have not been substantiated by other studies. It is not unusual, however, for findings to be weak and contradictory during the early phase of a research programme. To be useful to policy makers, findings need to be replicated with different samples and measurements, and confirmed through in-depth 'process tracing' of causal effects in individual cases. Existing studies have done important analytical groundwork and developed exploratory hypotheses that need to be further investigated and confirmed.
85 Theissen, supra n 79; Gibson, supra n 82.
International Journal of Transitional Justice, 2010,1-26
24   O.N.T. Thorns, J. Ron and R. Paris
Most of the methodological issues we raise here are beginning to be addressed in new research, which seems to indicate that the social science of TJ is maturing. Now that the justice cascade has provided more time and more cases for study, a critical research mass is beginning to emerge. An increasing number of researchers is working on this issue, there have been several recent large data collection efforts and we are beginning to see the first analyses of these new datasets.
Yet, while the new cross-national studies are addressing issues of causal analysis and scope conditions, their overall findings are still inconsistent. For instance, Kim and Sikkink control for other factors and find that countries that hold trials are less repressive afterwards, even in the context of civil war. Olsen et al. also find that TJ has positive effects for democracy and human rights in some of their analyses. Lie et al., however, find that only trials appear to have a positive effect on peace, but that military victories may provide a stronger alternative explanation for this effect. These studies consider different outcomes, of course, but the contrast shows that not all good things necessarily go together. They also point to the need for careful case-study research to investigate the causal relationships that various large-sample statistical studies are purporting to find.
Indeed, basic questions on TJ impacts are still unanswered, including: Do TJ mechanisms strengthen or undermine peace in transitional societies? Do they lead to improvements in the areas of human rights, rule of law and/or democratic practices in transitional societies? Do they foster reconciliation or exacerbate divisions at the societal level? To answer these questions, TJ researchers must grapple with the conceptual and methodological issues that all social scientists deal with, including the need for appropriate and precise measurements of TJ outcomes; separating TJ policies from other influences and factors, including countries' predispositions to certain outcomes; comparing countries with TJ policies to those without; comparing wide variations in TJ implementation across countries; and accounting for longer-term effects that are difficult to detect.
TJ impact researchers need to design studies that explicitly consider and evaluate alternative explanations of outcomes, including democratization, institutional reform and broader statebuilding efforts. In particular, policy practitioners and researchers should consider how TJ mechanisms may interact with other dimensions of peace negotiations, peacebuilding operations or postauthoritarian transitions. One recent trend in the literature on postconflict peacebuilding focuses on disentangling the contradictions between a peacebuilding enterprise's different goals.86 If TJ is anything like the other major components of peacebuilding - and there is no reason to believe that it is not - we should expect TJ efforts to interact in complex and often unpredictable ways with other statebuilding efforts.
Moreover, for research into TJ effects to be useful for policy makers, it needs to advance our understanding of scope conditions.87 What kinds of countries and
86 Roland Paris and Timothy D. Sisk, eds., The Dilemmas of Statebuilding: Confronting the Contradictions of Postwar Peace Operations (London: Routledge, 2009).
87 On scope conditions, see, Alexander L. George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (Cambridge, MA: MIT Press, 2004).
International Journal of Transitional Justice, 2010,1-26
State-Level Effects of Transitional Justice 25
postconflict situations are most conducive to TJ success, and which kinds are likely to provoke failure? Causal arguments about TJ should be carefully delimited by region, time, transition type, level of democracy, institutional capacity and other characteristics. This should be a key goal of further research. For instance, we must be cautious about generalizing from regional findings, as particular characteristics may make TJ work differently across geographic space.
Some scholars have already begun this task. The best studies we have reviewed are cautious about their claims, specifying the conditions under which they are likely to apply. Snyder and Vinjamuri, for instance, consider the role of intervening factors, such as the strength of both democratic forces and spoilers and the level of institutional development. As a result, their work provides important scope conditions for the more general claims about TJ's state-level impacts. Similarly, Barahona de Brito et al. find that TJ policies are more likely to advance democratization when linked to wider institutional reforms, which in turn depend on a range of institutional and political conditions. Finally, Lie et al. find that TJ does not have a clear effect across all postconflict countries, but that nonretributive TJ policies, including truth commissions, seem to be more successful when only reasonably democratic countries are considered. This finding in particular is fascinating, as it suggests that both TJ advocates and sceptics may be right, but that their arguments apply to different political conditions.
By uncovering the conditions under which specific TJ mechanisms are most likely to have desired effects on transitional societies, careful research can produce more policy-relevant knowledge than that provided by broad cross-national generalizations or idiosyncratic cases. So-called middle-range theories, which focus on nuanced scope conditions and causal pathways, tend to be more useful to policy makers because they provide contingent and specific generalizations.88 As Sikkink and Walling conclude, rather than making broad generalizations,
it is much more interesting to examine under what conditions trials can contribute to improving human rights and enhancing rule of law systems, or what sequencing or judicious combination of TJ mechanisms can help build democracy and resolve conflicts.89
By advancing such knowledge, social scientists are able to support the work of international lawyers and TJ policy makers. Although there are grounds to suspect that TJ is best suited to states with relatively strong institutions and a certain minimum level of democracy, these hypotheses, along with many others, must be systematically tested and retested.
Although strong claims of any kind about state-level TJ effects are not yet warranted, this does not mean that efforts to promote TJ should be abandoned. On the contrary, setting aside considerations of their empirical effects, moral and legal rationales exist for pursuing TJ policies. This article has focused on the
88 ibid.
89 Sikkink and Walling, supra n 9 at 443.
International Journal of Transitional Justice, 2010,1-26
26   O.N.T. Thorns, J. Ron and R. Paris
evidentiary foundations for claims that TJ has specific, discernible effects on transitional societies. Given the paucity and contradictory nature of the empirical findings to date, there appears to be an urgent need for more sustained, systematic, comparative analyses, and for greater attention to fact-based rather than faith-based claims.
International Journal of Transitional Justice, 2010,1-26
View publication stats