What Is My Argument? Making solid arguments based on firm evidence and logic is at the heart of critical thinking. After compiling and evaluating the relevant evidence, one of the first steps is to critically examine the key operating assumptions and identify those that clients can depend on to justify the decisions they make. Analysts need to recognize unsupported assumptions as early as possible to avoid relying on them when building an analytic framework Many unsupported assumptions turn out to be key uncertainties that require additional research and collection. Before even starting to write, analysts should think about how best to construct their argument. As they conduct their research and sit down to outline their drafts, analysts should ask themselves: _ • What Js,the key message? • What are the strongest arguments or claims that will support this primary thesis? • What is the best evidence to buttress these claims? More sophisticated analysts will consider how best they can portray their argument as a story or illustrate it with examples or graphics. Enriching the argument with stories or visuals will make it more compelling and deepen understanding of the underlying dynamics. This increases the chances the analysis will stick with the reader in the coming days, months, and years. When formulating the line of analysis, be sure to consider alternative explanations—or at least the null hypothesis—especially when generating evaluative or estimative products. This helps ensure that analysts do not fall victim to dangerous cognitive biases and intuitive traps and risk being wrong when it matters the most. If all can agree that there is no single uncontested history of what has happened in the past, then why should anyone hold ana-:; lysts to the standard of predicting only one version of the future? Analysts who master the discipline of instinctive^ observable phenomena and considering multiple future scenarios usually are 1140 Critical Thinking for Strategic Intelligence more proficient at developing robust analytic frameworks, identifying boundaries, avoiding logical fallacies, and escaping outdated mental models and 'mindsets. The practice of constructing competing scenarios and incorporating alternative views into a draft offers one of the best defenses against the pressures of politicization. Structured analytic techniques can provide a rigorous— and objective—foundation from which to view the data, identify conflicting or unsupported assumptions, and explore the evidentiary and logical basis for competing lines of analysis. If analysts can structure arguments in ways that help protagonists—and clients—focus on the key evidence and the underlying logic rather than argue over competing conclusions, they have done their job well Even the most rigorous analysis, however, is subject to error. In the intelligence profession, most experienced analysts aspire to be right more than 60 percent of the time. Given the frequent lack of data and the complexity of topics they cover, this is not an unreasonable standard. To meet or exceed this standard, however, you must carefully check your work before it goes out the door. Few would argue that discovering a major error or vulnerability before editing is far better than having to explain errors months after the paper is published to your peers, supervisors, or even a congressional committee. Good critical thinkers know well that conducting a Premortem Analysis will always take less time than responding to inquiries from postmortem investigators—and it is far less stressful. Are My Key Assumptions Well-Founded? SETTING THE STAGE Most analytic judgments are based on a combination of evidence and assumptions that influence how information is interpreted.\j^J^Uffi>pM&&i&-defined as something that is taken for granted or that is accepted as true or as certain to happen.2 A good critical thinker knows that some assumptions are unfounded and all assumptions must be examined critically. The challenge is that our cognitive processes make it difficult to discover or articulate our own assumptions. The key to overcoming this cognitive obstacle is to employ a process that helps make the implicit explicit. The individual or the team needs to "put on a different hat" that forces the analyst or the team to think more critically and examine suppositions and biases—including some that they may not even know are influencing the analysis. A structured technique known as the Key Assumptions Check provides that vehicle. LOOKING MORE DEEPLY The Key Assumptions Check is a systematic effort to make explicit and question the working assumptions that guide an analyst's interpretation of evidence and reasoning about any particular problem. Such assumptions are usually necessary and unavoidable as a means of filling gaps in the incomplete, ambiguous, and sometimes deceptive information with which an analyst must work They are driven by the analysts education, training, and experience. They also can be strongly influenced by the organizational context in which the analyst works. A Key Assumptions Check helps analysts avoid analytic traps, generate new insights, and "Euclid taught me that without assumptions there is no proof, Therefore, in any argument, examine tiie assumptions." —EXBeif Author and Research Mathematician1 J14Z t-'AKI lit: VVMAI \b IVtT AKUUiVlUt I ( discover knowledge gaps. Moreover, it is easy to use. Most analysts have difficulty identifying their own assumptions because many are rooted in sociocul-tural beliefs that are held unconsciQusly,Qr;. ^ to be true and not subject to challenge. Nonetheless, identifying key assumptions and assessing the overall impact the assumptions exert on the analysis are critical parts of a robust analytic process. In addition, the technique often uncovers hidden relationships and reduces the chance of surprise should new information render old assumptions invalid. The process of conducting a Key Assumptions Check is relatively straightforward in concept but often challenging in practice. In developing a list of key assumptions, work from the prevailing analytic line or lead C hypothesis back to the key factors that support it. The Key Assumptions Check can be conducted at any point in the analytic process. Most analysts prefer to conduct the check as they begin their analysis to avoid having to reorganize a draft or reconceptualize a section after discovering a faulty assumption late in the process. Reviewing the list of assumptions as you complete the project enables you to see if they still hold up or need to be modified because of events that occurred while you were drafting (see Figure 11.1), Being open-minded is essential when conducting a Key Assumptions Check. Involve others less familiar with the topic who can openly challenge the operating assumptions of the group. Based on our own experience in facilitating key assumptions exercises, around one in four key assumptions usually collapses on careful examination. The Key Assumptions Check involves an eight-step process. Figure 11.2 provides a template. The eight steps are as follows: 1. Gather a small group of individuals who are working the issue, along with a few "outsiders." The primary analytic unit already is working from an established mental model, so the outsiders are needed to bring other perspectives. 2. Ask participants to bring a list of assumptions to the meeting. If not everyone has a list, start the meeting with a silent brainstorming session. Ask each participant to write down several assumptions on index cards. 3. Collect the cards and list the assumptions on a whiteboard or easel for all to see. "When key assumptions and critical data are not challenged, the result at best is poor analysis; at worst it becomes the explanation for why we have stumbled into another major intelligence failure."4 —Randolph H. Pherson "Overcoming Analytic Mindsets: five Simple Techniques"5 ^fmriLEA \ I . rtl C !¥iy !\Ky MMU1 MjJIIUi li WtfCt!*TUUMUt?U f l*§JT FIGURE 11.1 Stop! Recheck Assumptions After you have completed your first draft, stop and review the key assumptions you generated at the start of your projict Conditions may have changed, and your thinking could have evolved. 4. Elicit additional assumptions. Work from the prevailing analytic line back to the key arguments that support it. Use various devices to help prod participants' thinking. Ask the standard journalist questions: Who, What, How, When, Where, and Why. a. Phrases such as will always, will never, or would have to be suggest that an idea is not being challenged and perhaps should be. b. Phrases such as based on or generally the case usually suggest that a challengeable assumption is being made. 5. After identifying a full set of assumptions, critically examine each assumption. Ask: a. Why am I confident that this assumption is correct? b. In what circumstances might this assumption be untrue? c. Could it have been true in the past but no longer be true today? d. How much confidence do I have that this assumption is valid? e. If it turns out to be invalid, how much impact would this have on the analysis? FIGURE 11.2 Key Assumptions Check Worksheet Key Assumptions Issue: Categories A. Supported B. With Caveats C. Unsupported -A Key Uncertainty Key Assumption A B C 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 6. Place each assumption in one of three categories: a. Basically solid and well supported b. Correct with some caveats c. Unsupported or questionable—the key uncertainties 7. Refine the list, deleting those assumptions that do not hold up to scrutiny and adding new assumptions that emerge from the discussion. 8. Consider whether any unsupported assumptions or key uncertainties should be converted into collection requirements or research topics. One technique analysts can employ to decide which category to assign to an assumption is to ask, "Can a decision maker or policymaker make decisions about moving resources or people based on this assumption?" If the answer is "yes," then the assumption can be rated as supported. If the answer is "it depends/' then the assumption merits a rating of with caveats and the caveat or caveats need to be recorded. If it would be inappropriate or hard to justify the movement of people or resources on the basis of the assumption, then the assumption should be rated as unsupported. One mistake some analysts make is not distinguishing among assumptions, assessments, and information or intelligence reporting (see Figure 11.3). The key is to decide what is reporting and what is analysis in your own mind and be careful to present it as either reporting or analysis in your writing. A Key Assumptions Check can prove valuable at any time in the drafting process. While most assumptions checks focus on substantive conclusions and the underlying analysis, checking assumptions about the quantity and credibility of the sourcing used to support the analysis can reap major dividends. FIGURE 113 Distinguishing Among information. Assumptions, Judgments, and Assessments A classic mistake many novice analysts make is confusing information or reporting with key assumptions, assessments, and judgments. • Information consists of ail the facts relevant to the case or the situation. The information describes what has or is happening and summarizes ali of the relevant reporting on the subject. • Assumptions guide an analyst's interpretation of evidence and reasoning about any particular problem and are often taken for granted. • Judgments are short sentences that highlight the most significant analytic points in a paper. They should include both facts and analysis—the "What" (Continued) (Continued) and the "So What." Good key judgments identify a trend, point out what is new, or expiain the meaning of the facts cited in the paper. They should be presented in order of importance with the most relevant judgment first. Use of the word because is a key indicator that the sentence contains a judgment. A summary usually describes what is discussed in a paper; a judgment conveys the purpose or main message of the paper. Drafting a set of key judgments is a critical skill as these judgments often may be the only part of the document a client has time to read. • Assessments are judgments about unknowns such as the following: o Who is or will be involved o What will happen o When it will take place o Where it will occur o Why it is occurring or will occur o What the implications are or will be o How it is likely to evolve Separating information from analysis or distinguishing reporting from assumptions, judgments, and assessments can be done in a variety of ways. The easiest approach is to first present the information or reporting and then draft a follow-on paragraph or paragraphs that contain the analysis and key judgments. Another strategy is to meid the reporting and the analysis into a series of paragraphs that tell a coherent story but identify the analysis with bold type. The key is to distinguish what is reporting and what is analysis in your own mind and be explicit about presenting this in your analysis. Do not assess what is known or has already been reported to have occurred. For example, one would not assess that hackers are using the internet to exploit children and advertise child pornography, but one would assess that use of the Internet by hackers for this purpose is growing and becoming more sophisticated. • Identifying key assumptions and assessing their validity are critical parts of ; a robust analytic process. • If an assumption is solid or supported, then policymakers and decision ; makers can make resource decisions based on this assumption, • Experience has shown that one in four key assumptions is unsupported ■ when subjected to critical examination. * An unsupported key assumption often becomes a key uncertainty that should be resolved through additional research or intelligence collection. • A judgment often contains the word because. Analysts must not confuse key judgments and key assumptions with information or reporting. Review Case Study II, "Puzzling Food Poisonings in Germany." • At the outset of this case, what key assumptions were reporters (and analysts) making about the outbreak of E. coli in Germany? How many were supported, in need of caveats, and unsupported? • Were one in four key assumptions unsupported, as is usually the case? * How would a Key Assumptions Check have helped German officials working this case? * If a Key Assumptions Check was conducted, what key uncertainties would have been identified suggesting priority topics for investigation? 1. The information used in this chapter was taken from Pherson Associates training materials (www.pherson.org). A discussion of the Key Assumptions Check and how it relates to other structured analytic techniques can also be found in Heuer and Pherson, Structured Analytic Techniques, 209-214. 2. The first definition for the word assumption is provided by Webster's Dictionary at www.merriam-webster.com/dictionary/assumption; the second definition is provided by the Oxford English Dictionary at http;//oxforddictionaries.com/definition/ assumption. 3. E. T. Bell, "What Mathematics Has Meant to Me," republished in Harmony of the World 75 Years of Mathematics magazine, eds. Gerald L. Alexanderson and Peter Ross (Mathematical Association of America, 2007), 4. Randolph H. Pherson, "Overcoming Analytic Mindsets; Five Simple Techniques" presentation to the National Security and Law Society Emerging Issues in National and International Security March 21-22, 2005, Washington College of Law, American University, Washington, DC. 5. Ibid. Can I Make My Case? SETTING THE STAGE The crux of the analytic process is making sense of and deriving knowledge from data. Most human brains do this naturally with varying degrees of facility, but some have difficulty putting the pieces together to create a meaning that is greater than the sum of its parts. Many of our students, for instance, have trouble extracting the "So What" from separate pieces of data that together might indicate drug cartel violence transcending a border or an attack plan nearing an operational state. They get stuck on the tactical meaning of the individual sources and cannot rise above the facts to identify and warn of impending danger. Our job as analysts is to give our clients insights that they do not receive from other sources. We are expected to interpret complex events. Our primary task is to characterize the forces and factors that drive events so decision makers can quickly understand what is going on and decide what actions they should take in response. The means by which we communicate what we know is by making a case or telling a story. Palm: Weil an argument's not the same as contradiction. LOOKING MORE DEEPLY Analytic products quite simply are what philosophers and logicians call logical arguments or lines of reasoning. They are vehicles that deliver analysts' thinking about what is true and what is not. The process of getting to the argument often includes what philosophers refer to as dialectic—^ type of conversation in which analysts challenge one another to come up with the best answers to implicit or explicit questions asked by their clients. Cfeese: It can be. Palim No it can't An argument is a connected series of statements to establish a definite proposition. Cieese: No it isn't Palin: Yes, it is! ft Itsn't just contradiction? —Michael Palin and John Cieese *The Argument Clinic/ Monty Python's Ffying Circus 1150 PART Hi: WHAT IS MY ARGUMENT? The prospect of building an effective argument can be daunting, particularly if you are breaking new analytic ground or exploring future trends. The process can flow more easily if you follow the Analysts Roadmap (see the inside back cover of this book). The analytic context or framework (see Chapter 3) and the AIMS process (see Chapter 4) guide you in setting the parameters for your argument. Your concept paper (see Chapter 8) steers you in planning your product, selecting research methods, and assembling supporting evidence. As we have discussed earlier, the purpose of analysis is to break an issue into its components. The function of synthesis is to build knowledge by combining and comparing the parts with other things, looking at the relationships among the parts, and assessing the totality of the system. The argument will put it all together to answer the client's questions and array the best evidence to support key points. BUILDING AN ANALYTIC ARGUMENT An argument is made up of a set of statements. One statement constitutes the conclusion; it should be supported logically by the other statements. The conclusion is based on reasonable evidence, connected to observable fact, and characterized by clear, traceable, and fair thinking. Here is a step-by-step approach:1 • Take a few minutes to ground yourself. o Review your AIMS (see Chapter 4). o Refine your question and scope out a reasonable answer, o Consider the main types of evidence that are most relevant to that answer. o Anticipate the questions and objections your reader might have to your evidence and answer. • Based on your preparatory work, make an assertion or claim. • Back it with supporting, observable evidence. • Link the two with reasons why the data is relevant to the claim. The claim and the reasons can usually be connected by the word because (see Chapter 17 for the importance of because in probability statements and levels of confidence), • Acknowledge other views and differences and provide evidence and reasons why you do so. • Refine your claims, reasons, and evidence. This process appears disarmingly simple but can quickly become complex when you have more than one question or multiple claims. Your highest-ordei claim (or thesis if you are writing a report in college) is supported by reasons which can become subclaims themselves and then are supported by their owr ■ ■* ■ reasons. The interconnected claims and reasons must eventually be based on evidence or assumptions. In the end, an argument is only as sound as the evidence that supports it. If your evidence is clearly and directly illustrative of your reason and claim, then your argument consists of laying out the facts and generalizing them as reasons that support the claim. A common example of this is inferring fire (claim) when we see smoke (evidence) because (reason) smoke only occurs when there is fire. Another example is as follows: Example Claim: Gangs pose an increasing threat to public safety in our community. Reason 1: They are moving into suburban neighborhoods as well as urban areas. Evidence: New markings and gang-related violence are appearing in suburban neighborhoods as well as urban areas. Reason 2; They are undertaking a broader range of activities. Evidence: Primary drug distributors are moving into the trafficking of weapons and illegal immigrants. Reason 3: More gangs are working internationally, taking advantage of the Internet. Evidence: Recent activity reported on northern and southern borders as well as new connections with foreign drug-trafficking organizations and intercepted incriminating e-mail communications. In analytic products, however, the relationships more often are not straightforward. The evidence may be ambiguous or clients may interpret it in different ways depending on their mindsets and experience. It may consist of weak indicators that are consistent with more than one scenario. This places the burden on analysts to characterize convincingly the reasons that link the evidence to the claim. How analysts bridge this conceptual gap provides the basis on which their clients evaluate the quality of their reasoning and evidence and builds client trust in the analysts expertise, understanding of their needs, and answers to their questions. The best way to learn how to produce effective analytic arguments is to practice doing them. Anthropologist Rob Johnston, in a study of professionals ''Forgotten were the elementary rules of (ogle, that extraordinary claims require extraordinary evidence and that what can be asserted without evidence can also be dismissed without evidence/ — Christopher Hitchem Author and journalist, Slate,' ■ October 2003 152 PART III: WHAT tS MY ARGUMENT? in intellectually demanding fields, such as intelligence analysts, astronauts, and anesthesiologists, identified six stages for developing expertise: (1) studying cases, (2) identifying patterns within cases, (3) generalizing patterns across cases, (4) formulating hypotheses, (5) creating models, and (6) testing models.2 He found that developing expertise takes seven to ten years and experience on 50,000 cases. The length of each stage depends on the individuals skills and the complexity of the issue. Similarly, in Outliers Malcolm Gladwell popularized the concept of the "10,000-hour rule," which posits that success results not so much from innate talent as from the drive to practice an activity for what he estimates to be some 10,000 hours.3 Based on our experience teaching and mentoring new analysts, we offer the following advice and admonitions to guide the development of expertise in argumentation: • Set in your mind the structure of claims, reasons, and evidence. The terminology gets confusing because the words can be used in different ways (see Figure 12.1).4 Using the parts of an argument not only will simplify structuring your product but also aid you in reviewing your paper and the work of others. • Make sure your claims, reasons, and evidence are specific and significant. Busy clients scan products for "news they can use" and skip over vague and generalized terminology that does not add to their knowledge base. Test your claims by assessing the'significance of their opposifes or showing them to your colleagues. • Include appropriate qualifications. Each component of your argument should reflect how you have dealt with the limitations of knowledge or other conditions by explicitly identifying your assumptions (see Chapter 11) or certainty (see Chapter 17). • Avoid reasons that are opinions or unlinked to evidence. We recall { vividly the frustration of one manager who bemoaned that his analysts ■ did not understand that their "personal opinions were not analysis." , Opinions are based on beliefs, emotions, and intuition. Analysis may \ reflect some of those, but only after linking them with evidence and \ consciously considering alternatives, ^Enrich your argument by reflecting the broader context and the work of others. In academic research, this can be done through lengthy citations and references, but in most analytic writing it must be done succinctly and often indirectly. THE ROLE OF FORMAL LOGIC Many critical thinking courses discuss deductive and inductive thinking and the basics of formal logic but often fall short in clarifying how these concepts make a difference in day-to-day analysis. We know of one organization that FIGURE 12.1 Elements of an Argument Booth Cotomb, Williams Claim Reasons (Warrant) Evictence í fSSĚĚSSBmSĚĚm Toul m in Claim * Modality (le,f usuatfy, rarefy) Warrant ssumptw ; * Backing í Gtouncís j i (Descriptions í • ot Experiences) i t— Hume Conclusions inferences Expenen«, j t t, - ,'. " ■ AusThink Contention Reason (Evidence) • Assumption. Presentation Lead {What and So what) Topic sentences (Why) (What) Í Supporting ! ! Evidence i Source: Copyright 2016 Pherson Associates, LLC. Al! Rights Reserved, required analysts in their performance reviews to identify specific examples of products that employed deductive and inductive reasoning, but the organization later gave up the practice. That does not mean that the principles defined by the Greek philosopher Aristotle some 2,300 years ago are not fundamental to good thinking, but the measurement of successful analytic products is based on the utility of the product for the client rather than on the logic employed. The two types of formal logic, induction and deduction help us understand how to connect claims and their support. • Deduction reaches a general conclusion (or claim) based on specific and complete evidence (or premises). If the evidence is true, the claim must be true. Example: Evidence: Selling drugs is an illegal activity. Evidence: John is selling drugs. Claim: John is committing an illegal activity. • Induction arrives at a generalized conclusion or claim from observed and incomplete evidence. The claim is outside the known facts and is possibly or probably true but could be false. The "inductive leap" is what gets the analyst from the known truth of the evidence to the claimed truth of the conclusion. Example: Evidence: Videotape shows an unidentified person trying to enter a sensitive facility. Evidence: A lock on a door of the gate shows signs of tampering. Claim: The facility is being targeted for attack. For most analysts, few if any of our analytic arguments are deductive. We rarely have the luxury of complete information in which only one answer can be true. Furthermore, the point of analytic argumentation is to posit and support claims because they have the possibility of supporting more than one plausible alternative. Inductive reasoning helps us as analysts understand how the patterns and trends we observe as part of our experience or research can be true, but it also reminds us that we may be missing critical data or neglecting to consider the full range of alternatives. Scholars over the past 150 years have broadened the field of argumentation beyond the rigid principles of formal logic to guide us in accounting for the complexities of our changing world. While lengthy discussion of argumentation theory is beyond the scope of this book, two concepts warrant attention because they are frequently mentioned in analytic training courses and reference books. • Abduction is referred to as a third kind of logical reasoning that is useful to analysts in testing hypotheses. Abductive reasoning starts with a set of facts, based on which the analyst develops a hypothesis that, if true, would provide the best explanation for those facts. In the nineteenth century, logician Charles Sanders Peirce identified abduction as part of the scientific method. The purpose of abduction is to generate the guesses or hypotheses whose consequences can be derived by deduction and evaluated by induction. This is an important concept for investigators, diagnosticians, and analysts who have reason to believe a certain precondition can explain the conclusion.5 Example: Evidence: The grass has been wet every time it has rained. Evidence: The grass is wet. Claim: It must have rained. • The Toulmin model is the basis for the argument framework used in this chapter. The terms—and indeed those used in most contemporary argument texts—are based on twentieth-century philosopher Stephen Toulmins updating of the classical model to account for contextual differences in relating an argument's claims and support (see Figure 12.2).6 Toulmin introduced fiWt^;:ii^'jbuiimin Model öf Argumentation Anne is one of Jack's sisters So, presumably Since any sister of Jack's may be taken to have red hair Anne now has red hair Unless Anne has dyed/gone white/lost her hair... On account of the fact that ali his sisters have previously been observed to have red hair Source: Copyright © 2003 Stephen E. Toulmin. Reprinted with the permission of Cambridge University Press. the term warrant to describe the belief, supposition, or principle that explains how the evidence supports the claim. Whether or not you need to state this explicitly depends in part on how intuitively obvious or commonly accepted your reason is for your claim. Analysts need to reconsider these connections when thinking through their article from the reader's perspective (see Chapter 16). MATCHING ARGUMENTS AND ANALYTIC PURPOSE How an analyst frames the issue depends on the nature of the problem and the purpose of the product. The arguments used in most intelligence writing reflect one or more of the types on the analytic spectrum (see Chapter 5 and Figure 5.4). • Descriptive analysis includes arguments that are data driven, reactive to events, and answer five of the journalist's standard six questions (the Five Ws and an H). Descriptive analytic products summarize and report information about an event, person, entity, or phenomenon but may also generalize by relating data presented to broader patterns and trends. The primary task for the analyst is to convince the client that the information relayed is an accurate and current rendition of reality. To do that, the supporting data needs to be as unambiguous, accurate, and authoritative as possible. The analysts' primary tools include: o Expert knowledge o A variety of credible sources that are appropriately characterized in terms of authority and accuracy throughout the product o Data displays, such as tables and graphs, in appropriate formats and scales o Examples of verified facts or representatives of the verified facts that led to the conclusion Reports and summaries may not be considered arguments in the classic sense, but analytic products should always contain the implications—the "So What"—and evidence to support these implications in order to answer clients' questions and needs. They usually rely on factual claims from indirect sources that also require corroboration or support. This means that the broader or more speculative the facts and implications presented, the greater the range of evidence required to support them. Acknowledging and accounting for exceptions or contradictory evidence is as critical to the clients' acceptance of reports and summaries as it is for other types of analytic products. Generalizations are the result of inductive reasoning in which we observe something a few times and assume based on our incomplete information that it happens often or all the time. Clients must see claims for generalizations as plausible and reasonable. Generalized claims are supported by identifying a number of verifiable examples, if the generalization has a descriptive or factual purpose, the examples will be facts. If, on the other hand, it involves explanatory or cause-and-effect relationships, then the examples will be relationships. The more far-reaching the generalization, the more examples will be needed to support it. Statistics are quantitative generalizations that draw conclusions about large sets of data based on smaller, but carefully defined, representative samples. Statistics support the likelihood of the generalized claim but do not prove it because the data are only a representative sample. Statistics can provide data and gravitas to the analysis, but too many tables or charts can weigh down the product and detract from the claims. We recall one report that illustrated this point well; it was peppered with superfluous line graphs but lacked a simple pie chart that would have illustrated the main claim. Example of Descriptive Analysis: Claim: An unknown anarchist with the online pseudonym BORDERCon-troL is trying to organize cyber attacks to take place from March 15 to 25 to protest immigration reform legislation coming to a vote at the end of March. Reason 1; BORDERControL is encouraging others to stage cyber attacks that would draw attention to the cause. Evidence: Hackers have been observed stealing e-mail addresses, flooding Web servers, defacing Web sites, and disrupting communications and financial systems. Evidence: BORDERControL specifically cited as targets the e-mails of US Senators and Representatives as found on their campaign Web sites, the military, other unnamed government entities, unnamed industries that use immigrant labor, and news services. Reason 2; BORDERControL is trying to rally hackers to the cause via a presentation titled "Electronic Civil Disobedience and the Border." Evidence: A press release calls for sympathizers to meet on March 10 but does not provide a venue. • Explanatory analysis includes arguments that are slightly less reactive and slightly more conceptually driven than descriptive arguments. They answer the journalist's "Why" question and identify the relationships between facts, events, observables, and trends. They often assert that one thing happened as a result of one or more other observables. A successful causal argument uses both direct and indirect evidence to establish the relationships and the reasonable arguments that one event caused another to happen. The broader or more ambiguous the relationship, the stronger the evidence required to substantiate the claim. The link between the two events often cannot be absolutely verified, but a successful causal argument leaves no doubts about the validity of the link. The degree of justification depends on the analytic intent. A Ml causal explanation for an effect requires the analyst to identify and support a number of causes, both direct and indirect, that might have an impact. For example, if the claim is that the price of oil will rise in the next quarter, the evidence would be that winter is approaching (increasing the demand for heating oil), instability in the Middle East is growing (undercutting supply), and two US refineries were just closed for repairs due to hurricane damage (further undercutting supply). To assign responsibility, the analyst needs to address the most immediate causes that relate to the effect. Analysts should always try to "look around the corner" to determine the future implications of current events. This means analysts should also consider the remote causes of current situations. Both necessary and sufficient causes can be cited to establish the relationship. • Necessary causes are those in which the result could not have happened without the cause. For example, the acoustic effect of thunder does not occur without the electrostatic discharge of lightning. Being a mammal is necessary to be a human, but it is not sufficient • Sufficient causes are those that might have caused the event, but other factors may have contributed as well. For example, erratic driving is sufficient for a police officer to stop a car, but it could result from a number of causes. • Potential causes can be clustered or chained so that in combination they contribute to the cause (see Figure 12.3). They can also be defined as indicators and arrayed against various scenarios to determine which are most diagnostic or helpful in distinguishing among the scenarios.7 In addition to defining the types of causes, explanatory arguments can contain: • Analogies to compare relationships. The analyst must be sure the phenomena are equivalent and that the key factors are the same in both cases. When people are involved, motive must be established to keep from making unwarranted assumptions about actions. • Clues as to the certainty of the relationship. Words and phrases such as caused, resulted in, produced, or were responsible reflect relationships characterized by certainty. When the relationships are less certain, analysts should use words and phrases such as contributed to, facilitated, and influenced. FIGURE 123 Causes of Economic Growth and Slowdown Source; Stock Trend Investing, www.stocktrendiavesting.com. [160 PART III: WHAT iS MY ARGUMENT? Example of Explanatory Analysis: Claim: Escalating rivalry between the Sinaloa and Juarez drug cartels in Juarez has led to increased violence on the US side of the border in El Paso, Texas, Reason 1: Drug-related violence by the cartels has surged in El Paso. Evidence: Five people were murdered at a local drug rehabilitation clinic this week; the Juarez cartel has targeted drug rehabilitation facilities in other border cities, but this is the first in El Paso. Evidence: The Sinaloa cartel attempted to move drugs across the border in daylight. Trucks crossing the border eluded capture by driving at high speed and firing automatic rifles at customs officers. Reason 2: The cartels are attempting to establish "drug safe houses" in El Paso to backstop operations as they vie to control drug activity in the region. Evidence: Law enforcement and customs officers discovered three tons of marijuana and a cache of small arms in a ranch house outside El Paso and arrested four men connected with the Sinaloa cartel. Evidence: Another rancher nearby was reported missing. Evidence: Government officers have begun visiting properties outside of El Paso in response to reports of a greater cartel presence there. * Evaluative analysis includes an argument with the intent to assess or to establish the meaning or implications of the issue or event by making judgments on key questions. Is the threat increasing or decreasing? Is the risk higher or lower? Is the weapon operational or not? Is the government stronger or weaker? The critical part of an evaluation is detailing clear and precise definitions of a reasonable and complete set of criteria to use in the appraisal. The best way to achieve this is by using criteria that have assumed common values, identifying their effect, citing authoritative sources, and comparing them to known facts. The analysts assessments are based on their own expertise and the logic and power of the attributes used to establish the criteria. If the criteria are commonly recognized or linked to reputable sources or other well-established expertise, then the analytic judgment will be well received. Example of Evaluative Analysis: Claim: The loosely organized hacking movement known as the Collective is increasing efforts to build capability to attack industrial control systems. While no targets appear to have been damaged, growing familiarity with the systems could quickly lead to operational attacks. CHAPTER 12: Can I Make My Case? 161 j Reason 1: Collective members have posted threats on the Internet against several industrial plants in the Midwest. (Intent: threatening statements) Evidence: Web site announcement of a threat against a water treatment plant in Arkansas; calls appear on the Internet encouraging people to organize protests against energy companies. Reason 2: Collective members are seeking information about industrial software systems and showing knowledge of system security controls. (Intent: building knowledge) Evidence: Twitter posts share results of browsing for information on specific software and code that would enable access. Reason 3: No evidence of actual attacks but some indications of increasing capabilities. Evidence: There are no postings of malicious activity or reports from potential targets. Internet discussion covers vulnerabilities of control systems operations and design showing growing sophistication. * Estimative analysis addresses the question, "What happens next?" It usually forecasts one or more possible scenarios. In such future-oriented projections, analysts typically base their arguments on available data and historical precedent but must rely more heavily on creative brainstorming and logical reasoning to form their analysis. The essence of producing solid estimative analysis is to first identify a set of the key drivers that are most likely to shape the future. Some of these drivers are fairly constant, such as demographic trends, but others can be extremely volatile, such as popular sentiments, civil-military tensions, or the quality of leadership. Both need to inform good estimative analysis, but most of the analytic ferment should revolve around the key uncertainties. Cause-and-effect relationships argued in explanatory analysis might form the basis for a future in which the same relationship among actors continues. For example, political analysts may argue that particular factors present before an election were influential in determining the election results and the same factors are likely to produce a similar outcome in future elections. Or a doctor tells his patient that the results of a series of medical tests indicate the presence of disease. In the future, similar test results would point to the same disease. A good critical thinker knows intuitively that there is always more than one credible potential future. Solid estimative analysis will provide several future trajectories and include signposts or indicators to tip off the client about which of the various futures is beginning to emerge, Two other characteristics of good estimative analysis are the capacity to bound future trajectories and inform clients of circumstances about which they do not need to be concerned. Policymakers and decision makers are accustomed to hearing what could go wrong from intelligence and security 162 PART III: WHAT IS MY ARGUMENT? analysts, but rarely are they told that they do not need to waste intellectual capital and resources worrying about a particular development. Warning about what might go wrong is a core function for many analysts, but often the policymaker or corporate decision maker would rather hear what can be done to make things work better or improve the profit line. Example of Estimative Analysis: Claim: The United States will become less dependent on nuclear energy in the next decade. Reason 1: Nuclear energy will become increasingly expensive relative to other sources. Evidence: The price of natural gas is already coming down with the discovery of new repositories. Reason 2: New alternatives to nuclear energy will emerge. Evidence: The government and private industry are investing substantial sums of research and development funds to make current alternative energy sources cheaper and are aggressively looking for others. TECHNIQUES FOR SCOPING OUT YOUR ARGUMENT The more robust your argument and evidence, the more credible you appear to your client. Our own model for how to produce quality analysis was our first division chief, who expected his analysts to stop and think through what was new and significant about their issue and then type out flawlessly a one-page current intelligence article. He could do this on a manual typewriter with no typographical errors. While he made this look simple, the process he followed internally was obviously more complex. We have learned over the years a variety of techniques to help develop, track, and evaluate our arguments before we sit down to draft a product. • Topic sentence outlines are easy ways to plan the main points and flow of your product, to identify and understand the argument in another's written argument, or to evaluate your own argument once you have a written draft (see Figure 12,4). Western students are taught to begin paragraphs with topic sentences, but some Western and non-Western writers of arguments may place the main point later in the paragraph. The following steps can assist in making a topic sentence outline: o Write what you think will be your topic sentences or—if you have the written product—copy or highlight the first line of every paragraph. CHAPTER 12: Can i Make My Case? 163 o Condense these lines into the three components: conclusions or claims, reasons, and evidence. o If the argument is not clear, generate or scan the document for additional claims, reasons, and evidence, adding them to the outline. o Rearrange the lines if necessary to make a clear argument, o Identify any gaps that remain in the argument. • Argument Maps, like the topic sentence outline, evaluate the argument by separating the claims and evidence, but they do so graphically so it is easier to think about complex issues and share with others the reasoning behind the conclusion.8 An Argument Map visualizes the argument parts as a tree diagram that starts from a conclusion or claim, branches to reasons (which may become subclaims) and evidence that support or conflict with one another, and ends when the components represent beliefs or suppositions rather than factual evidence. These suppositions are the arguments key assumptions; the mapping process provides a logical and systematic way to identify them (see Figure 12.5), The technique is most useful before a product is written to ensure no holes exist in the argument. It is also a good way to evaluate the strength of the evidence and to determine if the argument can be made more effectively or elegantly. Like other techniques, such as scenarios generation. Argument Mapping can be very complex and take a long time to complete, or it can be a quick way to check the quality of your work. Analysts should be able to evaluate and justify the arguments in their products. Argument Mapping requires some training and practice to ensure you are doing it correctly, but commercial software can greatly ease the process. Using sticky notes on a whiteboard can also work well in more informal sessions. Color-coding as follows can also be effective: o State the conclusion or main claim, placing it at the top of the tree, o Link the reasons and warrants with green lines to the conclusion, o Connect the evidence with green lines to the reasons each supports. o Identify the objections, challenges, or counterevidence, linking them with red lines to the reasons or evidence in the case of counterevidence they oppose. o Specify rebuttals, if any, with orange lines. An objection, challenge, or counterevidence that does not have an orange-line rebuttal suggests a flaw in the argument. o Evaluate the argument for clarity and completeness, ensuring that red-lined opposing claims and evidence have orange-line rebuttals. I im mm fit: whai ib my AKbUMtN 11 'WSW^ diitime ol"th'^: Declaration of lnd^pendehte: DECLARATION OF INDEPENDENCE When, in the course of human events, it becomes necessary for one people to dissolve the poiitical bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the laws of nature and of nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation. [SCOPE NOTE] We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable rights, that among these are life, liberty, and the pursuit of happiness. [PRIMARY ASSERTION] Such has been the patient sufferance of these colonies; and such is now the necessity which constrains them to alter their former systems of government. [EVIDENCE SUPPORTING VIOLATION OF PRIMARY ASSERTION] In every stage of these oppressions we have petitioned for redress in the most humble terms: our repeated petitions have been answered only by repeated injury. [PRIMARY ATTEMPT TO ADDRESS VIOLATIONS] Nor have we been wanting in attention to our British brethren. [SECONDARY ATTEMPT TO ADDRESS VIOLATIONS] We, therefore, the representatives of the United States of America, in General Congress, assembled, appealing to the Supreme Judge of the world for the rectitude of our intentions, do, in the name, and by the authority of the good people of these colonies, solemnly publish and declare, that these united colonies are, and of right ought to be free and independent states. [FINAL CONCLUSION] * Venn Analysis is a visual technique that helps analysts explore the logic of arguments by graphically displaying relationships among classes through overlapping circles. The overlaps indicate elements of separate groups that have something in common. This technique is useful in organizing and conveying similarities and differences among categories of items, as shown in the diagram of critical thinking skills presented in Figure 12.6. Commercial office software and freeware offer Venn diagram templates that create the overlaps automatically. The process is simple: o List the classes and members of each class. o Make a series of circles that overlap where classes have members in common. When applied to argumentation, Venn Analysis can be used to reveal invalid reasoning or validate the soundness of an argument. The technique also CHAPTER 12: Can 1 Make My Case? 165 FIGURE 12.5 Argument Map of Osama bin Laden's Declaration of War Against the United States, 1996 There is a Zionist Crusader Alliance (ZCA) - Israet is Zionist The United States is a Crusader The United States I and Israel are allied Duty to expel the ZCA from Saudi Arabia Trie ZCA is occupying Saudi Arabia Troop presence since 1991 Corrupted the government - Controlled the economy Made the King an infidel King adopting secular laws Sharia taw is required of all Muslims ZCA's laws the most to gain Using defense forces to support the ZCA, not defend the country The ZCA is not allowed to be in Saudi Arabia ZCA are infidels Infidels are not allowed in Saudi Arabia Duty to fight the ZCA elsewhere Muslims elsewhere suffer because of the ZCA - List of Jihads Clear conspiracy H between the ZCA and the UN Muslims are their main target Youth "feel and strongly believe" it is their duty to fight the ZCA everywhere Mohammed said infidels cannot be in the Land of the Two Holy Mosques That land now extends to alt of Saudi Arabia L Our duty to expel the ZCA _ All Muslims are obligated to defend Muslim lands No one else will do it FfGURE 12.6 Venn Diagram of the Components of Critical Thinking Source: Copyright 2016 Pherson Associates, LLC. Ail Rights Reserved. helps analysts organize their thinking, look for gaps in logic, or examine the quality of an argument.9 • Storytelling is a powerful mental organizer that has been part of human intellectual tradition since the cavemen, but it is rarely mentioned as a specific analytic technique. Just as business planners, doctors consulting with patients, and lawyers addressing juries use narrative to make their forecasts, diagnoses, and arguments compelling and easier to understand, a good analyst should create story lines for their clients, generate scenarios, and ponder "What Ifs" (see Figure 12.7). Narrative engages the entire brain—logic, emotion, and imagination— drawing on memories of past experiences, culture, and learning. Researchers have shown that the brain visualizes the entire "chunk" of data in a story-based picture that is recalled more easily than disparate pieces of information. When the story is incomplete, we fill in the gaps by relating the story to our personal experience. The downside is that stories reflect our personal biases, but the upside is that they can help us smooth rough spots in our analytic thinking. For analysts, the story is a narrative framework for presenting or organizing information or, as legal writing expert Kenneth Chestek describes it, the "scaffold on which you hang your argument."10 In contrast to the FIGURE 12.7 Comparing Analysis and Storytelling Elements Analysis SISiHiSBIijBi^ill^Biili Who Characters What, How Plot When, Where Setting Why Theme So What Goal left-brained Five Ws and an H, storytelling has analogous elements that speak to the right side of our brain. Stephen Denning identifies eight types of stories for business leaders, of which two are particularly useful for analysts: knowledge-sharing stories are about problems, and future stories are about forecasting, scenarios, and vision,11 Cornell professor Daphne Jameson came to similar conclusions after observing language used in business meetings, noting that managers preferred narrative to deal with conflicts, to unify groups, and—most interesting—to share reasoning on complex issues.12 Managers turned to stories rather than abstract arguments or statistical measures in part because they allowed more context. Narrative and traditional analysis share many of the same key attributes, suggesting they are left-brain or right-brain variations on ways to communicate complex information. Effective stories include the following characteristics: o Have a key point or takeaway o Include vivid detail that will be remembered o Grab attention with a strong beginning o Are structured to now through several short segments o Keep sentences simple with minimal dependent clauses HOW CAN I GO WRONG? Most analysts are not and will probably never be formal logicians, but they should be able to distinguish a solid argument from a weak one. We encourage our students to practice by reading and listening to arguments, then evaluating why they are effective or fallacious. Newspaper op-ed pages provide good of examples of well-reasoned analysis, and advertisements, political campaigns, "In formal logic, a contradiction is the signal of defeat, but in the evolution of real knowledge it marks the first step in progress toward a victory." —Alfred North Whitehead (British Mathematician and Philosopher) Science and the Modem World 192S PART III: WHAT IS MY ARGUMENT? and much cable television commentary can provide stunning examples of how-arguments can be poorly formed, unsupported, and misused. Analysts can avoid common logical errors by carefully defining the terms they use and rejecting unfamiliar, ambiguous, or controversial terminology Some words can become "hot buttons" for those who are seeking to politicize or sensationalize analysis, particularly if a sensitive assessment is leaked to the public through the media or Internet. This has become a particularly important issue for homeland security analysts with regard to civil rights and civil liberties issues. Logical fallacies involve a flawed relationship between an arguments claim and its supporting facts or logic. Using old or outdated information—even when more up-to-date information is lacking—is an obvious example of an unacceptable practice. Being aware of the types of common fallacies helps analysts avoid them by revealing the logical disconnect between their premises and conclusions. The fallacies that are most problematic for analysts involve arguments in which the evidence is logically irrelevant to the conclusion or fails to provide sufficient evidence for the conclusion, • Circular argument (or tautology) uses the claim or conclusion as part of the supporting argument. Example: President Reagan was a great communicator because of his ability to speak compellingly to the American people. • Inadequate sampling occurs when the sample used as a measure to draw a conclusion is too small. Example: I think the just-released movie will become a big hit based on the three people we interviewed leaving the theater. • Hasty generalization uses a general claim based on insufficient or unrepresentative evidence. It often includes stereotypes and words like alt, every, always, and never. Example: Deaths from drug overdoses have doubled in the past three years. Therefore, more Americans than ever are dying from drug abuse. • False analogy supports an argument with evidence that is not essentially similar. Example: If we can crack the genetic code, we should be able to find a cure for the common cold. • False dichotomy occurs when a set of opposing forces or possibilities is reduced to only two options, misrepresenting the complexity of the situation. Example: It must be either my way or the highway. CHAPTER 12: Can I Make My Case? 169 • Non sequitur asserts a logical relationship between a conclusion and a premise where none exists. The conclusion does not follow the premise. Example: Joe Smith is a devoted father; therefore, he will be an excellent governor. • Post hoc, ergo propter hoc (Latin for "after this, therefore because of this") claims that if an event preceded another, it must have caused the subsequent event to occur. Example: Since Chief Jones took office, the crime rate has decreased by 7 percent. Chief Jones should be applauded for reducing the crime rate. • Slippery slope relates the first and last steps in a causal chain when the intervening steps have not occurred. Example: I didn't get the job I applied for, so I guess I should forget about a career as an analyst. • Distraction (or red herring) brings irrelevant points to distract attention from the issue being argued. Example: If we build a fence along the entire border, the illegal immigration problem will be solved. • Ad hominem argument (Latin for "to the man") is against the person making the argument rather than against the argument. Example: Browns argument for political reform is worthless because he is on his third marriage. • Ad popuium argument (Latin meaning "to the people") uses popular behavior or opinion to argue a point rather than citing factual evidence. Example: Most people approve of the new traffic rules; therefore, they must be working. • Appeal to authority occurs when the opinion of a recognized expert is automatically seen as valid. Example: Reverend Jones said we should picnic in the shade so this is our only option. • Analysts convey their thinking to their clients by arguments or sets of statements that contain a conclusion supported logically by the other arguments or statements. • Arguments are natural follow-ons to frameworks, the AIMS process, and— for longer products—concept papers. • The best way to learn how to generate effective arguments is to practice! • Analysts should pay particular attention to their claims, reasons, and evidence and make them specific and significant. • Analysts almost always use inductive thinking because their data is incomplete; this means they are always vulnerable to being wrong. • How to frame the argument depends on whether the analyst is engaged primarily in descriptive, explanatory, evaluative, or estimative analysis. • Topic sentence outlines and Argument Maps are good techniques for scoping an argument. • Storytelling is an underused, but powerful, thought organizer. • Some of the most dangerous logical fallacies to avoid are false analogies, hasty generalizations, false dichotomy, and post hoc, ergo propter hoc arguments. Review Case Study I, "Countering the Iranian Nuclear "Threat: Stuxnet and Its Broader Implications." • What is the primary assertion made by the analyst in this case study? What key evidence and logic does the analyst provide to support this claim? • What are two key secondary assertions and the key evidence and logic used to support the claims? • Are any contrary views or evidence presented that would challenge the key assertion or the secondary assertions that you identified? • Can you find examples of each type of analysis—descriptive, explanatory, evaluative, and estimative—as they appear in the case study? • Can you find any examples of logical fallacies in the case study? 1. The discussion on argument is informed by a number of works on argumentation but is principally drawn from Pherson Associates training materials (www.pherson .org); Louis M. Kaiser and Randolph R Pherson, Analytic Writing Guide (Reston, VA; Pherson Associates, 2014); Wayne C. Booth, Gregory G. Colomb, and Joseph M. Williams, The Craft of Research, 3rd ed. (Chicago: University of Chicago Press, 2008); Joseph M. Williams and Gregory G. Colomb, TTte Craft of Argument, 3rd ed. (New York: Pearson Education, 2007); Stephen E. Toulmin, The Uses of Argument, 2nd ed. (Cambridge, UK: Cambridge University Press, 2003); Frans H. van Eemeren et. al., Fundamentals of Argumentation Theory (Mahwah, NJ: Lawrence Erlbaum, 1996); and Katherine J. Mayberry, Everyday Arguments (Boston: Houghton Mifflin, 2009). 2. Rob Johnston, "Analytic Culture in the US Intelligence Community—an Ethnographic Study" Center for the Study of Intelligence, 2005, https://www.cia.gov/library/ center-for-the-study-of-intelligence/csi-pubiications/books-and~monographs/ analytic-culture-in-the-u-s-inteltigence-community. 3. Malcolm Gladwell, Outliers (New York: Little Brown, 2008), 39-42. 4. The word claim can refer to the main claim that underpins the product or to subclaims. Reasons, when supported by other reasons, become claims in themselves. Warrants provide additional backing for reasons, connecting a reason to a general consequence of a general circumstance (i.e., fire results from smoking debris, or animal flee when they sense danger). For a single source with more detailed treatment of argumentation, we recommend Booth, Williams, and Colomb, The Craft of Research. 5. Williams and Colomb, The Craft of Argument, 235-242. 6. Stephen E. Toulmin, The Uses of Argument (Cambridge, UK; Cambridge University Press, 2003). 7. See Chapter 6 in Heuer and Pherson, Structured Analytic Techniques, 133-161, 8. This discussion and the technique conventions are based on information from the Austhink Web site, www.austhink.com/critical/pages/argument_mapping.html. 9. See Heuer and Pherson, Structured Analytic Techniques, 72-77. 10. Kenneth D. Chestek, "The Plot Thickens: The Appellate Brief as Story," Journal of the Legal Writing Institute 14 (2008): 131. 11. Stephen Denning, The Leaders Guide to Storytelling: Mastering the Art and Discipline of Business Narrative, 2nd ed. {San Francisco: Jossey Bass, 2011). 12. Daphne A. Jameson, "Narrative Discourse and Management Action," Journal of Business Communication 38, no. 4 (2001): 476-511.