Proving Analysts Wrong – Part II
Do you have trouble admitting when you’re wrong?
Most of us do because it is hard to admit we have made a mistake. Once we have come to a conclusion (like which political candidate to support), we tend to accept data that supports our view and ignore data that would undercut that decision. We fall into the traps of Confirmation Bias, Ignoring Inconsistent Evidence, Relying on First Impressions, and the Anchoring Effect. Structured Analytic Techniques (SATs) are designed to save us from these pitfalls. This issue of the Analytic Insider presents Argument Mapping, the second of a select group of analytic techniques, including Indicators and Analysis of Competing Hypotheses, that can spur analysts to admit they were mistaken in their initial analysis.
Technique #2: Argument Maps
Argument Maps are used to test a single hypothesis through logical reasoning. An Argument Map is a tree diagram that starts out with a lead hypothesis or conclusion and then branches out to show the logic, evidence, and assumptions that support or undermine that hypothesis or conclusion. An Argument Map makes it easy for analysts and, more importantly, decision makers to clarify and organize their thoughts. With the entire map arrayed in front of them, they can more easily and accurately evaluate the soundness of the analysis and their resulting decisions.
Imagine if you are engaged in a discussion of whether Brexit makes sense or Obamacare is worthwhile. Those on each side of the issue will marshal piles of evidence and logic to support their view, leaving it to you to decide who has made the stronger case. Each side will emphasize what best supports their view and ignore facts or logic that argue against their position. When the issues are this complex, it becomes very hard to figure out what makes the most sense. This is an ideal setting where a structured technique can help you sort through the thinking process.
An Argument Map forces both sides to put everything on the “same table” in an organized fashion. Each side lists the arguments (or claims) and evidence that support the hypothesis, the claims and evidence that undermines it, rebuttals for all claims, and assumptions inherent in the argument. With all this information arrayed in the context of a single diagram, both sides can then stand back and evaluate all aspects of the issue. Once all the evidence is arrayed, evaluated, and prioritized, the map will show whether the conclusion is fundamentally supported by the evidence, logic, and assumptions or if it falls apart.
The construction of an Argument Map makes it much harder for advocates to focus attention only on arguments that best support their position. All evidence must be evaluated within the context of the entire argument. If one side does not agree with the conclusion, they are free to add new data or new logic at the appropriate location in the map or alter initial assumptions.
When a team of analysts constructs an Argument Map, a picture will emerge showing the overall strength or weakness of the initial hypothesis. If the analysts agree to approach the topic with an open mind, a consensus will usually form within the team over what is the proper analytic bottom line. In most cases, analysts will freely admit that some initial positions they took were wrong because the technique forced them to 1) consider new evidence, 2) weigh the significance of all the evidence, 3) discover possible faults in their logic, or 4) recognize that some of their initial assumptions were flawed.
Globalytica’s case study, Iraq WMD: Facts, Fiction, and Yellowcake, uses an Argument Map to show how each argument that supported the conclusion that Iraq had imported yellowcake from Niger can be refuted. Click here to purchase the case study today.
Is Iraq importing yellowcake from Niger for its nuclear weapons program?