One of the greatest strengths of this volume is the honesty of the authors, who openly discuss the strengths and significant limitations to the use of counterfactuals, even when guided by Tetlock and Belkin’s criteria. The volume presents a clear framework and takes a useful first step towards developing a more systematic, coherent approach for employing counterfactuals in social science research. However, the volume also highlights a number of serious problems for using counterfactuals. One of the primary challenges to this analytical approach is identified by the editors in their introduction. They present six normative criteria for judging the use of counterfactuals, described as ‘most likely to contribute to the ultimate social-science goals of logically consistent, reasonably comprehensive and parsimonious, and rigorously testable explanations’ (p. 17). Yet, Tetlock and Belkin acknowledge that these standards are open to interpretation and will, at times, conflict with one another. While a positive step in developing counterfactual reasoning as an accepted social science methodology, this issue will require further attention if counterfactuals are to be effectively used in the study of world politics.

Expanding upon more common criticisms regarding intentional biases associated with the selection and evaluation of counterfactuals, the contributors review many psychological biases that affect their application. Olson, Roese and Deibert (Commentary 2) identify the motivational and cognitive processes which may lead researchers to adopt and assess counterfactuals based on their preferred theories and ideologies, to focus on exceptional or unusual antecedents or upon negative outcomes. Rather than turning us away from their use, awareness of the potential for psychological biases warns scholars to exercise caution in their counterfactual reasoning.

The ‘small-N problem’ that plagues researchers of world politics has found its way into counterfactual analysis as well. The volume implicitly argues that counterfactuals may be used to address this problem by presenting researchers with a way to increase their population of cases. While the contributors make successful claims in other areas, using counterfactuals for such purposes is highly-questionable. While the potential for bias alone should be enough to question such usage, Fearon (Chapter 2) provides an additional argument against this practice when he argues that counterfactuals may be of most use in areas where ‘causal mechanisms and regularities’ are already well understood. Unfortunately, in world politics, he concludes, we do not possess such understanding in most issues of interest.

Foremost among the difficulties of employing counterfactuals is that it presents a truly arduous task for researchers, who quickly find themselves cast in the role of science fiction ‘time travelers’, whose seemingly innocuous actions have somehow changed the course of history. And, much like the butterfly in chaos theory whose beating wings produce hurricanes on the far side of the globe, scholars using counterfactuals to make even minimal changes in historical events are placed in the unenviable position of accounting for a universe of potentially significant shifts in the time line.

Illustrating this problem is the fine chapter by Khong examining the Munich case and the plausibility of the statement, ‘Had Britain confronted Hitler in 1938, he would have backed down and World War II might have been avoided’. Arguing for its plausibility, Khong observes that the scenario requires only a minimal re-write of history, though the historical consequences of the action remained unclear (for example, perhaps Hitler would have welcomed an earlier start to the war). Indeed, despite detailed historical analysis of the case and the plausibility of rival hypotheses, Khong still falls victim to the ‘time-traveler paradox’. Specifically, while one can readily envision some of the possible changes to history, one remains severely limited in taking into account all potential impacts. Analysts lack perfect information of the historical context. They also lack understanding of all the potential cause-effect connections brought to life by their historical alteration.

For example, while it is possible that Hitler would have backed down from a confrontation in 1938, this outcome might have merely delayed the war to a later, more opportune time. In the short term, the ‘lesson’ of Munich in Khong’s scenario could be stated as ‘standing up to aggressors preserves the peace’. Yet, alternative ‘lessons’ are also conceivable if one considers possible long-term consequences of this scenario. German military modernization was not complete in 1939. Naval construction plans for a sizable surface fleet capable of challenging Britain, comprising Bismark-type vessels and large quantities of submarines, were not scheduled for completion until 1945. Technological innovation ranging from Tiger tanks, V-1 and V-2 rockets and jet aircraft occurred during the war, but Germany lagged behind in producing the quantities needed for victory. Had the war been delayed until the early-1940s, how might these innovations, coupled with Germany’s revolutionary blitzkrieg tactics, altered the course of the war? A German victory from a delayed start to WWII would certainly provide a different ‘lesson’ for Khong’s Munich scenario, which did not consider the impact of such military-technological innovation. This illustrates the challenge of using counterfactuals, no matter how carefully. Analysts face the immense difficulty of accounting for all important historical variables, all conceivable interactions, and both the short-and long-term consequences of their historical alterations in any policy case.

While pointing out the difficulties associated with using counterfactuals, the contributors to the volume also present a number of alternative uses for the approach. One such alternative is the ability to distinguish between plausible and implausible arguments. Fearon argues that the main benefit provided by imposing rigor on counterfactual reasoning is that it ‘forces one to be clearer about, or to “unpack”, the nature of the explanatory exercise one is engaged in’. Whether or not one uses counterfactual arguments in their research, they may be used as an exercise to better understand the causal arguments made in our studies of political phenomena. This is not only useful for academics when designing or assessing research but, also, for policymakers who can utilize counterfactual reasoning to better understand the arguments they propose, or are presented with, in their work. Another application for counterfactuals is offered by Weber (Chapter 12) who argues they should be used as a theory-generating device. While he is skeptical of using counterfactuals as a tool for theory testing, he sees counterfactuals as a way to ‘open minds, raise tough questions about what we think we know, and to suggest unfamiliar or uncomfortable arguments that we had best consider’. While data is a scarce resource that some may try to alleviate through counterfactuals, Weber argues that theories, arguments and ideas are also a scarce resource and counterfactuals can be a useful way to generate them.

As this review illustrates, the problems associated with the use of counterfactuals have not disappeared. On the contrary, this volume brings to light additional obstacles to the inclusion of counterfactual reasoning as an accepted methodology for the social sciences. What the contributors have done is identify the problems that need to be overcome and have started us along the path towards their resolution by providing clear methodological criteria for using counterfactuals. In sum, the contributors have presented a convincing argument for the continued study and use of counterfactual arguments for the examination of world politics. We agree with the editors when they say ‘...we conclude that disciplined use of counterfactuals – grounded in explicit standards of evidence and proof – can be enlightening in specific historical, theoretical, and policy settings. And that, we suspect, is the most important lesson of this book’ (p. 38).