Introduction for Intelligence Analysis and Policy Making
INTRODUCTION
INTELLIGENCE ANALYSIS is essential to the development and implementation of optimal foreign, defense, and national security policy (Jervis 2010; Fingar 2011). Yet its performance—the quality of the analysis and its ability to inform and shape policy making—outside the US context has been neglected by scholars. In Canada, several federal government departments have intelligence analysis units, but how their products are used by policy makers is poorly understood—often even by the analysts themselves. In this context, our book addresses these important questions: How well does intelligence analysis support policy making in Canada, and how can it be improved? What lessons does the Canadian experience hold for other intelligence communities?
This research will be of interest in and beyond Canada. In Canada, there is no book available for the thousands of members of the intelligence and national security community or for scholars and students interested in the role of intelligence analysis in federal policy making. Moreover, Canada is a close intelligence partner for the US, and a member of the most important intelligence-sharing partnership in the world, the Five Eyes (Australia, Canada, New Zealand, the UK, and the US). This book, as such, represents the first comprehensive empirical study on this topic and an important case study for the field of intelligence studies by proposing an analysis based on the Canadian case but relevant to other national contexts.
Scope of the Book: Intelligence Analysis and Policy
It is important, first, to delineate the scope of this book. Our research focuses on how intelligence analysis supports high-level foreign, national security, and defense policy making in Canada. Our parameters exclude important areas such as intelligence support to law enforcement or to military operations.1 We also do not study the multiple operational aspects of the intelligence machinery, especially not the collection of various types of intelligence.
This makes our purpose quite specific: analysis represents a small portion of the intelligence enterprise, as far more human and financial resources are devoted to collection and operations. Yet much of the interactions of senior policy makers with the intelligence world, in the bureaucracy and in the political realm, focus on analytical products. Analysis is thus far more important in terms of high-level policy making than its limited budget suggests.
This raises a first, essential question: What is intelligence analysis? There are, of course, multiple definitions, most of which are US-centric. Jack Davis, a former Central Intelligence Agency analyst who wrote considerably on the topic, defines it as such: “The mission of intelligence analysts is to apply in-depth substantive expertise, all-source information, and tough-minded tradecraft to produce assessments that provide distinctive value-added to policy clients’ efforts to protect and advance US security interests” (2015, 124). According to another definition by Adda Bozeman, intelligence analysis should “facilitate the steady pursuit of long-range policy objectives even as it also provides guidance in the choice of tactically adroit ad hoc responses to particular occurrences in foreign affairs” (1992, 2).
Narrowing the scope of these broad definitions, Loch Johnson and James Wirtz explain that intelligence analysis, at the strategic level, “contributes to the processes, products and organizations used by senior officials to create and implement national foreign and defense policies. Strategic intelligence thus provides warning of immediate threats to vital national security interests and assesses long-term trends of interest to senior government officials. Strategic intelligence is of political importance because it can shape the course and conduct of US policy” (Johnson and Wirtz 2004, 2).
In this book, we define intelligence analysis as the process whereby analysts gather and synthesize data obtained through a wide range of sources, both unclassified (open source) and classified, and present their findings in a range of products, written and oral, disseminated across government and with international partners. The core value-added of the work of intelligence analysts is to extract insights from the available data. These insights aim to be not merely interesting for the consumers of their written and oral products but also useful to their work in developing and implementing foreign, defense, and national security policy (see George and Bruce 2008; Marrin 2011; and Lowenthal 2017 for comparable definitions).
The second element in framing the scope of our research, after defining intelligence analysis, is to emphasize that the focus here is not on tradecraft (i.e., the methodology of intelligence analysis) or subject matter expertise (the need for analysts to acquire a body of empirical knowledge on the area of their work), the two dimensions typically receiving the most attention in intelligence studies. Instead, we focus on a third dimension: the relevance of the work of analysts, or how it is integrated into broader policymaking processes. Indeed, as the conclusion to what is probably the most important textbook on US intelligence analysis clearly states, “Analysts are of no use to their official customers by being merely the smartest and best-informed experts on a topic. They must address the policy needs of whoever now sits in the White House, executive departments, and congressional offices” (George and James 2008, 304). This cannot be emphasized enough and is the central pillar upon which this book is based. As the chapter on intelligence analysis of the 2005 report by the Silbermann-Robb Commission (often known as the WMD Commission) argues, “The best intelligence in the world is worthless unless it is effectively and accurately communicated to those who need it” (Silberman and Robb 2005, 26).
The third element, and arguably the most difficult, is to define success as it pertains to the impact of intelligence analysis on clients. There is no consensus among scholars or practitioners on how to define, let alone measure, success (Marrin 2012a). At the most general level, analytical units perform optimally when they generate insights that help clients develop and implement better policy. Successful analysis educates and enlightens policy makers and allows them to better do their work (Ghez and Treverton 2017). Kovacs eschews such ambitious goals and suggests more modestly that analysis is “useful” when it “makes a difference—either by bringing about a change in policy or decisions, or by supporting an existing policy which was about to be changed” (1997, 148). Similarly, Betts argues that the “most useful analysis . . . is that which helps to make policy something other than it would have been in the absence of the analysis” (1988, 187). This is a useful starting point, but it remains general and can carry different implications.
Some analytical products focus on a narrow topic, aimed at only one or a few clients, for a specific purpose (a yes or no decision). Others are produced for a more general audience, with a broader (and difficult to measure) objective, such as educating this readership on the topic in question, without necessarily directly supporting a particular decision process. Therefore, to succeed, an analytical product can aim to achieve a range of outcomes, including description, explanation, prediction, and achievement of policy leverage. For example, if there is a crisis in Venezuela, what do policy makers expect in terms of analytical support? Depending on their mandate and the circumstances, they might require a product, written or oral, that describes or explains the crisis (what are its drivers and consequences), makes predictions (where is it heading), or supports the development of policy leverage (what can Canada do, what are the consequences of adopting certain courses of action).
As many interviewees highlighted during the course of our research, it is difficult to measure how intelligence analysis supports policy making, in Canada or elsewhere. Senior policy makers are rarely willing to provide the detailed feedback to analytical units that would be necessary to rigorously determine whether their requirements are fulfilled. In the words of one interviewee, “You know when you fail, but you don’t know when you succeed.”2
In the Canadian context, moreover, senior policy makers rarely perceive that they have a strong need for intelligence, in the sense of having a clear, well-defined intelligence question to which they require a precise answer, which they will then knowingly and actively incorporate into their decision-making process. More often than not, as we discuss throughout the book, intelligence tends to fall more into the “nice-to-know” or “interesting” category of inputs into their work, and less frequently into the “need-to-know” or “need-to-have” category.3 In the words of one interviewee, “Many requests for briefings we get are from senior policy makers who are interested in an issue and want context, but they rarely have a direct, precise need. . . . We are often told, ‘I learned a lot,’ but rarely are we told, ‘I needed that.’”
In this context, it is challenging for intelligence analysis units to produce timely, well-informed, rigorous, and most importantly relevant analysis. That is, analysis answering the appropriate policy questions, in the right format, at the right time, and for the right people. These insights should be accurate, timely, forward-looking, and help avoid unwanted surprise (Betts and Mahnken 2003). To achieve this, analytical organizations must foster the development of subject matter expertise and rigorous tradecraft skills, which they often do, but they must also develop a deep understanding of the policy process—which they often fail to do (Juneau 2017). Indeed, as will be discussed in this book, this third element is easier said than done; it is at this level that the intelligence community in Canada faces some of its most important challenges.
Finally, the book covers a period starting with the attacks of September 11, 2001 until 2019. We chose this starting point because there is a clear “before and after”: intelligence’s role suddenly increased as it assumed a more prominent place in Canadian national security policy deliberations. Client expectations increased, in the realm of counterterrorism and in foreign policy making more generally. We conducted our research in 2018 and 2019, and we wrote the book in 2019 and 2020. The COVID-19 pandemic that started in 2020 serves as a useful cutoff point. It has undeniably affected how the intelligence community, at the levels of both analysis and operations, functions. We suspect, in particular, that some of the trends we describe in this book, such as the growth of senior policy makers’ needs for intelligence and enhanced ties between the intelligence community and nontraditional partners, will intensify. We intend to explore this in future projects; without the benefit of more hindsight, it would be difficult to study this systematically.
Methodology
Our research relied on multiple data-gathering methods. First, we conducted sixty-eight interviews with serving and retired national security practitioners, mostly from Canada but approximately 15 percent from allied countries. Within Canada, interviewees were individuals with experience on the analytical (as analysts or managers) and the consumer side (policy makers using intelligence analysis). Interviewees ranged from desk officers and analysts all the way to deputy ministers (equivalent to deputy secretaries in the US) and heads of agencies. We also interviewed senior political advisers to current and past ministers and prime ministers. Finally, we also interviewed serving and retired practitioners in the US, which allowed us to better understand the Canada-US intelligence relationship, on which Canada is so dependent, and to draw some limited comparisons between the Canadian and US experiences. Our interviewees thus represent a broad sample, both in the nature of their work and their level in the hierarchy.
While interviews represent the main source of primary and original data, we also rely on other sources: mining of official government documents (e.g., the annual Report on Plans and Priorities, parliamentary testimonies, and reports by review and oversight bodies), media reports, conferences and conversations held under the Chatham House rule, and (limited) academic literature on the topic in the Canadian context. Moreover, senior officials in the Canadian intelligence community provided us with declassified internal reports on intelligence policy relations.
Our own experience—we worked in the national security community for a combined total of fourteen years prior to joining academia—also guided our research. One of the two authors (Juneau) has also been actively involved with the Canadian intelligence community since he left government, notably to provide training on strategic intelligence support to policy making. This experience allows him to continuously explore, through what amounts to participant observation, intelligence policy dynamics in Canada.
Throughout the book, we highlight similarities and differences between the roles, missions, and procedures of Canada’s intelligence analysis community and that of its main counterparts. Comparisons with the US are possible, but differences in size and structure imply that the analytical function in Canada must position itself differently to optimally support clients. Comparisons with other countries are thus useful: the Australian and British intelligence communities (and those of other democracies to a lesser extent) are closer to the Canadian one in size and structure. In this context, our study of the Canadian case positions us to offer lessons that are generalizable to other national contexts.
Finally, we wish to highlight the limitations of the book. Researching the work of any country’s intelligence community is inherently challenging, since there is little publicly available information. If conducting such research is a challenge in the US, it is an even greater problem in Canada, where there is less publicly available information about the national security and intelligence community. That said, we believe we have been able to mitigate these challenges. Our book focuses on the process and machinery of intelligence analysis and its interaction with the policy world, not on the substance of the analysis. Some specific aspects of the machinery of the intelligence policy interface must remain classified, but as many of our interviewees emphasized, much can be said without revealing information that should remain away from the public eye. Certainly, more can be said about structure and machinery than about the content of the analysis, much of which is classified. We also do not study other aspects of intelligence that need to remain highly inaccessible, especially on the operational side. Even though we recognize that our research can simply not reveal the complete picture, we are confident that we uncovered sufficient information, especially through interviews, to provide an overview of the evolution of the relationship between intelligence analysis and policy making in Canada since 2001.
Interviews: Approach and Context
We based the recruitment of participants on our preexisting contacts and on “snowball sampling,” where existing study subjects assist in the recruitment of further interviewees. We targeted individuals who had experience producing, consuming, and/or managing intelligence analysis; many had experience in two or all three of these areas. Upon accepting an invitation, we sent interviewees a list of questions in advance that formed the roadmap for a semistructured interview. We conducted interviews between October 2018 and May 2019 in Ottawa, in a range of locations, including government offices, our offices at the University of Ottawa and Carleton University, public spaces (such as restaurants), and over the phone. We also conducted sixteen interviews with Canadians and Americans in Washington, DC, in May 2019. Most interviews lasted between sixty and ninety minutes, although some went longer.
Ultimately, we were pleasantly surprised with the access we achieved. Our requests were seldom turned down, and individuals often proved very helpful in providing further contact information, if needed. The atmosphere of most interviews was relaxed, with several interviewees stating that it had been a rare opportunity for them to think of “big picture” questions. Others expressed that they felt the interview was a kind of “therapy” to try to work out some of the issues they had been mulling over the course of their career.
Unsurprisingly given the subject matter, interviewees agreed to speak on the basis of strict anonymity. As such, we never identify interviewees, either by name or place of work. Even where interviewees (mostly retired individuals) agreed to go on record, we have chosen to keep their comments anonymous, for the sake of consistency and also on the grounds that the Canadian intelligence and national security community is so small that the identification of one person might accidentally lead to the identification of another, if only by process of elimination.
A note on context is also necessary, to understand the environment in which interviews took place and the scope of what our interviewees could speak to. We conducted our interviews during one of the greatest periods of change in the Canadian national security and intelligence community in terms of its legislative framework, the evolving threat environment, and expectations of transparency.
In June 2017, the government of Canada introduced sweeping legislative changes now known as the 2017 National Security Act.4 This legislation overhauled national security and intelligence oversight and review mechanisms with a view to enhancing accountability and, to a certain extent, transparency. Notably, departments with national security responsibilities that had never been subject to comparable external scrutiny before (Global Affairs Canada and the Department of National Defence / Canadian Armed Forces in particular) are now subject to review and oversight by a range of bodies. When we conducted our interviews, however, these changes were still in their infancy and their full impact was not yet known.
The new legislation granted the Communications Security Establishment new powers to engage in “active” (offensive) and “defensive” cyber activities and the Canadian Security Intelligence Service (CSIS) the ability to legally obtain, store, and use datasets. Even though core mandates and functions within the intelligence community largely remain unchanged, it was clear that many interviewees were unsure about what enhanced review and oversight would mean for their organization, including for analytical units.
Moreover, these legislative changes came at a time when the threat environment was fluid. This period continued to see concern over threats to democratic institutions and electoral meddling from malicious state actors, especially heading toward a Canadian federal election in October 2019. In addition, there were rising concerns about foreign state and state-championed investment in sensitive economic sectors. The growing salience of these issues came in the aftermath of the emergence of the threat from extremist travelers / foreign fighters and returnees following the rise and decline of the Islamic State in Iraq and Syria (Canadian Security Intelligence Service 2018).
While these threats had existed in some fashion for decades, their new manifestations required the readjustment of tools and mindsets as well as growing cooperation with nontraditional partners such as Elections Canada, Passport Canada, and, increasingly, Innovation, Science, and Economic Development Canada (formerly Industry Canada). This was yet another series of novel steps the intelligence and national security community had been taking only relatively recently, as we conducted our interviews.
Finally, these changes were taking place at a time when Canadian citizens and the media were demanding greater transparency and accountability of the intelligence community. Since the massive leak of information about the US intelligence community and its allies (including Canada) by Edward Snowden in 2013, concerns over the activities and oversight of intelligence services in Western countries have increasingly turned into demands for more transparency about what these agencies and departments are doing in the name of national security. Canada has been no exception. There has been pressure exerted by the Trudeau government, coordinated through Public Safety Canada, to improve how the intelligence community engages and informs Canadians about what it does. This includes the establishment of the National Security Transparency Advisory Group in 2019, cochaired by one of the authors (Juneau)—although this occurred immediately after we completed our interviews (Public Safety Canada 2019).
In this sense it is important to appreciate that our interviews took place during a period of reform, transition, and uncertainty for the intelligence community. While we believe our data to be broadly accurate, our findings must also be understood as providing an overview of key trends affecting the intelligence/policymaking nexus up to 2019. Indeed, the community continues to evolve as this book goes to press, out of a need to respond to legislative reforms, evolving threats, and, of course, its own desire to improve.
Key Findings
Overall, our research identified three broad conclusions on the performance of intelligence analysis in Canada since 2001.
Scope for improvement. Interviewees differed on the details, but a clear overall consensus emerged. The performance of the Canadian intelligence analysis community in supporting policy making has improved since 2001 but remains far from optimal; it has gone, roughly, from poor to satisfactory. In the 2000s, the world of intelligence analysis in Canada was largely disconnected from strategic-level policy making. Analysts and their managers had few connections with their clients, while the latter rarely asked for it or perceived a benefit from using it. By the end of the 2010s, the situation had evolved in a generally positive direction: though there remains much scope for further improvement, analytical units have become better connected to the wider policymaking process. Yet these improvements are uneven, with interviewees describing pockets of excellence, pockets of mediocrity, and everything in between. As many interviewees noted, the community has “consistently underperformed,” too often contenting itself with “muddling through.”
Structural barriers to improvement. Because Canada is a relatively safe country, policy makers often believe that they have a limited need for intelligence, especially because of Canada’s geographic location and proximity to the US. Moreover, Canada has traditionally been able to neglect security and intelligence issues because the costs of missteps are usually low. As a result, Canada’s national security culture is immature and unsophisticated in comparison to its allies. This implies, in particular, that even though there remains scope for much improvement, further improving the community’s performance will be difficult.
Electroshocks. Much of the improvement of recent years appears to be the result of what we call electroshocks to the system. These result from the sudden emergence of a new threat that the national security and intelligence community is initially poorly set up to address, forcing it to adapt. Importantly, these shocks are different from intelligence failures and strategic surprises (typically the source of major reform in other national contexts). They did not lead to sudden, important negative consequences like the 9/11 attacks in the US. In the Canadian experience, we identified four such post-9/11 shocks: war in Afghanistan (2005–2011), foreign investment by adversarial states (2012), emergence of foreign fighters (2013), and rise of online threats to democratic institutions (2016). Such events steadily increased the incentive for consumers to take intelligence analysis more seriously, a reflex that most senior members of the Canadian national security policy and diplomatic communities have not traditionally strongly had. These events also drive the analytical community to “get its act together,” in the words of one interviewee. We also fully expect that with hindsight, we will eventually view the COVID-19 pandemic—which falls outside the scope of this book, since it is still ongoing as we finish writing it—as another such shock.
Overview of the Book
The book proceeds on the assumption that four elements are required for a successful intelligence policy dynamic: overall governance and structure of the community; management of individual analytical organizations; management of the interface between intelligence analysis and policy; and approaches to analysis. Using this framework, the following four chapters assess the performance of intelligence analysis in Canada. They identify its main strengths and flaws and propose explanations. They also each conclude with a case study.
Chapter 1 provides an overview of the Canadian intelligence and national security community. It highlights some of the characteristics that drive the nature of the community’s internal dynamics. Weak central leadership, in particular, implies that personalities are especially important in the Canadian system relative to our allies. Next, the chapter looks at some of the structural factors that impact the performance of the analytical community: poorly defined niches and mandates, a lack of a foreign human intelligence service, and uncertainty regarding the impact of changes related to enhanced oversight and review. The chapter concludes with a case study of the Privy Council Office’s Intelligence Assessment Secretariat.
Chapter 2 examines in detail how the intelligence community manages its analytical units. It highlights the importance of everyday bureaucratic practices to a successful intelligence policy dynamic, including hiring, training, offering exchanges and secondments, and defining career paths. Our findings build on the assessments of chapter 1: given the decentralized nature of the intelligence and national security community and the importance of personalities, we find that the management practices of analytical units lack standards in terms of hiring, training, and defining career paths. We then offer a case study of the Intelligence Assessment Branch of CSIS at the end of the chapter.
Chapter 3 focuses on the cultural factors that impact the management of intelligence policy dynamics. This includes the relatively low intelligence and policy literacy that pervades the policy and intelligence communities respectively, challenges with the intelligence priorities process, different approaches to client relations, and continued problems with silos between the various intelligence analysis bodies. We also explain here why there is no hard politicization of intelligence analysis in Canada—contrary to the experience in the US. The chapter ends with a case study of the relationship between Global Affairs Canada and intelligence matters.
Chapter 4 presents the findings of our research on what makes an intelligence product useful, including format and responding to government priorities, and what kinds of analysis are desirable. The chapter also examines the tendency to overclassify in the Canadian system, which has a serious impact on the ability to share and use intelligence in policy making. The Canadian Forces Intelligence Command within the Department of National Defence / Canadian Armed Forces is the concluding case study.
Chapter 5 provides recommendations to address the problems and challenges identified throughout the book. We examine what can be done to remedy or mitigate certain weaknesses and propose measures that the intelligence community can take to improve its performance. Moreover, we examine how this can be accomplished in an era requiring enhanced transparency amid a series of fluid national security challenges.
Finally, the conclusion offers a synthesis of the book’s key themes. It begins with a discussion of whether we can truly say there is a Canadian intelligence and national security community, and what the experience of this “community” says about the role of expert communities more broadly in government. It also reflects on what the findings of the book suggest for how international structure shapes outcomes within a government context. And, the conclusion examines challenges ahead for the Canadian intelligence and national security community and its analytical bodies: What happens if Trumpism continues to deteriorate the foundations upon which Canada’s national security has rested upon for decades? How should the community focus its evolution? Can Canada draw upon the experiences of its allies? And, can the intelligence community manage relations with nontraditional partners when addressing unconventional threats when it struggles to maintain relationships within itself?
We also offer, in a short appendix at the end of the book, an overview of the main organizations that form the Canadian national security and intelligence community. Readers unaccustomed to its structure and the mandates of its key players may wish to read it before moving on to chapter 1.
Notes
1. In this context, the book does not focus much on the Royal Canadian Mounted Police (RCMP), where the bulk of the analytical work supports law enforcement operations. To be clear, this does not imply that there is no strategic analysis within the RCMP: analytical work to support operations does not have to be solely tactical and operational in its nature. In addition, the RCMP has, in recent years, boosted its strategic analysis capacity to support policy making within the force. This analysis, however, has by its nature a limited impact outside the RCMP and, as such, largely remains outside the scope of this book. (Interestingly, the RCMP has chosen to colocate its strategic policy and strategic analysis teams to ensure the close integration of their work.)
2. This difficulty in measuring success is also highlighted by Canada’s National Security and Intelligence Committee of Parliamentarians in its Annual Report 2018 (2019, 51). See the report for details on a draft framework developed by the Privy Council Office (but eventually discarded as too unwieldy) intended to measure how effectively the intelligence community was delivering on the government’s intelligence priorities.
3. As one official reviewing this chapter pointed out, this is largely true in the policymaking context but not in an operational one.
4. For an early explanation of the changes in Bill C-59, which created the act, see Forcese and Roach (2017).