Strategic Capability Roadmap Version 1 0 Analytic Framework Gary Christopher DRDC DGMPRA Debbie Blakeney Roman Petryk Ben Taylor Leonard Kerzner Van Fong Mark Ball DRDC CORA Andrew Beard DSTL-London UK exchange DRDC CORA TR 2009-013 December 2009 Defence R D Canada Operational Research Division Strategic Planning Operational Research Team Chief Force Development Strategic Capability Roadmap Version 1 0 Analytical Framework Gary Christopher DRDC DGMPRA Debbie Blakeney Roman Petryk Ben Taylor Leonard Kerzner Van Fong Mark Ball DRDC CORA Andrew Beard DSTL-London UK Exchange Defence R D Canada – CORA Technical Report DRDC CORA TR 2009-013 December 2009 Principal Authors G L Christopher D E Blakeney TL Workforce Modelling and Analysis DGMPRA Strategic Planning Operational Research DRDC CORA Approved by Original signed by Charles Morrisey Charles Morrisey Section Head Joint and Common Approved for release by Original signed by Dale Reding Dale Reding Chief Scientist Defence R D Canada – Centre for Operational Research and Analysis CORA © Her Majesty the Queen in Right of Canada as represented by the Minister of National Defence 2009 © Sa Majesté la Reine en droit du Canada telle que représentée par le ministre de la Défense nationale 2009 Abstract …… In 2005 the Chief of Defence Staff of the Canadian Forces CF mandated that Capability-Based Planning be institutionalized as a part of a centrally driven top-down approach to Force Development FD within the Department of National Defence Consequently over the last three years military and defence analyst staff have developed and implemented the first version of a Canadian end-to-end capability-based FD process Scenarios derived from policy and strategic guidance capture the scope and scale of potential operations in which the CF could participate During the Capability Planning Process scenarios are analyzed to define capability requirements The Capability Management Process evaluates current and projected force structures of the CF against those capability requirements to identify adequacies deficiencies and surpluses Through the Capability Integration Process potential solutions alternatives for the deficiencies are identified Finally optimization methods are employed to determine the best set of alternatives affordable within the available budget to maximize CF capability The results form the Strategic Capability Roadmap a 20-year plan for CF capability development The Capability Planning Capability Management and Capability Integration processes are supported by a set of dedicated analysis tools which collectively have come to be referred to as the analytic framework This report documents these analysis tools and processes of the analytic framework employed to produce the first Strategic Capability Roadmap Résumé … En 2005 le Chef d’état-major de la Défense des Forces canadiennes FC a rendu obligatoire l’institutionnalisation de la planification fondée sur les capacités dans le cadre d’une démarche centralisée et de haut en bas de développement des forces DF au sein du ministère de la Défense nationale Par conséquent au cours des trois dernières années le personnel militaire et les analystes de défense ont élaboré et mis en place la première version canadienne d’un processus de DF fondé sur les capacités d’un bout à l’autre Les scénarios qui s’inspirent de la politique gouvernementale et des orientations stratégiques rendent bien compte de la portée et de l’échelle des opérations auxquelles pourraient être appelées à participer les FC Dans la première partie du processus planification des capacités on analyse des scénarios pour définir les objectifs en matière de capacités Dans la deuxième partie gestion des capacités on évalue les capacités actuelles et projetées des FC en regard des objectifs définis dans la première partie afin de cerner les domaines où il y a des ressources en quantité suffisante en quantité insuffisante ou en quantité excédentaire Dans la troisième partie intégration des capacités on établit des solutions de rechange visant à remédier aux insuffisances décelées au cours des étapes précédentes Enfin à l’aide de méthodes d’optimisation on détermine l’ensemble de solutions qui permettront le mieux de maximiser les capacités des FC dans les limites d’un budget établi Ce processus de DF aboutira à la publication de la Feuille de route des capacités stratégiques un plan de développement des capacités des FC pour les 20 prochaines années Les trois parties du processus planification gestion et intégration reposent sur un ensemble d’outils d’analyse spécialisés que l’on désigne maintenant comme le cadre d’analyse Ce rapport rend compte des outils d’analyse et des processus qui ont servi à produire la première feuille de route des capacités stratégiques DRDC CORA TR 2009-013 i ii This page intentionally left blank DRDC CORA TR 2009 013 Executive summary Strategic Capability Roadmap Version 1 0 - Analytical Framework Gary Christopher Debbie Blakeney Roman Petryk Ben Taylor Leonard Kerzner Andrew Beard Van Fong Mark Ball DRDC CORA TR 2009-XX Defence R D Canada – CORA In 2005 the Chief of the Defence Staff CDS of the Canadian Forces CF mandated that Capability-Based Planning CBP be institutionalized as a part of a centrally driven top-down approach to Force Development FD within the Department of National Defence DND Consequently for the last three years military and defence analyst staff have developed and implemented the first iteration of an end-to-end capability-based FD process The FD process ultimately produces a scheduled list of programs necessary to meet the demands of the future security environment over the next 20 years This capability plan is formally referred to as the Strategic Capability Roadmap SCR The Canadian DND FD process is an integrated sequence of activities starting from government strategic guidance and delivering force elements for employment by Operational Commands The primary analysis phases of the FD process implemented in DND consists of three parts • Capability Planning What the CF needs to be able to do • Capability Management How well the CF will be able to meet its requirements • Capability Integration How the CF’s plans should change to better meet its requirements In the Capability Planning process scenarios derived from government policy are analysed to determine the CF’s Capability Goals the quality and quantity of capability required to conduct both domestic and international operations The Capability Planning process consists of five steps as shown in Figure E-1 Scenario Analysis Steps Key Output Scenario X from Scenario Set 1 Mission Analysis Concept of Operations for Scenario X 2 Validate Content of Framework List of Relevant Activities for Scenario X 3 Score Activities Priority List of Activities for Scenario X 4 Set Measures of Capability MoC MoC MoC Answers for Scenario X 5 Capability Goal Document Prep Capability Goals for Scenario X Figure E-1 Capability Planning Process Steps DRDC CORA TR 2009-013 iii In the first Capability Planning process step mission analysis is carried out on a single scenario from the FD scenario set Through a modified version of the Operational Planning Process mission analysis is performed on the chosen scenario to produce multiple CF Courses of Action COAs that could address the scenario’s threat One CF COA is approved by the Capability Development Board and consequently becomes the CF’s Concept of Operations for the scenario The Concept of Operations is used to validate the content of the capability framework consisting of 16 capabilities Each capability is decomposed into a three-tiered hierarchy of functions and activities The activities assigned to a capability create an exhaustive list of all the actions required to achieve that capability effect in any scenario in the FD scenario set Not every activity associated with a capability would be relevant for each FD scenario In the second step to validate the content of the capability framework for the chosen scenario the activities that would be utilized for the scenario are identified and subsequently prioritized through a top-down risk assessment procedure In the third step an analysis tool CATCAM facilitates the prioritization of activities by first weighting the scenario’s mission effects then scoring the activities against the mission effects in terms of frequency and consequence to produce an overall numerical score for each activity In the fourth step of the Capability Planning process a series of Measures of Capability MoC questions is used to quantify and qualify the mission-specific attributes of each capability Each capability has a unique set of MoC questions that must be answered in the context of the given FD scenario Each MoC question is mapped to one or more associated activities A Capability Goal for a particular FD scenario refers to the answers to its MoC questions and the priority list of its activities The final step of the Capability Planning process documents the capability goals in a series of capability reports In the Capability Management process CF capability current and projected is assessed against the defined capability goals using the ForGE tool Each MoC is assessed independently with the proportion of MoCs satisfied providing a measure of CF capability status CF capability status is presented in the Capability Outlook Figure E-2 while the operational implications are evaluated via a likelihood-of-success scale and displayed through the Risk Outlook Figure E-3 Identified capability deficiencies are assessed a mission-value score based on the activities they impact and the activity scores determined during Capability Planning The ANDREW tool facilitates the determination of the mission-value scores and the ranking of the capability deficiencies The Capability Outlook Risk Outlook and ranked set of capability deficiencies are the outputs of the Capability Management Process iv DRDC CORA TR 2009-013 Figure E-2 Capability Outlook Figure E-3 Capability Outlook In the Capability Integration process multiple capability alternatives are postulated for each deficiency Alternatives can involve the acquisition of new equipment changes to tactics techniques and procedures upgrading current equipment employing new concepts of operation assigning a new role to existing equipment or combinations of these options Alternatives can completely or partially resolve a deficiency and can vary widely in resource demands Each capability alternative is fully defined in terms of the cost lifecycle personnel requirements deficiency closure and implementation risk The final and arguably most challenging activity of the Capability Integration process is the determination of the optimal set of capability alternatives that will maximize CF capability while remaining affordable within the constraints of the defence budget An optimization model was developed using Phoenix Integration software which attempted to maximize the value of deficiencies resolved maximize compliance with the Objective Force a concept for the ideal future CF minimize additional personnel requirements minimize cost and minimize DRDC CORA TR 2009-013 v implementation risk The optimization model employed a genetic algorithm to search the solution space to identify the solution that best met the goals of the optimization while abiding by the imposed constraints Over 35 thousand potential solutions were examined to identify the “best” set of capability alternatives for development Once the best set of alternatives was identified mathematical models were applied to assess the level of risk associated with the implementation of the capability solution and explore potential development schedules The prioritized list of capability alternative components projects identified in the optimal solution has been staffed through senior management boards On 16 July 2008 the Defence Management Committee endorsed the project listing of the Strategic Capability Roadmap 1 0 The SCR represents DND's first 20-year Force Development plan The SCR itself along with the tools developed to support the CBP process have been used widely since then to support Departmental investment decisions SCR 1 0 was one of the key inputs perhaps the key input used to feed the Investment Plan specifically in terms of the projects that would be funded and the timeframe that those projects would receive funding The SCR and the analysis tools are being applied in investment trade-off decisions required to maintain the Investment Plan when project cost increases or implementation schedule changes occur In the future projects will be accepted into the Investment Plan on the basis of their assessment under this Capability-Based Planning framework rather than on the strength of stand-alone operational requirements arguments This confirms the future role of centralized joint Force Development in DND Notwithstanding the success of the newly-developed DND CBP process with the analytic framework that has been accepted and adopted as the Force Development process for the CF improvements are warranted In the haste to produce the first SCR some important factors could not be catered for to the extent they warrant In addition the degree of integration among the tools within the analytic framework could be enhanced Work is currently underway in the Strategic Planning Operational Research Team and Chief of Force Development to deal with these issues before the initiation of the next SCR currently planned to begin in 2010 vi DRDC CORA TR 2009-013 Sommaire Strategic Capability Roadmap Version 1 0 - Analytical Framework Gary Christopher Debbie Blakeney Roman Petryk Ben Taylor Leonard Kerzner Andrew Beard Van Fong Mark Ball DRDC CORA TR 2009-XX R D pour la défense Canada – CORA En 2005 le Chef d’état-major de la Défense CEMD des Forces canadiennes FC a rendu obligatoire l’institutionnalisation de la planification fondée sur les capacités PFC dans le cadre d’une démarche centralisée et de haut en bas de développement des forces DF au sein du ministère de la Défense nationale MDN Par conséquent depuis les trois dernières années le personnel militaire et les analystes de défense ont élaboré et mis en place la première version d’un processus de DF fondé sur les capacités d’un bout à l’autre Ce processus de DF produira comme résultat final une liste planifiée des programmes nécessaires pour rencontrer les exigences du futur dans le contexte de la sécurité de l’environnement pour les 20 prochaines années Ce plan des capacités est désigné formellement comme la feuille de route des capacités stratégiques FRCS Le processus de développement des FC du MDN est une suite intégrée d’opérations débutant par l’orientation stratégique du gouvernement et fournissant des éléments de force destinés à être employés par les commandements opérationnels La phase d’analyse principale du processus de DF qui est mis en œuvre au MDN compte trois parties • Planification des capacités Ce que les FC doivent être capables de faire • Gestion des capacités À quel point les FC pourront satisfaire aux exigences • Intégration des capacités Comment les plans des FC doivent être modifiés pour mieux rencontrer ces exigences Dans le processus de planification des capacités les scénarios tirés de la politique gouvernementale sont analysés pour définir les objectifs en matière de capacités la qualité et la quantité des capacités nécessaires pour mener les opérations au pays et sur la scène internationale Le processus comporte cinq étapes comme le montre la Figure E-1 DRDC CORA TR 2009-013 vii Étapes de l’analyse des scénarios Principaux résultats Scénario X du jeu de scénarios 1 Analyse de la mission 2 Validation du contenu du cadre 3 Cotation des tâches 4 Établissement des critères de mesure des capacités 5 Prép du doc sur les obj en matière de capacités Concept d’opérations pour le scénario X Liste des tâches pertinentes pour le scénario X Liste des tâches prioritaires pour le scénario X Critères de mesure des capacités – Scénario X Objectifs en matière de capacités pour le scénario X Figure E-1 Étapes du processus de planification des capacités À la première étape de la planification des capacités une analyse de la mission est effectuée sur un seul scénario tiré du jeu de scénarios de DF Grâce à une version modifiée du processus de planification opérationnelle on soumet le scénario choisi à l’analyse de la mission pour produire de multiples plans d’action des FC qui visent à enrayer la menace contenue dans le scénario Le Conseil de développement des capacités approuve un de ces plans d’action qui devient par conséquent le concept d’opérations des FC pour le scénario en question On se sert du concept d’opérations pour valider le contenu du cadre des capacités qui comprend 16 capacités Chaque capacité est décomposée en trois niveaux hiérarchiques des fonctions et des tâches Les tâches liées à une capacité constituent une liste complète de toutes les mesures nécessaires pour obtenir l’effet indiqué dans n’importe quel scénario du jeu de scénarios de DF Toutes les tâches liées à une capacité ne sont pas pertinentes dans chaque scénario de DF Afin de valider le contenu du cadre des capacités pour le scénario choisi on désigne les tâches auxquelles on recourrait pour réaliser ce dernier puis on les classe par ordre de priorité au moyen d’une évaluation descendante des risques propres aux tâches À la troisième étape de la planification des capacités l’outil d’analyse EACMEC Équipe d’action 3 du CEMD – Méthodologie d’évaluation des capacités facilite la hiérarchisation des tâches en permettant tout d’abord de pondérer les effets de la mission prévue dans le scénario puis de noter les tâches en fonction des effets de la mission au regard de la fréquence et du niveau de conséquence de manière à produire une cote numérique globale pour chaque tâche À la quatrième étape de la planification des capacités on répond à une série de questions dites « critères de mesure des capacités » CMC pour quantifier et qualifier les caractéristiques d’une capacité dans le cadre d’une mission particulière À chaque capacité correspond un jeu unique de questions CMC auxquelles il faut répondre dans le contexte d’un scénario de DF donné Chaque question CMC est mise en rapport avec une ou plusieurs des tâches associées à la capacité Un objectif en matière des capacités établi pour un scénario de DF particulier renvoie aux réponses aux questions CMC pertinentes et à la liste des tâches prioritaires connexes La dernière étape du processus de planification des capacités consiste à produire une série de documents sur les objectifs en matière de capacités Dans le processus de gestion des capacités on évalue les capacités ― actuelles et projetées ― des FC en regard des objectifs définis lors de la planification en se servant de l’outil de mise sur pied viii DRDC CORA TR 2009-013 de la force et d’évaluation MPFE Chaque CMC est évalué de façon indépendante et on jauge l’état des capacités des FC d’après la proportion de critères satisfaits La Figure E-2 Aperçu des capacités décrit cet état tandis que la Figure E-3 Aperçu des risques expose le risque opérationnel qui découle de l’Aperçu des capacités et qui est évalué au moyen d’une échelle de probabilité de réussite des missions On attribue une cote à chaque insuffisance de capacités repérée en fonction des tâches sur lesquelles cette lacune a une incidence et des cotes calculées pour ces tâches durant le processus de planification L’outil ANDREW facilite le calcul des cotes et le classement des insuffisances de capacités À l’issue du processus de gestion des capacités on dispose donc d’un aperçu des capacités d’un aperçu des risques et d’un classement des insuffisances de capacités Capacité et objectif des forces Éléments de la structure des forces AGIR Production des effets aérospatiaux Production des effets terrestres Production des effets maritimes Opérations spéciales Opérations non cinétiques Effets particuliers APERÇU GLOBAL DU DOMAINE AGIR COMMANDER Aide au commandement Communications Ciblage des effets interarmées APERÇU GLOBAL DU DOMAINE COMMANDER DÉTECTER Renseignement Surveillance et reconnaissance n tio t a r n t e us ill em D’ eul s APERÇU GLOBAL DU DOMAINE DÉTECTER PROTÉGER Protection des forces APERÇU GLOBAL DU DOMAINE PROTÉGER MAINTENIR EN PUISSANCE Maintien en puissance Services de soutien Mouvement Mise en service et désactivation dans le théâtre APERÇU GLOBAL DU DOMAINE MAINTENIR EN PUISSANCE Figure E-2 Aperçu des capacités Stabilisation au Canada sur le continent Contrôle au Canada sur le continent Stabilisation scène internationale Contrôle international Niveau intraétatique n ti o ra nt t e us ill em D’ eul s Contrôle international Niveau interétatique Figure E-3 Aperçu des risques DRDC CORA TR 2009-013 ix Dans le processus d’intégration des capacités on pose plusieurs options de capacités pour chaque insuffisance par exemple acquisition de nouveau matériel modification de la tactique des méthodes et des procédures mise à niveau du matériel existant utilisation de nouveaux concepts d’opérations affectation du matériel existant à de nouveaux usages ou une combinaison de ces options Les options peuvent servir à corriger entièrement ou partiellement une lacune et peuvent requérir des besoins en ressources fort différents de l’une à l’autre Chaque option de capacités est exposée clairement suivant plusieurs critères coût cycle de vie besoins en personnel résolution de l’insuffisance et risque associé à la mise en œuvre La dernière étape du processus d’intégration des capacités ― et la plus exigeante sans doute ― est celle qui consiste à déterminer quel sera l’ensemble optimal d’options qui permettront de maximiser les capacités des FC à un coût abordable dans les limites du budget de la défense On a élaboré à cette fin un modèle d’optimisation au moyen du logiciel de Phoenix Integration qui visait à maximiser la valeur des insuffisances résolues et le degré de conformité au projet de Force objective un concept désignant l’état idéal des FC dans l’avenir et à limiter au minimum les besoins additionnels en personnel le coût et le risque associé à la mise en œuvre Ce modèle d’optimisation utilise un algorithme génétique qui recherche dans l’espace-solutions celle qui satisfait le mieux les objectifs de l’optimisation tout en respectant les contraintes imposées On a donc examiné plus de 35 000 solutions possibles afin de déterminer le « meilleur » ensemble d’options de capacités pour le développement des forces Une fois cette étape franchie on a eu recours à des modèles mathématiques pour évaluer le niveau de risque associé à la mise en œuvre de la solution choisie et étudier des plans potentiels de développement La liste des options de capacités projets retenues dans la solution optimale et classées par ordre de priorité a été proposée par l’intermédiaire des conseils de la haute direction Le 16 juillet 2008 le Comité de gestion de la Défense a approuvé la liste de projets de la Feuille de route des capacités stratégiques FRCS - Version 1 0 qui est le premier plan de développement des forces sur un horizon de 20 ans au MDN Depuis cette date on a souvent recours à la FRCS ainsi qu’aux outils qui ont été élaborés pour les besoins du processus de planification fondée sur les capacités pour éclairer les décisions d’investissement du Ministère La FRCS - Version 1 0 est l’une des principales ressources disponibles sinon la principale ressource disponible servant à nourrir le Plan d’investissement plus particulièrement à déterminer les projets qui devraient être financés et le calendrier de financement La FRCS et les outils d’analyse servent en outre à éclairer les décisions visant à maintenir le Plan d’investissement lorsque le coût des projets augmente ou que le calendrier de mise en œuvre est modifié Dans l’avenir on déterminera les projets à inclure dans le Plan de financement selon l’évaluation qui en aura été faite dans le cadre du processus de planification fondée sur les capacités plutôt que selon la valeur des arguments indépendants relatifs aux besoins opérationnels Cela confirme le rôle futur d’une autorité centrale interarmées de développement des forces au MDN Malgré que le processus de planification fondée sur les capacités élaboré récemment au MDN soit maintenant reconnu comme le processus de développement des forces au Canada des améliorations restent à faire Dans la précipitation à créer la première version de la FRCS on n’a pas accordé toute l’attention qu’il aurait fallu à certains aspects majeurs De plus il y aurait lieu d’accroître le degré d’intégration des outils au sein du cadre d’analyse L’Unité de recherche opérationnelle en planification stratégique et le Chef – Développement des forces s’appliquent actuellement à résoudre ces questions avant la mise en chantier de la prochaine version de la FRCS qui doit avoir lieu en 2010 x DRDC CORA TR 2009-013 Table of Contents Abstract …… i Résumé … i Executive summary iii Sommaire vii Table of Contents xi List of Figures xiii List of Tables xiv Acknowledgements xv 1 Introduction 1 1 1 Background 1 1 2 CBP Process 2 1 3 CF Force Development Process 4 1 4 Strategic Capability Roadmap 5 1 5 Aim and Scope 6 2 Capability Planning Process 7 2 1 Overview of Capability Planning Process 7 2 2 Mission Analysis 8 2 3 Validation of the Capability Framework 8 2 4 Prioritization of Activities 10 2 5 Setting Measures of Capability 13 2 6 Capability Goal Documents 13 2 7 Mission Effect Documents 14 2 8 Force Goals 14 3 Capability Management Process 15 3 1 Overview of the Capability Management Process 15 3 2 Evaluation of Capability Status 15 3 2 1 Assessment Criteria 15 3 2 2 Force Structure Elements 17 3 2 3 Force Generation and Evaluation Tool 17 3 3 Capability Outlook 22 3 4 Risk Outlook 24 3 5 Identification of Capability Deficiencies 27 3 6 Ranking Capability Deficiencies 28 4 Capability Integration Process 32 4 1 Capability Integration Process Overview 32 DRDC CORA TR 2009-013 xi 4 2 4 3 4 4 4 5 4 6 4 7 4 8 4 9 4 10 4 11 4 12 Identification of Alternatives 32 Alternative Cost 33 Deficiency Closure 33 Development Risk 34 Objective Force Compliance 36 Alternative-Deficiency Interdependence 38 Available Funding 39 Identifying the Best Set of Alternatives 40 Program Financial Risk 46 Creation of “The List” 49 Scheduling 50 5 Conclusion 52 6 The Way Ahead 54 References 56 Annex A Capability Alternatives Form 59 Annex B Estimating Indirect Costs of Alternatives 61 Annex C SCR Optimization Model 63 C 1 Figure of Merit 63 C 2 Number of Personnel 64 C 3 Cost 65 C 4 Risk 65 C 5 Objective Force Compliance 66 C 6 Integration of Models and Optimization of Solutions 67 C 6 1 Integration of Models to Compute a Single Solution 67 C 6 2 Constrained Multi-objective Optimization of Solutions 68 List of symbols abbreviations acronyms initialisms 75 Distribution list 76 xii DRDC CORA TR 2009-013 List of Figures Figure E-1 Capability Planning Process Steps iii Figure E-2 Capability Outlook v Figure E-3 Capability Outlook v Figure 1 Red Team Capability-Based Planning Process 3 Figure 2 Force Development Framework 4 Figure 3 Force Development Scenarios 5 Figure 4 Capability Planning Process 7 Figure 5 CATCAM Tool with Sample Data 10 Figure 6 Mission Effect Frequency and Consequence Definitions 11 Figure 7 Frequency Consequence Score Matrix 11 Figure 8 Activity Frequency and Consequence Definitions 12 Figure 9 Activity Score Calculation 13 Figure 10 ForGE Tool Main Interface 19 Figure 11 ForGE Capability Assessment Interface 20 Figure 12 Sample ForGE Results 21 Figure 13 Capability Outlook – Partial View 23 Figure 14 Capability Outlook – Consolidated View 24 Figure 15 Risk Outlook – Classes of Operations 26 Figure 16 Risk Outlook 27 Figure 17 The ANDREW Tool with Sample Data 30 Figure 18 Sample Capability Deficiency List with Mission-Value Scores 31 Figure 19 Sample Phoenix SCR Output 42 Figure 20 Phoenix Results for FOM Cost and Personnel 43 Figure 21 Phoenix Results Constrained for Cost and Personnel 44 Figure 22 Optimal Solution Objective Force Compliance and Risk 45 Figure 23 Alternative Cost Probability Models 47 Figure 24 SCR Solution Financial Risk 48 Figure 25 SCR Solution Cumulative Risk Profile 49 Figure C-1 Process Diagram for a Single Solution 68 Figure C-2 Process Diagram for Optimization using the Genetic Algorithm 74 DRDC CORA TR 2009-013 xiii List of Tables Table 1 Capabilities by Domain 9 Table 2 Aerospace Effects Production Capability Functions Activities 9 Table 3 Example Measures of Capability Questions 13 Table 4 Sample of Capability Evaluation Criteria 16 Table 5 Force Structure Elements 18 Table 6 Risk Outlook Assessment Scale 25 Table 7 Capability Deficiency Template 28 Table 8 Example Capability Deficiencies 28 Table 9 Deficiency Activity-Impact Assessment Scale 29 Table 10 Deficiency Closure Values 34 Table 11 Capability Alternative Risk Assessment Model 35 Table 12 Objective Force Compliance Factors 37 Table 13 Alternative-Deficiency Dependence Scale 38 Table 14 Cost Uncertainty Scale 46 xiv DRDC CORA TR 2009-013 Acknowledgements Development and employment of the analysis framework for the Strategic Capability Roadmap SCR was a joint effort between the Operational Research analysts of Defence R D Canada on the SCR team and the military members of the Directorates of Capability Planning and Military Capability Management within Chief of Force Development The analysts developed the initial concepts for the framework tools which were subsequently refined in a multi-disciplinary working group setting The final result was truly a product of collaboration The authors would be remiss if the contribution of the former members of the Strategic Planning Operational Research Team and the Directorate of Defence Analysis were not also acknowledged This group of analysts and military staff officers too numerous to name individually developed many of the preliminary foundational strategic planning concepts components that eventually came to be recognized as capability-based planning The SCR team built upon many of these earlier developments to construct the detailed capabilitybased planning process of the Department of National Defence employed in the production of the first Strategic Capability Roadmap DRDC CORA TR 2009-013 xv This page intentionally left blank xvi DRDC CORA TR 2009 013 1 Introduction 1 1 Background Upon becoming Chief of the Defence Staff CDS in the spring of 2005 General Rick Hillier set up four Action Teams to determine what changes were required in the areas of Command and Control Force Generation Operational Capability and Institutional Alignment to achieve fundamental transformation of the Canadian Forces CF 1 The key intent behind establishing these CDS Action Teams CATs was to accelerate transformation of the CF towards becoming a more relevant more responsive more effective force to provide leadership and response to threats to Canada and around the world The CATs were given three months to conduct their assessment and provide recommendations The mandate of CAT 3 Operational Capability was to identify the set of affordable sustainable capabilities that should be resident within the CF The initial plan of CAT 3 was to review strategic direction conduct a capability gap analysis and provide force structure recommendations that would enable changes needed to achieve transformation and implement the vision of the newly published Defence Policy Statement 2 The CAT 3 Team decided to employ the Capability-Based Planning CBP methodology to complete its task due to the comprehensive and objective nature of the methodology 3 Although the strategic planning division Director General Strategic Planning within the Department of National Defence had adopted the CBP approach for centralized force structure planning prior to the establishment of the CAT 3 Team only a few limited tools had been developed to assist deriving capability requirements based on strategic guidance The CAT 3 Team quickly realized that additional refinements and tools would be required to carry out their assignment They set out to develop these tools and produced planning scenarios of hypothetical CF operations and a capability assessment model that allowed capability requirements to be defined and compared to one another 3 4 The CAT 3 Team also engaged a “Red Team” from the Defence R D Canada Centre for Operational Research Analysis to review and validate their proposed capability assessment process and tools 5 The Red Team provided an independent endorsement of the analysis process advocated by CAT 3 As the CAT 3 activity progressed it became clear that there was a substantial development effort required to establish the analysis process to assess CF capability requirements As it became obvious that developing the methodology and applying it to produce a result would not be possible within the timeframe provided to the CATs the focus and objective for CAT 3 shifted from producing a comprehensive assessment of CF capability requirements to advancing the development of the process and tools for capability assessment The final recommendation from the CAT 3 work was to adopt the CBP process as the basis for the CF Force Development FD planning process and to continue to develop the tools to support it 4 Following the conclusion of the CAT 3 work the CDS mandated that CBP be institutionalized as a part of a centrally driven top-down approach to FD within the Department of National Defence DND 6 Consequently over the last three years military and defence analyst staff have developed and executed the first iteration of an end-to-end capability-based FD process This FD DRDC CORA TR 2009-013 1 process ultimately produced a recommended list of programs necessary to meet the demands of the future security environment over the next 20 years 7 This capability plan was used to inform the production of the Departmental Investment Plan 1 2 CBP Process One of the most complete and thorough descriptions of the CBP process was provided by Technical Panel 3 Joint Concepts and Analysis of the Joint Systems and Analysis Group of The Technical Cooperation Program TTCP 7 1 However as thorough as this description is it speaks mostly about the concept of the process and provides only a general description of the different steps and activities that constitute the CBP process This TTCP manuscript does not provide a detailed step-by-step manual or procedure for the application of the CBP methodology Recognizing that the TTCP CBP process must be adapted to accommodate national planning procedures and resources the CAT 3 Red Team attempted to modify the process to fit DND requirements and constraints The Red Team also attempted to extend the explanation of the CBP steps to provide greater detail of the activities involved and the supporting tools needed although this expanded description still stopped short of a detailed step-by-step manual for Canadian CBP A graphical representation of the Red Team CBP process for DND is shown in Figure 1 It was used as the starting point to define the CBP FD process ultimately employed by DND and the CF to create the first version of the SCR In simple terms the CBP process begins with a review of strategic guidance documents and policies to produce a set of planning scenarios that form the basis for capability evaluation Each planning scenario is evaluated to determine the collection of capabilities capability goals required by the CF to fulfill their operational mission in the context of the scenario After the complete set of capability goals across all scenarios has been established the ability of the CF force structure to provide the capabilities is assessed This assessment may lead to the identification of capability deficiencies and capability surpluses Following a determination of the resources available for capability transformation alternatives to address the capability deficiencies are identified Finally investment optimization is performed to identify the most effective capability investments that can be made under the constrained resources to produce the most effective CF These results are encapsulated into a Capability Investment Plan Subsequently all possible changes to the planning conditions policies security environment resources technology advances etc are examined and the CBP process cycle is repeated While the CBP cycle is presented as a sequence of steps and activities that are carried out in a consecutive order there are numerous opportunities for revisiting earlier actions revising results and proceeding forward again As the ultimate result of the CBP process is an optimized force structure ideally affordable and sustainable there are numerous points in the CBP cycle where it may become evident that capability requirements and or available funding will preclude defining a force structure that can be produced and sustained within available resources When this expected situation becomes clear one should revisit earlier steps in the process which define constraints and conditions that drive capability requirements and subsequently the unaffordable 1 The Technical Cooperation Program is a formal program of information-sharing and collaboration in defence research and development matters between the governments of Canada the United States the United Kingdom Australia and New Zealand 2 DRDC CORA TR 2009-013 force structure One should examine the types of operations in which the forces will operate the roles and responsibilities of the forces within these operations concept of operation etc in an effort to relax requirements that in turn will reduce development and maintenance costs to allow the new resultant force structure to be affordable 0 1 Government Guidance Defence Priorities 2 Define set of Planning Scenarios 3 Identify Tasks and assess relative importance 4 Identify Capabilities and assess relative importance 5 Map assess CF Force Structure elements to required capabilities 6 Specify Capability Goals 7 Determine Concurrent and Recurrent Mission Demands 13 Assess Changes to 1 Government policy Defence Priorities 2 Threats Potential Operating Theatres 3 Technology Advances 4 Concepts of Operation 5 CF Capability deterioration 8 Evaluate Current CF force structure against capability goals and mission demands Identify capability mismatches Outputs 1 Prioritized Capability Requirements 2 Linkage between Capabilities and Missions 3 CF Capability mismatches balances Direction could be provided 9 Assess funding for new investment 10 Identify feasible Options to fill Gaps 11 Conduct Balance of Investment Analysis 12 Define Capability Development Plan Figure 1 Red Team Capability-Based Planning Process DRDC CORA TR 2009-013 3 1 3 CF Force Development Process The Canadian FD process as envisioned by Chief of Force Development CFD is illustrated in Figure 2 The Figure identifies where the CBP process fits within the broader DND FD process This schematic diagram shows an integrated process starting from government strategic guidance and delivering force elements for employment by Operational Commands It is recognized that there are important feedback loops with policy and direction being informed by the art of the possible While work in many if not all areas goes on continuously the key products such as the Strategic Capability Roadmap SCR and the Investment Plan IP will be formally generated every three years The first versions were produced in 2008 with a second edition of the SCR planned for 2010 along with a new IP Thereafter work will follow three year cycles Capability Employment Operational Commands Advice ReRe-Assess Strategic Guidance Capability Based Planning Key Products Foreign Policy Defence Planning Directives Future Security Analysis Capability Planning Management and Integration Defence Policy Advice Effects Resources Defence Capability Plan CF Campaign Plan Investment Plan Business Plan Outputs Joint Navy Strategic Capability Roadmap SCR Army Air Force ReRe-Assess The output from CBP informs Defence Policy and assists senior decision makers engaged in both short-term Horizon 1 and longer range Horizon 2-3 capability development Figure 2 Force Development Framework Within the defined FD process the CBP sub-process is represented by two aggregated activities Future Security Analysis and Capability Planning Management Integration Future Security Analysis takes strategic guidance provided by Defence Foreign Policy and other Defence Planning Directives and translates this guidance into a set of discrete planning scenarios that provide the foundation for the analysis of capability requirements Following on the work of the CAT 3 Team the Directorate of Future Security Analysis in CFD produced 18 FD scenarios 9 spanning the spectrum of conflict as shown in Figure 3 4 DRDC CORA TR 2009-013 Figure 3 Force Development Scenarios The second activity Capability Planning Management Integration in the CBP component of the FD process is the primary analysis phase of the process This analysis phase of the FD process consists of three distinct parts • Capability Planning What the CF needs to be able to do • Capability Management How well the CF can will be able to meet its requirements • Capability Integration How the CF should change to better meet its requirements In Capability Planning the scenarios are evaluated to determine the capability requirements or goals of the CF Capability Management assesses the current and planned CF force structure’s ability to meet the capability requirements The output of this part of the analysis is a capability status report culminating in the Capability and Risk Outlook Lastly Capability Integration examines possible Force Development alternatives new equipment changes to tactics techniques and procedures or new concepts of operation to address identified shortfalls in capability and determines the “best” set of alternatives for the CF to adopt to maximize their operational capability 10 1 4 Strategic Capability Roadmap On 25 October 2007 the Vice Chief of the Defence Staff formally tasked CFD to produce the first version of a Strategic Capability Roadmap with a target delivery date of July 2008 12 Up to that point in time development of the procedures and tools for the CBP process for DND had been progressing at a steady determined pace The scenario analysis process and tools had been exercised and refined through the capability goals assessment of eight scenarios The steps of the DRDC CORA TR 2009-013 5 military capability management process had been identified and the Directorate of Military Capability Management had just been stood up the prior summer Work was underway to extend the level of detail of the Capability Management process to allow it to be executable by the Force Development community and the needed tools to support the process were being built Development of the Capability Management and Integration processes with accompanying analysis tools was not expected to reach completion before summer 2008 With the initiation of the SCR tasking the pace of development greatly accelerated Not only did the development to the process and tools need to be completed ahead of the original schedule but the process would have to be executed in earnest with participation from the broad CF FD community with sufficient time to allow results to be evaluated by DND senior management This presented an extreme challenge to CBP process developers and necessitated an extreme reduction in the scope scale and sophistication of the analytic methods and models Only the essential analysis components crucial to ensure objectivity and rigour to the process would be adopted and assembled for this first SCR 1 5 Aim and Scope In the course of producing the first SCR the Operational Research team developed a collection of analytical tools and methods to support the Capability Planning Management and Integration phases of the process The aim of this report is to document in detail the concepts and functions of this suite of tools that provided the analytical framework for the SCR The report will cover the analysis procedures and tools starting from the evaluation of the Force Development Scenarios to the determination of the optimal capability alternatives set for CF Force Development i e the conclusion of the Capability Integration activity The actual results of the SCR Version 1 0 will not be described in this report as the emphasis is on the analytical underpinnings of the SCR The SCR Version 1 0 results are available from the Director of Capability Management and are recorded in the minutes of the Joint Capability Requirements Board meetings of July 2009 7 6 DRDC CORA TR 2009-013 2 Capability Planning Process 2 1 Overview of Capability Planning Process In the Capability Planning process firstly scenarios from DND’s FD scenario set are analysed to determine CF Capability Goals Capability Goals describe the variety and quantity of capability required for an individual scenario Once all or a selected subset of scenarios from the 18 FD scenario set have been analysed the Capability Goals derived for each scenario are aggregated to create Force Goals It is the Force Goals that articulate ‘what’ the CF must be able to do to meet the demands of the future security environment To determine Capability Goals for an individual FD scenario the Directorate of Capability Planning D Cap P within the CFD organization leads a Joint Capability Planning Team JCPT through a six-step scenario analysis process shown in Figure 4 JCPTs are formed for the analysis of each individual scenario and they consist of Subject Matter Experts SMEs from the Level 1 organizations across the DND CF Figure 4 Capability Planning Process An overview of each of the six steps Figure 4 for creating Capability Goals for an individual scenario is provided in the following sections of this report At the end of this chapter the process for determining Force Goals is described A more in-depth description of the entire Capability Planning process is provided in 11 Note that due to time constraints the production of SCR v1 0 was based on the analysis of only eight out of the 18 FD scenarios Of the eight analysed scenarios three were Domestic Continental scenarios and five were International scenarios However when the eight analysed scenarios were mapped to nine broad scenario classes i e Domestic Disaster Relief Domestic Asymmetric Threat Domestic Sovereignty Assertion Continental Defence DRDC CORA TR 2009-013 7 International Failed State Assistance International Stabilization International Counterinsurgency and Major Regional Conflict and Baseline Commitments it was found that all but two scenario classes namely Domestic Asymmetric Threat and Baseline Commitments were covered Consequently the corresponding Force Goals were considered by CFD to provide the ‘80% solution’ and were deemed sufficient for the purposes of creating the first version of the SCR 2 2 Mission Analysis In the first step of the scenario analysis process Figure 4 mission analysis is carried out on a single scenario from the FD scenario set Through a modified version of the Operational Planning Process 13 the JCPT identifies multiple CF Courses of Action COAs for the chosen scenario that would address in varying degrees the scenario’s threat One CF COA is approved by the Capability Development Board and consequently becomes the impetus behind the CF’s Concept of Operations for the scenario 2 2 3 Validation of the Capability Framework In Step 2 of the scenario analysis process Figure 4 the Concept of Operations is used to validate the content of the capability framework The capability framework was developed by JCPTs through scenario analysis 11 It consists of 16 capabilities which are shown in Table 1 categorized by capability domain The capabilities in the Command Sense Shield Sustain and Force Generation domains are considered “enabling” capabilities while the capabilities in the Act domain are considered “act” capabilities Enabling capabilities provide support to act capabilities that produce direct effects within operations Each capability in the framework is decomposed into a three-tiered hierarchy of functions activities and example activities For example the functions activities and example activities of the Aerospace Effects Production capability are provided in Table 2 Functions are the components that make up a capability while activities are the set of actions that make up a function and example activities are examples of “how” the activities can be achieved by the CF presently The complete capability framework is provided in Annex A of 10 The activities assigned to a capability create an exhaustive list of all the actions required to achieve that capability effect in any scenario in the FD scenario set Consequently not every activity associated with a capability would be relevant for each FD scenario To validate the content of the capability framework for the chosen scenario the JCPT identifies the example activities that would be utilized for the scenario assuming the Concept of Operations determined in Step 1 An activity is considered to be relevant to the FD scenario if at least one of its corresponding example activities is deemed relevant to the FD scenario 2 The Capability Development Board CDB consists of representatives at the Director General level from across DND The mandate of the CDB is to assist CFD in formulating decisions direction and guidance and recommendations pertaining to his role as the central FD authority for the Canadian Forces 8 DRDC CORA TR 2009-013 Table 1 Capabilities by Domain Domain Capability Command Command Support Communications Joint Effects Targeting Intelligence Surveillance Reconnaissance Aerospace Effects Production Land Effects Production Maritime Effects Production Special Ops Effects Production Non-Kinetic Effects Production Force Protection Sustainment Support Services Movements Theatre Activation Deactivation Force Generation Sense Act Shield Sustain Generate Table 2 Aerospace Effects Production Capability Functions Activities and Example Activities Capability Aerospace Effects Production Functions Activities Example Activities Conduct Air Intercept Defend Friendly Aerospace Deny Aerospace Conduct Defensive Counter Air to the Opposing Conduct Ground Based Air Defence Defeat OPFOR Aerospace Force OPFOR Conduct Anti-Air Warfare Assets Conduct Fighter Sweep Provide Freedom Provide Aerospace Control Combine Forces for Ops of Manoeuvre in Conduct Combined Air Operations the Aerospace Conduct Supression of Enemy Air Defence Destroy or Suppress Conduct Covert Operations OPFOR Aerospace Assets Conduct suppression of Surface-to-Air and Surface-to-Air on the Ground or at Sea Missile threats Conduct Offensive Counter Air Conduct Air Escort Protect Own Aerospace Conduct Combat Air Patrol Assets Monitor Aerospace DRDC CORA TR 2009-013 9 2 4 Prioritization of Activities In Step 3 of the scenario analysis process the relevant pan-capability activities as determined in previous step are prioritized through a top-down risk assessment of activities against the desired mission effects of the given FD scenario The CDS Action Team 3 Capability Assessment Methodology CATCAM tool facilitates the prioritization of activities by first assigning weights to the scenario’s mission effects then scoring the activities against the mission effects to produce an overall numerical score for each activity 3 The overall numerical score for each activity represents its relative importance in the scenario The standardized mission effects used in the Capability Planning process are Control Shape Stabilize Shield Project and Sustain and Informed Direction which were derived through the spiral development of the methodology A portion of the CATCAM tool with sample data is provided in Figure 5 Capability Surveillance Reconnaissance Functions Wide Area Collection Effect Effect Effect Control Informed Direction hh hm Collect Strategic Information eml elm Survey Areas emm hl ehl ehh ell Activities Focused Collection Track Targets of Interest Identify Assess Targets of Interest emh Shield ml hh Figure 5 CATCAM Tool with Sample Data The JCPT weights mission effects in CATCAM by assessing the risk to the success of the scenario mission if the CF could not create the mission effect A mission effect’s risk assessment consists of a frequency and consequence level The definitions of high h medium m and low l mission effect frequency and consequence are given in Figure 6 In Figure 5 the mission effect risk assessments are highlighted in orange A risk assessment of hm is assigned to the Informed Direction mission effect in the Figure which means that it is of high frequency and medium consequence to the scenario mission 3 In support of CAT 3 CATCAM was developed to compare the value of disparate CF capabilities 14 A modified version of the original CATCAM tool is being used in the Capability Planning process 15 10 DRDC CORA TR 2009-013 The CATCAM tool translates the mission effect risk assessments into numerical scores using the frequency consequence score matrix shown in Figure 7 Using the sample CATCAM data from Figure 5 the hm risk assessment assigned to the Informed Direction mission effect would be translated into a numerical score of 0 5 Finally CATCAM calculates a weight for each mission effect with respect the scenario mission by normalizing the mission effect numerical scores The weights of the three mission effects shown in Figure 5 would be Control 0 55 Informed Direction 0 28 and Shield 0 17 Frequency What is the frequency with which this mission effect will be applied within the scenario mission High Medium Low This mission effect will be needed almost always 80% throughout the duration of the scenario mission This mission effect will be needed often 20-80% throughout the duration of the scenario mission This mission effect will be rarely needed 20% throughout the duration of the scenario mission Consequence What is the risk of scenario mission failure if the CF cannot achieve this mission effect High Medium Low hm Scenario mission will fail if this mission effect cannot be achieved Scenario mission will be mission impeded if this missio effect cannot be achieved Scenario mission will suffer minimal impact if this mission effect cannot be achieved High Frequency Effect Medium Consequence Effect Figure 6 Mission Effect Frequency and Consequence Definitions Frequency Consequence Low Medium High High 0 5 0 5 1 0 Medium 0 3 0 3 0 75 0 1 0 3 0 75 Low Figure 7 Frequency Consequence Score Matrix DRDC CORA TR 2009-013 11 Once the mission effects have been assigned a weight the activities are prioritized by assessing what the risk to the success of creating the mission effects would be if the CF could not conduct the activity The “enabling” activities are assessed against all seven mission effects while the “act” activities are only assessed against the Control Shape and Stabilize mission effects When the JCPT determines that an activity is required to achieve a particular mission effect either to do or enable the mission effect a risk assessment is entered in the intersecting box in the CATCAM tool The risk assessment methodology is very similar to that described above for mission effects The definitions of high h medium m and low l activity frequency and consequence are given in Figure 8 Frequency What is the frequency with which this activity will be applied to produce this mission effect High will be used This activity will almost always 80% to produce this mission effect Consequence What is the risk of failing to achieve the mission effect if the CF cannot perform this activity High Medium This activity will be used often 20-80% to produce this mission effect Medium Low This activity will be rarely used 20% to produce this mission effect Low ml Medium Frequency Doer Activity Low Consequence Doer Activity ehm Mission effect cannot be achieved if this activity is not performed Mission effect may still be achieved if this activity is not performed This activity has minimal impact on the achievement of this mission effect High Frequency Enabler Activity Medium Consequence Enabler Activity Figure 8 Activity Frequency and Consequence Definitions In Figure 5 the Survey Areas activity is given an hl high frequency low consequence risk assessment against the Informed Direction mission effect The square is coloured dark purple to indicate that the Survey Areas activity is directly involved in achieving the Informed Direction mission effect A box is coloured light purple and an e is placed in front of the risk assessment when the activity enables the achievement of the mission effect The activity risk assessments are then translated by CATCAM into numerical scores through the frequency consequence score matrix Figure 7 Finally CATCAM calculates an activity’s overall score by taking the weighted sum of its numerical scores with the mission effect weights For example the calculation of the overall score for the Assess Targets of Interest activity from Figure 5 is illustrated in Figure 9 Listing the activities in descending order of their overall scores creates the ranked list of activities for the given scenario Activity scores are documented in the Capability Goal documents for a FD scenario and are ultimately used in the Capability Management process to prioritize capability deficiencies 12 DRDC CORA TR 2009-013 Capability Surveillance Reconnaissance Effect Effect Control Informed Direction Shield 0 55 x 0 75 0 28 x 1 0 17 x 0 Activity Function Identify Effect Assess Targets of Interest 0 41 0 28 0 0 69 Figure 9 Activity Score Calculation 2 5 Setting Measures of Capability In the fourth step of the scenario analysis process Figure 4 the JCPT answers a series of questions designed to quantify and qualify the mission-specific attributes of a capability These questions are referred to as the Measures of Capability MoC questions and were developed by the first JCPTs during the initial scenario analysis workshops Each capability in the framework Table 1 has a unique set of MoC questions that must be answered in the context of the given FD scenario Table 3 provides some examples of MoC questions used in the Capability Planning process The complete set of MoC questions is provided in Annex B of 10 Table 3 Example Measures of Capability Questions Domain Capability MoC Question Sense Surveillance and Reconnaissance Sustain Movements How quickly must Surveillance Reconnaissance assets be cued How long must wide-area surveillance be sustained How far must forces and resources be moved to from theatre within theatre For a given capability each of its MoC questions is mapped to one or more of its associated activities An activity is linked to a MoC question if it quantifies or qualifies an attribute that is relevant to the activity MoC questions and answers are documented in the Capability Goal documents for a FD scenario and are ultimately used in the Capability Management process to identify capability deficiencies 2 6 Capability Goal Documents The fifth step of the scenario analysis process is to create 16 Capability Goal documents for the given FD scenario one for each capability in the capability framework Table 1 A single Capability Goal document consists primarily of the answers to the MoC DRDC CORA TR 2009-013 13 questions for that capability and the priority list of its associated activities for the given FD scenario A Capability Goal document also contains a definition of the capability a listing of the capability’s functions and activities and an overview of the key findings of the scenario analysis with respect to capability requirements Note that for the remainder of this report a Capability Goal for a particular capability and FD scenario is referring to the answers to its MoC questions and the priority list of its activities for one FD scenario 2 7 Mission Effect Documents In the final step of the scenario analysis process Figure 4 seven Mission Effects documents are created A single Mission Effect document consists primarily of the key points that highlight reinforce or illustrate how each applicable capability contributes to the creation of that mission effect and an explanation of how the CF must produce that mission effect The Mission Effect document also provides a definition of the mission effect 2 8 Force Goals After the scenario analysis process described step-by-step above was conducted on eight out of the 18 FD scenarios the results were “rolled-up” to produce Force Goals for the SCR version 1 0 A Force Goal for a specific capability is the aggregate of its Capability Goals answers to MoC questions and priority list of activities across the eight FD scenarios The aggregate answer to a specific MoC question was taken as the “worst case” i e most demanding answer to the MoC question across the eight FD scenarios In this way achieving the Force Goal would ensure that the Capability Goals of the individual scenarios would be satisfied For example for the MoC question see Table 3 How long must wide-area surveillance be sustained the maximum time reported from the eight scenarios was chosen as the aggregate answer The activity priority lists were aggregated by first creating an aggregate Domestic Continental scenario activity priority list the average of the activity scores from the three Domestic Continental scenarios and an aggregate International scenario activity priority list the average of the activity scores from the five International scenarios The overall aggregated priority list of activities was formed by taking the average of the Domestic Continental activity score and the International activity score for each activity A comprehensive description of the methodology used to create Force Goals is provided in 14 Force Goals are ultimately the articulation of ‘what’ the CF must be able to do to meet the demands of the future security environment It is the collection of 16 Force Goals one for each capability in Table 1 that are used in capability assessment within the Capability Management process to identify and prioritize capability deficiencies The capability assessment Capability Management process is described in the next section of this report 14 DRDC CORA TR 2009-013 3 Capability Management Process 3 1 Overview of the Capability Management Process In the Capability Management process current and planned CF capabilities are evaluated against the capability goals established through the Capability Planning process Capability deficiencies are identified the Capability Outlook and Risk Outlook views are produced as capability and operational effectiveness status reports and the relative importance of the capability deficiencies are determined This section will describe the activities and analysis tools employed in carrying out the Capability Management process It is at this juncture in the CBP process where force structure elements are evaluated against the capability goals to determine where capability sufficiency excess and deficiency exist This is the first point in the process where equipment considerations are included in the evaluation activity It is the logical point to begin examining force structure as capabilities are ultimately provided by a combination of equipment personnel and doctrine 3 2 3 2 1 Evaluation of Capability Status Assessment Criteria Capability goals defined through the Capability Planning process provide the basis for evaluating capability status In particular the Force Goals were used for this evaluation in the production of the first version of the SCR Recall that the capability goal documents contain an overview of the capability and associated functions a definition of the capability quantitative and qualitative descriptors derived from the MoC questions and the associated set of prioritized activities The challenge facing the SCR Team was to take this amalgamation of factors defining capability and restructure it in a way that would allow an assessment of the degree to which CF can provide the capability It was decided that the capability assessment criteria would be derived from the set of factors defined by the MoC responses examined in the context of conducting the relevant activities of the capability The MoC responses identified specific capability levels of performance related to reach response speed consequences interoperability survivability etc This combination of capability requirements provided a rich set of evaluation criteria to assess the ability of the CF to provide the capability Table 4 displays a sample of the evaluation criteria used to assess the status of the Aerospace Effects Production capability within the Act domain The number of evaluation criteria employed to assess the status of capability within the CF varied from approximately a dozen criteria for the Act domain capabilities to over two hundred criteria for the Command domain capabilities DRDC CORA TR 2009-013 15 Table 4 Sample of Capability Evaluation Criteria Number 16 MoC Factor 1 Reach 2 Reach 3 Capacity 4 Consequence 5 Consequence 6 Consequence 7 Interoperability 8 Temporal 9 Temporal 10 Survivability 11 Survivability Evaluation Criteria Can the Force Element conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets across the Canadian airspace including air defence identification zone Can the Force Element conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets across an expeditionary JOA Coalition - Canadian lead of 400 km x 400 km or JOA Coalition - Other Nation lead Cdn Specialist Lead 2100 x 1500 km Can the Force Element conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets in support of four concurrent lines of operation within one operation Can the Force Element suppress enemy air defence across all the potential AD systems that could be encountered in the 8 types of operations Can the Force Element conduct Electronic Warfare jamming against the air defence and aerospace systems expected in the 8 types of operation Can the Force Element defeat destroy suppress all OPFOR aerospace assets expected in the 8 types of operations in the air and on the ground Can the Force Element coordinate the conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets with these entities CF OGDs Coalition Forces Host Nations Provincial Municipal police forces NORAD and US Government Can the Force Element begin conducting these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets in an Expeditionary operation within 90 days with an operational response time of hours to days immediate for air traffic control with a tactical response time of seconds to minutes and in a Domestic operation begin conducting these activities within hours Can the Force Element redirect the conduct of these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets in an Expeditionary operation within hours to days at the operational level and seconds to minutes at the tactical level and in a Domestic operations within minutes Can the Force Element conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets in the following environmental conditions temperature range from -60 to 60 degrees Celsius very low to very high humidity hurricane and dust storms in hurricane and dust storms high sea state low no visibility cold sea temperature Sea Level to 60 000 foot altitude and in the following geography harsh deserts urban rural suburban Arctic tundra and archipelago mountainous terrain littoral proximity to int'l and US airspace domestic and international airways as applicable Can the Force Element conduct these activities deny friendly aerospace defeat OPFOR aerospace assets combine forces destroy suppress aerospace assets on ground sea protect own aerospace assets in the following threat conditions Asymmetric threat Small arms IED's technicals suicide bombers terrorists Conventional forces and capabilities CBRN threat and cyber threat as expected in the 8 types of operations DRDC CORA TR 2009-013 3 2 2 Force Structure Elements The second critical input to be defined to allow the capability assessment to be performed is the set of force structure elements A force structure element is a self-sufficient unit within the CF that can be utilized to provide capability within an operation The unit can perform a tactical operational or strategic function The unit specified as a force structure element is the lowest-level unit a fundament asset considered for deployment as part of a force package 16 This level of unit can be considered as a basic building block for assembling a force package for an operation or mission Multiple units may be aggregated to provide a larger force contribution but as a general rule no smaller unit sub-unit or subset than the one specified as a force structure element would be considered for inclusion in a force package To benefit from the results of an earlier effort to identify force elements for capability costing and to promote as much consistency in strategic planning as possible the set of force structure elements was developed from the list utilized in the strategic costing model developed within CFD 17 18 This set of force elements was provided to each Capability Manager as a starting point for discussion Each Capability Manager then adjusted the set to enhance completeness and to better reflect how units within their capability domain were organized for operations There was also a requirement to anticipate new force structure elements that would be created in the future based on acquisition and development plans At any given time there are a wide variety of Force Development plans and proposals promulgated within the Department however only a subset will ever come to fruition To ensure that the set of force structure elements reflected expected reality to a high degree some threshold needed to be created to determine if a proposed force structure element should be included in the evaluation set or not Following a concerted discussion within the SCR Team it was decided that only FD projects that had reached at least the “project identification” phase also referred to as “D” status as defined in the Project Approval Guide 19 would be included in the set of force structure elements for evaluation At this level of approval a deficiency had been formally recognized and a project to deal with the situation has been initiated It was felt that when this level of approval was reached there was sufficient commitment to ensure that the project would be completed The final set of force structure elements used to assess capability status in SCR v1 0 is shown in Table 5 3 2 3 Force Generation and Evaluation Tool The Force Generation and Evaluation ForGE tool extended and expanded upon earlier conceptual work done by the Strategic Planning Operational Research Team Dr Andrew Billyard in developing a method to assess the individual contributions of force structure to providing capability as well as providing an aggregate assessment of the entire CF’s ability to meet all facets of each capability The ForGE tool was developed as a user-friendly spreadsheetbased software application that allows the set of force structure elements to be evaluated against the capability assessment criteria to produce a rating of how completely a capability requirement is satisfied where deficiencies exist where surplus may exist and the strength of each force element’s contribution to the capability DRDC CORA TR 2009-013 17 Table 5 Force Structure Elements No No Force Element 1 Strategic Command Control C2 - HQ 33 Air Expeditionary Support Wing 2 Strategic Communication Information System CIS 34 Maritime Tactical Helo CH124 CH148 3 Operational C2 fixed HQ 35 Tactical Utility Helo CH 146 4 Operational CIS fixed 41 Medium-Heavy Lift Helo 5 Deployable Joint C2 HQ 42 Domestic SAR Rotary 6 Deployable Joint CIS 43 Domestic SAR Fixed Wing 7 Land Tactical C2 CIS 44 Tactical Fighter 8 Naval Tactical C2 CIS 45 9 46 47 11 Air Tactical C2 CIS C2 Communications Computers C4 IntelligenceSurveillance-Reconnaissance ISR Satellite Special Operations Task Force JTF2 CSOR Maritime Joint NBCD Air-Air Refueler Aerospace Management and Control includes CADS and Radars Intelligence-Surveillance-Reconnaissance ISR Fixed Wing 48 Air Mobility CC 144 CC 130J CC 138 12 Disaster Assistance and Response Team 49 Strategic Airlift Transport CC 150 C177 13 Destroyer DDH 50 ISR Unmanned Air Vehicle 14 Frigate FFH 51 Air Demonstration Unit 15 Support Ship AOR JSS 52 Joint Support Group 16 Submarine SSK 53 Field Hospital 17 Coastal Defence Vessel 54 Joint Task Force Support Unit 18 Arctic Offshore Patrol Vessel 55 Field Ambulance 19 Fleet Diving Unit 56 Service Support Unit Land 20 Indirect Fire Regiment 57 Joint Signals Unit 21 Direct Fire Regiment Armoured 58 Forward Logistics Site Support Unit 22 Direct Fire Regiment Air Defence 59 Symmetric Infantry Battalion 10 18 Force Element 23 Light Infantry Battalion 60 Engineer Support Unit 1ESU 24 Motorized Infantry Battalion 61 Psychological Operations Company 25 Armoured Regiment 62 Military Police Unit 26 Armoured Reconnaissance Regiment 63 Port Security Harbour Defence Unit 27 Combat Engineer Regiment 64 Airfield Engineering Squadron 28 Engineer Support Regiment 65 CF Network Operations Centre CFNOC 29 Combat Service Support excluding Field Hospitals Field Ambulance and Land Service Support Unit 66 Cdn Material Support Group CMSG incl Supply Ammo and Workshop Depots 30 Territorial Defence Battalion 67 Civilian-Military Cooperation CIMIC Organization 31 EW Squadron 68 Public Affairs Organization 32 Ranger Patrol Group DRDC CORA TR 2009-013 Figure 10 displays the top-level main user interface of the ForGE tool Through this interface the user identifies the timeframe of interest for the capability assessment cell C1 and the scenario cell C2 that will be used to specify the capability requirements assessment criteria In this implementation of ForGE Scenario 0 indicates that the Force Goals will be used in place of scenario-specific capability requirements The set of capabilities for evaluation are listed in row 4 beginning in cell D4 The set of force elements available to be evaluated for their capability contributions are listed in column B beginning in cell B9 4 ti v s tra u l l I ly e On Figure 10 ForGE Tool Main Interface To conduct an evaluation of the contribution a force structure element provides to a capability one selects the cell at the intersection of the column and row of the capability and force element of interest then clicks on the “Assess Cell” button contained in cell B6 This action opens up the Capability Assessment interface shown in Figure 11 Here the capability and force structure element selected for evaluation are identified along with the relevant set of capability assessment criteria Evaluation results are captured in the cells in column A beginning in cell A13 For each capability assessment criterion the force structure element is assessed as to whether it is able to satisfy the capability criterion or not The force element is assessed as either fully satisfying the criterion input a 1 or not input a 0 No intermediate levels of capability provision are available 4 Note that the ForGE interface window shown in Figure 10 has been scrolled such that columns F through H and rows 9 through 36 are hidden DRDC CORA TR 2009-013 19 in this initial version of the ForGE tool Comments related to the assessment can be captured in column E Here the justification for the assessment and any supporting references can be identified t s tra Il lu i ve O nl y Figure 11 ForGE Capability Assessment Interface The results of the capability assessments are displayed on the main interface of the ForGE tool A sample of results is shown in Figure 12 The “0” and “1” sequences in the intersection cells i e I17 show the results of the capability assessment for the individual force structure elements Each digit in the result sequence indicates whether the corresponding capability criterion can be satisfied by the force element For example cell I43 identifies that there are 12 assessment criteria for the Aerospace Effects Production capability and that the Tactical Fighter force element can address eight of the criteria the first second and sixth through eleventh criteria The tactical Fighter in 2008 is unable to provide the third fourth fifth and twelfth elements criteria of Aerospace Effects Production capability The overall level of contribution from a force element to a capability is indicated in column C The overall contribution level is calculated by determining what proportion of the capability criteria the force element is able to provide If the proportion of capability criteria provided by the force element is 80 percent or higher the contribution is deemed to be “strong” and an “S” is placed in column C If the proportion of criteria satisfied if greater than zero but less than 80 percent this is a “partial” contribution and a “P” is placed in the appropriate cell If a force element does not contribute in any way to the capability the cell in column C is left blank 20 DRDC CORA TR 2009-013 ti v s tra Il lu ly e On Figure 12 Sample ForGE Results The aggregate results of all the force structure elements in the CF to provide the capability are indicated in rows 5 and 6 in the cells below the capability name In row 6 the sequence of numbers indicates the number of force elements that can address each criterion defining the capability Again each position in the sequence refers to a specific capability criterion A “0” in the sequence indicates that no force elements can satisfy that capability criterion This identifies a capability deficiency and the digit in the sequence is coloured red A “1” indicates that there is only one force element able to address the capability criterion and the number is coloured green A “2” or larger number identifies that there are multiple force elements that can provide this element of capability 5 These are coloured blue and could indicate some redundancy or possible excess in the capability The proportion of capability criteria addressed by force elements in the aggregate result defines the overall state of the capability from the perspective of completeness In the case of the Air Effects Production capability 11 of the 12 criteria are addressed so 92% of the elements of the capability are provided by the CF force structure in 2008 The capability is assessed as being 92% complete If 90% or more of the capability is provided the cells with the name of the capability 5 Numbers greater than 9 are aligned vertically with the first digit appearing above the second as shown in cell J6 of Figure 12 i e 10 is displayed with each numeral stacked vertically with the “1” placed above the “0” DRDC CORA TR 2009-013 21 and the percent completion are coloured green indicating that the vast majority of the capability elements are addressed If more than 70% but less than 90% of the capability criteria are provided then the cells are coloured yellow indicating that minor shortfalls exist within the capability If less than 70% of the capability criteria are satisfied then the capability is deemed to have serious deficiencies and the cells are coloured red These thresholds for colour-coding capability status were set by the Chief of Force Development and approved by the Vice Chief of Defence Staff These thresholds are contained within the program code of the model Once all the capabilities have been assessed against the existing force structure elements for the specified time period the results are archived The resultant status of the capabilities represents a capability snapshot for the time period As the SCR was intended to project out 20 years capability snapshots for future years were required The procedure for doing these assessments was to advance the timeframe by one year then consider if there were any changes to the force structure elements such as the introduction of a new force element or the retirement of an existing element or their performance that would alter the provision of capability components defined by the criteria If force structure changes were predicted to occur then the affected force elements and capabilities would be assessed If no changes were expected in a given year no additional assessments were required ForGE results for years where changes occurred would be assessed and archived For example if a new force element is scheduled for delivery in 2010 then the impact of this new force element on the CF’s ability to satisfy the evaluation criteria is assessed in a new ForGE spreadsheet for the year 2010 Potentially this new force element will satisfy some evaluation criteria that were not satisfied in previous years and therefore an identified capability deficiency for 2008 will not exist in 2010 Similarly if a force element is expected to reach its End Life Expectancy without replacement in 2017 a new deficiency might exist starting in 2017 3 3 Capability Outlook While each capability snapshot produced by the ForGE tool provides a wealth of information on the existing or predicted capability status of the CF it does require a concerted effort to acquire a high-level perspective of the overall state and anticipated evolution of CF capability Earlier prototyping work for capability reporting developed a singular chart referred to as the Capability Outlook which could provide this strategic view of CF capability The advance provided by the SCR Operational Research Team was to automate the production of the Capability Outlook from the ForGE results 20 Figure 13 shows a partial view of an example of the Capability Outlook in its expanded form 6 As can be seen from Figure 13 the Capability Outlook is a spreadsheet-based model In the actual spreadsheet there is a control that activates the procedure to populate the chart The procedure basically extracts from each archived file of ForGE results the overall status of each capability which force elements contribute to the capability and strength of the force elements contribution This information is then placed in the Capability Outlook spreadsheet Recalling that ForGE results for future years are only assessed and recorded if changes to force elements occur there will be gaps in capability status in the Outlook when the initial transfer is performed Under the 6 Please note that in Figures where actual results would be considered sensitive or classified illustrative results are presented This is indicated on the applicable Figures 22 DRDC CORA TR 2009-013 assumption that capability remains the same until a change is identified the final step in the procedure is to fill in the gaps in capability status by recording the previous year’s status From the single Capability Outlook chart one can identify which force structure elements contribute to the capability the strength of the contribution partial or strong and the overall completeness or health of the capability Blanks in the contribution lines indicate that the force structure element does not contribute to the capability in that particular year which could be the result of the force element being retired or not yet acquired This information is provided for each capability for each year of the Capability Outlook 20 years in the case of the SCR Capabilities are grouped within their assigned capability domain in the Capability Outlook An overall assessment of the health of the capability domain is also provided by determining the lowest status of all the capabilities in the domain in other words the domain can only be as strong as its weakest capability Figure 13 Capability Outlook – Partial View From the hypothetical results provided in Figure 13 it can be seen that nine force structure elements contribute to the Maritime Effects Production capability One of these force elements the Arctic Offshore Patrol Vessel does not begin contributing to the capability until 2015 suggesting that this system will be acquired and achieve initial operating capability by that year The health of the Maritime Effects Production capability is green in 2008 to 2012 indicating that the capability is at least 90% complete It can also be seen that the strength of the contribution from the Frigate force element diminishes from strong S to partial P in 2012 suggesting performance-reducing changes are expected at that point in the lifecycle of this system The following year the health of Maritime Effects Production changes to yellow 70-90% complete DRDC CORA TR 2009-013 23 and eventually to red 70% complete in 2015 after the contribution of the Destroyers changes from strong to partial Similar trends can be observed in the other capabilities A more consolidated view of the Capability Outlook can be produced by collapsing the results showing which force structure elements contribute to what extent to the capability Figure 14 shows an example of the consolidated view of the Capability Outlook 7 Here only the evolution of the health of the capabilities and domains is evident as indicated by the colour-coding This view provides the most concise picture of the entire status of CF capabilities with the ability to examine the underlying force structure element data producing the assessments Figure 14 Capability Outlook – Consolidated View 3 4 Risk Outlook While the Capability Outlook shows the current and projected state of health of capabilities it does not address the issue of what the operational consequences would be if CF capability were in this state To answer this “so what” question the Risk Outlook was developed The Risk Outlook provides an estimation of the operational risk imposed on the CF as a result of the Capability Outlook The Risk Outlook is produced by examining each Force Development scenario and evaluating year by year the ability of the CF to fulfill their assigned role and responsibilities with capabilities in the state indicated by the Capability Outlook The evaluation takes the form of a risk assessment using the assessment scale shown in Table 6 For simplicity only three possible 7 Note that due to resources limitations the Generate domain and capability was excluded from the analysis for SCR Version 1 0 24 DRDC CORA TR 2009-013 results were used in this first version of the risk assessment The first result identifies that with the given state of CF capabilities the CF is highly likely 90 percent probability or higher to achieve success in their mission within the scenario The second possible result states that the CF would still be likely to achieve mission success but it would be with some difficulty Here probability of success would be in the range of 50 to 90 percent The last possible result is that the CF would be unlikely less than 50 percent probability to succeed in their mission with the identified state of capability Table 6 Risk Outlook Assessment Scale Mission Success is Highly Likely Mission Success will be Impeded threatened but still Likely Mission Success is Unlikely The Risk Outlook is prepared in a working Group setting with SMEs from each of the Capability Domains One scenario is evaluated at a time With the scenario selected the SMEs for each capability domain provide their risk assessment based on the Capability Outlook supported with explanatory justification for each year covered by the Outlook solely from the perspective of their domain With all the capability domain assessments complete the group as a whole consolidates the results to arrive at an overall risk assessment for the scenario across each year in the outlook With the desire to produce a Risk Outlook as succinct and meaningful as possible it was felt that listing the risk assessment results for each of the scenarios individually would be information overload and could mask a true appreciation of the overall operational risk associated with the Capability Outlook It was decided to create mission groupings from similar scenarios and aggregate the risk assessment results Mission classes based on the scenario effects required were defined as shown in Figure 15 The FD scenarios were mapped to these mission classes and the risk assessment results were aggregated to produce the Risk Outlook DRDC CORA TR 2009-013 25 Figure 15 Risk Outlook – Classes of Operations Figure 16 provides an example of the Risk Outlook The results shown are for illustration purposes only and do not necessarily reflect the actual results produced for the SCR The Risk Outlook chart provides a simple easily understood appreciation of the current and future ability of the CF to successfully conduct various types of operations For example from Figure 16 the International Control Inter-State mission category is shown to have unlikely mission success for the entire 20-year period The International Stabilize mission category is shown to have high likelihood of mission success starting in the year 2013 26 DRDC CORA TR 2009-013 Figure 16 Risk Outlook The Risk Outlook combined with the Capability Outlook provide a complete strategic view of the evolving state of the CF from a capability perspective 3 5 Identification of Capability Deficiencies Having assessed the current and future state of CF capability the final activity in the Capability Management process is to formally specify existing and predicted capability deficiencies for corrective action Capability deficiencies were formally identified through capability domain working groups employing the data and results of the Capability Outlook To ensure that capability deficiencies possessed the same type of scope scale description and information a template for specifying capability deficiencies was defined Table 7 displays the template used by the capability domain working groups to identify deficiencies within their domain Three types of deficiencies were defined An “inability” indicated that the CF did not possess any form of the capability A “lack of capacity” denoted that the CF possessed the capability but lacked sufficient quantity to meet the requirements of all individual missions Lastly an “inadequate capability” indicated that some form of the capability existed in the CF but that some elements facets of the capability were missing The component of the capability that was lacking was specified from the set of activities or functions used in Capability Planning If the capability deficiency only manifested itself in certain geographic or environments conditions this was identified The scale or level of operation strategic operational or tactical impacted by the deficiency was specified Finally the timeframe when the deficiency would appear was recorded Deficiencies could DRDC CORA TR 2009-013 27 currently exist in the CF or could occur at some later time due to force elements moving into a new phase of their life cycle for example 8 Table 7 Capability Deficiency Template Some examples of identified capability deficiencies are shown in Table 8 These deficiencies are related to different activities within the capability domains They cover all the different types of capability deficiencies as well as different timeframes from the present to sometime in the future If the timeframe is not identified in the deficiency description the deficiency currently exists Table 8 Example Capability Deficiencies Domain Sustain Act Sense 3 6 Deficiency Insufficient capacity and capability to activate and deactivate a theatre at the operational level Inability to provide continuous end-to-end asset visibility in real-time Insufficient capacity to conduct long range antisubmarine warfare commencing in 2013 Inability to provide sufficient indirect fire support Inadequate capability to conduct Surveillance and Reconnaissance of urban areas Ranking Capability Deficiencies Anticipating that the resources available to DND and the CF would be insufficient to allow all capability deficiencies to be resolved the SCR needed to ensure that the most important deficiencies would be addressed To do this the deficiencies needed to be given some form of score to allow them to be ranked in terms of operational importance Knowing that as part of the 8 Horizon 2 covers the planning period from five to 10 years in the future while Horizon 3 is the period beyond 10 years in the future 28 DRDC CORA TR 2009-013 capability planning scenario assessment process activities related to capabilities had already been assessed and given a priority value based on the CATCAM assessment linking deficiencies to the activities they would impact upon would allow a value score to be calculated In this way capability deficiencies would assume the aggregate value of the activities upon which they impact To properly calculate this aggregate value some assessment of the degree to which the activity is impacted by the capability deficiency must be made Table 9 shows the evaluation scale used for this assessment A simple three-value scale was used based on the frequency that the activity could not be performed due to the capability deficiency under consideration A rating of “high” meant that the activity would almost always greater than 80 percent of the time be prevented from being performed when the deficiency existed “Medium” impact was assigned when the activity could “often” 20-80% of the time not be performed because of the deficiency Lastly an impact rating of “Low” was assessed when the capability deficiency affected the activity but only prevented it from being successfully conducted less than 20 percent of the time i e “rarely” Table 9 Deficiency Activity-Impact Assessment Scale Weight Rating Definition High This Deficiency will almost always 80% of the time prevent this Activity from being successfully performed 0 9 Medium This Deficiency will often 20 - 80% of the time prevent this Activity from being successfully performed 0 5 Low This Deficiency will rarely 20% of the time prevent this Activity from being successfully performed 0 1 The “weight” value associated with the ratings in Table 9 was based on the average probability of occurrence and used as a multiplication weighting factor to adjust the contribution of the activity based on the degree of impact The aggregate score assigned to the capability deficiency was calculated by doing a weighted sum of the activity Activities that were not affected by the capability deficiency were not included in the value calculation The aggregate score was referred to as the “mission-value score” of the capability deficiency To facilitate the determination of the mission value scores for the capability deficiencies a spreadsheet-based tool ANDREW Activity-based Neoteric Deficiency Ranking and Evaluation Workbook was developed Figure 17 displays the ANDREW tool with some sample assessments Capability deficiencies are numbered and listed across the columns at the top while DRDC CORA TR 2009-013 29 the activities are listed down the rows on the left side of the worksheet Activities are grouped by capability and function Only activities affected by the capability deficiency are evaluated Impact ratings H High M Medium or L Low are selected by means of a drop-down menu Note that separate evaluations are done for performing the activity in Domestic Continental “Dom” and in International “Int” operations Hence each capability deficiency receives two missionvalue scores one for Domestic Continental operations and one related to International operations The aggregate mission-value scores of the deficiencies are automatically calculated and displayed in the worksheets labelled “Dom Pri” and “Int Pri” ti v s tra u l l I ly e On Figure 17 The ANDREW Tool with Sample Data Figure 18 displays a sample of the set of capability deficiencies with their mission-value scores for domestic continental operations The deficiencies have been placed in their ranked order based on the mission value score and the titles have been colour-coded according to their associated capability domain The Score Chart column provides a visualization of the relative differences in the mission value scores since the difference in rank between two deficiencies is not necessarily proportional to their difference in score In the production of the first SCR approximately 87 capability deficiencies were identified for the 2008-2028 timeframe The mission-value-score ranking of deficiencies was presented to FD SMEs in multiple working group meetings to confirm validate the position of deficiencies within ranked listing 30 DRDC CORA TR 2009-013 t s tra Il lu i ve O nl y Figure 18 Sample Capability Deficiency List with Mission-Value Scores Recognizing that there was some uncertainty associated with the mission value scores it was felt unreasonable to suggest that each deficiency could be exactly positioned within the ranked list such that differences of one-hundredth of a point could decide the position within the ranking For the final ranking it was proposed that deficiencies should be grouped into sets with similar mission value and all deficiencies within the same set would be treated as equally ranked The statistical technique known as univariate clustering was applied to the ranking data Univariate clustering assigns the elements deficiencies of the set among a defined number of clusters such that the sum of the variances of each cluster is minimized Using this technique one chooses the number of clusters desired and the elements are assigned to the clusters placing the most similar valued elements in the same cluster For the SCR 10 clusters were used 9 The average mission value score for each cluster was then assigned to all the deficiencies within the cluster 9 The number of clusters chosen was an arbitrary decision based on the expectation of having eight to 10 deficiencies in each cluster DRDC CORA TR 2009-013 31 4 Capability Integration Process 4 1 Capability Integration Process Overview This final phase of the CBP process takes the ranked capability deficiencies from the preceding process as input and seeks to identify the best set of capability investment alternatives to resolve the deficiencies to the greatest extent possible Given the limitation of a fixed annual budget this is a multi-dimensional constrained optimization problem Once the preferred set of capability alternatives is identified program risk must be estimated and an implementation schedule must be defined The output of the process forms the results of the SCR and is utilized as one of the key inputs to develop the Departmental Investment Plan 4 2 Identification of Alternatives Capability alternatives are proposed courses of action to address capability deficiencies Capability Managers within the CFD organization were responsible for generating alternatives for the capability deficiencies in their assigned domains Typically three or more alternatives were developed for each deficiency Alternatives could be the acquisition of new equipment systems changes in operating procedures concepts of operation doctrine or tactics assigning a new role to an existing system or upgrading an existing system Alternatives could also include pursuing research to develop new technology Alternatives did not need to fully address the capability deficiency but could simply be potential solutions that reduced the magnitude of the deficiency The limited time allowed for this activity in the production of the first SCR meant that most alternatives proposed were conventional solutions that were already being considered by FD staffs While not a shortcoming of the SCR analytic framework or CBP process this effect was recognized as a shortcoming in the production of the first SCR In the future a more comprehensive examination of potential solutions to the capability deficiencies will be conducted A specific form was prepared to capture all the details related to each alternative This ensured that each alternative was specified in the same way to facilitate comparisons The form also allowed all the alternatives’ information to be automatically consolidated into an archive for further processing later in the Capability Integration process The standardized set of data to be provided for each alternative included • Alternative identifier • In-service life cycle • Rough-order-of-magnitude cost • Cost accuracy • Personnel changes • Project stand-up timeframe 32 DRDC CORA TR 2009-013 • Deficiency closure – completeness and availability and • Risk - technological and implementation A copy of the Capability Alternatives Form is provided in Annex A One form was submitted for each capability alternative identified For the first version of the SCR approximately 200 capability alternatives were identified To facilitate data transfer and validation an automated data-extraction tool was developed in Visual Basic code in Microsoft Word® 21 This tool extracted the data for each alternative and recorded it in a designated spreadsheet Following the data extraction the tool checked that each required data field had an entry specified and where possible the data entry was validated against an expected value range Erroneous or missing data was flagged and the Capability Alternatives Form was returned to the responsible Capability Manager for resolution 4 3 Alternative Cost A rough-order-of-magnitude ROM cost was identified for each proposed capability alternative This cost was to include the initial acquisition cost if applicable and all additional costs expected to be incurred over the in-service lifetime of the alternative For comparability and affordability evaluation as will be explained in detail later each alternative’s acquisition cost was converted into an Equivalent Annual Cost EAC by dividing the ROM cost by the in-service life-cycle duration Annual costs for increases in military and civilian personnel were determined from estimated average annual salaries derived from strategic costing research 22 23 and the DND Cost Factors Manual 24 The annual costs for each incremental military and civilian person required were $107 000 and $74 900 respectively The EAC for acquisition combined with the salary costs for increases in personnel constituted the direct costs of the alternative It should be noted that as a consequence of the DND plan 35 to expand the CF budget allocation estimates already accounted for the personnel cost of an anticipated additional 2000 military members To avoid double counting cost estimates for military personnel requirements were based on increases beyond 2000 members Finally to obtain a true appreciation of the total cost of the alternative the direct cost needed be combined with estimated indirect costs for National Procurement NP Operations and Maintenance O M Equipment Support Training Basing and Research Development R D Again from past strategic costing research and historical cost data estimates for these indirect costs were developed based on the equipment “class” of the alternative Annex B provides the details of the indirect cost calculations 4 4 Deficiency Closure The overall ability of a capability alternative to resolve a deficiency was a function of the degree to which the deficiency could be closed High Medium Low completeness and the expected availability High Medium Low resulting from serviceability and fleet size of the alternative Values between zero and one were associated with each level of the scale for completeness and DRDC CORA TR 2009-013 33 availability and the overall deficiency closure value was calculated as the product of the two values see Table 10 Table 10 Deficiency Closure Values Availability Completeness 4 5 High 1 00 Medium 0 75 Low 0 25 High 1 00 Medium 0 75 Low 0 25 1 00 0 75 0 25 0 75 0 56 0 19 0 25 0 19 0 06 Development Risk Development risk associated with each capability alternative was captured through four evaluation criteria technological maturity technological supportability implementation interdependencies and implementation change These risk assessment criteria were taken from the Risk Assessment Questionnaire provided within the Treasury Board Policy on Project Management 25 As capability alternatives are more conceptual in nature than projects and in the interest of keeping the completion of the Capability Alternatives Form as simple as possible only a small subset of the risk factors proposed in the Risk Assessment Questionnaire were used It was felt that the four factors selected could be answered for all the alternatives and would adequately capture the magnitude of the development risk associated with the proposed alternative Each risk assessment criteria had four corresponding statements describing different levels of risk as shown in Each statement had an associated point score For each risk criteria the statement that most closely matched the situation was selected Only one selection for each criterion was permitted As proposed in the Treasury Board risk methodology the average score from all the criteria is calculated An average score of one to three 1≤Average Score≤3 is Low Risk greater than three and up to five 3 Average Score≤5 is Medium Risk and an average score greater than 5 and up to seven 5 Average Score≤7 is High Risk Each capability alternative was given a risk rating of low medium or high based on this procedure 34 DRDC CORA TR 2009-013 Table 11 Capability Alternative Risk Assessment Model Change due to Implementation Implementation interdependencies Technology Supportability Technology Maturity Risk Factor Description Score The initiative involves implementation of a commercial or military off-the-shelf COTS MOTS solution with no integration or customization requirements 1 The initiative involves minor modifications to a COTS or MOTS product 2 The initiative involves major modifications systems integration hardening or complex militarization of a off-the-shelf product 5 The initiative involves research and development leading edge technology or the introduction of new technology to DND 7 There is every reason to believe that the proposed technology represents a solid foundation for the foreseeable future 1 Certain components may reach the end of their lifecycle before the system does but there is a high probability that there will be an upgrade path for replacement 2 Certain components may reach the end of their lifecycle before the system does and there does not appear to be a logical upgrade path 5 Various components appear to have reached the end of their lifecycle and more advanced technology exists in the market or technology foundation has yet to be determined 7 There are no interdependencies between this initiative and others 1 This initiative is dependent on other initiatives and there is a high degree of confidence that the initiatives will be on time and deliver the required capability 2 This initiative is dependent on other initiatives and there is a moderate degree of confidence that the initiatives will be on time and deliver the required capability 5 This initiative is highly dependent on other initiatives and there is a low degree of confidence that the initiatives will be on time and deliver the required capability 7 This initiative will impose very little change if any 1 This initiative will impose minor change 2 This initiative will impose significant changes 5 This initiative will present an entirely new way of conducting force generation force employment activities 7 DRDC CORA TR 2009-013 35 4 6 Objective Force Compliance Concurrently with the specification of alternatives to resolve the identified deficiencies CFD led an initiative to define a coherent comprehensive description of the force envisioned for the future the force of 2028 referred to as Objective Force 2028 From 26 “Objective Force 2028 offers a snapshot of the Canadian Forces CF of the future against which Force Development efforts can be compared and is intended to provide decision support for Force Development activity The characteristics competencies and required outputs of the Objective Force become the goal of our collective efforts to optimize CF capabilities … This combat capable force possessing the attributes of adaptability and endurance will be effective in the complex and information driven battlespace of the future The CF will be able to apply fires and conduct influence activities against defined targets achieving defined effects with precision Central to the creation of shaping control or stabilization effects the CF will be sufficiently adaptable This means being robust resilient responsive flexible and agile ” While the capability goals defined through capability planning represent estimated requirements based on expectations for the future the Objective Force identified general characteristics for adaptability of the CF that went beyond the Force Development scenarios These characteristics attempted to define requirements for the ability of the force to meet new unforeseen situations This added another dimension for the evaluation of the capability alternatives The essence of the Objective Force was distilled into a small number of key attributes that could be used to judge how well each alternative fit the vision of the future force These key attributes are listed in Table 12 and became evaluation criteria to estimate the Objective Force compliance of each capability alternative As shown in the Table 13 Objective Force compliance factors were defined However not every compliance factor was relevant to every capability domain Seven of the compliance factors were universal to all domains Three attributes applied only to the Act Shield and Sustain domains while a different set of three attributes applied only to the Command and Sense domains Each capability alternative was judged against 10 compliance factors If the alternative was assessed to meet an individual attribute then it is given a score of one 1 for that factor if not then it was given a score of zero 0 The scores for all the factors were added and divided by 10 giving each alternative an Objective Force compliance value as a percentage i e 80% compliant 36 DRDC CORA TR 2009-013 Table 12 Objective Force Compliance Factors Serial Title 1 Combat Capable kinetic and non-kinetic 2 Lines of Operation LOOs Capacity 3 Precision 4 Adaptability 5 Responsiveness Reaction Time and or Reach temporal 6 Interoperability 7 Operability in Austere Threat Environments 8 Efficiency and or Economy of Effort 9 Modular and or Scalable 10 Agility 11 Network Enabled or Enabling 12 Rapid Decision-making 13 Knowledge and or Information Sharing DRDC CORA TR 2009-013 Description The alternative is capable of combat by delivering a kinetic or non-kinetic effect The alternative has the capacity to sustain 1 LOO indefinitely or sustain 2 LOOs for short periods of time The alternative is able to deliver capability with precision or it enables contributes to precision delivery of capabilities effects Applies to kinetic and nonkinetic effects The alternative can be employed deployed in more than one environment OR The alternative can deliver an effect in more than one environment The alternative increases the responsiveness or reaction time for applying the capability OR the reach range of application of the capability The alternative can deliver capability by more than one asset type OR the alternative is interoperable with other services partners i e joint interagency or within comprehensive approach The alternative is designed to operate under adverse conditions and or hardened equipped against nonconventional threats The alternative takes into consideration potential for future resource constraints Optimizes reduces requirements from the status-quo Leverages technology to reduce resources The alternative consists of “building blocks” that can be tailored to create different effects or provide surge capacity or cover a greater area within AOR Not to be confused with and separate from capacity required for multiple LOO Question 2 The alternative provides a physical agility to redirect or re-role the capability quickly OR a psychological agility to increase awareness ability to think and draw conclusions Not to be confused with Question 4 The alternative will contribute or facilitate creating a CF DND-wide network The alternative reduces decision loop time or reduces steps within the decision loop Promotes selfsynchronization The alternative provides more information better resolution or wider distribution The alternative provides education training Capability Domains ACT SHIELD SUSTAIN ACT SHIELD SUSTAIN ACT SHIELD SUSTAIN ALL ALL ALL ALL ALL ALL ALL COMMAND SENSE COMMAND SENSE COMMAND SENSE 37 4 7 Alternative-Deficiency Interdependence It was recognized that the performance of alternatives could be dependent on the state of other capabilities A simple example is an alternative involving fighter aircraft The overall effectiveness and capability provided by this alternative could be affected by a capability to provide air-to-air refuelling If a deficiency exists in the air-to-air refuelling capability the ability of the fighter aircraft alternative to fully address the intended capability requirements could be diminished To account for this interdependence between alternatives and deficiencies a matrix and rating scale were established A simple three-level scale was used to rate the level of dependence of an alternative on a deficiency No Dependence Weak Dependence or Strong Dependence Each level in the rating scale had an associated dependence value that would be used to adjust the degree of closure the alternative could provide to its associated deficiency The dependence value is the degree to which the performance of alternative would be degraded without the closure of an associated deficiency With no dependence there would be no expected degradation hence a zero value With strong dependence severe but not total degradation would be expected A value slightly less than one total degradation was chosen Weak dependence would lie somewhere between the two extremes 50% degradation was arbitrarily chosen The rating scale and weights are shown in Table 13 Table 13 Alternative-Deficiency Dependence Scale Degree of Dependence No Dependence Dependence Value 0 Weak Dependence 0 5 Strong Dependence 0 95 Each alternative was defined with a closure value associated with the degree to which the alternative could resolve close its associated deficiency This closure value was estimated under the assumption that all required support for the alternative would be available as required This closure value then needed to be adjusted based on the alternative’s dependence on other deficiencies being closed i e the support required If the alternative was not dependent on a deficiency or if the deficiency on which the alternative was dependent was fully resolved 100% closed then the alternative’s closure of its deficiency would be the original value identified with the alternative When there was a dependence on a deficiency and the deficiency was not fully resolved the ability of the alternative to close its deficiency was adjusted by a factor related to the degree of dependence and the degree of closure of the depended-upon deficiency The final closure value associated with the alternative and its deficiency was calculated as the product of the initial closure value multiplied by the dependence value multiplied by the degree of closure of the depended-upon deficiency as indicated in Equation 1 38 DRDC CORA TR 2009-013 Cij' Cij Nd ∏ As 1 − Did ⋅ 1 − MAX C kd k 1 d 1 d ≠ j where C ij' is the final closure value of Deficiency j by Alternative i with dependence included Cij is the original closure value of Deficiency j by Alternative i 1 Did is the dependence value of Alternative i on Deficiency d Nd is the total number of Deficiencies and As MAX Ckd is the maximum closure value of Deficiency d by all the Alternatives As in the solution set k 1 Of course the final deficiency alternative closure values can only be calculated once the entire alternative solution set is identified which has implications on how the optimal set of capability alternatives would be determined as will be explained later 4 8 Available Funding The final constraint to be resolved before the activity to determine the best set of alternatives for development could begin was available funding In long-term planning and lifecycle management average equivalent annual cost has been shown to be an effective approach to use to assess the affordability of system that will be acquired and replaced in a cyclical manner 27 28 Under the assumption that the collection of all systems within a program will be maintained and refreshed repeatedly one can estimate an equivalent annual cost by dividing the total lifecycle costs of these systems by their planned lifecycle duration in years Comparing the equivalent annual cost against the annual program funding provides an effective means to assess long-term sustainability affordability of the program If the total equivalent annual cost of all the systems exceeds the annual budget there will be some point in time when the replacement of some of the systems will not be affordable and lapse in capability will be unavoidable without an increase in funding Note that actual expenditure schedules must be examined in detail to determine the exact point in time when the budget would be exceeded The Government of Canada is moving to use an accrual accounting procedure to fund major capital programs within the Department of National Defence 35 Under accrual accounting the acquisition cost of systems is spread out into equal annual expenditures over the entire lifecycle of the systems Under accrual accounting the actual cash phasing of the acquisition component of the program matches the estimates of equivalent annual cost To determine the equivalent annual funding available for new programs alternatives would involve estimating the equivalent annual cost of all existing systems within the CF and DND and comparing this total to the annual budget of the Department for capital acquisition Clearly the timeframe available to produce the SCR would not allow this comprehensive estimation to be performed As a substitute the cash phasing of the capital program out to 2028 which existed was examined to determine at what level in the out-years did capital expenditures stabilize and remain constant Given that the major fleet replacements were already factored into this projection via accrual accounting of the cash phasing this surrogate method to estimate the DRDC CORA TR 2009-013 39 equivalent annual funding available for capability alternatives investment was judged to provide the best estimate possible in the time available Through this approach it was estimated that DND had an equivalent annual funding buffer of 3 85 billion dollars $3 85B in current 2008-year dollars available for new capability investment This value was used as the funding constraint in determining the optimal set of capability alternatives for Force Development 4 9 Identifying the Best Set of Alternatives Identifying the best set of capability alternatives was fundamentally an optimization problem In the simplest terms the challenge was to identify the set of alternatives that provided the greatest resolution of the deficiencies most capability within the available funding limit This optimal set of capability alternatives would provide the best value for money At this point all the required components to perform the optimization had been assembled All possible alternatives were known For each alternative the equivalent annual cost could be determined by dividing the total cost by the in-service lifecycle the degree of deficiency closure was specified the value of the alternative deficiency mission value score was known as well as personnel cost risk Objective Force compliance and interdependence with other deficiencies The basic optimization problem was to find a set of alternatives that would deliver the highest aggregate capability at lowest risk with greatest Objective Force compliance for a given investment budget cost and personnel The solution could be a set of up to 87 alternatives one alternative for every deficiency chosen from among the 200-plus alternatives Since the level of capability offered by an alternative could be a function of the presence of other alternatives in the solution this multi-objective optimization problem was very complex and not amenable to manual approaches An optimization tool was built using the Phoenix ModelCenter 7 1 programming software suite from Phoenix Integration Inc The Phoenix ModelCenter suite hereafter simply referred to as Phoenix provides an optimization engine based upon a genetic algorithm The optimization engine can be linked to other software tools to provide data or evaluate options In the case of the optimization for the SCR Microsoft Excel® spreadsheets were utilized to provide the input data on the capability alternatives and evaluate the value of potential solutions In the Phoenix model an initial set of potential solutions is randomly generated A set of 120 solutions was used in the case of the SCR Each solution contains an entry for each deficiency These entries identify which alternative if any has been selected for the deficiency Each deficiency can have at most one alternative selected or no alternative selected in the latter case the deficiency is entirely unresolved The overall value referred to as the Figure of Merit FOM of each solution is calculated as the sum of the value of each deficiency adjusted by the degree it is closed by an alternative as shown in Equation 2 The FOM is a dimensionless variable that captures the overall value of SCR solutions on a continuous numerical scale 40 DRDC CORA TR 2009-013 Nd Ni Fs ∑∑ SjCij' Asi 2 j 1 i 1 where Fs is the FOM of Solution s S j is the mission value score of Deficiency j C ij' is the final closure value of Deficiency j by Alternative i with dependence included Equation 1 Asi has a value of one if Alternative i is in Solution s otherwise Asi equals zero Nd is the total number of deficiencies and Ni is the total number of alternatives The final FOM for the solution is a weighted sum of the adjusted deficiency values for International and Domestic Continental operations 10 As well as the FOM cost total personnel risk and Objective Force compliance are also determined for each solution Comparing these solution attributes against ordinal scales and thresholds allows each solution to be ranked The next generation of solutions is formed by cloning solutions evolving solutions and mutating solutions In cloning solutions a number of the best solutions 16 in the case of the SCR are carried over into the next generation To evolve new solutions the genetic algorithm then takes the top-ranked best solutions and randomly selects alternatives identified in these solutions to create new solutions for the next generation Finally in mutating solutions some existing solutions are chosen and a number of the identified alternatives are randomly replaced with ones selected from the broader pool In this way a new generation of solutions is populated from the best individual solutions of the previous generations with some random variability mimicking the biological evolutionary process of nature This process is repeated multiple times to produce many new generations of solutions With each generation of solutions a measure of improvement over the previous generation is calculated based how on many of the new solutions are better than the best of the previous generation Propagation of new generations of solutions continues until the degree of improvement between generations drops below a set threshold then the process stops For the SCR if fewer than three better solutions were produced in 25 generations the process was terminated Annex B provides a more detailed explanation of the process and the equations used to calculate the attributes of the solutions and generations This process results in an intensive directed search of the solution space in an attempt to locate the best highest FOM deficiency closure highest Objective Force compliance lowest risk solution within the set constraints of cost and personnel Because this procedure uses a heuristic algorithm and terminates when very little improvement is being obtained there is no guarantee that the best solution is located However given the intensity of the search in the vicinity of the best solution there is a very high probability that very good solutions possibly including the best solutions will be found In the iterations performed for the SCR 35 to 55 thousand solutions were generated for each combination of constraints 10 Recall that in the data collected for each capability alternative separate closure values were specified for each of Domestic Continental and International operations DRDC CORA TR 2009-013 41 Figure 19 shows the Phoenix results for one set of SCR constraints Domestic International weighting funding and personnel growth Each coloured dot in the chart represents one solution which is a set of selected capability alternatives The solution is plotted on a vertical scale of the aggregate FOM value and a horizontal scale of total equivalent annual cost The colour of the dot indicates the risk level associated with the solution The points marked with crosses ‘ ’ identify local optima Each cross represents a combination of alternatives where the same or better level of capability FOM cannot be delivered for less cost These points describe the efficiency frontier All points on the frontier represent an efficient use of resources It is impossible to find a point above the line and any point below the line represents a less efficient use of resources From the chart one can see that the majority of the solutions are coloured in the green-to-blue colour range indicating relatively low risk There are a few high risk red solutions These high risk solutions occur quite low on the chart far away from the area of optimal solutions Examining how the density of solutions varies in the graph provides a sense of the direction dictated by the genetic algorithm in searching for better solutions The poorer solutions located at the bottom of the chart are quite sparse As you move to the area of better solutions the density of solutions increases rapidly These characteristics of the solution set shown in Figure 19 were present in every Phoenix result for the SCR and came to be affectionately referred to as the Phoenix jellyfish ly e On FOM military value ti v s tra Il lu Figure 19 Sample Phoenix SCR Output 42 DRDC CORA TR 2009-013 Effective presentation of the Phoenix output is limited to three dimensions attributes such as FOM cost and risk It is up to the user to select which attributes to display As risk was not a significant discriminator among solutions results were also viewed as a function of FOM cost and personnel substituting personnel for risk Figure 20 shows one such set of results While the Phoenix optimization algorithm uses the constraints to direct the search of the solution space to concentrate in the area of the best solutions it does not exclude solutions that exceed the constraints Post-processing of the results using Phoenix functions is employed to filter the solutions to identify only the feasible ones For the SCR total military personnel growth was originally constrained to 2000 people Figure 21 displays Phoenix results where a personnel growth constraint of 2000 people and a funding constraint of $3 85B have been imposed From the Figure it can be seen that only solutions that meet the constraints retain a colour while all the solutions that exceed the imposed constraints are greyed out Also the new set of solutions that define the optimal set is shown with the crosses ‘ ’ This set of optimal solutions can be extracted for further detailed examination ly e On FOM military value ti v s tra u l l I Figure 20 Phoenix Results for FOM Cost and Personnel DRDC CORA TR 2009-013 43 ti ve s tra u l l I O nl y Figure 21 Phoenix Results Constrained for Cost and Personnel A separate examination was performed to explore the relationship between Objective Force compliance and risk for the Phoenix optimal solutions that lay in the vicinity of the funding limit Figure 22 shows one graph used to view this relationship For the optimal solutions produced by the Phoenix tool Objective Force compliance at the solution-level was taken to be the average of the Objective Force compliance scores of the capability alternatives contained in the solution Development risk for the solution was measured as the proportion of high risk alternatives contained within the solution For more details refer to Annex B 44 DRDC CORA TR 2009-013 Figure 22 shows that all the optimal solutions produced from this particular Phoenix run had high Objective Force Compliance 80% and relatively low risk 5% high risk alternatives This was the situation for all the Phoenix results In the end Objective Force compliance and development risk were not useful discriminators of the optimal solutions The final selection of the preferred optimal solution was based principally on the maximum FOM that could be obtained within the funding limit Optimal Solutions Objective Force Compliance vs Risk 100 % Objective Force Compliance 90 ti v s tra u l l I 80 ly e On 70 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 % High Risk Alternatives Figure 22 Optimal Solution Objective Force Compliance and Risk The final runs were conducted for different balances between North American and expeditionary missions As has already been noted the significance of capabilities and hence the figure of merit capability offered by alternatives varies between mission types Weighting factors between these mission types were used to represent different strategic priorities Optimization was repeated for a number of weightings to ensure that final recommendations were not artefacts of any particular weighting The weighting factors were varied from 4 1 to 1 4 North American-toExpeditionary priority as directed by CFD The final weighting factor 1 5 1 selected for the SCR was chosen by the Joint Capability Review Board DRDC CORA TR 2009-013 45 4 10 Program Financial Risk A key consideration and major concern of DND senior management was the financial risk associated with the recommended optimal set of capability alternatives for the SCR Recall that each alternative has a defined level of cost uncertainty identified with it In the optimization process cost uncertainty was not included directly as part of the optimization The allowance for cost uncertainty was established in a committee setting The final set of optimal solutions mapped against FOM and cost were presented to the Joint Capability Requirements Board with a request for direction on the size of the financial buffer that should be reserved to deal with potential cost over-runs due to the uncertainty in the cost estimates of the alternatives To assist with this decision analytic results in the form of probability estimates of the likelihood of exceeding the program funding limit were presented to the Board Cost uncertainty for each capability alternative was identified using the three-level scale shown in Table 14 The level of cost uncertainty was determined primarily as a function of the phase the planning was at in the project development process The cost uncertainty ranges were defined in accordance with accepted project management practices described in the Project Management Body of Knowledge 29 and the Cost Estimate Classification System 30 Greatest uncertainty was associated with alternatives that were basically new ideas for which no detailed definition work had been completed Here the cost of the capability alternative was estimated as possibly reaching as high as three times the cost estimate provided and could be as low as half the estimated cost At the other end of the scale where planning for the alternative was at the project identification stage alternative cost could be 75% greater than the estimate to as low as 25% less than the estimate These cost boundaries established defined ranges for the costs of each capability alternative Table 14 Cost Uncertainty Scale To assess the probability of an SCR solution exceeding the funding limits the cost estimates for the alternatives were treated as probability distributions The provided cost estimate was taken to be the most likely cost while the cost uncertainty defined the minimum and maximum values of the cost range To ensure a robust perspective of the financial risk three separate analyses were performed using different probability models to represent the alternatives’ cost distribution In one iteration alternative cost was modelled as a standard triangular distribution 31 32 shown at the top of Figure 23 Anecdotal evidence suggested that projects rarely complete under budget To investigate this perspective a second version of the triangular distribution was used with the 46 DRDC CORA TR 2009-013 minimum and most likely costs being the same i e the cost estimate of the alternative This version of the cost distribution was referred to as the pessimistic triangular distribution and is shown in the middle chart of Figure 23 The last distribution examined was one where alternatives’ cost was modelled as a uniform distribution i e all possible costs within the defined range are equally likely This distribution is shown at the bottom of Figure 23 Alternative Cost Probability Profile Standard Triangular Distribution Probability Cost estimate Low er Uncertainty Limit Equivalent Annual Cost Upper Uncertainty Lim it Alternative Cost Probability Profile Pessimistic Triangular Distribution Probability Cost Estim ate Low er Uncertainty Lim it Equivalent Annual Cost Upper Uncertainty Lim it Alternative Cost ProbabilityProfile Uniform Distribution Probability Cost Estimate Low er Uncertainty Lim it Equivalent Annual Cost Upper Uncertainty Lim it Figure 23 Alternative Cost Probability Models DRDC CORA TR 2009-013 47 To perform the financial risk assessment a program-cost model was developed using the @RISK® Monte Carlo simulation software package for Microsoft Excel® In this model the cost of each capability alternative in the solution is determined by sampling from the associated cost distribution The total cost of the entire program is then calculated by summing all the individual alternative costs Performing this procedure 1000 times generates a probability distribution of total program cost which could be compared to the SCR funding limit Figure 24 presents some indicative results of the financial risk assessment at the program level based on the standard normal triangular distribution From the Figure solution 25778 has a financial risk level of 80 percent which means there is an 80 percent probability of the entire program set of alternatives exceeding the funding limit The solutions shown are optimal solutions in sequential order moving down the efficiency frontier away from the funding cost constraint Because the level of cost uncertainty is unique to each capability alternative a solution further away from the funding limit does not necessarily have less financial risk than a closer solution This situation occurs for the first two solutions 25778 and 22652 shown in Figure 24 Financial Risk SCR Solution Financial Risk 90% 80% 70% t s tra Il lu i ve O nl y 60% 50% 40% 30% 25 77 8 22 65 2 18 98 6 48 89 1 42 96 0 49 75 0 32 23 7 31 27 4 33 56 8 39 48 1 20% 10% 0% Optimal Solutions Figure 24 SCR Solution Financial Risk To better understand the degree to which a program could be at risk from a costing perspective a cumulative risk profile was prepared based on the individual cost contributions of the capability alternatives in the solution As the cumulative risk profile is a function of the order in which alternatives are added the alternatives in the solutions were first placed in a sequence based on best value sum of the mission value scores of the deficiencies closed by the alternative for money cost of the alternative The assumption here is that one would want to minimize the risk for the alternatives that offer the best cost-benefit values Figure 25 presents the cumulative risk profile for solution 25778 for the three different alternative cost distributions The profile shown 48 DRDC CORA TR 2009-013 in the Figure actually displays the inverse of the risk profile namely the cumulative probability that the program can be achieved within the defined funding limit For this solution there is no risk for the vast majority of alternatives Irrespective of the cost distribution only the last eight alternatives are at risk from a financial perspective The degree of risk varies according to the cost distribution used SCR Financial Achievability 1 00 0 90 Probability 0 80 0 70 ti v s tra Il lu 0 60 0 50 ly e On 0 40 0 30 0 20 0 00 Cmd 02A Cmd 08A Cmd 10A Cmd 13A Cmd 03A Cmd 12A Cmd 16A Cmd 14A Act 03A Act 14C Act 08C Act 13B Act 04A Act 11C Act 06A Sus 12A Cmd 05A Sus 06A Sus 19A Sen 04A Act 05B Cmd 01B Sus 07A Act 17B Sus 02A Sus 04A Cmd 15A Cmd 04C Sus 05A Sus 18B Sus 14C Sus 10A Sus 01A Sus 23A Act 07A Shd 07A Sus 03A Sen 07A Shd 06A Sus 11B Sen 03A Sus 08A Sen 01C Sus 13A Act 10A Shd 08A Sus 15B Sus 09B Act 02A Sus 17A Sen 10A Sus 21C Act 01A Sen 08B Act 18A Sus 26A Sen 09A Shd 01C 0 10 Alternatives Normal Triangular Dist Pessimistic Triangular Dist Uniform Dist Figure 25 SCR Solution Cumulative Risk Profile 4 11 Creation of “The List” Recognizing that the factors considered in characterizing capability alternatives and conducting the optimization were but a limited subset of all possible factors influencing the cost effectiveness of a solution the final chosen Phoenix solution was reviewed in detail by a working group composed of members of the broad DND CF FD community In two offsite retreats this working group examined each deficiency and every capability alternative included in the final solution debating and validating the suitability and acceptability of the chosen alternative The vast majority of the capability alternatives in the recommended solution were confirmed however a couple of solution alternatives were switched with other alternatives identified as possibilities to resolve the deficiency These switches were suggested for reasons of existing familiarity with the technology which would ease training and implementation and for reasons of improved costeffective supportability maintainability At the end of the retreats a final accepted SCR solution was established through this combined “science and art” process To obtain endorsement of the SCR solution by senior management it was acknowledged that the SCR solution would need to be integrated with all the other existing and Government-mandated FD initiatives to provide the total Force Development picture Stakeholders would want to be comfortable with a set of tangible projects that could be executed under a new Investment Plan DRDC CORA TR 2009-013 49 and were less interested in abstract concepts of alternatives and degrees of deficiency closure Existing and mandated initiatives were already established as defined projects different in format and scope than the detail of the capability alternatives of the SCR To provide for a seamless coherent integration it was decided to translate the capability alternatives of the SCR into “alternative components” pseudo-projects In some cases this was a simply a name change In other cases the alternative was decomposed into a number of pseudo-projects to conform with the procedures employed to manage projects within DND The identification of the alternative components would also assist the post-SCR activity to develop the IP The final set of projects and alternative components was assembled To allow ranking the complete set and to facilitate prioritization for the production of the IP the project additions to the SCR component set were assigned mission-value scores Government-mandated projects identified in the Canada First Defence Strategy 35 were given the highest mission-value score 100 projects introduced in previous Government announcements were assigned a mission-value score of 95 projects that were in implementation were given a value of 90 and projects that were approved but not in implementation were allotted a score of 85 The high mission-value sores would ensure that the projects would retain their high priority for completion and would be immutable in design and schedule Capability alternative components assumed the mission-value score of the alternative from which they were derived The final list of projects and components was assembled in various views priority ranking capability domain preferred implementation timeframe and principal stakeholder The list comprising some 322 strategic-level projects was again distributed for review and endorsement by the FD stakeholders The final agreed list provided full or substantial closure to 72 of 87 deficiencies identified a further five deficiencies for research activity and left only 10 deficiencies unaddressed 4 12 Scheduling Having established the breadth of capability development activity that would be conducted over the 20-year time period the last remaining requirement of the SCR was to provide a view of the implementation schedule As the IP would provide specific milestones for the various project development phases along with detail cash phasing the SCR implementation schedule was intended to provide a general strategic-level appreciation of the development sequencing and a starting point for the detailed planning of the IP The 20-year timeframe was divided into four five-year periods with the aim of identifying which projects and components could achieve initial operating capability within each time period To do this scheduling the actual acquisition funding available for each time period needed to be determined Projects that were mandated by the Government and or already approved for implementation were considered as foundational elements of the SCR As such they were considered as unchangeable in cost or schedule These projects were factored into the funding line accounting for accrual funding and explicit cash phasing as appropriate Remaining funding available in each five-year period for capability development was thus determined The remaining alternative components and projects at early stages of approval approximately 158 in number identified in the SCR were then scheduled against the available funding To simplify the scheduling challenge it was assumed that all the acquisition cost of a project would be expended 50 DRDC CORA TR 2009-013 within the five-year period in which it was assigned As the major fleet replacements were already accounted for as foundational elements this simplifying assumption was felt to be reasonable A simple spreadsheet tool was developed to assist with the scheduling of the SCR components The tool took as input the rank-ordered set of capability components to be scheduled The components were ranked based on their value-for-money score mission-value score divided by component cost allowing the most cost effective components to be scheduled first Each component had a preferred implementation period five-year period associated with it The scheduling tool dealt with each component in sequence attempting to place it in the preferred time period if available funding allowed As each component was placed in a time period the available funding for the time period was decremented appropriately When all the available funds in a desired time period were expended the component was assigned the next subsequent period with sufficient available funding From this methodology the lower ranked alternative components had the greatest risk of being delayed in their implementation The component implementation schedule derived from the scheduling tool was combined with the schedule for the foundational elements to produce the overall strategic capability development schedule for the 20-year timeframe It was understood from the outset that this simple scheduling approach would not account for inherent linkages between components which would logically dictate the relative scheduling of the group This method could not appropriately cater to cyclically sequenced capability upgrades for instance It was accepted that the scheduling tool would at best produce a preliminary scheduling solution which would require refinement The plan was to produce and distribute the preliminary project component implementation schedule then refine the schedule in a working group meeting by swapping projects components between time periods to produce a coherent schedule The scheduling tool was created with imbedded functionality to allow project component swapping to be conducted while keeping track of funding adjustments within the time periods After the preliminary implementation schedule was produced representatives from across the Force Development community met for two days to refine the schedule In the end the effort was unsuccessful as stakeholders could not accept delaying the execution of some projects seen as being important in the near term to allow advancing important projects in other domains However the project component list was endorsed and transferred to the Chief of Program organization for subsequent programming An Investment Plan that matches the anticipated funding profile was subsequently created which maintains the intent of the SCR list DRDC CORA TR 2009-013 51 5 Conclusion Based upon the results and recommendations of the CDS Action Teams CAT 3 in particular the Chief of the Defence Staff in 2005 directed that Capability-Based Planning be institutionalized as the Force Development process for the CF In 2007 the Vice Chief of the Defence Staff directed that a Strategic Capability Roadmap be produced by summer 2008 thereby enormously accelerating the development of the DND CBP process and analytic toolset Development of the DND CBP process extended the earlier concept work of the Directorate of Defence Analysis the Joint Concepts and Analysis Technical Panel of TTCP and CAT 3 While this prior work defined the concepts for the CBP process it did not provide a detailed procedure and toolset that could be directly implemented in DND Developing the CBP procedure and tools for DND became the mandate for the Strategic Planning Operational Research Team and the CFD Directorates of Capability Planning and Military Capability Management Prior to the tasking to produce the SCR a methodology and tools had been developed to determine CF capability requirements goals based on a set of planning scenarios derived from strategic planning guidance This methodology and set of tools had been employed in the analysis of several scenarios had been refined and proven as effective Preliminary prototypes of the tools to assess CF capability status reported through the Capability Outlook had been developed Lacking were the process and analysis methods models to go from the identification of capability deficiencies to the articulation of an optimal capability development plan The SCR tasking provided approximately 10 months to complete the development of the CBP components and employ them to define an affordable long-term capability development plan maximizing CF effectiveness The SCR goal was accomplished The final steps in the process going from capability goals to a defined affordable capability development plan were specified Analysis tools ForGE Capability Outlook Risk Outlook ANDREW Phoenix Integration etc were rapidly built tested validated and employed In the end an optimized prioritized list of projects and capability alternative components was created which could be traced back to policy and strategic guidance through a series of rigorous objective evaluations The effort to produce the first SCR was deemed a success on several fronts by CFD and the Vice Chief of the Defence Staff First an end-to-end practical CBP process that could be performed within the resource constraints of DND was defined and proven A comprehensive framework of analysis tools was build to provide the analytical foundation for the CBP process The process and tools were effectively utilized to produce a coherent comprehensive capability development plan The Risk and Capability Outlooks have been used widely by senior leadership over the past year When the Canada First Defence Strategy was announced senior leaders i e the CDS and Deputy Minister requested updated views of the Outlooks particularly the Risk Outlook to provide context When the Investment Plan was first developed in early fall 2008 a variety of Capability and Risk Outlook views were demanded to provide context to the investment decisions These simple decision support tools the Risk Outlook in particular have resonated very well at the highest levels in the Department Further for large capital projects as initial approval to proceed is sought the strategic context and value of the project are expected to be identified using the Capability and Risk Outlooks 52 DRDC CORA TR 2009-013 The prioritized list of projects and alternative components has been staffed through senior management boards On 16 July 2008 the Defence Management Committee endorsed the project listing of the Strategic Capability Roadmap 1 0 The SCR represents DND's first 20-year Force Development plan The SCR itself along with the tools developed to support the CBP process have been used widely since then to support Departmental investment decisions SCR 1 0 was one of the key documents perhaps the key document used to feed the Investment Plan specifically in terms of the projects that would be funded and the timeframe that those projects would receive funding The SCR and the analysis tools are being applied to investment trade-off decisions required to maintain the Investment Plan when project cost increases or implementation schedule changes occur In the future projects will be accepted into the Investment Plan on the basis of their assessment under this CBP framework rather than on the strength of stand-alone operational requirements arguments This confirms the future role of centralized joint Force Development in DND The process and tool set utilized to produce the first version of the SCR have been briefed to NATO and Allies 33 34 with positive feedback The feedback and interest received to date suggest that this DND approach to CBP is the state of the art for nations with the resource levels of Canada DRDC CORA TR 2009-013 53 6 The Way Ahead While the newly-developed DND CBP process with the analytic framework has been accepted and adopted as the Force Development process for the CF improvements are warranted In the haste to produce the first SCR some important factors could not be catered for to the extent they warrant In addition the level of integration among the tools within the analytic framework could be enhanced While several measures of capability attempted to measure capacity requirements they were found to be inadequate The capability deficits identified through the ForGE assessments primarily concentrated on the nature of the capability requirements and only weakly addressed the quantitative aspect Where quantitative measures were specified for capability capacity there were limited to primarily the number of lines of operations that would need to be conducted simultaneously within a single operation scenario This was compensated for to some extent during the Risk Outlook evaluations and the detailed specification of capability deficiencies In addition capacity demanded by concurrent operations was for the most part left entirely out of the evaluation While some preliminary research had been conducted on how to assess capacity requirements for multiple operations it was incomplete during the time period within which the SCR was produced This was recognized as one of the major weaknesses of the first SCR Ideally the assessment of capacity would be explicitly evaluated as part of the ForGE assessments to the same degree as the other components of capability Research is underway to develop an improved methodology to capture capacity deficiencies in a more robust and traceable manner From the onset of the initiative to produce the SCR it was recognized that FD staff resources were insufficient to conduct the analysis for all 16 capabilities Capability Managers and staff had been established in CFD for all capability domains except Generate The decision was taken that due to resource limitations the Generate domain would not be included in the first version of the SCR Since the completion of SCR Version 1 0 the Generate section has been established in CFD This section is currently conducting the capability analysis for the Generate capability Results from this Generate capability assessment will be added to the next version of the SCR At the time when the first SCR was being produced only eight of the 18 FD scenarios had been analyzed An assessment of the capability factors examined through the scenarios indicated that these eight scenarios provided an approximately “80 percent solution” to the capability requirements of the CF Most of the additional scenarios provided minor additions to the capability components already evaluated additions such a different geographic conditions additional climatic conditions etc The majority of the spectrum of operations had been examined through the first eight scenarios analyzed One of the exceptions to this was analysis of the baseline Domestic scenarios These scenarios were intended to capture every-day routine commitments placed upon the CF by the Government These commitments include surveillance of the North fisheries patrols search and rescue etc The capability requirements that would be derived from these scenarios would be critical to determining the total capacity requirements of the CF Recognizing the importance of these scenarios they were given top priority for the next round of analysis At the time of the 54 DRDC CORA TR 2009-013 production of this report CFD had already initiated the analysis of the baseline scenarios with an objective of updating the SCR to include their requirements by the fall of 2009 A review has also been started to examine the alignment of the current set of scenarios with Policy and projections of the future security environment that have occurred since the scenarios were first developed This effort will validate the current scenarios identify redundancy and determine where gaps exist where new scenarios will be required The Strategic Planning Operational Research Team has undertaken to improve the tools of the analytic framework before the initiation of the next cycle of the SCR Research is underway to refine the measures of capability that define capability requirements particularly those that capture capacity demands Modifications to the format of the results of scenario analysis are being explored to improve the transfer transformation of the measures of capability to the evaluation criteria for the ForGE tool to produce the Capability Outlook Improved standardization of the evaluation criteria between the different capability domains is also being addressed The ForGE tool is being reconstructed to expand its functionality in linking capability assessments to supporting evaluations and analysis studies operational trials exercises operations etc and to improve the ease of employment of the tool The analytic framework process is being extended to explicitly account for capability requirements from concurrent operations This is the last factor that must be included to provide a complete assessment of the capacity component of capability In addition to extending the methodology to provide a more complete assessment of capability sensitivity analysis is also strongly recommended for future versions of the SCR Many evaluation scales and assessments are utilized within the analysis process that ultimately produces an optimal capability development solution How sensitive the final solution is to changes in these scales assessments is unresolved Future versions of the SCR should incorporate sensitivity analysis to ascertain how robust the final solution is to minor changes in the evaluations within the analytical framework Having established the value and benefits of the CBP process the analytic framework and the SCR senior leadership in DND have indicated their intention to produce a new version of the Strategic Capability Roadmap SCR 2 0 commencing in the summer of 2010 with a delivery date one year later DRDC CORA TR 2009-013 55 References 1 Hillier R J CDS Planning Guidance – CDS Action Teams Chief of Defence Staff Department of National Defence 10 March 2005 2 A Role of Pride and Influence in the World – DEFENCE Canada’s International Policy Statement Department of National Defence Government of Canada 2005 3 Billyard A Blakeney D Operational Research Support to the Chief of Defence Staff Action Team 3 Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA TR 2005-22 December 2005 4 CDS Action Team 3 Operational Capabilities Final Report Department of National Defence 2005 5 Burton R G L Christopher P Chouinard L Kerzner and Maj K Simonds Procedures processes and Tools of Capability-Based Planning An Outline for a Canadian Approach Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA TR 2005-35 November 2005 6 Hillier R J CDS TRANSFORMATION SITREP 01 05 Chief of the Defence Staff Department of National Defence 6 July 2005 7 Parker LCol R G L Christopher D Blakeney R Petryk B Taylor V Fong M Ball L Kerzner et al Strategic Capability Roadmap Briefing to the Joint Capability Review Board Department of National Defence July 2008 8 Guide to Capability-Based Planning TTCP Joint Systems Analysis Group Technical Panel 3 The Technical Cooperation Program 2004 9 Archambault P M C C Morrisey and M L Roi DND Force Development Scenarios Chief Force Development Project Report 24 November 2006 10 Blakeney D Billyard A Parker LCol R McLay LCdr R Deroche MWO C McDonald Cdr D The Capability Planning Process for Force Development Force Planning Scenarios 6-8 Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA TR 2009-034 July 2009 11 D BLAKENEY et al The Capability Planning Process for Force Development – after Four Force Planning Scenarios Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA Technical Memorandum TM2007-16 June 2007 12 Initiating Directive for the Production of the Strategic Capability Roadmap SCR Vice Chief of Defence Staff Department of National Defence Letter 195-0-1 D Cap P 25 October 2007 56 DRDC CORA TR 2009-013 13 CF Operational Planning Process Department of National Defence B-GJ-005-500 FP00 2002-11-06 2002 14 The Eight-Scenario Roll-Up Analysis for Capability-Based Planning CFD Final Report to Capability Development Board Department of National Defence February 2008 15 Billyard A P A User manual for CATCAM 3rd Edition Defence R D Canada – Centre for Operational Research and Analysis DRDC Technical Note TN 2007-06 June 2007 Internal Publication 16 Strategic Capability Planning for the Canadian Forces Vice Chief of Defence Staff Department of National Defence 13 June 2000 17 Solomon Dr B and L F Kerzner An Overview of the Strategic Costing Model presentation to Defence R D Canada – Centre for Operational Research and Analysis Annual Meeting October 2006 18 Blakeney D E A P Billyard L F Kerzner B Solomon and P L Chouinard Operational Research Tools Supporting the Force Development Process for the Canadian Forces Information and Security Journal Volume 23 No 1 2008 19 Project Approval Guide Director Force Planning and Program Coordination Department of National Defence January 2008 20 Ball M G The Outlook Tool - A Tool from the Strategic Capability Roadmap version 1 0 Tool-suite Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA TN 2009-028 June 2009 Internal Publication 21 Beard A Pelletier E Microsoft Word Form Collation Tool A Tool from the Strategic Capability Roadmap Version 1 0 Tool-suite Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA TN 2009-030 August 2009 22 Chouinard P and I Wood The Department of National Defence Strategic Cost Model Development Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA Technical Report 2007-14 September 2007 23 Solomon Binyam Paul Chouinard and Leonard Kerzner The Department of National Defence Strategic Cost Model Volume 2 Theory and Empirics Defence R D Canada – Centre for Operational Research and Analysis DRDC CORA Technical Report 2008-03 October 2008 24 Cost Factors Manual 2007-2008 Assistant Deputy Minister Finance and Corporate Services publication A-FN-007-000 AG-001 December 2007 25 Treasury Board Policy on Project Management 2-02 Government of Canada June 1994 26 Objective Force 2028 Chief Force Development Department of National Defence July 2008 DRDC CORA TR 2009-013 57 27 Funk R W and G L Christopher Capital Asset Amortization Model CAAM Defence R D Canada – Centre for Operational Research and Analysis Operational Research and Analysis Project Report 9510 September 1995 28 North Atlantic Treaty Organization Handbook on Long Term Defence Planning NATO Research and Technology Organization Technical Report RTO-TR-069 AC 323 SAS025 TP 41 April 2003 29 Project Management Body of Knowledge Guide Materiel Knowledge Network Project Management Institute 2004 30 Recommended Practice Cost Estimate Classification System Cost Engineering Volume 39 Number 4 Association for then Advancement of Cost Engineering International 1997 31 Emond E J Cost Estimation using Three-Point Estimates Defence R D Canada – Centre for Operational Research and Analysis ORAE Project Report PR303 May 1985 32 Vincent P A Methodology for Quick and Easy Risk Assessment and Cost Estimation Defence R D Canada – Centre for Operational Research and Analysis Directorate of Logistics Analysis Research Note 93 3 April 1993 33 Blakeney D G Christopher A Beard and R Petryk Determining Capability Goals and Prioritizing Capability Deficiencies for the Canadian Forces Proceedings of the NATO RTO SAS Specialist Meeting on Capability-Based Long Term Planning 2008 RTO-MPSAS072 November 2008 34 Taylor B R Petryk G Christopher D Blakeney and A Beard Producing an Integrated Capability Roadmap for the Canadian Forces Proceedings of the NATO RTO SAS Specialist Meeting on Capability-Based Long Term Planning 2008 RTO-MPSAS072 November 2008 35 Canada First Defence Strategy Department of National Defence Government of Canada 2008 36 Groves Major Richard Personal Communication to B Solomon Defence R D Canada – Centre for Operational Research and Analysis Assistant Deputy Minister Material Director Materiel Group Comptrollership Department of National Defence 20 May 2009 37 Iburg Peter Personal Communication to L Kerzner Defence R D Canada – Centre for Operational Research and Analysis Assistant Deputy Minister Finance and Corporate Services Directorate of Budget Department of National Defence November 2007 38 Mitchell Melanie An Introduction to Genetic Algorithms MIT Press 1998 58 DRDC CORA TR 2009-013 Annex A Capability Alternatives Form Deficiency DETAILS ALTERNATIVE insert ID Title In-Service Life Cycle years Identify the span of time in years that this Alternative would be in service ROM Cost Specify the Capital cost of the Alternative including Upgrades Updates and incremental infrastructure costs in thousands of dollars IAW D Cap P Costing Instructions Level of Definition Cost Accuracy Check the box beside the statement that most closely matches the situation Capital Cost $ $000s Project ROM Costing figures based on project estimates Cost accuracy 75% -25% or better dependent on project phase ref Mat Knet PMI Concept ROM Costing based on detailed feasibility or concept studies conducted by ECS L1staff Cost accuracy 100% -50% ref Association for Advancement of Cost Engineers Cost Estimate Classification System Placeholder ROM costing data based on 30 year cost model historical data analogous projects Cost accuracy 200% 50% ref Association for Advancement of Cost Engineers Cost Estimate Classification System Personnel Changes Identify incremental changes in personnel explicitly indicate if it is “ ” or “-“ change Project Standup Deficiency Closure a Completeness To what extent is the alternative able to close the Deficiency Check the box beside the applicable answer for both the Domestic Continental and International scenario contexts Deficiency Closure b Availability How often is the alternative available to close the Deficiency Check the box beside the applicable answer for both the Domestic Continental and International scenario contexts DRDC CORA TR 2009-013 Military personnel Civilian personnel Year the Project Office would stand up YYYY Dom Con Int Dom Con Int High The alternative is able to close almost all 95% of the deficiency Medium The alternative is able to close the majority 50% - 95% of the deficiency Low The alternative is able to close a minimal amount 50% of the deficiency High The alternative is available almost all of the time 95% to close the deficiency Medium The alternative is available the majority of the time 50% - 95% to close the deficiency Low The alternative is available a minimal amount of time 50% to close the deficiency 59 Technology Risk a What is the level of technology maturity and what are the integration customization requirements Check the box beside the statement that most closely matches the situation The initiative involves implementation of a commercial or military off-the-shelf COTS MOTS solution with no integration or customization requirements The initiative involves minor modifications to a COTS or MOTS product The initiative involves major modifications systems integration hardening or complex militarization of a off-the-shelf product The initiative involves research and development leading edge technology or the introduction of new technology to DND Technology Risk b How supportable are the key technologies in the initiative Check the box beside the statement that most closely matches the situation There is every reason to believe that the proposed technology represents a solid foundation for the foreseeable future Certain components may reach the end of their lifecycle before the system does but there is a high probability that there will be an upgrade path for replacement Certain components may reach the end of their lifecycle before the system does and there does not appear to be a logical upgrade path Various components appear to have reached the end of their lifecycle and more advanced technology exists in the market or technology foundation has yet to be determined Implementation Risk a There are no interdependencies between this initiative and others What is the level of initiative interdependencies Check the box beside the statement that most closely matches the situation This initiative is dependent on other initiatives and there is a high degree of confidence that the initiatives will be on time and deliver the required capability This initiative is dependent on other initiatives and there is a moderate degree of confidence that the initiatives will be on time and deliver the required capability This initiative is highly dependent on other initiatives and there is a low degree of confidence that the initiatives will be on time and deliver the required capability Implementation Risk b What will be the magnitude of change that this initiative will impose upon force generators and force employers in terms of force structure concepts of operation doctrine and procedures etc Check the box beside the statement that most closely matches the situation 60 This initiative will impose very little change if any This initiative will impose minor change This initiative will impose significant changes This initiative will present an entirely new way of conducting force generation force employment activities DRDC CORA TR 2009-013 Annex B Estimating Indirect Costs of Alternatives To obtain a true appreciation of the total cost of an alternative the direct cost needed be combined with estimated indirect costs for National Procurement NP Operations and Maintenance O M Equipment Support Training Basing and Research Development R D From past strategic costing research and historical cost data estimates could be made for these indirect costs based on the equipment “class” of the alternative Six equipment classes were used to categorize the alternates Mission Aircraft Ships Combat Vehicles Generic Mission Vehicles Civilian Pattern Vehicles Advanced Sensors Transport Aircraft and Information Technology These equipment classes have been used by Assistant Deputy Minister Material ADM Mat staff in estimating costs for capital programs Indirect annual costs were estimated as Equation B 1 specifies the formula used to estimate the annual NP cost of an alternative The NP cost is based on the capital acquisition cost CapitalEAC of the alternative multiplied by a factor related to the class of the equipment These factors were provided by ADM Mat staff 36 NP CapitalEAC NPClassFactor where NPClassFactor B 1 2 00 for Mission Aircraft Ships 1 00 for Combat Vehicles 0 80 for Generic Mission Vehicles 0 56 for Civilian Pattern Vehicles 2 70 for Advanced Sensors Transport Aircraft and 0 50 for Information Technology Equation B 2 calculates the annual O M cost of an alternative The O M cost is a function of the capital acquisition cost and the Military personnel cost The coefficients in the equation were derived through regression analysis of specific O M cost data provided by Assistant Deputy Minister Finance and Corporate Services staff 37 O M CapitalEAC 0 25 MilPersCost 0 35 B 2 Equipment support training and basing costs are estimated as shown in Equations B 3 B 4 and B 5 respectively The coefficients shown in the equations are averages derived from detailed data in the Strategic Cost Model 23 EquipmentSupport CapitalEAC 2 3 3 B 3 Training MilPersCost 2 7 0 B 4 Basing MilPersCost 3 3 7 0 B 5 DRDC CORA TR 2009-013 61 The associated R D cost of an alternative is a function of the total of all the other system costs as shown Equation B 6 The coefficient was calculated based on the total budget allocation of Defence Research and Development Canada compared to the entire defence program As a general rule of thumb R D cost can be estimated as approximately two percent of total system cost R D CapitalEAC MilPersCost CivPersCost NP O M EquipmentSupport Training Basing 2 0 98 0 B 6 Finally the total Equivalent Annual Cost EACij associated with alternative i for deficiency j was determined by Equation B 7 which combines the estimated indirect costs with the direct costs for acquisition CapitalEAC Military Personnel MilPersCost and Civilian personnel CivPersCost EACij CapitalEACij MilPersCostij CivPersCostij NPij O Mij EquipmentSupportij Trainingij Basingij R Dij 62 B 7 DRDC CORA TR 2009-013 Annex C C 1 SCR Optimization Model Figure of Merit Figure of merit FOM scores were used to capture relative military values for alternatives and solutions sets of selected of alternatives For a given mission type Domestic Continental or International the FOM of Alternative i for Deficiency j Fij was determined using the equation Fij S j Cij Nd ∏ As 1 − Did ⋅ 1 − MAX C kd C 1 k 1 d 1 d ≠ j where S j was the mission value score of Deficiency j Cij was the closure of Deficiency j by Nd Alternative i and the product ∏ d 1 d ≠ j As 1 − Did ⋅ 1 − MAX C kd accounted for the potential k 1 degradation in military value when Alternative i was dependent on an enabling capability provided through the closure of Deficiency d 11 Nd was the total number of deficiencies As was the number of alternatives in the solution set Did measured the strength of Alternative i ’s dependency on the capability associated with Deficiency d and Ckd was the closure of Deficiency d provided by Alternative k Cij and Ckd took on values in the range 0 1 0 when there was no alternative selected to address the deficiency to 1 when an alternative was selected which closed the deficiency to the maximum extent possible while the values Did fell in the range 0 0 95 0 if there was no dependency on the enabling capability 0 5 when there was weak dependency and 0 95 when the dependency on the enabler was strong 12 Qualitatively this meant the potential FOM of an alternative S j Cij was unaffected when there were no dependencies or when the enabling capabilities were fully provided However reductions to the potential FOM accumulated multiplicatively with every deficiency that remained in the enabling capabilities 11 Domain Managers provided two values associated with the closure of each alternative—the c completeness C ij i e degree to which the alternative could mitigate the deficiency if available and the a availability C ij i e likelihood the alternative would be accessible and able to perform as required The closure of an alternative was then given by the product Cij Cij C ij c 12 a Cijc and Cija took on values in the range 0 1 Cijc 0 or Cija 0 when no alternative was provided for c a c a the deficiency C ij 0 25 or Cij 0 25 when the completeness or availability of the provided alternative c were minimal C ij 0 75 or C ij 0 75 when the completeness or availability were intermediate and C ij 1 or Cij 1 when the completeness or availability were maximal Hence the product C ij Cij C ij a c a evaluated to a number in the range 0 1 DRDC CORA TR 2009-013 63 The solution FOM Fs for a given mission type was simply the sum of the alternative FOMs across all the Nd deficiencies That is C 2 Ni Nd Fs ∑∑ Fij Asi i 1 j 1 where Asi equals one 1 if the Alternative i is in the solution set and zero otherwise Finally the mission weighted solution FOM Fsw was the weighted Domestic Continental and International solution FOMs It was given by Equation C 3 Fsw w d c Fsd c wi Fsi sum of C 3 where w d c and wi were the respective Domestic Continental i e North American and international i e expeditionary mission weights and Fsd c and Fsi were the respective Domestic Continental and International solution FOMs calculated as described through equations C 1 and C 2 C 2 Number of Personnel Different sets of personnel were required by different alternatives and solutions The number of civilian personnel associated with Alternative i for Deficiency j Pijc and number of additional military personnel associated with Alternative i for Deficiency j Pijm were specified Pijc and Pijm took on integer values 0 indicated no alternative was selected to address the deficiency The number of civilian personnel associated with the solution Psc was simply the sum of alternative Pijc ’s across all the deficiencies That is C 4 Ni Nd Psc ∑∑ Pijc Asi i 1 j 1 where Asi equals one 1 if the Alternative i is in the solution set and zero otherwise 64 DRDC CORA TR 2009-013 Similarly the number of additional military personnel associated with the solution Psm was simply the sum of Pijm ’s across all deficiencies C 5 Ni Nd Psm ∑∑ Pijm Asi set and zero otherwise i 1 j 1 where Asi equals one 1 if the Alternative i is in the solution set and zero otherwise C 3 Cost Equivalent Annual Costs EACs were calculated for every alternative and solution The EAC of Alternative i for Deficiency j EACij was determined from estimates for in-service life and Rough Order of Magnitude ROM cost data The in-service life was determined as the number of years between the Initial Operational Capability IOC and end-of-life dates for the alternative The ROM cost of the alternative accounted for acquisition infrastructure equipment and midlife upgrades operation and maintenance equipment and base support research and development as well as personnel and training costs as explained in Annex B The cost associated with a solution EACs was determined using Equation C 6 C 6 Ni Nd EAC s ∑∑ EACij Asi i 1 j 1 where Asi equals one 1 if the alternative i is in the solution set and Asi equals zero otherwise C 4 Risk Each alternative was provided with technology and implementation risk scores Together these scores provided a measure of the overall risk of each alternative In detail every Alternative i for Deficiency j was assigned a technology risk score Rijt and an implementation risk score Riji 13 13 tm More specifically each Alternative i was assigned a technology maturity risk score Rij and a ts tm technology supportability risk score Rij Rij measured the likelihood and severity of initiative delays or ts underperformance due to the state of technological readiness while Rij measured the likelihood and severity of initiative underperformance or unsupportability due to technological obsolescence The technology risk of an alternative was then given by the sum Rij Rij Rij Furthermore each t tm ts ii alternative was assigned an implementation interdependency risk Rij and an implementation io ii organizational risk Rij Rij measured the likelihood and severity of reductions to alternative function or io benefit due to complex relationships to other enabling initiatives while Rij measured the likelihood and DRDC CORA TR 2009-013 65 The overall risk associated with the alternative Rij was calculated as the sum of technology and implementation risks Rij Rijt Riji C 7 Rij took on integer values in the range 4 28 14 Rij was considered high if it fell in the range 21 28 medium if it fell in the range 13 20 and low if it fell in the range 4 12 Assuming mitigation strategies would be in place to resolve issues with low and medium risk alternatives the overall risk of a solution Rs was determined as the proportion of alternatives in the solution with high risk Rs was therefore determined using the equation C 8 1 Ni Nd Rs − ∑∑ H ij Asi As i 1 j 1 where H ij identified if alternative i for deficiency j was a high-risk alternative H ij 0 if Rij ≤ 20 H ij 1 if Rij ≥ 21 Asi equalled one 1 if the Alternative i is in the solution set and zero otherwise and As was the total number of alternatives forming the solution a non-zero positive integer assuming solutions required the selection of at least one alternative C 5 Objective Force Compliance The Objective Force 2028 is a document and a vision integrating strategy concept and other products from across the CF and Department of National Defence DND into a coherent force goal It provides a high-level description of the characteristics competencies and operational requirements of the future CF against which Force Development efforts can be compared and capabilities optimized Every alternative i for deficiency j was assessed for how well it complied with the force goals described in Objective Force 2028 The compliance of each alternative Oij was given as a percentage The objective force compliance of a solution Os was then determined severity of reductions in function or benefit because of a departmental inability to change force structure concepts of operation doctrine and procedure as required by the alternative The implementation risk score of an alternative was then given by the sum Rij Rij Rij i 14 ii io Rijtm Rijts Rijii and Rijio took on discrete values from the set 1 2 5 7 1 if the individual measure of risk was negligible 2 if the risk was low 5 if it was intermediate and 7 if it was high Thus Rijt Rijtm Rijts and Riji Rijii Rijio took on integer values in the range 2 14 and the Rij Rijt Riji evaluated to an integer in the range 4 28 66 DRDC CORA TR 2009-013 as the average compliance across the set of selected alternatives That is C 9 1 Ni Nd Os − ∑∑ Oij Asi As i 1 j 1 where As was the total number of alternatives forming the solution a non-zero positive integer assuming solutions required the selection of at least one alternative and Asi equalled one 1 if the alternative i is in the solution set and zero otherwise C 6 Integration of Models and Optimization of Solutions C 6 1 Integration of Models to Compute a Single Solution In describing how solutions were optimized the process to compute a single solution must be discussed The procedure to compute a single solution is depicted in Figure C-1 and involved the following steps 1 Start 2 Select an allowed combination of alternatives i A 15 3 Calculate the mission weighted FOM Fsw using Equation C 3 for chosen Domestic Continental and International mission value weights w d c and wi 4 Calculate the number of civilian personnel Psc using Equation C 4 and the number of additional military personnel Psm using Equation C 5 5 Calculate the risk Rs using Equation C 8 6 Calculate the objective force compliance Os using Equation C 9 7 Calculate the cost EAC s using Equation C 6 8 Output data from steps 2—7 9 End 0 15 There were at least one and as many as five alternatives provided for each deficiency A solution was a set of chosen alternatives i A At most one alternative could be selected for each deficiency or no alternative could be chosen for a deficiency A combination of alternatives was considered as feasible solution if it contained as few as one alternative or as many alternatives as there were deficiencies DRDC CORA TR 2009-013 67 Fsw Fsw Psc Psm Psc Psm EACs EACs Rs Rs Os Os Figure C-1 Process Diagram for a Single Solution The process diagram for a single solution Figure C-1 depicts activity execution and information flow through the various steps Process start and end are displayed as rounded rectangles regular rectangles depict processing steps parallelograms depict key processing step outputs and arrow connectors indicate execution flow from one step to another Immediately after start is a processing step which generated an allowed combination of alternatives i A After an allowed combination was generated processing steps to determine Fsw Psc Psm Rs and Os were executed The processing step to determine EAC s was executed after the processing step for Psc and Psm determined the number of additional personnel associated with the solution Output was provided from each of the processing steps before end C 6 2 Constrained Multi-objective Optimization of Solutions A genetic algorithm GA 38 was used to perform a constrained multi-objective optimization of solutions The GA is a type of evolutionary algorithm making use of techniques inspired from evolutionary biology It was started by randomly generating an initial population of solutions After being generated each solution in the initial population was assessed for fitness and the fittest ones were allowed to propagate This generated a new set of solutions whose population size was equal to that of the previous generation Each solution in this new generation was then assessed for fitness and a convergence criterion was tested If the solutions were not converged 68 DRDC CORA TR 2009-013 then the process of propagating the fittest solutions assessing fitness and testing for convergence was repeated Once the criterion indicated that solutions were converged the process was terminated This process is summarized as follows 1 Start 2 Generate an initial population 3 Calculate properties of all solutions in the population and output data 4 Assess the fitness of each solution in the population 5 Propagate the fittest solutions in the current generation 6 Calculate properties of all solutions in the new population and output data 7 Assess the fitness of each solution in the new population 8 Test the convergence criterion 9 Return to step 5 if not converged 10 End In greater detail the initial generation g 1 contained S unique allowed combinations of alternatives i A S The composition of solutions in the initial population was determined by the value of a parameter called the seed Different seed values generated different initial populations The GA assessed solution fitness f i A S based on mission-weighted FOM Fsw using Equation C 3 cost EAC s using Equation C 6 and number of additional military personnel Psm using Equation C 4 The fitness of a solution increased with increasing Fsw Cost and number of additional military personnel constrained the solution space by applying a penalty decreasing the solution fitness when solutions exceeded specified limits on EAC s or Psm Based on direction from Chief Force Development acceptable solutions were to employ no more than 3000 additional military personnel and cost no more than $3 85 billion year These constraints were implemented by means of a penalty function The penalty that reduced the weighted FOM and hence solution fitness was calculated using Equations C 10 and C 11 DRDC CORA TR 2009-013 69 f i A S Fsw 1 − EAC s − EAC slim Psm − Psm lim TV EAC slim Psm lim TV 0 C 10 TV 2 5 ⋅ Tp TMax when EAC s EAC lim s m m lim or P P s s C 11 otherwise Where f i A S is the fitness value for solution h TV is the sum of percentages by which each of the constraints military personnel and cost was violated EACslim is the limit for solution cost $3 85 billion year Psm lim is the limit on additional military personnel 3000 TMax is the maximum percentage by which a constraint can be violated before the solution is considered infeasible set to 5% Tp is the basic percent penalty set to 50% to be applied to the FOM This form of penalty function and penalty values TMax and Tp were chosen after some experimentation and consultation with the technical support of the software company If the sum of a solution’s constraint violations was small then its FOM value was penalized by a small amount On the other hand if the constraint violations were large the exponential value of 2 5 ensured significant heavy penalization By penalizing solutions in this manner without completely eliminating them from the pool genetic information of “marginally infeasible” solutions with good performance is maintained for the benefit of future generations For example this would preserve an otherwise excellent solution with 3001 additional military personnel potentially allowing it the opportunity to mutate or breed thereby forming a similarly fit solution with less than or equal to 3000 military personnel Once the GA converged and processing halted all infeasible solutions remaining in the optimal set were filtered out This process obtained results high in quality and fully compliant with senior military direction on constraints for additional personnel and cost The GA propagated fittest solutions to create a new generation g 1 of allowed solutions i A S with population size S by a cloning the top few solutions into the new population offspring solutions were identical to parent solutions 16 b evolving the majority of the fittest solutions by generating new solutions with inherited properties offspring solutions were hybrids of parent solutions and c occasionally producing a mutant whose properties can be significantly different from those of the solutions in the current generation offspring solutions had properties not found in the parent population Each form of propagation played a significant role a cloning ensured survival of the fittest members of a population i e the best members of successive sets of solutions were at least as good as the best members of previous generations b evolution through inheritance of properties ensured beneficial solution traits were explored in 16 Progenitor solutions which propagate are called ‘parents’ while new solutions deriving from the ‘parents’ are referred to as ‘offspring’ 70 DRDC CORA TR 2009-013 more detail i e local optima of the solution space were examined comprehensively and c mutation ensured the GA broadly searched the spectrum of possible solutions i e provided a mechanism to search for the globally optimal solution s 17 In essence this was a multi-objective optimization which maximized FOM while minimizing solution cost In a multi-objective optimization problem best solutions are those which are Pareto optimal that is these solutions lie along a frontier line where for every value of cost no better FOM is possible The GA uses a Pareto-compliant multiple elitist scheme in determining the solutions of each successive generation Because of this the GA is focussed to converge upon solutions belonging to the Pareto optimal set The approach used to decide when to stop the GA was to examine the rate at which Pareto optimal solutions are discovered as the GA produces successive generations of solutions The frequency of finding such solutions over consecutive generations will tend drop as the algorithm converges to the Pareto optimal set As long as the probability of mutation is sufficiently high the GA will not get permanently trapped in local optima and it will spend minimal time in nonglobal optima efficiently sampling the space of solutions To formulate the stopping criterion a tolerance is specified as a minimum percent change in the Pareto optimal set resulting from a given generation of solutions If the percentage of new solutions in the Pareto set exceeds this minimum the generation has produced a significant improvement If the percentage of changed solutions falls below the minimum the given generation is said to have not produced a significant improvement Then if significant improvement in this sense is not observed after a sufficient number of successive generations it is assumed that the GA has adequately converged to the Pareto optimal set of solutions After some experimentation and discussions with the technical support at the software company the GA’s parameters were set as follows Population Size number of solutions in a generation 120 Successive Generations without Improvement to stop the GA 25 Minimum Change of Pareto Population Required for Improvement 2 5% Mutation Probability 0 05 This selection of parameters balanced the need to accurately find the Pareto optimal set with the need to deliver results under time constraints With more time available or better computational resources the percent change in the approximately Pareto optimal set could have been decreased the number of successive generations without improvement could have been increased or both 17 Mutation provides a mechanism to improve the likelihood of finding the globally optimal solution s However with any heuristic algorithm there is no guarantee of fining the global optimum DRDC CORA TR 2009-013 71 The detailed optimization procedure used by the GA is depicted in Figure A2 below The steps are described as follows 1 Start 2 Generate an initial population g 1 and assess its members a Select S unique allowed combinations of alternatives i A based on the initial seed value b Calculate properties of all S solutions in the population and output data 18 i Fsw using Equation C 3 for chosen w d c and wi ii Psc using Equation C 4 and Psm using Equation C 5 iii Rs using Equation C 7 iv Os using Equation C 9 v EAC s using Equation C 6 vi Output data from steps 2 a and 2 b i—2 b v c Assess the fitness of each solution f i A S in the population i Compare Fsw EAC s and Psm for each solution to determine fitness ii Rank solutions in decreasing order of fitness 3 Generate a new population g 1 and assess its members a Propagate the fittest solutions of the preceding generation to produce a new generation of allowed solutions i A with population size S i Clone the top few solutions ii Evolve the majority of the fittest solutions through inheritance i e hybridization iii Occasionally produce a mutant 18 Observe how steps 2 b and 3 b in this procedure are identical to steps 3 through 8 described in the preceding section to compute a single solution 72 DRDC CORA TR 2009-013 b Calculate properties of all S solutions in the new population and output data i Fsw using Equation C 3 for chosen w d c and wi ii Psc using Equation C 4 and Psm using Equation C 5 iii Rs using Equation C 8 iv Os using Equation C 9 v EAC s using Equation C 6 vi Output data from steps 3 a and 3 b i—3 b v c Assess the fitness of each solution f i A S in the new population i Compare Fsw EAC s and Psm for each solution to determine fitness ii Rank solutions in decreasing order of fitness 4 Test the convergence criterion 5 Return to step 3 if not converged 6 End The process diagram for optimization using the GA Figure C-2 depicts activity execution and information flow through the various steps Process start and end are displayed as rounded rectangles regular rectangles depict processing steps parallelograms depict key processing step outputs arrow connectors indicate execution flow from one step to another and decision points are represented by diamonds Immediately after the start are processing steps that initialize the values of the generation count g and the solution number s to 1 Following these steps are the iterative processing steps to generate a population of size S Allowed combinations of alternatives i A S are determined by the seed parameter value when g 1 and by propagation of the fittest solutions when g 1 After generating the population of solutions the procedure re-initializes s to 1 and enters an iterative loop to determine Fsw Psc Psm Rs Os EAC s and outputs data for each of the S solutions in the population Following this is the processing step that determines the fitness f i A S of each solution in the current generation The procedure then enters a decision point where a convergence condition is tested If the solutions are not yet converged the procedure increments g by one unit re-initializes s to 1 and continues by propagating and evaluating a new generation of solutions If however it is determined that the solutions have converged the process is terminated DRDC CORA TR 2009-013 73 s 1 s s 1 i A S s S s 1 s s 1 s S f i A S Figure C-2 Process Diagram for Optimization using the Genetic Algorithm 74 DRDC CORA TR 2009-013 List of symbols abbreviations acronyms initialisms ANDREW CAT CATCAM CBP CDS CF CFD COA COTS DND DRDC Activity-based Neoteric Deficiency Ranking and Evaluation Workbook CDS Action Team CAT 3 Capability Assessment Methodology Capability-Based Planning Chief of the Defence Staff Canadian Forces Chief Force Development Course of Action Commercial Off The Shelf Department of National Defence Defence Research Development Canada DRDKIM Director Research and Development Knowledge and Information Management EAC FD D Cap P FOM ForGE GA IP JCPT LOO MoC MOTS NP O M R D ROM SCR SME TTCP Equivalent Annual Cost Force Development Director Capability Planning Figure of Merit Force Generation and Evaluation Genetic Algorithm Investment Plan Joint Capability Planning Team Line Of Operation Measure of Capability Military Off The Shelf National Procurement Operations and Maintenance Research Development Rough Order of Magnitude Strategic Capability Roadmap Subject Matter Expert The Technical Cooperation Program DRDC CORA TR 2009-013 75 Distribution list Document No DRDC CORA TR 2009-013 16 1 5 2 LIST PART 1 Internal Distribution by Centre Authors 8 hard copies 8 PDF CS DRDC CORA PDF TL SPORT DRDC CORA 5 hard copies Library DRDC CORA 1 hard copy 1 PDF 24 TOTAL LIST PART 1 14 hard copies 10 PDF 1 1 1 1 1 1 1 1 1 1 1 LIST PART 2 External Distribution by DRDKIM ADM S T PDF CS DGMPRA PDF VCDS hard copy CFD PDF D Cap P PDF D Mil CM PDF C Prog PDF CMS PDF CLS PDF CAS PDF DRKIM PDF 11 TOTAL LIST PART 2 1 hard copy 10 PDF 35 TOTAL COPIES REQUIRED 15 hard copies 20 PDF 76 DRDC CORA TR 2009-013 DOCUMENT CONTROL DATA Security classification of title body of abstract and indexing annotation must be entered when the overall document is classified 1 ORIGINATOR The name and address of the organization preparing the document Organizations for whom the document was prepared e g Centre sponsoring a contractor's report or tasking agency are entered in section 8 2 Defence R D Canada 305 Rideau Street Ottawa Ontario K1A 0K2 3 SECURITY CLASSIFICATION Overall security classification of the document including special warning terms if applicable UNCLASSIFIED TITLE The complete document title as indicated on the title page Its classification should be indicated by the appropriate abbreviation S C or U in parentheses after the title Strategic Capability Roadmap Version 1 0 - Analytical Framework U 4 AUTHORS last name followed by initials – ranks titles etc not to be used Christopher G Blakeney D Petryk R Taylor B Kerzner L Beard A Fong V Ball M 5 DATE OF PUBLICATION Month and year of publication of document December 2009 7 6a NO OF PAGES 6b NO OF REFS Total containing information Total cited in document including Annexes Appendices etc 100 38 DESCRIPTIVE NOTES The category of the document e g technical report technical note or memorandum If appropriate enter the type of report e g interim progress summary annual or final Give the inclusive dates when a specific reporting period is covered Technical Report 8 SPONSORING ACTIVITY The name of the department project office or laboratory sponsoring the research and development – include address Defence R D Canada 305 Rideau Street Ottawa Ontario K1A 0K2 9a PROJECT OR GRANT NO If appropriate the applicable research and development project or grant number under which the document was written Please specify whether project or grant 9b CONTRACT NO If appropriate the applicable number under which the document was written 10a ORIGINATOR'S DOCUMENT NUMBER The official document number by which the document is identified by the originating activity This number must be unique to this document 10b OTHER DOCUMENT NO s Any other numbers which may be assigned this document either by the originator or by the sponsor DRDC CORA TR 2009-013 11 DOCUMENT AVAILABILITY Any limitations on further dissemination of the document other than those imposed by security classification Unlimited 12 DOCUMENT ANNOUNCEMENT Any limitation to the bibliographic announcement of this document This will normally correspond to the Document Availability 11 However where further distribution beyond the audience specified in 11 is possible a wider announcement audience may be selected Unlimited 13 ABSTRACT A brief and factual summary of the document It may also appear elsewhere in the body of the document itself It is highly desirable that the abstract of classified documents be unclassified Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph unless the document itself is unclassified represented as S C R or U It is not necessary to include here abstracts in both official languages unless the text is bilingual In 2005 the Chief of Defence Staff of the Canadian Forces CF mandated that CapabilityBased Planning be institutionalized as a part of a centrally driven top-down approach to Force Development FD within the Department of National Defence Consequently over the last three years military and defence analyst staff have developed and implemented the first version of a Canadian end-to-end capability-based FD process Scenarios derived from policy and strategic guidance capture the scope and scale of potential operations in which the CF could participate During the Capability Planning Process scenarios are analyzed to define capability requirements The Capability Management Process evaluates current and projected force structures of the CF against those capability requirements to identify adequacies deficiencies and surpluses Through the Capability Integration Process potential solutions alternatives for the deficiencies are identified Finally optimization methods are employed to determine the best set of alternatives affordable within the available budget to maximize CF capability The results form the Strategic Capability Roadmap a 20-year plan for CF capability development The Capability Planning Capability Management and Capability Integration processes are supported by a set of dedicated analysis tools which collectively have come to be referred to as the analytic framework This report documents these analysis tools and processes of the analytic framework employed to produce the first Strategic Capability Roadmap 14 KEYWORDS DESCRIPTORS or IDENTIFIERS Technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document They should be selected so that no security classification is required Identifiers such as equipment model designation trade name military project code name geographic location may also be included If possible keywords should be selected from a published thesaurus e g Thesaurus of Engineering and Scientific Terms TEST and that thesaurus identified If it is not possible to select indexing terms which are Unclassified the classification of each should be indicated as with the title Capability-Based Planning Strategic Capability Roadmap Capability Planning Capability Integration CATCAM ForGE Capability Outlook Capability Deficiency Mission Value Score Phoenix Integration Genetic Algorithm Risk Force Development Analytic Framework Capability management Capability Goal Measures of Capability Force Structure Risk Outlook ANDREW Capability Alternative Optimization Objective Force R i j DEFENCE DEFENSE if DHDC CU HA