FINAL EVALUATION REPORT

EVALUATION OF SEARCH AND RESCUE
SERVICES PROGRAM

PROJECT NUMBER 6B183
MARCH 2, 2017


EVALUATION DIRECTORATE
CHIEF FINANCIAL OFFICER SECTOR
FISHERIES AND OCEANS CANADA

ACKNOWLEDGEMENTS

The Evaluation Directorate would like to thank all individuals who provided input to the evaluation of Search and Rescue Services Program. In particular, the Directorate acknowledges the time and effort of key informants who took time to share insights, knowledge and opinions during interviews. The Evaluation Directorate also acknowledges the time and effort given by senior program management from the planning phase through to reporting.

ACRONYMS

CAF
Canadian Armed Forces
CCG
Canadian Coast Guard
CCG SAR Program
Canadian Coast Guard’s Search and Rescue Services Program
CCGA
Canadian Coast Guard Auxiliary
DFO
Fisheries and Oceans Canada
DND
Department of National Defence
IRB
In-Shore Rescue Boat Service
JRCC
Joint Rescue Coordination Centre
MRSC
Maritime Rescue Sub-Centre
RCAF
Royal Canadian Air Force
SISAR
Search and Rescue Program Information System
SMMS
Search and Rescue Mission Management System

TABLE OF CONTENTS

EXECUTIVE SUMMARY

Introduction

The evaluation of the Canadian Coast Guard’s Search and Rescue Services Program (CCG SAR Program) was conducted by DFO’s Evaluation Directorate and covers a five-year period from April 2011 to March 2016. The evaluation was completed in accordance with Treasury Board’s Policy on Results and the Financial Administration Act. Its main objective was to examine need, effectiveness and efficiency. The current evaluation also addressed the information needs of Senior Management in the Canadian Coast Guard (CCG) by examining the preparedness of the organization concerning its on-water presence and the frequency of certification exemptions for seagoing personnel. In addition, the evaluation examined the source of the maritime search and rescue incidents in order to detect trends in the type of vessel/search object. Lastly, the evaluation also followed-up on management actions developed by CCG in response to the recommendations in last evaluation completed in 2011-12.

Program Profile

Overall, search and rescue in Canada is a shared responsibility among federal, provincial/territorial and municipal governments, as well as air, ground and maritime volunteer search and rescue organizations. Marine and air search and rescue are both federal responsibilities, while ground SAR is the responsibility of the provinces, territories, and municipalities, except for federal Crown lands. The result is a federal search and rescue system in which military, civilian, and volunteer resources work together under the operational control of a Joint Rescue Coordination Centre or Maritime Rescue Sub-Centre to coordinate and control search and rescue responses. The national objective is to prevent loss of life and injury through search and rescue alerting, responding and aiding activities using public and private resources.

For its part, the CCG is responsible for the provision of search and rescue services in all maritime areas of federal responsibility (i.e. in the Great Lakes/St. Lawrence River system and all of Canada’s territorial coastal waters). The expected result for the CCG SAR Program is that the loss of life to mariners in distress is minimized. As such, the CCG is responsible for the detection of maritime incidents and the delivery of maritime search and rescue response in coordination with its partners. This is in addition to the provision of maritime resources as necessary to assist in aeronautical search and rescue, as well as humanitarian operationsFootnote 1.

The national response system was developed because of the size and remoteness of Canada. The CCG maritime search and rescue response components include: primary resources (comprised of large CCG vessels, lifeboat stations, and Inshore Rescue Boats); and secondary search and rescue resources (CCG vessels which have another program as their primary mandate, for example a vessel tasked with science activities and having a secondary mandate for the provision of search and rescue). In addition to the primary and secondary search and rescue resources, CCG also relies on aeronautical resources from DND as well as the volunteer aeronautical network known as the Civil Air Search and Rescue Association, the Canadian Coast Guard Auxiliary (CCGA), and vessels of opportunity.

The CCG SAR Program mandate is derived from several Acts and international conventions. The Constitution Act, 1867 gives the Parliament of Canada exclusive legislative jurisdiction over matters of navigation and shipping. With respect to coast guard services, the Oceans Act (S.C. 1996, c. 31) states that the powers, duties and functions of the Minister of the Department of Fisheries and Oceans extend to the maritime component of the federal search and rescue program. Under the Canada Shipping Act, 2001 (S.C. 2001, c. 26), the Minister may designate persons as Rescue Coordinators to organize search and rescue operations. Finally, through several international conventions, Canada has committed itself to making adequate arrangements for maritime search and rescue within its area of responsibility.

The CCG SAR Program includes the CCG Search & Rescue Coordination and Response sub-program and the CCGA sub-program. The CCG Search & Rescue Coordination and Response sub-Program provides coordination and response services to increase the likelihood of survival of people at risk of loss of life or injury in the maritime environment, ensuring they have access to assistance. Whereas, the CCGA subprogram is structured into five federally incorporated, not-for-profit volunteer organizations and are separate legal entities from the Government of CanadaFootnote 2. The majority of CCGA members are commercial fishers and pleasure boaters who donate their time and vessels to assist the CCG SAR Program and work in close partnership with the CCG. The remaining CCGA members are volunteers from local communities who crew community-based, dedicated SAR response vessels.


Evaluation Methodology

This evaluation adopted a theory-based approach whereby program performance was measured against intended results articulated in the program logic model. The chosen design was able to demonstrate the extent to which the program is achieving issues of need, effectiveness and efficiency, in line with the Treasury Board’s Policy on Results. Extensive use of triangulation was undertaken as an analytical method, where multiple lines of evidence helped corroborate findings. This involved a combination of key informant interviews and a review of administrative data to assess if the program had the level of inputs needed to support the coordination and response of maritime search and rescue incidents. Part of the analysis included an examination of the actual response(s) tasked to a search and rescue incident based on its type and severity. In addition, the evaluation considered the ability of the CCG to manage the costs related to both the coordination of and response to maritime search and rescue incidents; and, the degree to which the search and rescue services that are provided were cost efficient. The evaluation questions were determined subsequent to preliminary discussions with CCG Senior Management.

Evaluation Findings

As a result of its mandate to detect, coordinate and respond, there is a continuing need for the CCG SAR Program as evidenced by the steady number of maritime search and rescue incidents.

Evidence demonstrates that the CCG is successful in the detection of maritime incidents as well as the delivery of maritime search and rescue responses (i.e. in coordination with its partners). Overall, lives are being saved with the CCG and CCGA each being both prepared and available to assist people in need of help in the maritime environment. Known risks to search and rescue coverage are being mitigated in collaboration with search and rescue partners. For example, in the CCG Atlantic Region where the number of days planned and days delivered have decreased for large vessels, the CCG is mitigating this situation with the support of the Royal Canadian Navy. Evidence from the evaluation also highlights that the CCGA contributes to the overall efficiency of the CCG SAR program by handling 24% of all maritime SAR taskings.

Over the last five years, the number of operational days where the exemptions sought for rescue specialists doubled (from 4% to 8%), representing a total number of 2,668 operational days in 2015. There is evidence that new rescue specialists are being trained and many existing rescue specialists are being re-certificated. Nonetheless the CCG is not achieving its crewing levels and a solution is required in order to avoid possible impacts on the preparedness of the program.

There is evidence that the CCG SAR Program is managing costs through its efforts to optimize search and rescue resources when tasking a response to a maritime search and rescue incident as well as through its use of lower cost service delivery mechanisms (e.g. Inshore Rescue Boat Service and CCGA).

An obstacle to the maritime search and rescue system achieving even greater efficiency are the high number of incidents involving pleasure craft. Unlike other on-water users (e.g. commercial & fishing) where there have been reductions in the number of incidents, the pleasure craft group not only represents the single largest search object category, but the number and severity of incidents are also continuing to rise. Reducing the number and severity of search and rescue incidents involving pleasure craft may reduce both the number of lives lost as well as the overall number of maritime distress calls.

Recommendations

The evaluation made one recommendation concerning the increase in the number of operational days where exemptions were sought for rescue specialists.

Recommendation 1: CCG should develop and implement a strategy aimed at meeting the required ship’s crewing profile outlined in the Canadian Coast Guard Fleet Order #535 (i.e. Minimum number of designated rescue specialists on board Canadian Coast Guard Ships with a complement of four or more).

1.0 INTRODUCTION

1.1 Purpose of the Evaluation

This report presents the results of the evaluation of Canadian Coast Guard’s Search and Rescue Services Program (CCG SAR Program), including the Search and Rescue Coordination and Response sub-program and the Canadian Coast Guard Auxiliary (CCGA) sub-program. The evaluation was conducted by the Evaluation Directorate within Fisheries and Oceans Canada in accordance with Treasury Board’s Policy on Results and the Financial Administration Act. The main objective of the evaluation was to examine need, effectiveness and efficiency.

1.2 Evaluation Scope and Context

The evaluation covered the 5-year period from April 2011 to March 2016 and included the National Headquarters (NHQ) and all 3 Canadian Coast Guard (CCG) Regions: Western, Central and Arctic, and Atlantic. The evaluation started in March 2016 and will be presented to the Performance Measurement and Evaluation Committee in March 2017.

The scope of this evaluation was calibrated in order to focus on key issues identified in the previous evaluationFootnote 3. The current evaluation was also designed to address the information needs of CCG Senior Management by examining CCG preparedness concerning its on-water presence and certification exemptions for CCG seagoing personnel. In addition, the evaluation examined the source of the maritime search and rescue incidents in order to detect trends in the type of vessel/search object.


2.0 PROGRAM PROFILE

2.1 Program Context

Overall, search and rescue in Canada is a shared responsibility among federal, provincial/territorial and municipal governments, as well as air, ground and maritime volunteer search and rescue organizations. Marine and air search and rescue are both federal responsibilities, while ground and inland water SAR is the responsibility of the provinces, territories, and municipalities, except for federal Crown land and waterways. Federal SAR activities are managed primarily by the Canadian Coast Guard for marine incidents and the Canadian Forces (Royal Canadian Air Force) for air incidents.

Canada has integrated its maritime and aeronautical elements into a single search and rescue system. The result is a federal search and rescue system in which military, civilian, and volunteer resources work together under the operational control of a Joint Rescue Coordination Centre (JRCC), or Maritime Rescue Sub-Centre (MRSC) to coordinate and control search and rescue responses. The national search and rescue objective is to prevent loss of life and injury through search and rescue alerting, responding and aiding activities using public and private resources.

For its part, the CCG is responsible for the provision of maritime search and rescue services in all maritime areas of federal responsibility (i.e. in the Great Lakes/St. Lawrence River system and all of Canada’s territorial coastal waters). As such, the CCG is responsible for the detection of maritime incidents and the delivery of maritime search and rescue response in coordination with its partners. This is in addition to the provision of maritime resources as necessary to assist in aeronautical search and rescue, as well as humanitarian operationsFootnote 4. The expected result for the CCG SAR Program is that the loss of life to mariners in distress is minimized. The current level of service is to save 90% of the lives at risk. In 2015-16 the percentage of lives saved was measured at 99.16%.

The approach for maritime search and rescue services was to develop a national response system because of the size, and remoteness, of the Canadian maritime search and rescue region. Maritime search and rescue response components include primary and secondary resources. Primary resources are comprised of large CCG vessels, lifeboat stations, and Inshore Rescue Boats. It is the CCG service standard that primary resources depart within 30 minutes or less 99% of the time after receiving a SAR taskingFootnote 5. Secondary resources include CCG vessels which have another program as their primary mandate. For example a vessel tasked with science activities and having a secondary mandate for the provision of search and rescue. In addition to the primary and secondary search and rescue resources, CCG also relies on aeronautical resources from DND as well as the volunteer aeronautical network known as the Civil Air Search and Rescue Association, CCGA volunteer response units, and vessels of opportunity.

The CCG SAR Program mandate is derived from several Acts and international conventions. The Constitution Act, 1867 gives the Parliament of Canada exclusive legislative jurisdiction over matters of navigation and shipping. With respect to Coast Guard services, the Oceans Act (S.C. 1996, c. 31) states that the powers, duties and functions of the Minister of the Department of Fisheries and Oceans extend to the maritime component of the federal search and rescue program. Under the Canada Shipping Act, 2001 (S.C. 2001, c. 26), the Minister may designate persons as Rescue Coordinators to organize search and rescue operations. Finally, through several international conventions, Canada has committed itself to making adequate arrangements for maritime search and rescue within its area of responsibilityFootnote 6.


The CCG SAR Program includes the CCG Search & Rescue Coordination and Response sub-program and, the CCGA sub-program, described below.

2.1.1 Search & Rescue Coordination and Response

The CCG Search & Rescue Coordination and Response sub-Program provides coordination and response services to increase the likelihood of survival of people at risk of loss of life or injury in the maritime environment, ensuring they have access to assistance by: coordinating and delivering on-water response to maritime search and rescue cases; supporting the safety of life at sea; assisting the Department of National Defence in response to aeronautical and humanitarian cases; providing search and rescue response capacity; and managing partnerships essential for the efficient coordination of activities. Through communication, coordinating, and delivering maritime search and rescue response, and operational awareness, the CCG SAR Program increases the chances of rescue for people caught in dangerous on-water situations.

2.1.2 Canadian Coast Guard Auxiliary

The CCGA is organized into five federally incorporated, not-for-profit volunteer organizations. The Minister of Fisheries and Oceans maintains formal contribution agreements with each of the five regional CCGA corporationsFootnote 7. The corporations are separate legal entities from the Government of Canada and work in close partnership with the CCG. The majority of CCGA members are commercial fishers and pleasure boaters who donate their time and vessels to assist the CCG SAR Program. The remaining CCGA members are volunteers from local communities who crew community-based, dedicated response vessels 24 hours a day, 7 days a week. As of fiscal year 2015-16, the CCGA has approximately 4,000 members and access to approximately 1,100 vessels.


2.2 Partners, Enablers and Vessels of Opportunity

In addition to the collaboration of the CCGA, the CCG SAR Program is delivered with the support from a mixture of partners and enablers. More specifically, the aeronautic needs for maritime search and rescue are met by the Royal Canadian Air Force (RCAF); whereas, its on-water and marine related response needs (i.e. certificated professionals, vessels, Air Cushioned Vehicles, helicopters as well as small craft) are provided by the Coast Guard's Fleet Operational Readiness Program. Moreover, all federal on-water resources (exclusive of primary resources) are considered secondary resources with search and rescue taskings taking priority over all other planned activities. Lastly, vessels of opportunity are any other vessel not mentioned above, close enough to provide assistance to a vessel in distress. Under the Canada Shipping Act and international law, every vessel at sea is required to assist in a maritime distress situation.

Another integral contributor to the delivery of CCG SAR Program is the CCG Marine Communications and Traffic Services (MCTS) Program. MCTS is responsible for the detection of maritime search and rescue incidents and provides a critical communications link between vessels in distress and the JRCCs/MRSC. This information is essential to deploying search and rescue teams.

2.3 Program Resources

Table 1 provides the 2015-2016 financial and human resource details for the CCG SAR Program.Footnote 8

Table 1 - CCG SAR Program Resources
2015-16 Financial Resources ($ millions)
  Planned Authorized Available Actual Difference
Total $30,508,166 $31,104,608 $31,104,607 $596,441
2015-16 Human Resources (FTEs)
  Planned Actual Difference
Total 134.0 123.3 10.7

Source: 2015-16 DFO Departmental Performance Report

Table 1 provides the 2015-2016 financial and human resource details for the CCG SAR Program. (This does not include the resources for the certificated crew and the CCG platforms made available by the CCG Fleet Operational Readiness Program) The table has two rows. The first row are the Financial Resources and lists the planned spending, financial authorities available for use, actual amount spent as well as the difference between planned and actual spending. The second row are the Human Resources details for the program and lists the planned, actual and difference. Read across the row to learn the relevant information.


2.4 Logic Model and Performance Measurement

The logic model that guided the evaluation is attached as Annex A. It provides an overview of the linkages between the various levels of outcomes. A performance measurement strategy was developed in 2012 and recently updated in March 2015. As of winter 2016, the strategy is being implemented and the associated data was being collected and analyzed.

3.0 EVALUATION METHODOLOGY

3.1 Evaluation Approach and Design

This evaluation adopted a theory-based approach whereby program performance was measured against intended results articulated in the program logic model. The chosen design was able to demonstrate the extent to which the program is achieving issues of relevance, effectiveness and efficiency, in line with the Treasury Board’s Policy on Results. Extensive use of triangulation was undertaken as an analytical method, where multiple lines of evidence helped corroborate findings. This involved a combination of key informant interviews and a review of administrative data to assess if the program had the level of inputs needed to support the coordination and response of maritime search and rescue incidents. The analysis included an examination of the actual response(s) tasked to the search and rescue incident based on its type and severity (i.e. M1, M2, M3 and M4)Footnote 9. In addition, the evaluation considered the ability of the CCG to manage the costs related to both the coordination of and response to maritime search and rescue incidents; and, the degree to which the search and rescue services that are provided were cost efficient.

Annex B provides the detailed evaluation matrix. The evaluation questions were determined subsequent to preliminary discussions with CCG senior management where a need to examine the following issues was identified:

  • Preparedness in terms of the on-water presence for CCG search and rescue & non-search and rescue assets;
  • CCGA preparedness and availability;
  • Frequency of rescue specialist certification exemptions for CCG seagoing personnel;
  • Investigating the trend in maritime search and rescue incidents by type of vessel/search object;
  • Following-up on management actions developed by CCG in response to the recommendations in the last evaluation completed in 2011-12; and,
  • Effectiveness and efficiency.

3.2 Data Sources

The evaluation utilized existing administrative and financial data, and where required supplemented with additional data sources. The details of the sources used are described more fully below:

  • Financial data was provided by the Financial Management Advisors situated within CCG;
  • Administrative data was sourced from CCG databases (e.g. Search and Rescue Program Information System), program documents and other departmental publications;
  • Documents reviewed included various materials obtained from CCG national and regional offices, CCGA business plans as well as previous DFO evaluation and audits, and other federal government and external sources; and,
  • Key informant interviews were held with fifty-three respondents: CCG senior management (n=16); CCG NHQ (n=8), CCG Region (n=12), Enablers (n=7), Partners (n=10). These included interviews with the CCG Commissioner, Deputy Commissioner, Assistant Commissioners, Director Generals, Directors, and Search and Rescue Superintendents or senior managers in each region including National Headquarters. The majority of headquarter interviews were conducted in person, and with regional interviews by phone.

3.3 Methodological Limitations and Mitigation Strategies

Although the evaluation encountered some challenges and limitations that are outlined below, these limitations were mitigated, as much as possible, through the use of multiple lines of evidence and triangulation of data. This approach was taken in order to demonstrate reliability and validity of the findings and to ensure that conclusions and recommendations are based on objective and documented evidence. See Table 2 for details on limitations and mitigations.

Table 2 Methodological Limitations and Mitigation Strategies
LimitationsMitigaiton Strategies
Although large amounts of data regarding maritime search and rescue was available from various databases, frequently “other” and/or “blank” categories were used to classify the information. Considering the size of the data in these categories, this created a challenge when analyzing trends across all categories. For example, as part of the analysis of incidents by vessel type, the “other” category was second only to pleasure craft. Considering this data can represent a large portion of the information within the various databases, the evaluation team had considered, but rejected, the idea of excluding these categories. The evaluation team instead mitigated this limitation by retaining this data and working closely with the CCG program to validate charts and tables prior to their use in the evaluation.
As part of its efforts to calibrate the scope of the evaluation, the evaluation team conducted limited analysis of CCG financial data when analyzing the CCG use of lower cost service delivery mechanisms tasked in response to a maritime search and rescue incidents. The main strategy to mitigate the limitation was to expand the input to a wider variety of key informants as well as to use the results from a previous evaluation and document review.

Table 2 provides the methodological limitations and mitigation strategies. The first column are the two limitations whereas the second column are the corresponding mitigation strategies.

4.0 FINDINGS

4.1 Need

Key Finding: As part of its mandated role to detect, coordinate and deliver maritime search and rescue responses, there is a continuing need for the CCG SAR Program as demonstrated through the steady number of maritime incidents.

CCG is responsible for the detection of maritime search and rescue incidents and the delivery of a maritime search and rescue response in coordination with its partners. During the 5-year time frame of the evaluation, the CCG coordinated responses to approximately 7,000 maritime search and rescue incidents per year for a total of just over 35,000.Footnote 10 Of these, CCG primary and secondary search and rescue vessels were tasked to respond to approximately 47% of all maritime search and rescue incidents; whereas the CCGA were tasked to respond to 24%. See Table 3 for more details for an overall count of search and rescue incidents.


Table 3 All classifications of Search and Rescue Incidents by ResponderFootnote 11
2011 2012 2013 2014 2015 5 YR AVG
Candian Armed Forces (CAF) 957 1011 1071 961 962 992
Civil Air Search and Rescue Assoication (CASARA) 162 129 148 155 120 143
Canadian Coast Guard 4336 4577 4050 3881 4058 4180
Candian Coast Guard Association (CCGA) 1941 1962 1732 1831 2085 1910
Other Federal 57 41 70 68 70 61
Vessels of Opportunity 1751 1959 1937 2068 2022 1947

Source: SISAR database

This table provides details on the classifications of search and rescue incidents by responder from calendar year 2011 to calendar year 2015 as well as the overall 5-year average. There are seven rows. The headings for the calendar years are the first row. The second row is the responder data for the Canadian Armed Forces, the third is the responder data for Civil Air Search and Rescue Association, the fourth is responder data for the Canadian Coast Guard, the fifth is responder data for the Canadian Coast Guard Auxiliary, the sixth is responder data for the Other Federal category and the seventh row is the responder data for vessels of opportunity. Read across each column to learn the number of incidents by responder.


4.2 Effectiveness

Key Finding: The CCG and CCGA are both prepared and available to assist people in need of help in the maritime environment and lives are being saved.

Between 2011-12 and 2015-16 there has been upward trend in the percentage of lives saved in the maritime environment (Graph 1). In 2015-16 the percentage of lives saved was measured at 99.16% and represents a 0.84% increase with the number of maritime search and rescue incidents relatively static over the same 5-year period.Footnote 12

Graph 1

graph1

Source: SISAR database

Graph 1 is a bar graph illustrating the percentage of lives saved versus lives at risk per year. The percentage of lives saved versus lives at risk in 2011-12 was 98.32%, in 2012-13 it was 98.96%, in 2013-14 it was 98.71%, in 2014-15 it was 98.93%, and in 2015-16 it was 99.16%.


In addition, evidence demonstrated that primary CCG search and rescue vessels are achieving a reaction time of 30 minutes or less in 97% of incidents.Footnote 13 This represents an improvement from the previous evaluation where the CCG SAR Program service standard was met for 96% of the incidents between 2006 and 2010. Graph 2 illustrates the average CCG reaction time across all types of maritime search and rescue incidents.

Graph 2

graph2

Source: SISAR database

Graph 2 is a bar graph showing the average reaction time for the Canadian Coast Guard calculated in minutes for maritime SAR incidents. The targeted reaction time is 30 minutes or less. CCG is achieving this 97% of the time.


CCG Maritime Search and Rescue Presence

Nationally the numbers of planned days by CCG in support of the CCG SAR Program are being met; however, certain risks were identified in terms of maritime search and rescue presence at the regional level. More specifically, there are risks with search and rescue coverage in the off-shore zones for the Atlantic Region. Graph 3 demonstrates that large vessels, which have the capacity to patrol in the off-shore environment, have experienced a decrease in both days planned and days delivered. This decrease is attributable to a combination of planned and unplanned maintenance for large vessels in the Atlantic region. The CCG is mitigating this risk by calling upon its partners, such as the Royal Canadian Navy, to assist in alleviating the coverage risks.

Graph 3

graph3

Source: iFleet and MariTime databases

Graph 3 is a bar graph showing the days planned versus days delivered for Canadian Coast Guard large vessels in the Atlantic Region. The days planned in 2012-13 were 1,609 whereas the days delivered were 2,116; the days planned in 2013-14 were 1,587 and the days delivered were 1,748; the days planned in 2014-15 were 1,574 and the days delivered were 1,879; and, the days planned in 2015-16 were 1,379 and days delivered were 1,252.

CCGA is Prepared and Available

The CCGA continues to be an important contributor to maritime search and rescue through the provision of thousands of volunteer hours and vessels. Evidence from administrative data and key informants noted that the CCGAs were largely available when needed resulting in a national average of 24% involvement in responses to maritime search and rescue incidents (i.e. although actual regional levels vary significantly)Footnote 14 . A similar result was noted in the 2012 Search and Rescue evaluation. In the current evaluation, what key informants noted frequently was the adaptability of auxiliary model in how it operates across the country ranging from the owner/operation model in the Newfoundland and Labrador Auxiliary Region where over 90% of the membership are involved in the fishery to the Royal Canadian Marine Search and Rescue in the Pacific which mostly operates community boats from more than 40 marine rescue stations.

Graph 4 demonstrates that the number of trained members were largely stable between 2013 and 2016; however, data on the longer historical trend has shown an overall decrease in both members and vessels. These risks are known and similar concerns were raised in the 2007 SAR Needs AnalysisFootnote 15 as well as in the 2012 Search and Rescue evaluation. Key informants noted concerns regarding the capacity of CCGAs to maintain or replenish the volunteer base and that these issues are further intensified when the retirement of CCGA members also entails the withdrawal of an owner-operator auxiliary vessel. Finally, 15% of the CCGA taskings are M1 or M2 type of incidents where there is distress or potential distress.

Graph 4

graph4

Source: CCGA Annual Business Plans

Bar Graph 4 shows the number of trained Canadian Coast Guard Auxiliary members over a 3-year period between 2013-14 to 2015-16. The first group of bars is for the Maritime Auxiliary Region where there were 831 trained members in 2013-14. In 2014-15 and 2015-16 there were 806 trained members in each fiscal year. The second group of bars is for the Central and Arctic Auxiliary Region where there were 754 training members in 2013-14. In 2014-15 there were 784 trained members whereas there were 811trained members in 2015-16. The third group of bars is for the Newfoundland and Labrador Auxiliary Region where there were 715 members in 2013-14. In 2014-15 there were 721 trained members and 726 trained members in 2015-16. The fourth group of bars is for the Quebec Auxiliary Region where there were 668 members in both 2013-14 and 2014-15. In 2015-16 there were 589 trained members. The final group of bars is for the Royal Canadian Marine Search and Rescue (i.e. Pacific Auxiliary Region) where there were 1,062 trained members in 2013-14. In 2014-15 there were 1,073 trained members and 1,019 trained members in 2015-16.


Certification Exemptions for CCG Seagoing Personnel

Key Finding: There has been an increase in certification exemptions for Rescue Specialists.

The CCG Rescue Specialist program was developed in response to needs for a higher level of rescue capability and emergency pre-hospital care within the CCG Fleet. In order to be considered a "Rescue Specialist", CCG personnel must meet specific training requirements such as the "Medic "A" course. The Rescue Specialist certificate is a voluntary service performed in addition to regular responsibilities. As outlined in CCG Fleet Order # 535, it is a CCG internal policy to carry rescue specialist capabilities onboard all CCG vessels having a crew complement of four or more. Over the last five years, the number of operational days where exemptions for rescue specialists were sought doubled from 4% to 8% (see graph 5).Footnote 16 In 2015 this represented 2,668 operational days.

Graph 5

graph5

Source: Seagoing Personnel Certification Exemption Tracking System and iFleet

Graph 5 is a line graphing showing the percentage of operational days requiring rescue specialist exemptions from 2011 to 2015. In 2011, 4.2% of the operational days required exemptions. In 2012, 4.9% required exemptions. In 2013, 10.3% required exemptions. In 2014, 5.4% and in 2015, 8.3% of the operational days required rescue specialist exemptions.

Key informants and evidence from documents confirmed that new rescue specialists are being trained and many of the existing rescue specialists are being re-certificated. Despite this, the CCG continued to have challenges achieving the crewing levels. Key informants noted part of the challenge is not having clearly defined targets for the pool of trained specialists. Nonetheless, interviewees voiced that a solution would be required in order to avoid possible negative impacts on the preparedness of the CCG SAR Program of not having the required seagoing personnel onboard at all times. Moreover, several key informants noted the increased efficiency of having a rescue specialist onboard. For example, for those maritime SAR incidents where a RCAF search and rescue helicopter with a DND rescue technician is tasked, the CCG rescue specialist provides the initial support and then ensures that the patient is ready for transport.


National Exercise Plan

The evaluation undertaken in 2012 found that the CCG SAR Program did not have a national standard or approach for conducting large scale exercises. Consequently, a recommendation was made for CCG to develop a national standard or approach for conducting large scale exercises including the development of a process for the production and dissemination of After Action Reports. Through its management response, CCG committed to determining the number of annual exercises needed (both large on-the water exercises and tabletop) in order to facilitate a more national approach to exercise planning and reporting on lessons learned. Key informants and document review confirmed that, although exercises did occur between 2011 and 2016, there was not yet a national plan in place. The same key informants noted the importance of exercises in preparing for a maritime incident; however, the number of exercises, how they should be conducted, and with whom, had not yet been identified or standardized nationally.

Although not yet approved, a draft version of a national exercise plan was circulating within the CCG as of fall 2016. Updates on the status of this management response are provided regularly by the CCG to the DFO Performance Measurement and Evaluation Committee. The most recent update noted a reorganization within the group, loss of staff and emerging priorities all having an impact on the completion of the national plan.

4.3 Efficiency

Key Finding: The CCG SAR Program is operating in an economical and efficient manner to support the achievement of results.

As part of the evaluation the planned and actual financial resources and FTEs were analyzed. The second part of the analysis examined search and rescue coverage concerning the optimization of available resources when tasking a response to a maritime search and rescue incident. Lastly, the analysis of efficiency examined the use of lower cost service delivery mechanisms when responding to an incident.

Financial Information

The CCG SAR Program expenditures from 2011-12 to 2015-16 which includes expenditures for both CCG and CCGA funding are outlined in Table 4. Overall, funding remained stable and was largely within the authorities available for use. As noted earlier, lives are being saved, people in need of help are being assisted and CCG and CCGA are both prepared and available. Considering that these results are being achieved in the context of reduced expenditures the evaluation determines that the CCG SAR Program is achieving its expected outcomes in an economical manner.

Table 4
Financial Resources ($ millions) Human Resources (FTEs)
Year Planned Authorities
Available for
Use
Actual Variance from
Planned
Planned Actual Difference
2011-12 $32,524,378 $33,181,183 $37,695,431 -$5,171,053 124 127 3
2012-13 $32,930,626 $35,488,840 $34,547,572 -$1,616,946 170 148 -22
2013-14 $32,213,522 $35,151,917 $34,756,121 -$2,542,599 104 125 21
2014-15 $29,066,000 $34,248,198 $33,990,806 -$4,924,806 106 127.7 21.7
2015-16 $30,508,166 $31,104,608 $31,104,607 -$596,441 134 123.2 -10.8

Note: Costs associated with the certificated crew and operating vessels to support the delivery of this program are not included above. These costs are reported separately and are assumed by CCG Fleet Operational Readiness

Table 4 provides details on both the financial and human resources from fiscal year 2011-12 to fiscal year 2015-16. There are five rows, one for each fiscal year. There are five columns for Financial Resources. Read across each column to learn the planned spending, authorities available for use, actual spending, and variance from planned. There are three columns for Human Resources. Read across each row to learn the planned versus actual levels. Read each row for each fiscal year.

Search and Rescue Coverage

Key Finding: The CCG SAR Program is optimizing the use of available resources by adjusting the resources tasked to meet the needs and severity of the maritime SAR incident.

SAR coverage is the ability of a maritime search and rescue resource to arrive in time to save a life. Key informants noted that a focus of the CCG SAR Program is to ensure that this is accomplished with maximum efficiency. A key aspect of achieving this is the ability to scale the response to a maritime search and rescue incident based on the severity of the event. More specifically, the CCG SAR Program is optimizing the use of available resources from the range of response units (e.g. CCG primary and secondary search and rescue, DND aeronautical, CCGA and vessels of opportunity) and adjusting the resources tasked to meet the needs and severity of the incident. Table 5 confirms that over the 5-year period covered by the evaluation the tasking of multiple resources to an incident was reserved for the most severe distress incidents.

Table 5: Median number of responders per incident 2011-12 to 2015-16
M1 M2 M3 M4
Atlantic 2 1 1 1
Central and Arctic 3 1 1 1
Western 2 1 1 1

SISAR database

Table 5 provides details on the median number of responders tasked per search and rescue incident from 2011-12 to 2015-16. There are five columns. The first column lists each of the three CCG regions. The remaining four columns are the classifications for the maritime search and rescue incidents. Read across each row to learn the median number of responders tasked per incident.

Use of lower cost service delivery mechanisms

Key Finding: Savings for the overall delivery of maritime search and rescue services are being realized through the use of volunteers and seasonal employees.

The evaluation team included input from a variety of key informants and documents as well as used the results from a previous evaluation as a means to complete its analysis of lower cost service delivery mechanisms.

Inshore Rescue Boat Service

Inshore Rescue Boat (IRB) Service is a seasonal dedicated, primary search and rescue resource available for tasking by the JRCC or a MRSC. An IRB unit may also be tasked by the Regional Operations Centre, when necessary, for non-search and rescue taskings. A review of CCG administrative data confirmed that trained summer students make up 86% of the IRB crew and each year more than 97 candidates are hired as members of an IRB crew through the Federal Student Work Employment Program. There are 25 IRB stations located across the 3 CCG regions. Each station is equipped with a six to eight metre Fast Rescue Craft capable of operating at speeds in excess of twenty four knots. In 2013, the CCG partnered with the Royal Canadian Navy to allow naval reservists to be assigned to the IRB Service as part of their career training and development.

Key Informants noted that the IRB Service provides an important, incremental increase to search and rescue coverage in areas of high traffic density during the peak periods for recreational boating. The operational season for the IRB Service starts mid-to-late May and ends early September. Interviewees also expressed opinions that without the IRB Service it would be challenging for the CCG and its partners to meet the surge in maritime search and rescue incidents that occur in the summer months. Administrative data also confirmed that the IRB Service represents approximately 18% of CCG’s overall number of maritime search and rescue taskings but increases to roughly 30% during the peak months of June, July and August (see table 6).

Table 6 - IRB Tasking during June, July and August
% IRB of Surge 2010 2011 2012 2013 2014
June 42% 37% 41% 47% 32%
July 30% 30% 31% 34% 33%
August 27% 31% 28% 33% 31%

SISAR database

Table 6 provides details on tasking to the Canadian Coast Guard Inshore Rescue Boat Service from 2010 to 2014 during a 3-month period (i.e. June, July and August). There are six columns. The first is the months. The remaining four columns are one for each of the calendar years. Read across each row to learn the percentage of tasking to the Inshore Rescue Boat Service per incident by each fiscal year.

Canadian Coast Guard Auxiliary

CCGAs receive approximately $5.5 million annually in contributions. Key informants and administrative data confirmed that the CCG SAR Program leverages close to 4,000 volunteers and approximately 1,100 vessels in 5 CCGA regions. Evidence from the 2012 evaluation of the CCG SAR Program confirmed that the CCGA sub-program represents significant cost avoidance in terms of human resources and assets. More specifically, every $1 expended, resulted in a cost avoidance of approximately $43.Footnote 17 The same evaluation estimated that without the support of the CCGA, the CCG would need to spend some $337 million to purchase a similar fleet of vessels and an additional $200 - $300 million annually in salaries and benefits. Graph 6 represents the tasking to CCGA regions over a 3-year period.

Graph 6

graph6

Source: SISAR Database

Graph 6 is a line graph showing the percentage of Canadian Coast Guard Auxiliary taskings from 2013-14 to 2015-16. For the Royal Canadian Marine Search and Rescue (i.e. Pacific CCGA) in 2013-14 it was 30% and in 2014-15 it was 32% whereas for 2015-16 it was 33%. For the Central and Arctic Region in 2013-14 it was 13% and 2014-15 it was 12% whereas for 2015-16 it was 14%. For the Quebec Auxiliary in 2013-14 it was 16% and 2014-15 it was 18% whereas for 2015-16 it was 22%. For the Maritime Auxiliary in 2013-14 it was 21% and in 2014-15 it was 16% whereas for 2015-16 it was 21%. For the Newfoundland and Labrador Auxiliary in 2013-14 it was 16% and in 2014-15 it was 22% whereas for 2015-16 it was 15%.


4.4 Critical Success Factors and Key Barriers

Key Finding: The number and severity of maritime SAR incidents involving pleasure craft is increasing.

Workload: Maritime Search and Rescue Incidents (type of vessel/search object)

Maritime search and rescue incidents have various combinations of causes, severity and vessel types. However, it is pleasure craft that have historically represented the largest, single search object category. More specifically, from April 2011 to March 2016, there were just over 21,000 maritime search and rescue incidents involving pleasure craft; whereas, those involving fishing and commercial vessels were approximately 1,800 and 225 respectively. The “other” and “blank” categories combine for a 5-year total of approximately 2,700.Footnote 18 Key informants noted that during the timeframe between 1993 to present day, safety and prevention strategies existed through the Office of Boating Safety. Irrespective of where in federal government these prevention and boating safety programs were located, evidence from the evaluation demonstrates that the number and severity of incidents resulting from pleasure craft continued to rise whereas those involving commercial and fishing vessels types are decreasing (see Graph 7, 23-year trend in maritime search and rescue incidents).


Graph 7

graph7

Source: SISAR Database

Graph 7 is a line graph showing the 23-year trend in the number of maritime search and rescue incidents by search object from April 1993 to March 2016. There are seven categories of search objects including air, commercial, fishing government, other, pleasure and blank. The “Pleasure” craft category is the most frequent search object followed by “other”, “fishing”, “blank”, “commercial” and “government".

In 2015-16, pleasure craft represented 70% of all responses to M1 incidents, 80% of M2, 70% of M3 and 45% M4. Moreover, search objects stemming from pleasure boating activity have increased from 47% of incidents in 1993-94 to 54% in 2015-16. A recent evaluation conducted by Transport Canada noted that there is a large recreational boating population in Canada and estimated there were approximately 10.5 million Canadians who boated at least once in the past year.Footnote 19 In the same evaluation Transport Canada determined that there was no measureable evidence of the impact on boating safety attitudes, behaviour or overall awareness about boating safety. Moreover, search and rescue coordinators observed in 2014 that there were still prevention and awareness issues as evidenced by the lack of use of a life-preserver and lack of proper sailing plans.Footnote 20

Search and rescue is a last resort when safety and prevention measures fail. Key informants noted that addressing the number of maritime search and rescue incidents involving pleasure craft by increasing the priority of safety and prevention strategies could possibly assist in reducing both the number of lives lost as well as the overall number of maritime distress calls.


5.0 CONCLUSIONS AND RECOMMENDATION

5.1 Conclusions

Need

As a result of its mandate to detect, coordinate and respond, there is a continuing need for the CCG SAR Program as evidenced by the steady number of maritime search and rescue incidents. More specifically, the CCG is the recognized federal authority to coordinate maritime search and rescue services and is a participant to various international treaties, conventions and agreements. Consequently, there are requirements that Canada must make adequate arrangements for maritime search and rescue within its area of responsibility.

Effectiveness

Evidence demonstrates that the CCG is successful in the detection of maritime incidents as well as the delivery of maritime search and rescue responses (i.e. in coordination with its partners) in areas of federal responsibility. Overall, lives are being saved with the CCG and CCGA each being both prepared and available to assist people in need of help in the maritime environment. Known risks to search and rescue coverage are being mitigated in collaboration with search and rescue partners. More specifically, in the CCG Atlantic Region where the number of days planned and days delivered have decreased for large vessels, the CCG is mitigating this situation by calling upon the Royal Canadian Navy to assist in alleviating the coverage risks.

Over the last five years, the number of operational days where exemptions for rescue specialists were sought doubled (from 4% to 8%), representing a total number of 2,668 operational days in 2015. The CCG Rescue Service provides a higher level of rescue capability and emergency pre-hospital care within the Coast Guard Fleet. There is evidence that new rescue specialists are being trained and many existing rescue specialists are being re-certificated. Nonetheless the CCG is still not achieving its crewing levels. Consequently, a solution would be required in order to avoid possible negative impacts on the preparedness of the CCG SAR Program for not having the required rescue specialist personnel onboard CCG vessels.

As a final point, evidence from the evaluation highlights the important contribution of the CCGA to the maritime elements of federal search and rescue system. The CCGA contributes to the overall efficiency as a result of its volunteers and vessels by handling 24% of maritime SAR taskings. It also contributes to overall effectiveness by providing response to 15% of incidents where there is distress (M1) or potential distress (M2). Although the number of volunteers and vessels has largely stabilized within the timeframe of evaluation, key informants raised concerns over potential implications to the availability of both CCGA members and vessels should there be similar decreases in the future.

Efficiency

There is evidence that the CCG SAR Program is managing costs through its efforts to optimize search and rescue resources when tasking a response to a maritime search and rescue incident as well as through its use of lower cost service delivery mechanisms. This combined with evidence that the program funding remained stable and was increasingly in-line with planned spending, confirms the program is operating in an economical and efficient manner.

A significant obstacle to the maritime search and rescue system achieving even greater efficiency are the high number of incidents involving pleasure craft. Unlike other on-water users (e.g. commercial & fishing) where there have been reductions in the number of incidents, the pleasure craft group not only represents the single largest search object category, but the number and severity of incidents are also continuing to rise. An evaluation completed by Transport Canada confirmed that a large recreational boating population exists in Canada but that prevention efforts are likely being limited by the fact that there is a relatively small federal boating safety program. In the same evaluation Transport Canada determined that there was no measureable evidence of the impact of current federal efforts on boating safety attitudes, behaviour or overall awareness about boating safety.

5.2 Recommendation

As a result of the findings in the evaluation, one recommendation is made concerning the preparedness and availability of the CCG Rescue Specialist service. Annex C presents the Management Action Plan and identifies how the program will address the recommendation.

The recommendation acknowledges that the CCG SAR Program is aware of the increase in certification exemptions for CCG Seagoing Personnel with regard to rescue specialists. Similar concerns were noted in the Search and Rescue Needs Analysis, 2007 and the Canadian Coast Guard Search and Rescue and Canadian Coast Guard Auxiliary Evaluation Report 2012. In addition, the spring 2013 Auditor General of Canada report on Federal Search and Rescue Activities noted the CCG had difficulty maintaining an adequate number of rescue specialists. The recommendation is intended to ensure that the CCG is both prepared and available to provide advanced-level first aid and/or offshore survival and rescue techniques. Considering that the certificated crew and the CCG platforms are made available to the CCG SAR Program by the CCG Fleet Operational Readiness Program, any response to recommendation would need to be developed collaboratively between the two.

Recommendation 1: CCG should develop and implement a strategy aimed at meeting the required ship’s crewing profile outlined in the Canadian Coast Guard Fleet Order #535 (i.e. Minimum number of Designated Rescue Specialists on board Canadian Coast Guard Ships with a complement of four or more).

ANNEX A: PROGRAM LOGIC MODEL


graph8

Annex A shows the logic model which outlines the logical flow from the immediate outcome of the program to its long term outcome. The outcomes are connected by arrows. The immediate outcome is “The maritime component of the federal Search and Rescue system is prepared and available to respond”. The intermediate outcome is “People in need of help in the maritime environment are assisted”. The long term outcome is “Loss of life to people in maritime distress is minimized”.

ANNEX B: EVALUATION MATRIX


ISSUE/QUESTIONS INDICATOR LINE OF ENQUIRY
    INTERVIEWS DOCUMENT & FILE REVIEW ADMINISTRATIVE DATA FINANCIAL DATA SISAR DATABASE
   
SAR
MANAGEMENT
PARTNERS &
STAKEHOLDERS
       
RELEVANCE
1. Is there a continued need for SAR?
  • Continuing need for CCGA transfer payment
  • Opinions of SAR Management, Partners& Stakeholders
2. Is SAR aligned with Government of Canada and DFO priorities?
  • Degree of alignment of SAR with Government of Canada objectives and priorities
  • Degree of alignment of SAR with DFO objectives and priorities
3. Is SAR aligned with federal roles and responsibilities?
  • Evidence of alignment between SAR’s objectives and the legislated authorities
PERFORMANCE
EFFECTIVENESS
4.1 To what extent is the CCG component of the federal SAR system prepared and available to respond?
  • Number of days delivered vs. planned, by vessel and region (2011-12 to 2015-16)
  • % of time that a CCG vessel did not have required number of certificated rescue specialist(s) as per CCG Orders (2011-12 to 2015-16)
4.2 To what extent are CCGA members prepared and available to respond to maritime SAR incidents?
  • Number of trained CCGA members and number of CCGA vessels (2011-12 to 2015-16)
  • Opinions of SAR program managers, Partners and Stakeholders
4.3 To what extent are people in need of help in the maritime environment assisted?
  • % of primary search and rescue vessels meeting reaction time of 30 minutes or less for maritime incidents (2011-12 to 2015-16)
  • % of maritime Search and Rescue responses by CCGA relative to the total number of Maritime SAR taskings (2011-12 to 2015-16)
  • Opinions of SAR program managers, Partners and Stakeholders
4.4 To what extent is loss of life to people in maritime distress minimized?
  • % of lives saved relative to total reported lives at risk in the maritime environment (2011-12 to 2015-16)
  • Opinions of SAR program managers, Partners and Stakeholders
RESOURCE UTILIZATION
5. Is the CCG optimizing the use of available resources when tasking a response to a SAR incident?
  • Breakdown of taskings by incident classification (M1, M2, M3, M4)
  • Key informant and opinions
6. Could the efficiency of SAR activities be improved?
  • Maritime SAR incidents (type of vessel/search object)
  • Key informant and opinions
OTHER
7. Are there any external factors and/or challenges that may have impacted the results of the program? Did the program have any unintended results?
  • Opinions of key informants
  • Evidence from documents

This table presents the evaluation issues, questions, indicators and data sources. Evaluation issues and questions are listed in the first column under the headings of Relevance, Performance and finally Resource Utilization. Indicators aligning with the issues and questions are listed in the second column and the sources for each line of inquiry (i.e. interviews, document & file review, administrative data, financial data and Search and Rescue Program Information System) are listed in the third column. Read across each row to learn the details of each evaluation issue and question by corresponding headings.

 

ANNEX C: MANAGEMENT ACTION PLAN


RECOMMENDATION

Rationale: Over the last five years, the number of operational days where exemptions for rescue specialists were sought doubled from 4% to 8% representing a total number of 2,668 operational days in 2015. The rescue specialist program was developed in response to needs for a higher level of rescue capability and emergency pre-hospital care within the CCG Fleet. As outlined in CCG Fleet Order # 535, it is a CCG internal policy to carry rescue specialist capabilities onboard all CCG vessels having a crew complement of four or more. Although there is evidence that new rescue specialists are being trained and many existing rescue specialists are being re-certificated, the CCG is still not achieving its crewing levels.

The recommendation acknowledges that the CCG SAR Program is aware of the current increase in certification exemptions for CCG Seagoing Personnel with regard to rescue specialists. Similar concerns were noted in the Search and Rescue Needs Analysis, 2007 and the Canadian Coast Guard Search and Rescue and Canadian Coast Guard Auxiliary Evaluation Report, 2012. In addition, the spring 2013 Auditor General of Canada report on Federal Search and Rescue Activities noted the CCG had difficulty maintaining an adequate number of rescue specialists. The recommendation is intended to ensure that the CCG is both prepared and available to provide advanced-level first aid and/or offshore survival and rescue techniques. Considering that the certificated crew and the CCG platforms are made available to the CCG SAR Program by the CCG Fleet Operational Readiness Program, any response to recommendation would need to be developed collaboratively between the two.

Recommendation 1: CCG should develop and implement a strategy aimed at meeting the required ship’s crewing profile outlined in the Canadian Coast Guard Fleet Order #535 (i.e. Minimum number of Designated Rescue Specialists on board Canadian Coast Guard Ships with a complement of four or more).

STRATEGY

CCG is working toward the Rescue Specialist position going from voluntary to mandatory. CCG expects this will reduce the number of exemptions required.



MANAGEMENT ACTIONS DUE DATE (BY END OF MONTH) STATUS UPDATE: COMPLETED / ON TARGET / REVISED DATE AND REASON FOR CHANGE OUTPUT
Create a work description that integrates the Rescue Specialist duties into an appropriate position. July 2016 Completed
Identify with Regions the issues and circumstances causing the need for the exemptions and develop options available to reduce them. May 2017
Determine CCG’s ability to make more employees available for both the initial and re-certification training while continuing initial and re-certification training for rescue specialists. May 2017
Develop the combined positions for the Rescue Specialist duties. October 2017 National Work Description was sent in to Classification on July 5, 2016 for review. Once the position is classified, regional positions will be created as part of the vessel complement. This will eliminate the voluntary aspect for this position and make it a permanent status on the Crewing Profile. Union may need to be consulted once the positions are added to the vessel crewing profile. Develop the combined positions for the Rescue Specialist duties.

Annex C provides details on the management action plan. The plan presents the strategies and actions that the program intends to implement in response to the recommendations. There is one recommendation in the evaluation. Read across each row to learn the justification for the recommendation followed by a brief overview of the approach the program will take to address the one recommendation. The remaining information includes a specific deadline for the program to complete each task along with status update on the actions taken and a list of outputs developed by the program in response to the recommendation (e.g. terms of reference, work plan, strategy proposal, policy changes, a report or other item).