Archived – Evaluation of the Icebreaking Program

Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Project Number 6B123
Final Report (Short Version)
February 15, 2011

Table of Contents

List of Acronyms

AFB
Air Force Base
BNA
British-North America
C&A
Central and Arctic
CCG
Canadian Coast Guard
CCGAPS
Canadian Coast Guard Automated Performance Solution
CCGS
Canadian Coast Guard Ship
CGS
Canadian Government Ship
DFO
Fisheries and Oceans Canada
EC
Environment Canada
IB Program
Icebreaking Program
IISPA
Ice Information Service Partnership Agreement
IODIS
Icebreaking Operations Data Information System
LOS
Level of Service
MCTS
Marine Communications and Traffic Services
MOU
Memorandum of Understanding
MRRS
Management, Resources, and Results Structure
MSC
Military Sealift Command
NHQ
National Headquarters
PAA
Program Activity Architecture
PMF
Performance Measurement Framework
RPP
Report on Plans and Priorities
RCMP
Royal Canadian Mounted Police
ROC
Regional Operations Centre
SAR
Search and Rescue
SLSA
St. Lawrence Seaway Authority
SOLAS
Safety of Life at Sea
UNCLOS
United Nations Convention on the Law of the Sea
US or USA
United States of America


Executive Summary

Evaluation Objective

The main objective of this evaluation is to determine to what extent the Icebreaking Program (IB Program) is managed effectively and efficiently and whether it had achieved its stated objectives and results. As such, the evaluation examines the extent to which the IB Program demonstrates Value for Money or relevance and performance (effectiveness, efficiency and economy). 

The timeframe for this evaluation, a first for the Icebreaking Program, covered the period from 2005-06 to December 31, 2009 and was undertaken between November 2009 and August 2010.

Program description

The IB Program’s overall objectives are:

  • To facilitate the safe and timely movement of maritime traffic through and around ice-covered waters;
  • To minimize the effect of flooding caused by ice jams; and
  • To assist in the re-supply of northern communities for which there is no commercial service.

The IB Program contributes to the achievement of those objectives by carrying out the following activities: Route Assistance, Ice Routing and Information Services, Harbour Breakouts, Flood Control/Ice Management, Northern Resupply, and Supporting Arctic Sovereignty.

The icebreaking services are delivered with the support of CCG’s fleet and the Canadian Ice Service (CIS) of Environment Canada (EC), which provides ice information. Also, the federal government has established national (e.g. with Transport Canada, Government of Quebec, Government of Nunavut, etc.) and international agreements (e.g. with the United States of America, etc.), both formal and informal, which delineate operational commitments of icebreaking resources.

The clients of the Icebreaking services include domestic and foreign commercial vessels, ferries, ferry terminal operators, fishing vessels, Canadian and foreign government vessels, owners and operators of the Canadian marine ports and harbours infrastructure, fish processing companies, and the general public.

Methodology

Since there are no measurements before the program began, nor a comparison group against which to assess other plausible causes for the outcome, a non-experimental design was chosen for this evaluation. This method allows to measurements after the program has been implemented. This model was chosen for a number of reasons. First, the IB Program is a full coverage program and a control group could not be established. It is intended to be delivered across Canada where it is needed and could not be withheld from any area or region by virtue of its safety oriented intended outcomes. Secondly, there were no measures in place before the IB Program was introduced; therefore a pre-test of the clients in locations receiving the treatment was not possible.

The evaluation questions were determined on the basis of the new evaluation policy and by reviewing documents and the results of the planning phase interviews with key contacts.

The analysis methods used for this evaluation were tailored to the nature and availability of the data to be gathered, which were in turn linked to the evaluation questions. Extensive use of triangulation was used as an analytical method. Document review, interviews, survey, case studies, direct observation and program database were used as methodologies to support triangulation in this evaluation. Key informants included program personnel, partners and clients.

Over the course of the evaluation, the following limitations in the methodology were observed:

  • Non-experimental Design. Since there are no measurements before the program began, nor a comparison group against which to assess other plausible causes for the outcome, it is difficult to attribute impacts to the program. The program’s operationalizations (activities) were described in great detail increasing the evaluation's construct validity. Lastly the evaluation employed a variety of evaluation methods (e.g. interviews, document review), to assess the impact of the program.
  • Depth of environmental analysis. As part of this analysis, we identified and studied similar programs in the United States, Finland, Sweden and Russia. A macro level comparison of programs was undertaken focusing on the difference between programs with respect to their overall administration of icebreaking services from a delivery perspective.
  • Online survey: It should be noted that due to the small number of respondents (26), that responses are not representative of all potential users and findings from the survey are more suggestive in nature. Readers are to use caution in interpreting results from surveys.

  • Data from interviews. Due to the small number of clients interviewed their responses are not statistically representative for generalizing the findings to all users of the program's services. In this context, both the client and program personnel key informant interviews are based on their extensive experiences. As there is a likelihood of subjectivity, the qualitative evidence was used in combination with other lines of evidence to ensure neutrality for reporting purposes.

Evaluation findings and conclusions

Relevance

The IB Program is well aligned with departmental and government wide priorities. The Program is aligned with the federal government priorities, as well as a number of important strategies, legislation and a departmental strategic outcome. For example, the Program aligns with Canada’s Northern Strategy priorities, as well as some of the objectives of The Canada Shipping Act, 2001. The IB Program also aligns with the DFO strategic outcome of Safe and Accessible Waters, and supports the CCG’s mission.

The evaluation also concludes that the roles of the federal government and of the CCG are appropriate in delivering the IB Program. The IB Program is mandated by several pieces of legislation, such as the Constitution Act, 1867, s. 91(10) and the Oceans Act.

Finally, this evaluation concludes that the IB Program does address several continuing needs for Canadians. Vessel escorts and ice routing and information services help maintain maritime safety by reducing the risk of ice damage to ships, and decrease transit times. The northern re-supply activity is important, because it provides vital supplies to northern communities. The IB Program also ensures a Canadian presence in the Arctic, which is beneficial to Canada. Harbour breakouts are a necessity in ensuring that ships, marine facilities, and infrastructure do not sustain damage from ice and ships can access marine facilities. Without flood control, ice jams can form and cause flooding, which can result in loss of life and property.

Efficiency

The evaluation has concluded that the IB Program has been efficient in its delivery of services.

Overall the roles and responsibilities of the IB Program are defined and understood within CCG and between two of its main partners, CIS and United States Coast Guard (USCG). However, the roles, responsibilities and accountabilities in the regions, between the Ice superintendents and Superintendents, Regional Operation Center (ROC) should be clarified, documented and communicated with regards to tasking ships. Presently, some Ice Superintendents are tasking ships, however it is stated in the Fleet and Safety Manual that this is a responsibility of the Superintendent, ROC. Due to workload issues, the routing responsibility has been relaxed such that Ice Service Specialists (ISS) from Environment Canada have often been permitted to prepare and issue recommended routes, when they should be restricted to providing information on ice and weather conditions and identifying the ice regime along the recommended route. An ISS is not an expert with respect to routing and may not take into consideration factors that are considered important for safe and efficient navigation.

Recommendation 1: It is recommended that the IB Program should establish the following with external service delivery partners: with respect to operational responsibilities of regional icebreaking superintendents, the CCG should clarify with Environment Canada (EC), the role and responsibilities of the Ice Service Specialists (ISS) and in particular with regard to ice routing.

The evaluation found that if operational constraints arise during operations, the IB Program has mechanisms in place to resolve issues e.g. by having conference calls between regions.

The IB Program participates in the Fleet Planning process annually. The November 2009 National meeting in Montreal, a first, was a good initiative to discuss the availability of resources as well as how the stakeholders could work together to improve the efficiency of the program.

Overall, the IB Program monitors its performance through several means e.g. Level of Service (LOS), Ice Information Service Partnership Agreement (IISPA), Icebreaking Operations Data Information System (IODIS). The evaluation found that some the performance information provided though the IISPA and IODIS have been used to formally report results of the IB Program in the DPR, briefings to the commissioner, etc.

Although performance reporting measures are in place to assess the overall performance associated with the IISPA (an agreement with CIS), it has not been possible to quantify the success of the agreement because of limited performance information provided by the CIS. The evaluation also concluded that the IB Program does not measure how its collaboration with the USCG has contributed to the achievement of its results in the Central and Arctic region. In addition, through the data analysis of IODIS it was found that the IB Program could not provide performance information on all of its operational outputs (indicators) and strategic outcomes established in the LOS document.

As a result, an area where the IB Program could improve its efficiency could be with regards to performance measurement. The IB Program should review its different sources of performance data e.g. LOS, IISPA, IODIS, etc. to ensure that the appropriate data is being gathered in a consolidated way which supports a performance measurement strategy that is aligned to the MRRS and respects program and evaluation performance requirements.

Recommendation 2: It is recommended that the Icebreaking Program develop and implement a Performance Measurement (PM) Strategy that is aligned with the MRRS while respecting program and evaluation performance requirements. This PM Strategy would include a program profile, a logic model, a performance measurement framework, an evaluation and reporting strategy as well as a data collection strategy.

The evidence led evaluators to conclude that complimentary services (i.e. availability of ice information, website such as MARINFO and ice booms) and partnership agreements (i.e. with the CIS and the USCG) contribute towards improving the efficiency of the program. Ice Information Systems, specifically the availability of ice information, has improved efficiency by reducing reliance on icebreakers. These partnership agreements have resulted in some regions delivering the program in a zonal approach; the purchase of satellite imagery to create ice charts and routings; and the delivery of joint operations and cooperation of ice breaking services. However, there was no performance information on how these efficiencies have provided monetary savings to the CCG.

Examples of how these complimentary services have resulted in delivering the IB Program efficiently are the purchase of satellite imagery to create ice charts and routings; and the delivery of joint operations and cooperation of icebreaking services. In addition, Ice Information Systems, specifically the availability of ice information, have improved the efficiency of the program. Providing information such as voyage planning, recommended ice routes and tactical navigation reduces the reliance for icebreakers by increasing navigator autonomy.

While there was no performance information on how these efficiencies have provided monetary savings to the CCG, results from the evaluation confirm the usefulness of such activities in improving the efficiency of the program.

Recommendation 3: It is recommended that the Icebreaking Program continues to support complimentary services (i.e. availability of ice information, website such as MARINFO and ice booms) and partnership agreements (i.e. with the CIS and the USCG) that contribute to improving the efficiency of the program in all regions. These measures will continue to increase the efficiency of the program by increasing navigator autonomy and reducing reliance on icebreakers.

Effectiveness

The IB Program contributes to the safe, efficient and timely movement of vessels through ice-infested waters. In addition, IB services reduce the effect of flooding caused by ice jams on the St. Lawrence River, and the program assists in re-supplying northern communities that do not have commercial service.

Four key documents facilitate the delivery of IB services with its partners and clients:

  • Joint Industry-Coast Guard Guidelines for the Control of Oil Tankers and Bulk Chemical Carriers in Ice Control Zones of Eastern Canada (JIGs) – ), a responsibility identified in the MOU between TC and CCG;

  • Levels of Service (LOS) – agreement between the CCG, specifically the IB Program and external clients that use the icebreaking services;

  • Agreement providing for coordination of icebreaking activities of Canada and the United States of America on the Great Lakes–St Lawrence Seaway System (Canada/US agreement) – agreement between Canada and the US to collaborate in delivery of icebreaking services on the Great Lakes; and,

  • Ice Information Services Partnership Agreement (IISPA) – agreement between CCG and Environment Canada’s (EC’s) Canadian Ice Service (CIS).

Two of the documents had a five-year review cycle in place, while the other two, specifically JIGs and LOS, did not. The documents should be regularly reviewed and updated to ensure they remain effective. There is room for improvement in this regard.

JIGs is considerably out of date. The agreement, for which Transport Canada has the lead responsibility, has not been revised since 1979, despite a review in 2005. It is also vague and cannot effectively support operations. Given the potential for serious environmental harm from marine oil or chemical spills, such guidelines should be kept up to date.

The agreements need to be reviewed and updated on a regular basis to ensure they support the types and levels of service required to meet client needs, taking into account the program’s capacity. The IB Program’s ability to meet increasing or changing demands is tempered by its service capacity. While client feedback on the whole leans towards the positive, some comments and other feedback highlight areas of concern. The LOS in effect during the period covered by the evaluation dated back to 2001. Both clients and program staff noted that conditions have changed since then such that the 2001 version was out of date. That version which was based on the 2007 Levels of Service (LOS) Review, has since been replaced as of May 2010,. A review of the newest version was outside the scope of this evaluation. A 10-year period between updates is too long, as a result of evolving needs of the clients of the IB Program and potential changes in the capacity of the fleet of icebreakers.

Recommendation 4: It is recommended that the IB Program adhere to a five-year review and update cycle for all agreements pertaining to the delivery of icebreaking services to keep pace with evolving client needs and changing IB Program service delivery capacity.

The LOS identifies the target response times within which route assistance, facilities and maintenance, and flood control services are to be provided. For the period covered by the evaluation, the IB Program met these targets generally more than 90% of the time.

However, the 2001 LOS did not include the IB Program’s prioritization system, which separates service requests into five levels of priority. Furthermore, the priorities as defined lack clarity, detail and coverage of all ferry services giving rise to different interpretations and confusion. While the wording for Priority II, governing ferry services, was changed following a workshop in November 2009, the prioritization system should be formally incorporated as part of the IB Program’s LOS document.

Recommendation 5: It is recommended that language surrounding the service requests for the five levels of priority of the Icebreaking Program’s prioritization system in the LOS document be more clearly defined and detailed specifically with respect to coverage for ferry services.

The IB Program does not have complete performance data for services delivered through IISPA. IISPA requires monthly and seasonal CIS reports to be submitted to the Manager, IB Program, which have not been provided systematically over the period covered by the evaluation.

Recommendation 6: It is recommended that the IB Program should establish the following with external service delivery partners: It is recommended that the IB Program should ensure with respect to IISPA, submission of regular reports (monthly, seasonal, both or otherwise as deemed appropriate by the IB Program) from CIS to the Manager, IB Program.

Economy

The evaluation concluded that the IB Program has been producing enough outputs to produce the desired result at the lowest cost. Overall, the program has generally been able to maintain its Levels of Service above 90% (Graph 12), with fewer available ships (Table 3).

In addition, through the collaboration between the CCG and USCG on the Great Lakes, the IB Program has been minimizing its resources, by having two CCG vessels on the Great Lakes and leveraging up to nine USCG vessels while maintaining its Level of Services.

A comparative analysis of the literature compared Canadian icebreaking services to four countries (Russia, Finland, Sweden and the United States of America) regarding the delivery models (private vs. public) and comparison of resources i.e. fleet, and service fees.The IB Program could review how other countries are delivering their icebreaking services to see if CCG can learn and apply these practices in Canada. By comparing how other countries deliver their services, CCG could apply techniques that have been successful in other countries and further minimize the use of resources in the delivery of services.

1. Introduction

1.1 Context of the Evaluation

This evaluation report aims to present the results of the evaluation of the IB Program (IB Program). As identified by the Evaluation Policy, all direct program spending must be evaluated every five years. This evaluation was slated in the Department of Fisheries and Oceans (DFO) five-year evaluation plan and focuses on the core issues in assessing value for money as defined by the Treasury Board Evaluation Policy (2009). These core issues include: relevance and performance, including effectiveness, efficiency and economy of the IB Program.

The timeframe for this evaluation, a first for the IB Program, covers the period from 2005-06 to December 31, 2009. The evaluation assesses the extent to which IB Program has achieved its outcomes stemming from activities as stated in the Logic Model – IB Program. The evaluation is inclusive of the National Capital Region as well as four regional offices; Maritimes, Quebec, Central and Arctic, and Newfoundland and Labrador. Site visits were carried out in Sarnia, ON, the headquarters of the Central & Arctic region as well as in Quebec City, headquarters of the Quebec region. The evaluation was conducted by DFO’s Evaluation Directorate and was undertaken between November 2009 and August 2010.

1.2 Objectives of the Evaluation

The main objective of this evaluation is to determine to what extent the IB Program is managed effectively and efficiently and whether it had achieved its stated objectives and results. As such, the evaluation examines the extent to which the IB Program demonstrates relevance and performance (effectiveness, efficiency and economy). In this report we draw the lessons learned from each of these issues and develop recommendations for improving the program going-forward.

2. Program Profile/Background

2.1 History of Icebreaking in Canada

The federal government of Canada has provided an icebreaking service off Prince Edward Island since 1873, beginning with the Northern Light, employed off the coast of Prince Edward Island. Exploration of Canada’s Arctic, which peaked in the late 1890’s, precipitated the establishment of regular Arctic patrols in the 1920’s during the short summer navigation season. With "cold war" developments at the beginning of the 1950s, the first of the modern icebreakers were built to improve access to the north and supply defense sites and northern communities. Since confederation, demand has steadily increased, consequently, icebreaking services have evolved and developed as well.

2.2 Partners, Stakeholders and Clients

The icebreaking services are being delivered in partnership with other CCG/DFO programs, federal departments, other governments and agencies. The Icebreaking Services are delivered with the support of CCG’s fleet, which provides the icebreakers and the Canadian Ice Service (CIS) of Environment Canada (EC), which provides ice information.

The federal government has established national and international agreements, both formal and informal, which delineate operational commitments of icebreaking resources.

The clients of the Icebreaking services include domestic and foreign commercial vessels, ferries, ferry terminal operators, fishing vessels, Canadian and foreign government vessels, owners and operators of the Canadian marine ports and harbours infrastructure, fish processing companies, and the general public.

2.3 Program Objectives

The following are the IB Program objectives:

  • To facilitate the safe and timely movement of maritime traffic through and around ice-covered waters;
  • To minimize the effect of flooding caused by ice jams; and
  • To assist in the re-supply of northern communities for which there is no commercial service.

According to the DFO Program Activity Architecture (PAA), IB Program (or Icebreaking Services, as it is called in the PAA) is a sub-activity within the CCG program activity under the strategic outcome of ‘Safe and Accessible Waterways’.

2.4 Program Activities

There are six sub-activities of the IB Program, include:

Route Assistance - escorting ships and organizing convoys to travel through ice-infested waters, and freeing beset vessels.

Ice Routing and Information Services - providing ice information, ice reconnaissance to survey and forecast ice conditions, and broadcasts and ice routing advice.

Harbour Breakouts - breaking out approaches and clearing ice from wharf faces of port terminals and facilities, keeping ice clear of barge operations and the ships at anchor, and facilitating acceleration of ice clearance at the end of the ice season.

Flood Control/Ice Management - monitoring ice conditions and water levels in anticipation of flood risks, preventing formation of ice jams and excessive build-up of ice and facilitate ice flow during spring break-up.

Northern Resupply - transport dry cargo and fuel during the annual re-supply of Northern settlements and military sites.

Supporting Arctic Sovereignty - Canadian presence in the Arctic, assisting and supporting ships making Arctic voyages and exploratory or demonstration voyages, participating in the support to user communitites and local population, maintaining effective control over foreign navigation, and providing support to other government departments, agencies, and organizations in ice-infested waters.

2.5 Governance and Resources of the Program

The IB reports to Navigations Systems sector of the CGG. The Navigation Systems sector is part of the Maritimes Services Division of the CCG. The CCG is a special operating agency of DFO. The IB program has personnel both at the national headquarters (NHQ) and in four of the CCG regions (Central and Arctic, Maritimes, Newfoundland and Labrador, Quebec).

The following table summarizes IB financial information from 2005/06 to 2009/10:
Table 1: Icebreaking Program (Icebreaking Services sub-activity) Total Cost


TOTAL SUB-ACTIVITY COST

Spending Profile

2005-06

2006-07

2007-08

2008-09

2009-10

Mains

Actuals

Mains

Actuals

Mains

Actuals

Mains

Actuals

Mains

Actuals

Salary

2,203.1 1,107.3

2,227.9

845.8

2,274.6

1,031.9

779.1

1,131.6

1,022.5

943.6

Non-Salary O&M*

19,707.0

15,977.1

17,099.2

17,045.8

19,083.2

22,694.8

18,784.0

27,677.7

18,566.8

20,176.4

Vote-Netted Revenue (VNR)

(2,452.7)

(1,032.3)

(2,452.7)

(889.1)

(2,452.7)

(923.0)

(2,452.7)

(853.2)

(2,435.0)

(883.2)

Total Program Costs

19,457.4

16,052.0

16,874.4

17,002.5

18,905.1

22,803.7

17,110.4

27,956.1

17,154.3

20,236.8

Mains = Planned Spending
Actuals = Actual Spending
*The Non-Salary O&M includes funds for Marine Research and Development, vessel fuel, ice reconnaissance, minor capital and priorities and contingencies. These funds are not part of the Icebreaking Program budget, but part of the Icebreaking Services sub-activity, which is part of other directorates within the CCG.

2.6 Logic Model of the Program

For the purpose of this evaluation, the following program logic model was simplified from the logic model provided by the program.

For the purpose of this evaluation, the following program logic model was simplified from the logic model provided by the program.

3. Methodology

A multiple-lines-of-enquiry approach was used to study the issues and questions that were the focus of this evaluation. This section outlines the scope and methods of our approach, the evaluation design, evaluation questions, the methodological approach, analytical methods as well as the limitations of the evaluation.

3.1 Project Management

The evaluation was conducted by an evaluation team led by a senior evaluation manager within the Evaluation Directorate at DFO. The team collaborated with IB Program personnel on preparing a list of documents to review, identifying key informants and stakeholders and reviewing and providing feedback on interview guides and various reports.

3.2 Evaluation Design

For this evaluation we used a non-experimental design, in which measurements are taken after the program has been implemented. This model was chosen for a number of reasons. First, the IB Program is a full coverage program. It is intended to be delivered across Canada where it is needed and could not be withheld from any area or region by virtue of its safety oriented intended outcomes and a control group could not be established. Secondly, there were no measures in place before the IB Program was introduced; therefore a pre-test of the clients in locations receiving the treatment was not possible.  

3.3 Evaluation Questions

The evaluation questions were determined on the basis of the new evaluation policy and by reviewing documents and the results of the planning phase interviews with key contacts. The key questions are discussed briefly below. Annex A features an evaluation matrix organized by topic: relevance and performance (effectiveness, efficiency and economy). 

3.4 Secondary Data Sources

Secondary data was examined, which included document review.

Document Review
The document review consisted of the analysis of documents provided by the program and included both public and internal program documents and consisted of public documents i.e. annual reports and internal documents prepared by IB Program personnel. A literature review was also used in order to make a comparative assessment of the program using studies of similar programs in other countries or jurisdictions.

3.5 Primary Data Sources

Primary data was collected by the evaluation team specifically these consisted of key informant interviews, a survey, case studies, and observations at workshops held by the IB Program.

Interviews
Interviews were conducted with key informants in order to obtain useful information on the perceptions and opinions of individuals who play a significant role in or have experience with the program's design or implementation, as well as information on the importance and effectiveness of the program. Fifty-six key informants were interviewed and including managers, superintendents, fleet planners, major crown project managers, program representatives and cost recovery advisors. Ten clients were also contacted to discuss the IB Program purpose and impact (more specifically, its ability to obtain results) and ask for their opinion on the services provided and the role of government.

Survey
An online survey with Program clients was conducted for this evaluation. EKOS, a firm specializing in the development and implementation of surveys, conducted the survey on behalf of DFO. A total of 119 clients received the survey and 26 completed the questionnaire, for a response rate of 22 per cent. Due to the small sample size associated with this study when describing results of the survey, frequencies, rather than percentages, are used (e.g., 19 out of the 26 respondents rather than 73 per cent of respondents) to convey the more qualitative nature of the analysis.

Case studies
Two case studies were conducted for the Evaluation of the IB Program. The first case study was on the collaboration between the CCG and the United States Coast Guard in the Great Lakes. The second was on the collaboration between the CCG and the Canadian Ice Service. The case study methodology includes a thorough review of IB program documents, key informant interviews conducted by DFO evaluation staff and data from performance reports.  

Direct Observation
Evaluators attended two workshops held by the IB Program. The site visits occurred in May 2010 for the Quebec regional office as well as in Sarnia, at the headquarters of the Central and Arctic region. Evaluators attended the National Icebreaking Post-Season (2009) workshop, the National Icebreaking Pre-Season (2010) workshop, the Shipping Federation pre-winter, the Post Arctic (2009) workshop as well as the Polar Icebreaker workshop (2009).
 
Program Database
The Icebreaking Operations Data Information System (IODIS) is the IB Program data management system, which was implemented in 1989. IODIS was created as a reporting system used to monitor the efficiency of the icebreaking fleet and to support the Icebreaking service standards. As IODIS collects data per operational season: Summer Operations in the Canadian Arctic (from late June to October) and Winter Operations on the East Coast, Gulf, Saint-Lawrence River and Great Lakes (from mid-December to April or May) the analysis is based on winter 2004-2005 to winter 2009-2010 in order to cover the scope (April 1, 2005 to December 2009) of the evaluation. However, the Canadian Coast Guard Automated Performance Solution (CCGAPS) is the reporting tool used in this evaluation. The CCGAPS Icebreaking cube is linked to IODIS data.

3.6 Analytical Methods

The analysis methods used for this evaluation were tailored to the nature and availability of the data to be gathered, which were in turn linked to the evaluation questions. Extensive use of triangulation was used as an analytical method. In the social sciences, triangulation means that more than two methods are used in a study with a view to double (or triple) checking results using multiple lines of enquiry to corroborate findings.

3.7 Methodological Limitations and Constraints

Over the course of the study we observed some shortcomings in the methodology over the course of the study. In order to minimize their impact on the results of the evaluation, we employed information collected from a variety of sources using a variety of methods. In other words, we combined the various methods discussed earlier to arrive at the same conclusions, thereby reinforcing our assessment as to their validity. None of these shortcomings significantly jeopardized the validity or accuracy of the evaluation results.

  • Non-experimental Design. A non-experimental design involves taking measurements after a program has been introduced, with no control group. The model below illustrates as follows:

 

Exposure to the Icebreaking Program

Measurement After Exposure

Clients and Beneficiaries

X

01

The model above shows that it is difficult to clearly measure the effects of the IB Program. Since there are no measurements before the program began, nor a comparison group against which to assess other plausible causes for the outcome, it is difficult to attribute impacts to the program (01 may be affected by multiple factors). In other words, it is difficult to draw a clear conclusion about the incremental or net impact of the program. Although this model lacks scientific rigour, the rigour of this design was increased by describing activities, outputs and outcomes through a logic model, enabling evaluators to make causal linkages, to logically argue that results can be attributed to the program (internal validity). Further, a thorough program profile complemented the logic model. The program’s operationalizations (activities) were described in great detail increasing the evaluation's construct validity. Lastly the evaluation employed a variety of evaluation methods (e.g. interviews, document review), to assess in the context of the above construct, the impact of the program.

  • Depth of environmental analysis. As part of this analysis, we identified and studied similar programs in the United States, Finland, Sweden and Russia. We undertook a macro level comparison of programs focusing on the difference between programs with respect to their overall administration of icebreaking services from a delivery perspective.
  • Online survey:  It should be noted that due to the small number of respondents (26), that responses are not representative of all potential users and findings from the survey are more suggestive in nature. Readers are to use caution in interpreting results from surveys.
  • Data from interviews. The sample of clients interviewed reflected the types of clients served by the IB Program but was not statistically representative for generalizing the findings to all users of the program's services. As for the limited number of program personnel involved in the delivery of the IB Program, this limited the number of perspectives and hence there is a risk of bias in their responses. In this context, both the client and program personnel key informant interviews are based on their extensive experiences. As there is a likelihood of subjectivity, the qualitative evidence was used in combination with other lines of evidence to ensure neutrality for reporting purposes. 

4. Major Findings

4.1 Relevance

Relevance Question #1: To what extent are the objectives of the Icebreaking Program aligned with departmental and government wide priorities?

Key finding:  The Icebreaking Program aligns with Government of Canada priorities, the Canada’s Northern Strategy and pieces of legislation, and the DFO strategic outcome of Safe and Accessible Waters.

Findings

The IB Program aligns with two Government of Canada priorities. The first, Stimulating our Economy, the program supports by contributing to the economic prosperity of Canadian industries and communities on the St Lawrence Seaway. The second, Supporting Families and Communities, the program supports by minimizing the effects of flooding caused by ice jams by improving land use along rivers and reducing risk of ice damage.   

The IB Program also aligns with a number of important strategies, pieces of legislation, and a departmental strategic outcome. For example, the IB Program aligns with Canada’s Northern Strategy priorities of sovereignty and economic development through the program’s re-supplying missions to isolated northern communities. The IB Program objectives are aligned with some of the objectives of theCanada Shipping Act, 2001. IB Program activities align with the DFO strategic outcome of Safe and Accessible Waters. Finally, the IB Program supports the CCG’s mission of providing services in support to government priorities and economic prosperity and contributing to the safety, accessibility, and security of Canadian waters.

Evidence

The review of the documentation found that the IB Program provides services enabling year-round navigation on the St Lawrence Seaway, therefore contributing to the economic prosperity of Canadian industries and communities. This is supported by The Great Lakes St. Lawrence Study, 2007 which examined the infrastructure needs of the Great Lakes St. Lawrence Seaway system and found that Marine transport on the Great Lakes produces $3.4 billion in business revenue annually and more than $4.3 billion in personal income. It also generates direct and indirect employment for as many as 150,000 people in Canada and the United States1. The marine transport of material and goods provide input to industries, contributing to Government of Canada priority to Stimulate our Economy.

The analysis found that the IB Program activities which help minimize the effects of flooding caused by ice jams also work in support of the Government priority of Supporting Families and Communities by improving land use along rivers and reducing risk of ice damage to structural components of bridges and marine facilities, such as the Quebec Bridge.

The IB Program objectives also align with two of the priorities of Canada’s Northern Strategy, a government wide effort introduced in the 2007 Speech from the Throne; Exercising our Arctic Sovereignty and Promoting Social and Economic Development. The IB Program provides advice and ice information to assist CCG’s icebreaker’s missions to transport dry cargo and fuel in the re-supply of isolated northern communities that are not connected by road or rail.

The IB Program objectives are aligned with some of the objectives of The Canada Shipping Act, 2001 such as to promote an efficient marine transportation system, to protect the marine environment from damage due to navigation and shipping activities.

Our review of documents and observations found that, at the departmental level, the IB Program, by facilitating the informed, safe, and timely movement of maritime traffic through and around ice-covered Canadian waters, supports and links to the DFO strategic outcome of Safe and Accessible Waterways for Canadians.

The key activities of the IB Program, such as the provision of ice routing and information, route assistance, ice management (flood control), harbour breakouts, northern re-supply and supporting Arctic sovereignty, support the CCG’s mission of providing services in support of government priorities and economic prosperity and contributing to the safety, accessibility and security of Canadian waters.

Relevance Question #2: Is the current role of the federal government and CCG appropriate in delivering the Icebreaking Program?

Key finding:  Legislation mandating the federal government, as well as signed Canadian and international agreements, demonstrate that the current roles of the federal government and CCG are appropriate in delivering the Icebreaking Program.

Findings

Overall, the roles of the Government of Canada and CCG are appropriate in delivering the IB Program. Legislation mandating the federal government, as well as signed Canadian and international agreements, are the main drivers behind this finding. Finally, interviews confirmed that by virtue of the types of services it provides, no other entity in the public or private sector could undertake to provide these services.

The IB Program derives its mandate from several pieces of legislation. The principle authority for icebreaking services derives mainly from the Constitution Act, 1867, s. 91(10) and the Oceans Act. The IB Program supports other government departments in their efforts to meet their obligations under a number of Acts including the Arctic Waters Pollution Prevention Act, the Navigable Waters Protection Act as well as the Marine Transportation Security Act.

The program has an operational agreement2 to provide icebreaking services in Quebec region to coordinate icebreaking operations in the region in over 15 rivers. There is also the Agreement between CCG, C&A region, DFO and Environment Canada, Atmospheric Monitoring Division, Prairie and Northern region, regarding the re-supply of Eureka, Nunavut and A Memorandum of Understanding Regarding the Delivery of Petroleum Products and Dry Goods from Nanisivik, Nunavut, to Kugaaruk, Nunavut 2006 to 2010 with Environment Canada engaging the CCG to re-supply northern communities. 

According to the Terms of Union, Canada is obligated to maintain a transportation link between Newfoundland and the mainland, as well as Prince Edward Island and the mainland – this is accomplished through the services provided by the IB Program. 

The IB Program also has a directive (Icebreaking Operations Directive 4, December 2009) specifying the conditions under which the Program may provide services in support of sealing vessels or with respect to Fisheries Management and RCMP law enforcement during the seal hunt.

The Government of Canada is a signatory to several international obligations whose conditions require collaboration amongst sovereign states. At the international level, the Government of Canada has three agreements with the United States Government with regards to the provision of icebreaking services. These agreements are the Canada/United States Icebreaking Agreement for the Great Lakes and the Agreement between Canada and the USA on Arctic cooperation, as well as a formal MOU between the two countries for Icebreaking Support to “Pacer Goose” of the U.S. Military Sealift Command. Canada also agreed to contribute to the North Atlantic Ice Patrol under the Agreement Regarding Financial Support of the North Atlantic Ice Patrol.

Finally, findings from interviews indicate that the CCG’s IB Program is in the best position to fulfill the federal government’s obligations with regards to icebreaking, and that it could not be achieved by other departments or the private sector.

Evidence

CCG’s mandate for delivering icebreaking services derives from the Constitution Act, 1867, s. 91(10), where the Parliament of Canada was given Legislative Authority for Navigation and Shipping. The Oceans Act entrusts the Minister of Fisheries and Oceans with the responsibility to ensure safe, economical, and efficient movement of ships in Canadian waters, Section 41.(1)(a) and ice breaking and ice management, Section 41.(1)(a)(iii).

The IB Program supports other government departments in their efforts to meet their obligations under a number of Acts, including the Arctic Waters Pollution Prevention Act, the Navigable Waters Protection Act) as well as the Marine Transportation Security Act, all three of which are the primary responsibility of Transport Canada.

In the Quebec region, even if flood control in rivers is under Quebec Provincial Government jurisdiction, an operational Partnering Agreement was made in 1995 between the ‘Direction générale de la sécurité civile’ and the CCG, entrusting the responsibility to the CCG IB Program, Quebec region to coordinate icebreaking operations in the region in over 15 rivers.

According to the 1949 Newfoundland Act, (U.K.), 12-13 Geo. VI, c. 22, Canada is to maintain a freight and passenger steamship service between North Sydney, N.S. and Port-aux-Basques, NL. As for the Prince-Edward-Island, the government of Canada is obligated to establish and maintain services for the conveyance of mails and passengers between the Island and the mainland of the Dominion year round. Both cases do not explicitly imply mandate for icebreaking, but it implies that icebreakers, thus services from the IB Program, will be utilized to maintain the transportation link between the islands and the mainland when waters are covered with ice.

The Icebreaking Operations Directive 4 (December 2009) specifies the conditions under which CCG icebreaking services may be provided in support of sealing vessels which include all distress and emergency situations and may also include escort services to assist sealing vessels out of heavy ice conditions and into workable-ice or open water. Icebreakers may be used as a platform to assist Fisheries Management personnel with enforcement and monitoring functions, or to assist the RCMP in the event there are attempts to disrupt the seal hunt.

Due to heavy ice in the high Arctic and the challenges the voyage presents to conventional commercial ships, an Agreement between CCG, C&A region, DFO and Environment Canada, Atmospheric Monitoring Division, Prairie and Northern region, regarding the resupply of Eureka, Nunavut was made. Based on this Agreement, the CCG consents to provide vessels and equipment, to monitor the delivery of dry cargo and to direct or provide for the loading of dry cargo aboard CCG icebreakers in communities such as Nanisivik. A Memorandum of Understanding Regarding the Delivery of Petroleum Products and Dry Goods from Nanisivik, Nunavut, to Kugaaruk, Nunavut 2006 to 2010 engages the CCG to deliver dry cargo to the community of Kugaaruk (Pelly Bay). Nunavut and Environment Canada engage the CCG for this service as no commercial carriers operate in these areas.

The Government of Canada is a signatory to several international obligations whose conditions require collaboration amongst sovereign states. The Government of Canada has three agreements with the United States with respect to icebreaking services. The Canada/United States Icebreaking Agreement for the Great Lakes provides for the coordination of icebreaking activities of Canada and the United States in the Great Lakes.This arrangement includes designation of geographical areas within the Great Lakes and connecting waterways where each organization has principal responsibility for icebreaking. According to the Agreement between Canada and the USA on Arctic cooperation, both countries agreed to undertake to facilitate navigation by their icebreakers in their respective Arctic waters, to develop cooperative procedures for this purpose, and to exploit their mutual icebreaking capabilities to develop and share research information, in order to advance their understanding of the marine environment of the area. Finally, Canada and the US have a formal MOU for Icebreaking Support to “Pacer Goose” of the U.S. Military Sealift Command. Since 1992, the CCG has provided an icebreaker to escort U.S. Military Sealift Command (MSC) ships in the annual re-supply of Thule AFB in Greenland, Operation Pacer Goose. The agreement is predicated on a reciprocal exchange of ship days so that each service can avoid lengthy transits between the eastern and western Arctic.

Elsewhere internationally, the Government of Canada, along with other countries, signed the Agreement Regarding Financial Support of the North Atlantic Ice Patrol in 1956 in accordance with the provisions of Chapter V, Regulations 5 and 6, of the International Convention for the Safety of Life at Sea (SOLAS). Each party agreed to contribute annually to the support of the Ice Patrol which is managed by the Government of the United States.

Finally, eight of the interviewees indicated that the CCG’s IB Program is in the best position to fulfil the obligations of the Government of Canada with regards to icebreaking and that these obligations could not be fulfilled by other departments or the private sector.

Relevance Question #3: To what extent is the Icebreaking Program addressing an actual and continuing need?

Key finding:  The services that the Icebreaking Program provides are addressing several actual and continuing needs.

Findings

The IB Program supports the Arctic during the summer months from May to November, providing services such as assistance and support to ships making Arctic voyages, support to communities and local populations through the carrying out of various program activities, maintaining effective control over foreign navigation, and providing support to other government departments, agencies, and organizations in ice-infested waters.3 

Climate change is yielding a longer summer season in the Arctic, as well as changes in the ice cover. Changing ice coverage has potential impacts, including the possibility of the opening of the Northwest Passage. Canada’s Arctic sovereignty has become an increasingly important issue over the last decade. The Coast Guard responds to an important need with respect to supplying dry cargo and fuel to Northern settlements and military sites (Eureka and Pelly Bay). These missions are important with respect to ensuring a Canadian presence in the Artic. Furthermore, DFO Science is relying on the IB Program to conduct seismic and bathymetric surveys in the Arctic in support of Canada’s claim to waters past the 200 nautical mile limit in order to fulfill the requirements of UNCLOS. 

The IB Program is responding to a number of other important needs in the Arctic. For example, environmental response to pollution in the Arctic is becoming an increasingly important need. Increased traffic in the Arctic could potentially result in an increase in marine incidents, possibly resulting in oil spills and other pollution situations.4  Climate change is causing earlier break-up and later freeze-up of the Arctic ice, and potentially allowing multi-year ice (which is the most dangerous type of ice) to flow down into shipping lanes from the Arctic Archipelago. An increased likelihood of marine traffic in the North in dangerous ice conditions may also result in an increase in demand for Search and Rescue and Environmental Response. 

The IB Program fulfills an important need by providing navigational assistance to domestic and foreign commercial vessels, ferries, fishing vessels, and government vessels through ice covered waters, as well as the freeing of vessels beset in ice. Vessel escorts help maintain maritime safety by reducing the risk of ice damage to ships transiting ice-infested waters; they help manage and protect the marine environment by escorting ships carrying environmentally dangerous goods through ice-infested waters; and they facilitate marine trade, commerce, and ocean development by decreasing transit time for travelling through ice-infested waters and providing shipping companies with the assurance that help is available if needed.  25 of 36 key informants interviewed noted that vessel escorts are an important activity. 

Ice routing and information services exist to address the same needs as vessel escorts – the need for assistance in navigating through ice infested waters. Ice information makes transiting icy waters less risky and reduces the number of vessel escorts needed because ship captains have the tools they need to avoid ice themselves, and reduces the cost of economic activity by providing the most efficient route for the ships. Ice routing and information was seen as an important activity by 12 of 36 key informants interviewed.

In order for marine trade to exist in the Arctic during the summer and in Southern waters during the winter, harbour breakouts must be conducted to ensure that ships, marine facilities, and infrastructure do not sustain damage from ice and that ships can access the marine facilities. The harbour breakouts activity directly contributes to the objective of facilitating marine trade, commerce, and ocean development. Harbour breakouts were reported as an important need by six of 36 key informants interviewed.

Overall, looking at perceptions of how the Program is addressing client needs, findings from the survey suggest that an equal number of survey respondents held the opinion that the Program meets their needs both to a great extent (eight) and to a little extent (eight). The survey also found that 17 out of the 26 respondents surveyed held the opinion that there are needs not currently being addressed by the Program. 

Evidence

The Arctic
Two heavy, three medium and one light icebreakers are dispatched to the Canadian Artic during the summer months from late June to November to provide Icebreaking Services. Icebreaker ships are the only platform used by CCG in the High Arctic that can operate in this type of environment. The CCG also operates a medium icebreaker to support a dedicated science role and two special navaids tenders to service aids to navigation on the Mackenzie River. The presence of icebreakers plays a key role in securing these waters on behalf of Canada’s interests of sovereignty, Search and Rescue, environmental response, northern resupply, science, and for vessel escorts. There are no other vessels in Canada capable of meeting the needs of the Arctic.

Climate change is yielding a longer summer season in the Arctic and changing ice coverage has potential impacts which include the possibility of the opening of the Northwest Passage. Graph 1 below illustrates the gradual decline since the 1980s in the accumulated ice coverage in the Eastern Artic.

Graph 1

Graph 1 below illustrates the gradual decline since the 1980s in the accumulated ice coverage in the Eastern Artic.

Canada’s Artic sovereignty has become an increasingly important issue over the last decade. The article Securing the Northwest Passage Essential states that “with increased shipping traffic looming on the horizon, experts agree Canada must [increase] its presence in the Arctic or risk losing its claim to these contested waters… when the passage becomes an increasingly high-traffic zone, now infrequent challenges such as oil and chemical spills, smuggling and ships in distress will become more frequent. For these reasons, a June 2008 report by the Senate Committee on Fisheries and Oceans recommended Canada get serious about managing the expected increase in shipping traffic, or risk challenges to its already somewhat tenuous claims to the North.”5 

The CCG responds to an important need by supplying dry cargo and fuel to Northern settlements and military sites (Eureka and Pelly Bay) each year. These missions are important to ensuring a Canadian presence in the Artic. For example, The Dry Cargo Resupply Activity Summary by the Government of Nunavut for 2009 states that “Annual sealift is critical for economic re-supply to arctic communities and for economic development…Use of the marine mode provides shippers with the lowest costs, and the 2005 Sealift Review found the cost of shipping by air can be eight to eleven times the cost of sealift. This is why almost all of non perishable goods needed in Nunavut move by water…”6. Northern re-supply was reported as an important need by six of 36 informants interviewed.

Furthermore, DFO Science is conducting seismic and bathymetric surveys in the Arctic to support Canada’s claim to waters past the 200 nautical mile limit in order to fulfill the requirements of UNCLOS. Over the past couple of years, DFO Science has also been involved in International Polar Year research. The icebreaker Louis S. St.-Laurent is the platform that has been used for UNCLOS missions, and the IB Program plays an important role in supporting science during these operations by providing the ice routing and information services, Ice Service Specialists, etc.

Environmental response to pollution in the Arctic is becoming an increasingly important need. Increased traffic in the Artic could potentially result in an increase in marine incidents, possibly resulting in oil spills and other pollution situations.7  For example, in Model Negotiation on Northern Waters, Feb 2008, teams of non-governmental experts expressed concerns that “increased shipping will bring heightened environmental risks, especially in the form of oil spills and disruption of indigenous peoples and marine life”. The Arctic environment is highly sensitive due to a number of different factors, and is therefore particularly vulnerable to pollution. 

Climate change is causing earlier break-up and later freeze-up of the Arctic ice, and potentially allowing dangerous multi-year ice to flow down into shipping lanes from the Arctic Archipelago. An increased likelihood of marine traffic in the North in dangerous ice conditions may result in an increase in demand for Search and Rescue, which can only be carried out by the heavy icebreakers, as they are the only vessels capable of handing missions in the Arctic. 

Finally, a main activity of the IB Program is providing navigational assistance to domestic and foreign commercial vessels, ferries, fishing vessels, and government vessels through ice covered waters, as well as the freeing of vessels beset in ice. In spite of the downwards trend in first year ice, there is still sufficient ice to warrant continued vessel escorts through these waters. 

Other General Needs

Vessel Escorts: During the winter months, from mid-November to May, icebreakers are dispatched south of 60° North latitude. As in the Arctic during the summer, vessel escorts are a main activity of the IB Program during the winter as well. The LOS document states that vessel escorts help maintain maritime safety by reducing the risk of ice damage to ships transiting ice-infested waters; they help manage and protect the marine environment by escorting ships carrying environmentally dangerous goods through ice-infested waters; and they facilitate marine trade, commerce, and ocean development by decreasing transit time for travelling through ice-infested waters and providing shipping companies with the assurance that help is available if needed. Icebreaking services enables trade during this time of year. Graph 2 below has lines showing the average and trend for ice coverage in the Gulf of St. Lawrence. Although the trend is decreasing, it is clear that it will be many years before the ice cover is completely nonexistent. 

Graph 2

Graph 2 below has lines showing the average and trend for ice coverage in the Gulf of St. Lawrence. Although the trend is decreasing, it is clear that it will be many years before the ice cover is completely nonexistent.

Finally, 25 of 36 key informants interviewed noted that vessel escorts are an important activity. Two of the 25 key informants who felt there was a need for vessel escorts also felt that the need has been decreasing over the last few years, due to higher quality commercial ships that need less help navigating through the ice.
 
It is also important to note that the federal government has an obligation to provide ice breaking services for the ferries from Newfoundland and from P.E.I. (which the Confederation Bridge replaced the ferry) to the mainland as part of the agreements that were reached when these provinces each joined the Confederation.

Search and rescue (SAR): Furthermore, South of 60° North latitude the Climate Change Risk Assessment Report found that “increased severity and frequency of severe weather events, storm surges, and coastline changes due to sea level rise could lead to a rise in the number of emergency situations involving vessels.”  These situations potentially translate into increased SAR operations in the more southern regions in winter as well as in the Arctic. 

Ice routing and information:  Ice routing and information services exist to address the same needs as vessel escorts – the need for assistance in navigating through ice infested waters. The CIS supports safe and efficient shipping, contributes to Arctic civil, military, and environmental security and Arctic sovereignty, assists with the UNCLOS Northern continental shelf delimitation, supports Northern communities, and enables offshore industry. They do this through the provision of Ice Service Specialists, Airborne Ice Reconnaissance, Satellite Data Acquisition and Processing, Analysis and Forecasting Services, Ice Information Capture and Display Systems, Training, and more. The ice information provided makes transiting icy waters less risky and reduces the number of vessel escorts needed because ship captains have the tools they need to avoid ice themselves, and reduces the cost of economic activity by providing the most efficient route for the ships. It also helps to prevent flooding by monitoring ice conditions and detecting jams. Ice routing and information was seen as an important activity by 12 of 36 key informants interviewed. It is clear from this that the ice information services are an important and needed service.

Harbour Breakouts: in order for marine trade to exist in the Arctic during the summer and in Southern waters during the winter, harbour breakouts must be conducted to ensure that ships, marine facilities, and infrastructure do not sustain damage from ice and that ships can access the marine facilities. The IB Program Levels of Service (LOS) document states that the harbour breakouts activity directly contributes to the objective of facilitating marine trade, commerce, and ocean development by providing “increased access to marine facilities; reduced risk of ice damage to structural components of ports and marine facilities; maintenance of approaches to St-Lawrence Seaway Authority (SLSA) locks to prevent unscheduled closure of the system; and early breakout of harbours [permitting] fishermen to begin their fishing season earlier and [equalizing] access to fish among inshore fisheries.”  Harbour breakouts were reported as an important need by six of 36 key informants interviewed.

Overall, looking at perceptions of how the Program is addressing client needs, survey suggest that results are split in terms of the extent to which the Program is seen to be addressing these needs. An equal number were of the opinion that the Program meets their needs both to a great extent (eight) and to a little extent (eight). Respondents were also asked if there were any needs the IB Program is not addressing. Results reveal that 17 out of the 26 respondents surveyed responded that there are needs not currently being addressed by the Program. Some of the unmet needs mentioned (through a follow-up open-ended question) include assisting in protecting offshore drill rigs from packed ice, better coverage of late season activities, and a lack of icebreaking assets in the Great Lakes.

4.2 Efficiency

Efficiency Question #4: To what extent is the governance structure clear, appropriate, and efficient for the Icebreaking Program?

Key finding: Overall the roles and responsibilities of the Icebreaking Program are defined and understood within CCG.

Findings

Documents and interviews confirmed that overall the roles and responsibilities of the IB Program are defined and understood within CCG. The two case studies also confirmed that the roles and responsibilities between the IB Program and CIS and USCG respectively, were clearly defined and understood, however the Agreement between Canada and the USCG does not provide formal guidelines for the provision of icebreaking services on the Great Lakes.   

The evaluation found that there are potential issues with respect to the roles, responsibilities and accountabilities in the regions, between the Ice Superintendents and Superintendents, Regional Operations Centres (ROCs). Due to workload issues, the routing responsibility has been relaxed such that Ice Service Specialists (ISS) from Environment Canada have often been permitted to prepare and issue recommended routes, when they should be restricted to providing information on ice and weather conditions and identifying the ice regime along the recommended route. An ISS is not an expert with respect to routing and may not take into consideration factors that are considered important for safe and efficient navigation.    

Finally, the IB Program has weekly (daily when required) conference calls between the regions and headquarters. This provides a ready platform to resolve operational constraints as they arise. The IB Program has an MOU in place that provides a conflict resolution mechanism to ultimately resolve issues at the Regional Director level.

Evidence

Program roles and responsibilities (R&R)
The IB Program provides information to the public via its Internet and intranet websites. The intranet site provides details of the services that are delivered. The intranet provides the mission of the IB Program, which is to “provide icebreaking services of benefit to Canadian commerce, sovereignty and riparian interest, with regard to the marine environment, consistent with client needs and government expectations”. The intranet also provides the roles of the program at headquarters and in the regions, as well as the responsibilities of each team member in the program. The Internet site provides the objectives of the program, the Level of Service and other general information. Almost all interviewees (23 of 28) consider the governance structure (i.e. roles, responsibilities and accountabilities) to be clear.

Regional R&R
The evaluation found that the Standard Regional Organizational Structure document specified that the Coast Guard Regional Icebreaking Superintendents (or ice superintendent) are jointly responsible for managing the effective delivery of icebreaking services to mariners. During the winter ice season, the Icebreaking Superintendents are located in the ROCs. They work with the Superintendents of the ROCs to coordinate the deployment and tasking of CCG icebreaking and aerial reconnaissance resources with the other Regional Ice Superintendents in order to provide seamless service delivery of icebreaking services through a zonal approach. 

In certain cases, the Ice Superintendent must multi-task within a Regional Operations Centre. The multi-tasking has resulted in organizational challenges as described in the document Standard Regional Organizational Structure as follows:

…the workload and responsibilities for both positions result in a reduced capability to support the Icebreaking Program activities, such as:

  • ensuring vessels are assigned to the Icebreaking Program and tasked as per the approved LOS
  • conducting client consultation before and after the ice season,
  • managing daily communications with clients, the public and media during the ice season
  • participating in more frequent National Icebreaking Program meetings (currently reduced to half a day per year), and
  • actively participating in R&D projects.

The Icebreaking Superintendent, as certified ship’s navigation officers, are responsible for producing safe recommended routing though or around ice covered waters. Due to workload issues, this responsibility has been relaxed such that Ice Service Specialists (ISS) from Environment Canada have often been permitted to prepare and issue recommended routes, when they should be restricted to providing information on ice and weather conditions and identifying the ice regime along the recommended route. 

This could be an issue for CCG, particularly in the Arctic, as the recommended route may not take into consideration the hydrographic and geographic features of the area, the accuracy of the marine charts, the scale of the ice analysis chart (lack of features such as shoals and rocks), the draft and capability of the vessel, the ability of an icebreaker to respond safely if a vessel needs assistance in shallow waters, etc.8

R&R between the CCG and CIS
As for the governance structure, roles and responsibilities between the IB Program and CIS under the Ice Information Service Partnership Agreement (IISPA), the case study found these to be clear. Collaboration and communication between the two parties support daily decision-making and contribute to efficient icebreaking operations. At the IISPA Initial Review Workshop (November 2009) there was consensus that the current structure of the partnership works well. Also, all of the key informants directly involved with the CIS, as well as five key informants not directly involved, mentioned that the roles, responsibilities and governance structure between the CCG and the CIS were clearly defined and understood.

R&R between the CCG and USCG
With respect to the Icebreaking Program and the USCG, the case study found that the roles and responsibilities between the CCG and the USCG are considered clearly defined and understood, although they are not written into the Agreement providing for coordination of icebreaking activities of Canada and the United States of America on the Great Lakes–St Lawrence Seaway System.

R&R between Fleet and the IB Program
Input from a key informant and observations by the evaluator during workshops and site visits confirmed that the roles and responsibilities of the Ice and ROC Superintendents with regards to tasking ships are not as clear as they should be. Specifically, in some regions the Ice Superintendent is allowed to task icebreakers, although it is clearly the role of the Superintendent ROC. It is specifically stated in the Fleet and Safety Manual that the Superintendent, Operations Centre “is responsible for planning, organizing and coordinating the day-to-day utilization of regional vessels”.

Operational Constraints
The evaluation also examined whether there is a process in place if operational constraints arise and found that the program personnel from headquarters and all regions involved with ice have daily teleconference calls to coordinate services and discuss issues regarding the IB Program. The ROC superintendents may participate if needed. Finally, the MOU Standing Operational Guidelines on the Icebreaking Delivery in the Gulf of St. Lawrence signed December 21, 1999 states that for conflict resolution “All unresolved disagreements that occur during operations will be referred to the appropriate Directors Operational Services. If unresolved at this level, the issue will be referred to the Regional Directors involved.” 

Efficiency Question #5: To what extent is the design and delivery of the Icebreaking Program clear, appropriate, and efficient?

Key finding:  While the program monitors its performance in several ways e.g. through IISPA, IODIS, etc., a Performance Measurement Strategy has not been implemented.

Findings

The IB Program participates in the Fleet Planning process annually. The process provides sufficient time for Fleet and Regional Ice Superintendents to ensure they have sufficient resources in place to meet the approved LOS. All planners agreed that the November 2009 National meeting in Montreal was a good mechanism to discuss the availability of resources, as well as how the stakeholders could work together to improve the efficiency of the program.

The IB Program monitors its performance through several means e.g. Level of Service (LOS), Ice Information Service Partnership Agreement (IISPA), Icebreaking Operations Data Information System (IODIS. However, with respect to an overall performance measurement strategy (PMS), it has yet to implement one. The draft PMS developed by the Marine Program Performance Measurement Team in 2002 was not implemented. A subsequent draft PMS created in late 2009 still remains to be implemented. The IB Program reports on its activities and results annually through the Management, Resources, and Results Structure (MRRS), specifically through the Performance Measurement Framework (PMF). It does not measure how complementary programs, e.g. ice booms, have contributed to the achieving its results.

Case study findings indicate that although performance reporting measures are in place to assess the overall performance associated with the IISPA, it has not been possible to quantify the success of the agreement because of limited performance information provided by the CIS. It also was found in the case study on the Collaboration between the CCG and USCG in the Great Lakes that because there is no joint PMS in place to monitor and assess the success of the operations between the Coast Guards, it has not been possible to quantify their impact on CCG’s levels of service. However, all case study key informants emphasized the interoperability of the two Coast Guards i.e. they cannot operate efficiently and effectively without each other.

The IB Program’s target response times for the icebreaking fleet were reached on average at least 90% of the time (Graph 12) throughout the period covered by the evaluation. However, a data analysis of IODIS revealed that the IB Program could not provide performance information on all of its operational outputs (indicators) and strategic outcomes established in the LOS document.

Finally, the evaluators examined whether the data collected is being used for decision-making. They found that the performance information provided though the IISPA and IODIS have been used to report results of the IB Program in the MRRS, for CCG business planning, Report on Plans and Priorities (RPPs), Departmental Performance Reports (DPRs) and for the Commissioner’s Daily Briefing. There was insufficient evidence to conclude whether the data was used for decision-making.

Evidence

Planning
For planning purposes, the IB Program works closely with the CCG’s Fleet Planning. The IB Program must request, on an annual basis, the services of the Fleet to respond to the needs of the IB Program’s external clients and to meet its LOS commitments. The planning process, commencing in September of any given year, focuses on the upcoming fiscal year. This provides sufficient time for Fleet to review and assess whether it has sufficient resources and for Regional Ice Superintendents to ensure that operational and capital planning of icebreaking vessels and aircraft is in accordance with the approved Levels of Service.9

A November 2009 national meeting was held in Montreal attended by IB Program personnel, ROC Superintendents and Fleet Planning to discuss the 2009-2010 and 2010-2011 winter seasons. Evaluators observed and key informants confirmed that the national meeting was a good mechanism to discuss the availability of resources, as well as how the stakeholders could work together to improve the efficiency of the program. In short, the planning cycle is clearly defined and understood by the parties involved, however some improvements can be made such as by continuing annual meetings where all partners involved in the delivery of IB Program can discuss the availability of resources.

Performance Measurement Strategy
The IB Program did not implement a PMS covering the period of the evaluation. The draft PMS developed by the Marine Programs Performance Measurement Team in 2002 was not implemented. The program, in collaboration with CCG’s Performance Measurement Division, developed a draft RMAF/RBAF in December 2009, which includes a draft PMS linked to the program’s objectives and which indicates key outputs, outcomes and indicators. Data sources, targets and data collection responsibilities are still to be determined.

Finally, input from key informants and the document review indicate that the IB Program does not measure how complementary services (for example, ice booms) have contributed to achieving program results.

Management Resources Results Structure/Performance Measurement Framework
The IB Program reports activities and results annually through DFO’s MRRS, specifically through the department’s PMF. The document review confirmed that the information on program outputs for the MRRS is in accordance with the LOS and the IISPA. As part of its reporting, the program relies on information from the Transportation Safety Board for one of the expected results.

The expected result “Facilitating the safe and timely movement of maritime traffic through and around ice-covered waters”, and its related performance indicator Informed, safe and timely movement of maritime traffic through and around ice-covered waters is facilitated have been consistently used by the program for several years in the CCG PMF. Finally, the evaluation notes that the Maritime Services Risk Profile 2008-09 stated that “There is a need to improve program management tools to support the effective and efficient management of the CCG IB Program. Namely, the performance measurement and risk management tools require further development to support icebreaking decisions.”

Canadian Ice Services - IISPA
The ice information service provided by the CIS through IISPA is essential to the IB Program. It not only helps the program meet its levels of service and all three objectives, it also supports the CCG in its role in Canada’s Northern Strategy. Case study findings indicate that although measures are in place to assess the overall performance associated with the IISPA, it has not been possible to quantify the success of the agreement because of limited performance information provided by the CIS. 

CCG and USCG on the Great Lakes
It was found in the case study on the Collaboration between the CCG and USCG on the Great Lakes that because there is no joint PMS in place to monitor and assess the success of the operations between the Coast Guards, it has not been possible to quantify their impact on CCG’s levels of service. In addition, due to the lack of performance measurement information on the joint activities on the Great Lakes, it has not been possible to quantify the impact of these operations on the objectives of the IB Program. However, all case study informants emphasized the interoperability of the two Coast Guards; i.e. they cannot operate without each other. Also due to the lack of information, it was not possible for the evaluation to assess whether the joint operations contributed to long-term efficiencies and whether or not the resources were utilized in an economical manner. Although both Coast Guards monitor their activities, performance information is not shared nor jointly compiled.

Levels of Service
The Icebreaking Operations Levels of Service document defines the services that are provided and identifies delivery targets, such as response times. It was first published in October 1990. The version in effect for the period covered by the evaluation dates back to 2001 and was based on a 1997 report by the Joint Industry / CCG Icebreaking Task Force. The most recent version, issued May 2010, falls outside the scope of this evaluation. The 2001 LOS document provides a list of performance measures, which includes indicators and targets. The performance measures established in the LOS are monitored through IODIS.

IODIS was created as a reporting system to monitor the efficiency of the icebreaking fleet and to assess the extent to which service standards are met. The performance information from the icebreaking fleet is entered into IODIS MOBILE by personnel onboard the ships. The IODIS MOBILE files are sent by email to the HQ IODIS Administrator for uploading into the IODIS National Data base. A first quality check is done by the HQ Administrator. Then, a weekly report of the Icebreaking activities is sent to the Regional Offices for a second quality check. IODIS gathers large amounts of data yearly on services provided to clients by the fleet of icebreakers.

The program’s target response for the icebreaking fleet (LOS standards) were reached on average at least 90% of the time (Graph 12) throughout the period covered by the evaluation. However, a data analysis of IODIS revealed that the IB Program could not provide performance information on all of its operational outputs (indicators) and strategic outcomes established in the LOS document.

Finally, the evaluators examined whether the data collected is being used for decision-making. They found that the performance information provided though the IISPA and IODIS have been used to report results of the IB Program in the MRRS and for CCG business planning, RPPs, and DPRs. It also provides performance information for the Commissioner’s Daily Briefing every Thursday morning during the ice seasons (Arctic and winter). There was insufficient evidence to conclude whether the data was used for decision-making.

Efficiency Question #6: To what extent could the efficiency of the Icebreaking Program and its activities be improved?

Key finding:  The program is making good use of complementary services to icebreaking and partner agreements to provide efficient operations, not only for the program itself but in terms of overall resources that include the use of Coast Guard icebreakers.

Findings

Evidence from the evaluation demonstrates that complementary services to icebreaking (e.g. availability of ice information and ice booms) and partnership agreements contribute towards improving the efficiency of the program. While there was no performance information on how these efficiencies have provided monetary savings to the CCG, results from the document review, case studies and key informant interviews confirm the usefulness of such activities in improving the efficiency of the program.

Ice Information Systems10, specifically the availability of ice information, has improved efficiency by reducing reliance on icebreakers. Providing information such as voyage planning, recommended ice routes and tactical navigation has given navigators more autonomy thus reducing the need for icebreakers. In addition, the evaluation found that ice booms contribute to the program’s efficiency by assisting with the formation of an ice cover, reducing the risk of ice jams. As such, ice booms form an integral part of the Quebec Regional Ice Management Strategy, which also includes radars, video cameras and other sensors such as current and water level indicators that reduce the frequency of helicopter reconnaissance flights on the St. Lawrence River.

Three partnership agreements were identified during the evaluation as having contributed to the increased efficiency of the program. These partnership agreements have resulted in some regions delivering the program in a zonal approach; the purchase of satellite imagery to create ice charts and routings; and the delivery of joint operations and cooperation of icebreaking services.

Evidence

Complimentary Services
Existing technologies such as those that provide ice information contribute to improving the efficiency of the IB Program by reducing reliance on icebreakers. Ice information and ice routings allow ships to navigate around the ice or through the lightest ice conditions. The availability of this information from the program has led to less damage to the ships and less reliance on icebreakers. For example, the Quebec region has developed the Marine Information Portal, a website known as MARINFO which includes daily information on ice conditions in the St. Lawrence River and in the Gulf during the ice seasons. Four key informants stated that MARINFO has increased the program’s efficiency by providing information such as position and activities of Coast Guard Icebreaking Units, ice bulletins, ice monitoring system (cameras), ice routing support model, ice recommended routes, ice charts, ice analysis chart, water temperatures and so forth. This prevents the deployment of icebreakers or helicopters, for example, because the information for navigation in ice is readily available on the website. Furthermore, as noted in the IISPA case study, eight of the key informants stated that information such as the ice information and ice routings has assisted with overall decision making and thus contributed to improving efficiency. Specifically, the partnership between DFO and Environment Canada has allowed CIS’s staff to work within the CCG ice offices in support of daily operations. This cooperation reduces duplication of information, allows better communication and efficient decision making.

In addition, the document review found that the Research and Development (R&D) activity of the IB Program primarily focuses on the development and improvement of Ice Information Systems for voyage planning (to assess ice conditions that a vessel is likely to encounter along its planned route), development of recommended ice routes for marine shipping (providing ice routing advice to ships so that they may navigate safely around difficult areas of ice), and tactical navigation in ice (planning the route to take advantage of optimum ice conditions). Ultimately, these systems increase the autonomy of navigators as they receive enhanced information about current ice conditions and have less reliance on the CCG for icebreaking (i.e the use of the icebreakers). These systems will also enable the Polar Icebreaker to remain in the Arctic for up to nine months a year rather than four months a year. During those nine months, the Polar Icebreaker requires the appropriate tools to navigate in darkness, which are available through Ice Information Systems. These systems will continue to evolve through pilot projects as well as through additional research and development.

Several key informants in the Quebec region and the document review revealed that the ice booms of the Waterways Management Program are a complimentary service that contributes to efficient operations. Ice booms are physical structures complemented by an array of remote sensing devices such as cameras and radar stations installed along the navigation channel11. Ice booms on Lake St. Pierre increase the efficiency of artificial islets located on both sides of the navigation channel. Together, these structures also assist with the formation of an ice cover at the beginning of the winter season and help keep it in place all winter and out of the navigational channel12.  

Placed at strategic locations, the ice booms are an integral part of the Regional Ice Management Strategy which is based on the operation of all control works (booms, artificial islets and reservoirs). By controlling ice formation, ice booms help minimize the use of icebreakers.13

Partnership Agreements
The evaluation also found that three partnership agreements contribute to the increased efficiency of the program. The first partnership agreement is the Standing Operational Guidelines on the Icebreaking Program Delivery in the Gulf of St. Lawrence, an agreement between DFO’s Quebec, Newfoundland and Labrador and Maritime regions. These three regions have agreed to deliver the IB Program through a zonal approach. This agreement gives the Quebec Region operational control of major portions (main corridor) of the Gulf. For example, prior to the zonal approach, each region had their ships and was not under any obligation to share resources, whereas now they are. Key informants indicated they consider this zonal approach as increasing efficiency, although several reported that tensions exist between regions with respect to actual resource sharing.

Results from the document review and IISPA case study suggest that the CIS and the CCG partnership agreement also improves the program’s efficiency. Specifically, through this partnership agreement, satellite imagery is purchased to create ice charts. These charts provide information such as the location of the ice edge, ice type, floe size, landfast ice and deterioration, which permits the CCG to establish ice routings for navigators. As previously mentioned, the ice charts and ice routings are provided to navigators to safely navigate through ice-infested waters without the assistance of icebreakers, which contributes to the efficient use of resources beyond the program itself, i.e. the efficient use of Coast Guard resources. In addition, the IISPA provides for ISSs onboard ships and in the ROC (in each region) to assist CCG employees in analysing ice information. 

Finally, the CAN/US Agreement providing for coordination of icebreaking activities of  Canada and the United States of America on the Great Lakes–St. Lawrence Seaway System achieves efficiencies by enabling the two Coast Guards to work together in delivering icebreaking services. As per the Agreement, the Coast Guards “may make icebreaking facilities [e.g. vessels, buildings etc.] and personnel available to one another”. This provision of the Agreement is applied frequently when CCG’s icebreakers are completely tasked. The vessels operated on the Great Lakes by the Coast Guards differ: the CCG operates two icebreakers which are larger and capable of handling a broader range of ice conditions compared to the majority of the USCG’s eight or nine vessels, which include icebreakers, tugs and buoy tenders. A major difference between the two partners is that CCG vessels operate 24/7, whereas the USCG ships operate in the daytime only. Results from the case study suggest that the joint operations and cooperation made possible by the Agreement have become essential to providing efficient icebreaking services on the Great Lakes. For example, as noted in the case study, a USCG tug can break ice where there is limited manoeuvrable space while a larger CCG icebreaker can open the Seaway. The USCG icebreakers are often made available to support CCG activities. The most recent demonstration of this cooperation was during the winter of 2009/2010. Because CCG’s resources were reduced due to CCGS Samuel Risley being out of operation with an unforeseen breakdown, the USCG offered additional resources in support of ice operations on the Great Lakes. Similarly, the opening of the St. Lawrence Seaway occurs during a period of high demand. CCG icebreakers are required at the same time for other purposes in other regions, such as for seal fisheries, flood control in the Quebec Region and ferry services (e.g. Magdalen Islands, Blanc-Sablon) when there is still ice in the Gulf of St. Lawrence. During this time, there is also a need in Thunder Bay for harbour breakouts. In support of the CCG, the USCG provides the service for Thunder Bay, allowing CCG resources to concentrate on serving the St. Lawrence Seaway and Gulf.

In summary, results from the evaluation found that complimentary services such as ice information, ice booms (Integrated Ice System) and partnership agreements contribute to the efficiency of both the IB Program itself and the use of Coast Guard resources.

4.3 Effectiveness

Effectiveness Question #7: To what extent has the Icebreaking Program achieved its three objectives?

Key finding:  Evidence shows that the IB Program has produced its intended outputs and has contributed to the achievement of its objectives.

Findings

There was evidence that the IB Program has produced its intended outputs for most of its key objectives and survey results suggest that there is overall achievement of these as well. 

With respect to facilitating the safe and timely movement of maritime traffic through and around ice-covered water, the program provided ice reconnaissance with vessels and helicopters, ice information to client ports and client vessels (see Graph 3). The survey covered a small population of clients as such the results are suggestive and should not be representative of the all the clients of the IB Program. Survey results reveal a strong belief that ice information and ice-routing advice allow clients to navigate safely, timely, and efficiently through ice-infested waters;  the program provided route assistance (see Graph 4) and conducted harbour breakouts (see Graphs 5,6,7).  

Overall, survey results suggest fairly positive views on this issue with, for example, 16 out of 26 respondents feeling the IB Program facilitates efficient vessel transits and movements through or around ice covered waters to at least some extent, and 15 out of 26 feeling that the Program facilitates timely vessel transits and movements through or around ice covered waters to at least some extent.

With respect to minimizing the effect of flooding caused by ice jams in the St. Lawrence River, the program provided icebreaking services as well as reconnaissance missions for flood control (see Graph 10). 

With respect to assisting in the re-supply of the northern communities for which there is no commercial services, the program began collecting data in IODIS since the summer of 2009 making it difficult to assess achievement of this outcome. Other operational data is collected from a number of CCG sources, however assessing these for inclusion went beyond the scope of the evaluation.

The Icebreaking Program’s response time met the required standard more than 90% of the time for the period 2005-06 to 2009-10, even though some years had above average ice conditions.  

Evidence

A program’s objectives/outcomes are accomplished through the provision of its services/activities. The analysis below describes the services provided by the IB Program. Illustrating the production of services and outputs provides a certain degree of assurance that a program is poised to achieve its outcomes. An absence of the provision of services would indicate that a program was at risk of not achieving its outcomes. A discussion of the production of outputs or services is conducted below for the following services or program activities in the context of its respective objectives/outcomes.

Objective #1: To facilitate the safe and timely movement of maritime traffic through and around ice-covered water;

Views on Impacts of Icebreaking Program
Survey respondents were asked to what extent the service provided by the IB Program facilitated a number of outcomes. Results suggest fairly positive views on this issue. Eighteen out of the 26 respondents were of the opinion that the IB Program facilitate safe vessel transits and movements through or around ice covered waters to some (6) or a great (12) extent. Sixteen out of 26 responded that the IB Program facilitate efficient vessel transits and movements through or around ice covered waters to at least some extent, and 15 out of 26 responded that the Program facilitates timely vessel transits and movements through or around ice covered waters to at least some extent.

Ice Routing and Information services

The program provided ice reconnaissance with vessels and helicopters, ice information to client ports and client vessels during the scope of the evaluation. These services are tracked using IODIS and via the IISPA (i.e. hours of helicopter and aircraft reconnaissance). In 2009, CIS started to provide the number of Recommended Ice Routes. Graph 3 below is a summary of ice routing and information services provided in all regions. The winter of 2009-2010 was identified as a light ice year and as a result fewer ice routing and information services were provided. However, from the winter of 2006-2007 to 2008-2009 services increased in at least two regions. In Newfoundland, route assistance and ice routing and information services increased during the winter 2006-2007 compared to other years. The climactic temperature was above normal until March. During March and April temperatures dropped to below or slightly below normal but extreme wind created strong ice pressure and ice pack against the Newfoundland coast during April, causing significant problems for the shipping and fishing industry. The CCG and CIS implemented special forecast and ice information reconnaissance to keep the fishermen, the CCG and the entire marine community up to date about upcoming weather at that time. Even though this information was provided, more than 100 vessels became beset along the northwest coast of Newfoundland during this time. Consequently, icebreaker assistance was in higher demand for the period. 

Graph 3

Graph 3 below is a summary of ice routing and information services provided in all regions.
Source: CCGAPS

Survey results suggest a strong belief that ice information and ice-routing advice allow clients to navigate safely, timely, and efficiently through ice-infested waters. Nineteen out of the 26 respondents agree with this idea to a great extent.

Route Assistance
Route assistance is measured by the number of escorts provided to clients such as domestic and foreign commercial vessels, ferries, ferry terminal operators, fishing vessels, Canadian and foreign government vessels, owners and operators of the Canadian marine ports and harbours infrastructure, fish processing companies, and the general public The graph below provides the total number of escorts accomplished by the IB Program per year since 1997.

Graph 4

Graph 4  provides the total number of escorts accomplished by the IB Program per year since 1997.
Source: CCGAPS

As shown in Graph 4, in 1999-2000 and 2009-2010 the winters were warmer, thus fewer vessel escorts were required during those years. As described in the Icebreaking Operations Levels of Service, there are limitations that may affect the delivery of route assistance services and they are: weather restrictions, severity of ice conditions, physical restrictions, safety restrictions and availability of resources. During peak periods, such as the Seal Hunt, there are fewer icebreakers available for icebreaking support so the numbers do not reflect this.

Facilities and Port Maintenance i.e. Harbour break outs
Facilities and Port Maintenance are measured through the number of hours for harbour breakouts. As such Graphs 5, 6, and 7 describe the number of hours per year (from 2004-2005 until 2009-2010) for the four regions for the three types of harbour breakouts i.e. commercial, fishing, and assistance in shipping harbour involved in the delivery of these services.

Graph 5
Commercial harbour breakouts are provided in ports where commercial icebreaking tugs are not available, as per Icebreaking Directive # 2.

Graphs 5, 6, and 7 describe the number of hours per year (from 2004-2005 until 2009-2010) for the four regions for the three types of harbour breakouts i.e. commercial, fishing, and assistance in shipping harbour involved in the delivery of these services.
Source: CCGAPS

Graph 6

Graphs 5, 6, and 7 describe the number of hours per year (from 2004-2005 until 2009-2010) for the four regions for the three types of harbour breakouts i.e. commercial, fishing, and assistance in shipping harbour involved in the delivery of these services.
Source: CCGAPS

Graph 7

Graphs 5, 6, and 7 describe the number of hours per year (from 2004-2005 until 2009-2010) for the four regions for the three types of harbour breakouts i.e. commercial, fishing, and assistance in shipping harbour involved in the delivery of these services.
Source: CCGAPS

Regions spend more time on commercial harbour breakouts than fishing harbour breakouts and assistance shipping in harbours. Graph 6 demonstrates that the Newfoundland and Labrador region spends more hours assisting in icebreaking for fishing harbour breakouts than any other region, whereas Quebec region spends more hours than the other regions on providing assistance in shipping harbours.

Performance Measurement of the first objective
The first objective of the IB Program has been measured by the Transportation Safety Board of Canada since 1989. Graph 8 below demonstrates that there has been an overall decrease in shipping accidents damaged by ice since 1989. However, for the period by the evaluation there was an increase of vessels damaged by ice in 2007 and 2008, which exceeded the 2004-2008 average of 11 damaged vessels. This was due to the difficult ice season and the number of small sealing vessels damaged in icy conditions during the annual seal hunt fishery.  

Graph 8

Graph 8 below demonstrates that there has been an overall decrease in shipping accidents damaged by ice since 1989.
Source: Transportation Safety Board of Canada

Objective #2: To minimize the effect of flooding caused by ice jams in the St-Lawrence river;

Flood Control
The Quebec region and Central and Arctic are the two main regions that deliver this activity as reflected in the following graph.

Graph 9

The Quebec region and Central and Arctic are the two main regions that deliver this activity as reflected in the following graph.
Source: CCGAPS

Another measure for this activity is the number of icebreaking missions for flood control (Graph 10 below).

Graph 10

Another measure for this activity is the number of icebreaking missions for flood control (Graph 10 below).
Source: CCGAPS

Objective #3: To assist in the re-supply of the northern communities for which there is no commercial services;

Northern Re-Supply
The program collected data with respect to Arctic issues in general, but mostly for the sealift operations, in the summer 2009. Data is entered by vessel crews and regional offices when the activity is performed. There is no data available for services such as Northern Re-supply prior to the summer of 2009 in IODIS. Other operational data is collected from a number of CCG sources, however assessing these for inclusion went beyond the scope of the evaluation.

Overall services in perspective
Data analysis shows that the breakdown of services rendered on average for 2004 to 2010 is approximately 51% for route assistance, 16% facilities and port maintenance, 10% ice routing and information and around 23% flood control every year, as demonstrated by the following graph.

Graph 11

Data analysis shows that the breakdown of services rendered on average for 2004 to 2010 is approximately 51% for route assistance, 16% facilities and port maintenance, 10% ice routing and information and around 23% flood control every year, as demonstrated by the following graph.
Source: CCGAPS

Overall compliance with LOS response times for activities and outputs
The LOS defines the activities of the IB Program, as well as delivery targets. As such, these targets have been used by the program for performance measurement, such as in the annual Performance Measurement Frameworks. In Graph 12 it is possible to see that generally the % LOS response time per service was maintained above 90% across the years even though some years have been considered above normal ice years. Overall, the IB Program has been successful in achieving its response rate identified in the LOS for the period covered by the evaluation.

Graph 12

In Graph 12 it is possible to see that generally the % LOS response time per service was maintained above 90% across the years even though some years have been considered above normal ice years. Overall, the IB Program has been successful in achieving its response rate identified in the LOS for the period covered by the evaluation.
Source: CCGAPS

When asked about overall satisfaction with the IB Program, survey results suggest somewhat mixed views, but with a lean to the positive. A roughly equal number of respondents indicate that they are very satisfied with the Program (eight) as indicate they are not satisfied (seven), while the plurality (ten) are somewhat satisfied with the Program.

Effectiveness Question #8: To what extent are agreements in place, up-to-date, and effective to support the implementation and delivery of the Icebreaking Program?

Key finding:  Of the four main agreements that support the delivery of IB Program services, two have not been regularly reviewed or updated, and the other two have five-year review and update cycles and are up for renewal in December, 2010.

Findings

The evaluation examined whether agreements are in place that have impacted the delivery of services, whether they are up to date and effective. As such, there are four main agreements that support the delivery of IB Program services:

  • Joint Industry-Coast Guard Guidelines for the Control of Oil Tankers and Bulk Chemical Carriers in Ice Control Zones of Eastern Canada (JIGs), a responsibility identified in the MOU between TC and CCG.
  • Levels of Service (LOS)
  • Agreement providing for coordination of icebreaking activities of Canada and the United States of America on the Great Lakes–St Lawrence Seaway System (Canada/US agreement)
  • Ice Information Services Partnership Agreement (IISPA).

Status

The agreements need to be reviewed and updated on a regular basis to ensure they support the types and levels of service required to meet client needs, taking into account the program’s capacity. Changing climate conditions are making ice navigation far more hazardous, particularly in the Arctic. As well, demand continues to increase for icebreaking services in all regions requiring such services. These factors have implications for the services that are covered by the agreements.

The evaluation found that two of the agreements have not been regularly reviewed and updated. JIGs is (a guideline issued by Transport Canada which CCG provides support to) considerably out of date, having come into effect in November 1995. The agreement, for which Transport Canada has the lead responsibility, has not been revised since, despite a review in 2005. JIGs applies only to the Maritimes and Newfoundland & Labrador Regions, unlike the other agreements which cover more than one region. The LOS document in effect for the period covered by the evaluation was dated 2001. That version has since been replaced as of May 2010, which was based on the 2007 Levels of Service (LOS) Review. A review of the newest version was outside the scope of this evaluation. As such, the evaluation cannot confirm whether or not the 2010 LOS document is in fact up-to-date.

The other two agreements have five-year review and update cycles. The Canada/US agreement has adhered to this cycle since 1990, and IISPA was under review at the time of the evaluation in preparation for renewal at the end of its first five years. Both agreements are up for renewal in December 2010.

The importance of ensuring all agreements are up-to-date is underscored in the CCG’s introductory message to the 2010 LOS document. It cites the 2007 Report of the Auditor General of Canada’s recommendation that the CCG make better progress in achieving up-to-date national policies, standards and levels of service.

Effectiveness

There are specific aspects of each agreement that could be improved to strengthen their effectiveness. The LOS would be more effective if it included the IB Program’s service priorities were applied consistently across the country, as the priorities have been a source of confusion and open to interpretation from one region to the next. To avoid such confusion and varying interpretation, the priorities need to be more clearly defined and detailed.

The IB Program does not have complete performance data for either the Canada/US agreement or for services delivered through IISPA. The Canada/US agreement could better support operations by requiring the two Coast Guards to have an integrated set of performance data on the joint operations, in order to provide a comprehensive picture of the services provided. IISPA requires monthly and seasonal CIS reports to be submitted to the Manager, IB Program. During the scope of the evaluation the monthly reports were made available to the IB Program 42% of the time. No seasonal reports have been submitted since a winter 2006-07 report. Either the agreement needs to be strengthened in this regard or some other action is required to ensure the agreement is fully implemented. The lack of operational data for services delivered through these last two agreements hinders the CCG’s ability to fully assess both service delivery and the effectiveness of these agreements.

Evidence

 JIGs

Status and Effectiveness
The 1995 JIGs was under review in 2005 by the Shipping Federation, CCG and Transport Canada, but wasn’t updated then. No recent progress has been made with respect to its current. Evaluators observed this issue being discussed at a 2009 National Icebreaking Post-season Workshop. A review of the document confirmed that these guidelines are outdated. A key informant noted that the guidelines provide only a vague directive and no criteria as to when to activate Ice Control Zones. This comment suggests the agreement is ineffective in supporting the delivery of the IB Program.

LOS

Status: Current Version Released May 2010
Although an updated Maritimes Services LOS document came into effect in May 2010, which was based on the 2007 Levels of Service (LOS) Review, the Icebreaking LOS has not followed a five-year review and renewal process since being introduced in 1990. The 2001 version, with revisions based on a 1997 Joint Industry-Canada Coast Guard task force report, was in effect for the period covered by this evaluation. Therefore, the evaluation is based on the 2001 version.

Changing conditions necessitate the update of this document more frequently than has occurred. This was noted in a few of the interviews with program staff and clients. The LOS was discussed in 14 of the 54 evaluation interviews, and six of the 14 respondents observed that client needs, available resources and the climate have all changed since 2001.

A survey conducted as part of the evaluation suggests that clients consider the LOS to be out of date. There is a high level of awareness of the LOS among clients, with 24 of 26 respondents being at least somewhat familiar with the LOS. Of the 24, only five thought the LOS was up-to-date, while 11 did not and the remaining eight did not know or gave no response. 

The evaluation does not address the extent to which the May 2010 Maritimes Services LOS version has dealt with any shortcomings of the 2001 version. As such, the evaluation cannot confirm whether or not the 2010 LOS document is in fact up-to-date.

Effectiveness
Comments from the CCG’s 2007 LOS review provide evidence of client concerns about a lack of capacity to meet changing needs. For example, Great Lakes and Arctic shippers were working to increase the duration of their season, and they were concerned that the CCG would not be able to extend its icebreaking LOS dates to encompass an expanded season. Other feedback indicates there were concerns about inconsistencies in service priorities across the regions and the IB Program’s ability to respond to unforeseen events, as can be seen from the following comments from clients of the IB Program that have been provided to the program and are available on the program’s website:

  • “The priorities for icebreaking services seem to differ from one region to another across the country, and there is concern about the lack of flexibility in icebreaker deployment to respond to unforeseen events.” (Quebec Region)
  • “The Coast Guard in Newfoundland region seems to give higher priority to icebreaker services for fishing ports than Quebec region. This situation is negatively affecting Lower North Shore residents, particularly during the opening of the seal hunt. Therefore request: ensure that the same level of service is provided across Canada; ensure better coordination between Coast Guard icebreaking activities and Fisheries and Oceans Canada fishing monitoring activities.” (Quebec Region)

Six key informants confirmed that problems have arisen in applying the IB Program’s prioritization system from one region to the next. The prioritization system, shown in Table 2, is not part of the LOS (2001). The priorities are neither clearly expressed nor sufficiently detailed according to the key informants.

Table 2: IB Program Prioritization System

Priority

Service

I

Distress, emergencies and flood control

II

[Note: The wording shown here has been updated following discussions in late 2009.]
Service requests from ferry services provided in accordance with the Terms of Confederation will be given priority. Other ferry services will receive priority as deemed appropriate by the CCG.

III

Ships with vulnerable cargoes (i.e. pollution potential, dangerous goods, perishable) and vessels transporting cargo that is vital to the survival of communities.

IV

Marine traffic, fishing vessels and commercial ports

V

Fishing harbour breakouts

The wording in various regional operational procedures has given rise to different understandings and interpretations of the priorities from region to region. Four key informants specifically mentioned Priority II, which deals with requests from ferry services, as a major source of confusion. Priority II distinguishes between

a)   ferry services provided in accordance with the Terms of Confederation (i.e., PEI Terms of Union and the Newfoundland Act), which obliges the federal government to maintain a navigable channel for ferry services from each of these provinces to the mainland year-round, and

b)   other ferry services.

Not only is the wording not clear, but the four key informants also noted a lack of reference to within-province ferry services as a source of further confusion. For example, Quebec has about seven ferry routes within the province requiring ice-breaking services, but Priority II as stated above does not address this.

Following the 2010 National Icebreaking Pre-Season Workshop held in November 2009, the wording for Priority II was changed, by the attendees of the workshop, to cover all ferry services and to more clearly distinguish between the two main categories now identified as A – Terms of Confederation ferries (higher priority than B) and B – Regular Ferries. Given that the LOS is intended to provide clients with a clear understanding of the services they can expect to receive, it is appropriate to expect that a more detailed and clearly defined prioritization table than the one above has been included in the updated 2010 LOS. 

Canada/US Agreement

Status: Scheduled for Renewal December 2010
As specified in the Case Study: Collaboration between the CCG and USCG in the Great Lakes, the Canada/US agreement has had a five-year renewal process in place since 1990. Renewal is through a formal Exchange of Notes, with the last exchange signed December 5, 2005. Hence, it is up for renewal in December 2010.

Effectiveness
While the agreement provides for coordination of icebreaking activities between the Canadian and US Coast Guards, it does not provide formal guidelines for their respective roles and responsibilities. However, this does not appear to be a hindrance in practice. As identified in the case study, the key informants consider the roles and responsibilities of the two Coast Guards to be clearly defined and understood with regards to their cooperation. As well, during a site visit to Sarnia, evaluation staff observed during discussions with both Coast Guards that they work together collaboratively.

The case study informants plus another key informant consider the agreement to be simple and flexible enough to address icebreaking needs. The case study informants indicated that the agreement gives the Coast Guards latitude in adjusting daily operations according to changing requirements, such that they are able to deal effectively with the unpredictable level of service need.

According to the document review, marine transport in the Great Lakes is predicted to increase in the future and potentially have an impact on icebreaking requirements for the region. Both the document review and the key informants indicated the agreement allows for cooperation between the two Coast Guards and that it gives them the autonomy and latitude to develop compatible systems for daily operations. Analysis of IODIS data shows that the icebreaking activities provided in the Great Lakes are variable and difficult to predict. All Icebreaking services provided by the USCG Icebreakers are not captured in IODIS. The agreement has a flexible structure that permits regular adjustments to deal with this situation.

However, a complete picture of the services provided is not available, as there is a lack of integrated information. The document review and key informants confirmed that data collection is done separately by each Coast Guard and the information is not shared. As no data is compiled on the joint operations, there is no clear picture of icebreaking activities on the Great Lakes.  

IISPA

Status: Scheduled for Renewal December 2010
IISPA provides the framework for the provision of ice information services from the CIS facility in Ottawa to the IB Program. The CCG’s main obligation under IISPA is a financial contribution for services to meet its ice information needs. IISPA, which forms an integral part of the overall MOU between DFO and Environment Canada, was first signed on December 13, 2005, covering a period of five years. It replaces Annex II of the Memorandum of Agreement Between the Atmospheric Environment Service, Department of the Environment and the Canadian Marine Transportation Administration, Department of Transport, 1984. At the time of this evaluation, the agreement was being reviewed by both organizations in order to renew it by the end of 2010.

As described in the Case study: Collaboration between the CCG and the CIS every year, in accordance with the agreement, an exchange of letters between DFO and EC confirms the level of service and contributions for the fiscal year. The CCG covers approximately two-thirds of the costs for CIS (salaries and operations) and contributed close to $9.3 annually for the period 2005-06 to 2009-10. This represents over 45% of the IB services sub-activity’s budget.

Effectiveness
To achieve the objective of safe and timely movement of maritime traffic through ice-covered waters, the IB Program uses ice information provided by CIS for effective icebreaking operations and to create recommended ice routing and advice for ships. This reduces the number of icebreaker escorts required and supports the provision of efficient and effective icebreaking services. The extent to which IISPA is effective in supporting service operations is directly affected by the level of services agreed to in the annual exchange of letters. If the CCG’s or the CIS’ level of support decreases, then it could impact the effectiveness of IISPA.

Both the document review and three key informants confirmed that ice information is essential for delivery of IB Program services. Many documents, such as the CCG ice information requirements and the Ice Services National Action Plan Review, verify that the CCG requires accurate and timely ice information in order to conduct safe, efficient and effective icebreaking operations and to provide precise ice routing and information to the marine community. Thus, IISPA is integral in supporting the delivery of services. Discussions at the IISPA Initial Review Workshop held in November 2009 indicated that the current governance structure, roles and responsibilities work well.

The need for ice information is affected by a number of factors such as climate change, trends in commercial traffic movements and economic activities. These factors are particularly important in the Arctic where marine activity is increasing, thus increasing the demand for icebreaking services and the need for ice information. The fact that the need for ice information will increase, especially in the Arctic, came up in the group interview with CIS personnel.

Any decrease in resource level by either CIS or the IB Program could have an impact on the other. Comments from three IB Program key informants, the CIS group interview participants and five others indicate they all concur that reduced resources would have an impact on either CIS or the IB Program. They suggested, for example, that reduced CIS resources could decrease the quality and timeliness of ice information, as well as any capacity for improvement. Less ice information could result in a need for more escorts and less effective icebreaking operations which affects fuel consumption and LOS response times, among other things.

In order for the CCG to monitor the services provided by the CIS on an ongoing basis, it was recommended in the IISPA that monthly reports be forwarded to the Manager, IB Program. The evaluation found these reports were made available to the IB Program 42% of the time. Seasonal reports (winter and summer) were available from winter 2004-05 to winter 2006-07, but no information has been made available since then. Therefore, it is difficult for the CCG to assess the level of service provided by CIS and identify gaps between actual and expected levels of service. Either the agreement needs to be strengthened with respect to provision of data form the CIS to the IB Program, or some other mechanism needs to be in place to ensure the agreement is fully implemented.

4.4 Economy

Economy Question #9: To what extent is the Icebreaking Program utilized in an economical manner?

Key finding:  The evaluation found that the IB Program has been operating in an economical manner.

Findings

Through the analysis of documents, key informant interview and available data, we found that the IB Program has been operating in an economical manner. The program has been able to maintain its Levels of Service (on average) above 90% (refer to Graph 12), with fewer available ships (see Table 3 below). Five vessels and one air-cushioned vehicle have been decommissioned since 1997. While these ships were decommissioned they have been replaced by two air-cushioned vehicles and one medium icebreaker. This finding was supported by key informants where in fact 45% noted that the program does not have enough ships to respond to service needs. Furthermore, the remaining icebreakers are aging, which could impact the overall capacity of the fleet to maintain its LOS in the future.

The IB Program has leveraged USCG icebreaking capacity in the Great Lakes and has met its LOS Standards 82% of the time (see Graph 13). For example, the CCG has only two icebreakers on the Great Lakes compared to nine icebreakers for the USCG.

A comparative analysis of the literature compared Canadian icebreaking services to four countries (Russia, Finland, Sweden and the United States of America) and found that i) some governments elect to provide icebreaking services through the private sector, solely through the public sector or through a combination of both; ii) Canada has the largest fleet of icebreakers in the world; iii) Canadian icebreakers are multi-tasked, which improves the fleet’s overall efficiency and is regarded as a leading-edge best practice; iv) some countries, including Canada, collaborate with one another at varying levels to provide icebreaking services; v) Russia, Finland and Sweden place some restrictions on the types of vessels that are eligible for icebreaking services and Finland further restricts which ports will receive icebreaking services; and vi) all (with the exception of the US) countries levy some form of fee for the provision of icebreaking services.

Evidence

Material resources
The evaluation found that the IB Program has minimized the use of material resources, such as ships. In spite of this, the program has been able to maintain its Levels of Service (on average) above 90% refer to Graph 12. Five vessels and one air cushioned vehicle have been decommissioned since 1997. While these ships were decommissioned they have been replaced by two air-cushioned vehicles and one medium icebreaker as described in the table below.
 
Table 3: Decommissioned vessels versus new vessels since 1997

# of Vessels in 1997

21

# of New Vessels since 1997

3

# of Decommissioned Vessels since 1997

6

# of Vessels in 2010

18

Forty-five per cent of key informants mentioned that the program does not have enough ships to respond to service needs. Twenty-five per cent of key informants contradicted this statement by saying that the program is at the minimum point for vessel resources, but 30% of them explained that for a given heavy ice year , the program would not be able to respond in time due to an insufficient number of available vessels. This unavailability of vessels could impact the LOS. In addition, 30% of all interviewees stated that a vessel’s availability to respond to client requests in a reasonable timeframe depends on ice conditions. Furthermore, the remaining icebreakers are aging, which could impact the overall capacity of the fleet to deliver the LOS of the IB Program. As demonstrated in the following table, all vessels are over 20 years old and in three regions (shaded) vessels are exceeding design life.

Table 4 Overall Average - Icebreaker Age in 2009

 

 

Total # of vessels

Average age in 2009

# and % of vessels exceeding design life

# and % of vessels over 20 years old

Maritimes

4

27.75

1

25%

4

100%

Newfoundland

4

23.25

0

0%

4

100%

Pacific

1

23

0

0%

1

100%

C&A

2

31.5

1

50%

2

100%

Quebec

5

30.4

2

20%

5

100%

 

 

 

 

 

 

 

Total average

16

27.18

4

19%

16

100%

Collaboration on the Great Lakes
As described in the Case Study on Collaboration between the CCG and the USCG on the Great Lakes, the USCG has approximately nine icebreakers available for icebreaking on the Great Lakes, while the CCG operates only two vessels. Both Coast Guards collaborate to break ice in various areas of the Great Lakes remote from one another. For example, the USCG may break ice for a harbour in Canadian waters in Lake Superior, allowing CCG icebreakers to concentrate on icebreaking in the eastern portions of the Great Lakes in international waters. The CCG has minimized the use of resources on the Great Lakes while maintaining their LOS standards as demonstrated in the following graph.

Graph 13

The CCG has minimized the use of resources on the Great Lakes while maintaining their LOS standards as demonstrated in the following graph.
Source: IODIS

Comparative analysis
Our review of the literature compared Canadian icebreaking services to four countries (Russia, Finland, Sweden and the United States of America). All countries provide similar icebreaking-related services, such as navigation coordination, escorts and re-supplying. However, some governments elect to provide icebreaking services either through the private sector, as is the case for Sweden, solely through the public sector, such as for Canada, Russia and Finland or through a combination of both, such as for the US. In Russia, some major companies have their own small icebreaker to service their installations and tankers.

Country capacity varies with Canada having the most icebreakers at 18, followed by the US at 13, Russia with approximately 13, Finland with seven and Sweden with eight. For some time now, all Canadian icebreakers have been assigned to a variety of dutiesconcurrently. Multi-tasking, as it is referred to, employs icebreakers for a variety of uses, such as SAR or buoy tendering. This approach improves the program’s efficiency, and is seen as a leading-edge best practice. Finland has recently adopted this approach, whereas there was no evidence to suggest Russia, Sweden or the US have done so.

Most countries collaborate to provide specific icebreaking services. For example, the CCG and the USCG have two bilateral agreements for the coordination of icebreaking efforts in the Great Lakes and in the Arctic (as identified in the Canada/US Case Study). Finland and Sweden work in close collaboration across the Baltic Sea Area, both strategically and operationally. They have developed an IBNet data system, which both countries use as a data collection/management system. The system transmits daily ice forecasts and changes in ice conditions, as well as special forecasts for various sea areas, to its icebreakers. During harsh winters, the Nordic countries share shipping assistance duties in the southern Baltic Sea and the Gulf of Bothnia. Finally, all countries generate and use some form of ice information, such as ice forecasts, ice charts and so forth, developed either by themselves or collaboratively. 

Russia, Finland and Sweden invoke various navigational and traffic restrictions. For example, during winter navigation, vessels are required to obtain an ice classification rating or to indicate their ability to navigate in ice if they are ice strengthened, and assistance is only given to those meeting the minimum requirements. Port closures are another means of restricting traffic. Finland restricts its icebreaking services to 23 of its 60 ports during the ice season In Canada, any vessel can enter Canadian waters regardless of its ice classification, and all ports remain open year round.

CCG’s IB Program levies an Ice Services Fee (ISF) during the winter ice season in southern Canada. The ISF is charged as a fixed price per ship, per transit, to a maximum of eight times per ice season and no more than three times within a 30-day period. In Russia, icebreaking dues correspond to a vessel’s category and the time of the year. Finland and Sweden have no special fee for icebreaking, but recover costs through fairway dues that are based on ship size and ice class.

4.5 Lessons Learned

The IB Program, through certain activities, contributes to effective and efficient use of resources beyond the program itself, specifically by facilitating the efficient use of Coast Guard resources. In fact, it was observed that some specific activities (e.g. ice routing and information services) and technologies (e.g. radar and MARINFO) are in place so that users rely less on CCG icebreakers and more on the IB Program. This allows for navigators to become more autonomous and navigate safely through ice without an icebreaker. Investing in technologies that make navigators more independent vis-à-vis navigation in ice with services provided by the IB Program allows CCG to concentrate its icebreaking resources on meeting other priority demands.

It was also observed that the success of the program is dependent on the climate of the icebreaking season and the amount of ice produced in a given year. The variability of the weather impacts the services required by clients.

5. Conclusions and Recommendations

5.1 Relevance

The IB Program is well aligned with departmental and government wide priorities. The Program is aligned with the federal government priorities of Stimulating our Economy, Supporting Families and Communities, as well as a number of important strategies, legislation and a departmental strategic outcome. For example, the Program aligns with Canada’s Northern Strategy priorities of sovereignty and economic development, as well as some of the objectives of theCanada Shipping Act, 2001. The IB Program also aligns with the DFO strategic outcome of Safe and Accessible Waters, and supports the CCG’s mission of providing services in support to government priorities and economic prosperity, and contributing to the safety, accessibility, and security of Canadian waters.

The evaluation also concludes that the roles of the federal government and of the CCG are appropriate in delivering the IB Program. The IB Program is mandated by several pieces of legislation. The principle authority for icebreaking services is derived mainly from the Constitution Act, 1867, s.91(10), which gives the Parliament of Canada the Legislative Authority for Navigation and Shipping, and the Oceans Act, which entrusts the Minister of Fisheries and Oceans with the responsibility to ensure safe, economical, and efficient movement of ships in Canadian waters, including icebreaking and ice management. 

Finally, this evaluation concludes that the IB Program does address several continuing needs for Canadians. All of the activities of the Program (route assistance, ice routing and information services, harbour breakouts, flood control/ice management, northern resupply, and support to Arctic sovereignty) are necessary. Vessel escorts and ice routing and information services help maintain maritime safety by reducing the risk of ice damage to ships. This also helps reduce the number of environmental incidents resulting in pollution. The northern resupply activity is important, because it provides vital supplies to northern communities at a much lower cost than other options. The IB Program also ensures a Canadian presence in the Arctic, which is beneficial to Arctic sovereignty. The IB Program has a significant impact on economic development in the North, as well as economic activity in southern waters. Vessel escorts and ice information services provide shipping companies with the most efficient route through ice-infested waters (therefore decreasing transit time), and provide assurance that help is available if needed. Harbour breakouts are a necessity in ensuring that ships, marine facilities, and infrastructure do not sustain damage from ice and ships can access the marine facilities. Without flood control, ice jams can form and cause flooding, which can result in loss of life and property. It is clear that the IB Program plays an important role in many different areas, and that it addresses several needs. While it is difficult to predict precisely how climate change will affect the ice cover and icebreaking in the future, climate change models and past patterns point to the conclusion that there will be a need for these services for years to come. 

5.2 Efficiency

The evaluation has concluded that the IB Program has been efficient in its delivery of services. This could be concluded by supporting evidence provided through document review, interviews, case studies, and direct observations.

Overall the roles and responsibilities of the IB Program are defined and understood within CCG and between CCG and two of its main partners, CIS and USCG. However, due to workload issues, the routing responsibility has been relaxed such that Ice Service Specialists (ISS) from Environment Canada have often been permitted to prepare and issue recommended routes, when they should be restricted to providing information on ice and weather conditions and identifying the ice regime along the recommended route. An ISS is not an expert with respect to routing and may not take into consideration factors that are considered important for safe and efficient navigation. The roles, responsibilities and accountabilities in the regions, between the Ice superintendents and Superintendents, ROC should be clarified, documented and communicated with regards to tasking ships.

Recommendation 1:
It is recommended that the IB Program should establish the following with external service delivery partners: with respect to operational responsibilities of regional icebreaking superintendents, the CCG should clarify with Environment Canada (EC), the role and responsibilities of the Ice Service Specialists (ISS) and in particular with regard to ice routing.

The evaluation found that if operational constraints arise during operations, the IB Program has mechanisms in place to resolve issues e.g. by having conference calls between regions.

The IB Program participates in the Fleet Planning process annually. The November 2009 National meeting in Montreal, a first, was a good initiative to discuss the availability of resources as well as how the stakeholders could work together to improve the efficiency of the program.

The IB Program monitors its performance through several means e.g. LOS, IISPA, IODIS and the evaluation found that the performance information provided though the IISPA and IODIS have been used to formally report results of the IB Program. 

Although performance reporting measures are in place to assess the overall performance associated with the IISPA, it has not been possible to quantify the success of the agreement because of limited performance information provided by the CIS. The evaluation also concluded that the IB Program does not measure how its collaboration with the USCG has contributed to the achievement of its results.

In addition, through the data analysis of IODIS it was found that the IB Program could not provide all the performance info of its operational outputs (indicators) and strategic outcomes established in the LOS document. As a result, an area where the IB Program could improve its efficiency could be with regards to performance measurement. The IB Program should review its different sources of performance data e.g. LOS, IISPA, IODIS, etc. to ensure that the appropriate data is being gathered in a consolidated way which supports a performance measurement strategy that is aligned to the MRRS and respects program and evaluation performance requirements.

Recommendation 2:
It is recommended that the Icebreaking Program develop and implement a Performance Measurement (PM) Strategy that is aligned with the MRRS while respecting program and evaluation performance requirements. This PM Strategy would include a program profile, a logic model, a performance measurement framework, an evaluation and reporting strategy as well as a data collection strategy.

Evidence from the evaluation was able to conclude that complimentary services (i.e. availability of ice information, website such as MARINFO and ice booms) and partnership agreements (i.e. with the CIS and the USCG) contribute towards improving the efficiency of the program.

Examples of how these complimentary services have resulted in delivering the IB Program efficiently are the purchase of satellite imagery to be used to create ice charts and routings; and the delivery of joint operations and cooperation of ice breaking services. In addition, Ice Information Systems, specifically availability of ice information has improved the efficiency of the program by reducing reliance on icebreakers. Providing information such as voyage planning, recommended ice routes and tactical navigation has given navigators more autonomy thus reducing the reliance for icebreakers.

While there was no performance information on how these efficiencies have provided monetary savings to the CCG, results from the document review, case studies and key informant interviews confirm the usefulness of such activities in improving the efficiency of the program.

Recommendation 3:
It is recommended that the Icebreaking Program continues to support alternatives and complimentary services (i.e. availability of ice information, website such as MARINFO and ice booms) and partnership agreements (i.e. with the CIS and the USCG) that contribute to improving the efficiency of the program in all regions. These measures will increase navigator autonomy and reduce reliance on icebreakers.

5.3 Effectiveness

The IB Program contributes to the safe, efficient and timely movement of vessels through ice-infested waters. In addition to providing route assistance and ice information for safe passage, it assists in maintaining open harbours for commercial, fishing and assistance shipping needs. Furthermore, IB services reduce the effect of flooding caused by ice jams on the St. Lawrence River, and the program assists in re-supplying northern communities that do not have commercial service.

Ice conditions are becoming more hazardous to navigate due to changing climate conditions, particularly in the Arctic. As well, shipping activity in the Great Lakes and the Arctic continues to increase. These two factors will continue to drive an increasing demand for a wide range of icebreaking services, such as ice reconnaissance, ice charts, route assistance and so forth.

Four key documents facilitate the delivery of IB services:

  • Joint Industry-Coast Guard Guidelines for the Control of Oil Tankers and Bulk Chemical Carriers in Ice Control Zones of Eastern Canada (JIGs) ), a responsibility identified in the MOU between TC and CCG.
  • Levels of Service (LOS)
  • Agreement providing for coordination of icebreaking activities of Canada and the United States of America on the Great Lakes–St Lawrence Seaway System (Canada/US agreement)
  • Ice Information Services Partnership Agreement (IISPA).

The documents should be regularly reviewed and updated to ensure they remain effective. The importance of this is underscored by the Auditor General of Canada, who previously cited a need to ensure national standards and levels of service are up to date. There is room for improvement in this regard. Two of the agreements had a five-year review cycle in place, while the other two, specifically JIGs and LOS, did not.

JIGs is considerably out of date. The agreement, for which Transport Canada has the lead responsibility, has not been revised since 1979, despite a review in 2005. It is also vague and cannot effectively support operations. Given the potential for serious environmental harm from marine oil or chemical spills, such guidelines should be kept up to date.

The extent to which clients consider the IB services responsive to their needs varies somewhat. The IB Program’s ability to meet increasing or changing demands is tempered by its service capacity. While client feedback on the whole leans towards the positive, some comments and other feedback highlight areas of concern. Although a new version of the LOS was issued May 2010, the one in effect during the period covered by the evaluation dated back to 2001. Both clients and program staff noted that conditions have changed since then such that the 2001 version was out of date. A 10-year period between updates is too long, as a result of evolving needs of the clients of the IB Program and potential changes in the capacity of the fleet of icebreakers.

Recommendation 4: It is recommended that the IB Program adhere to a five-year review and update cycle for all agreements pertaining to the delivery of icebreaking services to keep pace with evolving client needs and changing IB Program service delivery capacity.

The LOS identifies the target response times within which route assistance, facilities and maintenance, and flood control services are to be provided. For the period covered by the evaluation, the IB Program met these targets on average more than 90% of the time.

However, the 2001 LOS did not include the IB Program’s prioritization system, which separates service requests into five levels of priority. Furthermore, the priorities as defined lack clarity, detail and coverage of all ferry services giving rise to different interpretations and confusion. While the wording for Priority II, governing ferry services, was changed following a workshop in November 2009, the prioritization system should be formally incorporated as part of the IB Program’s LOS document.

Recommendation 5: It is recommended that language surrounding the service requests for the five levels of priority of the Icebreaking Program’s prioritization system in the LOS document be more clearly defined and detailed specifically with respect to coverage for ferry services.

The IB Program does not have complete performance data for services delivered through IISPA. Operational data provides an important source of information that contributes to keeping services and agreements current and effective.

Recommendation 6 It is recommended that the IB Program should establish the following with external service delivery partners:  It is recommended that the IB Program should ensure, with respect to IISPA, submission of regular reports (monthly, seasonal, both or otherwise as deemed appropriate by the IB Program) from CIS to the Manager, IB Program.

5.4 Economy

The evaluation concluded that the IB Program has been producing enough outputs to produce the desired result at the lowest cost. Overall, the program has been able to maintain its Levels of Service (on average) above 90% (Graph 12), with fewer available ships (Table 3).

In addition, through the collaboration between the CCG and USCG on the Great Lakes, the IB Program has been minimizing its resources, by having two CCG vessels on the Great Lakes and leveraging up to nine USCG vessels while maintaining its Level of Services.

A comparative analysis of the literature compared Canadian icebreaking services to four countries (Russia, Finland, Sweden and the United States of America) regarding the delivery models (private vs. public), comparison of resources i.e. fleet, and service fees. In summary, it was concluded that Canada has the largest fleet of icebreakers in the world; Canadian icebreakers are multi-tasked, which improves the fleet’s overall efficiency and is regarded as a leading-edge best practice; Russia, Finland and Sweden place some restrictions on the types of vessels that are allowed to receive icebreaking services; and, Finland, Russia and Sweden levy some form of fee for the provision of icebreaking services.

The IB Program should review how other countries are delivering their icebreaking services to see if CCG can learn and apply these practices in Canada. For instance, CCG could review the administration of service fees, delivery models (private vs. public or a mix of both), or restricting services to certain harbours. By comparing how other countries deliver their services, CCG could apply techniques that have been successful in other countries and further minimize the use of resources in the delivery of services.     

6. Management Action Plan

Management action plan – Icebreaking Program


Recommendations

Management Action Plan

Status Report Update

Actions Completed

Actions Outstanding

Target Date

Recommendation 1:
It is recommended that the IB Program should establish the following with external service delivery partners: With respect to operational responsibilities of regional icebreaking superintendents, the CCG should clarify with Environment Canada (EC), the role and responsibilities of the Ice Service Specialists (ISS) and in particular with regard to ice routing.

 

The IISPA (2005) defines clearly the duties of the ISS (p. 11):
The duties of the ISS are limited to the provision of information, and specifically exclude decision making and reporting on ship routing and ship navigation which remain the responsibility of the CCG’.

 

In agreement with CIS director, the Icebreaking Program Manager will send out a reminder to officers on their roles and responsibilities at appropriate times, such as when a new Ice Superintendent is hired. Such a reminder was issued in December 2010.

 

 

Completed

 

 

 

 

 

 

 

Recommendation 2:
It is recommended that the Icebreaking Program develop and implement a Performance Measurement (PM) Strategy that is aligned with the MRRS while respecting program and evaluation performance requirements. This PM Strategy would include a program profile, a logic model, a performance measurement framework, an evaluation and reporting strategy as well as a data collection strategy.

The Icebreaking Program supports this initiative and will develop a Performance Measurement Strategy (PMS) which deals with the management, accountability and risk factors related to the IBP Program.

 

Also as per DFO performance Measurement Action Plan, the program has submitted a PM Strategy to the DFO Evaluation Directorate.

Draft Performance Measurement Strategy (PMS) developed.

DFO Evaluation Directorate will review and provide comments on IB Program updated PM Strategy.

 

Finalize and implement the approved (by CFO and Commissioner of CCG) Performance Measurement Strategy across the program.

March 2011

 

 

 

March 2012

 

Recommendation 3:
It is recommended that the Icebreaking Program continues to support complimentary services (i.e. availability of ice information, website such as MARINFO and ice booms) and partnership agreements (i.e. with the CIS and the USCG) that contribute to improving the efficiency of the program in all regions. These measures will increase navigator autonomy and reduce reliance on icebreakers. 

The Icebreaking Program continually supports the delivery of complimentary services that increase navigator autonomy and reduce reliance on icebreakers. As the fleet is aging and subject to increased failures, the Icebreaking Program puts its attention to develop and support complimentary services to provide high-quality information to mariners. The Icebreaking Program took this avenue years ago through the initiation of R&D projects, increased use of satellite images, and improvement of ice information accessibility on websites. However, it should be noted that icebreaker escorts will still be needed given that no other solution may be provided to mariners when ice conditions, thickness and concentration, are too severe.

The Icebreaking Program works closely with the Canadian Ice Service to develop new tools, in particular satellite images, to provide high-quality information to mariners.

-A R&D project has been ongoing for two years. The project aims to discriminate, mostly in the Arctic, between first year and mutli-year (more dangerous) ice. This project is funded by the CCG, TC, and NRCan and received attention from the international community.

The program is also working with the National Hydraulics Center to test an Ice Pressure Model. This model will provide the information needed to determine areas where ice concentration can cause problems to navigation and thereby support improvements in ice routing.

As these are R&D projects, their completion depends either on results obtained or on the funding available. For the Ice Hazard Project, the Program is looking right now for funding with the oil industry. The Program estimates that the recommendation is continuously followed.

Completed

Recommendation 4:
It is recommended that the IB Program adhere to a five-year review and update cycle for all agreements pertaining to the delivery of icebreaking services to keep pace with evolving client needs and changing IB Program service delivery capacity.

The Icebreaking Program supports this recommendation to avoid outdated agreements. As part of their renewals, all but one are now based on a 5 year cycle. The remaining agreement (Joint Industry / CCG Task Force Agreement for Icebreaker requirements) is with the industry and is currently under review. The CCG has informed the industry last fall that it wished to include a 5 year renewal cycle.

All but one agreement are now based on a 5 year cycle.

Ensure the last agreement includes a 5 year renewal cycle. CCG expects this last agreement to be renewed in March 2011.

March 2011

Recommendation 5:
It is recommended that language surrounding the service requests for the five levels of priority of the Icebreaking Program’s prioritization system in the LOS document be more clearly defined and detailed specifically with respect to coverage for ferry services.

The Icebreaking Program considers the language surrounding its LOS priorities clear. However, the IB Program acknowledges that the application of these priorities, and particularly the one related to ferry services, may sometimes be difficult to accept by its clients due to the delays of service it may impose to other clients. As such the IB Program will take any opportunity to clarify the language of its LOS during client meetings.

A revised Levels of Service and Service Standards document was published on the internet in May 2010. The document includes greater clarity on the prioritization. Further, the industry has been consulted in June 2010 with regard to icebreaker requirements and LOS. Clarifications on the prioritization of ferries has been made and accepted.

 

Completed

Recommendation 6:

It is recommended that the IB Program should establish the following with external service delivery partners: with respect to IISPA, submission of regular reports (monthly, seasonal, both or otherwise as deemed appropriate by the IB Program) from CIS to the Manager, IB Program.

 

The Icebreaking Program supports the recommendation. In fact, the submission of regular reports is already included in the IISPA agreement with Environment Canada (Canadian Ice Service). The IB program has discussed this with CIS and both organizations agreed to ensure the reports are provided regularly. 

 

 

 

Completed

  

Annex A- Evaluation Matrix

Icebreaking Program (IB) Evaluation Matrix

QUESTIONS

INDICATORS

METHODOLOGY

RELEVANCE

1. To what extent are the objectives of the Icebreaking Program aligned with departmental and government wide priorities?

Degree of alignment between the objectives of Icebreaking Program and Government of Canada priorities

  • Document Review
  • Interviews

Degree of alignment between the objectives of the Icebreaking Program and DFO’s strategic outcomes

Extent to which changes would be required for a better alignment with departmental priorities

2. Is the current role of the federal government and CCG appropriate in delivering the Icebreaking Program?

Documents evidencing Government of Canada obligations regarding Icebreaking Program and ice information, specifically agreements signed and Level of Service documents

  • Document Review
  • Interviews

 

Documents evidencing Government of Canada obligations regarding marine safety, aids to navigation, etc.

3. To what extent is the Icebreaking Program addressing an actual and continuing need?

Evidence/demonstration of need for Icebreaking Program, in the shipping community as well as for the general public

  • Document Review
  • Interviews
  • Direct Observation
  • Survey

Evidence of a future need for icebreaking and the Icebreaking Program

Evidence of the consequences if the program did not exist

Duplication of objectives between programs

EFFICIENCY

4. To what extent is the governance structure clear, appropriate, and efficient for the Icebreaking Program?

Perception with regards to the governance structure i.e. roles, responsibilities, and accountabilities of Icebreaking Program and whether they are clearly defined, understood, and efficient

  • Document Review
  • Interviews
  • Case studies

Evidence of a process for handling operational constraints

5. To what extent is the design and delivery of the Icebreaking program clear, appropriate, and efficient?

Evidence that a Performance Measurement Strategy/performance measures/objectives are in place and are clear, appropriate, and efficient

  • Document Review
  • Interviews
  • Case studies
  • Direct Observation

 

Extent to which the present planning process (strategic plans) is appropriate and efficient

Extent to which data collected is being used for decision making

Degree to which the elements/factors impacted the performance of the IBP

6. To what extent could the efficiency of the Icebreaking Program and its activities be improved?

Extent to which there are alternatives/complimentary services that could impact the efficiency of the Icebreaking Program

  • Document Review
  • Interviews
  • Case studies

Extent to which partnerships and agreements with stakeholders increase or decrease the efficiency of the Icebreaking Program

Extent to which the Icebreaking Program is efficient compared to other alternatives

Extent to which research and development within Icebreaking Program could improve the efficiency of the services delivered and the services to be delivered in the future

EFFECTIVENESS

7. To what extent has the Icebreaking Program achieved its three objectives?

Extent to which the Icebreaking Program has facilitated the safe and timely vessel transits and movements through ice-infested waters

  • Document Review
  • Interviews
  • Case Study
  • Program Database
  • Survey

Extent to which the Icebreaking Program has minimized the effect of flooding caused by ice jams in the St-Lawrence river

Extent to which the Icebreaking Program assisted in the re-supply of the northern communities for which there no commercial services

8. To what extent are agreements in place, up-to-date, and effective to support the implementation and delivery of the Icebreaking Program?

Extent to which the Level of Service document, JIGS, Canada/US, IISPA are approved, relevant, up-to-date, and effective to achieve the expected results of the Icebreaking Program

  • Document Review
  • Interviews
  • Case Study
  • Survey
  • Direct Observation

 

Extent to which the agreements are understood, implemented, and coherently applied between the regions

ECONOMY

9. To what extent is the Icebreaking Program utilized in an economical manner?

Extent to which the Icebreaking Program utilize the least amount of resources needed to produce its outputs and activities

  • Document Review
  • Interviews
  • Program Database
  • Literature review

 

Extent to which there could be any economy/affordability alternatives considered for the Icebreaking Program


1 http://www.glsls-study.com/English%20Site/home.html
2 The operational agreement has been effective since September 1995 between GGC and the Government of Quebec.
3 Icebreaking Operations Levels of Service
4 Climate Change Risk Assessment Report
5 The North articles: Securing the Northwest Passage Essential, Nov 6, 2008
6 Dry Cargo Resupply Program Activity Summary Shipping Year 2009
7 Climate Change Risk Assessment Report
8 Canadian Coast Guard. Standard Regional Organizational Structure, August 28, 2007. 
9 Standard Regional Organizational Structure – Duties of the Icebreaking Superintendent (Maritime Services). August 28, 2007.
10 Ice Information Systems include the following systems: Ice Sensor Thickness Sensor, Ice Routing Model, ICEggs, Integrated Ice system, Advanced Radar System, Integrated Ice System, Cross Polarized Radar, IceNav Virtual Marine Radar, Ice Hazard Radar, etc.
11 Lavaltrie, Lanoraie & Yamachiche. Floating Booms: Master Plan. Garde cotiere candienne. December 2009.
12 Ibid.
13 Lavaltrie, Lanoraie & Yamachiche Flaoting Booms. Master Plan. Garde cotiere candienne. December 2009.