Evaluation of the Integrated Oceans Management Program

Evaluation Directorate
Project Number 6B135
Final Report
February 2012

Table of Contents

Acronyms

ADM
Assistant Deputy Ministers
AOI
Areas of Interest
DFO
Fisheries and Oceans Canada's
DG
Directors General
DM
Deputy Ministers
EA
Ecosystem Assessments
EC
Environment Canada
ENGO
Environmental Non-Governmental Organizations
FMPAS
Federal Marine Protected Areas Strategy
FPT
Federal/Provincial/Territorial
HOTO
Health of the Oceans
IOM
Integrated Oceans Management Program
LOMA
Large Ocean Management Areas
MPA
Marine Protected Areas
OAP
Oceans Action Plan
PAA
Program Activity Architecture
PCA
Parks Canada Agency
WWF
World Wildlife Fund

Executive Summary

Scope and Methodology

Evaluation Objective

The main objective of this evaluation is to determine to what extent the Integrated Oceans Management (IOM) Program is relevant, is managed effectively and efficiently, and whether it has achieved its stated objectives. As such, the evaluation examined the extent to which the IOM Program demonstrates value for money in its relevance and performance (including effectiveness, efficiency and economy), in accordance with Treasury Board's 2009 Policy on Evaluation. This evaluation covered the period from 2005-06 to the present and was undertaken between June 2011 and January 2012.

Program Description

The IOM Program is the program activity within DFO with responsibility for integrated management of oceans and application of marine conservation tools for the protection of sensitive or significant marine areas. The IOM Program provides federal and provincial government authorities, industry, non-governmental organizations, Aboriginal groups and Canadians with the tools and fora needed to collaboratively develop Integrated Oceans Management Plans that incorporate social, economic, and environmental considerations in decision-making. The ultimate intended outcome of the IOM Program is that oceans activities are managed in a way that preserve the ecological health of the oceans, while allowing for sustainable use of the oceans.

Evaluation Methodology

The evaluation used a non-experimental design and a multiple-lines-of-enquiry approach. Qualitative and quantitative data for the evaluation were obtained from the following sources:

  • Document and performance data review;
  • Twenty-four interviews with key informants, including senior managers, stakeholders and external experts;
  • Online survey of IOM partners and stakeholders (responses from 60 individuals for a response rate 36 per cent); and,
  • Two case studies.

Key Findings

Relevance

  • The evaluation evidence indicates that there is an ongoing need for oceans programming. Canada's oceans have great economic and social significance, yet are showing signs of stress in many areas. The views of key informants, surveyed IOM Program stakeholders, and public opinion data are also supportive of efforts to address oceans management. Integrated management is an approach that has proven effective in terrestrial planning and has been adopted by other countries to manage marine environments.
  • The IOM Program aligns with federal priorities, particularly those related to economic affairs (a clean environment). The IOM Program is a key aspect of the Government of Canada's National Water Strategy through leadership and responsibility for many of the Health of the Oceans Initiative components, and supports complementary federal strategies (e.g., the Federal Sustainable Development Strategy). IOM Program activities supported by the Health of the Oceans Initiative contribute to meeting international commitments related to biological diversity, specifically marine protected areas and marine protected area networks (Convention on Biological Diversity).
  • The IOM Program aligns with departmental priorities, addressing the strategic outcome of Sustainable Aquatic Ecosystems, as well as contributing to the strategic outcome Economically Prosperous Maritime Sectors and Fisheries by supporting sustainable development of oceans. The program's priority alignment is subject to a persistent tension within DFO between the ecosystem approach and an historical focus on single species management.
  • The Oceans Act and Canada's Oceans Strategy outline the federal mandate in integrated oceans management. The federal level and DFO make an appropriate and important contribution to oceans programming, in collaboration with other levels of government and other stakeholders. There is some sentiment that the department's role could be more fulsome to meet its leadership mandate stated in the Oceans Act for integrated oceans management.

Performance

Effectiveness
  • Evaluation evidence from all lines of inquiry indicates some progress toward the achievement of the immediate and intermediate outcomes of the IOM Program.
  • The IOM Program has engaged a wide array of stakeholders in integrated oceans management. Surveyed IOM Program partners/stakeholders who were engaged in consultations/committees were generally satisfied with these processes. Impacts were noted in areas such as the inclusion of a broad spectrum of views, but less impact was felt to have occurred in the areas of oceans management decision-making and stakeholder commitment to integrated oceans management.
  • Integrated Oceans Management Plans have not been implemented within the timeframe specified in the Department's strategic plan. To date, only one Integrated Oceans Management Plan has been endorsed by the Department, while Integrated Oceans Management Plans in four other Large Ocean Management Areas are in various stages of completion and approval. Because Integrated Oceans Management Plans are not at a stage of implementation, there have been limited impacts on policies or practices for oceans-related decisions.
  • Progress in establishing Oceans Act Marine Protected Areas and protecting and conserving marine ecosystems is mixed. There has been significant progress in the development of key strategic and operational supports to establish and manage a national network of protected areas. Progress has been slower in establishing new Marine Protected Areas (the Health of the Oceans Initiative goal of six new Oceans Act Marine Protected Areas by 2012 will not be met) and developing indicators and monitoring protocols for the management of existing Marine Protected Areas.
  • The program's ultimate outcome – management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans – is ambitious, achievable only in the longer-term and difficult to attribute the IOM Program's contribution to its achievement. Some question the plausibility of the program's design to achieve this objective.
  • Unintended outcomes of the program were generally few, but mostly positive in nature. Unintended outcomes were largely in the area of beneficial aspects of collaborations. A negative unintended outcome has been increased stakeholder expectations around integrated oceans management, which have not been met under the current program, resulting in a loss of credibility of DFO and scepticism about the federal commitment to integrated oceans management.
Efficiency
  • In general, financial resources allocated to the IOM Program are difficult to track over time due to evolutions in the scope of the program during the study period. However, resources are generally viewed to have been lean for the program. Factors that hinder efficiency include delays, multi-stakeholder consultation processes which are expensive and time-consuming, lack of coordination and absence of well-articulated program objectives and targets.
  • The terms of reference for interdepartmental committees related to oceans management were refreshed for the Health of the Oceans Initiative, and roles and responsibilities of federal departments were viewed as clear. The Deputy Minister and Assistant Deputy Minister committees met infrequently, however, and engagement of senior management and ownership of the oceans issue by other government departments was viewed as limited by some. Within DFO, there is reported persistent tension between oceans management and fisheries management, reflecting the department's dual role in these areas. NHQ and regional collaboration were felt to be improving, with continued attention to coordination and communication required.
  • Science has provided significant support to integrated oceans management, though resource challenges remain in the complex area of ecosystem science. Social, cultural and economic data are being collected within regions, often with the assistance of external expertise. Practical use and incorporation of the information within integrated oceans management processes has proven challenging.
  • The IOM Program is currently developing performance measurement and monitoring tools. However, during the period under study, performance measurement occurred informally, as well as formally for activities funded through the Health of the Oceans Initiative. Component leads are generally satisfied with the requirements for monitoring and reporting under the Health of the Oceans Initiative, and the performance measurement process proved useful for internal management and reporting functions.
  • The IOM Program has supported an increased federal focus on integrated oceans management. There are many lessons learned that may benefit efforts moving forward to address evolving priorities. Priority areas identified in the evaluation included: moving Integrated Oceans Management Plans toward implementation; honouring commitments on Oceans Act Marine Protected Area designation and marine protected area networks, and addressing Marine Protected Area management challenges; and improving coordination.
Economy
  • A number of suggestions were provided to improve the effectiveness of the IOM Program to achieve intended outcomes. There are a number of recommendations and developments internationally that may be instructive in undertaking program improvements.

Recommendations

1) Integrated Oceans Management Plans have not been implemented within the timeframe specified in the Department's strategic plan. To date, only one Integrated Oceans Management Plan has been endorsed by the Department. In order to demonstrate results, the Program needs well-articulated program objectives with an appropriate performance measurement strategy. A new draft performance measurement strategy, inclusive of Science and Policy components, has recently been developed.

Recommendation:
In order to regularly monitor and demonstrate results, the Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should establish and communicate to staff and stakeholders clear and achievable short, medium and long-term Integrated Oceans Management objectives aligned with the recently developed draft performance measurement strategy.

2) Improvement is needed in the areas of oceans management decision-making, stakeholder commitment to integrated oceans management and efficient committee processes. Although collaboration is improving, continued attention to coordination and communication is required.

Recommendation:
The Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should take an active leadership role in increasing the engagement of senior management from other government departments to advance achievement of Integrated Oceans Management commitments.

3) The Integrated Oceans Management Program has engaged a wide array of stakeholders in integrated oceans management. Consultations and committees were successful in encompassing a broad spectrum of views. However, in order to gain efficiencies and effectiveness, it is advisable to streamline stakeholder participation in the consultation processes.

Recommendation:
The Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should review and streamline existing collaborative processes to enhance efficiencies and effectiveness of stakeholder participation.

1. Introduction

1.1 Context of the Evaluation

This report presents the results of the evaluation of Fisheries and Oceans Canada's (DFO) Integrated Oceans Management (IOM) Program. In accordance with the Treasury Board Secretariat Policy on Evaluation (2009), the evaluation focuses on the extent to which the IOM Program has demonstrated value for money by assessing the core issues of relevance and performance, including effectiveness, efficiency and economy.

1.2 Scope

The main objective of this evaluation is to determine to what extent the IOM Program is managed effectively and efficiently and whether it has achieved its intended outcomes. The evaluation covered the period from 2005-06 to the present and was undertaken between June, 2011 and January, 2012. The evaluation was led by the DFO Evaluation Directorate. An external consultant was retained to develop the evaluation plan and to conduct the IOM Program evaluation. It should be noted that due to the significant overlap between the IOM Program and the federal horizontal Health of the Oceans (HOTO) Initiative, the IOM and HOTO evaluations were conducted simultaneously.

2. Program Profile

2.1 Background and Objectives

On January 31, 1997, the Government of Canada brought the Oceans Act into force, making Canada the first country in the world to have comprehensive oceans legislation. The Oceans Act provides a framework for current and future oceans management initiatives. Section 31 of the Oceans Act requires the Minister of DFO to "lead and facilitate the development and implementation of plans for the integrated management of all activities or measures in or affecting estuaries, coastal waters, and marine waters that form part of Canada or in which Canada has sovereign rights under international law".

The IOM Program is the program activity within DFO with responsibility for integrated management of oceans and application of marine conservation tools for the protection of sensitive or significant marine areas. Whereas decisions about oceans management have previously focused on a single ocean activity, integrated oceans management is founded on a holistic ecosystem approach. The IOM Program provides federal and provincial government authorities, industry, non-governmental organizations, Aboriginal groups and Canadians with the tools and fora needed to collaboratively develop Integrated Oceans Management (IOM) Plans that incorporate social, economic, and environmental considerations in decision-making. The program also contributes to meeting Canada's ocean-related international legal commitments. The ultimate intended outcome of the IOM Program is that oceans activities are managed in a way that preserve the ecological health of the oceans, while allowing for sustainable use of the oceans. The IOM Program is responsible for the implementation of Canada's Oceans Strategy, including two phases of funding: Oceans Action Plan (OAP) (Phase 1) (2005-06 to 2006-07) and the HOTO Initiative (2007-08 to 2011-12).

2.2 Program Activities

In the 2011-12 Departmental Program Activity Architecture (PAA), the IOM Program is a program activity within the Sustainable Aquatic Ecosystems strategic outcome. There are two program sub-activity areas under the IOM Program: ecosystem assessments and marine conservation tools.1

Integrated Oceans Management

Outreach to and engagement of federal and provincial /territorial authorities, Aboriginal groups, industry, non-government organizations and Canadians in a given ocean space is an integral aspect of integrated oceans management planning. The integrated oceans management sub-activity within the IOM Program includes development of the tools, forums and governance structures that support the collaborative development and implementation of IOM Plans. Five Large Ocean Management Areas (LOMAs) have been identified to pilot test the integrated oceans management approach. For each LOMA, there are collaborative governance, advisory and technical committees formed at the regional or sub-regional level to support integrated oceans management.

Integrated oceans management within each LOMA is informed by ecosystem science, including assessing the biophysical components within the LOMAs (the ecosystem assessments sub-activity) and, using science-based ecological overviews, developing social, cultural and economic overviews of ocean spaces (the integrated oceans management sub-activity). Based on these inputs, ecological, social, cultural and economic objectives and guidelines are determined, and management approaches, including the development of risk-based and decision support tools to achieve objectives, are selected.

Within the integrated oceans management sub-activity are also communications and relationship building activities (e.g., coordination of events or opportunities for public education (e.g., World Oceans Day) and the development of networks/opportunities for information sharing and decision/priority action support for a variety of stakeholders (e.g., the Oceans Web Portal, Centres of Expertise).

Finally, the integrated oceans management sub-activity includes responsibility for selected components funded under OAP Phase I and the HOTO Initiative related to collaboration and knowledge transfer. These components include, for example, oceans science technology-related initiatives and activities in the international realm.

Marine Conservation Tools

The marine conservation tools (MCT) sub-activity supports the sustainable management of ocean resources by providing options to secure critical aspects of the ecosystem from harm. The Convention on Biological Diversity (1993) contributed to international recognition of the value of marine protected areas and networks of marine protected areas as a mitigation strategy against declining marine environments2. Development of a national network of marine protected areas is a key international commitment of the Government of Canada (e.g., made at the 2002 World Summit on Sustainable Development, the 2004 Convention on Biological Diversity (CBD), and the 2010 CBD Conference of the Parties), as well as a domestic commitment (e.g., under the Oceans Act, Oceans Action Plan). Development of a marine protected area network was a key action identified by the OAP and received support from the HOTO Initiative.

Actions that occur under the marine conservation tools sub-activity are: identification of Areas of Interest (AOIs) and designation of Oceans Act MPAs to protect and conserve marine species and habitats, including the identification of conservation objectives and the creation of regulatory and management approaches to achieve conservation objectives; and advancement of a federal strategy and national network of marine protected areas.

Oceans Act MPAs are established within an integrated oceans management context (that is, in a collaborative manner with other partners and stakeholders and planned within a broad marine planning context that considers oceans uses and users to increase the effectiveness of these protected areas and to ensure that surrounding areas are managed in a complementary fashion.

Ecosystem Assessments

The ecosystem assessments (EA) sub-activity provides the science foundation for the IOM Program to inform integrated oceans management plans, marine conservation tools and for ongoing assessment and monitoring of ocean spaces. Science also provides support in identifying conservation objectives for AOIs and in identifying indicators and developing monitoring protocols and strategies for existing Oceans Act MPAs.

Conducting ecosystem assessments is interdisciplinary in nature. It requires a comprehensive understanding of ocean variables and their interactions, both living and non-living, ultimately demonstrating how human activities have the potential to impact marine ecosystems. Key science products include: Ecological Overview and Assessment Reports; Canadian Science Advisory Secretariat Science Advisory Reports on Ecologically and Biologically Significant Areas, identification of Ecologically Significant Species and Community Properties, risk assessment tools, ecosystem indicators and monitoring strategies frameworks.

Program Delivery

The delivery of the IOM Program occurs largely in the regions. DFO operates within six administrative regions – Newfoundland and Labrador, Maritimes, Gulf, Quebec, Central and Arctic, and Pacific. Service Level Agreements are negotiated annually between DFO headquarters and each region. Service Level Agreements indicate the level of investment in each region, program activities and deliverables – e.g., integrated oceans management governance, ecosystem assessments; development, designation and ongoing management of MPAs; and development and implementation of bioregional MPA network designs and action plans.

1Note that the that ecosystem assessment (science) has only recently been included as a sub-activity under oceans management. Also note that the sub-activity integrated oceans management is subsumed under marine conservation tools in the 2011-12 PAA. This program profile has separated the integrated oceans management sub-activity from marine conservation tools for the sake of consistency with the operating reality of the IOM Program.

2Canada's federal marine protected areas network is comprised of the following three core programs: Oceans Act Marine Protected Areas (DFO) – established to protect and conserve important fish and marine mammal habitats, endangered marine species, unique features and areas of high biological productivity or biodiversity; Marine Wildlife Areas (EC) – established to protect and conserve habitat for a variety of wildlife including migratory birds and endangered species; and National Marine Conservation Areas (PCA) – established to protect and conserve representative examples of Canada's natural and cultural marine heritage and provide opportunities for public education and enjoyment. Throughout the report, the term marine protected areas refers to marine areas protected by any of the relevant departments/agency. Oceans Act MPAs are marine protected areas established by DFO.

2.3 Program Resources

The IOM Program is financed through a combination of A-base and B-base dollars (the OAP Phase I and HOTO Initiative). Table 1 summarizes the A-base, or permanent, funding for the IOM Program – totalling $32.9M annually. With respect to B-base, or non-permanent, funding, in total, $17.05M in funding was allocated to DFO for 13 of 18 components in Phase I of the Oceans Action Plan.3 The HOTO Initiative includes about $21M in total funding to DFO for nine of 22 components over five years (excluding one component led by the Canadian Coast Guard).4

Table 1: A-Base Initial Budget Allocation IOM Program 2010-11
2010-2011
Initial Budget
Integrated
Management
Marine Conservation
Tools
Ecosystem
Assessment*
Total Oceans Management Program
Salaries $6,804,371 $2,441,280 $17,518,305 $26,763,956
O&M $2,926,867 $540,755 $2,605,580 $6,073,202
Class contributions $50,000 0 0 $50,000
Total $9,781,238 $2,982,035 $20,123,885 $32,887,158

* 2011-12. Note that the Ecosystem Assessment allocation includes science activities to support Oceans, as well as other programs within DFO such as Habitat and Species at Risk, as well as Hydrographic Products and Services.

Table 2: Oceans Action Plan - DFO Allocation by Component
Initiatives Funding Distribution
2005-06
($000)
2006-07
($000)
International Leadership, Sovereignty & Security
1. Gulf of Maine (Canada-US Collaboration) 150 150
Integrated Oceans Management for Sustainable Development
2. Ecosystem Science Reports 800 700
3. Identification of Ecologically Sensitive Areas 1,000 1,400
4. Ecosystem Objectives/Smart Regulations 1,000 1,200
5. Economic Assessments and Analysis 150 150
6. Targeted Regional Consultations/Engagement 500 200
7. Agreements with F/P/T governments and Aboriginal groups 800 900
8. Regional Management and Advisory Bodies 1,000 1,000
Health of the Oceans
9. Oceans Act Marine Protected Areas 1,600 1,600
10. National Strategy for Marine Protected Areas 170 130
11. Science Research & Advice for Marine Protected Areas 100 150
Oceans Science & Technology
12. Oceans Technology Network 100 100
13. Placentia Bay Technology Demonstration Pilots 1,000 1,000
Total DFO 8,370 8,680

Table 3: HOTO Initiative Funded Components and Resources Mapped to IOM Program Sub-activities
Component IOM (IOM)*
Marine Conservation Tools (Marine Conservation Tools (MCT))
Ecosystem Assessments (EA)
HOTO Funding
2007-08 to 2011-12
(000's)
HOTO Initiative
1. Component #13: Federal Marine Protected Areas Strategy Implementation MCT 1,250
2. Component #14: Development of Federal-Provincial-Territorial MPA Network MCT 2,500
3. Component #15: Arctic Council – Ecosystem Projects IOM 1,000
4. Component #16: Oceans Centres of Expertise IOM 3,000
5. Component #17: Collaboration with World Wildlife Fund (WWF) IOM 213
6. Component #18: Gulf of Maine IOM 750
7. Component #19: Marine Protected Area Establishment MCT 5,250
8. Component #20: Integrated Management and Canadian Environmental Assessment Act Assessment Tool Linkages IOM 1,450
9. Component #21: Ecosystem Science Support and Advice on Health of Oceans EA 5,500
Total IOM   20,913

* While not a sub-activity within the PAA, IOM functions as a sub-activity within the program and so has been identified separately here.

3Total funding for OAP was $28.4M.

4Total funding for the HOTO Initiative was $61.5M.

2.4 Logic Model

The logic model for the IOM Program presents activities and outputs, and intended direct outcomes, intermediate outcomes and ultimate outcomes. This logic model is presented on the next page.

The logic model of the IOM program

2.5 Program Partners and Stakeholders

Because oceans management is far reaching, having implications that go beyond the federal government, the IOM Program involves numerous partners and stakeholders, including:

  • Provincial/territorial representatives. Responsibility for managing and protecting the oceans involves both federal and provincial/territorial jurisdictions. Both levels of government share authority and/or are involved in aspects such as the network of marine protected areas and fisheries;
  • Regulating agencies that make decisions about oceans activities in a given space (i.e., federal, provincial and territorial agencies with authority for areas such as resource development and transportation);
  • Industry (such as fishing, shipping, aquaculture, oil and gas);
  • Non-governmental organizations (particularly environmental non-governmental organizations (ENGOs) and community groups);
  • Aboriginal organizations; and
  • Others, such as academia and international bodies.

The IOM Program does not have a clearly articulated or single beneficiary group. Ultimately, all Canadians benefit from healthy oceans and sustainable use of ocean resources, and more broadly, the conservation of ocean resources is important for future generations.

2.6 Governance

An Oceans governance model was established when the OAP was introduced in 2005-06 to coordinate and implement all oceans activities. The model includes federal government interdepartmental, federal/provincial/territorial (FPT) and regional committees to provide a forum for collaboration and joint action in support of the management of all activities in or affecting marine areas. In 2008, the federal interdepartmental committees' Terms of Reference were updated to clarify roles, responsibilities and reporting relationships.

Federal Interdepartmental Committees

There are three federal government interdepartmental committees:

  • The Deputy Ministers (DM) Interdepartmental Committee on Oceans is an ad hoc committee that is brought together when needed, though has not met since 2007-08.
  • The Assistant Deputy Ministers (ADM) Interdepartmental Committee on Oceans is a sub-committee of the Deputy Ministers Interdepartmental Committee. The committee is composed of representatives from the eighteen federal departments and agencies that are involved in the oceans through policies, programs, services, regulations, and/or other activities and is chaired by the ADM, Ecosystems and Oceans Science Sector, DFO (to June 2011 prior to a realignment within DFO). The committee is a forum for collaboration and joint action on matters related to the OAP and, more broadly, implementing the federal agenda as it touches on Canada's oceans. The Committee provides operational direction downward. This committee met three times in 2008 and 2009.
  • The Directors General (DG) Interdepartmental Committee on Oceans includes representation from HOTO Initiative partner departments (five federal departments) and central agencies – roughly 20 organizations in all. The committee provides advice and support to the ADM Interdepartmental Committee on Oceans and is also a forum for joint action in support of a national strategy for the management of all activities in or affecting marine areas. This committee met seven times since 2007-08.

Secretariat functions are performed within the Regional Operations Branch, Ecosystems and Oceans Science Sector at DFO to coordinate the activities of the interdepartmental committees (since 2010-11).

Federal -Provincial -Territorial

The Oceans Task Group was the federal/provincial/territorial body under the Canadian Council of Fisheries and Aquaculture Ministers; it was disbanded in 2010 after fulfilling much of its work plan. The Oceans Task Group provided a mechanism to undertake or advance activities related to integrated oceans management and MPA network development across jurisdictions. As co-chair of the Oceans Task Group, the Director General, Oceans (or designate) was responsible for bringing forward updates and issues pertaining to integrated oceans management and the development of Canada's Federal Marine Protected Area Strategy and National Framework for Canada's Network of Marine Protected Areas.

Regional Governance Framework

Regional Committees on Oceans Management (or similar) deal with oceans management issues on a regional scale, including the integrated oceans management work underway within the LOMAs (the mandate and composition of regional committees is discussed in more detail in Section 4.2 below). The Regional Committees on Oceans Management are to report on progress through the DG Interdepartmental Committee on Oceans and the ADM Interdepartmental Committee on Oceans.

3. Methodology

3.1 Project Management

The evaluation was conducted by an external evaluation team, managed by a senior evaluation manager within the Evaluation Directorate at DFO. The team collaborated with program personnel to identify relevant documents and key informants and stakeholders to include in the evaluation, and to review and provide feedback on study products and deliverables.

3.2 Evaluation Approach & Design

A non-experimental design was used for this evaluation, in which measurements are taken after the program has been implemented with no control group. This model was chosen because the IOM Program is a full coverage program and it is intended to be delivered across Canada and not withheld from any area or region. The evaluation employed a variety of methods (e.g., interviews, document review, survey) where the evidence drawn from these methods were triangulated to arrive at valid findings and conclusions.

3.3 Key Issues & Evaluation Questions

The evaluation questions cover both relevance and performance, including effectiveness, efficiency and economy. The IOM evaluation matrix (Annex A) summarizes the use of multiple lines of evidence and complementary research methods as a means to ensure the reliability of information and data to be collected.

Table 4: Evaluation Questions

Relevance
1) Is there a continued need for the IOM Program?
2) To what extent are the objectives of the IOM Program aligned with departmental and government-wide priorities?
3) Is the current role of the federal government appropriate in the delivery of the IOM Program?
Performance: Effectiveness
Immediate Outcomes
4) To what extent have other government departments/agencies and stakeholders been engaged in integrated oceans management?
Intermediate Outcomes
5) To what extent have Integrated Oceans Management Plans been implemented?
6) To what extent have Marine Protected Areas been designated?
7) To what extent are Integrated Oceans Management Plans being considered in oceans-related decisions?
Ultimate Outcome
8) To what extent are marine ecosystems being protected and conserved?
9) To what extent has the IOM Program made progress toward the management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans?
10) What, if any, unexpected results have occurred as a result of the IOM Program? (positive or negative)
Efficiency
11) To what extent was the IOM Program implemented as designed?
12) To what extent was the IOM Program delivered in an efficient manner? What factors supported/hindered efficiency?
13) To what extent are roles and responsibilities clear and appropriate:
  • among DFO, other federal departments/agencies, other jurisdictions and stakeholders involved in oceans governance;
  • among branches within DFO that participate in the IOM Program; and
  • between headquarters and regions delivering the IOM Program.
14) To what extent have science products and advice been sufficient and appropriate for IOM Program decision-making and ongoing monitoring and assessment of ocean spaces?
15) To what extent has the IOM Program been able to incorporate social, cultural and economic factors into its decision-making processes and governance?
16) To what extent do the performance monitoring/ measurement processes support decision-making and departmental accountability requirements?
17) What are the lessons learned in implementing the IOM Program thus far?
18) What are the key priorities for integrated oceans management moving forward?
Economy
19) Is there a more economical way to achieve the objectives of the IOM Program?

3.4 Data Sources

3.4.1 Document, Literature and Performance Data Review

The evaluation included a review of several secondary data sources. Documents that were reviewed included program and policy related materials, and policy documents (e.g., HOTO performance and accountability frameworks, monitoring and reporting templates, corporate planning and priority documents). External documents in the public domain were examined pertaining to the state of Canada's oceans and international experience with oceans management (e.g., establishment of marine protected areas). Finally, annual HOTO performance data for DFO components that are included within HOTO were reviewed.

3.4.2 Key Informant Interviews

In total, 24 interviews were conducted with key informants. The interviews were conducted with: DFO senior management and program managers (n=10), stakeholders (other federal government departments, provincial/territorial officials, representatives from Aboriginal organizations, non-governmental organizations, industry) (n=10); and experts (n=4).

The use of percentages to represent the qualitative information obtained from the key informant interviews is not appropriate. However, to ensure a common understanding of the terms used in the analysis and reporting of interview results, the following guidelines have been used:

  • "A few/a small number of interviewees" = less than 25 per cent;
  • "Some/a minority of interviewees" = 25 to 49 per cent;
  • "A majority of interviewees" = 50 to 75 per cent;
  • "Most interviewees" = over 75 per cent; and
  • "Almost all interviewees" = 95 per cent or more.

3.4.3 Survey of Partners/Stakeholders

An online survey was conducted with DFO IOM partners/stakeholders. Partners/stakeholders included participants in committees and consultations related to IOM (e.g., with respect to management of LOMAs and MPAs), including: federal/provincial/ territorial representatives; industry representatives (such as fishing, shipping, aquaculture, oil and gas); First Nations and Inuit representatives; non-governmental organizations and community groups, scientific/academic/educational organizations; and other stakeholders.

The survey contained a mix of closed-ended (categorical and scaled responses) and open-ended questions. Respondents provided information on the nature of their involvement in the IOM Program, feedback on aspects of the IOM Program, as well as a series of background questions (e.g., region, affiliation).

Based on lists of stakeholders assembled by DFO, 167 individuals were invited to complete the survey and had valid email addresses. In total, 60 individuals completed the survey for a response rate of 36 per cent5. The survey was launched in October 2011 and closed in November 2011.

5The distribution of respondents by affiliation is: federal government department/agency (25 per cent); provincial/ territorial government department/agency (23 per cent); academic/research/scientific organization (13 per cent); environmental or other non-governmental organization (13 per cent); industry/private sector (12 per cent); and Aboriginal organization/council (10 per cent).

3.4.4 Case Studies

Two case studies were completed internally by DFO Evaluation Directorate staff:

  • The Gully MPA. Located 200 kilometres off the coast of Nova Scotia, the Gully case study provides an illustration of processes for Oceans Act MPA establishment and management– i.e., from assessment to MPA designation to development of regulatory and management tools/approaches to monitoring and ongoing assessment.
  • Beaufort Sea Large Ocean Management Area, located in the extreme northwestern corner of Canada, is one of five priority areas identified for integrated ocean management planning. The Beaufort Sea case study was intended to illustrate the integrated oceans management approach – i.e., from science and social/cultural/economic assessments inputs to development of objectives/guidelines and implementation of management approaches. Only limited information from documentary sources could be collected for this case study, however.

3.5 Methodological Limitations

The strength of the methodological strategy for the IOM Program evaluation is a multiple lines of evidence approach that includes both qualitative and quantitative lines of evidence, and the views of both internal and external stakeholders. Study limitations include:

  • Conducting the HOTO and IOM Program evaluations simultaneously. There is significant overlap between the DFO HOTO components and the IOM Program. Conducting the evaluations simultaneously provided efficiencies in terms of use of evaluation resources. However, for DFO internal key informants, the interviews were quite lengthy and there was limited time to gather detailed responses for both programs.
  • Organizational change. Over the last five years, the IOM Program has been subject to a series of organizational changes. The ecosystem assessment sub-activity is a new sub-activity in the Program Activity Architecture that has only recently been incorporated into the IOM Program. As well, more recently, there has been a re-alignment of the program under a new program activity. There are challenges in defining the scope of the program over time and establishing the amount of financial resources for the Program.
  • Measurement challenges. The IOM program objectives have historically not been well-defined. The ultimate outcome of program – the management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans – is of a long-term nature and difficult to measure and clearly attribute the impact of the program specifically. This challenge is not unique to the IOM Program.

Other limitations of the individual lines of evidence include:

  • Review of documentation and literature: To bolster some of the study findings, external literature was sought regarding health of the oceans and Canada's progress on establishment of marine protected areas. Peer-reviewed literature, while preferable, could not be broadly canvassed and synthesized with the time and resources available for the study
  • Key informant interviews: Key informants contacted for interviews represented a cross-section of internal and external stakeholders. Priority candidates could not be reached in all cases.
  • Survey of stakeholders: The survey of stakeholders used industry standard methods for survey administration, including two email follow-up reminders and online and telephone assistance to support respondents. The response rate to the survey was low – just over 30 per cent – and given the finite initial pool of potential respondents, the final sample size for the survey is small. Representativeness of the sample to the population of IOM Program stakeholders is difficult to assess given the absence of known characteristics of the population. Therefore, results are suggestive, not definitive.
  • Case studies: The two IOM Program case studies were selected to represent activity within LOMAs and MPAs. Given the diversity of this activity, case study findings are illustrative and cannot be generalized to other components. As well, key informants for the Beaufort Sea case study were difficult to contact and, as a result, this case study is based largely on documentary sources only and was of limited utility to examine effectiveness of integrated oceans management in this LOMA.

The limitations identified above are common ones for program evaluations of broad environmental programs. Like other studies, this evaluation uses multiple lines of evidence to address the weaknesses of individual methodologies. This process of triangulation strengthens the evidence to answer the evaluation questions. Still, the IOM Program ultimate outcome of management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use is of a long-term nature, and difficult to measure and clearly attribute the impact of the program specifically.

4. Major Findings

This section presents key findings from the evaluation of the IOM Program. Findings are presented by evaluation issue (relevance and performance) for each evaluation question. For each evaluation question, a summary of the major findings is provided in a 'Key Finding' section.

4.1 Relevance

Evaluation Question #1: Is there a continued need (environmental, societal) for the IOM Program?

Key Finding: The evaluation evidence supports the continued need for programming to address the health of the oceans, which show signs of decline and under pressure due to increased use. Canada's oceans have significant societal importance, contributing to economic activity and growth, as well as meeting other social and cultural needs of Canadians.

A review of documentation supports the continued need to address the health of the oceans in Canada. Evidence indicates newly emerging signs of stress on marine environments caused by human and environmental activity6. Pollution, invasive species, oil drilling, and overfishing are just a sampling of the issues affecting the biodiversity and sustainability of Canada's oceans7. While concern over many of these issues has existed for some time, there continue to be activities that degrade habitat and damage ecosystems8. Many expect human activities in the ocean such as tourism, industrial processes, and shipping to intensify in the future9., particularly in Arctic areas as ice sheets recede10.

With respect to continued societal need, it has been estimated that the economic contribution of ocean sector activities represented an estimated $17.7 billion in direct gross domestic product in Canada in 2006 and over 171,000 people are employed in related industries11. In addition, there are cultural and social benefits of oceans (e.g., sources of food, recreation) to the one in five Canadians who lives in a coastline community and, more broadly, to all Canadians.

Evaluation key informants across all respondent categories agree that there is a continued need for attention to oceans management. Key informants cited issues such as declining sea ice in the Arctic, habitat degradation, depleted fish stocks and threats due to climate change and invasive aquatic species. Many key informants further noted the economic and societal significance of oceans, anticipating a trend toward increased use of oceans (for example, oil and gas exploration and development, offshore mining and, aquaculture). Public opinion generally echoes key informant views. General awareness of environmental issues affecting the oceans is high and Canadians expect the government to show leadership in this area12.

Eight in ten surveyed IOM stakeholders or more indicate that there is a continued need for federal attention to all of the key themes under IOM – science, engagement of stakeholders, and integrated oceans management planning. Three-quarters of stakeholders indicate that there is a continued need for federal action on designation of marine protected areas and protection of marine ecosystems.

Figure 1: Stakeholder Opinion on Continued Need for Federal Action

Continued Need for Federal Action

In addition to the need for attention to oceans, most key informants agreed that there is a need for an integrated oceans management approach to understand oceans from an ecosystem perspective, taking into account the cumulative impact of human and environmental interactions. As oceans spaces are complex and shared, integrated oceans management is an inclusive approach that ideally brings stakeholders together around common objectives. Key informants noted that integrated oceans management has significant historical success in terrestrial planning, which is now being emulated for marine spaces.

The integrated oceans management approach was also noted to be the international standard in oceans management, adopted by the U.S., European Union, and Australia among others. The Report of the Commissioner of the Environment and Sustainable Development(2005)13 summarized the potential benefits of integrated oceans management:

  • Resolving oceans-use conflicts – where pipelines or cables are to be placed in or near traditional fishing areas, an integrated approach would identify routings that minimize the impact on the fishery;
  • Creating opportunities for, and enabling the potential use of ocean resources – seabed mapping can be used to identify opportunities for new or existing industries;
  • Limiting and mitigating the negative impacts of human use of ocean resources – shipping traffic lanes can be rerouted to avoid areas where endangered marine mammals exist; and
  • Identifying priorities for management and science activities – using risk assessment, limited science and management resources can be invested in high-risk or high-opportunity areas.

Evaluation Questions #2: To what extent are the objectives of the IOM Program aligned with departmental and government-wide priorities?

Key Finding: The IOM Program and integrated oceans management are grounded in the Oceans Act/Canada's Oceans Strategy. In delivering on components of the HOTO Initiative, the IOM Program is also a key component of the federal National Water Strategy. The Program complements other federal strategies with respect to sustainable development and supports Canada's commitments in international agreements. The IOM Program is aligned to the department's strategic objective of Sustainable Aquatic Ecosystems, and is also seen to contribute to Economically Prosperous Maritime Sectors and Fisheries.

Federal priorities

The Oceans Act and Canada's Oceans Strategy include provisions for integrated oceans management. These authorities identify Canada's need to conserve and protect oceans environments, ecosystems and resources through integrated management to ensure economic prosperity and environmental sustainability.

According to program documents and key informant respondents, the IOM Program is aligned with federal government priorities pertaining both to conservation and economic prosperity, particularly the Government of Canada Economic Affairs Outcome Area, A clean and healthy environment. The 2007 Throne Speech committed the Government of Canada to implement a new water strategy to help clean up major lakes and oceans. Budget 2007 created the National Water Strategy, which allocated $105.3 million in funds to improve the quality of water in Canada's rivers, lakes and oceans, including funding for HOTO, which is led by DFO.

Documents and key informant respondents also indicate the complementarity of the IOM Program with federal strength in science and monitoring, and with other federal strategies, including the Federal Sustainable Development Strategy for Canada among others14.

Also notable, Canada has broad commitments and obligations under international agreements that confer a domestic responsibility for action on oceans, such as those set out in the United Nations Convention on the Law of the Sea (UNCLOS). More specifically, a global commitment to establish a network of marine protected areas was made at the 2002 World Summit on Sustainable Development, and the 2004 United Nations Convention on Biological Diversity Program of Work on Protected Areas. The global biodiversity targets in the CBD Strategic Plan for Biodiversity that was adopted at the 10th Convention of the Parties in 2010 call for 10 per cent of coastal and marine areas being protected by 2017. The Strategic Plan for North American Cooperation in the Conservation of Biodiversity is a shared commitment of the 3 North America Free Trade Agreement state members and, the North American Marine Protected Areas Network (NAMPAN) is nested under this Plan and is guided by the following goal: "Promote the cooperation for the conservation and maintenance of North American regions of ecological significance."

Departmental priorities

DFO's PAA serves to situate the IOM Program within the department's program activities leading to strategic outcomes. The Department's PAA has undergone significant and frequent revisions in the last five years. (The evolution of the PAA is summarized below). Currently (2011-12 PAA), the IOM Program is a program activity within the Sustainable Aquatic Ecosystems strategic outcome. Key informants also note that the IOM Program contributes to the strategic outcome of Economically Prosperous Maritime Sectors and Fisheries by supporting sustainable development of oceans.

Table 5: PAA Evolution: Oceans Management Program Activity and Sub-Activities

2005-06
Oceans Management
Integrated Management
Marine Protected Areas
Other Oceans Management

2008-09
Oceans Management
Integrated Oceans Management
Marine Conservation Tools

2011-12
Integrated Oceans Management
Marine Conservation Tools
Ecosystem Assessments*

* The Ecosystem Assessments sub-activity was formerly a part of the Aquatic Ecosystems Science sub-activity under the Science for Health and Productive Aquatic Ecosystems program activity. In addition to the ecosystem assessments function, the former Aquatic Ecosystems Science sub-activity included other science functions, mainly integrated management of scientific data that are not included under the 2011-12 Integrated Oceans Management Program Activity.

Several internal key informants, as well as external stakeholders and authors noted a challenge in the priority alignment of integrated oceans management within DFO that reflects a broader tension around the department's dual role15. Integrated oceans management is based on an ecosystem approach, including ecosystem science that takes into consideration the interactive effects of human and other activities, as well as environmental drivers in the same aquatic ecosystem. The ecosystem approach entails a much longer planning horizon and considers a broad spectrum of current and future interests of multiple uses of oceans resources. This is a significant departure from DFO's traditional approach, which focused on single-species analyses and on fisheries management. The internal philosophical and cultural shift resulting from adopting an ecosystem approach, and increased demands on departmental and science resources, continue to challenge the department in fulfilling its Oceans Act responsibilities. However, there has been a notable change in DFO's organization – bringing together oceans management and fisheries management within the Program Policy, as well as Ecosystem and Fisheries Management sectors. This was noted by several key informants as a promising development, having the potential to forge a stronger alignment between the IOM Program and departmental priorities with respect to fisheries management.

Evaluation Questions #3: Is the current role of the federal government appropriate in the delivery of the IOM Program?

Key Finding: DFO has a legislated (Oceans Act) role to play in the delivery of integrated oceans management. The delivery of the program is consistent with this role.

The Oceans Act provides a framework for current and future oceans management initiatives in Canada, calling for the Minister of DFO to lead and facilitate the development of a national oceans management strategy, among other responsibilities. Specifically, Canada's Oceans Strategy "provides the overall strategic framework for Canada's oceans-related programs and policies, based on the principles of sustainable development, Integrated Management and the precautionary approach. The central governance mechanism of the Strategy is applying these principles through the development and implementation of Integrated Management plans".

Implementation of integrated oceans management is not a DFO responsibility alone, however. The program involves other federal government departments, provincial, territorial, and municipal governments, Aboriginal groups and a wide range of other stakeholders.

Key informants agree that the federal government is playing an appropriate role in integrated oceans management, albeit with room for improvement in terms of achieving its leadership role mandated by the Oceans Act. A number of areas where key informants thought the current federal role in integrated oceans management could be improved were identified (and are discussed in more detail in Section 4.3).

6P. J. Ricketts and L. Hildebrand, "Coastal and Ocean Management in Canada: Progress or Paralysis?," Coastal Management 39, no. 1 (2011): 4-19. International Programme on the State of the Ocean, Implementing the Global State of the Oceans Report (IPSO, 2011), http://www.stateoftheocean.org/research.cfm.

7J.L. Smith, K. Lewis, and J. Laughren, A Policy and Planning Framework for Marine Protected Area Networks in Canada's Oceans, Halifax: World Wildlife Fund, 2006

8P. L. Stewart and B.T.L. White, "A Review of Contaminants on the Scotian Shelf and in Adjacent Coastal Waters: 1970-1995.," Canadian Technical Report of Fisheries and Aquatic Sciences 2351 (2001): xviii +158 pp.

9M. Molnar and N. Koshure, Cleaning up Our Ocean, David Suzuki Foundation/Sierra Club of BC/Living Oceans Society, 2009.

10International Programme on the State of the Ocean, Implementing the Global State of the Oceans Report.

11http://www.dfo-mpo.gc.ca/ea-ae/cat1/no1-1/no1-1-eng.htm

12Department of Fisheries and Oceans Canada. (2005). Canada's Federal Marine Protected Areas Strategy (p. 18). Retrieved from http://www.dfo-mpo.gc.ca/oceans/publications/fedmpa-zpmfed/pdf/mpa-eng.pdf
McAllister Opinion Research. (December 16, 2008). "BC opinion poll finds widespread concern over health of oceans." Canada Newswire. http://www.newswire.ca/en/story/232915/bc-opinion-poll-finds-widespread-concern-over-health-of-oceans
McAllister Opinion Research. (September 27, 2007). "Increasing majority call Canada's pollution laws inadequate." http://www.globescan.com/news_archives/em07_pr/em07.pdf

13Office of the Auditor General of Canada, Report of the Commissioner of the Environment and Sustainable Development to the House of Commons, 2005 (Chapter 1 Fisheries and Oceans Canada - Canada's Oceans Management Strategy.

14Priorities for Environmental Sustainability (Priority III: Protecting Nature; Goal 6; Ecosystem/Habitat Conservation and Protection; 6.3 Target: Marine Ecosystems - Improve the conservation of ocean areas and marine ecosystems by 2012)

15M. Rutherford, M. Dickinson and T. Gunton, "An Evaluation of the National Framework for Marine Planning in Canada", Environments Journal, Vol. 37 (3), 2010.

4.2 Performance: Effectiveness

Evaluation Question #4: To what extent have other government departments/agencies and stakeholders been engaged in integrated oceans management?

Key Finding: There are many processes in place to engage other government departments and stakeholders in integrated oceans management. These processes vary across regions based on jurisdictional realities and capacity of stakeholders. Information sessions, and technical, advisory and stakeholder committees have been established within LOMAs and for Oceans Act MPAs. Stakeholders themselves are generally satisfied with their participation, though they perceive engagement to have had limited influence on decision-making at this stage.

Diverse government authorities and stakeholders have an interest in oceans activities and management, and are engaged by DFO in integrated oceans management at the regional level through Regional Committees on Oceans Management (or similar). The mandate for the Regional Committees on Oceans Management is similar across the regions, though the structure and maturity of committees vary significantly. Some regions also make use of ancillary governance structures at the regional and sub-regional levels (e.g., in some regions, additional governance structures exist for the LOMAs themselves) and the governance framework must also respond to the evolving needs and priorities of provinces, territories and other federal departments/agencies to develop regional, provincial or interprovincial oceans governance bodies and structures. Though the composition of the Regional Committees on Oceans Management is variable, they are often chaired or co-chaired by senior regional DFO or other federal departments and provincial officials, and are a forum for engaging federal departments/agencies, provinces, territories, non-governmental organizations and Aboriginal groups including those bodies established under land claims agreements. Engagement occurs through community consultations, and the creation of technical, advisory and stakeholder committees.

In general, key informants feel that the IOM Program has engaged stakeholders in integrated oceans management very well to increase awareness of sustainable oceans management. In LOMAs there are varying voices of interest with specific constituencies (ENGOs, industry), along with government authorities that are representing the interests of the broader general public. According to key informants, the advantage of stakeholder involvement in an IOM approach is engaging this broad perspective. Stakeholder engagement and collaborative mechanisms are effective to establish trust, enhance communications, and to avoid duplication of efforts. Information and knowledge is shared through many means – federal Oceans Portal, formal MOUs, distribution of the results of consultations, and meetings. The approach ensures that a range of stakeholder and regional views are represented.

The Gully MPA case study illustrates these processes and some of the challenges. The Gully Advisory Committee is the main governance structure for managing the Gully MPA. Key informants and governance review documents found the Committee was effective in creating an open and consensual approach to decision-making among stakeholders. However, concerns were raised about the lack of representation and regular attendance from certain stakeholders such as fish harvesters, ENGO and Aboriginal groups. In particular, when the MPA was designated and the focus of the committee shifted to management, participation of stakeholders waned. The case study also noted the potential for capacity challenges, as there are several committees operating in the Maritimes, Gulf and Newfoundland regions, all requesting participation of stakeholders which are often the same organizations or representatives.

IOM partners/stakeholders who were engaged in consultation and committee/briefing processes and participated in the survey were generally satisfied with the engagement processes (though the sample size is quite small, n=42). About eight in ten stakeholders agree that the views of all stakeholders were considered in committee discussions and decisions, committee/consultation roles and responsibilities were made clear and were understood, that adequate outreach efforts were made to include a broad spectrum of stakeholders and that committee/consultation results were disseminated appropriately (responded 4 or 5 on a 5-point scale).

Weaker ratings were expressed regarding efficiency of committee management, identifying priority issues and providing adequate information on the nature/benefits of oceans activities to the marine environment.

Surveyed IOM partners/stakeholders who participated in committees and/or consultations/briefings felt that the most notable achievement of these processes was the inclusion of a broad spectrum of views from oceans stakeholders (seven in ten said this happened to a moderate or considerable extent - responded 3, 4 or 5 on a 5-point scale). Two-thirds felt that information and knowledge sharing across stakeholder communities was achieved to a moderate or considerable extent. Collaborative mechanisms were rated as less effective in terms of taking stakeholder needs and priorities into account in oceans decision-making and increasing stakeholder commitment to collaborative oceans management.

Figure 2: Stakeholder Opinion on Impact of Committees/Consultations

Impact of Committees/Consultations

Overall, these views were echoed by key informants. As mentioned above, while engagement of stakeholders was extensive through consultations and committees, leading to information sharing and the gathering of a broad spectrum of views, key informants perceived there to be challenges in translating this collaborative effort into concrete actions and decision-making. Challenges identified by key informants included:

  • Limited attention and commitment to collaborative processes at the senior management level, particularly the federal presence representing other government departments, and delays in approval of IOM Plans at the senior levels within DFO;
  • Regional differences in the effectiveness of engagement processes. For example, in the Gulf of St. Lawrence, the jurisdictional complexity of this LOMA has challenged the implementation of collaborative mechanisms. Therefore, the region has moved forward using other forums such as information sessions;
  • Challenges in managing expectations of stakeholders, in particular, clarifying the distinction between the advisory role of committees and the decision-making responsibility or authority of federal and provincial authorities; and
  • The geographical breadth of LOMAs, number of stakeholders and complexity of issues has led to inefficiencies in stakeholder engagement. Processes are protracted and have yielded limited results in terms of concrete actions and decision-making.

Evaluation Question #5 and #7: To what extent have Integrated Oceans Management Plans been implemented? To what extent are Integrated Oceans Management Plans being considered in oceans-related decisions?

Key Finding: Progress in the development of IOM Plans for the five designated LOMAs has been slower than originally anticipated, with two IOM Plans being completed and another two identified as being on track for completion in 2011-12. To date, one IOM Plan has been officially approved by DFO. Due to limited endorsement and implementation of IOM Plans, IOM Plans are having a limited impact on oceans-related decisions.

The development and implementation of IOM Plans in five established LOMAs was a significant priority under the IOM Program. LOMAs were identified and delineated as pilot sites to test the integrated oceans management approach to addressing ecosystem health and sustainable use of oceans resources. The five existing LOMAs have been selected on criteria such as: important living and non-living marine resources; high biological diversity and productivity; and many stakeholders competing for ocean space and resources. The five LOMAs are:

  • Placentia Bay/Grand Banks;
  • Eastern Scotian Shelf;
  • Gulf of St. Lawrence;
  • Pacific North Coast; and
  • Beaufort Sea.

The development of IOM Plans that include ecological, social and economic objectives is a key requirement of successful integrated oceans management. A central objective of Canada's Ocean Strategy is the development of a system of nested IOM Plans for all marine waters, and to establish within these a national network of marine protected areas. Accordingly, DFO's 2005-2010 Strategic Plan also set out targets for the establishment of IOM Plans in all five LOMAs by 2012. Under the IOM Program, funding was allocated to establish and support regional governance and planning bodies, and to conduct scientific and economic research to support the development of IOM plans within the five designated LOMAs.

Under the IOM Program, Ecosystem Overview and Assessment Reports and the identification of Ecologically and Biologically Significant Areas reports were completed in each of the five LOMAs. Approaches to mapping priority sensitive marine areas and protocols for identifying ecosystem objectives were developed. An economic assessment of marine activities in the five LOMAs was undertaken which assessed the economic impact (GDP, employment and income) of key marine industry, public and para-public sectors as well as an analysis of the economic interactions between the various sectors.

The evaluation findings indicate that while progress has been achieved in the development of IOM Plans, achieving departmental targets has been slower than originally anticipated due to a variety of challenges (see discussion below). Through the IOM Program, inter-jurisdictional regional committees on oceans management or Regional Committees on Oceans Management were established in each of the five LOMAs to develop IOM Plans. As of 2009-2010, two of the existing LOMAs had completed their IOM Plans and the Beaufort Sea IOM Plan was endorsed by the Minister of Fisheries and Oceans in 2010. IOM Plans for two of the remaining three LOMAs are expected to be completed by March 2012. Despite approval of the IOM Plans, in some regions, it has been possible to get buy-in of other government departments for management changes to make progress toward objectives achievement (e.g., Canada-Nova Scotia Offshore Petroleum Board endorsement of Coral Conservation Plan; Gully designation as Area to be Avoided).

While the evaluation evidence indicates that stakeholders and government agencies have been engaged in integrated oceans management, delays in the development and endorsement of IOM Plans by DFO, as well as lack of continuity and adequate funding of integrated oceans management were perceived by some to have created uncertainty and negatively impacted the effectiveness of the IOM Program. According to some key informants and noted also in the literature, IOM Plans are also noted to be weak in terms of timelines, designated agency responsibilities for implementation and specificity of actions and strategies16. Difficulties associated with a multi-stakeholder consultative approach (e.g., engagement, trust), the large volume of scientific and socio-economic data that needs to be collected/analyzed were also seen as factors that impeded the implementation of IOM Plans.

Although key informant opinion attributes the IOM Program with having increased awareness and understanding of the objectives and benefits of integrated oceans management, since the IOM Plans have not been widely implemented, there is little evidence to indicate that federal or provincial departments, agencies or regulatory bodies have begun to adjust their programs/policies to implement IOM objectives or commitments. Just over half of surveyed IOM partners and stakeholders consider that IOM Plans have had a moderate or considerable influence on policies, practices or decisions related to oceans activities. One in three believe the IOM Plans have had a limited impact.

Figure 3: Perceived Impacts of IOM Plans Evaluation

Impact of IOM Plans

Question #6 and #8: To what extent have Marine Protected Areas been designated? To what extent are marine ecosystems being protected and conserved?

Key Finding: Progress in establishing MPAs and protecting and conserving marine ecosystems is mixed. There has been significant progress in the development of key strategic and operational supports to establish and manage a national network of protected areas. Progress has been slower in establishing new MPAs (the HOTO goal of six new Oceans Act MPAs by 2012 will not be met) and developing indicators and monitoring protocols for existing MPAs.

The marine conservation tools sub-activity of the IOM Program includes two streams of activities, funded with a mix of A-base (departmental permanent funds) and B-base funds from the OAP and HOTO Initiative. The first stream is management of existing Oceans Act MPAs, while the second is the designation of new Oceans Act MPAs. In addition to these activities, the HOTO Initiative provided funding for advancing a Federal Marine Protected Areas Strategy and a National Framework for Marine Protected Areas. Each of the three broad activities related to Oceans Act MPA monitoring and designation, and the marine protected area network are described, in turn, below.

Management of Existing Oceans Act MPAs

To date, there are eight Oceans Act MPAs that have been designated. DFO regional offices are responsible for implementation and coordination of activities related to MPA management plans, including: promote compliance with regulations; monitor permitted activities within the MPA; coordinate and carry-out surveillance of the MPA and enforcement of regulations; provide outreach opportunities and materials; support and conduct research in the MPA; and evaluate and monitor management of the MPA to ensure objectives are met17. A science-based ecosystem approach is applied in the design of the Oceans Act MPA regulations and management wherein oceans uses and users are comprehensively assessed against the ecosystem features being protected. Oceans Act MPA management often occurs through the use of zones providing varying levels of protection based on conservation objectives and ecological sensitivities.

DFO Science received funding under the HOTO Initiative to develop indicators and monitoring protocols and strategies for the management of existing Oceans Act MPAs and to evaluate the effectiveness of Oceans Act MPAs in achieving their conservation objectives. Of eight Oceans Act MPAs, five require indicators/monitoring protocols and three have indicators and monitoring protocols and strategies developed (Eastport, Gilbert Bay, Basin Head, Gully and Musquash Estuary). Indicators have been identified for Tarium Niryutait and the development of monitoring protocols is underway. The development of indicators and monitoring protocols for the remaining two MPAs (Bowie Seamounts and Endeavour Hydrothermal Vents) are in varying stages of the identification or designation process.

The management of existing Oceans Act MPAs has encountered a number of challenges. The articulation of the conservation objectives of existing Oceans Act MPAs has presented overarching difficulties for DFO Science in identifying a set of practical indicators and monitoring protocols and strategies to guide their management and assess effectiveness. An example of this is in the Gully MPA. In 2010, the Canadian Science Advisory Secretariat produced the Gully Marine Protected Area Monitoring Indicators, Protocols and Strategies study which reviewed and assessed the state of existing ecological indicators for monitoring the Gully, while identifying and incorporating others to address the MPA's conservation objectives. The next step for implementation is to establish a program to conduct the ecosystem monitoring where 47 indicators have been recommended by Science. However, monitoring of this offshore MPA is expensive due to distance and the monitoring strategy as proposed cannot be sustained with current funding levels. The Gully case study illustrates further MPA management challenges including, for example, loss of momentum and internal departmental and stakeholder interest once the Gully was designated an MPA, the difficulties of coordinating MPA surveillance and enforcement of activities that are not regulated by DFO and anticipated resistance from the fishing industry to the ecologically based conservation objectives and proposed monitoring protocols.

Designation of New MPAs

A key objective for DFO under the HOTO Initiative was to designate six new Oceans Act MPAs by 2012. While selection of the HOTO AOIs was intended to be informed and coordinated by a marine protected area approach the National Framework for Canada's Network of Marine Protected Areas was approved too late to inform the selection of the Oceans Act AOIs under HOTO. DFO Science received funding under the HOTO Initiative to collect baseline information to help inform the identification of new Areas of Interest for Oceans Act MPAs. Ecosystem science provided advice and information in support of the selection of seven Areas of Interest for Oceans Act MPAs, and the development of ecological assessments,

With respect to Oceans Act MPAs, six HOTO AOIs have been approved and announced (Hecate Strait-Queen Charlotte Sound Glass Sponge Reefs, Laurentian Channel, Darnley Bay, Shediac Valley, American Bank and St Anns Bank). Because many of the announced Areas of Interest are not far enough along in the MPA establishment process, the HOTO timeline will not be achieved, although all Areas of Interest are expected to be designated by 2014. The following table provides an overview of the status of the six Oceans Act Areas of Interest as of October 2011.

Table 6: Status of Health of the Oceans Areas of Interest under the Oceans Act (October 2011)
Approved Area of Interest Date announced Status
Hecate Strait/Queen Charlotte Sound glass sponge reefs Pacific Announced June 2010 Draft regulatory intent close to completion. Consultations on regulatory intent are complete. Preparation of background regulatory documents anticipated by December 2011. Potential designation in 2012.
Darnley Bay
Central & (Arctic)
Announced October 2010 Consultations with F/P/T, Aboriginal partners and other stakeholders are underway. Preparation of background regulatory documents anticipated by April 2013. Potential designation in 2013.
Laurentian Channel (NFLD) Announced June 2010 Advisory Committee formed. Consultations with F/P/T, Aboriginal partners and other stakeholders underway. Preparation of background regulatory documents anticipated by spring/summer 2012. Potential designation in 2013.
Shediac Valley (Gulf) Announced June 2011 Advisory Committee being formed. Documents being prepared for consultation. Consultations to begin in winter 2012. Preparation of background regulatory documents anticipated by March 2013. Potential designation in 2014.
American Bank (Quebec) Announced June 2011 Advisory Committee to be being formed in early 2012. Documents being prepared for consultations. Consultations with F/P, Aboriginal partners and other stakeholders are under way to begin in Fall 2011. Preparation of background regulatory documents anticipated by March 2013. Potential designation in 2014.
St Anns Bank (Maritimes) Announced June 2011 Advisory Committee being formed. Documents being prepared for consultations. Consultations underway. Preparation of background regulatory documents anticipated by December 2013. Potential designation in 2014.

Delays in the designation of the new HOTO AOIs occurred for a number of reasons. As approval of HOTO funding did not occur until September 2007, funding was not available to DFO for Oceans Act MPA activities until as late as January 2008. This resulted in a delay in the implementation of many planned activities by almost a year. Workplans and priorities were adjusted in subsequent years in response; however, delays were particularly problematic for MPA establishment activities for which the HOTO five-year timeframe was considered to be already very compressed. Key informants also noted significant delays in the approval processes for HOTO AOIs, as well as protracted consultation processes for some. These challenges were echoed by surveyed IOM partners and stakeholders who identified a weak federal commitment, the time consuming nature of consultations, difficulties in resolving conflicting interests of stakeholders and lack of buy-in from industry, and insufficient resources as the key challenges in the establishment and management of marine protected areas.

Figure 4: Perceived Challenges of Implementing MPAs

Challenges of Implementing MPAs

Federal Marine Protected Area Strategy and National Framework for Canada's Network of Marine Protected Areas

Under the HOTO Initiative, DFO, Environment Canada (EC) and the Parks Canada Agency (PCA) were funded to implement the Federal Marine Protected Area Strategy and make progress toward a Network of Marine Protected Areas. Prior to HOTO, the three departments/agency had begun work to coordinate their marine protected area establishment activities, and HOTO funding built on these efforts. The Strategy responds to the need for cooperation and collaborative approaches among the three federal departments/agency with marine protected area mandates. The coordinated approach also led to a common education and engagement strategy for marine protected areas. DFO also received funding to advance coordination of a national (federal-provincial-territorial) network of marine protected areas.

The evaluation evidence indicates that, through HOTO, a considerable amount of the necessary groundwork to establish and implement a national MPA network has been accomplished. HOTO provided the funding to increase organizational capacity within DFO (2 FTEs), as well as EC and PCA (1 FTE each), which also have mandates for establishment of MPAs. An inter-departmental (DFO, EC, PCA) Federal Marine Protected Areas Strategy (FMPAS) Working Group and DG-level oversight committee were formed to implement the Strategy. A key guidance document was developed, the Federal Guide for Collaborative Planning of Marine Protected Areas (2009), and a national inventory of 810 federal-provincial-territorial MPAs and some 200 "contributory sites" was compiled. In 2008 a workshop of international MPA network experts was convened to advise on Canada's approach to MPA network planning. In 2010, Science advice was sought on how to adapt international MPA network design properties to the Canadian situation, and on an appropriate spatial framework for the national network of MPAs. As a result of the latter analysis, 13 bioregions for MPA network planning were identified. A federal-provincial-territorial Technical Experts Committee was also struck under the supervision of an Oceans Task Group reporting to the Canadian Council of Fisheries and Aquaculture Ministers (CCFAM). This group collaborated with DFO in drafting an overarching, multi-jurisdictional National Framework for Canada's Network of Marine Protected Areas (2011). Drafted by the federal-provincial-territorial Technical Experts Committee, the National Framework, approved in principle by CCFAM in September 2011, outlines: the proposed overarching vision and goals of the national MPA network; establishes the network components, design properties and eligibility criteria for which areas will contribute to the network; describes the bioregional MPA network governance structure; and provides the direction necessary to promote national consistency in bioregional MPA network planning. In addition, a number of outreach activities were undertaken (workshops, community events) and communications products/supports were developed (e.g., Oceans Portal, Spotlight on Marine Protected Areas in Canada (2010), Emerald Sea (Pacific Coast) video). Technical guidance to support implementation of the National Framework at the bioregional level is now being drafted.

To support development of a marine protected area network, DFO Science provided advice on biogeographic units to guide marine protected area network planning. Examples of DFO Science guidance documents developed through the Canadian Science Advisory Secretariat process include: Development of a Framework and Principles for the Biogeographic Classification of Canadian Marine Areas (2009) and Science Guidance on the Development of Networks of Marine Protected Areas (MPAs) (2009)18. During the current fiscal year, DFO Science will prepare a guidance document on "Representative" Marine Protected Areas for Network Planning19.

Impact of Oceans Act MPAs on marine conservation

In general, the documentary evidence and key informant opinion indicate that marine protected areas are an important conservation tool for oceans spaces under specific conditions. While there is some debate about the optimal size of marine protected areas and the correct framing of objectives and effective monitoring, marine protected areas are generally regarded as (one of many) effective marine conservation tools.

With respect to Canada's international commitments toward a marine protected area network, the Convention on Biological Diversity Strategic Plan for Biodiversity (2010) calls for 10 per cent of coastal and marine areas to be protected by 2017. Currently, marine protected areas protect one per cent of Canada's oceans and Great Lakes20. Australia has increased its protected areas from 4.5 per cent to 7.5 per cent of its ocean space over the last 10 years. Canada was ranked 70th among 228 countries in establishment of marine protected areas21.

In Canada, it is early days to determine the effectiveness of the marine protected areas that have been established, though some initial data is becoming available for several marine protected areas. For example, the Gully MPA case study pointed to several instances where policies and practices of stakeholder groups and regulators demonstrate consideration for marine conservation objectives and guidelines (e.g., the Canada – Nova Scotia Offshore Petroleum Board has enshrined conservation objectives into their economic impact assessment approach; oil and gas companies have established codes of conduct which forbids flying over or sailing through the Gully MPA; Integrated Fisheries Management Plans and Fishing licenses cite the MPA regulations for prohibitions and limitations on fishing in the MPA).
 
Half of IOM Program partners and stakeholders that were surveyed consider, at least to a moderate extent, that the establishment of protected areas is resulting in the conservation of the marine ecosystem in the designated space, and that MPAs are having an impact on the policies, practices or decisions related to oceans activities. A substantial minority, however, believe MPAs are having no impact or a limited impact in these areas.

Figure 5: Perceived Impacts of MPAs

Impact of MPAs

Evaluation Question #9: To what extent has the IOM Program made progress toward the management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans?

Key Finding: Through engagement of stakeholders and collaborative mechanisms, the IOM Program has provided a foundation to shift attitudes and increase understanding of an ecosystem approach to oceans management. However, preserving the ecological health of the oceans, while allowing for sustainable use is a long-term goal and dependent on the actions of many other authorities and stakeholders.

Key informants provided a mixed review of the IOM Program's progress toward preserving the ecological health of the oceans, while allowing for sustainable use of oceans resources. Internal key informants note that the ultimate objective of the IOM Program is broad and difficult to measure and will occur only in the longer-term and with action on the part of other responsible agencies.

Integrated oceans management entails a shift in attitude and culture. While difficult to assess this change, key informants were of the opinion that the IOM Program had laid this foundation. Stakeholders generally share this perspective, though also raised concerns that the IOM Program in its current form is unlikely to achieve its longer-term objective. Leadership and funding for IOM, as well as balanced consideration of conservation and development interests, were identified as prerequisites for further progress.

In a similar vein, surveyed IOM partners/stakeholders provided generally positive reviews for the IOM Program short-term intended outcomes related to contribution to collaboration/knowledge sharing and advancing science. A majority of surveyed stakeholders feel that there has been moderate to considerable impact on advancing collaborations and knowledge sharing, and in advancing scientific understanding of oceans ecosystems. Mirroring the concerns of key informants, survey respondents are less certain that the IOM Program has had an impact on preserving the ecological health of oceans or marine protected areas (about half of surveyed stakeholders indicate this impact has occurred to a moderate or considerable extent, and a significant proportion believe this occurred to no extent or only a limited extent).

Figure 6: Impact of Oceans Management Efforts

Impacts of Oceans Efforts Overall

Evaluation Question #10: What, if any, unexpected results have occurred as a result of the IOM Program?

Key Finding: Few unintended outcomes of the IOM Program were identified. Positive unintended outcomes include extending IOM collaborations to address additional or related issues. An unintended negative outcome is that stakeholder expectations have not been met with respect to integrated oceans management, leading to a loss of credibility of the process and DFO.

Key informants noted few unexpected results of the IOM Program. Several key informants reported positive results of interdepartmental collaboration that have extended beyond the IOM Program and HOTO. There has also reportedly been an unexpected increased cohesion among fisheries industries as they articulate their interests in an IOM framework.

A minority of key informants felt that the IOM Program resulted in a more unwieldy collaborative process than intended. As well, the Oceans Act and Canada's Oceans Strategy raised stakeholder expectations regarding their engagement and the expected results of integrated oceans management. The slow progress in approving and implementing IOM Plans and designating Oceans Act MPAs had led to some frustration among stakeholders and loss of credibility of the process and DFO as the process has brought to light some deficiencies in oceans management, knowledge, policy, and leadership.

16M. Rutherford, M. Dickinson and T. Gunton, "An Evaluation of the National Framework for Marine Planning in Canada", Environments Journal, Vol. 37 (3), 2010.

17Based on the case study of the Gully Oceans Act MPA.

18http://www.dfo-mpo.gc.ca/CSAS/Csas/Publications/SAR-AS/2009/2009_056_e.pdf; http://www.dfo-mpo.gc.ca/CSAS/Csas/Publications/SAR-AS/2009/2009_061_e.pdf

19http://www.dfo-mpo.gc.ca/csas-sccs/Schedule-Horraire/2011/12_12-13-eng.html

20There are almost 700 marine protected areas captured in the report by Fisheries and Oceans Canada, Spotlight on Marine Protected Areas in Canada, 2010; 83 managed by the federal government.

21Fisheries and Oceans Canada, Spotlight on Marine Protected Areas in Canada, 2010.

4.3 Performance: Efficiency

Evaluation Question #11: To what extent was the IOM Program implemented as designed?

Key Finding: The IOM Program was broadly implemented as intended. However, there have been significant delays in the development, endorsement and implementation of IOM Plans and the establishment of Oceans Act MPAs. Distributed regulatory authorities for oceans, combined with weak internal support for IOM have hindered the planned implementation of integrated oceans management. Other implementation challenges include: the absence of concrete actions or milestones in IOM Plans; intergovernmental and jurisdictional complexities; competing stakeholder interests; and insufficient funding.

Annual Departmental Performance Reports and key informant opinion indicate that, overall, integrated oceans management initiatives are being implemented as designed, with some adjustment to approach and timelines. For example, implementation of integrated oceans management is moving toward a bioregion-based framework which represents a departure in some regions from the LOMA-based framework used for regional IOM planning. With respect to timelines, as mentioned previously, there have been significant delays in the implementation of IOM Plans and establishment of Oceans Act MPAs.

Documentary sources and key informant opinion identify key structural factors that have negatively influenced the implementation of integrated oceans management. The Oceans Act extends only a facilitation and leadership mandate to the Minister of Fisheries and Oceans, and does not compel other Ministers to work with DFO on IOM implementation. Internal key informants perceived a lack of ownership of the oceans issue by other government departments as a result.

The Oceans Act has limited regulatory authority other than designating and managing MPAs (though the Oceans Act does confer authority to DFO for the development of guidelines associated with Marine Environment Quality). To date, DFO has opted to assume a facilitator role in integrated oceans management rather than a regulatory one. Competing priorities within DFO between its mandate to promote fisheries/aquaculture and its mandate to protect marine areas, combined with the lack of a DFO champion (within senior management) for oceans, has also posed a challenge to ensuring adequate support for the IOM Program and the integration of ecosystem-based management across the department.

Other factors cited by key informants as having an impact on the implementation of integrated oceans management activities include: intergovernmental and jurisdictional aspects and complexities; competing stakeholder interests and differences in the availability of human and financial resources to support ongoing participation in LOMA/MPA planning processes across stakeholder groups (i.e., lack of a level playing field); insufficient funding for the program; and the lack of clarity and consistency of policy direction.

Surveyed IOM Program partners and stakeholders identified three major challenges associated with IOM Program ocean activities: the lack of sufficient funding or resources; lack of timeliness/action; and weak commitment from all government departments.

Figure 7: Perceived Challenges of Oceans Activities

Challenges of Oceans Activities

Evaluation Questions #12: To what extent was the IOM Program delivered in an efficient manner? What factors supported/hindered efficiency?

Key Finding: Efficiency of the IOM Program is difficult to assess, due in part to evolutions in the structure and PAA placement of the program over time. Key informants indicate that the program allocation is limited and that efficiencies will be realized over time as the program moves toward implementation. Efficiency is hindered by factors such as delays, and the time and resource investments in collaborative management, among other impediments.

Resources dedicated to the IOM Program across the period under study are difficult to determine on a national basis, given the evolutions in the program during the period under study. The program relies on a mix of A-base and B-base (OAP Phase I, HOTO) funds. Annually, the Oceans Program is allocated internal resources of about $33M or less than five per cent of DFO's budget.

According to most internal key informants, the IOM Program has been delivered in an efficient manner, given the challenges of the collaborative approach. As noted by one key informant, "for little resources, we have tried to move mountains". A number of key informants expect efficiencies will be realized over time as the IOM Program begins to focus on implementation of IOM Plans.

Factors that reportedly hinder efficiency include inefficiencies associated with delays in approval of AOI and IOM Plans, and the "large table" approach to multi-stakeholder collaboration. Collaboration is time-consuming and expensive (e.g., including costs associated with preparation of consultation materials, travel and record keeping), and the breadth of stakeholder engagement was perceived to have slowed advancement of action on integrated oceans management. A more efficient approach offered by many key informants would be to target collaborative efforts more strategically, making a distinction in collaborative processes between those with an interest in a particular oceans management challenge in a given space, from those with a responsibility. Other factors that were noted to hinder efficiency include limited coordination (e.g., across regions and DFO sectors), high costs of monitoring and surveillance in remote locations and absence of well-articulated program objectives or targets.

Evaluation Questions #13: To what extent are the IOM Program roles and responsibilities clear and appropriate?

Key Finding: Interdepartmental roles and responsibilities are defined through interdepartmental committees. While clearly defined (the Terms of References for committees were refreshed in 2008), a common perception was that other federal government departments' commitment to integrated oceans management, particularly at senior levels, is limited. Collaboration within DFO suffers from a persistent disconnect between the "fisheries" and "oceans" mandate of the department more broadly. Collaboration between NHQ and regions was seen to be improving.

Roles and Responsibilities with respect to Other Government Departments, Provincial/Territorial Jurisdictions, Stakeholders

According to program documentation, interdepartmental cooperation occurs through a committee governance structure that was established under the OAP in 2005-06, comprised of 18 departments/agencies with mandates respecting oceans22. DFO serves as the lead department for federal interdepartmental collaboration on oceans. The Terms of Reference for the committees were revised following OAP Phase I.

Most internal key informants agree that the roles and responsibilities are clear within and among federal partner departments and agencies and confirm that interdepartmental cooperation around oceans management is taking place to a greater extent than prior to HOTO. A key recommendation of the OAP evaluation was a more strategic role for the interdepartmental committees. The senior level interdepartmental committees have met sporadically during the period under study and DFO as lead department (though with a 'facilitator' role only) was perceived to have few levers to encourage greater commitment. The challenges of fragmentation of federal responsibility for oceans management (and the lack of Ministerial authority to address fragmentation) have been noted in the literature also, leading some to identify this as a deficiency of the Oceans Act itself23. A common view among internal key informants is that oceans management is seen by other government departments as a DFO issue, and therefore, there is limited responsibility assumed for this issue on the part of other government departments.

In addition, to federal interdepartmental committees, provincial/territorial governments are engaged through the federal/provincial/territorial committee (now disbanded) and Regional Committees on Oceans Management are established at the regional level to advise on local LOMA and MPA processes. As reported previously, committee members were generally satisfied with the way committee roles and responsibilities have been articulated and the breadth of engagement.

Roles and Responsibilities within DFO

As indicated previously, DFO is undergoing a culture shift through a process of evolution away from a single species management approach and to an ecosystem-based approach. Roles and responsibilities within DFO with respect to the IOM Program are influenced by this broader evolution within the organization.

At NHQ more so than the regions, there was perceived to be a persistent disconnect between 'Fisheries' and 'Oceans' within the department, with oceans management widely viewed by internal key informants (as well as external stakeholders familiar with the department) as the lower priority element in DFO's policy agenda. As well, it was noted by a small number of key informants that the IOM Program has been negatively affected by a great deal of turnover at the senior levels. As noted earlier, a recent reorganization within DFO bringing the IOM Program and Fisheries Management within the same sector was viewed as a positive one by several key informants.

DFO is a highly decentralized department with significant personnel located in five administrative regions. With respect to integrated oceans management, NHQ assumes responsibility for policy and allocation of resources, while the regions, through service level agreements, are responsible for delivery. Relationships between NHQ and the regions were characterized as "improving" by some, and benefiting from the implementation of coordinating committees and greater internal communications. Still, some internal key informants felt additional policy guidance and directives would be helpful as regions assume responsibility for implementation of IOM Plans and management of MPAs. The Gully MPA case study suggests that NHQ could draw more effectively from the experience of regions that are learning lessons on the application of integrated oceans management to MPAs and with and within LOMAs.

 Evaluation Question #14: To what extent have investments in science been sufficient and appropriate for IOM Program decision-making and ongoing monitoring and assessment of oceans spaces?

Key Finding: Science has provided significant support to integrated oceans management, though there are resource challenges in meeting the complex needs of ecosystem assessment and management.

Science is a key foundation of decision-making at DFO. According to DFO's science framework for integrated management, "Ecosystem science is needed to inform the department's policies and management practices, and to determine the necessary features of our Science activities.

  • Research should improve our knowledge of key ecosystem relationships and linkages to human activities and be broadly applicable to all departmental responsibilities.
  • Monitoring and data and information management should produce ecosystem-focused products and services of value to all parts of the department.
  • Science advice should be provided in an ecosystem perspective and be integrated across client sectors."24

DFO Science coordinates the scientific advisory process to provide advice and products to support the work of the IOM Program. Departmental requests for science advice are assembled and prioritized at the DM level. DFO Science offers a suite of ocean science products, including ocean observations, data management, modelling and advice, as well as scientific advisory and status reports on stocks, ecosystems and habitats.

Key informants are generally satisfied with the science products and advice available through DFO Science and feel they have contributed to decision-making for integrated oceans management. The products are said to be fundamental to planning and decision making with respect to development of IOM Plans, identification of AOIs, and designation and management of MPAs.

Relationships with DFO Science are said to be strong or improving. Internal key informants indicated that there is increasing recognition of ecosystem science as a department priority, in addition to stock assessment which has been the traditional focus of the department. Where DFO does not have the resources or technical expertise required, regions have commissioned research from external organizations (universities, research institutes) using contractual arrangements.

As mentioned previously, there have been challenges in identifying practical and affordable approaches for Oceans Act MPA management, including the development of appropriate indicators and monitoring protocols and strategies. This is due, in part, to difficulties in defining indicators for the high level conservation objectives that have been identified for Oceans Act MPAs, coupled with the fact that the development and establishment of MPAs is a relatively new process. Limited funds to address research gaps and adapt monitoring and data and information management for ecosystem science was also identified as a challenge. As an example, the case study of the Gully MPA highlighted that the current lack of understanding of the structure and function of the Gully ecosystem has prevented the identification of efficient monitoring indicators of the MPA.

Evaluation Question #15: To what extent has the IOM Program been able to incorporate social, cultural and economic factors into its decision-making processes and governance?

Key Finding: Social, cultural and economic data are being collected within regions, often with the assistance of external expertise. Practical use and incorporation of the information within integrated oceans management processes has proven challenging.

The majority of key informants agree that social, cultural and economic data are being collected with the intention of informing integrated oceans management decision-making. Where DFO does not have the necessary expertise internally – particularly in the area of identifying and considering social and cultural factors – external expertise (e.g., from academia, other government departments) has been retained. For example, external experts were invited to identify culturally significant areas of the Gulf region, and the information has reportedly been useful in preparing regional or strategic environmental assessments.

Still, several key informants noted that incorporation of social, cultural and economic factors in decision-making is a critical challenge, particularly when decision-making is the responsibility of another agency (e.g., Transport Canada or Natural Resources Canada). Other challenges include: lack of consistent social, cultural and economic data across regions or within the LOMA; insufficient policy guidance on how to collect and use data in the decision-making process; and methodological challenges in identifying and assigning priorities, especially relating to cultural values.

Evaluation Question #16: To what extent do the performance monitoring/ measurement processes support decision-making and departmental accountability requirements?

Key Finding: There are few performance measurement and monitoring processes for the IOM Program. The activities of the IOM Program that were funded by HOTO utilize the performance monitoring and measurement processes and tools of this Initiative, which are well-regarded. HOTO performance reporting has proven beneficial to management of the DFO HOTO components and for internal departmental reporting as well.

The intended outcomes of the IOM Program have not been concretely identified and, because IOM Plans have not yet been implemented, are somewhat 'soft' and high level in their nature (e.g., shift in attitude and culture toward collaborative management, ecosystem approaches). The program has not concretely defined targets or timelines for completion of key activities and outputs, or a clear framework to gather performance information. The Gully case study, for example, noted challenges with performance measurement, stating the indicators that are collected for roll-up at the departmental level are at too high a level and not able to account for or recognise the progress being made at the MPA level. Note that the IOM Program is undergoing an internal process currently to design a performance measurement and monitoring framework.

For IOM Program activities funded by HOTO, a Risk Based Audit Framework/Results-based Management and Accountability Framework (RBAF/RMAF) (2007) guides performance measurement. DFO was charged with compiling biannual updates received from each partner department for each component into an annual performance summary report. To support performance reporting, a performance measurement template and a protocol for completion, approvals and transmission were developed. Performance data that are collected includes: linkage to PAA; expenditures; status of projects; evaluation and audit studies; work plan deliverables accomplished during the fiscal year and progress towards the five-year outcome of a HOTO component; and other key indicators to measure progress and indirect results. The completed templates for each component are then summarized by DFO in a one- to two-page table.

Most internal DFO key informants with responsibility for HOTO components hold positive views about the HOTO performance monitoring and measurement processes and tools. There were no significant difficulties with the templates or reporting processes. Additionally, some component leads reported using their HOTO performance reporting to benefit their own internal functions, allowing them to track their progress, remain focussed, document the project for continuity during staff absences or turnover, and make better-informed decisions.

Evaluation Questions #17 and #18: What are the lessons learned in implementing the IOM Program thus far? What are the key priorities for integrated oceans management moving forward?

Key Finding: The IOM Program has promoted an increased focus on integrated oceans management at the federal level. There are many lessons learned that may benefit efforts moving forward to address evolving priorities.

Key informants were asked to describe any lessons learned that have emerged from their experience with the IOM Program and future priorities for the program moving forward. A frequent comment from key informants was that, overall, the IOM Program was important in advancing an ecosystem approach with DFO and providing focus and commitment to integrated oceans management. The weight of opinion was that there should be a continued focus on oceans ecosystems and integrated oceans management. Lessons learned and future priorities that were identified follow.

Evolution of IOM Plans

The development of IOM Plans has been a protracted process – much longer than forecast in the department's 2005-2010 Strategic Plan which targeted five integrated management plans by March 2008. Key informants indicate that, despite the delays, the IOM Program has been beneficial in promoting an understanding of integrated oceans management and in guiding the planning and design of IOM Plans. Stakeholders' involvement in collaborative mechanisms around integrated oceans management has been widespread and collaborative processes have been successful in raising issues and engaging multiple interests. Some internal key informants and stakeholders have been frustrated by the protracted and often cumbersome nature of collaborative processes and delays in approval of IOM Plans in several LOMAs. There are related challenges in maintaining stakeholder interest over time. Finally, some argue that the IOM Plans themselves have been too high level and lacking the kinds of tangible goals to lead to concrete actions.

There was a strong sentiment among some key informants and IOM stakeholders that to achieve IOM Program objectives, DFO needs to shift its current approach. Integrated oceans management planning was perceived to be at a stage of maturity to focus on implementation; that is, operationalizing IOM Plans through the use of tools such as marine spatial planning, regulations, and conservation tools.

The suggested increased focus on IOM Plan implementation, and concrete actions and goals was accompanied by a recommendation on the part of several internal key informants for a more targeted or strategic process of identifying the 'right players to be around the table' for a given integrated oceans management objective or challenge in a given oceans space. For most key informants who discussed this priority, a model that increased engagement of regulatory or decision-makers around the table was a priority (while ensuring that stakeholders have input at appropriate stages of problem solving and have an opportunity to engage with government and share information). Note that several key informants reported that this more targeted engagement process is an evolution that has already begun within DFO. As well, commitment/championing of oceans management among senior managers with authorities respecting oceans was identified as a second condition for achieving more concrete action on integrated oceans management leading to tangible results.

Honouring MPA commitments

The designation of the six new Oceans Act MPAs, an objective of HOTO, is expected to be completed by 2014. Honouring this commitment and meeting Canada's international commitments under the Convention on Biological Diversity with respect to MPA network establishment and protection of the marine environment were identified as high priorities by key informants. However, the implementation of consultation and regulatory processes are remaining tasks for many AOIs, requiring additional investment in the short-term. In the longer-term, science work to refine conservation objectives, and indicators and monitoring protocols for new and existing MPAs is required. Additionally the development of operational strategies and enforcement supports for the management of MPAs is needed. With respect to MPAs, there is limited information on costing models for MPA establishment and management.

The National Framework for Canada's Network of MPAs that is now in place is expected to pave the way for a more coherent and streamlined selection of marine protected areas within a bioregional framework in the future. Still, resources were perceived by some to be inadequate to accomplish the tasks related to network design.

Surveyed IOM Program stakeholders appear to agree with this priority – when asked about their level of support for a National Framework for Canada's Network of MPAs, 61 per cent of surveyed stakeholders supported the framework to a great extent (4 or 5 on a 5-point scale) and another 22 per cent supported a National Framework to a moderate extent (responded 3 on a 5-point scale).

Addressing jurisdictional complexity and issue ownership

The need to address federal departmental and jurisdictional complexities of oceans management was identified as a lesson learned by a number of internal and external key informants. The specificity of mandates of federal departments and agencies with respect to oceans translates into challenges in leadership and authorities to advance any given issue. As mentioned previously, there are 18 federal departments and agencies with mandates related to oceans, and priorities for each department/agency are set internally. Key informants and surveyed stakeholders highlighted the significant leadership/authority required to advance oceans issues within this complex environment. DFO's role as facilitator is, according to some, limited to accomplish this task.

Several key informants urged greater leadership and attention to oceans management at the senior levels of all relevant government departments, with a closer alignment of efforts to the Oceans Act/Oceans Strategy.

Enhancing coordination

There were many key informant comments pertaining to the importance of coordination in the effective and efficient delivery of integrated oceans management. With notable exceptions, coordination has been weak in several areas, leading to inconsistencies and inefficiencies in delivery. Examples that were mentioned by key informants include:

  • MPAs: The development of the National Framework for a national marine protected area network was intended to inform and coordinate the selection of AOIs, however, was approved too late to inform the selection of the Oceans Act AOIs under HOTO. Processes for consultation and selection of MPAs were not well coordinated regionally and did not have central support in the early years. Opportunities for efficiencies across regions (sharing of tools) were lost. The lack of consistency was noted as a source of frustration by several stakeholders.
  • IOM Plans. Similarly, there have been significant regional differences in the development of IOM Plans, such as consultation processes, data gathering, and development of the Plans. While regional tailoring is necessary given the great variety in jurisdictional realities and ecological issues, several key informants urged that greater coherence and efficiencies in these processes could be achieved by common tools, and central policy guidance.
  • Indicator development and monitoring. Several key informants noted the potential for DFO to do better in terms of shared indicator development across the various sectors of the department (e.g., oceans management, habitat, species at risk and sustainable fisheries management). The need for coordination extends to efficient delivery of research and monitoring in priority IOM sites. Some positive program experience demonstrates how opportunistic MPA studies can be added with minimal cost to other cruises and Science missions without detracting from core project objectives.
  • Leveraging the efforts of other departments for Oceans Act MPA surveillance and monitoring will often be critical, especially in the offshore. An example of this is occurring in the Gully MPA. Maritime region Oceans staff have built relationships internally with science and fisheries management and externally with other government departments such as the Department of National Defence (DND), Transport Canada and other stakeholders to help DFO deliver on its mandate. Establishing a government approach has capitalized on aerial surveillance capabilities at Transport Canada and leveraged DND resources to perform fisheries enforcement.

Another priority that was mentioned by key informants included moving away from the LOMA pilots to include all oceans spaces within an integrated oceans management approach. The LOMAs were identified as pilots to test the integrated oceans management approach. While a few key informants noted that the LOMAs have been important in directing funding and focusing attention on specific high risk areas of the oceans, LOMAs appear to be increasingly seen as lacking sensitivity to variations within the LOMA (now differentiated as bioregions) and excluding key oceans spaces that also merit attention.

As well, several key informants noted that within the IOM Program, Oceans Act MPAs have received significant attention as a marine conservation tool (driven at least in part by the HOTO funding which focused on marine protected areas). However, moving forward, these key informants noted the need for flexibility in choosing the appropriate conservation tool or tools to address the highly variable challenges within a given sensitive marine area (which is consistent with the approach articulated in the National Framework and the approach to MPA network establishment).

Key informants' priorities were echoed to a great extent by surveyed IOM partners/stakeholders. Three in ten IOM partners/stakeholders surveyed perceive MPA network management/monitoring to be the most important priority for oceans activities in the future. Establishing clear goals and objectives is a priority for one in five, and establishing partnerships and collaborating with stakeholders is a priority for 17 per cent.

Figure 8: Stakeholder Opinion on Future Priorities for Oceans Activities

Future Priorities for Oceans Activities

In an open-ended question, surveyed IOM partners/stakeholders were asked what role(s) or contribution(s) the federal level should make to achieving progress on these priorities. Half of surveyed IOM partners/stakeholders mentioned a leadership role for the federal government to play in achieving the goals of IOM, as legislated by the Oceans Act. One in five also mentioned the federal government should play a funding role. Other federal contributions mentioned by respondents included: improve engagement with all relevant groups/stakeholders; empower local/regional groups/agencies to contribute to implementation of integrated oceans management; and advance ecosystem science.

22The three federal government interdepartmental committees were described in Section 2.5.

23D. McCrimmon, Critiquing Canada's Ocean's Act: A Review of the 1995-2008 Academic Literature, Prepared for the DFO Maritimes Region, July 2009.

24Fisheries and Oceans Canada, 2007, p. 2, http://www.dfo-mpo.gc.ca/science/publications/ecosystem/index-eng.htm#a3.

4.4 Performance: Economy

Evaluation Questions #19: Is there a more economical way to achieve the objectives of the IOM Program?

Key Finding: A number of suggestions were provided to improve the effectiveness of the IOM Program to achieve intended outcomes. There are a number of recommendations and developments internationally that may be instructive in undertaking program improvements.

The issue of economy concerns whether program outcomes are achieved at least cost and whether there are alternatives to the current program that could achieve the same outcomes at a lower cost. In this section, the evaluation findings presented include key informant opinions and suggestions in the documentary sources on alternative, more cost-effective approaches to achieve IOM Program objectives.

Not an alternative per se, several key informants urged that the IOM Program move forward more proactively to incorporate regulatory approaches into integrated oceans management. As mentioned previously, this approach could take the form of greater engagement of federal departments and provincial agencies with regulatory authority on an issue-and/or place-based basis. Another avenue proposed by a number of internal key informants is for DFO to assume greater regulatory authority; while DFO does not have regulatory authority over oceans uses (other than fisheries), the department does have authority under the Oceans Act to establish marine environmental quality guidelines to protect and manage unique and sensitive ecosystems. According to these key informants, the department could further explore the use of these overarching guidelines to manage oceans uses.

Key informants and case study respondents noted a number of other potential areas for improvement in the efficiency or economy of the program such as: employing electronic means of consultation; streamlining IOM Plan/MPA approval processes; contract out consultations; commission stakeholder groups to conduct some activities with respect to consultations or data gathering; and seek opportunities for stakeholder leveraged funding. The case study of the Gully MPA also proposed improving efficiency by integrating LOMA and MPA advisory committees to reduce the meeting burden, while improving collaboration. The case study also noted the potential efficiency benefits of sharing best practices from the experience of regions in management of MPAs within an integrated oceans management.

There was little in the documentary sources that were reviewed to demonstrate more economical ways to achieve the objectives of the IOM Program. However, Macleod and Leslie (2009)25 argue that a centralized and consistent federal approach may be most economical and effective. Regional planning is important, but this should be orchestrated centrally. The authors further argue for increased efforts to integrate the efforts of multiple sectors to achieve maximum economy of resources in undertaking collaboration, scientific research, and establishing MPAs26. This could include the delegation or contracting of tasks to academic, private and not-for-profit sectors while government maintains a management role. In this vein, key informants also suggested capitalizing on existing hard and soft infrastructure, like the research technology available at research institutes and academies, or volunteer and civil society networks for information sharing and consultation (though this approach would be less practical for remote/offshore marine protected areas).

With respect to marine protected areas, Australia has significantly increased the number of marine reserves and area under protection through the use of regional marine plans. In their assessment of the Australian approach, Stark and Ladell (2008) suggest that "combining MPA network planning with regionally-based marine-use planning is the most efficient way to achieve a network of MPAs that takes into the account the needs of government, multiple user groups and coastal communities"27. Jessen's (2011) review of global best practice in marine protected areas and marine planning notes advances in several areas: the designation of vast new marine protected areas (with significant no-take/high internal protection levels) such as in the US Northwestern Hawaiian Islands and Australian Great Barrier Reef; efforts to implement marine protected area networks (as opposed to marine protected area practice focused on individual sites) within an ecosystem-based management approach and marine spatial planning; and the definition of time-bound public processes to identify and implement marine protected area networks28.

Some authors have pointed to recent developments in the US as being of interest to Canada. In 2009 the US government struck an Interagency Ocean Policy Task Force, and in 2010 a National Ocean Council to provide a national approach to build on existing coastal and marine planning processes. The policy endorses an ecosystem-based approach to marine management, while recommending ways to increase coordination across the multiple agencies and laws pertaining to oceans management29.

Finally, Canadian Parks and Wilderness Society outlines several socio-economic concerns that need to be considered specifically in the establishment of MPAs:

  • Identify community-based initiatives and integrate local knowledge
  • Inventory current uses and activities
  • Identify opportunities for alternative uses / compatible activities (Jessen et al. 2011)30.

25Karen MacLeod and Heather Leslie, "State of Practice," in Ecosystems Based Management for the oceans (Island Press, 2009), http://books.google.ca/books?hl=en&lr=&id=yn4mL6u35tMC&oi=fnd&pg= PA314&dq=healthy+oceans+canada&ots=As0o1RxM7m&sig=fcLWyKzm8NXnDYWHorBYP2bLqAo#v=onepage&q&f=true.

26K. MacLeod and H. Leslie, "State of Practice," in Ecosystems Based Management for the Oceans (Island Press, 2009);
    M. Rutherford, M. Dickinson, and T. Gunton, "An evaluation of the national framework for marine planning in Canada," Environments 37, no. 3 (2010).

27J. Stark and K. Ladell, The International School of Marine Protected Areas Progress Report Card: Canada, Australia and the United States, A Report by Living Oceans Society, the Sierra Club British Columbia, and the David Suzuki Foundation, Vancouver, BC, 2008.

28S. Jessen, A Review of Canada's Implementation of the Oceans Act since 1997 – From Leader to Follower?, Coastal Management, 39:1, 20-56.

29D. McCrimmon, Critiquing Canada's Ocean's Act: A Review of the 1995-2008 Academic Literature, Prepared for the DFO Maritimes Region, July 2009. P. J. Ricketts and L. Hildebrand, "Coastal and Ocean Management in Canada: Progress or Paralysis?," Coastal Management 39, no. 1 (2011): 4-19.

30Jessen, S., et. al., "Science-based Guidelines for Marine Protected Areas and MPA Networks in Canada: An Overview" (Canadian Parks and Wilderness Society, 2011), http://cpaws.org/uploads/mpa_overview.pdf.

5. Conclusions

The findings of the evaluation lead to the following broad conclusions about the relevance and performance (effectiveness, efficiency and economy) of the IOM Program.

5.1 Relevance

  • The evaluation evidence indicates that there is an ongoing need for oceans programming. Canada's oceans have great economic and social significance, yet are showing signs of stress in many areas. The views of key informants, surveyed IOM Program stakeholders, and public opinion data are also supportive of efforts to address oceans management. Integrated management is an approach that has proven effective in terrestrial planning and has been adopted by other countries to manage marine environments.
  • The IOM Program aligns with federal priorities, particularly those related to economic affairs (a clean environment). The IOM Program is a key aspect of the Government of Canada's National Water Strategy through leadership and responsibility for many of the Health of the Oceans Initiative (HOTO) components, and supports complementary federal strategies (e.g., the Federal Sustainable Development Strategy). IOM Program activities supported by HOTO contribute to meeting international commitments related to biological diversity, specifically marine protected areas and marine protected area networks (Convention on Biological Diversity)..
  • The IOM Program aligns with departmental priorities, addressing the strategic outcome of Sustainable Aquatic Ecosystems, as well as contributing to the strategic outcome Economically Prosperous Maritime Sectors and Fisheries by supporting sustainable development of oceans. The program's priority alignment is subject to a persistent tension within DFO between the ecosystem approach and an historical focus on single species management.
  • The Oceans Act and Canada's Oceans Strategy outline the federal mandate in integrated oceans management. The federal level and DFO make an appropriate and important contribution to oceans programming, in collaboration with other levels of government and other stakeholders. There is some sentiment that the department's role could be more fulsome to meet its leadership mandate stated in the Oceans Act for integrated oceans management.

5.2 Performance

Effectiveness

  • Evaluation evidence from all lines of inquiry indicates some progress toward the achievement of the immediate and intermediate outcomes of the IOM Program.
  • The IOM Program has engaged a wide array of stakeholders in integrated oceans management. Surveyed IOM Program partners/stakeholders who were engaged in consultations/committees were generally satisfied with these processes. Impacts were noted in areas such as the inclusion of a broad spectrum of views, but less impact was felt to have occurred in the areas of oceans management decision-making and stakeholder commitment to integrated oceans management.
  • Integrated Oceans Management Plans have not been implemented within the timeframe specified in the Department's strategic plan. To date, only one Integrated Oceans Management Plan has been endorsed by the Department, while Integrated Oceans Management Plans in four other Large Ocean Management Areas are in various stages of completion and approval. Because Integrated Oceans Management Plans are not at a stage of implementation, there have been limited impacts on policies or practices for oceans-related decisions.
  • Progress in establishing Oceans Act Marine Protected Areas and protecting and conserving marine ecosystems is mixed. There has been significant progress in the development of key strategic and operational supports to establish and manage a national network of protected areas. Progress has been slower in establishing new Marine Protected Areas (the Health of the Oceans Initiative goal of six new Oceans Act Marine Protected Areas by 2012 will not be met) and developing indicators and monitoring protocols for the management of existing Marine Protected Areas.
  • The program's ultimate outcome – management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans – is ambitious, achievable only in the longer-term and difficult to attribute the IOM Program's contribution to its achievement. Some question the plausibility of the program's design to achieve this objective.
  • Unintended outcomes of the program were generally few, but mostly positive in nature. Unintended outcomes were largely in the area of beneficial aspects of collaborations. A negative unintended outcome has been increased stakeholder expectations around integrated oceans management, which have not been met under the current program, resulting in a loss of credibility of DFO and scepticism about the federal commitment to integrated oceans management.

Efficiency

  • In general, financial resources allocated to the IOM Program are difficult to track over time due to evolutions in the scope of the program during the study period. However, resources are generally viewed to have been lean for the program. Factors that hinder efficiency include delays, multi-stakeholder consultation processes which are expensive and time-consuming, lack of coordination and absence of well-articulated program objectives and targets.
  • The terms of reference for interdepartmental committees related to oceans management were refreshed for the Health of the Oceans Initiative, and roles and responsibilities of federal departments were viewed as clear. The Deputy Minister and Assistant Deputy Minister committees met infrequently, however, and engagement of senior management and ownership of the oceans issue by other government departments was viewed as limited by some. Within DFO, there is reported persistent tension between oceans management and fisheries management, reflecting the department's dual role in these areas. NHQ and regional collaboration were felt to be improving, with continued attention to coordination and communication required.
  • Science has provided significant support to integrated oceans management, though resource challenges remain in the complex area of ecosystem science. Social, cultural and economic data are being collected within regions, often with the assistance of external expertise. Practical use and incorporation of the information within integrated oceans management processes has proven challenging.
  • The IOM Program is currently developing performance measurement and monitoring tools. However, during the period under study, performance measurement occurred informally, as well as formally for activities funded through the Health of the Oceans Initiative. Component leads are generally satisfied with the requirements for monitoring and reporting under the Health of the Oceans Initiative, and the performance measurement process proved useful for internal management and reporting functions.
  • The IOM Program has supported an increased federal focus on integrated oceans management. There are many lessons learned that may benefit efforts moving forward to address evolving priorities. Priority areas identified in the evaluation included: moving Integrated Oceans Management Plans toward implementation; honouring commitments on Oceans Act Marine Protected Area designation and marine protected area networks, and addressing Marine Protected Area management challenges; and improving coordination.

Economy

  • A number of suggestions were provided to improve the effectiveness of the IOM Program to achieve intended outcomes. There are a number of recommendations and developments internationally that may be instructive in undertaking program improvements.

5.3 Recommendations

1) Integrated Oceans Management Plans have not been implemented within the timeframe specified in the Department's strategic plan. To date, only one Integrated Oceans Management Plan has been endorsed by the Department. In order to demonstrate results, the Program needs well-articulated program objectives with an appropriate performance measurement strategy. A new draft performance measurement strategy, inclusive of Science and Policy components, has recently been developed.

Recommendation:
In order to regularly monitor and demonstrate results, the Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should establish and communicate to staff and stakeholders clear and achievable short, medium and long-term Integrated Oceans Management  objectives aligned with the recently developed draft performance measurement strategy.

2) Improvement is needed in the areas of oceans management decision-making, stakeholder commitment to integrated oceans management and efficient committee processes. Although collaboration is improving, continued attention to coordination and communication is required.

Recommendation:
The Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should take an active leadership role in increasing the engagement of senior management from other government departments to advance achievement of Integrated Oceans Management commitments.

3) The Integrated Oceans Management Program has engaged a wide array of stakeholders in integrated oceans management. Consultations and committees were successful in encompassing a broad spectrum of views. However, in order to gain efficiencies and effectiveness, it is advisable to streamline stakeholder participation in the consultation processes. 

Recommendation:
The Senior Assistant Deputy Minister, Ecosystems and Fisheries Management Sector, in collaboration with the Assistant Deputy Minister, Program Policy, should review and streamline existing collaborative processes to enhance efficiencies and effectiveness of stakeholder participation.

IOM Program Evaluation Matrix

The IOM Program Evaluation Matrix
Issue Indicators Document Review Literature Review Key Informant Interviews Expert interviews Case Studies Online Survey
Senior Managers, Staff Stakeholders/ partners
Relevance
1.1 Is there a continued need for the IOM Program? Evidence of environmental/societal/economic need and emerging issues with respect to integrated oceans management
Perspectives of key informants, experts and others on the continued need for the IOM Program
X X X X X   X
1.2 To what extent are the objectives of the IOM Program aligned with departmental and government-wide priorities? Degree of alignment of Program with:
Government of Canada objectives and priorities
DFO objectives, priorities and strategic outcome (PAA linkage)
IOM Program objectives, vision and mission
X   X        
1.3 Is the current role of the federal government appropriate in the delivery of the IOM Program?  Program objectives consistent with federal jurisdiction
Perceived importance/appropriateness of federal government role in integrated oceans management
Perceived need for federal leadership in ensuring integrated oceans management.
X X X X X   X
2.0 Performance: Effectiveness
Immediate Outcome:
2.1 To what extent have other government departments/agencies and stakeholders been engaged in integrated oceans management?
Status and perceived effectiveness of governance structures and advisory processes
Number and types of stakeholders involved, perceived effectiveness of engagement (communication, transparency, outreach)
Evidence of mechanisms for exchanges, sharing of expertise and best practices
Nature of communications products/events – take-up/stakeholder participation/assessed effectiveness
X   X X   X X
Immediate Outcome:
2.2 To what extent have Integrated Oceans Management Plans been implemented?
Number/status of Integrated Oceans Management Plans X   X     X  
Immediate Outcome:
2.3 To what extent have Marine Protected Areas been designated?
Number of AOIs progressing through the MPA establishment framework
Number of Marine Protected Areas designated
Number or percentage of Marine Protected Areas with management plans in place or in development
Percentage or number of Marine Protected Areas with conservation objectives identified with monitoring protocols or plans in place or in development
Percentage or number of Marine Protected Areas with enforcement and compliance strategies in place or in development
Progress through the 8 step planning process for establishing bioregional networks of MPAs
Percentage of marine planning processes with conservation objectives with monitoring protocols in place
Observations of key informants on advancement of a national network of marine protected areas
X   X     X  
Intermediate Outcome:
2.4 To what extent are Integrated Oceans Management Plans being considered in oceans-related decisions?
Policies and practices of stakeholder groups/ regulators consider IOM Program
Observations of key informants, experts, survey respondents on influence of Integrated Oceans Management Plans
Illustrative examples of the how Integrated Oceans Management Plans are being considered in oceans-related decisions
X   X X X X X
Immediate Outcome:
2.5 To what extent are marine ecosystems being protected and conserved?
Number of MPAs designated/established
Policies and practices of stakeholder groups/ regulators consider marine conservation objectives/guidelines
Mechanisms in place to monitor compliance with conservation objectives
X   X X X X X
Ultimate Outcome:
2.6 To what extent has the IOM Program made progress toward the management of oceans activities in a way that preserves the ecological health of the oceans, while allowing for sustainable use of the oceans?
Views of key informants, experts and stakeholders on the potential of IOM Program to contribute to managing oceans activities such that the health of the oceans is preserved while sustainable use of the oceans is allowed
Perceived validity of program's theory
    X X X X  
2.7 What, if any, unexpected results have occurred as a result of the IOM Program? (positive or negative) Indirect or "spin off" results not associated with the original IOM Program intended outcomes (e.g., of collaborative mechanisms, of science investments)     X X X X  
3. Efficiency
3.1 To what extent was the IOM Program implemented as designed? Comparison of planned versus actual implementation
Challenges in implementation as reported in HOTO Performance Monitoring Reports
Perceived factors internal or external to IOM Program that affected implementation of the IOM Program
X   X X X X X
3.2 To what extent was the IOM Program delivered in an efficient manner? What factors supported/hindered efficiency? Views of key informants and experts on the extent to which activities are completed efficiently; suggestions for how to improve efficiency of the delivery of activities.
Strengths and weaknesses of delivery approach with respect to efficiency
Output costs:
  • Planned versus actual
  • Analysis in light of program theory and context
  • Analysis in light of quality/quantity/timeliness/appropriateness of outputs
X   X     X  
3.3 To what extent are roles and responsibilities clear and appropriate:
among DFO, other federal departments/agencies, other jurisdictions and stakeholders involved in oceans governance;
among branches within DFO that participate in the IOM Program;
between headquarters and regions delivering the IOM Program
Nature and appropriateness of involvement of departments/ agencies, jurisdictions, stakeholders
Program adaptations to jurisdictional realities
Perceived clarity of the stated roles and responsibilities, effectiveness of communication
Recommendations from OAP evaluation with respect to governance implemented
Cooperation around integrated oceans management between Headquarters and the regions
X   X X      
3.4 To what extent have science products and advice been sufficient and appropriate for IOM Program decision-making and ongoing monitoring and assessment of ocean spaces? Deliverables against workplans
Observation of key informants
X   X        
3.5 To what extent has the IOM Program been able to incorporate social, cultural and economic factors into its decision-making processes and governance? Status/usefulness of socio-economic assessment reports
Observations of key informants on DFO capacity
X   X        
3.6 To what extent do the performance monitoring/ measurement processes support decision-making and departmental accountability requirements? Current performance monitoring strategy; suggested improvements from OAP evaluation implemented
Perceived degree to which current reporting requirements elicit an accurate representation of actual performance
Perceived usefulness of performance data
Suggestions for improvement
X   X     X  
3.7 What are the lessons learned in implementing the IOM Program thus far? Strengths/benefits of the program
Reported challenges encountered in implementing IOM Program, as well as mitigation strategies,
Suggestions for improvement
X X X X X X X
3.8 What are the key priorities for integrated oceans management moving forward? Extent to which key thrusts of the program – integrated oceans management, ecosystem assessments, marine conservation tools – is the best way to achieve the program's intended outcomes
Perceived future priorities in terms of science investments, governance/education, priority principles for decision-making, etc.
  X X X X X X
4. Economy
4.1 Is there a more economical way to achieve the objectives of the IOM Program? Presence of comparable programs/alternative approaches
Possible (more economical) alternatives
Views of key informants and experts
X X X X X    

IOM Program Management Action Plan

IOM Program Management Action Plan Recommendations
Recommendations

Recommendation 1:
IOM Plans have not been implemented within the timeframe specified in the Department's strategic plan. To date, only one IOM Plan has been endorsed by the Department. In order to demonstrate results, the Program needs well-articulated program objectives with an appropriate performance measurement strategy. A new draft performance measurement strategy, inclusive of Science and Policy components, has recently been developed.

Recommendation:
In order to regularly monitor and demonstrate results, the Senior ADM, Ecosystems and Fisheries Management Sector, in collaboration with the ADM, Program Policy, should establish and communicate to staff and stakeholders clear and achievable short, medium and long-term IOM objectives aligned with the recently developed draft performance measurement strategy.

Strategy

The Program's response to all three Recommendations has been formulated at a time of program renewal and reorganization.  The management actions being proposed are consistent with the Oceans Path Forward, a visioning and renewal exercise and plan designed to refocus the Oceans Program on strategic priorities for targeted Integrated Oceans Management over the next five years.  The Management Action Plan is also consistent with immediate objectives that are focused on those valuable and vulnerable ecosystems that may be under pressure or at risk and the application of regulatory authorities within the Minister of Fisheries and Oceans' purview, where necessary, to maintain sustainable aquatic ecosystems that, in turn, support prosperous maritime sectors and fisheries.

This refocusing links well with the theme of each of the three recommendations: i) clarify IOM objectives; ii) streamline and strengthen interdepartmental collaboration and governance; and iii) strategically target stakeholder engagement to correspond to selected oceans management challenges. In general, the actions proposed are consistent with a more strategic, targeted approach in an environment of limited resources and large challenges - an approach that must be founded on internal alignment within DFO, and close collaboration on selected initiatives with federal and other partners.

Management Actions Actions Completed Actions Outstanding Target Date Supporting Evidence
1. Develop objectives for adaptive oceans management.   Improve internal alignment of effort by tabling IOM objectives annually at DFO planning fora (SOC, DMPC, etc). One annual meeting each year before March 31.        
  Convene annual IOM and MPA practitioner workshops to establish and refine short, medium and long-term adaptive management objectives to recommend to Senior Program Management for Oceans.      
2. Develop protocol and tools for monitoring progress against IOM objectives   Develop a monitoring and reporting plan for measuring progress toward IOM objectives, including timelines, and identification of roles and responsibilities.   March 31, 2013         
  Develop monitoring tools, such as a template, to facilitate IOM performance measurement (as was done to good result in support of HOTO monitoring and reporting. March 31, 2013    
  Complete "Phase 2" of the "new" Oceans program Performance measurement Strategy, including identification of additional performance indicators and data sources.   March 31, 2013    
  Link the IOM monitoring and reporting protocol to the departmental business cycle (eg, one-pass planning process, risk management, DPR & RPP). March 31, 2013    
3. Develop an approval process for IOM Plans.   Review approval process of IOM Plans to date to identify lessons learned.   July 31, 2012      
  Develop options for a streamlined approval process, with clearly defined accountabilities, for DM approval. March 31, 2013  
4. Communicate IOM results to Oceans stakeholders.   Add an annual highlights page to the Oceans section of the DFO website to communicate incremental IOM results.   Sept 30, 2012    
  Incorporate IOM maps (EBSAs, Marine Protected Areas, Large Ocean Management Areas, etc.) into a newly-developed, online  DFO maps portal     June 30, 2012  
Recommendations
Recommendations
Recommendation 2:
Improvement is needed in the areas of oceans management decision-making, stakeholder commitment to integrated oceans management and efficient committee processes. Although collaboration is improving, continued attention to coordination and communication is required. Recommendation:
The Senior ADM, Ecosystems and Fisheries Management Sector, in collaboration with the ADM, Program Policy, should take an active leadership role in increasing the engagement of senior management from other government departments to advance achievement of IOM commitments.
Strategy
See Strategy Statement under Recommendation 1, above.
Management Actions Actions Completed Actions Outstanding Target Date Supporting Evidence
1. Renew the Interdepartmental Committees on Oceans (ICOs)   Renewal of DGICO and ADMICO TORs, with input from and approval by committee members. Sept. 30, 2012    
  Create a "Forward Agenda" for ICOs to enhance capacity to coordinate oceans management efforts with federal partners. Sept. 30, 2012    
  Strengthen the relationship between the ICOs and the RCCOMs to facilitate national-regional interchange in planning efforts.   Dec. 31, 2012  
Recommendations
Recommendation 3:
The IOM Program has engaged a wide array of stakeholders in integrated oceans management. Consultations and committees were successful in encompassing a broad spectrum of views. However, in order to gain efficiencies and effectiveness, it is advisable to streamline stakeholder participation in the consultation processes. Recommendation:
The Senior ADM, Ecosystems and Fisheries Management Sector, in collaboration with the ADM, Program Policy, should review and streamline existing collaborative processes to enhance efficiencies and effectiveness of stakeholder participation.
Strategy
See Strategy Statement under Recommendation 1, above.
Management Actions Actions Completed Actions Outstanding Target Date Supporting Evidence
1. Develop a risk-based management framework.   Develop a risk management framework to identify priority areas and to engage necessary stakeholders on targeted oceans management issues.   Sept. 30, 2012        
2. Develop improved mechanisms and approaches to targeted engagement of Provinces, Territories, and First Nations.   Review inter-jurisdictional engagement efforts to date, and develop options for improved engagement processes targeting at specific, strategic oceans management issues. Dec. 31, 2012