An online survey was conducted with HOTO partners and stakeholders. Partners and stakeholders included participants in committees and consultations related to HOTO, including: federal/provincial/ territorial representatives; industry representatives such as fishing, shipping, aquaculture, oil and gas; First Nations and Inuit representatives; non-governmental organizations and community groups, scientific/academic/educational organizations; and other stakeholders.

Based on lists of stakeholders assembled by HOTO partner departments, 118 individuals were invited to complete the survey and had valid email addresses. In total, 37 individuals completed the survey for a response rate of 31 per cent. The survey was launched in October 2011 and closed in November 2011.

Note that survey respondents are representative of the DFO-led HOTO components. DFO provided the most extensive list of stakeholders and the response rate for these stakeholders was higher compared to other departments/agency. PCA did not submit stakeholder lists as their HOTO work has involved internal partners only thus far.

3.4.4   Case Studies

Four case studies were completed of selected HOTO components:

• Federal Marine Protected Areas Strategy (HOTO Components #1, #6, #13, aspects of #14). Protecting sensitive marine areas is a key intended outcome of HOTO. Three departments/agencies (EC, PCA and DFO) received HOTO funding to collaborate to achieve a more systematic approach to marine protected area planning and establishment.
• Arctic Marine Shipping Assessment (HOTO Components #5 and #11). The Arctic Marine Shipping Assessment was co-led by Canada, Finland and the United States from the Arctic Council, Protection of the Arctic Marine Environment Working Group. These HOTO components, led by TC and AANDC, provided an assessment of current and future marine shipping activities in the Arctic, and examined the potential impacts of shipping on indigenous communities and the environment. Included in this funding but separate from the Arctic Marine Shipping Assessment was an update to the Arctic Council’s Regional Programme of Action for the Protection of the Arctic Marine Environment from Land-based Activities. This update was intended to inform policy and decision-makers in Arctic Council states and the marine community in general.
• Enforcement of Ballast Water Regulation (HOTO Component #8). TC’s Ballast Water Control and Management Regulations (2006) require all ships coming from outside Canada (other than nearby American ports) to exchange their ballast water at sea or treat it if it is to be discharged in Canada to minimize the threat of invasive species from this source. This component supports enforcement of regulations (i.e., dedicated inspectors to the Ballast Water Enforcement Program, training and guidelines and testing). Collaboration with industry and various national and international authorities is a key component of enforcement.
• Gulf of Maine (HOTO Component #4). This component, led by EC, enabled more active EC participation with other Canadian federal and provincial departments, as well as with US federal and state departments, academic institutions and non-government and business interests in the Gulf of Maine Council on the Marine Environment. HOTO funding supports implementation of the Canada-US 5-year action plan including regional strategies for adaptation to climate change, indicator development, education and engagement and joint monitoring for toxic contaminants in the transboundary ecosystem.

3.5  Methodological Limitations

The strength of the methodological strategy for the HOTO evaluation is a multiple lines of evidence approach that includes both qualitative and quantitative lines of evidence, and the views of both internal and external stakeholders. Study limitations include:

• HOTO is a collection of 22 quite different and often discrete components, led by five federal partner departments/agency. Challenges were encountered in addressing evaluation questions in terms of HOTO overall (e.g., the relevance of HOTO, achievement of the direct outcomes of HOTO), as well as assessing the components in detail given the breadth of these efforts. The evaluation collected information at the level of the components, with later analyses conducted to synthesize the findings to the intermediate outcome level. The evaluation yielded lessons learned at the Initiative level, and not the component level.
• It should be noted that the outcomes that have been identified for some of the HOTO components are more aptly characterized as outputs only (e.g., Dash 7 retrofit, distribution of emergency spill response kits) and indicators related to achievement of intermediate outcomes as laid out in the Results-Based Management and Accountability Framework and Risk-based Audit Framework tend to closely parallel immediate outcomes. Therefore, there is some variability in the extent to which the components are amenable for analysis at the level of immediate or intermediate outcomes.
• While multiple lines of evidence were used for the evaluation, each of the methodologies are not equally appropriate for assessment of each of the priority needs areas. For example, the sample size for the survey of HOTO partners/stakeholders is quite small and largely represents those with some involvement in marine protected area establishment. Therefore, assessing achievements in the area of pollution control relied more heavily on other sources of evidence (e.g., documentary evidence, case studies).

Other limitations of the individual lines of evidence include:

• Review of documentation and literature: To bolster some of the study findings, external literature was sought regarding health of the oceans and Canada’s progress on establishment of marine protected areas. Peer-reviewed literature, while preferable, could not be thoroughly accessed and synthesized with the time and resources available for the study
• Key informant interviews: Key informants contacted for interviews represent a cross-section of internal and external stakeholders, and the five partner departments/agency. Priority candidates could not be reached in all cases. In addition, conducting the HOTO and IOM Program evaluations simultaneously proved challenging. There is significant overlap between the DFO HOTO components and the IOM Program. Conducting the evaluations simultaneously provided efficiencies in terms of use of evaluation resources. However, for DFO internal key informants, the interviews were quite lengthy and there was limited time to gather detailed responses for both programs.
• Survey of stakeholders: The survey of stakeholders used industry standard methods for survey administration, including two email follow-up reminders and online and telephone assistance to support respondents. In addition, telephone reminders were conducted for stakeholder groups where the responses were particularly low (TC and AANDC). The response rate to the survey was low – just over 30 per cent – and given the finite initial pool of potential respondents, the final sample size for the survey is small. Representativeness of the sample to the population of HOTO stakeholders is difficult to assess given the absence of known characteristics of the population.
• Case studies: The four HOTO case studies were selected to represent the three priority needs areas within the Initiative. Given the diversity of HOTO activity, case study findings are illustrative of HOTO outcomes and cannot be generalized to other components.

The limitations identified above are common ones for program evaluations of broad environmental programs. Like other studies, this evaluation uses multiple lines of evidence to address the weaknesses of individual methodologies. This process of triangulation strengthens the evidence to answer the evaluation questions. Still, the HOTO ultimate outcome of health of the oceans is of a long-term nature, and difficult to measure and clearly attribute the impact of the HOTO components specifically.

⁵Note that there was a discrepancy in the HOTO documentation around the intended ultimate outcome of the HOTO Initiative, referred to in some documentation as “keeping coastal waters clean”. The Results-based Management and Accountability Framework identifies the ultimate outcome of the initiative as “health of the oceans”.

4.  Major Findings

4.1  Relevance

 Evaluation Question #1 : Is there a continued need (environmental, societal) for HOTO?

A review of documentation supports the continued need to address the health of the oceans in Canada. Evidence indicates newly emerging signs of stress on marine environments caused by human and environmental activity.⁶ Pollution, invasive species, oil drilling, and overfishing are just a sampling of the issues affecting the biodiversity and sustainability of Canada’s oceans.⁷ While concern over many of these issues has existed for some time, there continue to be activities that degrade habitat and damage ecosystems.⁸Many expect human activities in the ocean such as tourism, industrial processes, and shipping to intensify in the future,⁹ particularly in Arctic areas as ice sheets recede.¹⁰

With respect to continued societal need, it has been estimated that the economic contribution of ocean sector activities represented an estimated $17.7 billion in direct gross domestic product in Canada in 2006 and over 171,000 people are employed in related industries.¹¹ In addition, there are cultural and social benefits of oceans to the one in five Canadians who live in a coastline community and, more broadly, to all Canadians.

Evaluation key informants across all respondent categories agreed that there is a continued need for attention to the health of the oceans. Key informants cited declining sea ice in the Arctic, habitat degradation, depleted fish stocks and threats due to climate change and invasive aquatic species. Many key informants further noted the economic and societal significance of oceans, anticipating a trend toward increased use of oceans (for example, oil and gas exploration and development, offshore mining, aquaculture). Public opinion generally echoes key informant views. General awareness of environmental issues affecting the oceans is high and Canadians expect the government to show leadership in this area.¹²

Three-quarters or more of surveyed HOTO stakeholders indicate that there is a continued need for federal attention in all of the key themes under HOTO – science, marine protected areas, engagement of stakeholders, and pollution prevention surveillance and control.

⁶ P. J. Ricketts and L. Hildebrand, “Coastal and Ocean Management in Canada: Progress or Paralysis?,” Coastal Management 39, no. 1 (2011): 4-19. International Programme on the State of the Ocean, Implementing the Global State of the Oceans Report (IPSO, 2011), http://www.stateoftheocean.org/research.cfm. .

⁷J.L. Smith, K. Lewis, and J. Laughren, A Policy and Planning Framework for Marine Protected Area Networks in Canada’s Oceans, Halifax: World Wildlife Fund, 2006

⁸P. L. Stewart and B.T.L. White, “A Review of Contaminants on the Scotian Shelf and in Adjacent Coastal Waters: 1970-1995.,” Canadian Technical Report of Fisheries and Aquatic Sciences 2351 (2001): xviii +158 pp.

⁹Molnar and Koshure, Cleaning up Our Ocean. David Suzuki Foundation, Sierra Club of British Columbia, Living Oceans Society.

¹⁰International Programme on the State of the Ocean, Implementing the Global State of the Oceans Report.

¹¹http://www.dfo-mpo.gc.ca/ea-ae/cat1/no1-1/no1-1-eng.htm

¹²Department of Fisheries and Oceans Canada. (2005). Canada’s Federal Marine Protected Areas Strategy (p. 18). Retrieved from
http://www.dfo- mpo.gc.ca/oceans/publications/fedmpa-zpmfed/pdf/mpa-eng.pdf
    McAllister Opinion Research. (December 16, 2008). “BC opinion poll finds widespread concern over health of oceans.” Canada Newswire. http://www.newswire.ca/en/story/232915/bc-opinion-poll-finds-widespread-concern-over-health-of-oceans

Figure 1: Stakeholder Opinion on Continued Need for Federal Action

While documentary sources and key informant and stakeholder opinion confirm the continued environmental and societal need for attention to health of the oceans, the continued need for each individual HOTO component is less certain. Evaluation findings regarding the priorities for addressing health of the oceans moving forward are discussed in more detail in Section 4.3.

Evaluation Question #2: To what extent are the objectives of HOTO aligned with departmental/agency and government-wide priorities?

Federal priorities

The Oceans Act includes provisions for many of the activities undertaken through HOTO, including the establishment of marine protected areas. The federal Oceans Strategy includes, among its four pillars, the health of the oceans.

According to program documents and key informant respondents, HOTO is aligned with federal government priorities pertaining both to conservation and economic prosperity, particularly the Government of Canada’s Economic Affairs Outcome Area, a clean and healthy environment. The 2007 Throne Speech committed the Government of Canada to implement a new water strategy to help clean up major lakes and oceans. Budget 2007 created the National Water Strategy, which allocated $105.3 million in funds to improve the quality of water in Canada’s rivers, lakes and oceans, including funding for HOTO.

Also notable, Canada has broad commitments and obligations under international agreements that confer a domestic responsibility for action on oceans, such as those set out in the United Nations Convention on the Law of the Sea (UNCLOS). More specifically, a global commitment to establish a network of marine protected areas was made at the 2002 World Summit on Sustainable Development, and the 2004 United Nations Convention on Biological Diversity Program of Work on Protected Areas. The global biodiversity targets in the Convention on Biological Diversity Strategic Plan for Biodiversity that was adopted at the 10th Convention of the Parties in 2010 call for 10 per cent of coastal and marine areas being protected by 2017. The Strategic Plan for North American Cooperation in the Conservation of Biodiversity is a shared commitment of the three North America Free Trade Agreement state members and, the North American Marine Protected Areas Network (NAMPAN) is nested under this Plan and is guided by the following goal: “Promote the cooperation for the conservation and maintenance of North American regions of ecological significance.” As well, Canada has obligations under the MARPOL Convention. The MARPOL Convention is the main international convention covering prevention of pollution of the marine environment by ships from operational or accidental causes. Finally, the International Convention for the Control and Management of Ships' Ballast Water and Sediments Ballast Water Convention to which Canada has committed will also soon come into effect.¹³

Documents and key informant respondents also indicate that funded HOTO activities support other federal strategies, including the Arctic Agenda¹⁴ and the Federal Sustainable Development Strategy for Canada among others.¹⁵

Departmental/agency priorities

HOTO partner departments’/agency’s PAAs serve to situate the HOTO components within departments’ program activities leading to strategic outcomes. The following table is a summary of partner departments’ components and priority alignment.

¹³The treaty will enter into force twelve months after the date on which not less than thirty States, constituting not less than thirty-five percent of the gross tonnage of the world’s merchant shipping, have ratified the Convention. To date, 28 States have ratified the BWM Convention, adopted in February 2004, representing about 25% of the world’s merchant shipping.

¹⁴Most recently, the 2011 Speech from the Throne indicated the current government’s priority of protecting Northern waters.

¹⁵Priorities for Environmental Sustainability (Priority III: Protecting Nature; Goal 6; Ecosystem/Habitat Conservation and Protection; 6.3 Target: Marine Ecosystems – Improve the conservation of ocean areas and marine ecosystems by 2012).

Evaluation Question #3: Is the current role of the federal government appropriate in the delivery of HOTO?

The Oceans Act provides a framework for current and future oceans management initiatives in Canada, calling for the Minister of DFO to lead and facilitate the development of a national oceans management strategy, among other responsibilities. HOTO has entailed the involvement of five federal departments and agencies with jurisdiction over specific facets that are implicated in HOTO activities. The health of the oceans is not a federal responsibility alone, however. Many initiatives involve provincial, territorial, and municipal governments, Aboriginal groups and a wide range of other stakeholders.

Key informants agree that the federal government is playing an appropriate role in oceans management. The federal level was described as playing an important leadership role in oceans management due to its legislative authority, as well as responsibility for transboundary waters (e.g., Gulf of Maine) and international commitments. The federal government’s focus on developing policies, structures and regulations was viewed as an important one to provide long term mechanisms to protect the oceans. A number of areas where key informants thought the current federal role was insufficient or could be enhanced were identified and are discussed in more detail in Section 4.3.

4.2. Performance

Evaluation Question #4: To what extent have HOTO’s intended direct outcomes, as identified by the five Partner Departments, been accomplished?

According to program performance data and program key informants, the immediate outcomes of the HOTO components – the component objectives as articulated by the component lead departments – have been accomplished/are nearing completion, or notable progress is evident. Several HOTO components funded one-time capital investments (Component #10 – the Dash 7 outfitting), supported operational requirements (Component #8 – ballast water regulations enforcement, Component #9 – pollution prevention surveillance) or had very clear tangible outputs (Component #22 – emergency spill response equipment). These components achieved their immediate objectives within the HOTO timeframe and with few challenges.
Deviations in achieving intended direct outcomes for some HOTO components had to do with delays in implementation due to factors often outside departmental/agency control (e.g., requirement for provincial/territorial supports for Areas of Interest prior to Ministerial approval, delays in Ministerial approval of Areas of Interest, organizational issues/policy shifts within external partner organizations (e.g., in the case of Component #20 Canadian Environmental Assessment Act linkages). HOTO components pertaining to marine protected areas have largely met objectives with respect to collaboration, planning and network development (Components #1, #6, #13 and #14). The identification of indicators and the development of monitoring protocols for Oceans Act MPAs has advanced, but been completed for only a couple of these areas owing to difficulties in identifying practical indicators for broadly-defined conservation objectives. The establishment of new marine protected areas has lagged.

Evaluation Question #5: To what extent has HOTO met the high priority needs for marine protected area establishment?

Marine protected area establishment was a significant priority under HOTO – ten^16-a of the 22 HOTO components focused on this area, representing an investment of about $29 million or about 47 per cent of the HOTO allocation. Given their unique mandates in establishment of marine protected areas, DFO, EC and PCA were variously identified as leads for these components. DFO, EC and PCA were funded to implement the Federal Marine Protected Area Strategy and make progress toward a Federal Network of Marine Protected Areas. Prior to HOTO, the three departments/agency had begun work to coordinate their marine protected area establishment activities, and HOTO funding enabled building upon these efforts. The Strategy responds to the need for cooperation and collaborative approaches among the three federal departments/agency with marine protected area mandates. The coordinated approach also led to a common education and engagement strategy for marine protected areas. The case study of the component underlined the importance of the Strategy in ensuring a coordinated approach in engaging the provinces and territories around marine protected areas.

DFO received funding (under Component #14) to advance coordination of a national (federal-provincial-territorial) network of marine protected areas. DFO Science was funded (under Component #21) to collect baseline information to help inform the identification of new Areas of Interest for Oceans Act MPAs, and develop indicators and monitoring protocols for the management and evaluation of existing Oceans Act MPAs. Finally, a key HOTO objective was to establish or assess the feasibility of nine new marine protected areas:

• Marine Wildlife Area at Scott Islands (EC lead);
• National Wildlife Area at Sable Island (EC lead) (eventually became a terrestrial national park reserve under PCA)
• National Marine Conservation Area in Lancaster Sound (PCA lead) (feasibility assessment leading to establishment);
• Establishment of six Oceans Act MPAs (DFO lead).

The evaluation indicates that, through HOTO, a considerable amount of the necessary groundwork to establish and implement a national marine protected area network and implement the Federal Marine Protected Areas Strategy has been accomplished (Components #1, #6, #13, and #14). HOTO provided the funding to increase organizational capacity in the three departments/agencies. A multi-departmental Federal Marine Protected Areas Strategy Working Group (DFO, EC, PCA) was formed to implement the Strategy. Two key guidance documents were developed, the Federal Guide for Collaborative Planning of Marine Protected Areas (2009) and an overarching, multi-jurisdictional National Framework for Canada’s Network of Marine Protected Areas (2011). Drafted by the federal-provincial-territorial Technical Experts Committee, the National Framework outlines: the proposed overarching vision and goals of the national MPA network; identifies the network components, design properties and eligibility criteria for which areas will contribute to the network; describes the proposed network governance structure; and provides the direction necessary to promote national consistency in bioregional network planning. To support marine protected areas establishment, a number of outreach activities were undertaken (workshops, community events) and communications products/supports developed (Oceans Portal, Spotlight on Marine Protected Areas in Canada (2010), Emerald Sea (Pacific Coast) video).

Ecosystem science support and advice (Component #21) provided input into the scientific underpinning for the delivery of many oceans related activities. Ecosystem science provided advice and information in support of the selection of seven Areas of Interest for Oceans Act MPAs, and the development of ecological assessments, as well as advice on biogeographic units to guide marine protected area network planning. Examples of DFO Science guidance documents developed through the Canadian Science Advisory Secretariat process include: Development of a Framework and Principles for the Biogeographic Classification of Canadian Marine Areas (2009) and Science Guidance on the Development of Networks of Marine Protected Areas (MPAs) (2009).^16-b During the current fiscal year, DFO Science will prepare a guidance document on “Representative” Marine Protected Areas for Network Planning.¹⁷

Ecosystem Science also made progress in the identification of indicators and the development of monitoring strategies and protocols that would be used for assessing the effectiveness of Oceans Act MPAs in achieving their conservation objectives. This work has advanced, albeit somewhat slowly due to difficulties in determining indicators for broad conservation objectives that have been set out for some Oceans Act MPAs. As of October 2011, five of eight existing Oceans Act MPAs have indicators and monitoring protocols developed (Eastport, Gilbert Bay, Basin Head, Gully and Musquash Estuary). Indicators have been identified for Tarium Niryutait and the development of monitoring protocols is under way. The development of indicators and monitoring protocols for the remaining two marine protected areas (Bowie Seamount and Endeavour Hydrothermal Vents) are at varying stages of scientific study.

HOTO intended outcomes with respect to marine protected area establishment (Components #2, #7 and #19) will not be achieved within the Initiative’s five-year timeframe. With respect to progress during the HOTO Initiative, the establishment of the Scott Islands as Canada’s first Marine Wildlife Area is in the consultation phase and a regulatory strategy has been drafted. A federal-provincial agreement was reached to designate Sable Island a National Park Reserve, rather than a marine protected area. The feasibility assessment for the proposed national marine conservation area in Lancaster Sound was delayed initially to address other outstanding issues in the area, but is now progressing with consultations. The Government of Canada announced its position on a proposed boundary for a national marine conservation area in Lancaster Sound in December 2010, as a starting point for consultations, and also announced that no exploration for petroleum resources would take place within that area. Updated assessments of the region's ecological values and energy potential have been undertaken.

With respect to Oceans Act MPAs, six Areas of Interest have been approved and announced (Hecate Strait-Queen Charlotte Sound Glass Sponge Reefs, Laurentian Channel, Darnley Bay, Shediac Valley, American Bank and St Anns Bank). ¹⁸ Because many of the announced Areas of Interest are not far enough along in the MPA establishment process, the HOTO timeline will not be achieved, although all Areas of Interest are expected to be designated by 2014. The following table provides an overview of the status of the six Oceans Act Areas of Interest as of October 2011.

With respect to Canada’s international commitments toward a marine protected area network, the Convention on Biological Diversity Strategic Plan for Biodiversity (2010) calls for 10 per cent of coastal and marine areas to be protected by 2017. Currently, marine protected areas protect one per cent of Canada’s oceans and Great Lakes.¹⁹ Australia has increased its protected areas from 4.5 per cent to 7.5 per cent of its ocean space over the last 10 years. Canada was ranked 70th among 228 countries in establishment of marine protected areas.²⁰

In general, the documentary evidence and key informant opinion indicate that marine protected areas are an important conservation tool for oceans spaces under specific conditions. While there is some debate about the optimal size of marine protected areas and the correct framing of objectives and effective monitoring, marine protected areas are generally regarded as (one of many) effective marine conservation tools. In Canada, it is early days to determine the effectiveness of the marine protected areas that have been established, though some initial data is becoming available for several marine protected areas. Just over half of surveyed HOTO partners and stakeholders consider, at least to a moderate extent, that the establishment of protected areas is resulting in the conservation of the marine ecosystem in the designated space, and that marine protected areas are having an impact on the policies, practices or decisions related to oceans activities.

¹⁹There are almost 700 marine protected areas captured in the report by Fisheries and Oceans Canada, Spotlight on Marine Protected Areas in Canada, 2010; 83 managed by the federal government.

²⁰Fisheries and Oceans Canada, Spotlight on Marine Protected Areas in Canada, 2010.

Figure 2: Perceived Impacts of Marine Protected Areas

The key challenges that have been encountered in establishing marine protected areas, according to key informants who indicated them, were significant delays in the approval processes for Areas of Interest for Oceans Act MPAs, as well as protracted consultation processes for some marine protected areas. These challenges were echoed by surveyed HOTO partners and stakeholders who identified a lack of will or weak commitment to marine protected areas at the federal level, difficulties in resolving conflicting interests of stakeholders, and insufficient resources as the three key challenges in the establishment and management of marine protected areas.

Figure 3: Challenges of Implementing MPAs

Evaluation Question #6: To what extent has HOTO met the high priority needs for pollution control?

HOTO’s efforts in the area of pollution control were led by Transport Canada (TC) and particularly focused on enforcement of ballast water regulations and pollution prevention surveillance (including a significant capital investment in aircraft equipment) (Components #8, #9 and #10). Other components addressed research activities with respect to arctic marine shipping and ship waste reduction, and provision of equipment for oil spill response readiness. Together, $26.3 million was allocated to this area, representing 43 per cent of HOTO funding.

A case study of Component #8 – enforcement of ballast water regulations – indicated that enhanced ballast water enforcement²¹ has increased inspection rates to 100 per cent inspection of vessels from overseas entering the St Lawrence Seaway. In this region, ballast water enforcement has benefited from a joint Canadian and US vessel inspection team to increase efficiency in inspection. Compliance with regulations is very high in the Great Lakes Seaway, at 97 per cent, with corrective action taken for the remaining three per cent before they enter the Seaway. Research on the incidence of observed invasive species indicates that there has been a near 100 per cent decrease in the introduction of invasive species to the Great Lakes from ships since the regulations were brought into effect in 2006²².

According to interview respondents, inspection rates for ships entering coastal waters (Pacific and Atlantic) are much lower at 20 to 25 per cent due to the great volume of shipping activity on these coasts. Solutions to managing ballast water in ocean environments are also more challenging, as they require treatment rather than exchange. This is the primary goal of the International Convention for the Control and Management of Ships’ Ballast Water and Sediments also known as the Ballast Water Convention. This treaty is administered by the International Maritime Organization. Ballast water enforcement funded by HOTO has allowed Canada to meet its obligations under the Ballast Water Convention and to ratify it on April 9, 2010.

HOTO funding has also enabled TC to augment the existing National Aerial Surveillance Program budget, resulting in an increase in patrols frequency (patrol hours), expanding the surveillance to areas that were previously not patrolled (such as the Arctic and the Labrador Coast), and improved remote sensing imagery. The increased visibility of NASP surveillance aircraft is perceived to provide a deterrent to ocean pollution incidents. Evidence gathered through NASP surveillance can and has been used to prepare and pursue prosecutions, fines and pleas from marine polluters.

NASP statistics, as presented in the following table, support the deterrent effect. Patrol hours increased since 2006-07 and, while the number vessels that were subject to surveillance has been variable since HOTO, there has been a decrease in pollution sightings since 2009-10, which is suggestive of a deterrent effect.

²¹Specifically, HOTO funding allowed for the hiring of three more dedicated inspectors to the Ballast Water Enforcement Program, as well as resource requirements to develop additional work instructions, conduct national training, train inspectors on the use of the ballast water database and risk assessment module, and life cycle management of the salinity test equipment inventory.

²² S. Bailey. et. al. “Evaluating Efficacy of an Environmental Policy to Prevent Biological Invasions”, Journal of Environmental Science and Technology., 2011, 45 (7), pp 2554–2561, Publication Date (Web): March 9, 2011

Table 5: National Aerial Surveillance Program Statistics

There are some notable, ancillary benefits of the NASP activities. The remote sensing data, for example, are an asset for surveillance, providing intelligence that is useful to many departments including EC (information for ice charts), TC and Department of National Defence (DND) (marine domain awareness information for marine security), as well as DFO (MPA surveillance and enforcement), and CCG (search and rescue).

Other output-based achievements in the area of pollution control include:

• Ship Waste Reduction Strategy (Component #12). The Ship Waste Reduction Strategy was implemented to support regulations pertaining to the disposal of ship waste and port waste reception facilities. During the HOTO implementation period, Canada ratified the international requirements for garbage pollution prevention -- the Regulations for the Prevention of Pollution by Garbage from Ships (Annex V of MARPOL). The Regulations apply to all ships in Canadian waters and to all Canadian ships everywhere and ban the discharge of garbage into internal waters and specify requirements for garbage discharge into the oceans. The Regulations also specify record-keeping requirements for commercial carriers. The HOTO-funded Waste Reduction Strategy contributed support for these regulatory activities, by showing how wastes can be managed to meet these regulations. With respect to port reception facilities, under the Canada Shipping Act, Canada has the authority to make regulations requiring ports to provide reception facilities, though to date has not regulated this area. In 2005, TC began to explore other non-regulatory options such as an approach introduced by the “Baltic Strategy” which lays down the no-special-fee system (all costs are integrated into port fees) and mandatory delivery of all wastes ashore. The HOTO-funded Waste Reduction Strategy facilitated the study and consultation processes and contributed support for related activities. Owing to the complex nature of how ports operate and multiple jurisdictions involved, consultations are continuing towards ensuring adequate facilities are maintained in Canadian ports and that new port developments account for waste management needs.
• Arctic Marine Shipping Assessment. With contributions from AANDC and TC, these components evaluated current and future trends, opportunities and threats in marine shipping traffic in the Arctic (Component #5 and #11). A case study of these components indicates that the report has been translated and is available online. The Arctic Marine Shipping Assessment report has been promoted at conferences and to Arctic Ministers, and component leads and partners feel the report will be effective in informing future policy decisions related to shipping and pollution control in the North.
• Spill Capacity Study and Emergency Response Strategy. This component involved distribution/replacement of emergency oil spill response equipment at ten existing depots and outfitted nine additional sites for emergency spill response in the Arctic (Component #22).

Key informants agreed that HOTO activities have met the high priority needs for pollution control, noting observable impacts on aquatic invasive species in the Great Lakes and other tangible increases in resources, such as the increased number of inspectors and aircraft, and greater access to pollution spill response equipment. Key informants also highlighted HOTO funding devoted to research that may contribute to future pollution control, such as studies related to ballast water treatment.

Evaluation Question #7: To what extent has HOTO met the high priority needs for Collaborative Oceans Management?

Diverse government authorities and stakeholders have an interest in oceans activities and management. The components under HOTO that address collaborative oceans management include Canada’s (EC and DFO) participation in the international Gulf of Maine Council (as two separate components), as well as fostering linkages with the WWF and alignment of marine environmental assessment tools with the Canadian Environmental Assessment Act Assessments. Four centres of expertise were also created under this component to address oceans management challenges. Total funding from HOTO for this priority needs area was $6.2 million or 10 per cent of the total allocation.

In addition to the components specified above, other HOTO components also featured a high degree of engagement and collaboration with stakeholders – for example, community consultations conducted by AANDC in support of the Arctic Marine Shipping Assessment report²³ and advisory, technical and stakeholder committees struck to gather input on the designation of marine protected areas that were led by DFO, EC and PCA. Engagement and collaboration activities included other federal departments, provincial/territorial representatives, Aboriginal organizations and non-governmental organizations. In the case study, the Federal Marine Protected Area Strategy and National Framework were also noted as key requirements for collaborative oceans management.

In general, key informants felt that HOTO addressed priority needs in the area of collaborative oceans management very well, or at least were “on the right track”. The Gulf of Maine case study (Component #4) is an illustration of enhanced international and domestic collaboration in this transboundary ecosystem as a result of HOTO. Under Component #4, EC received HOTO funds to enhance Canada’s contribution to the joint Canada-US Gulf of Maine Council five-year action plan. A positive, if unanticipated, impact of HOTO was the creation of a Canadian Steering Committee composed of federal departmental representatives (EC and DFO) and relevant ministries from three provinces to better understand jurisdictions’ activities in the Gulf of Maine and to coordinate Canadian participation in the Gulf of Maine Council. During the implementation of HOTO, increased coordination occurred with DFO’s development of the State of the Gulf of Maine Report (Component #18).

Survey respondents who were engaged in committee or consultation/briefing processes were satisfied with the engagement processes (though the sample size is quite small, n=28). Surveyed stakeholders were most likely to agree that the committee/consultation roles and responsibilities were made clear and were understood (over 80 per cent responded 4 or 5 on a 5-point scale). About 70 per cent of surveyed stakeholders each also agreed that committee/consultation results were disseminated appropriately, that the mandate of the committee/consultation was clear and that the views of all stakeholders were considered.

Weaker satisfaction levels were expressed regarding committee effectiveness in identifying priority issues (44 per cent agreed) and that consultation/briefings provided participants with adequate information on the nature/benefits of health on the oceans activities to the marine environment (46 per cent agreed).

Surveyed HOTO stakeholders who participated in committees and/or consultations/briefings rated the impact of these engagement processes highest for sharing of information and knowledge, inclusion of a broad spectrum of stakeholders and transparency in the conduct and management of activities (between 60 and 63 per cent indicated that this occurred to a moderate or great extent). Views were more divided and less certain about the effectiveness of collaborative processes in improving stakeholder commitment to collaborative management and in considering stakeholder views in oceans management decision-making (54 and 53 per cent of surveyed stakeholders respectively rated the committee/consultation processes as having this impact to at least a moderate or great extent).

²³Note that three community consultations were initially scheduled for this HOTO component, though six were conducted.

Figure 4: Stakeholder Opinion on Impact of Committees/Consultations

Evaluation Question #8: To what extent has HOTO made progress towards the health of the oceans?

Key informants and survey respondents perceive HOTO as having laid the groundwork for continued collaboration, knowledge sharing and information networks that will support future actions to support healthy oceans. This sentiment was echoed by surveyed HOTO stakeholders, where 32 percent of these stakeholders indicated that efforts on health of the oceans was contributing to a moderate or great extent to keeping Canada’s coastal waters clean.

Evaluation Question #9: What, if any, unexpected results have occurred as a result of HOTO?

A minority of key informants indicated unexpected results from HOTO. A few key informants highlighted that the breadth of participation and the extent of dialogue among various stakeholders in consultative/collaborative processes exceeded expectations. In some cases, collaboration on HOTO led to collaborations on other issues (e.g., establishment of a federal interdepartmental task group on criteria and conduct of reviews and assessments of resource potential in order to establish marine protected areas, networking between DFO science and the Canadian Healthy Oceans Network funded by the Natural Science and Engineering Research Council). Research activities have also spurred some unexpected successes and avenues of further inquiry (e.g., the efficacy of brine in treating ballast water).

In the area of pollution control, an unintended outcome was Canada’s participation in the emergency response to the US Deepwater Horizon oil spill. The onboard remote sensing equipment used by the National Aerial Surveillance Program was instrumental in providing strategic information during this spill. Similarly, Canada’s profile was raised internationally through its leadership role in the Arctic Marine Shipping Assessment.

A small number of key informants also identified some unintended outcomes of HOTO having to do with previously unknown gaps in knowledge or in practice (e.g., regional differences in the enforcement of ballast water regulations, baseline data needs in the establishment of marine protected areas, absence of international data on shipping activity in the Arctic). According to a few key informants, shifts in federal approaches to oceans management (e.g., a perceived move away from integrated oceans management) and regional differences in implementation have had an unintended negative effect of creating confusion and uncertainty among stakeholders.

4.3  Efficiency

Evaluation Question #10: Was HOTO implemented as designed? What external factors have influenced the implementation of HOTO?

Annual performance summaries and key informant views indicate that, overall, HOTO components were implemented as designed with minor modifications to activities and timelines. As approval of HOTO funding did not occur until September 2007, funding was not available to partner departments/agency until as late as January 2008. This resulted in a delay in the implementation of many planned activities by almost a year. Workplans and priorities were adjusted in subsequent years in response; however, delays were particularly problematic for activities for which the HOTO five-year timeframe was considered to be already very compressed (e.g., marine protected area establishment). Despite these challenges, tangible results were being reported in each of the three priority areas as of 2009-2010.

Implementation delays continued to be a factor beyond year one of HOTO for some components, leading to further modifications to timelines. Delays were due to: protracted federal contracting processes; extensive public consultations; lengthy senior management and Ministerial approval processes for Oceans Act Areas of Interest; and the federal election.

Aside from delays, there were other instances where implementation diverged more substantially from design. For the establishment of marine protected areas, the governments of Canada and Nova Scotia determined that Sable Island should be designated as a national park instead of a national wildlife area (Component #2). This arrangement will, in turn, have implications for the operation of the Sable Island Weather Station (Component #3). EC and PCA have initiated a collaboration on Sable Island, including the type of infrastructure required on the Island, potential ‘greening’ initiatives and alternative resupply methods to increase efficiency. In another example of divergence from the planned HOTO activities, work ceased on the Foxe Basin-Eastern Arctic Area of Interest as the Nunavut partner organization was not prepared to continue with this work in the absence of a long-term, overarching agreement on marine protected areas (Component #19). Under the ecosystem science support component, the Beaufort Sea monitoring pilot project was annulled (as the Conservation of Arctic Flora and Fauna’s Circumpolar Biodiversity Monitoring Programme superseded this need) in order to support the application of an Ecosystem Approach in the Beaufort to inform an oil and gas decision support tool (Petroleum and Environment Management Tool) (Component #15). Planned networking by the Oceans Centres of Expertise in Traditional Ecological Knowledge was reduced due to overlap with other traditional ecological knowledge initiatives (Component #16).

External factors that were noted by key informants as having an impact on the implementation of HOTO include:

• intergovernmental and jurisdictional aspects and complexities (e.g., negotiation of the text of the Arctic Marine Shipping Assessment report among eight participating countries, respecting treaty rights in marine protected areas such as Scott Islands, provincial jurisdiction over ballast water and commitments under international agreements and resolutions (e.g., United Nations));
• logistics and costs associated with implementing projects in the Arctic (e.g., few harbours, inadequate emergency response, local research institutions, small window of operations due to climate, the greater distances and the increased cost of transportation);
• competing stakeholder interests and consultation fatigue/limited capacity of some stakeholder groups to participate; and
• the lack of clarity and consistency of policy direction (referring to issues such as insufficient federal inter-departmental linkages around oceans management, DFO’s dual role to promote the fisheries/aquaculture industry, as well as conservation, regional inconsistencies in communications and practices in marine protected area designation and regulatory enforcement, decisions on marine protected areas that have not taken into consideration HOTO or ecological objectives and priorities).

Surveyed partners and stakeholders identified three major challenges associated with HOTO oceans activities: lack of sufficient funding for marine protected area implementation (“budget constraints”, “working level staff have no funds to deal with the issues that make or break marine protected area establishment – like licence buyouts”); lack of commitment from all government departments (“(lack of) will/vision to establish a whole of government approach to oceans management”, “(lack of) senior level commitment to make things happen on the marine protected area front”); and lack of meaningful engagement of stakeholders (“keeping non-governmental stakeholders engaged due to lengthy processes”, “lack of citizen engagement in decision-making process”, “not convinced real collaboration is occurring at the stakeholder level”).

Figure 5: Perceived Challenges of Implementing HOTO Activities

Evaluation Question #11: Was HOTO delivered in an efficient manner? What factors supported/hindered efficiency?

The following table presents the expenditure profile for each HOTO component and partner department/agency. A number of components (13 of 22 components) show lapsed funding to be 10 per cent or greater compared to planned expenditures according to the data available through to year four of HOTO.²⁴ For HOTO overall, actual expenditures are 15 per cent lower than planned expenditures. The amount of lapsed funding varies significantly by component and is related, at least in part, to implementation delays. In the first fiscal year of HOTO - 2007-2008 - approval from Treasury Board did not occur until half way through the year, and funds were not accessible until the final quarter. Workplans had to be adjusted and, in many cases across the five partner organizations and the 22 components, work in the first fiscal year was limited to reprofiling workplans and preparing for staffing. Consequently, a very high percentage of all lapsed funds (for the first four years) occurred in year one. While much of the unspent year one allocation was placed into planned carry-forwards, these were only partially recouped in year two due to a lack of capacity to spend the inflated year two allocation. By year three, with more HOTO-funded staff on hire, and component workplans reprofiled, variance dropped overall. In fact, for many components there began to be significant funding shortfalls. It is also important to note that some of the overall variance is attributed to double-counting of reprofiled funds and, in some cases, HOTO funding was reprofiled within departments to other priority program activities supportive of health of the oceans. The ability of HOTO partner departments/agency to fully spend the residual funds in the last year of HOTO to address HOTO objectives was not explored in the evaluation.

The perceived adequacy of HOTO resources varied across the components. For example, the demands on ecosystem science (Component #21) were seen to exceed available resources, particularly given the expense of scientific research in the Arctic and other remote locations. Planned expenditures for this component were fully spent during the first four years of the Initiative. The components pertaining to the implementation of the Federal Marine Protected Area Strategy experienced challenges in the allocation of the HOTO funds between salary and O&M dollars (salary being insufficient), which introduced workload challenges for staff in completing the work.

In general, however, internal key informants and case study respondents commonly noted that they had been efficient as they could be, given the resources they were provided and the timing of receiving them. Efficiency was supported through the use of existing tools (e.g., communications, governance mechanisms) and shared interests (e.g., pooling of resources among departments/agency involved in marine protected area establishment through the implementation of the Federal Marine Protected Areas Strategy).

Factors that reportedly hindered HOTO efficiency included inefficiencies associated with delays or suspension of workplans due to protracted internal approval processes with respect to financial approvals and Ministerial approvals of Oceans Act Areas of Interest. As well, according to some internal key informants, the multi-stakeholder collaboration process does not lend itself to timely delivery of the program. These processes require significant resources (both human and financial). According to one respondent, “you can be inclusive or efficient, not both”.

A common concern expressed by key informants was the short-term nature of the HOTO funding commitment. Many of the HOTO component activities require ongoing support – for example, regulatory enforcement and long-term monitoring and management of marine protected areas.

²⁴Only the pollution prevention (surveillance) Component #9 was slightly over budget at the end of year 4 (by four per cent)

Source: HOTO Financial Expenditure Summary_Sept 2011.xls.

Surveyed HOTO stakeholders offer another perspective on improvements to health of the oceans programming. There was little consensus among surveyed stakeholders, with a variety of opinions mentioning: greater involvement with local interests and need for greater transparency/unbiased agency (each mentioned by 14 per cent of respondents) and 11 per cent of respondents mentioned improvements involving increased public awareness/education; increased resources; and improved management processes.

Figure 6: HOTO Stakeholders Proposed Improvement to Health of the Oceans

Evaluation Question #12: To what extent are the roles and responsibilities clear and appropriate within and among HOTO partner departments and agencies? To what extent is interdepartmental cooperation around oceans management taking place?

DFO serves as the lead department for HOTO, and partner departments and agencies have responsibility for the delivery of their respective components. Most internal key informants agree that the roles and responsibilities are clear and appropriate within and among HOTO partner departments and agencies. The identification of the HOTO components with lead agency assignments was useful to specify the responsibilities.
According to program documentation, interdepartmental cooperation occurs through a committee governance structure that was established under the OAP in 2005-06. Note that interdepartmental membership is not restricted to HOTO partners, but is comprised of roughly 20 departments/agencies with mandates relating to oceans. There are three federal government interdepartmental committees:

• The Deputy Ministers (DM) Interdepartmental Committee on Oceans is an ad hoc committee that is brought together when needed, though did not meet during HOTO.
• The Assistant Deputy Ministers Interdepartmental Committee on Oceans is a sub-committee of the Deputy Ministers Interdepartmental Committee, chaired by the ADM, Ecosystems and Oceans Science Sector, DFO (to June 2011 prior to a realignment within DFO). The committee is a forum for collaboration and joint action on matters related to the Oceans Action Plan, including the HOTO Initiative and, more broadly, implementing the federal agenda as it touches on Canada’s oceans. The Committee provides operational direction downward. This committee met three times in 2008 and 2009.
• The Directors’ General (DG) Interdepartmental Committee on Oceans includes representation from HOTO Initiative partner departments/agency (five federal departments), other federal organizations with a stake in the Oceans agenda, as well as central agencies – roughly 20 organizations in all. The committee provides advice and support to the Assistant Deputy Ministers Interdepartmental Committee on Oceans and is also a forum for joint action in support of a national strategy for the management of all activities in or affecting marine areas. The Directors General (DG) Interdepartmental Committee on Oceans met seven times, to date, since the announcement of HOTO in October 2007.

Secretariat functions are performed within the Regional Operations Branch at DFO to coordinate the activities of the interdepartmental committees (since 2010-11). The Terms of Reference for the interdepartmental committees were revised in 2008. The secretariat has also helped to foster information sharing and collaboration through the HOTO Forum – a web-based information portal for interdepartmental committee members and a HOTO Leads Meeting, held at the mid-way point of the Initiative. There is also a web-based Forum which enables information sharing across the wider committee membership, beyond the five HOTO partners.

Most internal key informants confirm that interdepartmental cooperation around oceans management is taking place to a greater extent than prior to HOTO, with the caveat that some HOTO activities are less amenable to or require less interdepartmental collaboration than others and collaborative activities at the regional level do not occur in the same fashion or with the same success in all regions. Where interdepartmental cooperation has been effective, most key informants note their satisfaction with the format and the benefits of a more coordinated approach to oceans management.

A key recommendation of the OAP evaluation was a more strategic role for the interdepartmental committees. The senior level interdepartmental committees met sporadically during HOTO and DFO as lead department has few levers to encourage greater commitment. Improvements recommended by some key informants included greater commitment on the part of partner departments/agency and increased engagement of senior level management to improve on delivery of common initiative objectives and identified priorities.

Evaluation Question #13: To what extent do the performance monitoring/ measurement processes and tools support decision-making and accountability requirements?

The evaluation of the OAP recommended improvements to performance monitoring for future phases, and clarification and strengthening of the roles and responsibilities of the Secretariat with respect to performance measurements and monitoring.

For HOTO, a Risk Based Audit Framework/Results-based Management and Accountability Framework (2007) was developed. DFO was charged with compiling biannual updates received from each partner department/agency for each component into an annual performance summary report. To support performance reporting, a performance measurement template and a protocol for completion, approvals and transmission were developed. Performance data that are collected include: HOTO activity linkage to departmental/agency PAA; expenditures; status of projects; evaluation and audit studies; work plan deliverables accomplished during the fiscal year and progress towards the five-year outcome of a HOTO component; and other key indicators to measure progress and indirect results. The completed templates for each component are then summarized by DFO in a one to two-page table.

A strong majority of internal key informants with responsibility for HOTO components hold positive views about the HOTO performance monitoring and measurement processes and tools. There were no significant difficulties, and several interviewees described the tools specifically as “clear”, “simple” and “user-friendly”. Additionally, some component leads reported using their HOTO performance reporting to benefit their own internal functions, allowing them to track their progress, remain focussed, document the project for continuity during staff absences or turnover, and make better-informed decisions.

Only a few suggestions for improvement were made:

• Have finance groups fill in the financial data rather than the program staff, as the financial officers may have more precise knowledge of expenditure details.
• There should be an opportunity to adjust objectives and reporting to reflect evolution of the Initiative.
• Use a specific project code for HOTO to simplify the financial reporting process.
• Preference for annual (as opposed to biannual) reporting.

As mentioned previously (under Section 3.5), the specification of outcomes within the HOTO logic model presented challenges in answering evaluation questions related to achievement of intended outcomes. For instance, addressing “high priority needs” in areas of marine protected area establishment, pollution control and collaboration is challenging to demonstrate from the performance measurement information, and indicators tend to overlap with immediate outcomes. As noted previously, this is due, in part, to the output-based nature of some components. For others, such as those pertaining to marine protected area establishment, it is too early to tell whether such areas are having a positive impact on conservation of sensitive marine areas and health of the oceans, as they are not yet fully established and practical conservation objectives, and monitoring indicators and protocols are not yet in place.

The case study of the Federal Marine Protected Areas Strategy noted a challenge in the identification of targets, indicators and monitoring protocols to evaluate the effectiveness of the overall federal marine protected area network. While some work was undertaken in this area by DFO under the national marine protected area network initiative (Component #14), an overall monitoring and performance framework for marine protected areas is still pending. Interviewees point to the inherent difficulties in establishing common indicators within a specific bioregion and the need for greater clarity and direction from the respective departments/jurisdictions on monitoring and reporting requirements.

4.4.  Economy

Evaluation Question #14: Is there a more economical way to achieve the objectives of HOTO?

The issue of economy concerns whether program outcomes are achieved at least cost and whether there are alternatives to the current program that could achieve the same outcomes at a lower cost. In this section, the evaluation findings presented include key informant opinions and suggestions in documents on alternative, more cost-effective approaches to achieve HOTO objectives.

There were few program alternatives to HOTO that were identified by key informants. The HOTO components were selected for funding from a larger package that included a broader array of components, some of which were subsequently not funded. Comparative effectiveness of a different configuration of components or alternative governance model is difficult to estimate. A small number of key informants felt the absence of a HOTO component dedicated to integrated oceans management had a negative impact on the effectiveness and cohesiveness of HOTO.

There were some mentions of alternatives to the current federal approach for selected HOTO components. A few key informants mentioned the potential for a greater role for industry in bearing the costs of oceans management. An illustration of this would be in the area of pollution control, where regulations requiring ships to have systems on board for the treatment of ballast water would reduce federal costs for ongoing enforcement of the ballast water exchange regulations in the long-term. A small number of respondents noted that Canada currently has three departments/agencies with responsibility for marine protected areas and the potential for consolidation of activities within one agency or an umbrella agency was suggested as potentially leading to efficiencies. However, there is no evidence from the evaluation that efficiencies would, in fact, materialize and currently all three organizations hold legislated mandates to establish marine protected areas.

There was little in the documents that were reviewed to demonstrate more economical ways to achieve the objectives of HOTO. However, McLeod and Leslie (2009)²⁵ argue that a centralized and consistent federal approach to oceans management may be most economical and effective. Regional planning is important, but this should be orchestrated centrally. The authors further argue for increased efforts to integrate multiple sectors to achieve maximum economy of resources in undertaking collaboration, scientific research, and establishing marine protected areas.²⁶ This could include the delegation or contracting of tasks to academic, private and not-for-profit sectors while government maintains a management role. In this vein, a few key informants also suggested capitalizing on existing hard and soft infrastructure, like the research technology available at research institutes and academies, or volunteer and civil society networks for information sharing and consultation (though this approach would be less practical for remote/offshore marine protected areas).

With respect to marine protected areas, Australia has significantly increased the number of marine reserves and area under protection through the use of regional marine plans. In their assessment of the Australian approach, Stark and Ladell (2008) suggest that “combining MPA network planning with regionally-based marine-use planning is the most efficient way to achieve a network of MPAs that takes into the account the needs of government, multiple user groups and coastal communities”.²⁷ Jessen’s (2011) review of global best practice in marine protected areas and marine planning notes advances in several areas: the designation of vast new marine protected areas (with significant no-take/high internal protection levels) such as in the US Northwestern Hawaiian Islands and Australian Great Barrier Reef; efforts to implement marine protected area networks (as opposed to marine protected area practice focused on individual sites) within an ecosystem-based management approach and marine spatial planning; and the definition of time-bound public processes to identify and implement marine protected area networks.²⁸

Finally, Canadian Parks and Wilderness Society research outlines several socio-economic concerns that need to be considered specifically in the establishment of marine protected areas and could be continued or reinforced in future oceans programming:

• Identify community-based initiatives and integrate local knowledge;
• Inventory current uses and activities; and
• Identify opportunities for alternative uses and compatible activities (Jessen et al. 2011)

²⁵K. MacLeod and H. Leslie, “State of Practice,” in Ecosystems Based Management for the Oceans (Island Press, 2009)
²⁶K. MacLeod and H. Leslie, “State of Practice,” in Ecosystems Based Management for the Oceans (Island Press, 2009);
    M. Rutherford, M. Dickinson, and T. Gunton, “An evaluation of the national framework for marine planning in Canada,” Environments 37, no. 3 (2010).
²⁷J. Stark and K. Ladell, The International School of Marine Protected Areas Progress Report Card: Canada, Australia and the United States, A Report by Living Oceans Society, the Sierra Club British Columbia, and the David Suzuki Foundation, Vancouver, BC, 2008.
²⁸S. Jessen, A Review of Canada’s Implementation of the Oceans Act since 1997 – From Leader to Follower?, Coastal Management, 39:1, 20-56.

Evaluation Question #15: What are the lessons learned in implementing HOTO thus far? What are the key priorities for oceans management moving forward?

Lessons Learned

Key informants were asked to describe any lessons learned that have emerged from their experience with HOTO. A frequent comment from key informants was that, overall, HOTO was important in providing focus and commitment to identified priority needs areas. Dedicated staff and resources supported progress on many fronts. Lessons learned that were identified included:

• Coordination. There were many key informant comments pertaining to the importance of coordination in the effective and efficient delivery of oceans management. For many components, coordination was initially weak, though evolved toward greater coherence over time. Examples mentioned by key informants include:
  • The development of the National Framework for a national marine protected areas network was intended to inform and coordinate the selection of marine protected areas, however, it was approved too late to inform the selection of the HOTO Oceans Act Areas of Interest.
  • Processes for consultation and selection of Oceans Act Areas of Interest were not well coordinated regionally and did not have the benefit of central coordinating support within DFO in the early years of HOTO. Opportunities for efficiencies across regions (sharing of tools, etc.) were lost. The lack of consistency was noted as a source of frustration by several HOTO stakeholders.
  • Coordination across components was underdeveloped in the original HOTO submission. EC and DFO participation in the Gulf of Maine, for example, evolved toward greater coordination during HOTO through the creation of a Canadian Steering Committee to ensure a coordinated Canadian approach in the Gulf and on the Gulf of Maine Council.
• Realistic expectations and clear objectives and outcomes. Several components that had clear and tangible objectives and were not subject to external constraints were more straightforward in their implementation and had better success in achieving and demonstrating effectiveness. Other components had objectives that were more broadly cast, ambitious or amorphous, or which were subject to factors in the broader environment. Completion of these components proved very challenging in terms of both resources and timeframes for those charged with their implementation (e.g., marine protected area establishment).
• Collaboration and partnership. While collaborative oceans management was a priority intended outcome of HOTO (which five of the 22 components were identified as supporting), key informants more broadly noted the value of the engagement and collaboration that occurred within most of the components as a best management practice. Collaborations facilitate information sharing among interested parties, establish buy-in and often the pooling or leveraging of resources and expertise.
• Anticipate operational requirements and consider these within the funding request. This lesson learned was pertinent, for example, to the pollution control components for which funding was used to support operations – the distribution of emergency spill response equipment or aerial surveillance operations. These kinds of activities have ongoing resource implications, such as training and maintenance of physical equipment, that need to be recognized within the original allocation. With respect to marine protected areas, costing models for marine protected area establishment and ongoing monitoring and management were limited. Resources were perceived by some to be inadequate to accomplish these tasks.
• Experience, skill and continuity of the project lead. This factor was mentioned by a small number of key informants. Components that were led by experienced leads, often with existing networks or prior direct experience with the component activities/operations were noted to benefit from a rapid and strong start.

Priorities Moving Forward

When asked to identify what they believed to be the most important future priorities for the health of the oceans moving forward, many key informants generally noted the importance of continued funding under a future ‘HOTO2’. Many of the HOTO components are considered by key informants to be investments in addressing longer-term oceans management needs, which if not funded further, would represent a significant loss of momentum and effort invested to date. Moreover, several key informants urged greater leadership and attention to oceans management at the senior levels of government, with a closer alignment of efforts to the Oceans Act/Oceans Strategy to ensure a balance between protection and sustainable development of oceans.

In terms of specific priorities, the broad spectrum of HOTO components yielded similarly broad responses with respect to priorities moving forward. However, commonly noted were:

• Marine protected area designation and management. The HOTO marine protected area establishment objectives are expected to be met by 2014. Completing the designation process was noted by key informants as a high priority need for conservation of sensitive marine areas and to meet Canada’s international commitments under the Convention on Biological Diversity. However, implementation of consultation and regulatory processes remain for many marine protected areas, requiring additional investment in the short-term. In the longer-term, science work to refine conservation objectives, and develop indicators and monitoring protocols for new and existing marine protected areas is required, as well as strategies and enforcement support for management of marine protected areas. The National Framework that is now in place is expected to pave the way for a more coherent and streamlined selection of marine protected areas in the future. Surveyed stakeholders appear to agree with this priority – when asked about their level of support for a National Framework for Canada’s network of marine protected areas, 70 per cent of stakeholders supported the framework to a great extent and another 21 per cent supported a National Framework to a moderate extent.
• Regulatory enforcement. HOTO provided funds to support enforcement of ballast water regulations and pollution prevention surveillance through the NASP. Key informants pointed to the effectiveness of these efforts and, more generally, for the need for regulations of any kind to be accompanied by some form of enforcement regime to ensure compliance.
• Meeting Canada’s international commitments. As mentioned above, Canada is a signatory to the Convention on Biological Diversity, which specifies targets for protection of marine areas that are currently not being met. As well, endorsement of the International Convention for the Control and Management of Ships' Ballast Water and Sediments, to which Canada is a contracting party, is expected in the short-term. A number of key informants further named Canada’s participation and contribution to international organizations – the Gulf of Maine Council and the Arctic Council – as continuing priorities.
• The Arctic. The Arctic was mentioned by several key informants as a key priority for future oceans management. Changing climactic conditions and increasing pressure for resource development in the Arctic were said to suggest a need for attention in this region.
• Coordination and partnerships. Consistent with lessons learned mentioned above, a number of key informants further noted future priorities to include continued building of partnerships, and coordination and transfer of knowledge across partner departments/agency and across oceans management activities. One example is creating opportunities to pool efforts and resources to address multiple tasks. The use of aerial surveillance to monitor ocean pollution, while at the same time meeting objectives for data collection and enforcement of Oceans Act management strategies is an illustration of coordinated effort that is currently being explored.

Examples of other priorities mentioned by small numbers of key informants include: expansion of education and engagement of Canadians in oceans management and fostering action at the community level; addressing marine debris and near shore issues such as land-based sources of oceans pollution; continued promotion of HOTO products and success stories (e.g., the Arctic Marine Shipping Assessment report); and providing infrastructure support to small vessels.

Key informants’ priorities were echoed to some extent by surveyed HOTO partners/stakeholders. The most important future priorities identified in the survey include: marine protected area designation, management and monitoring (24 per cent) (though recall that survey respondents are drawn largely from stakeholders involved in marine protected area establishment); establish clear goals, targets or tangible actions (22 per cent); and undertake partnerships and collaborations (14 per cent).

Figure 7: Stakeholder Opinion on Future Priorities for Health of the Oceans

Among surveyed stakeholders, 43 per cent suggested that the federal government assume a leadership role aligned with the Oceans Act to implement these priorities and 27 per cent also mentioned committing funding and resources a key federal role to address priorities.

5.  Conclusions

The findings of the evaluation lead to the following broad conclusions about the relevance and performance (effectiveness, efficiency and economy) of the Health of the Oceans Initiative.

5.1.  Relevance

• The evaluation evidence indicates that there is an ongoing need for attention to the health of the oceans. Canada’s oceans have great economic and social significance, yet the documentary evidence indicates that oceans are under pressure in many areas. Key informants, HOTO stakeholders, and the public support efforts to address the health of oceans.
• HOTO aligns with federal priorities. HOTO is an aspect of the Government of Canada’s National Water Strategy and supports other federal priorities established in the Arctic Agenda, Canada’s Northern Strategy and Federal Sustainable Development Strategy. HOTO also supports Canada in fulfilling its commitments to the international treaties on biological diversity and pollution control.
• The Oceans Act and Canada’s Oceans Strategy outline the federal mandate in oceans programming, and HOTO contributes to the federal effort to fulfill this mandate. The federal level makes an appropriate and important contribution to oceans programming, in collaboration with other levels of government and other stakeholders.

5.2.  Performance

Effectiveness

• Evaluation evidence from all lines of inquiry indicates progress toward the achievement of the immediate outcomes of the HOTO components. Many of the objectives for the 22 components will be met during the HOTO timeframe.Components that have output-based immediate outcomes or support operational requirements were accomplished with few challenges. Those with immediate outcomes of greater complexity or dependent on external factors have encountered greater challenges.
• Progress on the HOTO intermediate outcome ‘meeting the high priority needs of marine protected area establishment’ is mixed. There has been significant progress in the development of key strategic and operational supports to establish and manage a national network of marine protected areas. A federal strategy and national framework for marine protected areas have been developed. Intended outcomes with respect to establishment of marine protected areas will not be met within the HOTO timeframe, though are targeted for completion by 2014. Continued development of appropriate conservation objectives, related monitoring indicators and protocols, and practical tools for marine protected area management is required and still at the early stages.
• High priority needs for pollution control have been addressed through four HOTO components. The ballast water regulations are enforced in all Canadian waters. In the Great Lakes and St. Lawrence Seaway in particular, the expansion of the enforcement of regulations pertaining to ballast water has been effective in achieving compliance. Since 2006, no new alien invasive species attributable to ships’ ballast water have been reported in the Great Lakes. HOTO has allowed the existing aerial surveillance program to expand its pollution patrol over all Canadian waters. Since the expansion, there has been a continuous reduction in the number of ship-source pollution spill sightings. HOTO has also highlighted shipping issues in the Arctic and supported the assessment of options for reduction and disposal of ship waste.
• There has been progress in addressing priority needs related to collaborative oceans management. Several HOTO components were dedicated to this effort, and collaboration took place to implement many others. HOTO enabled many government and stakeholder collaborations to be established or advanced. Surveyed HOTO partners and stakeholders are satisfied with these processes. Impacts were noted in areas such as information sharing and the inclusion of a broad spectrum of views, but less so in terms of influence on oceans decision-making and stakeholder commitment to collaborative management.
• The program’s ultimate outcome – health of the oceans – is achievable only in the longer-term and it is difficult to directly attribute HOTO’s contribution to its achievement. Some concrete outcomes, such as in the area of pollution control, have contributed to this objective; however, other evaluation evidence measuring the program’s performance on this measure was limited.
• Unintended outcomes of HOTO were few, but mostly positive in nature. Unintended outcomes were largely in the area of unexpected beneficial aspects of collaborations. As well, there was an unexpected positive outcome of the National Aerial Surveillance Program. The Program extended its assistance to the United States for monitoring an oil-spill emergency in the Gulf of Mexico, and acquired international recognition of the excellence of its services. An unintended negative outcome is that shifts in federal approaches to oceans management and regional differences in implementation have created confusion and uncertainty among stakeholders.

Efficiency

• Aside from delays experienced in the early years of HOTO, most components were implemented as designed. Implementation diverged from design in instances where external factors played a role in reshaping the objectives of the component. Implementation challenges included delays outside the control of the program and other factors such as: intergovernmental and jurisdictional complexities; logistics and costs associated with working in the Arctic; consultation fatigue/limited capacity of some stakeholder groups to participate; and lack of clarity and consistency of policy direction with respect to oceans management.
• The efficiency of the delivery of HOTO is mixed. While components were implemented with the funds available, concerns were expressed about the short-term nature of HOTO funding to address longer-term oceans management needs. Factors that hindered efficiency included delays and multi-stakeholder consultation processes that require significant time and resources.
• The roles and responsibilities of HOTO partner departments/agency were viewed as clear. Terms of reference for interdepartmental committees were refreshed for HOTO. However, the DM and ADM committees met infrequently, and increased engagement of senior management, as well as increased joint ownership of Initiative objectives was recommended by some.
• Performance measurement improved in response to recommendations from the Oceans Action Plan evaluation, with new tools and processes introduced to capture results from HOTO. Component leads are generally satisfied with the requirements for monitoring and reporting, and the performance measurement process proved useful for internal management and reporting functions. Challenges were encountered in evaluating HOTO intermediate and ultimate outcomes as many components are output-based or were not sufficiently advanced to measure impacts at this level.

Economy

• Few alternative models are available to deliver on HOTO’s objectives. There are some potential alternative program vehicles to deliver aspects of HOTO but these are not clear in terms of their effectiveness or efficiency benefits. Suggestions to improve effectiveness tended to converge around the need for consistent and coordinated approaches to implementation, and leveraging external resources such as from local level organizations, industry and science.
• HOTO supported an increased federal focus on oceans management. There are many lessons learned that may benefit efforts moving forward to address evolving priorities. In addition to urging greater leadership and attention to oceans management overall, and a closer alignment of efforts to the Oceans Act/Oceans Strategy, future priority areas identified in the evaluation were diverse, reflecting the variety of the HOTO components themselves. Priorities were identified that, if not funded further, would represent a significant loss of momentum and effort invested to date. Key priority areas that were identified included: continued support to fulfill HOTO objectives related to marine protected area establishment; meeting requirements for regulatory enforcement; honouring Canada’s international commitments; attention to the Arctic; and continued focus on coordination and building partnerships.

5.3.  Recommendations

Collaboration was highlighted as a best management practice which resulted in stronger efforts between Health of the Oceans Initiative partners to maximize the benefits of operational activities. Coordinating structures put in place for oceans planning had tangible benefits, such as building an inventory of current oceans activities within participating jurisdictions. Roles and responsibilities of federal interdepartmental committees were clearly articulated; however, there was less participation of senior management than originally anticipated. While some progress has been made in increasing the engagement of departments and agencies with regulatory authority in decision-making about oceans activities, there is still significant room for improvement.

In the event of Health of the Oceans Initiative funding renewal, the following recommendation should be addressed.

Recommendation:

It is recommended that the Assistant Deputy Ministers responsible for Health of the Oceans objectives, from all five partner departments and agencies (Fisheries and Oceans Canada, Aboriginal Affairs and Northern Development Canada, Environment Canada, Parks Canada Agency, and Transport Canada), work cooperatively to:

1. a) provide centrally coordinated policy direction to support and advance collaborative responses to current and emerging oceans management issues;

         b) establish a mechanism to support increased collaboration and maximize the benefits of Health of the Oceans activities; and

         c) develop strengthened performance measurement tools, including enhanced outcomes, indicators and protocols, in the event of a renewed federal oceans management program involving collaboration across multiple government departments and agencies.

Annex A: HOTO Evaluation Matrix

Annex B : Status Update on HOTO Components

*As identified in the HOTO Risk Based Audit Framework

ANNEX C : Management Action Plan