Internal Audit Report

Audit of the Aquatic Climate Change
Adaptation Services Program

Project 2014-6B265
Date: March 6, 2015

Logo canada government

TABLE OF CONTENTS

1.0 EXECUTIVE SUMMARY
2.0 BACKGROUND
3.0 AUDIT OBJECTIVE
4.0 AUDIT SCOPE
5.0 AUDIT APPROACH
6.0 AUDIT FINDINGS
7.0 AUDIT OPINION
8.0 STATEMENT OF CONFORMANCE
APPENDIX A — AUDIT CRITERIA

1.0  EXECUTIVE SUMMARY

In keeping with the Government of Canada’s ongoing commitment to climate change adaptation and providing scientifically-sound information to support adaptation planning and decision-making, in 2011, Fisheries and Oceans Canada received $16.5 million over five years for the Aquatic Climate Change Adaptation Services Program.

The overall objective of this five year program was to advance knowledge and understanding of the risks, impacts and opportunities created by climate change for Fisheries and Oceans Canada’s mandated areas of responsibility and to begin to develop science-based adaptation tools necessary to support the consideration of climate change in departmental decision-making. To accomplish this, the Program carried out two risk assessments to identify the potential risks and opportunities, created by climate change, for the Department's mandated areas of responsibility. These risk assessments were, the Science Risk Assessment and the Socio-Economic Risk Assessment. The results of these risk assessments were used to identify the priority areas of research and adaptation tool development under the Programs two competitive funding envelopes; the Understanding Climate Change Impacts Fund and the Adaptation Tools Fund.

The audit objective was to provide assurance that the Program has adequate controls/mechanisms in place to ensure that key climate change risks, impacts and opportunities are identified and incorporated into the Program’s decision-making to effectively support the Department in understanding and adapting to climate change.

Based on the audit findings, our opinion is that the Program has adequate controls/mechanisms in place to ensure that key climate change risks, impacts and opportunities are identified and incorporated into its decision-making and to effectively support the Department in understanding and adapting to climate change. The audit found that while the program is well structured and implemented, there is an opportunity for improvement in the area of its external collaboration and continuous monitoring. The recommended improvements are as follows:

  • Formally develop and document a systematic process for engaging in collaborative opportunities with external stakeholders outside of the competitive funding envelopes.
  • Formally assign and document the process for active monitoring and updating of key climate change risks, impacts, and opportunities, as well as, reviewing the Program’s initial risk assessment.

Management Response

Management is in agreement with the audit findings, has accepted the recommendations included in this report, and has developed a management action plan to address them.  The management action plan has been integrated in this report. 

2.0 BACKGROUND

In 2010, the Office of the Auditor General tabled a report of the Commissioner of the Environment and Sustainable Development which included a chapter entitled "Adapting to Climate Impacts". The Commissioner of the Environment and Sustainable Development audit examined five key federal departments, including Fisheries and Oceans Canada, whose mandates may be significantly affected by climate change and identified and assessed risks from climate change to their areas of responsibility. One of the recommendations in the report was that departments should identify the adaptation measures they need to take, in response to the risks that climate change presents for their areas of responsibility.

Following the Commissioner of the Environment and Sustainable Development audit, in 2011, the Government of Canada provided funding to a group of nine Departments/Agencies in the amount of $148.8 million towards the implementation of programs in support of helping Canadians adapt to a changing climate under the Government of Canada’s renewed Clean Air Agenda initiative. Fisheries and Oceans Canada received $16.5 million for a period of five years from this fund for its Aquatic Climate Change Adaptation Services Program. The Program is in its fourth year.

The Program’s objectives are: to advance knowledge and understanding of the risks, impacts and opportunities created by climate change for Fisheries and Oceans Canada’s mandated areas of responsibility, and to begin to develop science-based adaptation tools necessary to support the consideration of climate change in departmental decision-making. To accomplish these objectives, the Department began by developing four detailed regional risk assessments to identify key vulnerabilities to climate change. The Program also established two competitive funding envelopes for internal projects addressing a variety of climate change considerations. The two envelopes are the Understanding Climate Change Impacts Fund and the Adaptation Tools Fund. These funds are awarded through a competitive process involving a series of review committees that assess and evaluate each proposal against a set of criteria. The criteria, including the definition of priority areas of research and adaptation tool development, were developed specifically for the competitive funding envelopes and were based on the overall program objectives, the results of the risk assessments, and the application of the Framework for Identifying Fisheries and Oceans Canada Climate-Sensitive Decisions as well as the completed inventory of existing Canadian and international climate change adaptation tools.

The information and services provided by this Program benefit the Department’s stakeholders by facilitating the development of adaptive management strategies for activities occurring in the three oceans that surround Canada and its major inland watersheds.

3.0 AUDIT OBJECTIVE

The objective of the audit was to provide assurance that the Aquatic Climate Change Adaptation Services Program has adequate controls/mechanisms in place to ensure that key climate change risks, impacts and opportunities are identified and incorporated into the Program’s decision-making to effectively support the Department in understanding and adapting to climate change.

4.0 AUDIT SCOPE

The scope of the audit was risk-based and focussed on the execution of the Program, its risk assessment process and integration of such into the decision-making and project selection process.

5.0 AUDIT APPROACH

The audit team carried out its mandate in accordance with Treasury Board’s Policy on Internal Audit and the Internal Audit Standards for the Government of Canada.  The audit employed various techniques including a risk assessment of the audit entity, interviews, as well as reviews and analysis of documentation and information.

6.0 AUDIT FINDINGS

This section provides the observations and recommendations resulting from the audit work carried out. While the audit was conducted based on the lines of enquiry and audit criteria identified in the planning phase, this report is structured along the following main themes:

  • External Collaboration;
  • Program Management; and
  • Results and Performance.

For conclusions by audit criterion, please refer to Appendix A.

Based on the audit work performed and our professional judgment, the risk associated with each observation was rated using a three-point scale. The risk ranking (high, moderate, low) is based on the level of potential risk exposure we feel may have an impact on the achievement of Fisheries and Oceans Canada’s objectives, and is indicative of the priority Management should give to the recommendations associated with that observation. The following criteria were used in determining the risk exposure level:

Criteria Table
High Controls are not in place or are inadequate.
Compliance with legislation and regulations is inadequate.
Important issues are identified that could negatively impact the achievement of program/operational objectives.
Moderate Controls are in place but are not being sufficiently complied with.
Compliance with central agency/departmental policies and established procedures is inadequate.
Issues are identified that could negatively impact the efficiency and effectiveness of operations.
Low Controls are in place but the level of compliance varies.
Compliance with central agency/departmental policies and established procedures varies.
Issues identified are less significant but opportunities that could enhance operations exist.

6.1 External Collaboration

Since the effects and implications of climate change impact both government and society, external collaboration facilitates the understanding of external factors and their impacts on the Program’s objectives, risks and priorities. For effective external collaboration all parties involved must share the same expectations and ensure their respective needs, constraints and risks are understood and managed.

Those departments receiving funds under the Clean Air Agenda’s mandate to implement programs in support of helping Canadians adapt to a changing climate, are expected to collaborate with the other Clean Air Agenda Adaptation Theme departments/agencies, research institutions, and other levels of government.

Observations Section 6.1
Observations
Low

6.1.1 The Program actively participates in collaborative opportunities with other Clean Air Agenda Adaptation Theme departments/agencies and with external stakeholders (such as research institutions and other levels of government). There is an opportunity however, to develop a formalized process for engaging in collaborative opportunities with external stakeholders outside of the project selection and risk assessment processes.

The Program’s internal collaboration with other departmental sectors is very strong. The Program included a broad range of sectors in both the risk assessment and project selection processes. Further, in interviews with various departmental sectors during the planning phase of this audit, the Program’s strong collaboration with and amongst internal stakeholders (e.g., sectors and regions) was consistently highlighted as a best practice that should be shared with the rest of the Department.

External collaboration was examined in detail through interviews and document review with the following methods of collaboration being identified:

Formal Methods of Collaboration

The Program actively participated in all formal collaborative opportunities available to it, as identified below, and utilized the information received in these collaborative exchanges to further strengthen the Program’s risk assessments and inventory of existing climate change adaptation tools.

The Program actively participated in both of the Interdepartmental Steering Committees led by Environment Canada; the Director General Management Committee and the Director General Adaptation Policy Steering Committee.

The Program is also active in the Natural Resources Canada Adaptation Platform; a virtual area created by the Natural Resources Canada Climate Change Impacts and Adaptation Division. This is a forum where key groups from government, industry and professional organizations collaborate on adaptation priorities. It provides an environment for sharing of knowledge, capacity and financial resources. The Platform includes coordinated forums, as well as, a variety of working groups. Fisheries and Oceans Canada is involved in many of the communities within the platform and is active in posting and responding to posts. Fisheries and Oceans Canada also actively monitors responses, comments and questions related to items they post. Fisheries and Oceans Canada’s Economic Analysis and Statistics Directorate also participates actively in the Platform’s Economic Working Group by sharing information and practices at Fisheries and Oceans Canada with respect to the socio-economic impacts of climate change on adaptation, as well as identifying opportunities for interdepartmental collaboration for projects or initiatives.

Another formal external collaboration tool employed by the Program was surveys of external climate change work. As part of the risk assessment process within the Program, all assessments and workshops considered and integrated results obtained from external research and papers gathered via a web-based survey.

Similarly, under the Inventory of Existing Climate Change Adaptation Tools initiative, a survey of adaptation tools from a range of information sources such as academic literature-based resources, and a web-based survey of Canadian and international organizations (e.g., Environment Canada, Public Safety Canada, Natural Resources Canada, National Research Council, the United States and United Kingdom National Governments, trans-national organizations etc.) was carried out. This was supplemented by interviews with government departments and agencies with similar mandates to Fisheries and Oceans Canada and key Canadian and international organizations.

Finally, the Competitive Funding Envelopes project selection process encouraged collaboration by including the level of internal and external collaboration as an element in the project evaluation criteria used to assess and rate each project proposal. The level of collaboration within the Competitive Funding Envelopes projects was one of the Program’s performance indicators and the actual result was reported in the Department’s 2013-14 Departmental Performance Report with $0.25 in external funding and $0.55 of in-kind contributions for every dollar of program spending. This exceeded the Program’s original target rates for such collaboration.

Informal Collaboration

The Program also took advantage of informal collaborative opportunities. Based on discussions, interviews and subsequent review of supporting documentation, the Program has been actively exchanging climate change information and knowledge with other internal and external stakeholders via informal communications (such as emails and phone conversation).

Also, Fisheries and Oceans Canada scientists actively participated in many scientific reports and workshops on climate change, such as contributions to a chapter of the 5th Intergovernmental Panel on Climate Change Report as well as making presentations on risk assessments to various organizations (e.g., Transport Canada and the Canadian Meteorological and Oceanographic Society).

Fisheries and Oceans Canada has also been involved in a number of other external collaborations, outside of the project selection process, where mutually beneficial opportunities were identified. While the Program is internally focused, when it comes to collaboration with external parties outside of the project selection process, there is no formal process in place to evaluate/consider collaborative opportunities. This has resulted in external collaborations being done on an ad hoc basis. This increases the risk of there being a perception of a lack of transparency or fairness. Developing and documenting a process for engaging in collaborative opportunities with external stakeholders would help to mitigate this risk and will provide a record of activities that could inform decisions on the allocation of additional resources in priority areas.

Recommendations / Managment Action Plan
Recommendation Management Action Plan
R-#1. It is recommended that the Assistant Deputy Minister, Ecosystems and Oceans Science develop and document a formalized, systematic process for engaging in collaborative opportunities with external stakeholders outside of the competitive funding envelopes. The Program will develop or modify existing processes to provide for an opportunity to evaluate/consider collaborative opportunities with external stakeholders.
Office of Primary Interest: Ecosystems and Oceans Science Sector
Due Date: April 30, 2015

6.2 Program Management

Internal controls help ensure that objectives are met. In an environment with well-designed controls, risks to the objectives of the program are proactively identified, reviewed and considered. Therefore, it is crucial to develop an internal control environment to ensure all controls and mechanisms in place are performing effectively.

In the context of this audit, it is important for the Program to demonstrate that it has the controls and mechanisms in place to address areas of unacceptable risk to the project selection process and that within its risk assessment processes both risks and opportunities are given appropriate consideration. Also, the Program would be expected to formally document and assign the process for monitoring and updating key climate change risks, impacts and opportunities associated with the risk assessments. In reference to the project selection process itself, the Program would be expected to have a structure in place to communicate all aspects of the process and its decisions to the relevant key stakeholders.

Observations Section 6.2
Observations
Low 6.2.1 Effective and appropriate controls/mechanisms are in place to support the integrity/credibility of the project selection process. Within the Program's risk assessment process however, there is opportunity for formalizing controls to ensure that it is updated to reflect significant changes and reviewed on a regular basis.

Project Selection Process

Based on interviews, process mapping and walkthroughs of the Program’s project selection process, the Program has demonstrated that there is a process in place for project selection and adequate controls exist to mitigate the areas of risk identified.

While the selection process involves three separate departmental committees, the audit team did not find any significant duplication of effort. Each committee involved within the project selection process played a unique and significant role in the assessment of, and resulting decision for, each project proposal as follows:

  • The National Climate Change Adaptation Steering Committee focuses on the relevance of the proposed project related to the overall Program objectives and the priorities identified in the call for project proposals;
  • The National Climate Change Adaptation Technical Review Committee focuses on the technical aspect such as methodology, deliverables, data management, etc.; and
  • The National Science Directors Committee provides a nation-wide perspective, and ensures an appropriate regional balance. They are also responsible for recommending the final approval of the projects to the Assistant Deputy Minister, Ecosystems and Oceans Science.

Having multiple review committees with unique and significant roles in the project selection process is identified as a best practice that could be shared with the rest of the Department. Each of these committees also had documented governance structures in place that identify their roles, responsibilities and the committee’s membership.

Communication of the Project Selection Process and Results

As a result of interviews and document review, the audit found that the Program’s call for project proposals, including the project selection process, its criteria and the results of this process were adequately communicated to all relevant internal stakeholders. The selection process, including the call letter and established criteria, against which each project would be evaluated, were made available to prospective project proponents by way of e-mail communication from the office of the Assistant Deputy Minister of Ecosystems and Oceans Science, and the Department’s weekly ‘In the Loop’. The resulting recommendations or decisions, from this process, were then communicated to the project proponents via emails. Further, both the process and decisions were also communicated to the senior management of all relevant sectors through their participation in the various committees.

Risk Assessment Process

The Program has identified key climate change risks, impacts, and opportunities through various assessments and studies. This work resulted in the delivery of two products:

  • The Science Risk Assessment, which helped to ensure that the key climate change risks and opportunities were identified; and
  • The Socio-Economic Risk Assessment, which provided a balance of both socio-economic risks and opportunities and categorized them as such. Both of these assessments were used to inform the decision-making in the project selection process.

Further, based on interviews, the Program reviews current literature and research in the area of climate change and incorporates this information into its risk assessment process when necessary. However, no position or group has been formally assigned the responsibility for active monitoring of this type of research and reports. This could result in some key changes in climate change risks or opportunities possibly being missed. Also, the Program has indicated, through interviews, that it plans on reviewing the risk assessment again, however there is no formal process in place for such a review. By identifying and documenting a process and timing for such a review, it will ensure that the risk assessment is reviewed on a regular basis, as appropriate for the Program.

Recommendations / Managment Action Plan
Recommendation Management Action Plan
R-#2. It is recommended that the Assistant Deputy Minister, Ecosystems and Oceans Science formally:
  • Assign responsibility for active (continuous) monitoring and updating of key climate change risks, impacts, and opportunities to a position or group.
  • Document a process to review the risk assessment (timing and details to be determined by the Program).
Responsibility for active monitoring and assessing of climate risks, impacts and opportunities will be assigned within the sector, and a process to review the risk assessment will be documented.
Office of Primary Interest: Ecosystems and Oceans Science Sector
Due Date: April 30, 2015

6.3 Results and Performance

Internal controls should encompass the processes in place to monitor financial and operational performance on an ongoing basis and the degree to which performance results are fed back into the planning process. More specifically, management should monitor the actual performance against planned results and adjust course as needed.

Within the context of the Program, the Program’s management must monitor and assess projects to ensure their stated deliverables are on-track and continued alignment with the Program’s objectives.

Observations Section 6.3
Observations
Low

6.3.1 Program management monitors and assesses the projects to ensure their continued alignment with the Program's objectives.

Based on interviews and document review, the Program ensures that project proposals are aligned with Program objectives. Further, the Program has demonstrated that it has an adequate monitoring process in place to ensure that individual projects remain aligned with objectives identified in the project proposals. Document review confirmed that all approved projects submitted final or progress reports that reported on the deliverables identified within the project proposal.

To ensure on-going alignment of the overall portfolio of projects, the Program carries out an annual priority setting process to identify necessary changes to the project selection priorities to address areas that may require additional work. Based on this priority setting process, the Program makes the necessary adjustments to target new projects that address these priorities, thereby ensuring Program objectives are met.

7.0 AUDIT OPINION

Based on the audit findings, our opinion is that, the Program has adequate controls/mechanisms in place to ensure that key climate change risks, impacts and opportunities are identified and incorporated into its decision-making and to effectively support the Department in understanding and adapting to climate change. The audit found that while the program is well structured and implemented, there is an opportunity for improvement in the area of external collaboration and continuous monitoring. The recommended improvements are as follows:

  • Formally develop and document a systematic process for engaging in collaborative opportunities with external stakeholders outside of the project selection process.
  • Formally assign and document the process for active monitoring and updating of key climate change risks, impacts, and opportunities, as well as, reviewing the risk assessment.

8.0 STATEMENT OF CONFORMANCE

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The extent of the examination was planned to provide a reasonable level of assurance with respect to the audit criteria. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with Management. The opinion is applicable only to the entity examined and within the scope described herein. The evidence was gathered in compliance with the Treasury Board Policy and Directive on Internal Audit. The audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Quality Assurance and Improvement Program. The procedures used meet the professional standards of the Institute of Internal Auditors. The evidence gathered was sufficient to provide Senior Management with proof of the opinion derived from the internal audit.

APPENDIX A – AUDIT CRITERIA

Based on a combination of the evidence gathered through documentation examination, analysis and interviews, each of the audit criteria listed below was assessed and a conclusion for the audit criteria was determined using the following definitions:

Conclusion per Audit
  Conclusion on Audit Criteria Definition of Opinion
1 Criteria Met – Well Controlled Well managed or no material weaknesses noted, controls are effective.
2 Criteria Met with Exceptions – Controlled Requires minor improvements.
3 Criteria Met with Exceptions – Moderate Issues Requires improvements in the areas of material financial adjustments, some risk exposure.
4 Criteria Not Met – High Impact – Significant Improvements Requires significant improvements in the area of material financial adjustments, serious risk exposure.

The following are the audit criteria against which conclusions were drawn. In cases where significant improvements and/or moderate issues were observed, these were reported in the audit report.

Audit Criteria / Conclusions
Audit Criteria Conclusion on Audit Criteria Examples of Key Evidence/ Observations
Line of Enquiry 1 – External Collaboration
Criterion 1.1: Adequate communication processes/mechanisms, such as working groups and committees, exist and support sharing of timely, relevant, and reliable information between Program, Environment Canada, other Clean Air Agenda Adaptation Theme departments/agencies, and external collaborators. 1 6.1.1
Criterion 1.2: Procedures exist to ensure input and feedback from Environment Canada, other Clean Air Agenda Adaptation departments/agencies, and external collaborators (e.g., research institutions) are considered, acted on, and/or integrated to facilitate identifying climate change risks, impacts, and opportunities and adaptation mechanisms. 1 6.1.1
Criterion 1.3: Program leverages, where appropriate, collaborative opportunities to facilitate sharing and integration of external stakeholder's (other federal departments or levels of government, research institutions) input into the Program's decision–making process or science–based products. 2 6.1.1
Line of Enquiry 2 ' Program Management
Criterion 2.1: Effective and appropriate controls/mechanisms are in place to support the integrity/credibility of the project selection process. 1 6.2.1
Criterion 2.2: Program management has identified and incorporated the key climate change risks, impacts and opportunities into the risk assessment and the project selection processes.

Sub–Criterion 2.2.1: Program management has identified key climate change risks, impacts, and opportunities.

1 6.2.1

Sub–Criterion 2.2.2: Program management has incorporated the identified key climate change risks, impacts, and opportunities into the risk assessment and project selection processes.

1 6.2.1

Sub–Criterion 2.2.3 Responsibility for monitoring and updating key climate change risks, impacts, and opportunities in the decision–making process is clear and communicated.

2 6.2.1

Sub–Criterion 2.2.4: Active monitoring and updating of key climate change risks, impacts, and opportunities in the process of risk management and project selection is demonstrated.

2 6.2.1
Criterion 2.3: The Program's project selection and/or revised decisions are clearly articulated and communicated to key stakeholders. 1 6.2.1
Line of Enquiry 3 – Results and Performance
Sub–Criterion 3.1: Program management monitors and assesses projects to ensure their continued alignment with the Program's objectives. 1 6.3.1