Archived – Audit of Supporting Statistical Information on Fisheries


Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Project Number 6B205
Preliminary Survey Report
March 18, 2010

Table of Contents

1.0 Executive summary

Introduction

The Internal Audit Directorate was to conduct an audit of Supporting Statistical Information on Fisheries within Fisheries and Oceans Canada (DFO) in accordance with the DFO Multi-Year Risk-Based Audit Plan 2009-10 to 2011-12.

Initially set-up as a follow-up audit on the Audit of the Management Control Framework Supporting Statistical Information on Fisheries completed in 2006, the audit was realigned to complement the Audit of the Management Control Framework Supporting Senior Management Committee Information for Decision-Making completed in 2008 and to address the information for decision-making risk, which was identified as the second highest risk in the Departmental Corporate Risk Profile. Consequently, the renewed audit objective focused on ensuring that adequate controls are in place to ensure quality of information used to support decision-making pertaining to fisheries management, a key activity in achieving DFO’s strategic outcomes. 

Based on DFO Program Activity Architecture 2009-10, fisheries management is a program activity comprised of six sub-activities. Following a preliminary risk assessment, Resource Management was selected as the area of primary audit interest of the audit engagement as its outputs have a high impact on sustainability of fisheries and economic viability.

Background on the audit entity

Resource Management is responsible for making recommendations in support of fisheries management decisions such as total allowable catch, quotas allocation & sharing, fishing access including opening and closure of fishing areas or season etc. For major commercial fisheries, these fish harvest decisions are either made by the Minister or the Regional Directors General. Ministerial authority is typically required for the multi-regional fisheries, international fisheries and fisheries that are deemed “sensitive” such as lobster fisheries.

Decision memoranda along with relevant attachments are required when seeking decisions from the Minister or the Regional Directors General. The purpose of these memoranda is to provide information about a fishery or an issue along with options and recommendations for decision-making. The decision memoranda ensure that all stakeholders’ perspectives are reflected and that positive and negative outcomes for each option are described. Resource Management is responsible and accountable for the development of these memoranda. Sectors providing the information are accountable for the quality of their information products. The main providers of information typically involved in these decisions are the Science sector for its environmental information and the Policy sector1 for social and economic information.

Preliminary Survey Findings

The audit team referred to the Treasury Board Information Management Framework which provided a set of quality attributes associated with information quality. The Framework defines information quality as “characteristics or attributes of information that lead readers/users to trust it, including accurate, complete, reliable, understandable, relevant, current, accessible and timely”. This Framework was used by the audit team as a reference for the assessment of controls as it provided guidelines on how to achieve each of the eight components of information quality.

Based on preliminary audit work, Resource Management, Science, and Policy have adequate controls in place to ensure quality of information used in decision memoranda pertaining to the fish harvest decisions. The audit team is referring here to the existence of controls only as testing of control effectiveness was not conducted as part of the audit work.

Conclusion

Based on the results of the risk assessment of its preliminary findings, the audit team found no major risks in the context of information quality associated with the fisheries management decision memoranda. Though minor risks were identified, the audit team considered that they were addressed by sufficient mitigation strategies. The audit team is satisfied with the relevant control framework in place for Resource Management, Science and Policy and Economics. 

Given that adequate controls are in place to ensure the quality of information for decision-making pertaining to fisheries management decision memoranda, the audit team considered that pursuing a detailed examination in the conduct phase would not add value at this time. As the Internal Audit Directorate needs to conduct audit work only on areas of high risk and materiality to optimize the use of its resources, the Directorate recommends concluding the internal audit engagement at this stage. 

2.0 Introduction

2.1 Source of the internal audit engagement

The Internal Audit Directorate was to conduct an audit of Supporting Statistical Information on Fisheries within Fisheries and Oceans Canada (DFO) in accordance with the DFO Multi-Year Risk-Based Audit Plan 2009-10 to 2011-12.

Haut de la page

2.2 Realignment of the internal audit engagement

Initially set-up as a follow-up audit on the Audit of the Management Control Framework Supporting Statistical Information on Fisheries2completed in 2006, the audit was realigned to complement the Audit of the Management Control Framework Supporting Senior Management Committee Information for Decision-Making3 completed in 2008 and to address the information for decision-making risk, which was identified as the second highest risk in the Departmental Corporate Risk Profile. Consequently, the renewed audit objective focused on ensuring that adequate controls are in place to ensure quality of information used to support decision-making pertaining to fisheries management, a key activity in achieving DFO’s strategic outcomes.

Based on the DFO Program Activity Architecture 2009-10, fisheries management is a program activity comprised of six sub-activities: Resource Management, Aboriginal Policy and Governance, Salmon Enhancement Program, International Fisheries, Conservation and Protection, and Aquaculture. Following a preliminary risk assessment, the audit team opted to focus on Resource Management as it ranked highest in significance related to the information for decision-making risk. This risk assessment methodology is further detailed in the Internal Audit Risk Assessment section of this report.

As per the Program Activity Architecture, fisheries management is supporting DFO strategic outcome “Sustainable Fisheries & Aquaculture”. According to DFO, the principles of sustainable development maintain that social, economic and environmental issues are interconnected and must be integrated into the decision-making process. In other words, information for decision-making pertaining to fisheries management should also include these three components. In the context of fisheries management, social and economic information is generally provided by the Policy sector4 whereas environmental information is provided by the Science sector.

3.0 Background on the activity

3.1 Mandate of Resource Management

Resource Management delivers policies, programs and plans in partnership with Aboriginal groups and industry, in order to manage, protect and conserve fisheries resources. Resource Management ensures sustainability and provides for the allocation and distribution of harvestable surpluses among those dependent on the resource. Resource Management integrates input from other related the DFO program areas, other government departments and stakeholders to develop and implement fishing plans (i.e. Integrated Fisheries Management Plans) for fisheries. These plans integrate conservation, management and scientific objectives, and spell out the required measures to conserve and manage fisheries resources. Allocations between user groups and fleet sectors are also an important aspect of resource management. Fisheries are managed by allocating quotas to entire fleet sectors that fish either competitively or through enterprise or individual allocations, and by controlling effort, escapement or by-catch.

3.2 Activity Risk Assessment

The Fisheries and Aquaculture Management sector risk profile identified information for decision-making (the risk that sufficient and appropriate information will not be available on a timely basis to support decision making) as a moderate to high risk which was a rating consistent with the DFO Corporate Risk Profile 2008-09. For Resource Management, the information for decision-making risk was more specifically defined as the risk that decision makers will not have access to reliable information (biological, economic, and social) in a timely manner.

Resource Management needs reliable information to support their recommendations in support of decisions to stakeholders with widely diverging agendas. Increased transparency and predictability of decisions is expected. There is also an increasing level of scepticism among stakeholders in the information provided by the Department.

3.3 Governance Structure

As mandated under the Fisheries Act, the Species at Risk Act and related regulations, the role of the Resource Management Directorate encompasses conservation, responsible and sustainable harvesting practices, and equitable distribution of the resource among user groups. The directorate operates under Fisheries and Aquaculture Management, a highly operational and decentralized sector. The Resource Management regional offices are located throughout the seven administrative regions of DFO. While each regional office is responsible for their respective regional fisheries, the National Headquarters is responsible for transboundary and straddling stocks. Nonetheless, the national office also provides policy direction, ensures national consistency and provides inter-regional coordination and liaises with other DFO sectors and government departments and central agencies.

3.4 Description of the Management Framework

Resource Management is responsible for making recommendations in support of key decisions pertaining to fisheries management, including:

  • Total Allowable Catch

  • Quotas allocation & sharing

  • Fishing access including opening and closure of fishing areas or season

  • Fleet obligations and gear requirements

  • Changes and exceptions (i.e. requests from fishing industry outside of licensing policies)

For major commercial fisheries, these fish harvest decisions are not made directly by Resource Management; depending on the nature of the decision, these key decisions are either made by the Minister or the Regional Directors General. Ministerial authority is typically required for the following:

  • Multi-regional fisheries

  • International fisheries

  • Fisheries that are deemed “sensitive” such as lobster fisheries

  • Exception cases or major changes

Decision memoranda along with relevant attachments are required when seeking decisions from the Minister or the Regional Directors General. The purpose of these memoranda is to provide information about a fishery or an issue along with options and recommendations for decision-making. The decision memoranda ensure that all stakeholders’ perspectives are reflected and that positive and negative outcome for each option is described. Resource Management is responsible and accountable for the development of these memoranda. Sectors providing the information are accountable for the quality of their information products.

3.5 Description of key processes

Three key processes pertaining to fisheries management decision memoranda were identified for major commercial fisheries. As part of the planning phase, the audit team has documented these key processes and controls which are represented through process maps that can be found in Appendices A, B and C.

The development of the decision memoranda is generally initiated by the Post-Seasonal Review of a fishery to present recommendations for next cycle’s fishery management. However, it can also be triggered during the season when an important issue arises (i.e. new information found by Science).

An advisory committee is established for each major commercial fishery. The committee, led by Resource Management, includes participants from regions involved, applicable sectors (e.g. Science, Policy, Oceans, Legal, etc.) and external stakeholders (e.g. Industry, First Nations, Environmental Groups, etc.). As part of the Post-Seasonal Review, all participants of the advisory committee meet through a consultation meeting. Prior to the consultation meeting however, Resource Management would request information from all applicable sectors, namely, Science and Policy. After the consultation meeting, information from all stakeholders is consolidated by Resource Management.

Generally, the Resource Management officer dedicated to the fishery in question is responsible for writing decision memoranda and ensuring that all needed information is incorporated to support the recommendations. The draft memoranda is then shared with all internal stakeholders involved to ensure that information has been well interpreted and is revised by various levels of management before it is sent to the decision-maker that is, the Minister or Regional Directors General.

Haut de la page

4.0 Engagement planning activities

4.1 Approach to Audit Planning

The purpose of the audit planning phase is to understand the audit entity, the related processes and controls as well as to identify the main areas of risk. Furthermore, this phase is also used to determine whether to proceed or not with the conduct phase based on the information collected. Methodologies to perform the work during the planning phase included interviews with management and staff as well as reviews of key documents.

To gain a general understanding of the audit topic and related activities, the audit team conducted interviews with management and staff from Fisheries and Aquaculture Management, Science, and Policy sectors. As part of the planning phase, some interviews were also conducted with other sectors to better define our potential audit scope. Interviewees were selected in administrative regions and at National Headquarter.

As part of the planning phase, the auditors reviewed key departmental documents such as the DFO Program Activity Architecture 2009-10, the DFO Corporate Risk Profile 2008-09 including related sector and directorate-level risk profiles along with other relevant departmental policies and reports. In addition, the audit team also reviewed relevant legislation as well as Treasury Board policies, standards and guidelines. Finally, the team also examined and analysed relevant documentation provided by the interviewees including examples of decision memoranda and information products used in developing these decision memoranda.

The audit team also mapped out key processes related to the decision memoranda to gain a clear understanding of the controls in place and key players involved.

4.2 Internal Audit Risk Assessment

A risk assessment was conducted to direct the audit towards the area of greatest value-added for DFO. Though fisheries management was established as the focus of the audit engagement, this program activity was deemed too broad in the context of information for decision-making. As a result, the audit team used a two-fold risk assessment methodology to select the audit entity and subsequently identify and assess the risks associated with this audit entity. 

Risk Assessment Part I – Selection of audit entity

Through the review the DFO Corporate Risk Profile along with sectors and related directorate-level risk profiles, a preliminary list of audit entities related to fisheries management was established. 

Based on the potential risk drivers (causes and factors), the potential consequences and the current mitigations in place for each audit entity, the audit team rated the significance of the risks in relation to their effects. Ratings were based on information obtained through preliminary interviews, review of documentation obtained during the preliminary survey phase, and the analysis and professional judgment of the audit team. As a result, Resource Management was selected as the area of primary audit interest of the audit engagement as its outputs have a high impact on sustainability of fisheries and economic viability. 

Risk Assessment Part II – Identification of key controls related to information quality and assessment of associated risks

Once the audit entity was selected, the audit team examined its key decisions, related processes and controls to identify key risk areas to pursue in the conduct phase. The audit team found that Resource Management is responsible for making recommendations in support of key decisions pertaining to fisheries management and for presenting them in decision memoranda when seeking decision from the Minister or the Regional Directors General.

Through interviews with key stakeholders and documentation review, the audit team examined the development process of the decision memoranda produced by Resource Management including how the various types of information feed in to them. As mentioned previously, the information that feeds into the decision memoranda derives from internal and external sources. The internal sources of information covered by the audit team were Science and Policy.

As part of the preliminary survey, the audit team referred to the Treasury Board Information Management Framework which provided a set of quality attributes associated with information quality. The Framework defines information quality as “characteristics or attributes of information that lead readers/users to trust it, including accurate, complete, reliable, understandable, relevant, current, accessible and timely”. This Framework was used by the audit team as a reference for the assessment of controls as it provided guidelines on how to achieve each of the eight components of information quality. 

Through interviews and review of documents, the team examined the controls in place to ensure information quality for all three entities. The team looked at what information products were used to make these decisions and the controls in place to ensure the quality of these information products (e.g. signoff process). The team also looked at what types of information were contained in these information products and the controls in place to ensure the quality of the information. Findings associated with this exercise are presented in the following section.

Haut de la page

5.0 Preliminary Survey Findings

Based on preliminary audit work, Resource Management, Science, and Policy have adequate controls in place to ensure information quality of decision memoranda pertaining to fish harvest decisions. The audit team is referring here to the existence of controls only as testing of control effectiveness was not conducted as part of the audit work.

In developing the decision memoranda, Resource Management is overall satisfied with the quality of information obtained. Some concerns were raised about timeliness in obtaining the information however, the risks associated with this concern was rated as low to medium significance by the audit team as it was addressed by sufficient mitigating actions, as follows.

  • Science peer review process might be lengthy but is important to ensure information quality. In terms of timeliness, the Science advisory schedule process allows to prioritize requests based on risks and significance. For urgent matters, Science has a Special Response process in place. Though information may not be as accurate, there will always be a trade-off between accuracy and timeliness.

  • Policy has no formal agreements or formal planning process to prioritize requests for information products. Requests are often ad hoc and the branch plans according to the priority of the request and their own internal capacity. Based on interviews conducted, Resource Management did not raise significant concerns in this area as good cooperation between the two sectors exists. Furthermore, Resource Management tries to plan and notify Policy in advance about upcoming information requests. Suggestions to management could be made, however this area does not warrant further audit effort.

The next three sub-sections provide additional or complementary information by sector on the quality attributes reviewed.

5.1 Resource Management

DFO manages fisheries in accordance with the roles and responsibilities outlined in the Fisheries Act, taking into account environmental sustainability, economic viability, and the inclusion of stakeholders in the decision-making process. Resource Management is responsible for the development of decision memoranda that are used to inform decision-makers about options and recommendations related to fisheries management. Decisions are subsequently made based on the information contained in these memoranda. Therefore, it is critical that this information is of quality to ensure that decisions are made based on the best information possible.

Overall, the audit team found that Resource Management has adequate controls in place to ensure information quality of decision memoranda. In developing these memoranda, Resource Management uses information deriving from both internal and external sources that are authoritative (i.e. Science) and/or commonly recognized (i.e. fishing industry representatives).

This practice is consistent with DFO fisheries management priorities as it increases the reliability of information and ensures that all viewpoints are included in the decision-making process. Resource Management develops options and recommendations by consolidating the information from various sources. To ensure that information has been well interpreted, the directorate always shares the decision memoranda with the internal stakeholders involved. Furthermore, Resource Management also sends the decision memoranda to be signed off by the sectors’ senior management, thus, approving the information related to their sector and related suggestions. Decision memoranda also undergo quality assurance performed at various levels of management before they are sent to the decision-makers.

Concerns and opinions from external stakeholders are also reflected in the decision memoranda. However, as this information is not factual, it is indicated as such. Though decision memoranda are not shared with external stakeholders, minutes of the consultation meetings are made available to all participants and generally published on the DFO website for major commercial fisheries. This ensures that information is not biased and increases transparency of the decision-making process.

Resource Management uses the departmental Electronic Knowledge Management Environment (EKME) and the Ministerial and Executive Correspondence Tracking System (MECTS) to create and track decision memoranda protecting them from unauthorized alterations. The decision memoranda adhere to a departmental template that sets out specific format requirements making them clear in meaning and easy to read to the decision-maker. Adhering to a departmental template also ensures regional consistency in terms of presentation format. Though there is currently no documented guideline to provide assistance on the content of the decision memoranda, steps are taken by the Director General of Resource Management to develop such template.

5.2 Science

Resource Management relies on science advice and information to develop decision memoranda pertaining to fisheries management. Scientific information feeding into the decision memoranda is generally presented in the form of scientific papers and reports.

To develop their information products, Science uses data collected by their own sector and data collected by the regional statistics function5. Science has a data quality program using technology to ensure integrity of the data collected in-house (i.e. electronic sensors). Furthermore, science data undergo stringent sampling and testing methodologies.

Given that science is a practice founded on knowledge-base capable of resulting in a predictable outcome, information products are developed by scientists that have subject matter expertise and knowledge. Science also has documented guidelines to provide guidance on the development of their information products for Resource Management.

Science uses a formal peer review process to ensure the quality of its information products. Peer review is a formal process that examines the data and methodology used to create the scientific paper and ensure its accuracy. Science peer review abides by the Treasury Board Science Advice for Government Effectiveness principles and is published on DFO website. These principles provide guidance on how to improve science-based decision making, and a series of concrete guidelines to facilitate the adoption of the principles espoused. The peer review also includes internal and external participants rendering the process impartial and credible. The Canadian Science Advisory Secretariat is responsible to coordinate the peer review process for DFO. Terms of references between Science and Resource Management exist for any scientific advice request and clearly specify information needs, timeframes, and expectations in terms of deliverables. These terms of references are also published on DFO website.

Science has a documented process for information requests that prioritize and post them on a schedule available on DFO website. Requests submitted throughout the year are considered and integrated as required in this schedule. Information products provided to Resource Management adhere to this schedule. For urgent or ad hoc requests, Science has also a documented Special Response Process. This formal process ensures that information products are delivered to Resource Management in a timely manner.

5.3 Policy

In addition to scientific information, Resource Management also considers social and economic information when developing decision memoranda pertaining to fisheries management.  Policy presents this information in a vast array of information products.

To develop their information products, Policy uses data collected from external sources and data collected from the statistics function6.  When external sources are used, Policy employs sources of recognized credibility (i.e. Statistics Canada). In cases of data or information uncertainty, Policy would use a proxy and specify their limitations and confidence levels in the information products.  Based on this appreciation of assurance, the decision-makers can decide whether to include or not the information in their decisions.

Due to the vast array of their information products, Policy does not have documented guidelines for the development of all information products.  However, some of their recurrent and standard information products have documented guidelines.  The format of their information products varies according to the request of Resource Management.  As information requests are generally not documented in terms of references, both Resource Management and Policy rely on close cooperation. Although both entities at the regional level have expressed that having formal agreements would add administrative burden, Policy Sector, National Headquarter recognizes that a risk exists and is taking steps to formalize information requests for social and economic analysis.

Policy does not have a systematic and formal peer review process like Science.  External peer review is performed only on a needed basis when their information products involve sensitive or high risk issues.  Similar to Resource Management, information products from Policy undergo internal quality assurance performed at various levels of management before they are sent to Resource Management.

Haut de la page

6.0 Conclusion

Based on the results of the risk assessment of its preliminary findings, the audit team found no major risks in the context of information quality associated with the fisheries management decision memoranda.  Though minor risks were identified, the audit team considered that they were addressed by sufficient mitigation strategies.  The audit team is satisfied with the control framework in place for Resource Management, Science and Policy. 

Given that adequate controls are in place to ensure the quality of information for decision-making pertaining to fisheries management decision memoranda, the audit team considered that pursuing a detailed examination in the conduct phase would not add value at this time.  So as to optimize the use of its resources, the Internal Audit Directorate needs to conduct audit work only on areas of high risk and materiality.  As a result, the Directorate recommends concluding the internal audit engagement at this stage. 


7.0 Appendix A – Resource Management Decision Memorandum Process Map

diagramme de processus — note de décision de la gestion des ressources

8.0 Appendix B – Policy Information Product Process Map

diagramme de processus — produits d’information du secteur des politiques

9.0 Appendix C – Science Information Product Process Map

diagramme de processus — produits d’information du secteur des sciences

1 “Economic Analysis and Statistics” is the National Headquarter function within the Policy sector that provides input to Resource Management on social and economic information.  In the region, this function is referred as “Policy and Economics”.

2 Senior Management of Economic Analysis and Statistics confirmed that progress was still underway to implement the management action plan including Key Renewal Initiatives and raised concerns about the burden that the audit would have on the branch and regions at this time.  Consequently, conducting a follow-up audit on the Audit of the Management Control Framework Supporting Statistical Information on Fisheries would not add value at this time.

3 The Audit of the Management Control Framework Supporting Senior Management Committee Information for Decision-Making focused on the governance and accountability as well as related processes and controls for key departmental committees.

4 “Economic Analysis and Statistics” is the National Headquarter function within the Policy sector that provides input to Resource Management on social and economic information.  In the region, this function is referred as “Policy and Economics”.

5 Statistical Information on Fisheries is not examined as per the realignment of the internal audit engagement.

6 Statistical Information on Fisheries is not examined as per the realignment of the internal audit engagement.