Archived – Audit of the Management Control Framework Supporting Senior Management Committee Information for Decision-Making

Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Project Number 6B087
Final Audit Report
December 16, 2009

Table of Contents

List of Acronyms

CCG
Canadian Coast Guard
CCGMB
Canadian Coast Guard Management Board
DFO
Department of Fisheries and Oceans
DMC
Departmental Management Committee
FAMEC
Fisheries and Aquaculture Management Executive Committee
NSDC
National Science Directors Committee
RD
Record of decisions
TOR
Terms of reference

 

1.0 Executive Summary

1.1 Introduction

The Department of Fisheries and Oceans (DFO) has identified the risk of “sufficient and appropriate information not being available on a timely basis to support decision-making.”[1] Information for decision-making is a broad-based risk that includes multiple drivers such as gaps in data collection and in record-keeping. The risk is in fact too broad to be addressed with a single audit.  It was therefore decided, at the time that the terms of reference were finalized, to segment the audit response to this corporate risk in two separate engagements.

The current audit provides initial oversight into the information for decision-making risk, which will be further addressed in subsequent audit work outlined in DFO’s Multi-year Risk-based Audit Plan 2009-10 to 2011-12. This audit focused solely on the information used in support of decisions made by senior management committees. Generally, high-level decisions such as those made by senior management committees have a greater impact on organizational objectives. Thus there is greater need for reliance on appropriate and timely information for senior management committees in support of such decisions.

1.2 Objectives and Scope

The objective of the audit was to provide assurance that controls were in place and operating as designed, to ensure the information used by senior management committees for decision-making was appropriate and provided in a timely manner. The audit focused on governance and accountability as well as committee processes and controls; it did not provide assurance about the integrity and reliability of data and information systems.

Four senior management committees at national headquarters were assessed based on their roles in setting direction on strategic issues and establishing goals, priorities and policies. The audit examined committee terms of reference (TOR), decision-making process, procedures and information supplied for decisions taken between April 1, 2007 and September 30, 2008.

1.3 Statement of Assurance

In our opinion, the auditors have examined sufficient, relevant evidence and obtained sufficient information and explanations to provide a high level of assurance on the reported opinion or conclusions.

1.4 Summary of Observations and Recommendations

The audit has found good senior management committee foundations. Governance structures were in place for all four senior management committees and three had TORs. Some controls and procedures were in place; however, they were not complete, fully documented or operating consistently. It appears that committees are exercising their mandates [2]; however, inconsistent committee operations had an impact on the appropriateness and timeliness of information supplied for decision-making examined in the file review.

All senior management committees in DFO can increase their effectiveness by bringing more rigor to their governance structures and decision-making processes. Specifically, it is recommended that senior management committees periodically assess their performance to ensure roles and responsibilities are consistently carried out. A standard TOR should also be developed. Finally, it is recommended that committee processes be formalized and documented at key points in the decision-making process, as depicted in Appendix A of the present report.

2.0 Introduction

2.1 Background

DFO is a decentralized organization with eight of every ten employees located in regions across Canada. DFO operations are carried out in regions; national headquarters establishes objectives, policies, procedures and standards. The Canadian Coast Guard (CCG)—a special operating agency—is also decentralized and operates out of national headquarters and regions.
Decision Dimensions Pyramid
Decisions are taken at all levels of the department, as illustrated in the decision dimensions pyramid. Generally, the higher the level of decision-making, the greater the impact on the organizational objectives; thus there is increased reliance on appropriate and timely information.

The audit focused on decisions taken by senior management committees, given their complexity and long-term impacts on organizational objectives. The Departmental Management Committee (DMC) acts as DFO’s senior decision-making body. DFO is supported by a number of other senior management committees. This includes DMC sub-committees as well as regional and sectoral management committees. CCG and its management board are also supported by sub-committees and executive boards.

Senior management committees require high quality information to fulfill their function.[3] As DFO senior management identified in its 2008 corporate risk profile, information is an essential component of effective management. The presence of well functioning controls is essential to ensure information supplied to committees is appropriate and timely. For the purpose of the audit, those characteristics were defined as follows: 

  • Appropriate: Information for decision-making is developed by exercising due diligence, supports plausible options for decisions and reflects required consultations; and
  • Timely: Information for decision-making is supplied sufficiently in advance to support sound decision-making.

Information for decision-making is a broad risk for DFO. In the Fisheries and Oceans Canada 2008 Corporate Risk Profile, senior managers identified multiple drivers such as gaps in data collection and in record keeping. The risk is in fact too broad to be addressed with a single audit.  It was therefore decided, at the time that the terms of reference were finalized, to segment the audit response to this corporate risk in two separate engagements.

While the current audit provides initial oversight into the information for decision-making risk, it focused solely on decisions taken at the senior management committee level.  Generally, high-level decisions such as those made by senior management committees have a greater impact on organizational objectives.  Thus there is greater need for reliance on appropriate and timely information for senior management committees in support of such decisions.

Subsequent audit work will also be undertaken to further address this risk, as outlined in DFO’s Multi-year Risk-based Audit Plan 2009-10 to 2011-12, namely the Audit of Supporting Statistical Information on Fisheries. 

Top of page

2.2 Objectives and Scope

The objective of the audit was to provide assurance that controls were in place and operating as designed, to ensure the information used by senior management committees for decision-making was appropriate and provided in a timely manner. There was no testing of data with the intent of providing assurance about the integrity and reliability of data and information systems.

Four senior management committees at national headquarters were assessed: DMC, Canadian Coast Guard Management Board (CCGMB), National Science Directors Committee (NSDC) and Fisheries and Aquaculture Management Executive Committee (FAMEC). These committees were selected because of their roles in setting direction on strategic issues and on establishing overall goals, priorities and policies.

The audit focused on two lines of enquiry—governance and accountability as well as committee processes and controls—recognizing that information requirements have foundational and operational dimensions for senior management committees. To assess criteria under each line of enquiry, the audit examined committee TORs, decision-making processes, procedures and information supplied for decisions taken between April 1, 2007 and September 30, 2008.

2.2.1 Line of Enquiry 1: Governance and Accountability

Governance structures and accountabilities are in place and operate in accordance with generally accepted management principles and practices, to ensure that senior management receives appropriate and timely information for decision-making.

  • Criterion 1.1: Governance structures and accountabilities are in place and correspond to generally accepted management principles and practices.
  • Criterion 1.2: Governance structures and accountabilities operate in accordance with generally accepted management principles and practices.

2.2.2 Line of Enquiry 2: Committee Processes and Controls

Processes and controls (e.g. policies and procedures) to support the governance structure have been developed, implemented and are operating as designed, to ensure that senior management receives appropriate and timely information for decision-making.

  • Criterion 2.1: Internal policies and procedures related to information for senior management committee decision-making are in place and comply with generally accepted management principles and practices.
  • Criterion 2.2: Information obtained for senior management committee decision-making is in accordance with central agency and Department of Fisheries and Oceans policies and procedures or generally accepted management principles and practices.

2.3 Method

2.3.1 Planning Phase

A benchmarking exercise was conducted with other government departments to understand key controls related to information for senior management committee decision-making. The purpose was to identify best practices.

To gain a better understanding of senior management committees and the information for decision-making risk, a sample of DMC members was interviewed. The preliminary survey included a review of a sample of information supplied for decision-making. The DMC, CCGMB, FAMEC and NSDC decision-making processes were documented and their TORs were analyzed.

A judgmental sample of agenda items was selected for file review, based on the frequency of committee meetings. The sample was selected from important decisions and agenda items for which no purpose could be identified. A total of 33 items were selected from the four committees, 20 of which were confirmed as decisions after file review.

An audit TOR and program were developed, including objective, scope, lines of enquiry, criteria and methodology. Criteria were developed based on generally accepted management principles and practices, including Treasury Board Secretariat and DFO policies on information management, the Office of the Comptroller General’s Core Management Controls, the Criteria of Control Framework from the Canadian Institute of Chartered Accountants and best practices. The Deputy Minister approved the audit TOR on November 25, 2008.

2.3.2 Conduct Phase

A benchmarking exercise was conducted with other government departments to understand key controls related to information for senior management committee decision-making. The purpose was to identify best practices to strengthen existing DFO committee control frameworks. The results of that exercise are detailed in section 3.4 of the present report.

Checklists were developed based on the audit criteria to document and summarize the analysis of documentation supplied or confirmed by committee secretariats. This included: committee TORs; decision-making process flowcharts; other documented procedures to support decision-making; plus a sample of decision items, which includes supporting material submitted by decision sponsors and distributed by secretariats, as well as meeting agendas and RDs.

2.3.3 Reporting Phase

A status report was presented to the Departmental Audit Committee in March 2009 with preliminary findings. Observations were confirmed with individual committee secretariats and Information Management and Technology Services. Finally, a draft report was sent to the Deputy Minister and the chairs of the various committees for their concurrence.

3.0 Observations and Recommendations

In this section, observations and recommendations are presented by issue. Observations are based on the four senior management committees examined in the audit. Recommendations are addressed to the Deputy Minister and pertain to all DFO senior management committees.

3.1 Information for Senior Management Committee Decision-Making

Senior management committees require appropriate and timely information to support decision-making and fulfill their function. This includes, supporting material with a clear purpose, the decision sought and information on consultations with stakeholders. Supporting material must be distributed to members with enough time to allow for meaningful review and analysis; travel requirements should also be considered. To adequately follow up on decisions, RDs should identify decisions, offices of primary interest and timelines for implementation.

Lack of timely and appropriate information can lead to ineffectiveness or inefficiencies, for example, delays in decisions, taking inappropriate decisions or even repeating decisions.

A review of 20 decisions revealed weaknesses in appropriateness as none of the supporting material contained all of the required elements outlined above. Submission forms were used in 14 decisions and there was no consistent follow up when they were not. No evidence was provided for quality assessment, which could have addressed the appropriateness issue. Five decisions were not recorded in RDs, due to two committees not keeping RDs for all meetings and some RDs pending publication. Only three documented decisions included an office of primary interest and a timeline for implementation. The review also revealed some weaknesses in timeliness. Sixty-five percent of supporting material was distributed within established timelines—ranging from three days to one week—however, interviewed committee members were generally dissatisfied with timeliness. The average publication time for RDs was five months.

Lack of timely and appropriate information can lead to ineffectiveness or inefficiencies, such as, delays in decisions, taking inappropriate decisions or even repeating decisions. As outlined in the Fisheries and Oceans Canada 2008 Corporate Risk Profile, lower quality information also exacerbates other risks such as legal compliance. Assessing committee performance will help identify and adjust variations from established procedures. This will ensure information for decision-making is assessed and amended when necessary, prior to distribution.

Recommendation:

Top of page

3.2 Approved and Documented Governance Sturctures

Senior management committees require documented and approved governance structures to function effectively. A governance structure may be documented in a variety of formats, such as terms of reference or charters. It should clearly communicate the committee’s mandate, roles and responsibilities, membership, frequency of meetings as well as decision-making authority (i.e. how decisions are reached).[4] It should also include a provision for a periodic review to ensure continued relevance.

The absence of a periodic review—and recorded approval—of TORs may result in the governance structure not adapting to changes in its environment.

There are a number of key information management-related responsibilities which also need to be reflected in governance structures. Deputy heads are responsible to ensure that “information is shared within and across departments to the greatest extent possible, while respecting security and privacy requirements.”[5] To facilitate this, documented governance structures should include provisions for timely communication of decisions to stakeholders.

Of the four senior management committees reviewed, three had TORs with membership and frequency of meetings. Mandates as well as roles and responsibilities were included; however, these were sometimes vague or not explicitly defined. TORs had been updated within the last one to five years; however, they did not include provisions for periodic review and no record was kept of the approval of their last update. Decision-making authorities were also absent from the TORs. One TOR had a timeline for communicating decisions to key stakeholders.

Without a documented governance structure, senior management committees may not be functioning effectively, for example, not meeting frequently enough to address issues within their mandates. In addition, stakeholders may not be informed of decisions and the committee may not reach its decisions in a consistent manner, potentially having a negative impact on implementation of decisions. The absence of a periodic review—and recorded approval—of TORs may result in the governance structure not adapting to changes in its environment.

Recommendation:

3.3 Committee Decision-Making Process and Procedures

Deputy heads are responsible for “ensuring that decisions and decision-making processes are documented to account for and support the continuity of departmental operations, permit the reconstruction of the evolution of policies and programs, and allow for independent evaluation, audit, and review.” [6] Departmental employees are responsible to support deputy heads in the management of information. [7] If decisions are not documented, the department is at risk of not being in a position to fully answer for its directions. Decision-making processes should include accountabilities for information. In addition, a quality assessment function should be in place. [8] Expectations for accountabilities must be balanced with the capacity to deliver, that is, sufficient resources must be allocated and appropriate training must be provided to employees.[9]

Policies and procedures should also be established to guide the work of personnel in specific areas that have an impact on information for decision-making, namely: preparation and assessment of supporting material; communication and follow up on decisions; and records management. Procedures should be written and disseminated to appropriate parties. The level of formality and detail can vary based on the complexity of committees and should provide for continuity in operations. Procedures should be documented separately from TORs as governance documents are not meant to provide the necessary detail.

All four committees had decision-making processes in place which included accountabilities for information.

All four committees had decision-making processes in place which included accountabilities for information for decision-making and controls such as: the approval of agenda items; assessment of readiness of supporting material; preparation and communication of records of decisions; and follow up on action items. Two committees had accountabilities for information for decision-making outlined in their TORs. However, the decision-making processes were not documented and controls were not operating consistently.

Three committees had TORs and a summary form for submission of supporting material, which provided guidance on characteristics of supporting material—such as decision sought, options and recommendations—and a timeline for distribution. Communication and follow up of decisions are briefly addressed in all three TORs. Departmental procedures for records management are in place. However, formal procedures had not been developed to translate departmental requirements into committee record keeping requirements.

Without formal documented procedures—or if personnel are not aware of their roles and responsibilities—there is a risk that information for decision-making may be inconsistent or missing key elements. Templates or checklists could be used to communicate supporting material requirements to decision sponsors; secretariats could also use them to perform quality assessment. Procedures are required to ensure documents in support of decisions are stored in accordance with departmental and central agency requirements. Procedures for the communication of decisions would provide guidance on key elements to include in RDs and on the administrative actions for timely publishing. Finally, following-up on results of decisions will help ensure subsequent committee business is appropriately scheduled.

Recommendation:

3.4 Benchmarking with Other Government Departements and Agencies - Best Practices

A benchmarking exercise was conducted with four senior management committees of other government departments and agencies with science-related mandates. The purpose of the exercise was to identify best practices and procedures which could be used in DFO; they are outlined in a process map found in Appendix A of this report. In addition, the following table compares the best practices identified in some of the other organizations with the four DFO senior management committees reviewed in the audit.

Best Practices Found in Other Governement Departements and Agencies DFO Senior Management Committees
A B C D
1. Senior management committees have documented TORs. Yes No Yes Yes
2. Accountability is assigned for information for decision-making. Yes Yes Yes Yes
3. Procedures for the preparation of supporting material are documented (e.g. briefing notes or deck templates). Yes No Yes Yes
4. Procedures for the review and approval of supporting material are documented. No No No No
5. Procedures for communicating decisions and record keeping are documented. No No No No
6. Procedures to follow up on decisions and action items are documented. No No No No

Top of page

4.0 Conclusion

All senior management committees examined in the audit had governance structures in place. The one committee that did not have a TOR had developed a draft, which is well underway to being approved by the end of the audit. Although some improvements with documentation and clarity of TORs are required, overall, it appears that committees are exercising their mandates. Existing TORs are good foundations and committees will benefit from the enhancements recommended in this report.

Some controls are in place for information used in decision-making processes; however they are not complete or operating consistently. Interviewed committee members felt that supporting material was not always appropriate and timely and this is consistent with the conclusions of the file review. A strategy to assess performance will ensure established procedures are followed consistently. Standard TORs will promote rigor and consistency in governance structures. Finally, documented and formal procedures will provide continuity in committee operations and ensure information for decision-making is well prepared and assessed; decisions are documented and adequate follow-up occurs; and proper records are kept.

5.0 Management Action Plan

Recommendations Management Action Plan Status Report Update
Actions Completed Actions Outstanding Target Date
1. The Deputy Minister should develop a strategy to ensure each committee and its secretariat assesses its performance and adjusts as required to ensure effectiveness. (High significance) Evaluations of the performance and effectiveness of DMC and its sub-committees will be conducted including self-evaluations by each committee.     March 31, 2010
2. The Deputy Minister should develop a standard to ensure all senior management committees have documented and approved terms of reference. The standard should include: mandate; composition; frequency of meetings; roles and responsibilities; decision-making authority; provisions for periodic review and timely communication of decisions. (High significance) A standard will be developed by Executive Secretariat, in collaboration with the secretariats for other senior management committees, and presented to DMC for approval by January 2010.  The approved standard will then be adapted for use by every committee chaired by a DMC member.     Proposal to DMC for consideration by January 2010; implementation by March 31, 2010.
3. The Deputy Minister should ensure that all senior management committees have formal and documented procedures related to decision-making processes, including: preparation and assessment of supporting material; communication and follow up on decisions; and records management. Training and awareness sessions should also be provided to individuals responsible for official decision-making records. (Medium significance) Procedures will be formalized and documented.  Training on information management requirements will be provided. 
Discussions at DMC will be held regarding the kind of decisions it needs to focus on and the kind of information required to make sound decisions.
    Procedures: April 30, 2010
Information management training:  December 15, 2009

Appendix A: Best Practices Process Map

Best Practices Process Map

1.Fisheries and Oceans Canada, 2008 Corporate Risk Profile, April 29, 2009, p. 33.

2. This observation is generally supported by the 2008-09 Management Accountability Framework assessment of the corporate management structure. Treasury Board of Canada Secretariat, TBP Assessment, p. 8.

3.Office of the Comptroller General, Core Management Controls: A Guide for Internal Auditors, p. 13.

4.Office of the Comptroller General, Core Management Controls: A Guide for Internal Auditors, pp. 11 to 14.

5.Treasury Board of Canada Secretariat, Policy on Information Management, section6.1.3.

6.Treasury Board of Canada Secretariat, Policy on Information Management, section6.1.2.

7.Treasury Board of Canada Secretariat, Directive on Information Management Roles and Responsibilities, section 6.3.

8.Office of the Comptroller General, Core Management Controls: A Guide for Internal Auditors, p. 14.

9.Treasury Board of Canada Secretariat, Foundation Framework for Treasury Board Policies, section 5.2.