Archived – Audit of the Aboriginal Aquatic Resource and Oceans Management Program

Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

PROJECT NUMBER 6B093
Final Audit Report
June 18, 2009

Table of Contents

List of Acronyms
1.0 Summary
1.1 Introduction
1.2 Objectives and Scope
1.3 Statement of Assurance
1.4 Summary of Observations and Recommendations
1.4.1 Key Observations
1.4.2 Key Recommendations
2. Introduction
2.1 Background
2.1.1 Program’s Financial Data
2.2 Objective and Scope
2.2.1 Audit Criteria
2.3 Methodology
3. Observations and Recommendations
3.1 Management Control Framework
3.1.1 Responsibilities, Policies and Procedures
3.1.2 Training
3.2 Eligibility, Selection and Approval of Funding Applications
3.2.1 Selection and Funding of Recipients
3.3 Program Management
3.3.1 Issuing and Approving Agreements
3.3.2 Activity Reports
3.3.3 Payment Requests
3.3.4 Review of Audited Financial Statements
3.4 Risk and Results Management
3.4.1 Monitoring of Agreements and Risk Management
3.4.2 Results-based Management and Accountability Framework
4.0 Management Action Plan

List of Acronyms

RMAF - Results-based Management and Accountability Framework
IACMF - Integrated Aboriginal Contribution Management Framework
APF - Aboriginal Programming Framework
RBAF - Risk-based Audit Framework
APG - Aboriginal Policy and Governance Directorate
IAD - Internal Audit Directorate
FAA - Financial Administration Act
DFO - Department of Fisheries and Oceans
AAROM - Aboriginal Aquatic Resource and Oceans Management Program
GAAP - Generally Accepted Accounting Principles
PTP - Policy on Transfer Payments
AFS - Aboriginal Fisheries Strategy

Top of page

1.0 Summary

1.1 Introduction

The purpose of the Aboriginal Aquatic Resource and Oceans Management Program (AAROM) is to help Aboriginal groups build the capacity to successfully manage their resource-related activities and participate more effectively in multi-stakeholder and other decision-making processes used by Fisheries and Oceans Canada (DFO) in the management of oceans and aquatic resources.

The AAROM program is aimed at enhancing the ability of Aboriginal communities, working together, to participate in the advisory and decision-making processes related to the management of aquatic resources and oceans.

The AAROM program is currently being renewed under an integrated performance and risk management framework which includes six Aboriginal programs, and is referred to as the Integrated Aboriginal Contribution Management Framework (IACMF). The new streamlined approach is aimed at improving efficiency in service delivery, strengthening accountability and setting greater focus on achievement of results. It is in line with the new Policy on Transfer Payment (PTP) and reflective of best practices.

1.2 Objectives and Scope

The objectives of the audit are to:

  • provide assurance that a management control framework and effective and appropriate management practices were implemented under AAROM; and
  • determine the level of compliance with policies and procedures pertaining to the control of transfer payments.

The audit focused mainly on AAROM’s Capacity Building and Collaborative Management components, and on the review of a sample of ten recipient files covering the period 2004-2005 to 2008-2009 and representing the main regions affected by AAROM.

At the time of the audit (December 2008-January 2009), management was still working on the new IACMF which was approved by Treasury Board in March. Key strategy and related implementation plan are expected to be rolled-out in 2009-2010.

We wish to express our thanks for the co-operation and assistance given to the audit team by management and staff at headquarters (HQ) and in the regional offices.

1.3 Statement of Assurance

In our opinion, the auditors have examined sufficient, relevant evidence and obtained sufficient information and explanations to provide a high level of assurance on the reported opinion or conclusions.

1.4 Summary of Observations and Recommendations

Observations and recommendations of this report apply to the new IACMF and the Management Action Plan is developed in light of these new changes.

1.4.1 Key Observations

AAROM is generally managed and administered appropriately and in compliance with the requirements and terms and conditions of the Policy on Transfer Payments, and is supported by access to the appropriate systems, resources and internal controls. A management control framework and effective and appropriate management practices were implemented for AAROM.

However, the audit revealed the following areas where there is room for improvement:

Management Control Framework

  • The definitions of AAROM staff roles and responsibilities are not sufficiently detailed and do not include all steps of the program’s lifecycle.
  • The procedures and other management tools related to AAROM are incomplete, and staff do not have a guide setting out the program’s administrative and financial procedures.

Eligibility, Selection and Approval of Funding Applications

  • Documentation for the selection process requires improvement with respect to the terms of reference used by the review committee and the participation of various stakeholders in this committee.

Program Management

  • Some clauses required pursuant to the PTP are not always included in the contribution agreements.
  • Generally, although recipients produce the activity reports stipulated in the terms and conditions of the agreements, the requisite financial information and summary of results are not always submitted. Furthermore, the summary does not contain the information needed to assess the effectiveness of the contributions in achieving the program’s objectives.
  • The frequency and type of activity reports required by the Aboriginal Policy and Governance Directorate (APG) entail substantial work for recipients each time payment is requested, but they do not offer DFO any significant added value.
  • Budget information is not required in activity reports, even though it is considered a fundamental requirement in terms of financial control and budget tracking.
  • The audited financial statements include the balance sheet and income statement of the organization that administers funds provided by AAROM rather than only those statements pertaining to AAROM funding, thus diminishing the value of the internal control pertaining to audited financial statements.
  • For annual contributions equal to and exceeding $250,000, the recipient’s financial statements are not audited in accordance with the terms and conditions of the Results-based Audit Framework (RBAF), which requires a compliance audit pursuant to the contribution agreement. Moreover, documentation kept in recipient files pertaining to audited financial statements is not always complete.
  • APG’s compliance audit requirement, as set out in the RBAF and required under the PTP, is not based on a strategic assessment of recipients' risk profiles.

Monitoring and Risk Management

  • The monitoring of agreements is not formally documented, and risk management is not incorporated at all stages of the program’s lifecycle to ensure that the funds allocated to recipients are used for their stated purpose.
  • Risk assessment is not documented upon completion of monitoring actions.
  • Ongoing monitoring of results and performance is insufficient and does not reflect the data-gathering, analysis and commitments set out in the Results-based Management and Accountability Framework (RMAF).
  • An integrated performance and risk management framework for all of DFO’s Aboriginal programs is being developed.

1.4.2 Key Recommendations

The following key recommendations are aimed at strengthening the program’s management control framework. They have been grouped for simplicity of presentation. The Director General, APG should:

  • Document, update and communicate the roles and responsibilities of the various stakeholders responsible for managing AAROM, and in particular, ensure that monitoring, risk management and performance measurement expectations and responsibilities are clearly defined, communicated and applied at all stages of the program’s lifecycle.
  • Re-evaluate the frequency and type of activity reports to be provided by recipients, based on level of funding, level of risk to AAROM, work involved for recipients, and the review capabilities of employees responsible for reviewing these reports. Additionally, activity reports should include budget information so that it can be compared with actual costs incurred and the employees responsible for reviewing the reports can identify significant variances that require corrective action.
  • Establish procedures and guidelines concerning the documentation of monitoring actions and the evaluation of AAROM’s performance, i.e. data-gathering, compilation, analysis and dissemination.
  • Review expected results and performance indicators for AAROM to ensure that the program’s performance can be measured effectively.
  • Determine approaches to audit recipients based on a systematic risk assessment of recipient which would be measured and recorded during all steps of the program’s lifecycle. For example:
    • determine the opportunity to introduce on-site compliance audits (audit of contribution agreement) as an additional control based on the systematic assessment of risk;
    • establish guidelines on the verification of audited financial statement in order to clarify difference between financial audit and compliance audit and to standardized risk assessment based on information in the financial statement etc.
  • Establish or update administrative and financial procedures and guidelines, and train assigned staff, accordingly.

Top of page

2.0 Introduction

In accordance with the RBAF for AAROM, the program’s internal audit was conducted between December 2008 and January 2009, before the anticipated program’s renewal in March 2009.

The AAROM program is currently being renewed under an integrated performance and risk management framework which includes six Aboriginal programs, and is referred to as the Integrated Aboriginal Contribution Management Framework (IACMF). The new streamlined approach is aimed at improving efficiency in service delivery, strengthening accountability and setting greater focus on achievement of results.  It is in line with the new Policy on Transfer Payment (PTP) and reflective of best practices.

At the time of the audit (December 2008-January 2009), management was still working on the new IACMF which was approved by Treasury Board in March. Key strategy and related implementation plan are expected to be rolled-out in 2009-2010.

2.1 Background

The Aboriginal Aquatic Resource and Oceans Management Program is intended to assist Aboriginal groups in developing the necessary capacity to play a more active and effective role in key areas of conservation and sustainable use of aquatic resource and oceans management. Particular emphasis is placed on Aboriginal participation in multilateral decision-making and advisory processes that involve fishers, scientists, industry representatives, conservation groups, DFO officials, and other government departments.

DFO’s Aboriginal Programming Framework (APF) is intended to serve as a catalyst for effective management and governance in the area of aquatic and oceans resources and potentially other areas of Aboriginal governance.

The main objectives of APF are:

  • support the involvement of Aboriginal groups in various fora and processes used for the integrated management and planning of aquatic and oceans resources;
  • support the creation of new management bodies that bring together Aboriginal groups or communities that are located in the same watershed or ecosystem and share the same goals and interests with respect to the conservation and management of aquatic and ocean resources (and the enhancement of existing bodies where appropriate);
  • help Aboriginal groups access the technical, scientific and administrative expertise required to support their effective involvement in these areas;
  • build capacity in Aboriginal communities fisheries management and extending to various areas of DFO activity;
  • facilitate the resolution of treaty and Aboriginal rights issues by developing practical and durable approaches in areas of common interest, in order to establish a model for long term arrangements.

AAROM has three components:

Collaborative management:This component involves developing capacity for Aboriginal groups and hiring the qualified staff (biologists, technicians, fisheries managers, etc.) required to:

  • effectively manage their use and management of aquatic resources and oceans; and
  • participate effectively in DFO’s advisory and decision-making processes.

This component also includes hiring and training Aboriginal Fisheries Officers.

Capacity building: This involves enabling Aboriginal groups who are not eligible for the Collaborative Management component but who agree to work in this direction, to obtain funding to help them develop their professional, administrative and technical expertise so that they can satisfy the eligibility requirements of the Collaborative Management component. Some of the activities eligible for funding under this component are:

  • development and implementation of capacity-building strategies;
  • conduct of strategic planning exercises related to AAROM;
  • design and completion of capacity assessment / feasibility studies;
  • development of business plans; and
  • administrative, financial and legal preparations for the establishment of an aggregate body.

Economic opportunities: This component includes the following two sub-components, aimed at contributing to the long-term sustainability of AAROM and of the participating communities:

  • access: to support the voluntary retirement of commercial fishing licenses and transfer of commercial fishing opportunities (including vessels and gear) to eligible Aboriginal groups, and
  • aquaculture: funding to enable Aboriginal groups to pursue aquaculture-related activities.

To qualify for funding, Aboriginal groups in a single watershed or ecosystem must collaborate and comply with certain requirements related to management practices. AAROM recommends a community approach that recognizes that each group is at a specific stage of capacity development, and that not all groups have the same priorities and objectives.

To help groups qualify for funding under AAROM’s Collaborative Management component, the program allows Aboriginal groups to first apply for funding under the program’s Capacity Building component.

Eligible groups can also receive funding for commercial licences, vessels and gear and for capacity development so that they can take advantage of the opportunities afforded by aquaculture. Eligible groups can also obtain funding to train Aboriginal Fisheries Officers.

The program can also benefit groups that are located in areas managed by DFO and have not signed any comprehensive land claim agreements addressing matters covered by AAROM.

2.1.1 Program’s Financial Data

AAROM has a $59.4 million budget for five years, allocated as follows:

AAROM Funding Sources
(million $) 2004-05 2005-06 2006-07 2007-08 2008-09 Total
Contribution agreements (vote 10) 6.0 7.7 11.1 11.1 11.1 47.0
AFS reallocation to AAROM (vote 10) 0.0 2.1 2.1 2.1 2.1 8.4
Human resources and other expenses (vote 1) 0.8 0.8 0.8 0.8 0.8 4.0
Total 6.8 10.6 14.0 14.0 14.0 59.4

2.2 Objective and Scope

The objectives of the audit are:

  • to provide assurance that a management control framework and effective and appropriate management practices were implemented under AAROM; and
  • to determine the level of compliance with policies and procedures pertaining to the control of transfer payments.

The audit focused mainly on AAROM’s Capacity Building and Collaborative Management components, and on the review of a sample of ten recipient files covering the period 2004-2005 to 2008-2009 and representing the Maritimes, Pacific and Quebec Regions.

AAROM funding for commercial licences, vessels and gear as well as funding to support DFO’s internal capacity were excluded from the audit mandate since they represent low dollar amounts compared with the contributions paid under AAROM’s Capacity Building and Collaborative Management components.

2.2.1 Audit Criteria

The following audit criteria were identified specifically for AAROM following a preliminary assessment of the program’s risk that was completed during the audit planning stage. The criteria were grouped under the following four categories:

Management Control Framework

  • The roles and responsibilities of staff responsible for the program are clearly defined and communicated.
  • Policies, procedures and other management tools are made available to staff so that they can do their jobs effectively.
  • Staff assigned to the program are trained properly to evaluate funding applications, and to deliver and control the program.

Eligibility, Selection and Approval of Funding Applications

  • Information provided by program applicants allows for informed decision-making with respect to the granting of funds.
  • Funding applications are evaluated and approved impartially.

Program Management

  • The program is administered and monitored in compliance with the terms and conditions of the program and the PTP, and more specifically, on the issuance and approval of agreements.
  • AAROM payment requests are certified in compliance with the FAA and supported with the appropriate documentation, in compliance with the PTP and with program terms, conditions and internal procedures.

Risk and Results Management

  • Program follow-up on risk management and performance meets the requirements of the RBAF and the RMAF, in compliance with Treasury Board Secretariat policies.
  • Risk management strategies and practices are incorporated into all stages of the program’s lifecycle to ensure that the funds allocated to recipients are used for their intended purpose.

2.3 Methodology

The audit team carried out its mandate in accordance with Government of Canada internal audit standards, as set out in Treasury Board Secretariat’s Policy on Internal Audit. Our approach comprised the following activities:

  • review of documentation on the management control framework used for AAROM;
  • review of a sample of ten recipient files from the Pacific, Maritimes and Quebec Regions; and
  • interview key staff at HQ and in the Pacific and Maritimes Regional offices.

A sample of ten recipients was reviewed from a total of 44. This sample represents contributions totalling $12.9M, i.e., 28.9% of all signed agreements, and covers the program’s five-year term. During the AAROM audit planning phase, the program’s main risks and the internal controls deployed to minimize them were identified.

Top of page

3.0 Observations and Recommendations

Observations and recommendations of this report apply to the new Integrated Aboriginal Contribution Management Framework and the Management Action Plan is developed in light of these new changes.

This section presents observations on the effectiveness of the AAROM internal control framework and on compliance with contribution policies. They are presented based on the audit criteria identified in section 2.2.1.

To assist management in preparing its management action plan, the audit recommendations have been labelled according to the following three categories to reflect their degree of importance:

  • High significance – major control weaknesses and/or unacceptably high level of risk;
  • Medium significance – control weakness affecting operational efficiency and/or credibility; and
  • Low significance – control or process weakness contributing to inefficiency.

3.1 Management Control Framework

3.1.1 Responsibilities, Policies and Procedures

The roles and responsibilities of staff assigned to the program do not cover all stages of the program’s life cycle and could be defined in more detail.

The procedures and other management tools related to AAROM are incomplete, and staff do not have a guide describing administrative and financial procedures for the program.

As stipulated in the Policy on Transfer Payments (PTP) issued by Treasury Board Secretariat effective October 1, 2008, one of the deputy head’s responsibilities is “ensuring that cost-effective oversight, internal control, performance measurement and reporting systems are in place to support the management of transfer payments”. An appropriate control framework can be established and the programs managed consistently if the roles and responsibilities of the various stakeholders involved in contribution program management are defined and communicated, and policies, procedures and directives are prepared and distributed.

The review of documentation on roles and responsibilities, procedures and other management tools revealed the following:

When the AAROM program was launched and in subsequent years, the APG developed and communicated management tools and procedures, and posted them on an intranet site accessible to employees assigned to the program. The site includes an organizational chart and summary defining the roles and responsibilities of all stakeholders involved in AAROM management at headquarters and regionally. These documents were developed during the second quarter of 2006-2007.

Program risks were documented when the program was created and measures for minimizing these risks were developed in accordance with the PTP. Expected results and performance measures were also identified in accordance with the PTP. The logic model and performance measurement strategy were streamlined in 2007.

We noted that neither the procedures nor the definition of roles and responsibilities cover all activities in the program’s life cycle. Monitoring of projects funded by the program, program risk management and performance measurement are not documented and as a result, have not been communicated to staff assigned to the program.

Furthermore, the decentralization of financial operations in the Pacific Region was not reflected either in the procedures or in the summary of roles and responsibilities for the AAROM program, so the region documented the roles and responsibilities of its own stakeholders involved in AAROM management. However, this documentation does not cover all the stages in the program’s life cycle in terms of project monitoring, risk management and performance measurement.

Unlike other APG-managed programs, stakeholders responsible for managing AAROM do not have a guide describing administrative and financial procedures for the program and containing various management tools developed for the program. Given turnover among AAROM staff in some regions, this makes it difficult to maintain corporate memory.
The result is a lack of consistency in responsibilities among the regions, and non-compliance with policies, as reported in the sections below.

Recommendation (Medium significance)

1. We recommend that the Director General, APG ensures that:

i) the roles and responsibilities of the various stakeholders responsible for AAROM management are documented, updated and communicated;

ii) AAROM program procedures and guidelines are complete, updated and circulated to the various stakeholders responsible for the program’s management;

iii) monitoring, risk management and performance measurement expectations and responsibilities are clearly defined, communicated and applied at every stage of the program’s life cycle.

3.1.2 Training

Additional program management training requirements were identified for staff assigned to AAROM.

Although there is no specific training plan in place for AAROM, APG recently delivered training sessions and implemented alternative measures for providing continuous training to employees assigned to the program and for supervising new employees, to ensure that they achieve their program delivery and control objectives.

However, given the non-compliance with policies noted in the following sections, the branch should provide program management training to all staff assigned to AAROM once the program’s procedures are updated. The purpose of the training would be to standardize AAROM management processes and communicate updated procedures. At a minimum, the training should cover advances, subsequent payments, project monitoring and program oversight, risk management and performance measurement.

Recommendation (Low significance)

2. We recommend that the Director General, APG ensures that staff assigned to AAROM have enough regular training to ensure that the updated AAROM policies, procedures and directives are implemented consistently.

3.2 Eligibility, Selection and Approval of Funding Applications

3.2.1 Selection and Funding of Recipients

Information provided by program applicants allows for informed decision-making with respect to the granting of funds, and funding applications are evaluated and approved impartially. However, documentation for the selection process requires improvement with respect to the terms of reference used by the review committee and the participation of various stakeholders in this committee.

As set out in the new PTP, “the government is committed to ensuring that transfer payments are managed in a manner that respects sound stewardship and the highest level of integrity, transparency and accountability. Moreover, the government is resolved to ensuring that transfer payment programs are designed, delivered and managed in a manner that is fair, accessible and effective for all involved”.

To ensure an impartial selection process, APG created a review committee of managers from HQ and co-ordinators representing the regions. This committee meets regularly to evaluate funding applications and make recommendations on approvals. The evaluation is documented on a form listing several criteria, and scores are assigned to reflect the degree to which the application meets objectives.

Applicants can also access application information and procedures on an AAROM Web site, and DFO managers have met several times with potential applicants to explain the program and its requirements.

The review of the terms of reference and documentation from the review committee’s decisions revealed the following:

  • The criteria used in the selection process established by the review committee comply with the criteria initially submitted to Treasury Board.
  • Minutes are kept in order to document decisions made at review committee meetings.
  • The review committee’s mandate and participants’ roles and responsibilities are not sufficiently specific. The existing terms of reference do not define the participation of the regions or experts invited to review committee meetings, or the voting process or what constitutes a quorum.
  • Participants and experts invited to review committee meetings are not always identified in the minutes of committee meetings.
  • The selection process approval document does not name the members who attended, validated the information provided and assigned scores resulting in the acceptance or denial of a funding application.

Recommendation (Low significance)

3. We recommend that the Director General, APG update the terms of reference for the review committee responsible for selecting funding applications in order to enhance its transparency and accountability.

3.3 Program Management

3.3.1 Issuing and Approving Agreements

Generally, agreements and amendments are issued in accordance with the PTP and are approved in accordance with the delegated financial authorities set out in DFO’s delegation of financial authorities instrument.

Although the issuing and signing of agreements and amendments complies with the policies and directives, we noted the following:

Even though APG developed standardized models for contribution agreements, which were approved by DFO’s Legal Services, the clause requiring external audit of financial statements for agreements equal to and exceeding $250,000 per year was included in only six of the eight audited contribution agreements exceeding that amount. Furthermore, for agreements equal to or exceeding $100,000, the clause requiring that the applicant declare funding from other sources for the proposed project was not included in four of the ten audited agreements. This contravenes the terms and conditions of the program and of the PTP.

With the exception of the number of jobs created or maintained as a result of the agreement, none of the audited agreements stipulated their expected results or objectives, which should be related to the objectives of the approved program and described in terms of outcomes, not just in terms of expenditures and activities. As a result, there is no guarantee that recipients will regularly report on their activities in terms of expected outcomes, making follow-up review by program managers more difficult.

All contribution agreements contain a clause allowing recipients to retroactively claim eligible expenses incurred before the agreement is signed, as long as these expenses are incurred during the term of the agreement and the agreement has been duly signed. This clause is required because agreements are often not finalized until after their effective date.

Although this clause complies with the PTP, it increases the risk that the recipient will incur non-eligible expenses or will exceed the maximum allowed under the program, since the agreement is not finalized when the recipient begins incurring expenses. [Solicitor-client privilege].

Recommendation (Medium significance)

4. We recommend that the Director General, APG:

i) establish procedures for issuing contribution agreements to ensure that all clauses required by the PTP are included in all contribution agreements signed by APG;

ii) include in contribution agreements a description of the objectives and expected outcomes so that both recipients and AAROM managers are able to monitor the performance of the agreements;

iii) determine whether contribution agreements should continue including a clause allowing recipients to retroactively claim eligible expenses incurred before the agreement is signed provided that the expenses are incurred during the term of the agreement and that the agreement has been duly signed.

3.3.2 Activity Reports

Generally, although recipients produce the activity reports stipulated in the terms and conditions of the agreements, the requisite financial information and summary of results are not always submitted. Furthermore, the summary does not contain the information needed to assess the effectiveness of the contributions in achieving the program’s objectives.

The frequency and type of activity reports required by the APG entail substantial work for recipients each time payment is requested, yet do not offer DFO any significant added value; moreover, budget information is not required in activity reports, even though it is considered a fundamental requirement in terms of financial control and budget tracking.

The PTP specifies that one of the deputy head’s responsibilities with respect to transfer payments is “ensuring that cost-effective oversight, internal control, performance measurement and reporting systems are in place to support the management of transfer payments”. Accordingly, the APG requires that recipients submit the following reports for each payment request:

  • Narrative report on progress for each task involved in the activity in question for the period;
  • Report on eligible costs incurred, for each task involved in the activity, as identified on the budget work plan submitted in the appendix to the contribution agreement;
  • Report on eligible costs incurred, for each type of eligible cost, as set out in the appendix to the contribution agreement;
  • The statement of balance at month end, in the form as set out in the appendix to the contribution agreement.

Also, at the end of each fiscal year, recipients must submit, in addition to the above reports, a summary of results and audited financial statements.

According to our review of the files, the financial information and summary of results are not always submitted. See the section on activity reporting below.

The summary of results does not contain, as stipulated in the contribution agreement, the information required to report on the effectiveness of contributions in achieving the program goals as set out in the program terms and conditions. The current summary of results includes the number of jobs created (expressed in months of full-time employment) and the salaries and benefits paid. It also gives a description and dollar value of capital acquisitions, and the amount of funding from other sources and number of jobs created, as well as annual revenues generated from these funding contributions.

These reports are reviewed by the regional offices and HQ before payment requests are certified under section 34 of the FAA. The information in these reports must also be regularly compared with the summary of projected results, and with cash flow projections, both of which are submitted by recipients when AAROM-funded activities commence.

Activity reporting

Interim and final activity reports are used to support payment requests submitted by recipients and explain progress made to date. Generally, the 88 payment requests we audited were supported by activity reports submitted by recipients. However, our review of these reports showed that in some cases, the information provided by the recipients did not comply with all the reporting requirements of the contribution agreement:

Initial advances – Of the 19 payment requests for initial advances, 18 (95% of the sample) were supported by cash flow projections however, 18 were not supported by a summary of projected results, contrary to contribution agreements requirements.

As a result, follow-up on activity reports is inadequate because the employees responsible for reviewing reports cannot validate the projected results, which are tied to the objectives of the agreement.

Subsequent advances – Of the 51 activity reports related to requests for subsequent advances, 36 (70% of the sample) contained all the information required, in compliance with the terms and conditions of the contribution agreements. Information missing from the other 15 reports concerned the breakdown of costs incurred by eligible cost category or by task for the activities identified in the budget work plan.

When this information is not included in activity reports, employees responsible for reviewing reports cannot identify significant variances from the budget information in the agreement and cannot implement action plans for rectifying the situation in the event of major discrepancies.

Final payments – The 18 final payments we audited generally contained the information required by the agreement (refer to the six elements discussed earlier), with the exception of ten final reports (56% of the sample), which did not contain the summary of results detailing job creation, capital expenditures and funding from other sources. Furthermore, six payments (33% of the sample) did not contain any cost breakdowns by task for the activities identified in the budget work plan, however they did contain cost breakdown by type of eligible costs.

Evaluation of administrative requirements for activity reports

The PTP also requires that the deputy head ensure “that the administrative requirements on recipients are proportionate to the risk level. In particular, that monitoring, reporting and auditing reflect the risks specific to the program, the value of funding in relation to administrative costs, and the risk profile of the recipient”.

In our opinion, the work load imposed on recipients to produce activity reports for each payment request is excessive, does not take into account the varying levels of funding and does not significantly increase the quality of internal control relative to the program risks, given that two types of actual cost breakdowns are required for each payment request (i.e., by task as set out in the budget work plan appended to the agreements, and by allowable cost category, as set out in the appendix to the agreement).

Also, the frequency of activity reporting significantly increases the workload of employees responsible for reviewing these reports, because they have to analyse the information provided within tight deadlines.

As a result, there is a greater risk of delays in processing payment requests, because these activity reports are reviewed by several people in the regional offices and at HQ, which may penalize recipients with payment delays, particularly where AAROM contributions are used partly to pay employees and contractors.

The APG should review the frequency of activity reports currently required. In our opinion, the type of information currently required from recipients could be required quarterly, for example, and a streamlined activity report that still includes financial data and a narrative summary describing the progress of activities could be required monthly to satisfy Treasury Board Secretariat requirements.

Budget tracking requirements

We also noted that the format of the activity reports required by the APG does not contain any requirement that recipients submit budget against actual costs to reveal variances; such information would help employees responsible for reviewing activity reports track budgets more effectively.

At the present time, only actual incurred costs are required in activity reports, with the result that APG employees have to identify variances themselves, using budget information, which can significantly reduce the effectiveness of this control mechanism, given the volume of reports to review and the time constraints imposed on AAROM managers.

Because the recipients generally indicated in their statement of interest that they used accounting systems with budgeting modules, this additional requirement for financial information should not create additional work for them: it is a basic requirement for financial control and budget tracking.

Recommendation (Medium significance)

5. We recommend that the Director General, APG:

i) update and communicate AAROM procedures and directives concerning information to be included in activity reports for various payment types (initial and subsequent advances and final payments), to standardize the activity report review process;

ii) provide staff assigned to reviewing activity reports with training on the updated procedures and directives;

iii) re-evaluate the frequency and type of activity reports to be submitted by recipients, based on levels of funding, level of risk to AAROM, effort required of recipients, and the review capabilities of employees responsible for reviewing these reports; and

iv) include in activity reports budget information that can be used to compare projected and actual costs so that employees responsible for reviewing these reports can identify significant variances that require corrective action.

3.3.3 Payment Requests

Several payment requests were certified under section 34 of the Financial Administration Act (FAA) by program employees who do not have signing authority, as required in the program terms and conditions and as set out in DFO’s delegation of financial authorities instrument. Furthermore, several times during the term of the agreements, payments to recipients were under-estimated or over-estimated, but ultimately, the maximum amount negotiated for the entire term of the agreement was generally adhered to.

According to the program’s terms and conditions, the Director General, APG and Regional Directors General or their delegates can approve payments under the contribution agreement.
In our review of 88 payment requests tracked from the ten recipient files selected, we noted the following:

  • In 56 cases (64% of the sample reviewed), the payment request was certified under section 34 of the FAA by program employees who did not have authority to sign AAROM payment requests, contrary to the program terms and conditions and DFO’s delegation of financial authorities instrument. Three of these payments did not contain information on the signature – the payment request was missing.
  • For 38 payment requests (43% of the sample), the payment amount was underestimated or overestimated relative to the maximums set out in the agreements, costs incurred, cumulative payments, cash flow forecasts, etc., by as much as $85,000 per payment request (in three cases).

However, these variances were offset over the term of the agreement because, with the exception of one case, all payments issued for each of the ten contribution agreements audited equalled the maximum negotiated amount set out in the agreement, taking into account the 10% holdback paid after the agreement ended once the recipients had fulfilled all requirements. The exception was an overpayment of $61,000 which was subsequently amended retroactively.

  • Also, for six payment requests (7% of the sample), some financial information was missing from the recipients’ files (such as certain sections of the activity reports or cash flow projections), preventing the auditors from validating the amounts paid to the recipients.

Recommendation (Medium significance)

6. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the calculation of the amounts to be paid based on payment requests submitted by recipients and section 34 certification of payment requests;

ii) provide staff with training on the updated procedures and directives.

3.3.4 Review of Audited Financial Statements

For annual contributions equal to and exceeding $250,000, the recipient’s financial statements are not audited as set out in the terms and conditions of the RBAF, which requires a compliance audit in accordance with the contribution agreement.
APG’s compliance audit requirement, as set out in the RBAF and required under the PTP, is not based on a strategic assessment of recipients’ risk profiles.

The audited financial statements include the balance sheet and income statement of the organization that administers funds provided by AAROM rather than only those statements pertaining to AAROM funding, thus diminishing the effectiveness of the internal control pertaining to audited financial statements.

Moreover, documentation kept in recipient files pertaining to audited financial statements is not always complete and the procedures and directives aimed at ensuring that audited financial statements are reviewed uniformly are not documented.

As an internal control to support the management of AAROM contributions based on the program’s risks, the APG stipulates the following in the program’s RBAF:

  • “An audit will be required for all contributions equal to or exceeding $250,000 [per year]”.
  • “To be in compliance with TB’s requirements for recipient audits, reliance will be placed on the recipient’s auditor [engaged to audit financial statements] to ensure that the recipient audits are conducted according to generally accepted accounting principles (GAAP), generally accepted auditing standards (GAAS) and comply with the provisions of the PTP”.

We noted the following in our review of 12 audited financial statements for the ten recipient files selected:

  • The 12 auditors’ reports with the financial statements we reviewed only contain specific wording to the effect that “the audit was conducted in accordance with generally accepted auditing standards in Canada and that the financial statements present an accurate picture of the financial status of the audited organization and its operating results, according to GAAP in Canada”.

Consequently, audit reports submitted by recipients’ auditors do not include details on PTP compliance (i.e., compliance with all or some of the funding eligibility criteria), contrary to what was stipulated in the RBAF.

APG’s RBAF compliance audit requirement is not based on a strategic evaluation of recipients’ risk profiles, as stipulated in the PTP. Moreover, the contribution agreement clause requiring audited financial statements does not specify the type of audit required, contrary to the RBAF. This clause is worded as follows in the contribution agreements:

“For each of its fiscal years during which it will receive funding under this Agreement and at its own expense provide DFO with audited statements in a form acceptable to DFO with respect to all financial transactions related to the Contribution within 120 days following the end of the fiscal year referred to herein”.

Thus, the APG cannot use the compliance audit as a means of internal control in respect of recipients representing some risk to DFO to ensure that the funds allocated under AAROM are used for the purposes stated in the agreement and in compliance with the requirements and terms/conditions of the PTP. This type of audit involves mandating auditors, who will work on site, to audit the clauses of the agreement as well as conduct a financial audit limited to the transactions funded by the contribution program.

This situation is due to the fact that no plan was developed for systematically assessing the risk profile of recipients, throughout the term of the agreement, to determine which recipients could be subjected to an on-site compliance audit. This matter is addressed in detail in section 3.4 of this report.

  • The financial statements submitted by recipients cover all of the financial transactions of the organizations responsible for administering AAROM funds rather than only AAROM-funded activities. Because most of these organizations also manage funds from other DFO programs, other federal departments and other sources, the audited financial statements pertain to all of these funds, while the results of AAROM-funded operations are presented in a separate appendix attached to the financial statements.

This situation diminishes the effectiveness of the internal control pertaining to audited financial statements, since the statements are for consolidated funds from various sources under the responsibility of the organizations being audited. As a result, there is a greater risk that the funds from AAROM will be used for purposes other than those set out in the agreement, by the audited organization, but that this will not be detected in the audit since the mandate of the financial auditor pertains to the organization rather than to the use of funds allocated under AAROM.

The resulting decrease in internal control pertaining to audited financial statements could be offset by conducting compliance audits based on a risk assessment of each recipient.

  • Of the 12 audited financial statements we reviewed, only five (42% of the sample) included all the financial information, i.e., the auditor’s report, balance sheet, income statement, cash flow statement and notes to the financial statements. Of the remainder, the only information submitted was the appendix showing operating results pertaining to AAROM, accompanied by the auditor’s report. This lack of complete financial statements makes it more difficult for employees responsible for monitoring AAROM agreements to identify financial risks related to the financial position of the organization administering AAROM funds.
  • There is no specific procedure for reviewing audited financial statements, with the result that the review is not done consistently by employees responsible for monitoring AAROM agreements.

Recommendation (High significance)

7. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the audit of financial statements for annual contributions equal to and exceeding $250,000 in order to:

  • clarify the difference between a financial audit and a compliance audit of the terms and conditions of agreements pursuant to the PTP;
  • standardize risk assessment based on information in the financial statements, the assessment methodology and the type of documentation required;

ii) provide staff with training on the updated procedures and directives.

iii) determine the appropriateness of introducing on-site compliance audits as an additional internal control, based on a systematic assessment of recipients’ risk profiles.

3.4 Risk and Results Management

3.4.1 Monitoring of Agreements and Risk Management

Oversight and monitoring of agreements by staff assigned to AAROM is not formally documented, and risk management is not included in all activities in the program’s lifecycle to ensure that the funds allocated to recipients are used as stipulated.

The risk assessment is not documented upon completion of monitoring actions.

We noted a lack of procedures and directives pertaining to documentation of monitoring and risk assessment during the term of the agreement.

The PTP requires “ensuring that cost-effective oversight, internal control, performance measurement and reporting systems are in place to support the management of transfer payments”.

To that end, the monitoring controls implemented by the APG consist in reviewing monthly and final activity reports and reviewing audited financial statements for annual contributions equal to and exceeding $250,000, once the funding application has been approved following an assessment of the proposal and the applicant’s risk profile. Also, for all contributions equal to and exceeding $250,000, HQ reconciles financial data from the annual activity report with the audited financial statements, on an annual basis.

Also, employees and managers in the region conduct on-site visits to meet with the various stakeholders involved in AAROM-funded projects. These visits are also an opportunity to ensure that AAROM-funded projects are achieving their objectives and are consistent with the work plan included in the agreement.

AAROM-related risks and measures to minimize them are identified in the RBAF developed in April 2004 when the program was being developed. The RBAF satisfies the requirements of the PTP. The APG is in the process of reviewing the RBAF in its development of an integrated performance and risk management framework for all of DFO’s Aboriginal programs.

However, according to the documents kept in recipient files that we reviewed and the information gathered from AAROM employees we interviewed, we found that the following is occurring, making it difficult to ensure a corporate memory or proof of due diligence in the area of risk management:

  • None of the monitoring actions are formally documented.
  • The risk assessment is not documented once monitoring for integration and follow-up have been completed. For example, AAROM employees assigned to monitoring agreements could briefly assess the recipient’s risk profile after each monitoring action and enter the profile in a database designed to document recipients' risk profiles throughout the term of the agreements.
  • These situations are the result of a lack of procedures and directives on the documentation of monitoring and risk assessment activities during the term of the agreement.

Because there is no documentation on monitoring actions in the recipient files we examined, we were not able to make a determination on the appropriate use of funds allocated to recipients, in compliance with AAROM contribution agreements.

Recommendation (Medium significance)

8. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the documentation of monitoring actions, recipient risk profile assessment and the entering of risk profiles in a database to ensure adequate risk follow-up, documentation of due diligence, and the preservation of a corporate memory within the APG;

ii) provide staff with training on the updated procedures and directives.

3.4.2 Results-based Management and Accountability Framework

The RMAF satisfies PTP requirements but should be adjusted with respect to expected results and performance indicators to ensure more effective monitoring of the program’s performance. Furthermore, the ongoing monitoring of results and performance is insufficient and does not reflect the data-gathering, analysis and commitments set out in the RMAF.

An integrated performance and risk management framework for all of DFO’s Aboriginal programs is being developed.

According to the PTP, one of the deputy head’s responsibilities is "ensuring that a performance measurement strategy is established at the time of program design, and that it is maintained and updated throughout its lifecycle, to effectively support an evaluation or review of relevance and effectiveness of each transfer payment program".

The APG issued an RMAF in April 2004 when AAROM was being developed. The RMAF satisfies PTP requirements. However, we noted the following:

  • Expected results outlined in the performance strategy seem difficult to measure.
  • There are numerous performance indicators and they are not always relevant to evaluating the effectiveness of the program.

The RMAF lists numerous performance indicators requiring ongoing monitoring for the gathering, analysis and distribution of data. However, according to the documents kept in the recipient files that we reviewed and the information provided by the AAROM employees we interviewed, there is no formal process currently in place for gathering, analysing and distributing, on an ongoing basis, information pertaining to performance measurement, with the exception of the number of jobs created or maintained by AAROM-funded projects. The main performance monitoring activity, as performed by employees responsible for monitoring AAROM, consists in analysing activity reports submitted by recipients. However, the results of this analysis are not formally compiled for ongoing monitoring.

Furthermore, staff members responsible for monitoring the performance of AAROM-funded projects are in regular contact with DFO representatives in the regions during the promotion of new partnerships and opportunities for joint projects to ensure that the performance of AAROM-funded projects is monitored informally.

As a part of the renewal process, APG has developed an integrated performance and risk management framework for all of DFO’s Aboriginal programs.

Recommendation (Medium significance)

9. We recommend that the Director, APG:

i) review the expected results and performance indicators for AAROM to ensure that the program’s performance can be measured effectively;

ii) establish administrative and financial procedures and directives for AAROM concerning performance measurement, i.e., gathering, compiling, analysing and distributing data;

iii) provide staff with training on the updated procedures and directives.

Top of page

4.0 Management Action Plan

Recommendations Management action plan Monitoring report update
Actions completed Actions to be completed Timeframe

1. We recommend that the Director General, APG ensures that:

i) the roles and responsibilities of the various stakeholders responsible for AAROM management are documented, updated and communicated (Medium significance);

ii) AAROM program procedures and guidelines are complete, updated and circulated to the various stakeholders responsible for the program’s management (Medium significance);

iii) monitoring, risk management and performance measurement expectations and responsibilities are clearly defined, communicated and applied at every stage of the program’s life cycle (Medium significance).

A number of initiatives are being pursued under the umbrella of the Integrated Aboriginal Contribution Management Framework (IACMF) that reflect the modern principles of risk management and mitigation including:
  • A national roles and responsibilities template addressing all stages of the program lifecycle is being developed. The integrated program templates will be further populated and maintained by the respective DFO management regions given regional variances in program delivery models. A suite of reference materials are also being developed including an on-line e-book, associated Program Charters and supporting learning modules.
 

National program roles and responsibilities template.

 

June/July 2009

On-line e-book, associated Program Charters and supporting learning modules. Oct/Nov 2009
   
2. We recommend that the Director General, APG ensure that staff assigned to AAROM have enough regular training to ensure that the updated AAROM policies, procedures and directives are implemented consistently (Low significance).

Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle. A suite of reference materials are also being developed including an on-line e-book, associated Program Charters and supporting learning modules.

A regional “expert” will be assigned and be subject to more targeted training (Aug/Sept 2009) in order to provide ongoing assistance to regional and area personnel throughout the 2009-10 transition period.

Initial Atlantic training workshop held April 28-30, 2009.

Initial Pacific training workshop.

June 2009

Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training workshop. Oct/Nov 2009
3. We recommend that the Director General, APG update the terms of reference for the review committee responsible for selecting funding applications in order to enhance its transparency and accountability (Low significance). AAROM Review/Steering Committee Terms of Reference to be updated.   Revised Terms of Reference June 2009
4. We recommend that the Director General, APG:

i) establish procedures for issuing contribution agreements to ensure that all clauses required by the PTP are included in all contribution agreements signed by APG (Medium significance);

ii) include in contribution agreements a description of the objectives and expected outcomes so that both recipients and AAROM managers are able to monitor the performance of the agreements (Medium significance);

iii) determine whether contribution agreements should continue including a clause allowing recipients to retroactively claim eligible expenses incurred before the agreement is signed provided that the expenses are incurred during the term of the agreement and that the agreement has been duly signed (Medium significance).
A number of initiatives are being pursued under the umbrella of the Integrated Aboriginal Contribution Management Framework, including:
  • development of a common contribution agreement template for DFO Aboriginal Programs that reflect requirements of the October 1, 2008 Transfer Payment Policy (TPP) and supporting Aboriginal Directive with respect to the inclusion of standard clauses, monitoring and evaluation;
A focus of the new Aboriginal Directive on use of fixed, flexible and block funding arrangements and multi-year agreements will minimize the future use of the clause referenced in 4 iii).
  Common contribution agreement template June/July 2009
5. We recommend that the Director General, APG:

i) update and communicate AAROM procedures and directives concerning information to be included in activity reports for various payment types (initial and subsequent advances and final payments), to standardize the activity report review process (Medium significance);

ii) provide staff assigned to reviewing activity reports with training on the updated procedures and directives (Medium significance);

iii) re-evaluate the frequency and type of activity reports to be submitted by recipients, based on levels of funding, level of risk to AAROM, effort required of recipients, and the review capabilities of employees responsible for reviewing these reports (Medium significance); and

iv) include in activity reports budget information that can be used to compare projected and actual costs so that employees responsible for reviewing these reports can identify significant variances that require corrective action (Medium significance).

FY 2009-10 is a transition year. Most of the reporting requirements were rolled over from the previous year to current year on a one year basis. Future reporting requirements will be based on a risk based framework as set out in IACMF and new TPP.

In addition, overall program will include budget information that can be use to compare with actual expenses to improve the monitoring function.

 

     
Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle, including learning modules on Monitoring Agreements and Managing Payments, and Evaluating Agreements and Projects. Initial Atlantic training workshop held April 28-30, 2009. Initial Pacific training workshop. June 2009
Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training Oct/Nov 2009
6. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the calculation of the amounts to be paid based on payment requests submitted by recipients and section 34 certification of payment requests (Medium significance);

ii) provide staff with training on the updated procedures and directives (Medium significance).
Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle, including a learning module on Monitoring Agreements and Managing Payments. Initial Atlantic training workshop held April 28-30, 2009. Initial Pacific training workshop. June 2009
Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training workshop Oct/Nov 2009
7. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the audit of financial statements for annual contributions equal to and exceeding $250,000 in order to:
  • clarify the difference between a financial audit and a compliance audit of the terms and conditions of agreements pursuant to the PTP;
  • standardize risk assessment based on information in the financial statements, the assessment methodology and the type of documentation required (High significance);
ii) provide staff with training on the updated procedures and directives (High significance);

iii) determine the appropriateness of introducing on-site compliance audits as an additional internal control, based on a systematic assessment of recipients’ risk profiles (High significance).
Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle, including learning modules on Monitoring Agreements and Managing Payments and Assisting in Risk-based Audits. This will include introduction of an on-site assessment tool for Program Officers, with planned monitoring of contribution agreements to be scheduled in accordance with the risk profile of the recipient. Initial Atlantic training workshop held April 28-30, 2009. Initial Pacific training workshop. June 2009
Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training workshop. Oct/Nov 2009
On-site assessment tool. Sept/Oct 2009
8. We recommend that the Director General, APG:

i) establish administrative and financial procedures and directives for AAROM concerning the documentation of monitoring actions, recipient risk profile assessment and the entering of risk profiles in a database to ensure adequate risk follow-up, documentation of due diligence, and the preservation of a corporate memory within the APG (Medium significance);

ii) provide staff with training on the updated procedures and directives (Medium significance).
Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle, including “Assessing Applications / Making Recommendations for Funding” based on a modern risk management and mitigation approach. Planned monitoring of contribution agreements will be scheduled in accordance with the risk profile of the recipient.Overall program-specific risk profiles will be generated as a result. Initial Atlantic training workshop held April 28-30, 2009. Initial Pacific training workshop. June 2009
Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training workshop. Oct/Nov 2009
9. We recommend that the Director, APG:

i) review the expected results and performance indicators for AAROM to ensure that the program’s performance can be measured effectively (Medium significance);

ii) establish administrative and financial procedures and directives for AAROM concerning AAROM’s performance measurement, i.e., gathering, compiling, analysing and distributing data (Medium significance);

iii) provide staff with training on the updated procedures and directives (Medium significance).

The IACMF includes a new logic model and performance measurement strategy (PMS).

Four training sessions are planned for 2009-10 on the Management of Grants and Contributions incorporating a risk management approach across all stages of the program lifecycle, including Monitoring Agreements and Managing Payments as well as Evaluating Agreements and Projects.

The former will be aligned with the IACMF Performance Measurement Framework.

New logic model and PMS completed. Initial Pacific training workshop. June 2009
Assignment and training of regional “expert”. Aug/Sept 2009
Second Atlantic training workshop. Oct/Nov 2009
Second Pacific training workshop. Oct/Nov 2009

Top of page