Archived – Audit of the At-Sea Mentoring Initiative (ASMI) and the First Nations Fisheries Operations Management Initiative (FOMI)

Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Project Number 6B096
Final Audit Report
March 18, 2010

Table of Contents

List of Acronyms

AICFI
Atlantic Integrated Commercial Fisheries Initiative
APG
Aboriginal Policy and Governance Directorate
ASMI
At-Sea Mentoring Initiative
FOMI
Fisheries Operations Management Initiative
IACMF
Integrated Aboriginal Contribution Management Framework
PAYE
Payables at Year-End
PICFI
Pacific Integrated Commercial Fisheries Initiative
RBAF
Risk-based Audit Framework
RMAF
Results-based Management and Accountability Framework
TB
Treasury Board
T&C
Terms and Conditions

 

1.0 Executive Summary

1.1 Introduction

In 2003, the Department sought and received approval to amend the terms and conditions of the Marshall Response Initiative. Included in this amendment was the approval of the terms and conditions for two new initiatives, the At-Sea Mentoring Initiative (ASMI) and the First Nations Fisheries Operations Management Initiative (FOMI).  In 2004, the Minister announced funding of $6 million and $1.5 million respectively over the life of the initiatives spanning from December 1, 2003 through March 31, 2008. 

The purpose of these initiatives was to provide 34 Mi’kmaq and Maliseet First Nations (First Nations)  communities in New Brunswick, Prince Edward Island, Nova Scotia and the Gaspé region of Québec a foundation for developing a self-sustaining mechanism for continued fish harvesting, technology transfer and capacity building.  These initiatives were aimed at improving the chances that First Nations will be successful in the commercial fishery by continuing to provide fisheries training and mentoring and helping First Nations develop better fisheries management structures at the community level. 

The ASMI assisted First Nations to further develop skills required to fish safely and effectively and to maintain their vessels.  The objectives of the initiatives were not only to provide required training and mentoring but also to build the capacity in First Nations communities to meet their future training requirements independently, better use the access provided under the Fisheries Access Program and enhance economic returns for the benefit of their communities.

The FOMI was designed to support First Nations in their efforts to strengthen fisheries management practices at the community level.  The purpose of the FOMI was to assist First Nations in acquiring the tools and skills to build an internal capacity to establish and manage fisheries enterprises and contribute to a sustainable and self-reliant fishery. 

1.2 Objectives and Scope

The objectives of this audit are to provide assurance that:

  • The Management Control Framework and management practices for the At-Sea Mentoring Initiative and First Nations Fisheries Operations Management Initiative were in place and were adequate to ensure compliance, program effectiveness and financial integrity; and

  • The initiatives complied with the approved terms and conditions and were consistent with the regulatory policies and procedures for controlling contribution payments.

The total combined budget for these two initiatives was $7.5 million over four years.  Based on our audit risk assessment, the scope of this audit included the ASMI Contribution (Vote 10) expenditures, with limited examination of Operating expenditures, and excluded FOMI.

1.3 Statement of Assurance

In our opinion, the auditors have examined sufficient, relevant evidence and obtained sufficient information and explanations to provide sufficient assurance on the reported opinion or conclusions.

1.4 Summary of Observations and Recommendations

The following observations are presented by issues. Although the program has expired, similar activities are currently carried out under an integrated performance and risk management framework which includes six Aboriginal programs, and is referred to as the Integrated Aboriginal Contribution Management Framework (IACMF).  Recommendations of this report are designed to apply to this new framework and the Management Action Plan is developed in light of these recent changes.

Financial Management Controls

Control weaknesses in the financial management control framework resulted in a number of errors in the management of funds. There was limited evidence of project monitoring activities prior to making payments.  Many payments reviewed were not in compliance with the cash management principles of the Treasury Board Policy on Transfer Payments, the reporting requirements defined in the terms and conditions of the program and agreements in place.  While some policies and procedures were developed to aid in the delivery of the ASMI, detailed administrative and financial procedures were deemed to be lacking and management practices and controls were not always adequately documented.

The Director General, Aboriginal Policy and Governance should ensure that detailed, comprehensive administrative and financial procedures are in place for the Integrated Aboriginal Contribution Programs and that staff assigned receive the appropriate training.

Eligibility, Selection and Approval

A selection and approval process was developed and implemented, however the proposal assessments were not documented.

The Director General, Aboriginal Policy and Governance Directorate should ensure that the results of the selection and approval process for the Integrated Aboriginal Contribution Programs are properly documented.

Program Design

Program management allowed for adjustments to the program in response to changes in recipient’s need. While these changes were communicated to participating First Nations in attendance at semi-annual workshops, they were not always reflected in program documentation and therefore guaranteed to be communicated to all parties involved.

The Director General, Aboriginal Policy and Governance should ensure that changes to the program are documented and communicated to all parties involved and reinforced over the duration of the Integrated Aboriginal Contribution Programs.

Issuing and Approving Agreements

Generally, agreements and amendments were issued in accordance with the Treasury Board Policy on Transfer Payments, the terms and conditions of the program and were approved in accordance with the delegated financial authorities, with the exception of some agreements signed toward the end of the program which contained ineligible activities.

The Director General, Aboriginal Policy and Governance should ensure that agreements are in compliance with terms and conditions for the Integrated Aboriginal Contribution Programs before signing them.

Program Monitoring

The Results-based Management and Accountability Framework (RMAF) satisfies the Policy on Transfer Payments and Treasury Board approval requirements, but monitoring of results and performance did not reflect commitments set out in the RMAF. There were no formal processes in place for gathering, analysing and distributing, on an on-going basis, information pertaining to performance measurement.

The Director General, Aboriginal Policy and Governance should ensure that a performance measurement strategy is established, maintained and updated throughout the duration of the program, and implemented as designed  for the Integrated Aboriginal Contribution Programs.

2.0 Introduction

In accordance with the 2009-2010 to 2011-2012 Risk-based Audit Plan for the Department of Fisheries and Oceans Canada, an internal audit of the At-Sea Mentoring Initiative (ASMI) and Fisheries Operations Management Initiative (FOMI) was conducted.  The terms and conditions (T&C) for these initiatives expired on March 31, 2008 and required that an internal audit be conducted within 12 months following the end of the initiatives. 

Although these programs have expired, similar activities are currently carried out under the Atlantic Integrated Commercial Fisheries Initiative (AICFI) being managed as part of the integrated performance and risk management framework that includes six Aboriginal programs in total, and is referred to as the Integrated Aboriginal Contribution Management Framework (IACMF). The new streamlined approach is aimed at improving efficiency in service delivery, strengthening accountability and setting greater focus on achievement of results.  It is in line with the new Treasury Board (TB) Policy on Transfer Payments (2008) and reflective of best practices.

The new IACMF was approved in March 2009.  The framework’s key strategy and related implementation plan were rolled-out in 2009-2010. Recommendations of this report are designed to apply to the new framework and the Management Action Plan is developed in light of these recent changes.

2.1 Background

In 2003, the Department sought and received approval to amend the terms and conditions of the Marshall Response Initiative. Included in this amendment was the approval of the terms and conditions for two new initiatives, the ASMI and the FOMI. In 2004, the Minister announced funding of $6 million and $1.5 million respectively over the life of the initiatives spanning from December 1, 2003 through March 31, 2008. 

The purpose of these initiatives was to provide 34 Mi’kmaq and Maliseet First Nations (First Nations)  communities in New Brunswick, Prince Edward Island, Nova Scotia and the Gaspé region of Québec a foundation for developing a self-sustaining mechanism for continued fish harvesting, technology transfer and capacity building.  These initiatives were aimed at improving the chances that First Nations will be successful in the commercial fishery by continuing to provide fisheries training and mentoring and helping First Nations develop better fisheries management structures at the community level. 

In 2002-2003 an Atlantic-wide Fish Harvesting Mentor Training and Certification Program was carried out during which 80 fisheries mentors were certified. With certified mentors available, the Department decided to proceed with a dedicated First Nations Fisheries Mentor Deployment Program, as a pilot project, for the fiscal year ending March 31, 2004.  During this time, the Minister announced the new ASMI to help the Mi’kmaq and Maliseet First Nations to further develop skills to fish safely and effectively in various fisheries.  In 2004-2005, an internal audit of the pilot project was conducted with the intent that the results would assist in the implementation of the new initiative.

The ASMI assisted First Nations to further develop skills required to fish safely and effectively and to maintain their vessels.  The objectives of the initiatives were not only to provide required training and mentoring but also to build the capacity in First Nations communities to meet their future training requirements independently, better use the access provided under the Fisheries Access Program and enhance economic returns for the benefit of their communities.

The FOMI was designed to support First Nations in their efforts to strengthen fisheries management practices at the community level.  The purpose of the FOMI was to assist First Nations in acquiring the tools and skills to build an internal capacity to establish and manage fisheries enterprises and contribute to a sustainable and self-reliant fishery. 

There was considerable progress made with the ASMI. This progress has resulted in improved collaborative efforts of First Nation communities and the Department.  First Nation preparations leading up to delivery of the mentoring activities were considerable and the department provided support to assist them. First Nations benefited from the initiative through developing training plans, vessel winterizing mentoring and training, delivery of at-sea mentoring, and consequently, increased capacity.  

Top of page

2.2 Objectives and Scope

The objectives of this audit are to provide assurance that:

  • The management control framework and management practices for the At-Sea Mentoring Initiative and First Nations Fisheries Operations Management Initiative were in place and were adequate to ensure compliance, program effectiveness and financial integrity; and

  • The initiatives complied with the approved terms and conditions and were consistent with the regulatory policies and procedures for controlling contribution payments.

The audit focused on the ASMI Contribution (Vote 10) expenditures. Based on our preliminary risk assessment, minimal risk was determined to be associated with the FOMI and considering the low materiality and limited number of transactions, that component was excluded from our audit scope.  The Operating expenditures were not subject to direct examination as they were allocated for program delivery and included in the Directorate’s regular operating budget. The audit included the review of a judgmental sample of transactions associated with ASMI for the period of December 1, 2003 to March 2008 that was representative of the all regions involved.

MarshallFisheries Transition Strategy had a $7.5 million budget for 4 years. Cumulated expenditures totalled $7.5 million as shown in table below.

 

 Fiscal Year

Vote 11 Vote 10

 

ASMI/FOMI ASMI2 FOMI TOTAL

2004-2005

$   661,350

$   279,483

$940,833

2005-2006

517,851

1,025,214

161,902

$1,704,967

2006-2007

428,700

1,965,073

417,964

$2,811,737

2007-2008

501,825

905,449

654,796

$2,062,070

TOTAL

$2,109,726

$4,175,219   

$1,234,662

$7,519,607

Vote 10 represents items related to Grant and Contribution category and extracted from the Management Reporting System. Vote 1 represents items related to Operating category.

1 Amounts were traced to Management Reporting System with the exception of the $315K transferred to the Regions annually.
2Amount includes a total of at least $211.9K in overpayment, with some funds partly recovered at the time of this report.

2.2.1 Audit Criteria

The following audit criteria were identified specifically for the audit of the ASMI following a preliminary assessment of the program’s risk that was completed during the audit planning phase. The criteria were grouped under the following six lines of enquiry:

Program Implementation

  • The department has the authority (legislative and financial) to undertake the initiatives.
  • Recommendations from the previous audit on the pilot project are integrated in design of initiatives.
  • The department is aware of changing needs and addressed those changes with program changes.

Capacity to Deliver

  • Roles and responsibilities are well defined, communicated and understood for those involved in the program.
  • Those involved in the program have the information and tools to effectively deliver the program.

Eligibility, Selection and Approval

  • Funding per applicant is granted and approved based on an impartial decision process and eligibility criteria thus ensuring a transparent process for the Aboriginal communities.

Agreements and Terms and Conditions

Program is administered and monitored in compliance with the T&C of the contribution program, the Policy on Transfer Payments and the Financial Administration Act and more specifically:

  • Agreements are in place based on approved T&C and consistent with applicable TB Policy on Transfer Payments.
  • Rationale is provided for all amendments made to original agreement.
  • All parties sign agreements before the stated start dates and before eligible expenses are incurred.

Monitoring and Payments

  • Requests for payment under the initiatives are approved in compliance with the Financial Administration Act and are supported by proper documentation in compliance with the Policy on Transfer Payments, the T&C of the initiatives and the program internal procedures.
  • Project files are well documented, up-to-date and contain original claims, along with supporting documentation and monitoring/progress reports.

Risk Management and Program Monitoring

  • Program follow-up on risk management and performance meets the requirements of the Risk-based Audit Framework (RBAF) and the Results-based Management and Accountability Framework (RMAF) in compliance with the TB Secretariat policies.

2.3 Methodology

The audit team carried out its mandate in accordance with Government of Canada internal audit standards, as set out in TB’s Policy on Internal Audit.  The approach included the following activities:

  • Review of internal program documentation and reports as well as documentation and reports prepared by outside parties involved in the delivery and management of the program;

  • Review of relevant TB and departmental documentation (e.g. policies and agreements);

  • Review of a sample of eight First Nations and six other recipient files (including Service Providers and Training Plan Partners);

  • Limited review of eleven additional First Nations recipient files; and
         
  • Interviews with Program Management.

In total, 35 payments for First Nations and other recipient were reviewed including advances, progress and final payments, and Payables at Year-End (PAYES), along with supporting documentation.  These payments equated to 30 percent of total ASMI contribution spending of $4.05 million and covered the period of 2004-2005 to 2007-2008.

 

Top of page

3.0 Observations and Recommendations

This section presents observations on the effectiveness of the ASMI internal control framework and on compliance with approved Terms and Conditions and regulatory policies and procedures for controlling contribution payments. They are presented based on issues.

As the ASMI terms and conditions expired on March 2008, recommendations of this report apply to the new IACMF and the Management Action Plan is developed in light of these new changes.

To assist management in preparing its action plan, the audit recommendations have been labelled according to the following three categories to reflect their degree of importance:

  • High significance – major control weaknesses and/or unacceptably high level of risk;

  • Medium significance – control weakness affecting operational efficiency and/or credibility; and

  • Low significance – control or process weakness contributing to inefficiency.

3.1 Management Control Framework

In its review of the ASMI, the audit team found that all recommendations from the Audit of the Mentor Deployment Program had been implemented; the results of which were intended to assist in the implementation of the new initiative.  As a result of their implementation, the processes for the administration of the ASMI were improved.

Improvements included the development of a strategic communication plan to increase awareness of the Initiatives and the implementation of a selection process to select Service Providers. The policy and procedures were updated to include an application and approval process for mentor deployment projects and required documentation, including an approved training plan; guidelines for First Nations in selecting a Service Provider, guidelines for the selection of trainees and mentors; and guidelines for the evaluation of trainees and mentors.

3.1.1 Financial Management Controls

Control weaknesses in the financial management control framework resulted in a number of errors in the management of funds.

There was limited evidence of project monitoring activities prior to making payments.  Many payments reviewed were not in compliance with the cash management principles of the TB Policy on Transfer Payments, the reporting requirement set in the T&C of the program and agreement in place.  While some policies and procedures were developed to aid in the delivery of the ASMI, detailed administrative and financial procedures were deemed to be lacking and management practices and controls were not always adequately documented.

The TB Policy on Transfer Payments emphasizes accountability and transparency through sound management practices, responsible spending and effective controls.

In our review of 35 payments from the 14 recipients files selected, the audit noted a number of errors in the account verification process such as the lack of supporting documentation, calculations that were made incorrectly or could not be validated, PAYE set-up without required evidence and funds used for other purposes.  Incomplete records made it difficult to determine whether monitoring and decisions made by program staff were appropriate.  Specifically, we have found the following:

Supporting documentation and calculation – 19 out of 35 payments made to recipients were approved without all required documents stipulated in the agreements and 13 out of 35 payments were either calculated incorrectly or could not be validated as required information was missing or inadequate. There was inconsistency in the documentation provided as proofs of payment and cost sharing. The audit team found limited evidence of review of supporting document that was received and tabulation necessary to ensure that all costs were eligible and respected the cost sharing arrangement set in agreements to support the issuance of payment as described in the program’s policies and procedures. 

PAYE - The TB Policy on Payables at Year-End states that the entitlement of the recipient to be paid must be established and quantified before an amount is set up as a PAYE.  In five out of eight PAYE set up, there was no specific documentation contained in the contribution files reviewed that explained calculations and the rationale related to the PAYE.

Receiving and verifying required supporting documents is essential to good management of funds. Incomplete records make it difficult to determine if payments were issued as intended; were compliant with T&C of agreements (e.g. compliant with cost sharing arrangement) and whether monitoring and decisions made by program staff was appropriate and approved by management. It posed the risk that overpayments or underpayments are made resulting in debts due to the Crown and administration inefficiency.

As a result of the above, overpayments were made to recipients in three out of 14 files reviewed, resulting in debts due to the Crown.  Furthermore, Program Management confirmed the existence of other overpayments which combined totalled at least $211.9K.  Arrangements were taken for these funds to be reimbursed and repayment schedule negotiated to recover the full amount. Debt due to the Crown is a lengthy process to manage and could potentially create relationship issues with recipients. 

Based on Program Management’s budget logs, evidence exists that Operating funding provided for the ASMI were expended in support of the AICFI without appropriate authorization from TB.  In total, the budget logs clearly identify $95.8K of ASMI funding being spent on AICFI in 2007-2008.  Management acknowledged that they did in fact use ASMI funds to support the development of the AICFI program but that a request was put forward to transfer ASMI lapsed funding to the AICFI.

Policies and procedures are a key part of a management control framework, and critical to its success. Policies outline management’s expectations of how the organization should manage its operations, including the roles and responsibilities of the different parts of the organization.  Policies and procedures need to be consistent with TB policy and directives and must be communicated and easily accessible to the organization.

A manual was developed in 2005 with the intent of providing Program Management and other co-deliverers with the tools, policies and procedures to manage the ASMI.  The Policy and Procedures Manual (Manual) provided high-level direction, however it lacked detailed administrative and financial procedures required for day-to-day program delivery. As a result, the aforementioned errors were identified in the administration of contribution funds.

Additional tools were developed on an ad hoc basis to supplement the manual such as a payment checklist to help reconcile claims and ensure all supporting documents were received.  The audit team did not see evidence of consistent use of the checklist, which often resulted in insufficient documentation as identified in the T&C of the program, as discussed earlier in the report.

At a minimum, based on good management practices, the Manual should cover key information, such as roles and responsibilities for Program staff, procedures related to project monitoring, payment verification and documenting of monitoring actions.  It should also cover PAYE and carry forward requirements.

Recommendation (High significance)

1.   The Director General, Aboriginal Policy and Governance should ensure that detailed, comprehensive administrative and financial procedures are in place for the Integrated Aboriginal Contribution Programs and that staff assigned receive the appropriate training.

3.1.2 Eligibility, Selection and Approval

A selection and approval process was developed and implemented, however the proposal assessments were not documented.

Based on their experience with the Mentor Deployment Program, First Nations recommended a number of improvements to be considered in the development and implementation of the ASMI, including the need for a transparent approval process.  Information provided by program applicants allows for informed decision-making with respect to the granting of funds, and funding applications are evaluated and approved impartially. 

The audit of the Mentor Deployment Program concluded that there was no clear process for the request and approval of projects.  To address this deficiency, Program Management updated their Policy and Procedures Manual (Manual) in April 2005 to outline the application, review and approval process for projects and included documentation, such as a training plan template and a checklist to help review the Fisheries Training Plans and proposals for deployment of mentors associated with the ASMI.

To ensure an appropriate selection process, Program Management created two Technical Advisory Committees; one for the Québec/Gulf area and the other for the Scotia-Fundy area. Each Technical Advisory Committee, involved one or two people possessing technical expertise in fisheries training and were paid by the Department to help deliver the program.

At the commencement of the program, all First Nations affected by the Marshall decision were contacted by a representative of the Department to explain the new initiative and to encourage participation. As a prerequisite for participation in the ASMI, and as a basis for their Mentoring Deployment Proposal, each First Nation was asked to develop a detailed, multi-year training plan based on the template with the assistance of a Training Plan Partner, paid for by the Department. The Band coordinator, in collaboration with the Service Provider for the deployment, was then required to prepare a proposal for the deployment of mentors in accordance with the training plan.

The Technical Advisory Committee was to review/assess the Fisheries Training Plans and the Mentor Deployment Proposals and make recommendations to the Program Authority, who would give the final approval.  It was this assessment that was the basis on which funding decisions were made.  The review/assessment was to be based on the checklist included in Appendix C of the Manual which provided an outline for their evaluation.  The Manual did not require the Technical Advisory Committee to formally document their assessment on the form provided but did require that their assessment reflect the degree to which the application meets objectives and/or provide the required information for decision making. Based on the file review, there was no evidence that the tasks listed on the checklist were done or documented however some files contained Technical Advisory Committee correspondence related to the mentor deployment proposal and their support for the proposal. The lack of documentation on the project evaluation does not reflect fully the transparency and accountability of the selection process.

Recommendation (Medium significance)

2.  The Director General, Aboriginal Policy and Governance Directorate should ensure that the results of the selection and approval process for the Integrated Aboriginal Contribution Programs are properly documented.

3.1.3 Program Design

Program Management allowed for adjustments to the program in response to changes in recipient needs, however these changes were not always documented and communicated to all parties involved.

The department is aware of changing needs and addressed those changes with program changes. The mentoring initiatives involved co-delivery with Service Providers for the deployment of mentors.   As part of the program design, First Nations were required to engage a pre-authorized Service Provider to manage the process in the field.  Service Providers were to train, coordinate the mentors, dispatch the mentors and monitor, evaluate and report to Program Authority on all deployment activities.

Service Providers were to provide mentors through direct-hire or sub-contract and pay for their services.  During the course of this initiative, some First Nations recipients requested to hire and pay the mentors directly.  Program Management agreed to this change on the condition that they provide proof that service had been rendered and mentors paid and this was the case in one out of two Service Providers. Enhanced reporting requirements were reflected in the agreement signed between Fisheries and Oceans Canada and First Nations, and the contract between First Nations and Service Providers were structured accordingly but this change was not updated in the program documentation.  Program Management stated that these changes were clearly communicated at their semi-annual workshops to which all participating First Nations were invited to attend including the Service Providers, but was unable to provide evidence supporting the communication on this adjustment. With the changes not being formally reflected in the program documentation, potential applicants may not have easy access to common program knowledge. Furthermore, this change had an impact on the co-deliverer’s responsibilities and the verification of payment process.

Based on the file review conducted by the audit team these proof of payment and related details of training days were not always well supported which could have led to mentors not being paid appropriately for the training rendered.  Thus, Program Management did not always have assurance that mentors were being paid, and consequently funds reimbursed could have been used to subsidize other activities, however no complaints were received from mentors.

Recommendation (Medium significance)

3.   The Director General, Aboriginal Policy and Governance should ensure that changes to the program are documented and communicated to all parties involved and reinforced over the duration of the Integrated Aboriginal Contribution Programs.

3.1.4 Issuing and Approving Agreements

Generally, agreements and amendments were issued in accordance with TB Policy on Transfer Payments, the T&C of the program and were approved in accordance with the delegated financial authorities set out, with the exception of some agreements signed toward the end of the program which contained ineligible activities.

Agreements and amendments examined by the audit team as part of the file review all contained the mandatory clauses outlined in the 2001 TB Policy on Transfer Payments and were signed by the appropriate authority named in the T&C of the program.  However some agreements, particularly those signed near the end of the program, contained activities which were not eligible under the T&C of the program. 

Following the file review, 11 additional files containing agreements signed within the last three months of the program were identified for limited review.  These agreements had a total approved value of $351K, of which $225K was for training not eligible under the T&C of the program.  A total of $163K was actually incurred and subsequently reimbursed, some of which after the program’s T&C had expired. Program Management expressed that they were aware of this but that the decision was made in response to new Transport Canada regulations announced in the fall of 2007 that came into effect in December 2008, for On Board Personnel to have First Aid and Marine Emergency Duties training.  Program Management felt that there was a risk that fishers who had been mentored under the ASMI would not be in compliance with Transport Canada’s new regulatory requirements unless financial assistance was provided.  Program Management felt that timelines would not have permitted them to pursue any other course of action.

Department must obtain TB approval of the terms and conditions of all contribution programs. The Minister may approve technical changes and exceptions to the T&C previously approved by TB under certain conditions; departments must consult the Secretariat about whether TB approval is needed before implementing changes and inform the Secretariat in writing of any amendments or exceptions approved under these provisions within one month after the Minister has approved them.  Program Management did not engage in this process and approved the non eligible training without requesting appropriate approval for the change in T&C through required channels. 

Recommendation (Medium significance)

4.   The Director General, Aboriginal Policy and Governance should ensure that agreements are in compliance with terms and conditions for the Integrated Aboriginal Contribution Programs before signing them.

3.1.5 Program Monitoring

The Results-based Management and Accountability Framework (RMAF) satisfies the Policy on Transfer Payments and TB approval requirements but monitoring of results and performance did not reflect commitments set out in the RMAF. There were no formal processes in place for gathering, analysing and distributing, on an on-going basis, information pertaining to performance measurement.

To enable management and program officers to successfully carry out their oversight and management reporting roles, they must receive or have access to accurate, relevant, and timely information. Typically, financial and non-financial information is needed to spot inconsistencies, trends, and support decision making to take informed decisive action.

TB’s previous Policy on Transfer Payments, effective June 1, 2000, established that it is government policy to “manage transfer payments in a manner sensitive to risks, complexity, accountability for results and economical use of resources”. It also states that Departments must establish policies and procedures to ensure that a RMAF is prepared to provide for appropriate measuring and reporting of results. Conditional to the program approval, TB requested that a RMAF and a RBAF be developed.

A RMAF for the ASMI and FOMI initiatives was developed in August 2004 to facilitate tracking, reporting and monitoring activity. The RMAF introduced a Logic Model (an illustration of how activities are intended to achieve planned outcomes), an Ongoing Performance Measurement (PM) Strategy, an Evaluation Strategy and Reporting Strategy. 

The PM strategy called for ongoing monitoring of outputs and outcomes identified in the Logic Model in order to identify progress in achieving results.  While the RMAF was developed to meet TB requirements, monitoring of results and performance did not reflect the data-gathering, analysis and commitments set out in the RMAF. 

Current Program Management indicated that the RMAF developed for this program was not salient to the effective monitoring of results.  Accordingly, ASMI performance results were tracked using ad hoc methods.  Management reported senior management was updated on the status of the ASMI regularly through bi-weekly conference calls.  They by no means missed an opportunity to redirect and improve the program due to lack of monitoring.

Nonetheless, review of evidence provided suggests that performance results were infrequently tracked and results conveyed were inconsistent from time to time. There was no formal process in place for gathering, analysing and distributing, on an on-going basis, information pertaining to performance measurement.

Program Management undertook an independent review of progress made from April 1, 2004 to November 30, 2007 using an external consultant with the report being tabled at the end of 2007, one year before the end of program.  This report provided key findings and suggestions for improvements to take into account in the continuing follow-up work under the AICFI program as it encompasses some of the activities carried out under the ASMI. 

An evaluation of the initiative was not conducted in 2007-2008 as stated in the RMAF and required by TB but will be carried out by the Department’s Evaluation Directorate in 2010-2011.

TB’s current Policy on Transfer Payments (October 2008) states that administrative requirements on applicants and recipients should be proportionate to the risk level. In particular, monitoring, reporting and auditing should reflect the risk specific to the contribution program, the value of funding in relation to administrative costs, and the risk profile of the recipient.  The Aboriginal Policy and Governance Directorate has developed an integrated performance and risk management framework for all of the Department’s Aboriginal programs based on the current Policy. Program Management explained that many improvements were made with the AICFI performance framework and that they are using it on a regular basis.  The audit did not review the AICFI framework or the implementation of it.

Recommendation (Medium significance)

5.   The Director General, Aboriginal Policy and Governance should ensure that a performance measurement strategy is established, maintained and updated throughout the duration of the program, and implemented as designed  for the Integrated Aboriginal Contribution Programs.

Top of page

4.0 Conclusion

Considerable efforts have been made by the Department to respond to the Supreme Court of Canada’s 1999 Marshall decision, including continuous efforts to respond to the needs of the First Nations communities by creating new opportunities to better assist them. The ASMI mentoring process has been established as an effective method of technology and knowledge transfer to First Nations and its activities were quite beneficial despite the challenges to engage communities. ASMI helped the Department improve its working relationships with First Nations to build capacity in First Nations communities to meet their future training requirements independently, better use the access provided under the Fisheries Access Program and enhance economic returns for the benefit of their communities.

While a number of good control practices were identified and efforts were made while addressing the changing needs of recipients, there were areas of the ASMI policy and management control framework that either required strengthening or needed to be established.  Management has provided their assurance through completion of the Management Action Plan that issues identified in this report will be addressed in the implementation of the Integrated Aboriginal Contribution Management Framework.

5.0 Management Action Plan

Recommen-
dations
Office of Primary Interest Management
action plan
Status report update
Actions
completed
Actions
outstanding
Target
date

1. The Director General, Aboriginal Policy and Governance should ensure that detailed, comprehensive administrative and financial procedures are in place for the Integrated Aboriginal Contribution Programs and that staff assigned receive the appropriate training (High significance).

Director General, Aboriginal Policy & Governance

  • Director, Aboriginal Programs
  • Director, AICFI
  • Director, PICFI
  • Prepare an Agreement Administration Handbook, reviewed by DFO Finance, and provide training to staff.

 

AICFI:

  • Contribution Agreement Administrative Process
  • Payment checklist
  • Governance charter
  • Management Control Framework

PICFI:

  • Governance Charter

APG:

  • IACMF Agreement Administration Handbook

 

July, 2010

2. The Director General, Aboriginal Policy and Governance Directorate should ensure that the results of the selection and approval process for the Integrated Aboriginal Contribution Programs are properly documented (Medium significance).

Director General, Aboriginal Policy & Governance

  • Director, Aboriginal Programs
  • Director, AICFI
  • Director, PICFI
  • Documentation and implementation of program application procedures, including approval responsibilities and processes.

 

  • Development of the APG Information System to store documentation of high level assessment and approvals for program funding.

AICFI:

  • Capacity Building in Fisheries- An Application Guide of Mi-kmaq & Maliseet First Nations (AICFI Application Guide)

 

  • AICFI Management Information System in place, data entry reviewed including documentation regarding approvals.

APG:

  • Remaining Application Guidelines (Aboriginal Fisheries Strategy, Aboriginal Aquatic Resource Oceans Management, Aboriginal Fund for Species At Risk, PICFI)
  • APG Information System under development; Phase I will include storing of high level assessment and approval documentation.

 

July, 2010

 

 








August, 2010

 

3. The Director General, Aboriginal Policy and Governance should ensure that changes to the program are documented and communicated to all parties involved and reinforced over the duration of the Integrated Aboriginal Contribution Programs
 (Medium significance).

Director General, Aboriginal Policy & Governance

  • Director, Aboriginal Programs
  • Director, AICFI
  • Director, PICFI
  • Program changes documented and communicated to parties.

AICFI:

  • Reporting Procedures Handbook for Recipients
  • AICFI Application Guide
  • Component 4 – Commercial Fisheries Development Opportunity Source – Application Procedures
  • AICFI Internet and Intranet sites routinely updated with current program information.

APG:

  • All Program Governance Charters will include a section detailing that “any significant change to the administration of this program must be documented and approved by the appropriate authorities and communicated to all parties”.

 

May, 2010

4. The Director General, Aboriginal Policy and Governance should ensure that agreements are in compliance with terms and conditions for the Integrated Aboriginal Contribution Programs before signing them (Medium significance).

Director General, Aboriginal Policy & Governance

  • Director, Aboriginal Programs
  • Director, AICFI
  • Director, PICFI
  • Provide instruction to all DFO program personnel involved in the administration of contribution agreements in regards to adherence to IACMF terms and conditions.

AICFI:

  • Contribution Agreement Administrative Process Checklist
  • [Solicitor client privilege]

APG:

  • All Program Governance Charters will include a section detailing that “all staff involved in that administration of contribution programs should review on a consistent basis the IACMF terms and conditions” and “for activities which do not fall within the confines of the IACMF terms and conditions, full approval must be obtained from the appropriate authority”.

 

September, 2010

5. The Director General, Aboriginal Policy and Governance should ensure that a performance measurement strategy is established, maintained and updated throughout the duration of the program, and implemented as designed  for the Integrated Aboriginal Contribution Programs (Medium significance).

Director General, Aboriginal Policy & Governance

  • Director, Aboriginal Programs
  • Director, AICFI
  • Director, PICFI
  • Development of IACMF Performance Management Framework and regular reporting to Senior Management.

AICFI:

  • Key performance indicators tracked within the Management Information System
  • Annual reporting to Senior Management Aboriginal Policy Committee

APG:

  • IACMF Performance Measurement Framework

APG:

  • All Program Governance Charters will include a section detailing that “Program Authorities shall report to Senior Management against the IACMF Performance Measurement Framework at regular intervals”.
  • On-going key performance indicators will be tracked on a regular basis.

 

September, 2010

 

 











March, 2011