ABORIGINAL INLAND HABITAT PROGRAM

---

Project number 6B130
January 2012

TABLE OF CONTENTS

• 1.0 EXECUTIVE SUMMARY
• 1.1.      Introduction
• 1.2.     Objectives and Scope
• 1.3.      Overall Conclusion
• 1.4.      Summary of Findings and Recommendations
• 1.5.5   Recommendations
• 2.0    INTRODUCTION
• 2.1.     Background
• 2.2.     Objectives and Scope
• 2.3.    Methodology
• 3.0     Findings ans Recommendations
• 3.1.     Relevance
• 3.2      Performance
• 3.3      program design and Delivery
• 3.4      Lessons Learned
• 3.5      Should the Program Continue or be Terminated?
• 3.6      Recommendations
• 3.7      Conclusions

List of acronyms

 

1. EXECUTIVE SUMMARY

1.1 INTRODUCTION

The Aboriginal Inland Habitat Program (AIHP) was a contribution program whose objective was to enhance the ability of Aboriginal groups, working together, to participate in the decision making processes related to habitat management regulatory and non-regulatory activities.  The AIHP applied to Aboriginal groups located in the inland provinces (Alberta, Saskatchewan, Manitoba, Ontario and parts of Quebec) where the provinces manage the fisheries but Fisheries and Oceans Canada (DFO) maintains responsibility for the management of fish habitat.  In September 2004, funding was approved ($9M over five years) for the program.  In 2009, the terms and conditions and funding for the AIHP were renewed.

In 2008-09, an evaluation was conducted on the program.  It noted deficiencies in implementing the program, such as the lack of dedicated staff and signed agreements, as well as the ineffective use of program funding.  The evaluation recommended that a follow-up evaluation be carried out in 2010-11 and a decision be made at that time on whether the program should be continued or terminated.

1.2 1.2 OBJECTIVES AND SCOPE

This evaluation focused on the Core Issues in Assessing Value for Money in Evaluation as defined by the Treasury Board Policy on Evaluation (2009).  That is, the evaluation examined issues of relevance and performance, including effectiveness, economy and efficiency.

The evaluation was conducted from July to November 2010 in the National Capital Region, with data collection undertaken in the Central and Arctic and Quebec Regions and covered the period covered by the current evaluation included the 2009-10 fiscal as well as an examination of 2010-11 information, where available.

1.3 1.3 OVERALL CONCLUSION

The evaluation found that there was evidence that the program has made some progress in building collaborative management capacity and establishing infrastructure, governance, processes and relationships with Aboriginal groups.  It did not however make substantive progress in achieving the expected result of having Aboriginal groups collaborating and participating in the sustainable management and conservation of fisheries, aquatic and oceans resources.

Since the previous evaluation, AIHP staff had made a concerted effort to improve and measure the performance of the program.  There had been an increase in the number of contribution agreements entered into and there were an increasing number of communities represented by AIHP-funded organizations.  These improvements, in addition to listings/compilations of activities undertaken by Aboriginal organizations, did not demonstrate any impact on or linkages to the achievement of AIHP objectives or DFO priorities.

While many activities were being undertaken by the Aboriginal organizations and some capacity building for habitat management, particularly within the organizations themselves, had been achieved, it was not evident that there was a clear vision or strategic focus for the AIHP, particularly as it related to the continuing level of capacity building required by Aboriginal organizations or how they were going to address those needs in the future.

In addition, it was not clear how these organizations would assist DFO in delivering habitat regulatory and non-regulatory activities, given the limited involvement with DFO’s Habitat program.

Based on the evidence examined during the evaluation, the evaluation team is unable to conclude that the program had made substantive progress in achieving its expected results or that there was a reasonable expectation of doing so in the near future.

1.4 1.4 SUMMARY OF FINDINGS AND RECOMMENDATIONS

1.4.1 Relevance

The AIHP had demonstrated relevance in terms of a continuing need for the program and there was a role for Federal government intervention. The objectives of the AIHP were relevant to DFO objectives and the program was aligned with DFO’s Strategic Objective of Healthy and Productive Aquatic Ecosystems.

1.4.2 Performance

Performance is defined by the Treasury Board’s Policy on Evaluation as the extent to which effectiveness, efficiency and economy are achieved by a program.

Effectiveness

Since the previous evaluation, AIHP staff had made a concerted effort to improve and measure the performance of the program.  The program had made progress in achieving the immediate outcome of increasing collaborative management capacity, infrastructure, governance, processes and relationships in place with Aboriginal groups in terms of having AIHP organizations in place to assist communities.  However, the program is not making substantive progress in achieving its expected outcome of Aboriginal groups collaborating and participating in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

Efficiency and Economy

When the program was approved in 2004, it included funding for one full-time equivalent (FTE) position.  Despite receiving funding for an FTE, when the program was implemented, there was no full-time staff assigned to AIHP at National Headquarters.  In June 2008, a full-time position of AIHP Manager in the Central and Arctic Region was staffed on an acting basis and a competitive process was held to fill the position on a permanent basis.  The region had also assigned another individual to the program to support (and provide back-up) to the AIHP Manager in delivering the program. 

There was minimal funding provided to the Department for the implementation of the AIHP.  Since the program was delivered primarily by the Central and Arctic Region, program funding was provided to this Region.  There were no salaries or Operations and Maintenance funds available for the Quebec Region or NHQ. 

Because of problems with the initial implementation of the program, there were very few agreements entered into in the first three years of the program.  At the time of the evaluation, tools were in place to guide DFO staff and Aboriginal Organizations and contributed to program improvements.

At the time of the evaluation, the AIHP had program support in place, however, it was evident that the lack of dedicated DFO program staff greatly impacted how the AIHP was delivered in its early days, as well as its reputation both within and outside of DFO.  

1.4.3 Lessons Learned

Following are lessons learned that were identified during the evaluation:

Consultation Process

• Consultation with Aboriginal organizations and communities should have taken place early in the planning phase for the new program.  This could have helped set out the program objectives and determine the expectations of potential participants in the program.
• The consultation process could also have included engaging the provinces as they are managing fisheries in the inland provinces.

Linkages to Other Aboriginal Programs

• There could have been closer linkages with the other DFO Aboriginal programs.  The sharing of expertise that existed in the Ecosystems and Fisheries Management Sector could have increased the probability for the success of the AIHP. 

Integration with the Habitat Management Program

• The AIHP was not integrated into DFO’s Habitat Management Program.  The AIHP operated in isolation or in a silo on its own. 

Capacity to Deliver Program

• The AHIP was provided with a small amount of contribution funding for a potentially large number of Aboriginal groups and communities. 
• Adequate capacity (staff) to deliver the program should have been incorporated into the program.

Framework to Deliver Program

• A proper framework that sets out the governance structure, guidelines and tools such as reporting templates and an assessment grid should have been in place prior to implementing the program.

Strategic Direction for the Program

• Strategic direction or a vision as to what Departmental objectives would be addressed through the activities carried out by the Aboriginal organizations would have been beneficial. 

1.4.4 Should the Program Continue or be Terminated?

The 2008-09 evaluation recommended that a follow-up evaluation be carried out in 2010-11 and a decision be made at that time on whether the program should be continued or terminated.

When individuals, representing the Aboriginal organizations and DFO, were asked if they believed that the program should be continued, they were overwhelmingly of the view that the program should continue.  Many of the reasons given for the AIHP to continue focused on the improvements that had been made to the administrative and management aspects of the program.

In interviews, representatives of Aboriginal organizations expressed the view that poor management on the part of DFO caused the problems in the implementation and management of the program.  This view was supported by the lack of a consultation strategy and engaging the Assembly of First Nations to assist in rolling out the program; and the lack of a framework (e.g. policies, guidelines) for the AIHP at the time of its implementation. 

While the views of those interviewed during the evaluation are important, it was necessary to consider all of the evidence relating to the AIHP and the results achieved.  Based on the information reviewed for the case studies, the documentation review and interviews, there was evidence that AIHP had made progress in achieving the immediate outcome of: collaborative management capacity, infrastructure, governance, processes and relationships in place with Aboriginal groups.  The program, however, was not making substantive progress in achieving the intermediate outcome of Aboriginal groups collaborating and participating in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

The evaluation team was unable to conclude that the program had made substantive progress, after six years of funding, in achieving its expected results in terms of the intermediate outcomes or there was a reasonable expectation of doing so in the near future. 

1.4.5 RECOMMENDATIONS

It is recommended that:

1. The Terms and Conditions for the Aboriginal Inland Habitat Program not be renewed.
2. A close-out strategy for the program be prepared that would take into consideration the following:

• the need for a communication strategy that would inform Aboriginal organizations and communities about the termination of the program;
• the need for a human resource plan to take into consideration the impact on any affected DFO employees;
• The mechanism for dealing with multi-year contribution agreements that are in place; and
• The possible ways of dealing with the needs that the program was intended to address.

2.0 INTRODUCTION

In accordance with Fisheries and Oceans Canada’s (DFO) 2010-11 Departmental Evaluation Plan, an evaluation of the Aboriginal Inland Habitat Program (AIHP) was conducted.  This document reports on the result of the evaluation. 

In 2008-09, an evaluation was conducted of the AIHP.  That evaluation noted deficiencies in how the program was implemented and recommended that a follow-up evaluation be carried out in 2010-11 and a decision be made at that time on whether the program should be continued or terminated.

2.1 BACKGROUND

The Government of Canada is responsible for Habitat Management in the inland provinces. The AIHP was intended to support and assist DFO’s Habitat Management Program (HMP) in meeting its responsibilities under the Fisheries Act, Canadian Environmental Assessment Act (CEAA), other Environmental Assessment (EA) legislation in Canada, as well as the Species at Risk Act (SARA) to conserve and protect fish and fish habitat in support of ongoing and future fisheries in Canada.  

In the 1990s and early 2000s, there was a growing and maturing evolution in the nature of DFO’s programming and of its mandate generally.  This included a focus on the integrated management of fisheries and on broader watershed or ecosystem-based management.

There had also been growing demand and interest on the part of DFO stakeholders to engage the Department and other public agencies in the integrated planning and management of natural resources, including fisheries.  These led to the development of new approaches to fisheries management and the emergence of new models of governance and of shared stewardship. These new models included the capacity for local and Aboriginal traditional knowledge to contribute to resource and habitat management decision making processes and activities. 

Aboriginal groups, however, had little or no capacity, either institutional or financial, to engage effectively with DFO and others in such processes and, even less so, at a broad watershed or ecosystem level.  The implementation of the AIHP, a contribution program, was intended to enhance the capacity of Aboriginal groups, working together, to participate in the decision making processes related to habitat management regulatory and non-regulatory activities.

In September 2004, funding of $9M over five years was approved for the AIHP.  The terms and conditions and funding for the AIHP were subsequently renewed in 2009.   

The AIHP applied to Aboriginal groups located in the inland provinces (Alberta, Saskatchewan, Manitoba, Ontario and parts of Quebec) where the provinces manage the fisheries but DFO maintains responsibility for the management of fish habitat.

Through the AIHP, DFO intended to assist inland area Aboriginal groups to develop the capacity and to engage in collaborative management of fish habitat.  This approach was intended to allow for improved coordination with Aboriginal groups on habitat matters and more effective delivery of DFO’s habitat management program responsibilities.  The AIHP had two main components:  

Capacity Building

Capacity building was intended to facilitate the involvement of Aboriginal people in the collaborative management of the fisheries and fish habitat.  The AIHP contributed to the creation of this capacity by supporting Aboriginal organizations that have the governance, management and administrative infrastructure to coordinate the delivery of the AIHP contribution agreements.

Collaborative Management

Through contribution agreements with Aboriginal organizations, the AIHP directly supported Aboriginal groups and communities to collaborate in decision making processes that had potential impact on fish and fish habitat.  In addition, the AIHP enabled Aboriginal people to participate in initiatives/projects that support the conservation and enhancement of fish habitat.  

The program strived to build Aboriginal capacity, promote constructive relationships and the development of structures and processes relating to habitat management with DFO and others involved in habitat management (e.g., other government departments, provinces, municipalities, industry and stewardship groups).

Greater coordination with Aboriginal groups in inland areas on habitat matters was intended to allow for more effective delivery of DFO’s habitat management responsibilities.  It would also help Aboriginal groups and DFO work more effectively to understand the implications and identify the impacts on First Nations of projects or proposals that affect fish habitat.

On March 31, 2009, the Terms and Conditions (Ts&Cs) of all of DFO’s Aboriginal Contribution Programs, including the AIHP, were scheduled to terminate. In preparation for renewal of the Ts&Cs, DFO initiated the integration of DFO Aboriginal Contribution Programs under the Integrated Aboriginal Contribution Management Framework (IACMF). The IACMF emphasized the commonalities that existed among the key activities, outputs, and outcomes of DFO’s Aboriginal Contribution Programs. The creation of the IACMF allowed these programs to be represented in one Logic Model with one common Performance Measurement Strategy.

Integration of DFO Aboriginal Contribution Programs was based on the identification of Contribution Program “Results Chains” that focus on the commonalities of outcome themes among the six DFO Aboriginal Contribution Programs.  One of the “Results Chains” was identified as Aquatic Resource Management and Stewardship whose objective was to enhance the ability of Aboriginal communities to participate in collaborative management structures that contribute to aquatic resource and habitat advisory and decision-making processes. It was this chain that the AIHP is most closely aligned to. 

The AIHP’s expected results as stated in the IACMF were as follows:

• Collaborative management capacity, infrastructure, governance, processes and relationships are in place with Aboriginal groups; and
• Aboriginal groups collaborate and participate in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

The AIHP was aligned with DFO’s 2010-11 Strategic Outcome of Healthy and Productive Aquatic Ecosystems as a program sub-activity.  The expected result for the AHIP as identified in the Performance Measurement Framework for the Management Resources and Results Structure (MRRS) was:

• Aboriginal groups in Quebec, Ontario, and the Prairie provinces have the necessary capacity to contribute to the conservation and protection of fish and fish habitat and improvement of fish habitat.

2.1.1 Resources

The AIHP received initial approval as of April, 1, 2004 for a five-year period.  Funding for the AIHP was subsequently approved for two additional years as of April 1, 2009. 

(000 $)
	2006­2007	2007­2008	2008­2009	2009­2010*	2010­2011*	Total
Contributions	$ 1,875	$ 1,875	$1,875	$ 1,875	$ 1,825	$ 9,375
Operations and Maintenance (O&M))	0,225	0,225	0,225	0,225	0,225	1,125
Total	$ 2,100	$ 2,100	2,100	$ 2,100	$ 2,100	$ 10,500

*Funding was approved for 2 years from April 1, 2009 to March 31, 2011

At the time of the evaluation, there were two Full - Time Equivalent positions assigned to the delivery of the AIHP.

2.1.2 Governance

The Assistant Deputy Minister (ADM), Ecosystems and Fisheries Management (EFM), was accountable for the overall delivery of AIHP.  The Associate Assistant Deputy Minister, Ecosystems and Fisheries Management signed the AIHP agreements on behalf of Canada and was responsible for the overall co-ordination of the program.

At the beginning of the program, negotiation of the terms and conditions of the AIHP agreements with Aboriginal groups, as well as overall program delivery and administration, was the responsibility of the then Habitat Management Directorate at National Headquarters (NHQ).  In the second year of the program (February 2006), program delivery and administration was delegated to the Central and Arctic (C&A) Region insofar as signing authority for the approval of payments for the AIHP was sub-delegated to the Regional Director, Habitat. Because some Aboriginal organizations in Quebec were also eligible to participate in AIHP, there was involvement of DFO’s Quebec Region in the program. 

The program operated under the direction of the AIHP Steering Committee, a senior-level committee responsible for leadership and direction of the Program.  The Steering Committee was supported by an Operations Committee, which was responsible for the management and day-to-day implementation of the Program. 

Role of Regional Director, Oceans, Habitat and Species at Risk, Quebec Region

The Regional Director Oceans, Habitat and Species at Risk (RD OHSAR), Quebec Region was the senior DFO officer delegated responsibility pertaining to the AIHP.  Selected areas of responsibility were sub-delegated to the Regional Manager, Habitat Management, Quebec Region.

Role of Director General, Aboriginal Programs and Governance

The Director General (DG), Aboriginal Programs and Governance (APG) Directorate in the EFM Sector was accountable for program delivery pertaining to Aboriginal Contribution Programs that impacted areas where DFO manages the fisheries.  This responsibilty did not include the AIHP.

Part of the accountability of the DG, APG included the harmonization and integration of the frameworks, templates and processes used to manage and administer DFO Aboriginal Contribution Programs.  Therefore, although not accountable for AIHP delivery, the DG, APG provided guidance and advice to AIHP management pertaining to opportunities to utilize common frameworks, templates and processes required in the management and administration of the program.

2.2 Objectives and Scope

In accordance with the requirements of Treasury Board’s Policy on Transfer Payments, an evaluation of the AIHP was conducted prior to the renewal of the program’s Ts&Cs in 2009. The evaluation found deficiencies in implementing the program, such as the lack of dedicated staff and signed agreements, as well as ineffective use of program funding.  Because the program had begun to demonstrate some progress on implementation in 2008-09, the evaluation recommended that the AIHP be continued on the basis that an evaluation be conducted in 2010-11 and a decision be made at that time on whether the program should be continued or terminated.  

The current evaluation focused on the Core Issues in Assessing Value for Money in Evaluation as defined by the Treasury Board’s Policy on Evaluation (2009).  That is, the evaluation examined issues of relevance and performance, including effectiveness, economy and efficiency.

The current evaluation assessed the extent to which the AIHP had achieved immediate and intermediate outcomes stemming from the activities as stated in the Logic Model for the Integrated Aboriginal Contribution Management Framework as it applies to the AIHP.

In accordance with the Treasury Board’s Policy on Evaluation (2009), the current evaluation addressed value for money by including clear and valid conclusions about the relevance and performance, including effectiveness, efficiency and economy of the AIHP.  To address value for money, the evaluation assessed the core issues identified below.  In addition, the evaluation also identified lessons learned from the implementation and delivery of the AIHP. 

CORE EVALUATION ISSUES
Relevance
Issue #1:  Continuing Need for the Program	Assessment of the extent to which the Program continues to address a demonstrable need and is responsive to the needs of Canadians.
Issue #2:  Alignment with Government Priorities	Assessment of the linkages between Program objectives and (i) federal government priorities and (ii) departmental strategic outcomes.
Issue #3:  Alignment with Federal Roles and Responsibilities	Assessment of the roles and responsibilities for the federal government in delivering the Program.
Performance
Issue #4:  Achievement of Expected Outcomes	Assessment of progress toward expected outcomes (the evaluation will assess the outcomes as stated in the IACMF): Collaborative management capacity, infrastructure, governance, processes and relationships are in place with Aboriginal groups; and Aboriginal groups collaborate and participate in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources. Assessment of progress toward the expected result as stated in the Performance Measurement Framework of the MRSS: Aboriginal groups in Quebec, Ontario, and the Prairie provinces have the necessary capacity to contribute to the conservation and protection of fish and fish habitat and improvement of fish habitat   Assessment of the alignment between the expected results as stated in the IACMF and the expected results identified in the Performance Measurement Framework of the MRRS.
Issue #5:  Demonstration of Efficiency and Economy	Assessment of resource utilization in relation to the production of outputs.

The evaluation was conducted from July to November 2010 in the National Capital Region, with data collection undertaken in the Central and Arctic and Quebec Regions.  The period covered by the current evaluation included the 2009-10 fiscal as well as an examination of 2010-11 information, where available.

2.3 METHODOLOGY

The AIHP evaluation matrix included the use of multiple lines of evidence and complementary research methods as a means to ensure the reliability of information and data to be collected. The following research methods were used to gather qualitative and quantitative data for the evaluation:

2.3.1 Document and File Review

The review of relevant existing program documentation was a means of obtaining a comprehensive perspective on the results and activities of the AIHP. The document review included an examination of various program documents and administrative data. The following are types of documentation that were reviewed during the evaluation: DFO’s Performance Measurement Framework and Program Activity Architecture, Departmental Performance Reports, IACMF documentation, Reports on Plans and Priorities and other documents such as Speeches from the Throne.

In addition, a review and analysis of information contained in projects files, and of previous evaluation and audit reports was carried out.  A detailed review of the existing program data was undertaken to obtain information to address the evaluation questions and issues.

2.3.2 Key Informant Interviews

Interviews served as an important source of information for the evaluation by providing qualitative input from individuals who are closely involved in the design and delivery of the program or are key stakeholders of the program. The key informant interviews was one of the methodologies necessary to address most evaluation issues including those dealing with relevance, design and delivery, effectiveness, and efficiency.  

Interviews were undertaken at both NHQ and in the C&A and Quebec regions.  The interview groups included:

• DFO management and staff at NHQ and in the region(s) (14); and
• Aboriginal organizations (representatives from 8 of 13 organizations with agreements in 2009-10).

Interview guides were developed and tailored for each of the interview groups. The information obtained from the key informant interviews was used to address the evaluation issues.

2.3.3 Comparative Analysis of Similar Program

A comparison was made between the implementation of the AIHP with that of a similar program, the Aboriginal Aquatic Oceans and Resource Management Program to determine if any lessons learned or best practices were applicable to the AIHP.

2.3.4 Financial Analysis/Trends

An analysis was carried out regarding the utilization of financial resources approved for AIHP, particularly the trend in their usage.   The analysis included a comparison between the funding allocated to the program and the actual funds expended, as well as identifying trends in the usage of the funds allocated to the program since its initial approval in 2004.

2.3.5 Case Studies

Case studies assist in understanding complex issues and can support what may already be known through other research.  In the case of the AIHP, contribution arrangements with five Aboriginal organizations, as well as subsequent reporting on accomplishments, were examined for the years 2007-08 to 2009-10.  The purpose was to determine the activities that had been carried out by the organizations with the funding provided to them and to use the analysis of these activities to assist in assessing performance. 

2.3.6 DATA ANALYSIS PLAN

The data from each of the above evaluation methodologies was analysed to address each of the relevant evaluation issues/questions contained in the evaluation matrix. The data analysis strategy included the triangulation of multiple lines of evidence. This involved the extraction of the results from each line of inquiry that relate to each evaluation issue. The data obtained from each line of inquiry was analyzed for each evaluation question to develop a summary response to each evaluation question.

2.3.7 Limitations, Challenges & Mitigation Strategies

The evaluation methodology was designed to provide for multiple lines of evidence, as explained above, in support of the evaluation findings. The data and information was collected to respond to the main evaluation questions and issues. As in most evaluations, there were limitations and considerations that should be noted relating to the methodologies employed as well as general challenges. The following were some challenges encountered during the evaluation:

• there was a limited number of people within DFO who had direct involvement and knowledge of the AIHP;
• reporting by Aboriginal organizations was inconsistent;
• the anticipated data base that was to have been put in place under the IACMF was not yet available when the evaluation was carried out; and
• the evaluation relied heavily on qualitative information for the assessment of results.

Despite these limitations, the evaluation was designed to use multiple lines of evidence (e.g., interviews, document review and quantitative analysis), thus strengthening the reliability and validity of the evaluation results. 

3.0 FINDINGS AND RECOMMENDATIONS

3.1 RELEVANCE

When the AIHP was conceived, its objective was to enhance the ability of Aboriginal groups, working together as AIHP bodies, to participate in decision making related to habitat management regulatory and non-regulatory activities.  Through the establishment of AIHP bodies, DFO would assist Aboriginal groups in inland areas to develop the necessary capacity to carry out fish habitat management activities through a collaborative and integrated approach.  Greater coordination with Aboriginal groups in inland areas on habitat matters could allow for more effective delivery of DFO’s habitat management responsibilities.  The AIHP was targeted towards approximately 380 Aboriginal communities in Alberta, Saskatchewan, Manitoba, Ontario and Quebec, with a total Aboriginal population of approximately 650,000.

Consultation with Aboriginal groups can present unique challenges due to the lack of technical and financial capacity of Aboriginal groups to effectively engage in consultation activities.  The AIHP could help address these challenges in inland provinces by providing funding to Aboriginal groups to develop the capacity to effectively participate in decision making related to regulatory and non-regulatory decisions pertaining to fish habitat management.

Interviews with both DFO staff and Aboriginal organizations indicated that there was a need for the program to continue and that the need had increased over the past few years, particularly where there had been major commercial development or work on hydro dams.  Some of the reasons given for the need to continue the program were: 

• In the past, natural resources management was generally linked to forestry.  There was no capacity in Aboriginal communities to address fish habitat.  With the increasing impact of dam building, water levels, sewer discharge and leakages from highways, the communities need the capacity to deal with habitat issues. 

• During the first couple of the years of the AIHP, the Aboriginal organizations were developing their program and were gathering information.  They were building capacity in terms of technical skills and tools such as Geographic Information Systems (GIS) and starting to collect and manage data.  Now, they were starting to work with people in the Aboriginal communities and sharing knowledge.   

• The AIHP was the only program that had a focus on fish habitat and no other department or agency provided funding for this type of work. 

• Awareness of the program had increased.  2010-11 was the first year that the number of proposals received had exceeded the funds available.  The program has matured and more groups were submitting more complex proposals. 

In DFO’s Program Activity Architecture, the AIHP contributed to the Department’s Strategic Outcome of Healthy and Productive Aquatic Ecosystems, which was aligned with the Government of Canada’s outcome area of Clean and Healthy Environment.

Building stronger relationships and improving the quality of life of Aboriginal people in Canada was a key commitment of the federal government. DFO had an important role to play in meeting this commitment by helping to build capacity in Aboriginal communities. The AIHP could contribute to this role by contributing to capacity building to build knowledge and awareness, thus allowing First Nation communities to participate in decisions that affect fish habitat.

In the Progress Report on Aboriginal Initiatives from the Government of Canada - 2009-10, it was stated that governments and private industry were increasingly recognizing the tremendous economic potential of Aboriginal communities, particularly in the area of resource development. To realize this potential, the federal government was working closely with provinces and territories to develop effective approaches and processes to fulfill the legal duty to consult and, where appropriate, accommodate, with a focus on opportunities for inter–jurisdictional cooperation and collaboration. The AIHP could have contributed to this collaboration. 

Another aspect that indicated a role for federal intervention was the approach to consulting potentially affected Aboriginal groups in the context of an environmental assessment under the Canadian Environmental Assessment Act.  Where appropriate, the Crown consults with potentially impacted Aboriginal groups prior to making regulatory decisions, for example, the issuance of a Fisheries Act authorization. Potential adverse environmental effects considered during an assessment include effects on the current use of lands and resources for traditional purposes by Aboriginal persons. 

DFO has responsibilities under the Fisheries Act, the Species at Risk Act (SARA), the Oceans Act and the Canadian Environmental Assessment Act to ensure that Canada's oceans and inland waterways and resources are protected and managed for the benefit of present and future generations.

The Fisheries Act provides the legal framework for regulating impacts on fish and fish habitat associated with works, undertakings, operations and activities occurring in or around fresh and marine waters throughout Canada.

In general, in relation to the reviews of projects in and around fisheries waters, DFO’s primary focus is to ensure that the works and undertakings are conducted in such a way that ensures compliance with the following provisions of the Fisheries Act.

• Section 35 of the Fisheries Act prohibits all works or undertakings that result in the harmful alteration, disruption or destruction (HADD) of fish habitat without an authorization from the Minister of Fisheries and Oceans or under regulations made by the Governor-in-Council.

• Section 32 prohibits a person from killing fish by a means other than fishing unless the person is authorized to do so by the Minister of Fisheries and Oceans or under regulations made by the Governor-in-Council.

• There are other sections of the Fisheries Act that pertain to the conservation and protection of fish and fish habitat and these include section 20, 21 and 22 (fish passage and flow), and section 30 (fish guards), among others.
• Section 36 prohibits the deposit of deleterious substances into waters frequented by fish unless authorized by a regulation under the Fisheries Act or by another law of Parliament. Environment Canada, on behalf of the Minister of Fisheries and Oceans, administers section 36 of the Fisheries Act.  DFO relies upon advice provided by Environment Canada regarding issues pertaining to water quality.

The Habitat Management Program (HMP) conducts its activities in accordance with the Policy for the Management of Fish Habitat to maximize or increase the social and economic benefits derived by Canadians from productive fish habitats and the fisheries resources they support.

DFO had in place an Integrated Aboriginal Policy Framework which was to provide guidance to employees in helping to achieve success in building relations with Aboriginal groups. The Framework was to serve as a guide for the renewal of DFO’s Aboriginal policies and programs, to provide strategic policy direction for the development of operational guidelines and programs, and to guide DFO in discussions and collaboration with other federal agencies, provinces, territories, stakeholders and Aboriginal groups.

The Framework presented a vision of “supporting healthy and prosperous Aboriginal communities through:

• building and supporting strong, stable relationships;
• working in a way that upholds the honour of the Crown; and
• facilitating Aboriginal participation in fisheries and aquaculture and associated economic opportunities and in the management of aquatic resources.”

The Framework identified seven strategies to guide DFO’s progress toward the attainment of the vision. The objective of the AIHP could provide support to one of these strategies:

“Supporting increased Aboriginal participation in co-management of aquatic resources – by working with Aboriginal groups to increase their participation in the management and protection of aquatic resources, habitats and ocean spaces, including policy and program formulation, planning, resource management decision-making and program delivery”.

The need for a framework for DFO’s policies and programs as they relate to Aboriginal communities was recognized by the Department’s five-year Strategic Plan ‘Our Waters, Our Future’2005-2010. The Strategic Plan called for the development of a fully integrated departmental policy and program approach. It set out as a special over-arching objective, the need to strengthen and foster the relationship between the Department and Aboriginal peoples.

The Strategic Plan further identified, as one of its goals for maintaining and strengthening relationships between DFO and Aboriginal groups, to “Increase the involvement of Aboriginal groups in the decision-making processes in other areas of DFO’s responsibility including integrated oceans management, species at risk, habitat management, scientific research and aquaculture development”. The AIHP was directly linked to this goal.

3.2 PERFORMANCE

The Treasury Board’s Policy on Evaluation defines performance as the extent to which effectiveness, efficiency and economy are achieved by a program. The following sections report on the findings related to the effectiveness, efficiency and economy of the AIHP.

3.2.1 Effectiveness

Effectiveness is defined as the extent to which a program is achieving expected outcomes.  

At the time that the AIHP was implemented in 2004, a Results-Based Management and Accountability Framework (RMAF) that included a performance measurement strategy that identified expected outcomes as well as the indicators and data collection methodology to assess them had been prepared.  The strategy, however, was not fully implemented. 

In 2009, the former Aboriginal Policy and Governance Directorate of DFO’s then Fisheries and Aquaculture Management Sector (now Ecosystems and Fisheries Management) led the development of an Integrated Aboriginal Contribution Management Framework (IACMF).

Integration of DFO Aboriginal Contribution Programs was based on the identification of Contribution Program “Results Chains” that focused on the commonalities of outcome themes among the six DFO Aboriginal Contribution Programs, including the AIHP. 

Four key common Results Chains represented the integration of the individual program Results Chains from all six APG Aboriginal Contribution Programs.  These were:

• Aquatic Resource Management and Stewardship
• Food, Social and Ceremonial Fisheries Management
• Economic Opportunities
• Aquatic Resource Management Compliance and Accountability

The AIHP came under the Aquatic Resource Management and Stewardship results chain.  The expected outcomes attributed to the AIHP in the IACMF were as follows:

Immediate

• Collaborative management capacity, infrastructure, governance, processes and relationships in place with Aboriginal groups (Capacity Building element of AIHP).

Intermediate

• Aboriginal groups collaborate and participate in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources (Collaborative Management element of AIHP).  

In the Management Resources and Results Structure (MRRS) of DFO's Performance Measurement Framework, the expected result for the AIHP was stated as follows: 

“Aboriginal groups in Quebec, Ontario and the Prairie Provinces have the necessary capacity to contribute to the conservation and protection of fish habitat and improvement of fish habitat”.

The expected result identified in DFO’s MRRS aligned with the immediate Outcome set out in the IACMF. 

Measuring Effectiveness

In the report of the 2008-09 evaluation of the AIHP, it was recommended that: 

“The ADM, OHSAR, in consultation with the RDG, C&A Region, should provide a performance report to the Departmental Evaluation Committee for the 2009-10 fiscal year.”

In response to this recommendation, a draft Performance Report was prepared by AIHP staff and provided to the evaluation team during this evaluation.  The draft Performance Report was based on the expected outcomes stated in the IACMF.   The information, as provided in the draft Performance Report, was used in the assessment of the performance for the AIHP.  

Evaluation Question - Immediate Outcome

Key Performance Indicators:

• Number of AIHP Aboriginal Organizations designated.
• Number of Aboriginal Communities Represented

An Aboriginal Organization Designation Template was developed but it was not available in time to use for the 2009-10 agreements. The purpose of the template was to guide Aboriginal organizations in the development of a request for status as a Designated AIHP Aboriginal Organization with DFO.  Designation was an eligibility requirement in future years for AIHP funding.  It did not, however, guarantee AIHP funding.

For fiscal year 2009-10, the contribution agreements, already in place for the previous year, were amended as the program’s Terms and Conditions had not been renewed by April 1, 2009. 

The following table illustrates the number of contribution agreements that have been entered into since the program began.

Number of Contribution Agreements
2004-2005	2005-2006	2006-2007	2007-2008	2008-2009	2009-2010
1	1	2	13	13	13

The number of agreements entered into had increased significantly since the early years of the program.  While not illustrated in the above table, it was expected that 15 agreements would be in place for fiscal year 2010-11.  During interviews with key informants, it was mentioned that with the number of agreements increasing and awareness of the program becoming more widespread, the ability to meet the expectations of Aboriginal communities with the same amount of funding for the AIHP may be diminished.  A strategic approach would be required for allocating funding and consideration given to how many organizations the AIHP can realistically sustain, while contributing to building the capacity of Aboriginal communities.

The number of Aboriginal communities represented by agreements in 2009-10 was as follows:

Province	Contribution Agreements	Communities Represented
Québec	2	9
*Ontario	7	134
Manitoba	2	38
Saskatchewan	1	74
Alberta	1	44

*The number of agreements entered into for Ontario includes one with the Assembly of First Nations, which represents communities across Canada.  The number of communities represented in Ontario does not include those represented by the national organization.

Through interviews and a review of documentation, it was observed that there was a wide variation in the number of communities that individual Aboriginal organizations represent.   For example, one organization represented three communities while another represented as many 74 communities.  As the number of Aboriginal organizations and the communities they represent increased, so did the demand and expectations of what could be done under the AIHP.  

Key Performance Indicator:

• Number of Aboriginal Communities that have direct involvement in AIHP initiatives/projects.

The following table illustrates the number of communities that had direct involvement in AIHP initiatives/projects. 

Province	Communities Represented	Communities having direct Involvement
Québec	9	9
Ontario	134	92
Manitoba	38	16
Saskatchewan	74	12
Alberta	44	32

The draft Performance Report indicated the number of communities that had direct involvement in AIHP initiatives/projects.  There was no clear definition, however, of what “direct involvement in AIHP initiatives/projects” meant.  It appeared that direct involvement could have been interpreted as simply being contacted during the year by the Aboriginal organization. There were some examples provided where there was information shared or assistance provided to Aboriginal communities on habitat issues but conclusions on the effectiveness of this involvement could not be drawn from them.  

For some Aboriginal organizations, information on direct involvement was not available and therefore, their direct participation was counted as nil.

Key Performance Indicator:

• Number of Aboriginal people who have received training supported through the AIHP and or/have had direct involvement in AIHP initiatives.

There was information in the draft Performance Report in support of this indicator, however, the inconsistency in how it was reported did not lend itself to measuring performance.

• In the Performance Report, training was divided into:
  • training provided to AIHP staff of the Aboriginal organizations;
  • training provided to communities; and
  • Youth Initiatives.
• in some instances, the title of the course attended by Aboriginal people was provided and the number of participants was sometimes indicated; 
• in some instances, the name of the workshop/course or presentation was given followed by the name of the communities that participated but there was not always an indication of the number of individuals who attended; and
• there was no indication of the potential impact of the training. 

With the information obtained through interviews, documentation review, and the case study, it was evident that there had been many activities and initiatives undertaken.  However, the multitude of activities indicated that there was not a strategic approach to establishing what specific training was required and what level of capacity the Aboriginal organizations and communities required or were trying to achieve. 

The AIHP was making progress in achieving the immediate outcome of: “Collaborative management capacity, infrastructure, governance, processes and relationships in place with Aboriginal groups” in terms of having AIHP organizations in place to assist communities. The program, however, did not have a clear vision or strategy in terms of how these organizations would assist DFO in delivering habitat regulatory and non-regulatory activities.

Evaluation Question - Intermediate Outcome

Key Performance Indicators

• Nature and Scope of Aboriginal participation in collaborative habitat management activities, structures and processes/projects. 

In addressing this indicator, the Performance Report stated that:

• One Aboriginal organization had attended two meetings, one with a government agency, and the other with a non-government organization, to discuss their potential collaboration in training and a habitat restoration project. 
• One Aboriginal organization and one First Nation community had entered into partnerships with universities. 

The evaluation team expected to find Aboriginal organizations and communities collaborating with DFO Habitat and other groups involved in protecting and conserving habitat. There was no evidence of any collaboration in decision making processes for habitat or of significant involvement of the Aboriginal organizations with DFO Habitat staff.  Interviews did note that some efforts, particularly in Alberta, were made to involve DFO in AIHP activities but these were not widespread. 

Based on the evidence obtained through interviews and documentation reviews, the evaluation team was unable to conclude that the program was making substantive progress in achieving the intermediate outcome of Aboriginal groups collaborating and participating in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

During the evaluation, as a case study, the evaluation team examined reports (2007-08 to 2009-10) submitted by Aboriginal organizations that had received AIHP funding.  The purpose of doing this was to determine if there was any demonstration of progress being made against the expected outcomes and could contribute to the assessment of effectiveness for the evaluation.  Reports from five groups were examined, one from each province participating in the AIHP. 

In the 2008-09 evaluation report, it was stated that Aboriginal organizations had been reporting their activities to DFO, however, there was no systematic collection or analysis of data with which to assess the achievement of expected outcomes. This same issue existed at the time of this evaluation. 

An examination of reports showed that the reports varied from, detailed qualitative reports on the activities carried out for each year of AIHP funding, to the submission of a schedule (to support a request for payment) outlining activities completed during the year.  The representatives from the Aboriginal organizations were asked about the clarity of the reporting requirements for the AIHP.  There was a diversion of views on the reporting requirements, ranging from, there being clarity about requirements, to that of DFO having no vision for what was expected in reports from the organizations.  One group indicated that it had submitted reports each year to DFO but did not know what was done with them as it had received no feedback on them.  Most organizations felt that for the fiscal year 2010-11, the reporting requirements were much clearer but also more onerous. 

The review of the reports by the evaluation team supported many of the comments provided by representatives of the organizations:    

• the reports varied in terms of content and format;
• most reports represented a list of activities that had taken place during the year; and
• there was no evidence that the reports had been reviewed by DFO or any attempt made to analyse the information in order to assess the performance of the program. 

The reports examined demonstrated that many activities such as, meetings, training courses and awareness presentations had been carried out by the Aboriginal organizations receiving AIHP funding since 2007-08.  However, the listings of activities, such as meetings, courses attended, or presentations given or received, does not provide an indication of the capacity achieved or the level required by the organization or communities. Given the multitude of activities undertaken, there did not appear to be a focus or evidence of a strategy of how or when they would get to a level where they could participatent in habitat decision making processes. 

With the wide variation in the number of communities represented by the Aboriginal organizations, there was not any strategy to address how the communities would be impacted by capacity building activities.   The multitude of projects, training sessions, meetings and networking opportunities, made it difficult to identify any clear focus or strategic plan for how all of the various activities could lead to a collaborative approach for fish habitat management. 

The analysis of information also indicated that there were activities being carried out by Aboriginal organizations that did not seem to fall within the scope of the program:

• Comprehensive reviews of policies and legislation affecting fish habitat and First Nation treaty rights were being undertaken.  While some of the policies and legislation may impact fish habitat, such as the proposed new Fisheries Act, it would seem that better use on the funds would have been for Aboriginal organizations to increase capacity building or collaborative management initiatives. 

• Review of provincial fisheries management plans was carried out.   Work that was being carried out on provincial fisheries management plans may not fall directly within the intent of the AIHP (while realizing that habitat is an important component of fisheries management). 

• There were also indications that fish habitat restoration work had taken place.  Restoration work was not normally eligible under the AIHP unless it was part of a training exercise.   It was not clear if, or how the restoration work was used in training exercises. 

• There were also some activities noted that could have been more appropriately funded by other government programs, such as the Aboriginal Funds for Species at Risk or input into provincial fisheries management plans. 

• Some of the activities undertaken or training received did not appear to have a direct link to any DFO specific habitat projects or priorities, e.g., training related to defensive driving and English writing. 

While these activities were approved by DFO, it points to a lack of adequate assessment of proposals in terms of what would be acceptable under the program.  It was not until 2009-10 that assessment criteria was put in place to assist in managing the program.

As stated in the Relevance section of this report (section 3.1), the objectives and intention of the AIHP were linked to and aligned with overall DFO and federal government objectives.  However, the lack of integration and involvement of DFO’s Habitat program staff made it difficult to link the contribution of AIHP activities to habitat regulatory and non-regulatory decision making processes. 

Effectiveness Conclusion 

The Performance Report, prepared by program, measured the performance of the AIHP in terms of a numeric assessment of the success of the program: 

• there has been an increase in the number of contribution agreements entered into;
• there is an increasing number of communities represented by AIHP funded organizations; and
• there are listings of activities, such as training courses, presentations and meetings, undertaken by the organizations.

It was expected that there would be evidence of collaboration or participation in habitat management decision making processes. The report, however, did not provide any indication of impact or linkages to the achievement of AIHP objectives or DFO priorities.

The information reviewed during the evaluation indicated that many activities were being undertaken and that some capacity building, particularly within the Aboriginal organizations, was taking place.  However, it was not evident that there was a clear vision or strategic focus for the AIHP and what these activities undertaken would achieve.

Despite having organizations in place that could assist communities on habitat matters, there were no clear linkages to or integration with specific DFO’s habitat priorities or projects or the organization had participated in any habitat decision making processes.

Based on the information provided reviewed, there was evidence that the program had made progress in increasing collaborative management capacity, infrastructure, governance, processes and relationships with Aboriginal groups, however, the AIHP was not making substantive progress of participating in decisions related to the sustainable management and conservation of fisheries, aquatic and oceans resources.

This evaluation identified several outcomes resulting from the AIHP that were not initially expected.

• Relationships have been fostered between DFO and Aboriginal organizations and among Aboriginal communities. There was also more involvement by Aboriginal organizations and the provinces as well as with universities and fishermen/angler’s associations. 

• The lack of involvement by DFO Habitat staff particularly at the Area and District level was not expected.  The Aboriginal organizations had expected more opportunities to work with DFO staff on joint habitat projects but this did not happen.  The organizations also believed that a better knowledge of DFO roles, responsibilities and structures would have been helpful, particularly the delineation of responsibilities between the provinces for fisheries management and DFO for fish habitat.   

• There are mixed reactions concerning career opportunities in Habitat.  One organization mentioned that the habitat technicians they had hired expressed interest in pursuing university studies in biology.  On the other hand, another organization indicated that they were having difficulties in maintaining the interest of their habitat technicians. 

• Youth Initiatives - In most Aboriginal organizations, youth engagement had been more successful than anticipated either as a result of class room presentations or at youth camps. 

The following were some factors and challenges that impacted the performance of the AIHP: 

• Delay in implementation and overall lack of program support from DFO led to a lot of frustration on the part of the involved Aboriginal organizations.  Greater leadership from DFO for ensuring implementation according to expected timelines was needed.

• Efficiency: the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output. The level of input and output could be increases or decreases in quantity, quality, or both.

• Capacity to implement the AIHP - there was only one FTE allocated to the program delivery. 

•  The lack of DFO support for program implementation left Aboriginal organizations on their own to prepare proposals.  Some of the organizations had no capacity in developing proposals and were left on their own to determine what should go into them.

3.2.2 Efficiency and Economy

Efficiency and economy are defined in the Treasury Board’s Policy on Evaluation (2009) in the following manner:

• Efficience : Mesure dans laquelle les ressources sont utilisées de manière à produire un plus grand niveau d’extrants avec le même niveau d’intrants, ou le même niveau d’extrants avec un plus faible niveau d’intrants. Les niveaux d’intrants et d’extrants peuvent se traduire par des hausses ou des baisses de qualité, de quantité ou les deux.
• Economy: the minimum amount of resources needed to achieve the expected outcomes.

Contribution agreements entered into by DFO with Aboriginal organizations were considered to be the outputs for the program.  The agreements were produced using the least possible resources. 

The number signed agreements had increased since the program was implemented.  In addition, tools were in place to guide DFO staff and Aboriginal organizations, contributing to the efficiency of the AIHP.   

Because of problems with the initial implementation of the program, there were very few contribution agreements entered into in the early years of the program.  It was not until that 2007-08 that more than two agreements were signed in a fiscal year when thirteen agreements were signed. 

In 2008-09, the program entered into thirteen contribution agreements totalling $2, 021,900.  This was the first year that the program was able to fully utilize its approved funding and in fact exceeded its approved allocation of 1,875,000.  The excess was covered off by the recovery of funds that had been loaned to the AAROM program in previous years.  

In 2010-11, fifteen contribution agreements were entered into totalling $1, 846,950. This was the first year, the program received more proposals than the funding allocation for the AIHP could sustain.  The program had to enter into negotiations to reduce some proposals and reject others based on the new scoring grid/criteria that had been established.

When the program was approved in 2004, it included funding for one full-time equivalent (FTE) position.  Despite receiving funding for an FTE when the program was implemented, there was no full-time staff assigned to AIHP at NHQ.  When the delivery of the program was delegated to the Central and Arctic Region, the role of the AIHP Manager was an added job responsibility for a Habitat Manager in that Region. Prior to June 2008, there was a 14-month gap without a full time AIHP Manager in place.

A full-time position of AIHP Manager was staffed in C&A Region on a permanent basis and another individual assigned to support the delivery of the program.  In addition, the AIHP was part of the regular Habitat Program and oversight was carried out by the Regional Manger of Habitat and the Regional Director of Oceans, Habitat and Species at Risk (OHSAR) in the C&A Region.  The program was also supported by the Central and Arctic Region’s regional administration and finance groups. 

There was minimal funding provided to the Department for the implementation of the AIHP.  Since the program was delivered primarily by the Central and Arctic Region, program funding was provided to this Region, however, there were no salaries or operations and maintenance funds available for the Quebec Region or NHQ.  If DFO staff from Quebec Region or NHQ were required to travel on AIHP activities, it would have been funded by their own organization.   

While the Aboriginal organizations were complimentary in their comments regarding individuals filling the AIHP Manager role, it was evident that the lack of dedicated DFO staff to the program greatly impacted how the AIHP was delivered in its early days, as well as its reputation both within and external to DFO.

During the previous evaluation in 2008-09, it was noted that there were deficiencies in the administrative and management systems in place for AIHP. Since that time, the program had put tools in place to support the delivery of the AIHP.  The following tools were put in place to help improve the management of the AIHP: 

• AIHP Guide for Aboriginal Groups

This guide provided an overview of the steps to be taken by Aboriginal organizations to access support and funding through the Aboriginal Inland Habitat Program.  The guide would help Aboriginal organizations by providing a general background and description of the AIHP including objectives and Guiding Principles.  It also outlined DFO’s Integrated Aboriginal Contribution Management Framework. 

The guide also outlined the activities and costs that the AHIP would support, as well as describing the steps involved in the AIHP planning, application and program delivery processes.  It also clarified the criteria that would be used to assess AIHP applications.

• Aboriginal Inland Habitat Program - Guide to Implementation

The purpose of this AIHP Guide was to establish clear accountabilities and governance practices for AIHP program delivery and describe ongoing program delivery processes, decision-making, oversight mechanisms and reporting requirements.

In addition, there were two administrative staff working in the program and oversight was provided by the Regional Manager of Habitat and the Regional Director of OHSAR in the Central and Arctic Region. 

There was also increased involvement with the Aboriginal Program and Governance (APG) Directorate in the Ecosystems and Fisheries Management Sector, including reciprocal membership on APG and AIHP working groups. The Ecosystem Management Directorate (formerly Habitat Management Program) was part of the EFM Sector.  This could have led to the sharing of lessons learned and best practices. 

The AIHP was allocated $1, 875,000 in contribution funding each year.  In the evaluation conducted in 2008-09, it was reported that because of problems with the initial implementation of the AIHP, there were very few contribution agreements entered into in the early years of the program.  The funds that had been allocated to the program were not fully utilized for AIHP activities, resulting in funds lapsing or being reprofiled to support other DFO programs.  For example:

• In FY 2004-05, the Department made a request to Treasury Board to reprofile funds in the amount of $575,000 but it was rejected.  As a result, this amount lapsed. 
• In FY 2005-06, the Department was operating under Governor General Warrants and was unable to convert O&M funds to Grants and Contributions (G&C) to cover commitments that had already been approved under the Departmental Class G&C Program.  It was determined that AIHP had not spent its funding, so the program provided $1,080,000 to cover the Class G&C requirements. 
• In FY 2006-07, the AAROM program requested a loan of $1.1M from AIHP.  This was to be paid back over two years at $550K per year.  Also in 2006-07, AIHP provided $35K to fund two departmental Class G&C items for which commitments had been made. 
• In FY 2007-08, agreements were signed late in the fiscal year and the AIHP groups were unable to carry out the work that had been identified in the agreements.  Consequently, the program lapsed funds in the amount of $402,878.

The lapsing of funds as well as the transfer (reprofiling and loans) of funds to other departmental program was an indication that the Department was not in a position to effectively deliver the AIHP and spend funds in a judicious manner.

Since 2008-09, the AIHP had fully utilized the funds allocated to it and the demand for funding has been increasing.

3.3 PROGRAM DESIGN AND DELIVERY

Program Design

The AIHP was designed with two separate components: capacity building and collaborative management. There were no suggestions offered for alternatives program designs or delivery mechanism for a program such as the AIHP. 

Capacity Building

Capacity building was intended to facilitate the involvement of Aboriginal people in the collaborative management of the fisheries and fish habitat.  The AIHP contributed to the creation of this capacity by supporting Aboriginal organizations that have the governance, management and administrative infrastructure to coordinate the delivery of the AIHP contribution agreements.

The AIHP was intended to develop the capacity of Aboriginal communities and assist individuals to engage in collaborative management of fish habitat.  The AIHP was also intended to support training in areas related to fish habitat management, as well as, support Aboriginal communities undertake fish habitat initiatives/projects to build their ongoing capacity to contribute to the collaborative management of fish habitat.  

Collaborative Management

Through contribution agreements with Aboriginal organizations, the AIHP can directly support Aboriginal groups and communities with respect to collaboration in decision making processes that have potential impact on fish and fish habitat.  In addition, the AIHP could enable Aboriginal groups to participate in initiatives/projects that support the conservation and enhancement of fish habitat.  

Interviews during the evaluation with DFO staff and with representatives from Aboriginal organizations indicated that this was the proper design for the program in terms of the key elements of capacity building and collaborative management to support the needs of the Aboriginal peoples.

Program Delivery

Implementation of Program

Prior to the start of the program in 2004, it was decided that it would be necessary to determine if there were existing ‘aggregates’ of Aboriginal communities that could provide a foundation for AIHP capacity building.  After some research, the Habitat Management program approached six Aboriginal groups in the five inland provinces to see if they would be interested in participating in the AIHP.  Five Aboriginal groups responded by submitting statements of interest to participate in the program. 

Soon after the six original groups had been approached, a number of concerns were raised by other Aboriginal groups resulting in a re-thinking of the approach taken.  There was a general lack of awareness on the part of other Aboriginal groups of the existence of the AIHP and they felt excluded from the opportunity to participate in a program that could help them develop capacity to participate in habitat management decision making processes.  It was determined that broad consultations with Aboriginal groups were needed. 

In January 2005, the Assembly of First Nations (AFN) approached DFO to conduct a series of workshops intended to inform Aboriginal groups about the AIHP and seek their advice on its implementation. In March 2005, DFO entered into a contribution agreement with the AFN to facilitate the engagement of Aboriginal groups in dialogue around the management of fish habitat through capacity building and by encouraging inter-community dialogue and collaboration along watershed and broad ecosystem areas. 
The AFN hosted six workshops in the winter and spring of 2005.  A key recommendation arising from those consultations was a call for the coordinated delivery of the program within each of the five provinces.  Coordinated delivery meant that Aboriginal groups in each province would have the opportunity to jointly develop AIHP statements of interest to deliver the program in that province.

The Department endorsed the recommendations and, in addition, decided to make changes to the administration of the program by transferring day-to-day management of the program to the C&A Region (given that C&A accounts for four of the five AIHP provinces) and established a National Management Committee comprised of NHQ and regional staff.  The first AIHP agreements (excluding the AFN and Aboriginal Human Resources Development Council of Canada agreements to pursue the AFN recommendations) were put into place in 2007-08.

The failure to get the AIHP up and running early in the life of the program hampered its ability to succeed and, there were suggestions that the reputation of the program was severely damaged. 

Challenges with Design and Delivery

Delays in Implementing the Program

It had taken some time to recover from the initial difficulties in implementing the AIHP.  The program did not get off the ground until its third year of existence. 

Communications

Communication of the program was not carried out well both externally to Aboriginal organizations and internally with DFO Habitat Program staff. A communication strategy would have been helpful to provide awareness of the program.  Also, some Aboriginal organizations indicated a need for better communication of what role DFO plays in the management of fish habitat and its relationship with other federal departments and the provinces in this area. 

Capacity to Manage the Program

The capacity to manage the program was a common challenge raised by individuals within and external to DFO.  There was one FTE allocated to delivering the AIHP and that position was not filled on a full time basis in the early years of the program. 

In addition to the capacity to manage the program, it was also noted by some, in particular Aboriginal organizations, that there had been very little support or integration with DFO Habitat Management staff in DFO district offices.  Most did recognize that resource limitations in the Habitat Program were the reason for this.

Benefits of Program Design and Delivery

Involvement of Aboriginals

• After initial difficulties in implementing the AIHP, Aboriginal organizations and communities were asked for input/suggestions into how the program would be delivered.
• The AIHP was delivered at the ground level with Aboriginal organizations and communities.
• Individual communities can ask to become involved in the program through Aboriginal organizations. 

Flexibility of AIHP

• The flexibility of the AIHP terms and conditions which allowed for a variety of habitat related activities was seen as a benefit of the program design and delivery.

The use of multi-year contribution agreements gave a sense of certainty to the program and allowed Aboriginal organizations to plan beyond a one-year horizon.

3.4 LESSONS LEARNED

DFO staff and representatives from Aboriginal organizations were asked to provide views on any lessons learned from the implementation of AIHP.  A review of AAROM also considered lessons learned that may be applicable to the AIHP. 

Following are some issues identified during the evaluation from the views provided and documentation reviewed:

Consultation Process

• Consultation with Aboriginal organizations and communities should have taken place early in the planning phase for the new program.  This could have helped set out the objectives of the program and learn about the expectations of potential participants in the program.
• A consultation process could also have included engaging the provinces as they are responsible for management of the fisheries in the inland provinces.

Linkages to Other Aboriginal Program

• There could have been closer linkages with the other DFO Aboriginal programs.  The expertise of dealing with Aboriginal groups that existed in the Ecosystems and Fisheries Management Sector could have increased the probability of the success for the AIHP. 

Integration with the Habitat Management Program

• The AIHP was not integrated into DFO’s Habitat Management Program.  The AIHP operated in isolation or in a silo on its own. 

Capacity to Deliver Program

• The AHIP was provided with a small amount of contribution funding for a potentially large number of Aboriginal groups and communities.  The establishment of the program in itself raised expectations of the Aboriginal communities but the limited funds available impacted the ability to meet these expectations. 
• Adequate capacity to deliver the program should be incorporated into the program. Delivering a program with only one person (sometimes working part-time on the AIHP), raises the risk of the program not being able to succeed. 

Framework to Deliver Program

• A proper framework that sets out the governance structure, guidelines and tools such as reporting templates and an assessment grid should have been in place prior to implementing the program.

Strategic Direction for the Program

Strategic direction or a vision as to where the Department wanted to be with the AIHP would have been beneficial.  This would have helped the program set priorities and assist the Aboriginal organizations focus on what they wanted to achieve.

3.5 SHOULD THE PROGRAM CONTINUE OR BE TERMINATED?

The 2008-09 evaluation recommended that a follow-up evaluation be carried out in 2010-11 and that a decision be made at that time on whether the program should be continued or terminated.

When individuals, representing Aboriginal organizations and DFO, were asked if they were of the view that AIHP should be continued, overwhelming, they indicated that the program should continue.  Many of the reasons given for the AIHP to continue focused on the administrative and management of the program, including 

• internal governance had improved;
• the AIHP delivery mechanism was in place;
• evaluation criteria for proposals had been established;
• organizations were being asked to report on their achievements.  2009-10 was the first year that the Aboriginal organizations had to report on results; and
• the program had stabilized but there had not been enough time or money to achieve its expected outcomes. 

During interviews, representatives of Aboriginal organizations expressed the view that it was poor management on the part of DFO that caused the problems in the implementation and management of the program.  This view was supported by the lack of a consultation strategy and engaging the Assembly of First Nations to assist in rolling out the program; and the lack of a framework for the AIHP at the time of implementation. 
. 
The AIHP however, was not making substantive progress in achieving the intermediate outcome of Aboriginal groups collaborating and participating in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

While the information reviewed during the evaluation indicates that many activities were being undertaken and that some capacity building, particularly within Aboriginal organizations, had been achieved, it was not evident that there was a clear vision or strategic focus for the AIHP, particularly as it relates to the continuing level of capacity building required by the Aboriginal organizations or how they were going to address those needs in the future.

Despite having organizations in place that could assist communities on habitat issues, there were no clear linkages to or integration with specific DFO’s priorities or projects related to fish habitat.  In addition, there was no evidence that the Aboriginal organizations had participated in any decision making processes with DFO.  There was a clear vision or strategy on how these organizations would assist DFO in delivering habitat regulatory and non-regulatory activities.

Based on the above, the evaluation team was unable to conclude that the program had made substantive progress, after six years of funding, in achieving its expected results or that there was a reasonable expectation of doing so in the near future. 

3.6 RECOMMENDATIONS

It is recommended that:

1. The Terms and Conditions for the Aboriginal Inland Habitat Program not be renewed.
2. A close-out strategy for the program be prepared that would take into consideration the following:

• o the need for a communication strategy that would inform Aboriginal organizations and communities of the termination of the program;
• o the need for a human resource plan to take into consideration the impact on any affected DFO employees;
• o the mechanism for addressing multi-year contribution agreements that are in place; and
• o the possible means of dealing with the needs that the program was intended to address.

3.7 CONCLUSION

The evaluation found that there was evidence that the program had made some progress in building collaborative management capacity and establishing infrastructure, governance, processes and relationships with Aboriginal groups.  The program did not however make substantive progress in achieving the expected result of having Aboriginal groups collaborate and participate in activities, structures and processes aimed at the sustainable management and conservation of fisheries, aquatic and oceans resources.

Since the previous evaluation, AIHP staff had made a concerted effort to improve and measure the performance of the program.  There had been an increase in the number of contribution agreements entered into and there was an increasing number of communities represented by AIHP-funded organizations.  These improvements, in addition to listings/compilations of activities undertaken by Aboriginal organizations, did not demonstrate any impact on or linkages to the achievement of AIHP objectives or DFO priorities.

While many activities were being undertaken by the Aboriginal organizations and some capacity building for habitat management, particularly within the organizations themselves, had been achieved, it was not evident that there was a clear vision or strategic focus for the AIHP, particularly as it related to the continuing level of capacity building required by Aboriginal organizations or how they were going to address those needs in the future.

The evaluation found that there were many and a wide range of activities being carried out by Aboriginal organizations.  However, with the multitude of activities undertaken, it was difficult to determine if there was an underlying strategy guiding them or what they would achieve.  It was also not evident that there was a clear vision or strategy for the program, particularly as it relates to the continuing level of capacity building required by the Aboriginal organizations and how these needs will be addressed in the future.  In addition, it was not clear how these organizations would assist DFO in delivering habitat regulatory and non-regulatory activities, given the limited involvement to date with DFO’s Habitat program.

Based on the evidence examined during the evaluation, the evaluation team was unable to conclude that the program had made substantive progress in achieving its expected results or that there was a reasonable expectation of doing so in the near future. 

MANAGEMENT ACTION PLAN IN RESPONSE TO THE EVALUATION OF THE ABORIGINAL INLAND HABITAT PROGRAM

RECOMMENDATIONS
Recommendation 1: It is recommended that the Terms and Conditions for the Aboriginal Inland Habitat Program not be renewed when they terminate on March 31, 2012.
STRATEGY
Terms and Conditions for the Aboriginal Inland Habitat Program will not be renewed when they terminate on March 31, 2012.
MANAGEMENT ACTIONS	ACTIONS COMPLETED	ACTIONS OUTSTANDING	TARGET DATE	SUPPORTING EVIDENCE
Termination of the Aboriginal Inland Habitat Program Terms and Condition			March 31, 2012
RECOMMENDATIONS
Recommendation no 2 It is recommended that a close-out strategy for the program be prepared that would take into consideration the following:  the need for a communication strategy that would inform Aboriginal organizations and communities of the termination of the program; the need for a human resource plan to take into consideration the impact on any affected DFO employees; the mechanism for addressing multi-year contribution agreements that are in place; and the possible means of dealing with the needs that the program was intended to address.
STRATEGY
The close-out strategy for the program is part of the Strategic Review implementation.
MANAGEMENT ACTIONS	ACTIONS COMPLETED	ACTIONS OUTSTANDING	TARGET DATE	SUPPORTING EVIDENCE
Communications related to the close-out of AIHP	Aboriginal Organizations engaged in the AIHP were given written notice of the termination of the program by the Regional Director General of the Central and Arctic Region on November 10, 2011.	Completed	November 10, 2011	Letters to AIHP Aboriginal Organizations
	Questions regarding the conclusion of the program were collected from the AIHP Aboriginal Organizations.	Five questions from the AIHP Aboriginal Organizations have been submitted. A formal response is being developed by Ecosystems	January 2012	Written response from the Regional Director General, Central and Arctic Region.
Human Resource Plan for Affected DFO Employees	FTE associated with AIHP has received “affected” letter. AIHP Term Employee has been informed that their term will end on March 31, 2012.	Completed	December 12, 2011	Affected Letter
	Human Resource Plan for the affected employee is being developed by Human Resources in the Central & Arctic Region.		March 31, 2012	Affected Letter
Mechanism for addressing multi-year contribution agreements that are in place	Multi-year contribution agreements were made in principle only.  All signed contribution agreements terminate on March 31st, 2012. AIHP Aboriginal Organization were notified in writing at the beginning of the 2011-12 fiscal year, that funding beyond 2011-12 was subject to appropriation of funds in those subsequent years.  This was also stated in the letter to Aboriginal Organizations, giving notice of the termination	Completed	November 10, 2011	Letters to AIHP Aboriginal Organizations, and signed Contribution Agreements
Possible means of dealing with the needs that the program was intended to address	Incorporate the AIHP program goals into the regular business of the Habitat Management Program in the Quebec and Central and Arctic Regions		April 1st, 2012	Communications developed as part of the roll-out of the Strategic Review