Evaluation of the Atlantic Salmon Endowment Fund


Project number: 6B146
January 2012

TABLE OF CONTENTS

List of acronyms

APFC
Asia Pacific Foundation of Canada
ASF
Atlantic Salmon Foundation
ASCF
Atlantic Salmon Conservation Foundation
ASEF
Atlantic Salmon Endowment Fund
COSEWIC
Committee on the Status of Endangered Wildlife in Canada
DFO
Fisheries and Oceans Canada
FTE
Full-time Equivalents
HSP
Habitat Stewardship Program
NAAF
National Aboriginal Achievement Foundation
NASCO
North Atlantic Salmon Conservation Organization
O&M
Operations and Maintenance
PAA
Program Activity Architecture
PEI
Prince Edward Island
PSEF
Pacific Salmon Endowment Fund
SARA
Species at Risk Act
UN
United Nations

EXECUTIVE SUMMARY

INTRODUCTION

This report presents the results of the evaluation of the Atlantic Salmon Endowment Fund (ASEF).  The evaluation examined the extent to which ASEF demonstrated value for money by assessing the core issues of relevance and performance, including effectiveness, efficiency and economy in accordance with the Treasury Board’s Policy on Evaluation (2009).

The evaluation of ASEF covered the period from 2006/07 to 2010/11 and was conducted between April 2011 and November 2011.  The evaluation includes the department’s National Headquarters and four Regions: Newfoundland and Labrador, the Maritimes, Quebec and the Gulf.

PROGRAM PROFILE
Fisheries and Oceans Canada (DFO) announced the creation of ASEF in the 2005 federal budget as a one-time $30 million conditional grant to invest in the conservation and enhancement of wild Atlantic salmon and its habitat.  An Endowment Fund, such as this one, is a transfer of money donated to an institution, in many cases a charitable/non-profit organization, where the income earned by the property may be used for ongoing operations or other specified purposes.  In this case, the income earned from ASEF is used to fund projects that contribute to salmon restoration and conservation mainly in the freshwater environment, although marine environment projects are also considered for funding in some cases.

The Atlantic Salmon Conservation Foundation (hereafter referred to as “the Foundation”) is a non-profit organization that was chosen to invest the $30 million grant and disburse investment income in accordance with the Funding agreement between DFO and the Foundation. 

The purpose of ASEF is to achieve healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec.  The types of projects that are eligible for funding through ASEF include those that maintain, protect and enhance Atlantic salmon and their habitat, rebuild stocks and restore salmon populations, and are related to watershed planning

The objectives of fund are to:

  • effectively support community volunteer organizations in conserving, restoring and protecting wild Atlantic salmon and its habitat;
  • enhance cooperation and partnership among government, Aboriginal organizations, community volunteer groups and others in the interests of conserving, restoring and protecting wild Atlantic salmon and its habitat;
  • promote and improve conservation planning and management at the watershed level as the basis for ensuring effective use of and accountability for funds made available for wild Atlantic salmon conservation initiatives; and,
  • improve public awareness, education, and research respecting the conservation of wild Atlantic salmon and salmon habitat.

EVALUATION METHODOLOGY
The evaluation used a non-experimental design and a multiple-lines-of-enquiry approach.  This design was chosen because a control group could not be established for the purposes of making comparisons between pre and post-program, nor within or across areas or regions.  Qualitative and quantitative data for the evaluation were obtained from the following sources:

  • document and performance data review;
  • comparative analysis of three other federal endowment funds;
  • eleven interviews with key informants including Foundation Board and Committee members and funding recipients;
  • survey of 104 ASEF applicants (response rate of 42%); and,
  • review of 15 project files (chosen randomly) including application forms, project progress reports and project final reports.

Data analysis involved reviewing and summarizing input from the various research methods and then synthesizing the information to determine where the various lines of evidence reinforce one another (triangulation), where there is notable consensus or disagreement, and what other major notable points needed to be included in the evaluation report.

EVALUATION FINDINGS & RECOMMENDATIONS

Relevance
In general, the evaluation evidence indicates an overall need for ASEF.  The activities undertaken by the Foundation are contributing to the maintenance and restoration of healthy and diverse salmon populations and their habitat for the benefit and enjoyment of all Canadians.  These activities are consistent with the roles of the federal Government.

The evaluation evidence indicates that the original need for ASEF still exists.  The decline in wild Atlantic salmon that occurred between 1995 and 2004 has had an significant social and economic impact in Atlantic Canada and Quebec. By funding projects that aim to restore salmon habitat, ASEF is contributing to the improvement of regional economic and social issues.  Further, a comparative analysis of ASEF with similar programs has confirmed that the program does not overlap or duplicate objectives of similar programs, which confirms the unique need for ASEF.

ASEF is also consistent with the federal government’s role in conservation of wild Atlantic salmon in accordance with the Fisheries Act and the Wild Atlantic Salmon Conservation Policy.  The Fisheries Act is the primary legislative basis for fisheries management in Canada. It authorizes the Minister of Fisheries and Oceans to make decisions about the conservation of fisheries resources and habitat, to establish and enforce standards for conservation, and to determine access to and allocation of the resource.  The federal Government’s Wild Atlantic Salmon Conservation Policy represents Canada’s commitment and planned course of action for the conservation of wild Atlantic salmon.  Through the Policy, DFO plays a leadership role in international salmon conservation and is actively collaborating with external parties such as the Foundation to contribute more effectively to conservation through shared stewardship. 

The objectives of ASEF are still relevant to the federal government’s outcome of strong economic growth and are aligned with the departmental Strategic Outcome of Economically Prosperous Maritimes Sectors and Fisheries.  Further, the objectives of ASEF are contributing to broader government priorities such as supporting traditional industries, protection of water and land, and improving the lives of Canada’s Aboriginal people.

Effectiveness
Overall, the evaluation found that ASEF has achieved its immediate outcomes by increasing watershed planning, and engaging volunteer and stakeholder participation through both cash and in-kind contributions. ASEF has been successful in increasing watershed planning in all Atlantic Provinces.  Since the creation of the fund, the projects undertaken have led to the restoration and/or protection of 94,878 m2 of wild Atlantic salmon habitat.  The number of watershed planning projects is lower in Quebec, given that the provincial government has already undertaken this task as part of its delegated powers with respect to fisheries management.

ASEF has also led to engaging volunteer participation with respect to salmon conservation.  Since 2008, the Foundation has reported a total of 382 organizations involved in project proposals and has relied on a total of 949 volunteers to deliver its projects to date. 

The evaluation found that ASEF has led to increased stakeholder participation, both financially and in-kind.  The Foundation has been successful in engaging partners through cash and in-kind contributions.  Since 2008, the Foundation has raised over $2,218,287 dollars (including cash and in-kind) from external sources to fund conservation and habitat restoration of wild Atlantic salmon.

Furthermore, evaluation findings indicate that ASEF has achieved its intermediate outcomes by establishing partnerships with a significant number of interested organizations that undertake projects to rebuild the resource.  However, only limited funds (approximately $300,000) are available per year for projects. In addition, ASEF has achieved its intermediate outcome of establishing partnerships with new and recurring stakeholders. Through different types of outreach projects (education and awareness) and communication mechanisms (brochures, website, press releases, etc.) it appears that the Foundation is working towards increasing public awareness about salmon conservation. Some evidence suggests, however, that increasing awareness is a challenging task, since awareness may take many years, education and visibility.

Efficiency
ASEF has proven to be efficient through a clear governance structure, clear performance measurement and efforts to mitigate its most significant challenge, the impact of the 2008 recession.

The overall governance structure of ASEF is working efficiently through various committees and a Board of Directors supported by an Investment Committee, as required by the Funding Agreement.  

ASEF has also implemented several mechanisms to monitor the performance of projects.  The Funding Agreement between DFO and the Foundation was specific about reporting requirements, which the evaluation identified as a good practice. Close monitoring of projects by the Board of Directors and onsite visits have been accomplished by the Foundation to ensure that project results are linked to the outcomes established in the logic model.

Finally, the evaluation indicates that the Foundation has been mitigating the repercussions of the 2008 recession, which has been identified as its most significant challenge. A few months after receiving the $30 million endowment fund global markets were affected by a worldwide downturn and the value of ASEF plunged by $5,703,686.  To address the situation, the Foundation implemented a disbursement plan in order to rebuild the value of the fund.  In fact, the 2010 Annual Report states, “As of December 31, 2010, the market value of the fund was $30,086,289 as compared to the projected $29,212,423 demonstrating a modest excess over the year-end projection of the long-term financial strategy.”

Economy
Through the Foundation, ASEF has delivered projects economically by optimizing the use of resources, utilizing many volunteers, and leveraging significant funds from other sources. The Foundation has only two full time equivalents (FTEs) and relies on the work of many volunteers in order to carry out its mandate.  In addition, ASEF has leveraged funds from other sources in order to minimize the use of resources.  On average, the Foundation leverages $1.31 for every dollar granted1.  

There are no recommendations stemming from this evaluation.

1Note that administrative overhead costs are taken into account in leveraging calculations. In 2008 the leveraging ratio was $1.70 for every $1 granted, in 2009 $0.95 for $1 granted, and 2010 $1.28 for 1$ granted. However, leveraging ratio excluding administrative overhead would be $3.77 for every $1 granted (2008), $2.17 for $1 granted (2009), and $2.42 for 1$ granted (2010).

1.0 INTRODUCTION

1.1 CONTEXT OF THE EVALUATION

This evaluation focused on core issues as described in the Directive on the Evaluation Function of the Policy on Evaluation (2009).  That is, the evaluation examined issues of relevance and performance, including effectiveness, economy and efficiency.

The Atlantic Salmon Endowment Fund (ASEF) was created as a one-time conditional grant, which has been invested and managed by the Atlantic Salmon Conservation Foundation (ASCF), hereafter referred to as ‘the Foundation’.  The Foundation was formed by a group of volunteers who incorporated a non-profit organization in 2005 in order to prepare a proposal to Fisheries and Oceans Canada (DFO) to accept responsibility for ASEF.  The evaluation focused on the activities presented in the ASEF logic model developed by the Foundation and assessed the extent to which the fund is achieving its immediate and intermediate outcomes.  Since the fund has been in operation for less than five years, it is premature for the evaluation to focus on the fund’s ability to achieve its long-term outcomes.  Prior to this evaluation, no evaluation or audit of the fund was carried out.

1.2 SCOPE

The main objective of this evaluation was to determine the extent to which the program was managed effectively and efficiently and whether it achieved its stated objectives and results. The evaluation covered the period 2006/07 to 2010/11 and was undertaken between April 2011 and November 2011 by DFO’s Evaluation Directorate. The evaluation includes DFO’s National Headquarters and four Regions: Newfoundland and Labrador, Maritimes, Quebec and Gulf.

2.0 PROGRAM PROFILE

2.1 BACKGROUND & OBJECTIVES

Between 1971 and 1985, the estimated abundance of Canadian Atlantic salmon fluctuated between 0.8 and 1.7 million fish annually.  However, between 1995 and 2004, estimated abundance declined to about 0.4 to 0.7 million fish annually.2  When this decline was observed, DFO implemented a wide range of conservation measures, such as closing commercial fisheries and restricting recreational fisheries of wild Atlantic salmon stocks.  In addition, DFO announced the creation of ASEF in the 2005 federal Budget as a one-time $30 million conditional grant to invest in the conservation and enhancement of wild Atlantic salmon and its habitat.

An Endowment Fund is a transfer of money donated to an institution, in many cases a charitable/non-profit organization, where the income earned by the property may be used for ongoing operations or other specified purposes, while “…preserving the principal amount of the Conditional Grant over the long term against inflationary pressures and generating long term conservative growth for such purposes.”3  The ASEF was intended to provide a mechanism to engage stakeholders and expected to generate donations from external sources.  An endowment fund, such as this one, is anticipated to reduce long-term dependency on government support. 

Parties interested in administering ASEF were invited to submit a proposal to DFO. The Foundation was chosen to invest the $30 million grant and disburse investment income in accordance with the Funding Agreement between DFO and the Foundation.  As mentioned earlier, the Foundation is a non-profit, charitable organization with the goal of improving and strengthening the conservation of wild Atlantic salmon and its habitat in Atlantic Canada and Quebec4.   

The purpose of ASEF is to achieve healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec.  The types of projects that are eligible for funding through ASEF include those that maintain, protect and enhance Atlantic salmon and their habitat, rebuild stocks and restore salmon populations, and are related to watershed planning. The income earned from ASEF is used to fund projects that contribute to salmon restoration and conservation mainly in the freshwater environment, although marine environment projects are also considered for funding in some cases.

The objective of the fund is to:

  • effectively support community volunteer organizations in conserving, restoring and protecting wild Atlantic salmon and its habitat;
  • enhance cooperation and partnership among government, Aboriginal organizations, community volunteer groups and others in the interests of conserving, restoring and protecting wild Atlantic salmon and its habitat;
  • promote and improve conservation planning and management at the watershed level as the basis for ensuring effective use of and accountability for funds made available for wild Atlantic salmon conservation initiatives; and
  • improve public awareness, education, and research respecting the conservation of wild Atlantic salmon and salmon habitat.
2Report of the Working Group on North Atlantic Salmon (2008)
3Funding Agreement between DFO and the ASCF
4http://salmonconservation.ca/en

2.2 PROGRAM ACTIVITIES

In order to achieve the objectives of ASEF, the following activities are undertaken by the Foundation:

  • funding priority projects that contribute to the development of salmon and salmon habitat conservation plans for a watershed or sub-watershed (watershed planning);
  • encouraging cooperation between governments, Aboriginal groups and volunteer groups;
  • encouraging engagement and participation of stakeholders;
  • increasing awareness of habitat requirements through a communication strategy;
  • providing balanced allocation of funding that reflects geographical areas and priorities; and
  • managing the ASEF, and seeking other funding sources.

2.3 PROGRAM EXPECTED RESULTS/PERFORMANCE MEASUREMENT

ASEF’s activities are aimed at contributing to a longer term outcome of healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec as well as improved community capacity and involvement in salmon conservation.  A Performance Measurement Strategy was developed by the Foundation to describe the accountability and performance measurement reporting of ASEF.  Performance measures are in place and were used for this evaluation.

2.4 PROGRAM LOGIC MODEL

The logic model for ASEF (presented in Figure 1) was used for the evaluation.  This logic model was developed by the Foundation and has been in place since the inception of the program.

Key short-term outcomes of the ASEF are: an increase in watershed planning, measured in km of improved habitat; involvement of an increasing number of governments, Aboriginal and volunteer participants in more communities, measured in their numbers; and stakeholder participation, both financially and in-kind, measured in their dollar value.

The intermediate outcomes are: an increase in public awareness about salmon conservation, measured in the number of promotional material distributed; establishment of partnerships with stakeholder organizations, measured in terms of their numbers created and maintained; and organizations taking up responsibility for rebuilding the resource, measured in their numbers.

The final outcomes of the program are:  achievement of healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec and improved community capacity and involvement in salmon conservation.

                     Figure 1:  ASEF Logic Model
ASEF Logic Model

Source: The Atlantic Salmon Conservation Foundation, 2011 Strategic Business Plan

2.5 PROGRAM PARTNERS/CLIENTS/STAKEHOLDERS

The Foundation is responsible for investing and managing the $30 million ASEF grant.  The Foundation works towards achieving healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec by creating partnerships among volunteer conservation groups, Aboriginal organizations, governments and others. 

The stakeholders and beneficiaries of the ASEF grant include:

  • individuals, organizations and governments engaged in the conservation, restoration and protection of Atlantic salmon and its habitat;
  • Aboriginal peoples with cultural connections to the Atlantic salmon;
  • people who are employed in endeavours that depend on wild Atlantic salmon;
  • people who participate in recreation related to the Atlantic salmon;
  • people who derive sustenance from the Atlantic salmon as a source of food; and
  • the people of Canada to whom this natural resource belongs.

2.6 GOVERNANCE AND GRANTING PROCESS

ASEF is managed at arm’s-length from DFO.  There are no Full-time Equivalents (FTEs), Operations and Maintenance (O&M) or capital commitments for DFO in relation to the fund.  The Foundation has only two FTEs; the Executive Director and a Marine Biologist.  The remainder of the activities of the organization rely solely on volunteer support.  The Foundation has implemented a voluntary committee structure in order to promote inclusiveness and partnership, while assuring expert advice to ensure the use of ASEF for conservation purposes.  A description of the roles and responsibilities of the foundation’s Board and committees is provided below.

Board of Directors
Nine volunteer Directors are responsible and accountable for investing ASEF in accordance with the Funding Agreement with DFO.

Provincial Advisory Committees
Five Provincial Advisory Committees, responsible for identifying provincial priorities, reviewing funding proposals and recommending approved projects to the Board of Directors. 

Central Advisory Committee
The Central Advisory Committee is formed of technical experts who assist the Board of Directors in defining and adopting relevant conservation objectives, designing effective conservation and management tools and processes and monitoring Foundation progress and performance. 

Board Committees
Four Board Committees also report to the Board of Directors on specific issues, such as investment, audit and finance, policy and program and development. 

Figure 2 depicts the governance structure of the Foundation. 

Figure 2:  The Foundation Governance Structure

The Foundation Governance Structure

Source:  Atlantic Salmon Conservation Foundation, 2010 Annual Report

Through its governance structure, the Foundation undertakes and oversees multiple activities that are carefully selected in order to report on performance and provide feedback to DFO and to the general public on the extent to which anticipated outcomes, such as watershed planning, habitat conservation and restoration, partnership engagement and public education can be measured and quantified.  The Foundation holds an annual round of calls for funding proposals in the fall.  The Provincial Advisory Committees are then responsible for reviewing funding proposals and making recommendations on which projects should be approved for funding.  All applicants are required to submit a detailed funding proposal which clearly demonstrates how proposed projects will link up with regional priorities which meet ASEF’s expected outcomes. 

Applications for funding are ranked as follows:

Proposal Criteria Overall value
Potential for the project to improve conservation and restoration of wild Atlantic Salmon and its habitat

30 %

Technical merit and project feasibility 30 %
Partnership/cooperation among community groups, Aboriginal groups, governments and other organizations 25 %
Cost-effectiveness and budget detail and consideration 15 %

By ranking proposals, the Foundation is ensuring that approved projects are innovative and have a high probability of success with measurable results for on-the-ground conservation of wild Atlantic salmon and its habitat.  Recommended proposals are announced in the spring to allow work to commence before the opening of the conservation field season5.

5 Most conservation projects are carried out in late spring, summer and early fall.

2.7 PROGRAM BUDGET

As ASEF was a one-time $30 million grant, the table below illustrates the fund balance between 2007 and 2010.

Table 1: Endowment Fund Year End Balances

2007

2008

2009

2010

Revenue

(617 817)

(4 014 530)

4 323 606

2 788 861

Expenses Other6

375 687

429 381

426 681

452 629

Dépenses de subvention

-

266 271

211 250

347 800

Total
-excess of revenue over expenses

(993 504)

(4 710 182)

3 685 675

1 988 432

Fund balance (beginning of year)

30 000 000

(993 504)

(5 703 686)

(2 018 011)

Fund balance (end of year)*

(993 504)
29 006 496

(5 703 686)
24 296 314

(2 018 011)
27 981 989

(29 579)
29 970 421

General Fund and liabilities

88 024

247 281

179 534

252 758

Total

29 094 520

24 543 595

28 161 523

30 223 179

Source: Atlantic Salmon Conservation Foundation, 2011 Strategic Business Plan

6Other expenses include the administrative costs such as salaries and benefits, meetings, travel, professional fees, general office administrative overhead, public relations and communications as well as additional expenses such as for the investment management.

3.0 METHODOLOGY

3.1 EVALUATION APPROACH & DESIGN

A non-experimental design was used for this evaluation, in which measurements were taken after the program was implemented with no control group.  This design was chosen because a control group could not be established for the purposes of making comparisons between pre and post-program nor within or across areas or regions.

The analytical methods used for this evaluation were tailored to the nature and availability of the data to be gathered, which were in turn linked to the evaluation questions (Annex A). The rigour of this design was increased through the use of multiple lines of evidence and triangulation across those lines of evidence to reduce the occurrence of bias and to increase validity. In addition, the evaluation team also analyzed the evaluation data obtained from each line of inquiry for each evaluation question and developed a summary response to each evaluation question, taking into account the strengths and limitations of each line of inquiry. This enabled evaluators to make linkages, and logically argue that results can be attributed to the program.

The evaluation was conducted by an evaluation team led by a senior evaluation manager within the Evaluation Directorate at DFO. The team collaborated with Program personnel to prepare a list of documents to review, identify key informants and stakeholders and to review and provide feedback on interview guides and various reports.

3.2  DATA SOURCES

The research methods, described in more detail below, were used to gather qualitative and quantitative data for the evaluation: document review; comparative analysis; key informant interviews; file review; and a survey of the Foundation applicants. 

Although ASEF was announced in the 2005 federal Budget, funds were only transferred to the Foundation in 2007 and the first round of project funding was in 2008.  Therefore, only data from 2008, 2009 and 2010 were available for the evaluation.

3.2.1 DOCUMENT & PERFORMANCE DATA REVIEW

A review of documentation was conducted to assess all evaluation issues. This line of evidence was chosen because it can provide historical insight into the development and delivery of a program, verify data from other methodologies and provide primary data on issues such as relevance.  The document review consisted of the analysis of documents provided by the program and included both public and internal program documents (for example annual reports, previous audits, Speeches from the Throne, etc.).

Through its performance measurement strategy, the Foundation has collected a wide range of performance data to document progress towards the achievement of immediate and intermediate outcomes.  This information was also analysed for the evaluation.

3.2.2 COMPARATIVE ANALYSIS

A comparative analysis focused on the management of three other federal endowment funds: the Pacific Salmon Endowment Fund (PSEF), the Asia Pacific Foundation of Canada (APFC) and the National Aboriginal Achievement Foundation (NAAF). The most recent evaluations7 available for APFC and NAAF were used for the analysis. For example, it allowed for the comparison of objectives, governance structures and management of the fund of the different endowments to determine best practices and lessons learned.

A second analysis compared ASEF to perceived similar programs or organizations such as the Atlantic Salmon Federation (ASF) and the Habitat Stewardship Program (HSP) to determine any differences or similarities, including overlap or duplication in program delivery.

3.2.3 KEY INFORMANT INTERVIEWS

Key informant interviews were conducted to gather in-depth information for evaluation purposes, including views, explanations, examples, as well as contextual and factual information that addressed evaluation questions.  Planning interviews were conducted with Resource Management Staff at DFO8.  The evaluation team also conducted 11 interviews with key stakeholders, including the Foundation Directors, the Foundation Provincial Advisory Committee members, the Foundation Board Committee members (representatives from all four committees described in section 2.6), as well as funding recipients.

3.2.4 SURVEY

An online survey was administered to all funding applicants from the past three years to gather their view with respect to a number of focused evaluation questions related to the relevance and performance of the program. The survey was launched in September 2011 and remained open for 3 weeks. A total of 44 of 104 applicants responded to the survey, for a response rate of 42%.

3.2.5 FILE REVIEW

A review of 15 project files, including application forms, project progress reports and project final reports was undertaken as part of the evaluation in order to analyze the mechanism in place during the application process and the monitoring of projects.  One project was randomly chosen for each province (New Brunswick, Nova Scotia, Prince Edward Island, Newfoundland and Labrador and Quebec) for 2008, 2009 and 2010.  Final project descriptions posted online were studied in order to assess project alignment with the overall objectives of ASEF, DFO and the Government of Canada. 

3.3 METHODOLOGICAL LIMITATIONS, CHALLENGES & MITIGATION STRATEGIES

Limitations and considerations were noted relating to the methodologies employed, as well as mitigation strategies include the following:

Non Experimental Design. When using this model, it is difficult to clearly measure the net effects of ASEF. Since there are no measurements before the program began, nor a comparison group against which to assess other plausible causes for the outcome, it is difficult to attribute impacts to the program. In other words, it is difficult to draw a clear conclusion about the incremental or net effects of the program. Although this model lacks scientific rigour, the rigour of this design was increased by describing activities, outputs and outcomes through a logic model, enabling evaluators to make causal linkages, and logically argue that results can be attributed to the program (and increase the internal validity).

Data from interviews.  The small number of program personnel involved in the delivery of the program provided a limited pool for drawing key informants. In order to enhance the range of potential perspectives, key informants chosen for the evaluation were those with the most knowledge of ASEF and included the Foundation Directors, members of the Provincial Advisory Committees and members from all four Board Committees.  In addition, key informant responses were corroborated, where possible, with other lines of evidence to validate statements they presented as evidence.

7 http://www.international.gc.ca/about-a_propos/oig-big/2009/evaluation/apfc_fapc09.aspx?lang=eng&view=d et http://www.naaf.ca/sites/default/files/docs/Final_Evaluation_of_Endowment.pdf.

8 Because ASEF is managed at arm's-length from DFO, Resource Management Staff were unable to speak to the outcomes of the program. Therefore DFO staff was only consulted for the planning of the evaluation in order to provide key documents, list of key informants, etc.

4.0 MAJOR FINDINGS

4.1 RELEVANCE

Is there a continued need for ASEF?
Is there a role for the federal government in ASEF?
Are the objectives of ASEF aligned with federal Government priorities?

Key Finding:  The evaluation evidence indicates an overall need for ASEF.  The activities undertaken by the Foundation are contributing to the maintenance and restoration of healthy and diverse salmon populations and their habitat for the benefit and enjoyment of all Canadians.  These activities are consistent with the roles of the federal Government and are aligned with the Fisheries Act and Canada’s Wild Atlantic Salmon Conservation Policy. There is also no duplication with similar programs.

Continued need for ASEF
The document review has provided evidence that the original need for ASEF still exists.  The decline in wild Atlantic salmon has had an important social and economic impact in Atlantic Canada and Quebec. The recreational fishery for wild Atlantic salmon, which contributes over $56 million9 to the regional economies, has continued to require management restrictions, which limit its potential benefits to communities and to the economy of the region.  In addition, Aboriginal groups, which have long depended on Atlantic salmon for food, social and ceremonial fisheries, have had greatly reduced harvests.

Severe declines in salmon stocks have also been reported in the 32 rivers of the inner Bay of Fundy where Atlantic salmon have been assessed as “endangered” by the Committee on the Status of Endangered Wildlife in Canada (COSEWIC) and listed under Canada’s Species at Risk Act (SARA)10.  Salmon populations in numerous rivers in the Southern Uplands of the Atlantic coast of Nova Scotia are also at risk of extirpation or have already been extirpated.  In 2010, the Nova Scotia Salmon Association received $28,000 from ASEF to conduct research on survival patterns of 140 smolts from the Nova Scotia Southern Upland Rivers.  It is hoped that the data generated from this study will contribute to a better understanding of Atlantic salmon, which have been assessed by COSEWIC as “endangered” in this area.

Further, a comparative analysis of ASEF with similar programs such as the ASF and the HSP for Species at Risk has confirmed that the program does not overlap or duplicate objectives of similar programs, which confirms the need for ASEF.  Although there are similarities between programs, they are complementary in nature and not duplicative.  The ASF concentrates its efforts on the protection and scientific research of wild Atlantic salmon in the marine environment, whereas ASEF is intended to support wild Atlantic salmon mainly in the freshwater environment.  The goal of HSP is to protect important habitat for species at risk in general and does not concentrate its effort on wild Atlantic salmon stocks. The programs are seen as complementary; addressing different issues on the spectrum of conservation, protection and restoration of habitat.

As well, the majority of key informants (n=7) confirmed this by stating that while other programs had similar mandates they were not specific to salmon. This was also supported by 48% (n=21) of survey respondents, who stated that there was no duplication or overlap. On the other hand, a small percentage of survey respondents (9%, n=4) stated there was overlap or duplication of ASEF with other DFO, federal, provincial, or non-government activities, while 43% (n=19) responded “don’t know”. 

Overall, data from both interviews and the survey indicate that there is a continued need for DFO’s contribution to ASEF.  Eighty percent (n=35) of survey respondents and most key informants (n=10) indicated that the need for funding has continued or even increased over the past five years.  In addition, 65% (n=20) of survey respondents mentioned that they would not have been able to complete their project without funding from the ASEF; demonstrating the continued financial needs of recipients. Many key informants (n=5) expressed that additional financial resources are needed in order to continue funding projects that support restoration, habitat protection, conservation and enhancement of wild Atlantic salmon stocks.

Furthermore, 66% (n=29) of survey respondents indicated that one or more of their requests for funding from the Foundation was rejected within the last three years.  Of the rejected projects, 66% were unable to go forward due to lack of funding.

Federal government role in ASEF
The document review confirmed that activities undertaken by ASEF, through the Foundation, for the conservation of wild Atlantic salmon are consistent with DFO’s roles and responsibilities under the Fisheries Act and the Wild Atlantic Salmon Conservation Policy as well as the Department’s international obligations. 

The Fisheries Act is the primary legislative basis for fisheries management in Canada. It authorizes the Minister of Fisheries and Oceans to make decisions about the conservation of fisheries resources and habitat, to establish and enforce standards for conservation, and to determine access to and allocation of the resource. The federal Government’s Wild Atlantic Salmon Conservation Policy was finalized in 2009.  The Policy represents Canada’s commitment and planned course of action for the conservation of wild Atlantic salmon.  Through the Policy, DFO plays a leadership role in international salmon conservation and is actively collaborating with external parties such as the Foundation to contribute more effectively to conservation through shared stewardship.  Therefore, the Foundation is consistent with DFO’s responsibility towards the Wild Atlantic Salmon Conservation Policy.
 
In Quebec, the provincial government has received delegated powers with respect to fisheries administration, which apply to the management and control of fishing in freshwaters, as well as anadromous and catadromous species of fish in waters of the Province and tidal waters11.  This delegation is provided for in the 1990 Quebec Fishery Regulations.  Although DFO remains responsible for the application of the Fisheries Act, the province maintains responsibility for dealing with the conservation and protection of fish and fish habitat in Quebec waters.    To avoid duplication, the Foundation has adopted a funding allocation model based on the respective conservation needs of each province.   As such, DFO’s contribution to ASEF is helping achieve healthy and sustainable Atlantic salmon stocks in Atlantic Canada and Quebec which is in line with the Fisheries Act and Quebec Fishery Regulations.

Through the ASEF grant to the Foundation, DFO’s support of ASEF is also consistent with Canada’s international obligations such as those under the United Nations (UN) Convention on Biological Diversity as well as the North Atlantic Salmon Conservation Organization (NASCO). The UN Convention on Biological Diversity “…advocates the conservation of genes, species and ecosystems…”12 and the …objective of NASCO “…is to contribute, through consultation and cooperation, to the conservation, restoration, enhancement and rational management of salmon stocks subject to the Convention taking into account the best scientific evidence available”13. As such, the results of projects funded by ASEF contribute to many of the goals of the international agreements in which Canada is a party.

In addition, most key informants (n=10) and 93% (n=41) of survey respondents agreed that there is a role for federal government intervention in achieving healthy and sustainable wild Atlantic salmon stocks in Atlantic Canada and Quebec. 

Alignment of ASEF objectives with federal government priorities
The activities of ASEF are designed to help achieve the goals set out in the Wild Atlantic Salmon Conservation Policy by promoting enhanced community partnerships in the conservation of salmon and its habitat in Atlantic Canada and Quebec.  There is also an indication of alignment between ASEF activities and Government of Canada priorities as demonstrated through Speeches from the Throne 2007, 2008 and 2010 which include support of traditional industries such as fisheries, protection of water and land and improving the lives of Canada’s Aboriginal people.  In addition, most key informants (n=9) and 84% (n=37) of survey respondents agreed that the objectives of ASEF are aligned with federal Government and DFO priorities. 

The responsibility for ASEF falls under Integrated Fisheries Resource Management in DFO’s 2011-12 Program Activity Architecture (PAA).  The program is therefore aligned with the Strategic Outcome of Economically Prosperous Maritime Sectors and Fisheries which in turn contributes to the federal government’s priority of Strong Economic Growth.  The objectives of the ASEF are also aligned with DFO’s Strategic Plan (2005-2010) and DFO’s Report on Plans and Priorities (2006-07 to 2010-11) by supporting the department’s role in achieving healthy and productive waters and aquatic ecosystems for the benefit of present and future generations.  In addition, ASEF is contributing to DFO’s priority of deepening engagement and key partnerships as identified in the Department’s 2011-12 Report on Plans and Priorities.  Through ASEF, the Foundation has been working with clients and stakeholders and First Nations to extend the work on the protection and conservation of wild Atlantic salmon stocks.

In addition, the work undertaken by projects funded through ASEF is consistent with the Government of Canada’s legal obligation to protect wildlife species under the Species at Risk Act (SARA).  As previously discussed, ASEF is funding research on survival patterns of smolts in regions where salmon has been assessed by COSEWIC as an endangered species.

Through the Foundation, ASEF also funds multiple projects that are carefully selected in order to report on performance and provide feedback to DFO and to the general public on the extent to which anticipated outcomes, such as watershed planning, habitat conservation and restoration, partnership engagement and public education can be measured and quantified.  Furthermore, a review of project files, including progress reports and final reports, determined that projects funded by ASEF are aligned with the Foundation’s main objectives, which in turn are aligned with DFO and federal government priorities.

Conclusions
In general, the evaluation has concluded that ASEF is relevant, is addressing a continued need and is aligned with federal government priorities that support DFO’s legislated roles and responsibilities.

The evaluation found that there is a continuing need for ASEF as there is a federal government role in conservation of wild Atlantic salmon under the Fisheries Act and the Wild Atlantic Salmon Conservation Policy.  In addition, projects funded through ASEF also contribute to the international agreements to which Canada is a party.

Finally, the evaluation also concludes that the objectives of ASEF contribute to the federal government’s outcome of strong economic growth and are aligned with the departmental Strategic Outcome of Economically Prosperous Maritimes Sectors and Fisheries.  Further, the objectives of ASEF are contributing to broader government priorities such as supporting traditional industries, protection of water and land, and improving the lives of Canada’s Aboriginal people.


9 Wild Atlantic Salmon Conservation Policy.

11 Wild Atlantic Salmon Conservation Policy.

12 Wild Atlantic Salmon Conservation Policy.

13 http://www.nasco.int/

4.2 EFFECTIVENESS

4.2.1 To what extent has ASEF achieved its immediate outcomes?

Key Finding: Through the Foundation, ASEF has achieved its immediate outcomes by increasing watershed planning, and engaging volunteer and stakeholder participation through both cash and in-kind contributions.

Increased watershed planning
The evaluation evidence from documents, interviews and the survey indicate that ASEF has resulted in increased watershed planning in all Atlantic Provinces. A total of 24 watershed plans were developed via ASEF funding by community groups between 2008 and 2010 (12 in 2008, 7 in 2009 and 7 in 2010). As shown in Figure 3, funded projects have led to restored or protected habitat, which represents an increase of 57,064 m2 between 2008 and 2010. Also, most key informants (n=9) and 64% (n=28) of survey respondents support these findings by stating that program activities have led to increased watershed planning.



Figure 3: Number of square meters (m2) of improved habitat
Number of square meters (m2) of improved habitat
Source: ASCF, 2010 Annual Report

Watershed planning is being funded by ASEF in all the Atlantic Provinces and only limited funding has been allocated for watershed planning in Quebec.  As explained earlier, Canada has delegated powers to the Quebec government over fishery management, and the province has already developed extensive watershed plans.  The Foundation has addressed this issue by adopting a funding allocation model based on geographical conservation needs. 

Sustained involvement of a number of government, Aboriginal and volunteer participants in more communities
All funding applicants are required to demonstrate how partners will be engaged in their proposal.  As stated in the proposal criteria, 25% of their overall proposal mark is based on “Partnership/cooperation among community groups, Aboriginal groups, governments and other organizations”.

ASEF has reported a total of 382 organizations involved in project proposals since 2008 (including municipal, provincial and federal governments and First Nation communities), demonstrating a willingness of partners to be involved. Although the number of organizations involved has decreased slightly each year (2008 in 139, 2009 in 121, 2010 in 122), there seems to be a level of sustained involvement by organizations that are interested in wild Atlantic salmon.  In addition, all key informants and 75% (n=33) of survey respondents were of the opinion that program activities have led to increased involvement of the number of government, Aboriginal and volunteer participants. 

Since 2008, a total of 949 volunteers have provided a total of 14,611 hours of volunteer work (Figure 4).  In 2008, the Foundation worked with its largest number of volunteers (612), however they only provided on average 5.7 hours of work.  The number of volunteers decreased afterwards, but volunteers were more engaged and provided more hours of work (in 2009, 146 volunteers provided an average of 31 hours of work each, in 2010, 191 volunteers provided an average of 34 hours of work each).

Figure 4: Number of volunteers and hours of work
Number of volunteers and hours of work
Source: ASCF Database

Achievement of stakeholder participation, both financially and in-kind
Evaluation evidence from the document review, interviews as well as survey responses demonstrate that the Foundation has been successful in engaging project partners through financial and in-kind contributions. All key informants and 82% (n=36) of survey respondents are of the opinion that program activities have resulted in stakeholder participation both financially and in-kind.

The average funding in cash from external partners between 2008 and 2010 was $325,145.  Specifically, these external partners have contributed additional cash funding toward projects with increasing dollars from 2008 to 2010 ($267,487 in 2008, $293,896 in 2009, $414,053 in 2010). The Foundation has also been successful in engaging partners through in-kind contributions, with the exception of a drop of $136,626 in 2009 (Figure 5).  However, it is also demonstrated that the financial and in-kind contributions from funding recipients have been consistently decreasing since 2008.  At this time, it is too early to determine any trends as there have only been three years of project funding.

  Figure 5 : Value of Partner and Recipient Funding
Value of Partner and Recipient Funding
Source: ASCF Database

In addition, some key informants (n=4) spoke about the involvement of external partners both financially and in-kind. As an example, the Foundation has recently launched a fundraising initiative in collaboration with the Prince Edward Island Liquor Control Commission.  Throughout the fundraising campaign, participating suppliers will donate a portion of revenues from the sale of selected brands at the store retail outlets.  The Foundation will use these funds to help support river conservation projects in Prince Edward Island (PEI). 

4.2.2 To what extent has ASEF achieved its intermediate outcome?

Key Findings:  Through the Foundation, ASEF has achieved its intermediate outcomes by establishing partnerships with a significant number of interested organizations that undertake projects to rebuild the resource.  In addition, different types of outreach projects and communication mechanisms appear to be leading to increased public awareness about salmon conservation.

Achievement of organizations taking responsibility for rebuilding the resource
ASEF has achieved its intermediate outcome as the evaluation evidence indicates that external organizations have taken responsibility for rebuilding the resource.  The evaluation has found that external organizations are willing to undertake projects that contribute to wild Atlantic salmon conservation and are also willing to leverage further funds from external sources.

To date, the Foundation has received 157 project proposals (48 in 2008, 46 in 2009 and 54 in 2010). Of these project proposals, 65 (21 in 2008, 20 in 2009 and 24 in 2010) were approved/funded by the Foundation Advisory Committees and Board of Directors representing a total value of $804,934. Additionally, most key informants (n=10) and 80% (n=35) of survey respondents were of the opinion that ASEF activities have resulted in organizations taking responsibility for rebuilding the resource.

Based on the documentation reviewed, ASEF has funded, on average, 41% of project proposals received between 2008 and 2010.  This leaves a larger proportion (59%) of interested organizations without funding from ASEF, and as described earlier, those surveyed indicated that without this funding their projects did not go forward.  However, the number of proposals received demonstrates that there is interest in taking responsibility for the resource.  The Foundation does not have the financial capacity however to respond to the number of proposal submissions, since only approximately $300,000 per year is available to fund projects across five provinces (only $250,000 was available in 2009).

Achievement of the establishment of partnerships with stakeholders
The ASEF has achieved its intermediate outcome of establishing partnerships with stakeholders.  Between 2008 and 2010, ASEF has established partnerships with 242 organizations, specifically 88 in 2008, 90 in 2009 and 64 in 2010.  Of these partners, 48 were identified as recurring partners. Most key informants (n=10) and 77% (n=34) of survey respondents also support that ASEF activities have resulted in partnerships with stakeholders. Some key informants (n=4) also noted that applicants are returning from year to year.

Working toward increased public awareness about salmon conservation
Through different types of outreach projects and communication mechanisms, it appears that ASEF is working toward increasing public awareness about salmon conservation. The Foundation has supported 21 education and awareness projects between 2008 and 2010, and distributed approximately 600 brochures in 201114. Further, the Foundation has developed a website that has between 8,000 and 10,000 monthly hits15. The Foundation also issues, on average, two press releases per year, one announcing grants in the spring and another announcing the opening of the call for proposals in the fall. Some key informants (n=4) have said that ASEF activities have lead to increased awareness about salmon; while others (n=4) have mentioned that ASEF activities seem to be leading to increased awareness. Three additional key informants stated that increased awareness has not yet been achieved as this will take many years of both education and visibility. However, 63% (n=28) of survey respondents were of the opinion that the Foundation facilitated increased public awareness about salmon conservation.

Conclusions
The evaluation evidence indicates that ASEF has achieved its planned immediate and intermediate outcomes, with the exception of increasing public awareness, which the Foundation is working to achieve. Examples of achievements have been restoration or protection of habitat, watershed planning, stakeholder involvement (in-kind and in cash), organizations taking responsibility for salmon and establishing partnerships.

ASEF has achieved its expected immediate outcomes.  Since the creation of the fund, the projects undertaken have led to the restoration and/or protection of 94,878 m2 of wild Atlantic salmon habitat.  In addition, watershed planning projects are being undertaken in all Atlantic Provinces.  However, the number of watershed planning projects is lower in Quebec, given that the provincial government has already undertaken this task as part of its delegated powers with respect to fisheries management.

ASEF has also led to increased volunteer involvement with respect to salmon conservation.  Since 2008, the Foundation has reported a total of 382 organizations involved in project proposals, which demonstrates the willingness of partners to get involved in projects. Further, the Foundation has relied on a total of 949 volunteers to deliver its project to date.  The Foundation clearly supports partnership and cooperation among community groups, Aboriginal groups, government and other organizations.

The evaluation evidence also concludes that ASEF has led to increased stakeholder participation, both financially and in-kind.  The Foundation has been successful in engaging partners through cash contribution and through in-kind contributions.  Since 2008, the Foundation has raised over $2,218,287 dollars (including cash and in-kind) from external sources (including proposal recipients) to fund conservation and habitat restoration of wild Atlantic salmon.

The large amount of project proposals suggests that ASEF activities have resulted in organizations taking responsibility for rebuilding the resource. However, only a limited amount (approximately $300,000) is available per year to fund projects. In addition, ASEF has achieved its intermediate outcome of establishing partnerships with new and recurring stakeholders. Through different types of outreach projects (e.g. education and awareness) and communication mechanisms (e.g. brochures, website, press releases, etc.) it appears that the Foundation is working towards increasing public awareness about salmon conservation. However, some evidence suggests that the increased awareness is more challenging to achieve, since awareness may take many years, education and visibility.

14 The Foundation only started distributing brochures in January 2011

15The Foundation's website service provider

4.3 EFFICIENCY

To what extent is the design and delivery of ASEF appropriate to produce intended results?

Key Finding: ASEF has proven to be efficient through a clear governance structure, clear performance measurement and mitigating its most significant challenge, the impact of the 2008 recession.

Clear governance structure
Evidence from the evaluation indicates that the overall governance structure of the Foundation is working efficiently. Most key informants (n=8) interviewed stated that the governance structure is working well with clearly defined roles and responsibilities. The documentation reviewed also demonstrated that a clear governance structure is in place to address complexities regarding the geographical and stock status environment of Atlantic Canada and Quebec.16 Specifically, the governance structure, as discussed earlier, is comprised of six Advisory Committees (five provincial and one central) which support and provide advice and recommendations to the Board of Directors. The Board of Directors is also supported by an Investment Committee that has the responsibility, though not exclusively a requirement of the funding, to assess the investment performance of the Fund.  The Investment Committee was a requirement specified in the Funding Agreement between the Foundation and DFO to oversee “all matters related to the investment and the management of the Fund”17. Similarly, the Asia Pacific Foundation of Canada (APFC) Fund, another Endowment Fund of the Government of Canada, also specified in its Conditional Grant Agreement, that an Investment Committee was needed.18 

The comparative analysis indicated that the Foundation governance is comparable to other Foundations with Endowment Funds. All four Foundations reviewed have an Investment Committee with the responsibility to oversee the value of the grant.

Efficient performance measurement
ASEF has several mechanisms in place to monitor the performance of its projects.  Yearly business plans and annual reports (both requirements of the funding agreement), are published on the Foundation’s website and provide information on the results of funded projects. These results are gathered through different forms and templates, including the application form, progress report templates and final report templates.  A review of these forms and templates concluded that information is gathered to ensure projects are aligned with ASEF’s immediate and intermediate outcomes which in turn align with DFO priorities.  The application forms and report templates have been refined throughout the years to ensure that the necessary data is collected to support reporting requirements.  Funding recipients are required to submit progress reports every six months and a final report upon completion of their project.  The Foundation tracks all the performance data in an internal database; all interview respondents spoke about clear performance monitoring (either in funding proposals or monitoring while project is in place).

The review of documentation also indicated that the Foundation Board of Directors reviews the results of funded projects annually. As well, the Foundation also conducts site visits to ensure that recipients are accomplishing the results as planned.

Evidence of risk and challenges
The 2008 recession has proved to be the most significant challenge for ASEF, which was supported by documentation reviewed, a comparative analysis and interviews.  Only a few months after receiving the $30 million endowment fund, global markets were affected by a worldwide downturn.  The value of ASEF plunged by $5,703,686 in 2008. The Funding Agreement states that the “Investment decisions shall be made with the principal objectives of generating sufficient income to fund Ultimate Projects and Administrative Costs, preserving the principal amount of the Conditional Grant over the long term against inflationary pressures and generating long term conservative growth for such purposes.”19 The Foundation’s 2010 Annual Report states, “As of December 31, 2010, the market value of the fund was $30,086,289 as compared to the projected $29,212,423 demonstrating a modest excess over the year-end projection of the long-term financial strategy”. The other Endowment Funds examined were also affected by the recession. ASEF, PSEF and APFC have responded differently to the decline of their respective endowment fund’s value during that time. The APFC used a different approach than PSEF and ASEF, by leveraging additional funding through private and other government donations. A majority of key informants (n=7) spoke about the recession affecting the value of the fund and how the Foundation is working to increase the value of the fund.

Conclusions
The overall governance structure of ASEF is working efficiently by having a clear governance structure in place.  In addition, an Investment Committee, a requirement of the Funding agreement, is also in place to support the Board of Directors.

In terms of efficient performance measurement, ASEF has several mechanisms in place to monitor its performance of projects.  The Funding Agreement was specific about reporting requirements, which was perceived as a good practice by the evaluation team. As well, the business and annual reports were also made public on the Foundation’s website. Close monitoring of projects by the Board of Directors and onsite visits have been accomplished by the Foundation to ensure that project results are linked to the outcomes established in the logic model.

Evaluation evidence has indicated that ASEF has been mitigating the repercussions of the 2008 recession, which has been identified as its most significant challenge.

16Atlantic Salmon Conservation Foundation, 2010 Annual Report.

17Funding Agreement between DFO and the Atlantic Salmon Conservation Foundation.

18 Summative Evaluation of the Asia Pacific Foundation of Canada (March 2009).

19 Funding Agreement between DFO and the Atlantic Salmon Conservation Foundation.

4.4 ECONOMY

Is ASEF operating in a way that minimizes the use of resources to achieve its intended outcomes?

Key Finding: Through the Foundation, ASEF has delivered projects economically by using a minimum of resources, utilizing many volunteers, and leveraging significant funds from other sources.

The evaluation evidence indicates that through the Foundation, ASEF delivers its projects at a minimal cost. All key informants responded that the Foundation is using a minimum number of resources. As stated in the 2010 Annual report, “The Foundation is dependent on the work of many volunteers to fulfil its mission.” For instance, all members on Advisory Committees and on the Board of Directors are volunteers and are not compensated for their time. Some key informants (n=4) stated that it was economical for the Foundation to only have two FTEs and some (n=5) pointed to the large number of volunteers (n=949) working for the Foundation on projects funded by ASEF.

In addition, ASEF has leveraged funds from other sources in order to minimize the use of resources.  On average, the Foundation leverages $1.31 for every dollar granted20.  Figure 6 depicts the totals value of grants approved, the total amount of funds leveraged from other sources, and the total administrative fees between 2008 and 2010, and indicates that a greater proportion of funds are leveraged than granted by the Foundation.

   Figure 6 : Total Value of ASEF Projects per year
Total Value of ASEF Projects per year
Source: ASCF Database

Conclusions
ASEF has demonstrated that it is delivering projects economically by using a minimum number of resources and relying on many volunteers, who either undertake projects or work on the various committees. Additionally, between 2008 to 2010, ASEF has leveraged, on average $1.31 for every dollar spent.


20 Note that administrative overhead costs are taken into account in leveraging calculations. In 2008 the leveraging ratio was $1.70 for every $1 granted, in 2009 $0.95 for $1 granted, and 2010 $1.28 for 1$ granted. However, leveraging ratio excluding administrative overhead would be $3.77 for every $1 granted (2008), $2.17 for $1 granted (2009), and $2.42 for 1$ granted (2010).

4.5 BEST PRACTICES

It was observed throughout the evaluation that the detailed Funding Agreement between DFO and the Foundation resulted in strengthened accountability and clear expectations of the Foundation in the management and investment of ASEF. For instance, the grant agreement was specific about the fund investment conditions (e.g. Investment Committee), disbursement of the fund, program purpose, objectives, strategy and requirements for a strategic plan, annual business plan, audit and evaluation. DFO’s expectations were clearly stated and have resulted in the collection of performance data since the creation of the Foundation allowing DFO to measure and report on the program’s relevance and performance.

5.0 OVERALL CONCLUSIONS & RECOMMENDATIONS

In general, the evaluation has concluded that ASEF is relevant, is addressing a continued need and is aligned with federal government priorities that support DFO’s roles and responsibilities stated in legislation.  The evaluation evidence also indicates that ASEF has achieved its planned immediate and intermediate outcomes, with the exception of increasing public awareness, which the Foundation is working to achieve. Overall, the evaluation has concluded that the mechanisms established by the Foundation including performance measurement mitigation strategies for the 2008 recession impacts and, the overall governance structure of the fund have been efficient. The Foundation was also economical by leveraging funding from external sources. 

There are no recommendations stemming from this evaluation.

Annex I : EVALUATION MATRIX

QUESTIONS INDICATORS METHODOLOGY

RELEVANCE

1. Is there a continued need for DFO’s contribution to ASEF?

Key informants attest to the importance, in Canada, of the DFO’s contribution to ASEF

Evidence of issues related to DFO’s contribution to ASEF as still being ‘significant’

Comparison of DFO’s contribution to ASEF activities and mandate with the other similar programs

Perception of similarities/overlap/complement with other programs

  • Document Review
  • Interviews
  • Survey
  • Comparative Analysis

2. Is there a role for the federal government in DFO’s contribution to ASEF?

Program documentation provides rationale for current role of government as well as clear strong links to government objectives and priorities.

Documents demonstrate continuing relevance and alignment.

  • Document Review
  • Interviews
  • Survey

3. Are the objectives of DFO’s contribution to ASEF aligned with the federal government priorities?

Degree of alignment of DFO’s contribution to ASEF with Government of Canada and priorities.

Degree of alignment of DFO’s contribution to ASEF with DFO objectives, priorities and strategic outcomes.

  • Document Review
  • Interviews
  • Survey
  • File review

EFFECTIVENESS

4. To what extent has DFO’s contribution to ASEF achieved its immediate outcomes?

4.1 To what extent has DFO’s contribution to ASEF increased watershed planning?

Number of watershed plans developed by community groups

Number of project proposals associated with watershed plans in support of watershed plans

Number of kilometers of improved habitat

Perceptions of achievement of immediate outcomes by key informants

  • Examen de documents
  • Entrevues
  • Sondage

4.2 To what extent has DFO’s contribution increased the involvement of a number of government, Aboriginal and volunteer participants in more communities?

Report the numbers of organizations involved in project proposals

Volunteer efforts in support of ASCF – Number of hours

Perceptions of achievement of immediate outcomes by key informants

  • Document Review
  • Interviews
  • Survey

4.3 To what extent has DFO’s contribution to ASEF resulted in stakeholder participation, both financially and in-kind?

Report degree of financial and in-kind participation by funding recipient

Perceptions of achievement of immediate outcomes by key informants

  • Document Review
  • Interviews
  • Survey

5. To What extent has DFO’s contribution to ASEF achieved its intermediate outcomes?

5.1 To what extent has DFO’s contribution to ASEF resulted in organizations taking up responsibility for rebuilding the resource?

Number of project applications received

Number of approved and signed project agreements

Numbers of partner organizations contributing to project activity

Numbers of newly identified partnerships

Perceptions of achievement of intermediate outcomes by key informants

  • Document Review
  • Interviews
  • Survey

5.2 To what extent has DFO’s contribution to ASEF resulted in partnerships with stakeholders becoming established?

Number of genuine partnerships established and maintained.

Perceptions of achievement of intermediate outcomes by key informants

  • Document Review
  • Interviews
  • Survey

5.3 To what extent has DFO’s contribution to ASEF increased public awareness about salmon conservation?

Number of hits on ASCF web site

Number of ASCF press releases and public announcements

Amount of documentation distributed

Perceptions of achievement of intermediate outcomes by key informants

  • Document Review
  • Interviews
  • Survey

EFFICIENCY AND ECONOMY

6. To what extent is the design and delivery of DFO’s contribution to ASEF appropriate to produce intended results?

Roles and responsibilities are clear and well defined

Evidence of existing performance measurement strategy (PMS) and reporting system

Evidence of risk/challenges that may or has impacted the success of DFO’s contribution

  • Document Review
  • Interviews
  • SurveyComparative analysis

7. Is DFO’s contribution to ASEF operating in a way that minimizes the use of resources to achieve its intended outcomes?

Key informant views of delivery modifications to DFO’s contribution to ASEF that would make it more economical

  • Document Review
  • Interviews
  • Comparative analysi