Archived – Evaluation of the Small Craft Harbours Divestiture Class Grant Program (SCH-DCGP)

Archived information

The Standard on Web Usability replaces this content. This content is archived because Common Look and Feel 2.0 Standards have been rescinded.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Project Number 6B138
February, 2011

Table of Contents

List of Acronyms

ADM
Assistant Deputy Minister
CFO
Chief Financial Officer
DEC
Departmental Evaluation Committee
DFO
Department of Fisheries and Oceans
DNHP
Divestiture of Non-Core Harbours Program
DPR
Department Performance Report
FRHA
Fishing and Recreational Harbours Act
HQ
Headquarters
IPI
Information Portuaire/Port Information System
MOU
Memorandum of Understanding
NCR
National Capital Region
PAA
Program Activity Architecture
PM
Performance Measurement
PWGSC
Public Works Government Services Canada
RMAF
Results-based Management and Accountability Framework
SCH
Small Craft Harbours
SCH-DCGP
Small Craft Harbours Divestiture Class Grant Program
SSE
Small Scale Evaluation
TB
Treasury Board
TBS
Treasury Board Secretariat

 

Executive Summary

Introduction

This evaluation report presents the results of the evaluation of the Small Craft Harbours Divestiture Class Grant Program (SCH-DCGP). Hereinafter SCH-DCGP will be referred to as “the Program”   Small Craft Harbours is a program activity on the 2010-2011 DFO Program Activity Architecture (PAA). This evaluation was scheduled in the 2010-2011 Department of Fisheries and Oceans (DFO) five-year evaluation plan and focuses on the core issues in assessing value for money as defined by the Treasury Board Evaluation Policy (2009).  The main objectives of this evaluation are therefore to determine to what extent the Program is relevant, is managed effectively and efficiently, and whether it had achieved its stated objectives and results in an economical manner.  The timeframe for this evaluation covers the period from 2005-2006 to 2010-2011. The Program was last evaluated in February 2006 as part of the 2006 renewal process. The present evaluation was conducted by the DFO’s Evaluation Directorate and was undertaken between August 2010 and January 2011.

Program Profile

DFO is the custodian of over one thousand small craft harbours, dispersed across five regions in Canada.  Recreational, low-activity and derelict fishing harbours were targeted for divestiture in order to efficiently utilize scarce resources on core fishing harbours.  The program was created in 2001 to negotiate grant agreements with potential recipients and transfer the title of harbours to provinces, municipalities, local non-profit organizations or First Nations.

The table below provides the amount of funds used for the Program from 2005-06 to 2009-10:

Year

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

Expenditure

$410,000

$575,000

$810,000

$2,275,000

$975,000

Methodology

A non-experimental design was used for this evaluation, in which measurements are taken after the program has been implemented with no control group. The evaluation employed a variety of methods (e.g. interviews, document review) where the evidence drawn from these methods were triangulated to arrive at valid findings and conclusions.

For this evaluation, the evaluation team used a Small Scale Evaluation (SSE) approach. The SSE approach considers whether evaluators can approach an evaluation in a more resource-efficient manner, while focusing on the core value for money issues in a robust manner commensurate with the size and risk associated with the program.  

In order to determine whether the Program is a low risk program from an evaluation perspective, a risk-based assessment was conducted that examined several risk factors.  The risk-based assessment determined whether a broader or narrower scope and breadth (level of effort) was required to demonstrate relevance and performance.  In short, a more resource-efficient, economical evaluation can be conducted if evaluators can demonstrate that over the period being covered by the evaluation:

  1. Recent previous evaluation or performance findings demonstrate that the program was relevant and effective;
  2. There haven’t been any major changes with regards to internal factors (resources, activities and processes) and external factors (social, legal, economic and political conditions) and these have remained fundamentally unchanged;
  3. The level of complexity with respect to the delivery of the program is relatively simple i.e. a centralized vs. decentralized delivery, a small versus large number of partners or staff delivering the program etc.;

Evaluators found that the 2006 evaluation clearly demonstrated that it was relevant and performed well, that there were no major changes to internal and external factors, and program delivery is low in complexity. Therefore a SSE approach was deemed appropriate for the evaluation of this program.

Findings and Conclusions

Relevance

The evaluation concluded that the Program as a mechanism to divest harbours is relevant.  As of fall 2010, there remains up to 360 harbours that could be divested with this mechanism.  Furthermore, communities value these harbours and seek some continued use from them.  The divestitures of harbours continues to be a priority as it was originally defined by the 1995 Program Review and aligns with other government priorities, such as a DFO strategic outcome safe and accessible waterways. Finally, the 1995 Program Review determined that provinces, municipalities and private interests were more appropriate owners of non-core harbours, and by virtue of ownership and legislation, harbour divestiture does fall within the mandate of the minister.

Performance

For the most part, the current findings are consistent with the 2006 evaluation findings. Overall, we can conclude that the Program is effective. The evaluation found that 16 divestitures were completed as a result of using the Program compared to 11 planned between 2005-2006 and 2010-2011.

Harbours disposed of via the Program have remained safe and accessible to the public for a minimum of five years despite the fact that there is no formal monitoring system in place. The safety and accessibility of divested harbours via the Program are passively monitored on a complaint basis. There was no record of any complaints received regarding the safety and accessibility of harbours divested via the Program. The passive approach to monitoring frees up resources which can then be focused on core harbours.

Finally, the results from the key informant interviews indicate that to some extent, the Program enables activities/resources to be focused on core harbours. As a result of the Program, 16 harbours were divested thus contributing to the decreasing number of harbours left to be divested; indirectly resulting in activities/resources to be focused on core harbours.

The current evaluation confirmed what the 2006 evaluation found that the Program offered a relatively faster and less effort intense way to divest. The level of effort is greater when DFO conducts the repairs prior to disposal compared with the Program, where DFO offers a grant. When DFO conducts the repairs there is an element of project administration and management that is not involved when using the Program and therefore takes more time and effort. The Program involves the preparation and signing of a grant agreement followed by the transfer agreement at which point the grant recipient conducts the repairs. When DFO conducts the repairs prior to disposal, the potential recipient does not sign the transfer agreement until the repairs are completed. As such there is a potential risk that interested recipients may decide that they no longer want to take ownership of the harbour leaving DFO with a refurbished harbour it intended to divest. In the case of the Program, the level of risk is significantly lower as potential recipients accept the harbour “as is” by signing the transfer agreement prior to conducting repairs themselves.

National coordination of the Program is provided by National Headquarters (NHQ) in Ottawa and program operations are delivered by five regional offices.  Regions are responsible for their own divestitures using the Program and as such, roles and responsibilities vary depending on the region given the unique characteristics of each divestiture. The evaluation findings suggest that there is a general consensus with respect to roles and responsibilities.  

The 2006 evaluation found that several factors contribute towards increased costs when DFO carries out the repairs of a harbour on behalf of a potential recipient.  The current evaluation compared the steps involved in either divesting with a grant versus repairing a harbour on behalf of a grant recipient and found that the level of effort is greater when DFO conducts the repairs prior to disposal compared to the Program.  Furthermore, a grant further reduces the risk of recipients who unexpectedly withdraw their support, leaving DFO with a refurbished harbour it intended to divest and absorbing those costs. 

The current evaluation found that since the last time the program was evaluated, fewer harbours were divested using the Program and the average cost to divest harbours using the Program has more than doubled. The current evaluation confirmed the 2006 evaluation findings that contribution agreements were not seen as a viable alternative.  As for the demolition of a harbour, there are many potential hidden costs such as habitat remediation, or adjacent property erosion, and in some cases, it may not be a viable option if it is the only harbour in the area. 

In short, the evaluation concluded that the use of a grant as a tool to divest is generally cost-effective.

1. Introduction

1.1 Context of the Evaluation

This evaluation report presents the results of the evaluation of the Small Craft Harbours Divestiture Class Grant Program (SCH-DCGP). Hereinafter SCH-DCGP will be referred to as “the Program”. In accordance with the Transfer Payments Policy and the Policy on Evaluation, the Program is defined as an on-going grant and contribution and as such, must be evaluated every five years. The Terms and Conditions for the Program are scheduled for renewal in fiscal year 2010-2011, and an evaluation is required in 2010-2011 by Treasury Board as part of the renewal process.

Small Craft Harbours is a program activity on the 2010-2011 DFO Program Activity Architecture (PAA). This evaluation is scheduled in the 2010-2011 Department of Fisheries and Oceans (DFO) five-year evaluation plan and focuses on the core issues in assessing value for money as defined by the Treasury Board Evaluation Policy (2009). These core issues include: relevance and performance, including effectiveness, efficiency and economy of the Program.

The timeframe for this evaluation covers the period from 2005-06 to 2010-11. The Program was last evaluated in February 2006 as part of the 2006 renewal process. The evaluation assesses the extent to which the Program has achieved its outcomes stemming from activities outlined in the Program Logic Model. The evaluation is inclusive of the National Capital Region as well as five regional offices.  The evaluation was conducted by the DFO’s Evaluation Directorate and was undertaken between August 2010 and January 2011.

For this evaluation the evaluation team used a Small Scale Evaluation (SSE) approach. The SSE approach considers whether evaluators can approach an evaluation in a more resource-efficient manner.  Evaluators conducted a risk-based assessment to determine whether a broader or narrower scope and breadth (level of effort) was required to demonstrate relevance and performance.  An internal assessment was conducted by the Evaluation Directorate. An SSE approach was deemed appropriate for the evaluation of this program.

1.2 Objectives of the Evaluation

The main objective of this evaluation is to determine to what extent the Program is managed effectively and efficiently and whether it had achieved its stated objectives and results in an economical manner. As such, the primary focus of this evaluation was to assess value for money, and to provide clear and valid conclusions about the relevance and performance, including effectiveness, efficiency and economy of the program and where required, recommendations for improving the program going-forward. The results of this evaluation will be part of the 2010-2011 renewal process as well as for program management decision-making.

2. Program Profile/Background

DFO is the custodian of over one thousand small craft harbours, dispersed across five regions in Canada.  The Small Craft Harbours (SCH) Program was created in 1977 when the Fishing and Recreational Harbours Act wasadopted that same year.  The Act enables the Minister of DFO to acquire, contribute to, maintain, operate and repair fishing and recreational harbours facilities.  Small Craft Harbours Program activities are linked to the DFO strategic outcome: Safe and Accessible Waterways.  Following a Program Review in 1995, the Small Craft Harbours Program has undertaken to rationalize its harbour inventory via a program of divestitures and removals.  Recreational, low-activity and derelict fishing harbours were targeted for divestiture in order to utilize scarce resources on core fishing harbours.  Priority is given to the divestiture of recreational harbours, as they are seen to be more closely aligned with provincial or community interests in tourism and local economic development than with the federal mandate.

Potential recipients of harbour facilities via grant

Like all divestitures, harbour facilities divested via the Program, are offered at nominal cost, in order of priority, to the province, municipality, local non-profit organizations, or First Nations and First Nations owned corporations (for profit and non profit), and an amalgamation of such interested parties.  The process begins with community meetings where the local users are advised that a harbour is to be disposed of and interested parties willing to assume ownership are asked to come forward.  Where there is no interest in the facility, it is offered at market value through a public tendering process.  If this elicits no interest, the facility is demolished or removed, but only as a last resort. 

Methods of transferring harbours

The Small Craft Harbours Program has used two methods to accomplish the transfer of a harbour at a nominal value.  In one, if work is required to bring the harbour up to acceptable condition for the recipient, the Program completes the work before the transfer of ownership.  In the second method, instead of undertaking the repairs or improvements itself, the Small Craft Habours Program provides a grant to the recipient to accept the harbour as is.  The amount of this grant must not exceed the costs that DFO would have incurred had it done the work, and must be less than $4 million per recipient.  The recipient also assumes full responsibility for the harbour, including all future liabilities relating to its current condition. 

The Small Craft Harbours Program has found the second method to be simpler, less costly and less time-consuming than the first method.  The recipients also found the second method more attractive as they can undertake repairs according to their own priorities. 

Funding the Program

The Program is not a funded program; rather it is a funding mechanism whose funding is sourced from within DFO Reference Levels, exercised as required through vote transfers in the  Supplementary Estimates from Vote 1 (Operating) to Vote 10 (Grants and Contributions).  As part of its regular programming, DFO transfers about $1.5 million per annum from the Small Craft Harbours maintenance budget to fund the cost of divestitures. In Budget 2008, however, the four-year Divestiture of Non-core Harbours Program was established to help DFO move forward with harbour divestitures, with an amount of $45 million ($5 million in each of 2008-09 and 2009-10; $15 million in 2010-11; and $20 million in 2011-12) intended for both methods of divestiture. 

Table 1 provides the amount of funds used for the Program from 2005-2006 to 2009-2010:

Year

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

Expenditure

$410,000

$575,000

$810,000

$2,275,000

$975,000

Table 1: Expenditures for grants from 2005-2006 to 2009-2010.

Progress

Under the Program, 16 harbours have been divested at a cost of approximately $5 million, between August 2005-2006 and 2010-2011 across four regions.  They comprise of one in the Maritimes, five in Central and Arctic, nine in Quebec, and one in Pacific.

Overall, there are still approximately three hundred small craft harbours remaining to be divested.  It is not currently possible to assess which method will be used to divest those small craft harbours as the choice of divestiture methodology is determined during the negotiation process with potential recipients.  Given the current level of funding for divestitures, despite the influx of temporary funds via the four-year Divestiture of Non-core Harbours Program, it can be anticipated that the pace of divestitures will continue to be slow due to the complexity of the transactions, mandatory environmental assessments, and the high cost of those harbours remaining to be divested.  At this pace, it will take the Small Craft Harbours Program decades to divest all non-core harbours from its inventory. 

The main activities of the Program are to negotiate grant agreements with potential recipients, and; transfer the title of harbours to provinces, municipalities, local non-profit organizations or First Nations.

The Program’s internal and external partners are entities that assist in implementing the program. Internal partners at DFO include DFO’s regional finance groups, DFO’s financial advisors at Headquarters, and DFO’s Legal Services. External partners include Public Works and Government Services Canada (PWGSC).

Program beneficiaries are entities that benefit from the Program. Beneficiaries include Fisheries and Oceans Canada; Recreational boaters; Commercial fishers; and Communities (Municipalities, provincial governments, First Nations and Non-profit organizations).  

The Program is administered locally by the Regions, with the Regional Director Small Craft Harbours being responsible for verifying that recipients meet the eligibility requirements.  Overall direction is from Small Craft Harbours in Headquarters, reporting to the Assistant Deputy Minister (ADM), Ecosystems and Fisheries Management.

2.1 Logic Model of the Program

A logic model for the Program was developed in 2006 as part of the Results Based Management Accountability Framework (RMAF). For the purpose of this evaluation, the logic model has been adapted; wording was slightly changed to reflect current terminology; one of the intermediate outcomes was shifted to the final outcome, and the previous final outcome was removed.  This was done to better the results of the program over the timeframe covered by this evaluation.

Small Craft Harbours Divestiture Class Grant Program logic model: 2005-2006 to 2009-2010

3. Methodology

Evaluators used a multiple-lines-of-enquiry approach to study the issues and questions that were the focus of this evaluation. This section outlines the scope and methods of our approach, the evaluation design, evaluation questions, the methodological approach, analytical methods as well as the limitations of the evaluation.

3.1 Project Management

The evaluation was conducted by an evaluation team led by a senior evaluation manager within the Evaluation Directorate at DFO. The team collaborated with Program personnel on preparing a list of documents to review, identifying key informants and stakeholders and reviewing and providing feedback on interview guides and various reports.

3.2 Evaluation Design

A non-experimental design was used for this evaluation, in which measurements are taken after the program has been implemented with no control group. The model below is illustrated as follows:

 

Exposure to the Program

Measurement After Exposure

Clients and Beneficiaries

X 01

This model was chosen because the Program is a full coverage program and it is intended to be delivered across Canada and not withheld from any area or region and that it is appropriate to demonstrate the extent to which a program achieves issues of relevance, efficiency and economy. The evaluation employed a variety of methods (e.g. interviews, document review) where the evidence drawn from these methods were triangulated to arrive at valid findings and conclusions. 

For this evaluation the evaluation team used a Small Scale Evaluation (SSE) approach. The Policy on Evaluation states that evaluations need to collect and analyze evidence on the outcomes of policies and programs to make judgments about their relevance and performance (efficiency, effectiveness and economy).  As a result, there is a requirement to ensure that evaluations of varying scope and breadth meet this commitment while at the same time are conducted in an efficient and economical manner commensurate to the level of risk and complexity associated with the program.  The SSE approach considers whether evaluators can approach an evaluation in a more resource-efficient manner, while still conducting a robust evaluation that produces cogent findings, conclusions and recommendations.   

In order to determine whether the Program is a low risk program from an evaluation perspective, a risk-based assessment was conducted that examined several risk factors.  The risk-based assessment determined whether a broader or narrower scope and breadth (level of effort) was required to demonstrate relevance and performance.  In short, a more resource-efficient, economical evaluation can be conducted if evaluators can demonstrate that over the period being covered by the evaluation:

  1. Recent previous evaluation and/or performance monitoring data demonstrate that the program was relevant and effective;
  2. There haven’t been any major changes with regards to internal factors (resources, activities and processes) and external factors (social, legal, economic and political conditions) and these have remained fundamentally unchanged;
  3. The level of complexity with respect to the delivery of the program is relatively simple i.e. a centralized vs. decentralized delivery, a small versus large number of partners or staff delivering the program etc.;

An internal assessment was conducted by the Evaluation Directorate. A variety of information sources and data collection methods were used to determine whether a SSE approach was appropriate for the Program. Evaluators found that the Program 2006 evaluation clearly demonstrated that it was relevant and performed well, that there were no major no major changes to internal and external factors, and program delivery is low in complexity. Therefore, an SSE approach was deemed appropriate for the evaluation of this program.

3.3 Evaluation Questions

The questions were determined on the basis of the new evaluation policy, by reviewing documents and the results of Phase I interviews with key contacts. Annex A features an evaluation matrix organized by the key issues covered: relevance and performance (effectiveness, efficiency and economy).

3.4 Secondary Data Sources

Secondary data was examined including: the Program’s RMAF Performance indicators; document reviews, audit interview results and program databases.  

The document review consisted of the analysis of documents provided by the program and included both public and internal program documents. Sources of information reviewed as part of the document review included; program administrative data, including Information Portuaire/Port Information (IPI) database; Program Terms and Conditions; RMAF and PAA; the 2006 evaluation of the Program; and results from the 2010 audit interviews1.

3.5 Primary Data Sources

Primary data was collected by the evaluation team specifically for this evaluation study. They consisted of 10 key informant interviews with program personnel, grant recipients and partners.

Evaluators sampled and conducted interviews with five program personnel and two partners. The purpose was to obtain information on the perceptions of experienced key informants. These individuals play a significant role in or have extensive experience with the program's design or implementation, as well as information on the importance of the program. Given this evaluation was a SSE, three clients (grant recipients) were randomly selected from the 16 divested harbours to discuss the Programs purpose and impact and ask for their opinion on the design and delivery of the program.

3.6 Analytical Methods

The analytical methods used for this evaluation were tailored to the nature and availability of the data to be gathered, which were in turn linked to the evaluation questions. Triangulation was used as an analytical method. In the social sciences, triangulation means multiple lines of evidence were used to substantiate the findings.

Given that this evaluation was an SSE, an iterative process to analyse data was used.  For example, for each evaluation question, we commenced with the gathering and analysis of documentation.  Initial findings were identified.  This was then followed by our key informant interviews of program personnel, partners and clients.  An analysis of this data was conducted, followed by a close examination of the findings. Where findings were deemed to require additional evidence, further data collection and analysis ensued until the conclusions were sufficiently supported by the findings. Conversely, if initial findings were not supported by multiple lines of evidence, there were not presented as such in this report.

3.7 Methodological Limitations and Constraints

Over the course of the evaluation we observed some shortcomings in the methodology. In order to minimize their impact on the results of the evaluation, we corroborated with information collected from a variety of sources using a variety of methods. In other words, we combined the various methods discussed earlier to arrive at the same conclusions, thereby reinforcing our assessment as to their validity. None of these shortcomings significantly jeopardized the validity or accuracy of the evaluation results.

  • Non Experimental Design. When using this model, it is difficult to clearly measure the net effects of the Program. Since there are no measurements from before the program began, nor a comparison group against which to assess other plausible causes for the outcome, it is difficult to attribute impacts to the program. In other words, it is difficult to draw a clear conclusion about the incremental or net effects of the program. Although this model lacks scientific rigour, the rigour of this design was increased by describing activities, outputs and outcomes through a logic model, enabling evaluators to make causal linkages, and logically argue that results can be attributed to the program (and increase the internal validity).

  • Data from interviews. The sample of grant recipients interviewed reflected the types of clients served by the Program but was not representative of all users of the program's services. Because the limited number of program personnel involved in the delivery of the Program, this limited the number of perspectives. In light of this, and where possible, their responses were corroborated with other lines of evidence to validate the statements they presented as evidence. Though the evidence put forward by respondents was considered robust and sufficient to arrive at grounded conclusions, should it have been necessary, the SSE approach did allow room to interview additional respondents.

  • Economy Analysis. Evaluators were unable to compare the costs to repair a harbour using the Program against the costs to repair a harbour when DFO undertook those repairs.  This is because no two harbours are alike and data was not available to compare divestiture of a single harbour using costs or cost estimates for either the Program or if DFO were to conduct the repairs.  In order to mitigate this, evaluators defined the generic level of effort to divest a harbour based on the steps involved for either the Program or if DFO were to conduct the repairs.

4. Major Findings

4.1 Relevance


Relevance Question #1.1: Is there a continued need for the Small Craft Harbours Divestiture Class Grant Program?

Key Findings

The evaluation found that there is a continued need for the Program as there are still up to 400 non-core harbours remaining to be divested. The evaluation also found that non-core Small Craft Harbours are, in general, an asset that communities do not wish to lose and that grants allow communities to put these harbours to good use.  Finally, the evaluation found that a grant is a mechanism that allows for a faster and less expensive approach to divest harbours.

Evidence

The current evaluation found that there are a total of 400 non-core harbours (Graph 1) remaining as potential candidates for a divestiture through the use of a grant.

Graph 1: Distribution of the Conditions of Remaining Non-Core Harbours

The 2006 evaluation found that grant funding allowed DFO to divest harbours to communities who viewed their harbour as a key resource within their community and to maintain safe access to them. The current evaluation found that non-core Small Craft Harbours are, in general, an asset that communities do not wish to lose. Indeed, evidence from the document review also found that derelict harbours may not be used extensively by the fishing industry but continue to be important to the local community for other uses such as recreation, tourism, or use as a storm haven. All program key informants interviewed (5) stated that by virtue of working close with recipients, the grant provided positive impacts for both the program and the communities which are acquiring the ownership of the harbour.

In terms of utility, the 2006 evaluation found that a grant offers a quick, easy way to divest. Transferring a property using a grant with the recipient undertaking the repairs allows the department to make a complete break with the property. The current evaluation confirmed what the 2006 evaluation found in terms of the grant being a mechanism that allows for a faster and less expensive approach to divest harbours. Indeed all program key informants interviewed (5) stated that divesting with a grant was faster and less expensive than a divestiture which involves DFO undertaking the repairs of the harbour. As stated in Section 4.3 (Efficiency), the current evaluation found that the level of effort is greater when DFO conducts repairs prior to disposal compared with the Program. When DFO conducts the repairs, there is an element of project administration and management that is not involved when using the Program and therefore takes more time and effort. Overall, the results from the clients interviewed suggest that they favour a grant over repairs undertaken by DFO as recipients can adapt the repairs of the harbour to meet their needs. Indeed, the two clients interviewed confirmed that with the grant they received they were able to adapt their harbours to meet their community’s needs.

Relevance Question #1.2: To what extent are the objectives of the Small Craft Harbours Divestiture Class Grant Program aligned with departmental and governmental wide priorities?

Key Findings

The evaluation found that the Program represents important ongoing work that aligns with various departmental and governmental wide priorities.  For example, the Program continues to be aligned with the 1995 Program Review’s proposition to dispose of recreational and low-use/derelict commercial fishing harbours to local recipients. The Program was also aligned with the former Treasury Board Secretary (TBS) policy on Alternative Service Delivery, however the more recent 2010 Budget speech highlighted identifying opportunities for savings and the improvement of service delivery, however did not specifically mention divestitures.  Finally, the Program aligns with the PAA strategic objective Safe and Accessible Waterways.

Evidence

The document review stated that Small Craft Harbours continues to align with the 1995 Program Review proposition to dispose of recreational and low-use/derelict commercial fishing harbours to local recipients. In other words, the Program helps devolve recreational and low-activity fishing harbours to provincial or community interests.

The Program also aligned with the former Treasury Board Secretariat (TBS) policy on Alternative Service Delivery. This policy’s objective was to ensure that the potential for innovative organizational arrangements for service delivery is explored within departments as well as outside the departmental structure. It should be noted that this TBS Alternative Service Delivery Policy has since been replaced in part by the Policy on Reporting of Federal Institutions and Corporate Interest on April 1st 2007. This new policy does not explicitly address the issue of alternative service delivery. However, as the 2010 Budget speech noted, the Government is still interested in reviewing Government administrative functions and overhead costs in order to identify opportunities for additional savings and improve service delivery.

The alignment of the Program to the 2010-11 strategic outcome of “safe and accessible waterways” is explained by the fact that the Program final outcome “Enhance ability to focus resources on harbours that are critical to the fishing industry” aligns with the overall Small Craft Harbours program intermediate outcome “Core commercial fishing harbours that meet principal and evolving needs of the commercial fishing industry and coastal/shore communities” which in turn aligns with the DFO strategic outcome “Safe and accessible waterways”.

Relevance Question #1.3: Is the current role of the Federal Government appropriate in delivering the Small Craft Harbours Divestiture Class Grant Program?

Key Findings

This evaluation found that the current role of the Federal Government is aligned with the Small Craft Harbours program. The evidence for this alignment is based on the results of the Program Review which found that the non-core harbours were better aligned with provincial, municipal or private interests.  In addition, the Crown is the legal owner of the harbours through the Fishing and Recreational Harbours Act (FRHA) giving the minister the mandate to acquire harbours as well as the right to dispose of harbours through the Federal Real Property Regulations.

Evidence

The 2006 evaluation found that the decision to dispose of recreational and derelict/low activity harbours to local recipients formed part of DFO’s Program Review2 commitments and that they were better aligned with provincial, municipal or private interests. The current evaluation re-examined those findings and confirmed that the Small Craft Harbours passed the Program Review’s ‘role of government’ test, but because the Federal Government owns the harbours, it is by default involved in the divestiture process.

The current evaluation confirmed that the Crown has ownership of the scheduled harbours and has the right to dispose of them. By virtue of the Fishing and Recreational Harbours Act (FRHA), the Minister of DFO has administration and control of small craft harbours where the Crown is the legal owner of the harbours.  The FRHA provides for the acquisition, administration and development of certain fishing and recreational harbours in Canada. Under this Act, the minister is mandated acquire harbours and DFO administers the use, management and maintenance of every scheduled harbour. Finally, the minister has a right to dispose of these harbours under section 4.1 (1) of the Federal Real Property Regulations. 

4.2 Effectiveness


Effectiveness Question #2.1: To what extent has the Small Craft Harbours Divestiture Class Grant Program disposed of recreational, derelict and low activity fishing harbours in a timely and cost-effective manner?

Key Findings

The evaluation found that a total of 132 harbours were divested since the 2006 evaluation. There were 16 divestitures completed using the Program compared to 11 planned divestitures. Therefore 116 harbours were disposed of via repairs or demolished by DFO. The decision to elect for the divestiture of a non-core harbour either by the Program versus having DFO conduct the repairs is dependent upon the recipient’s preference. This underscores that the Program is a mechanism or another option to divest non-core harbours.

The extent to which the Program has divested non-core harbours in a timely manner will be addressed in Section 4.3 (Efficiency) and the extent to which the Program has divested non-core harbours in a cost-effectiveness manner will be examined in Section 4.4 (Economy).

Effectiveness Question #2.2: To what extent have the disposed harbours remained safe and accessible to the public for a minimum of five years?

Key Findings

The 2006 evaluation found that all the harbours divested through the Program were still in active operation. Regions were not actively monitoring the maintenance of safe public access and use; however they rely on their ongoing relationships with the recipients and their involvement in other harbours. Monitoring of these harbours was done passively namely by receiving complaints from local people. A more passive approach to monitoring harbour accessibility avoided showing an active interest in the divested harbours which could have been misinterpreted that DFO still had an active interest in those harbours and that continued federal financial involvement was possible. Actively monitoring divested harbours would not allow the program to divert resources to core harbor activities.

The current evaluation found that harbours disposed of via the Program have remained safe and accessible to the public for a minimum of five years despite the fact that there is no formal monitoring system in place. The safety and accessibility of divested harbours via the Program are passively monitored on a complaint basis. There was no record of any complaints received regarding the safety and accessibility of harbours divested via the Program.

Evidence

As a pre-condition, grant recipients must agree to keep divested harbours safe and accessible to the public for a minimum of five years. The majority of program key informants (4 of 5) stated that the divested harbours via the Program have remained safe and accessible. None of the program key informants interviewed were aware of any complaints made as a result of safety and/or accessibility issues. It is assumed that harbours must be safe and accessible if no complaints were received.

All grant recipients (3) interviewed indicated they had not received any complaints from users. When asked if they have a plan in place to keep the harbours safe and accessible; only one recipient indicated they were planning to develop strategies that will ensure they have ongoing funding to maintain the reliability and the accessibility of the harbour.

When asked if safety and accessibility was being monitored, half of program key informants (2 of 4) stated it was being monitored.  All program key informants (5) interviewed agreed that the harbours are not actively monitored i.e. no formal monitoring process in place, however, monitoring was occurring passively on a complaint basis.  The majority of grant recipients (2 of 3) interviewed indicated that the harbours are safe and accessible as a result of the repairs completed using the Program.  

Effectiveness Question #2.3: To what extent has the Small Craft Harbours Divestiture Class Grant Program enhanced the ability of Small Craft Harbours to focus resources on core harbours?

Key Findings

The 2006 evaluation found that although divesting non-core harbours allowed the Small Craft Harbours to free up human and financial resources the overall impact of this shift in resources was minimal. In addition, it was found that regional managers believed that the divestment of assets had freed up human and financial resources although the amounts are minimal as these harbours did not require much from DFO before divestiture.

The current evaluation found that to some extent, the Program enables activities/resources to be focused on core harbours. The Program is a tool used to divest non-core harbours and therefore contributes (16 harbours were divested via the Program) to the decreasing number of harbours left to be divested; indirectly resulting in activities/resources to be focused on core harbours. 

Evidence

When asked whether the Program enables activities/resources to focus on core harbours, all program key informants interviewed (5) indicated that to some extent3 the Program does enable activities/ resources to focus on core harbours. According to two of the five program key informants, it should be assumed that with fewer harbours to divest, resources can be focused on core harbours.

As stated in Section 4.3 (Efficiency), the current evaluation found that the level of effort is greater when DFO conducts repairs prior to disposal compared with the Program. When DFO conducts the repairs, there is an element of project administration and management that is not involved when using the Program and therefore takes more time and effort.

4.3 Efficiency


Efficiency Question #3.1: To what extent is the design and delivery of Small Craft Harbours Divestiture Class Grant Program appropriate to produce outputs?

Overall Key Findings

To determine the extent to which the design and delivery of the Program is appropriate to produce outputs, the following was examined; the level of effort required for the divestiture process using the Program compared to DFO completing the repairs; the challenges/barriers faced when using the Program; the extent to which roles and responsibilities are clear and well defined; and the extent to which the performance data supports decision –making and departmental accountability requirements.

Overall, the Program is operating efficiently with respect to the level of effort required to divest non-core harbours using the Program and the general understanding of the roles and responsibilities within and between a region and NHQ. However, some improvements can be made with respect to the collection and usefulness of Program performance data and the identified challenge/barrier faced when using the Program, for example, the timeliness of supplementary estimates.

Level of Effort Required for Divestiture Process

Key Findings

The current evaluation confirmed what the 2006 evaluation found in terms of the Program offering a relatively faster and less effort intense way to divest. The level of effort is greater when DFO conducts the repairs prior to disposal compared with the Programs, where DFO offers a grant. When DFO conducts the repairs there is an element of project administration and management that is not involved when using the Programs and therefore takes more time and effort. The Programs involves the preparation and signing of a grant agreement followed by the transfer agreement at which point the grant recipient conducts the repairs.

When DFO conducts the repairs prior to disposal, the potential recipient does not sign the transfer agreement until the repairs are completed. As such there is a potential risk that interested recipients may decide that they no longer want to take ownership of the harbour leaving DFO with a refurbished harbour it intended to divest. In the case of the Program, the level of risk is significantly lower as potential recipients accept the harbour “as is” by signing the transfer agreement prior to conducting repairs themselves.

Evidence

The Small Craft Harbours Program uses two methods to accomplish the transfer of a harbour; repairs are completed by DFO before the transfer of ownership or a grant is provided to a recipient to accept the harbour as is.

When comparing the steps involved in the Programs and when DFO conducts the repairs prior to disposal (see Figure 1), the evaluation found that the level of effort is greater when DFO conducts the repairs as the amount of time and effort required for the administration and management of a project is greater than the preparation and approval of a grant agreement.  There is considerable amount of work involved with respect to the procurement activities and contract management activities. 

Figure 1: Divestiture Process Chart

The majority of program key informants (4 of 5) interviewed indicated that the difference between DFO conducting repairs prior to disposal and the Program is the project administration and management component required when DFO conducts the repairs. The Program involves the preparation and signing a grant agreement and a transfer agreement while if DFO were to conduct the repairs, there would be the administration and management of an entire construction project (RFP, tender process, procurement process, manage repair process, etc.). The document review found that a divestiture when DFO conducts the repairs may take up to five years to complete, including at least one year for planning and two for contracting and administrative work.  

Similar to program key informants, all PWGSC partners (2) interviewed indicated that the Program is faster than if DFO were to conduct repairs prior to disposal. When DFO conducts repairs prior to divestiture, PWGSC is often responsible for the conduct or management of the repairs. The repair stage is often the longest stage in the divestiture process by the sheer fact that the activities (i.e. the elaboration of the plans and the estimate, the call for tender and the execution of the works) involved must be undertaken by the federal government.  

Challenges/barriers faced when using the Program

Key Findings

The evaluation identified one main challenge/barrier when using the Program; the main funding challenge was the timing of the supplementary estimates and the unexpected costs or inaccurate repair estimates. The timing of supplementary estimates, i.e. the process of converting funds from operating (vote 1) to grant and contribution (vote 10) can result in a delay or loss of a planned divestiture. The unexpected costs or inaccurate repair estimates can lead to the grant recipient not having enough funds to repair a harbour. Such a challenge could result in the recipient not being able to keep the harbour safe and accessible to the public for a minimum of five years as stipulated in the Transfer Agreement.

Evidence

The Program is not a funded program, rather it is a funding mechanism whose funding is sourced within DFO Reference Levels, exercised as required through vote transfers in the supplementary estimates from vote 1 (operating) to vote 10 (grants and contributions). When funding is not converted from vote 1 to vote 10 in a timely manner it can result in a delay or loss of a planned divestiture. The majority of program key informants (4 of 5) interviewed stated that the timing of supplementary estimates can cause delays and/or loss of a planned divestiture. For example, if DFO enters into negotiations with a potential recipient at the end of the fiscal year, it will not be possible to convert funds due to the timing of supplementary estimates; there are only three opportunities a year to convert funds via supplementary estimate process.

Program Roles and Responsibilities

Key Findings

National coordination of the Program is provided by National Headquarters (NHQ) in Ottawa and program operations are delivered by five regional offices.  The Director General (DG) of Small Craft Harbours and the Small Craft Harbours National Management Committee are responsible for national oversight of the divestiture program, inclusive of the Program. Regions are responsible for their own divestitures using the Program and as such, roles and responsibilities vary depending on the region given the unique characteristics of each divestiture. When asked about NHQ and regional roles and responsibilities, there was a general consensus with respect to roles and responsibilities among program key informants, however, each key informant’s interpretation varied slightly within their region, at NHQ and between regions and NHQ.

Evidence

When asked about roles and responsibilities, program key informants identified the following regional roles and responsibilities: oversee /supervise the grant program; leading negotiations with potential recipients and ensure recipients meet the requirements set out in the Terms and Conditions; process of land transfers; ensure data is being inputted into the Port Information System; notify NHQ of supplementary estimate conversions if additional funds are required; and manage regional divestiture budgets.

Given each region is responsible for the management of their own divestiture using the Program, roles and responsibilities vary depending on the region as every grant is site specific. For example; while there is a general process used to divest harbours using the Program however, each site has its own unique issues.

The following NHQ roles and responsibilities were also outlined by program key informants; overall support and coordination; program and policy design; cash management inclusive of facilitating and leading the supplemental estimate conversions; management of national budget; liaise with finance; oversee information systems (IPI) and monitor grants; and provide guidance to the regions. All program key informants interviewed felt that for the most part, roles and responsibilities were clear; however they varied slightly from region to region and from site to site.

Program Performance Data

Key Findings

All components of a Performance Measurement (PM) Strategy, with the exception of a detailed program profile were included in the program’s 2006 Results-based Management Accountability Framework (RMAF). There are seven performance indicators outlined in the RMAF and data has been consistently collected for six of those seven indicators. Results from the program key informant interviews indicated that the data is easy to collect and for the most part is up to date and reliable.

The evaluation strategy outlined in the RMAF states there will be an annual evaluation of the program to assess the identified performance indicators with specific focus on two of the seven indicators; however, there was no evidence to support that an evaluation has been conducted since the 2006 evaluation.

The Program data is currently used to report on divestitures overall and included in reports such as the semi-annual Port Information System reports as well as the Departmental Performance Report. The evaluation strategy included in the Terms and Conditions states the Program data/results will be reported in the Harbour Disposal Program Annual Report, however, there was no evidence that such a report has been produced. There was no evidence found during the evaluation that performance data is used for specific Program reporting requirements. Results from the evaluation indicate that the Program performance data is not formally being used to support decision-making and departmental accountability requirements; however it is used to demonstrate the usefulness and need for the program.

Evidence

The evaluation found that the Program has an approved 2006 RMAF inclusive of a logic model, performance indicators, an evaluation strategy and a reporting strategy. The program is currently in the process of developing a formal PM Strategy, and an approved PM Strategy is expected to be in place for the renewal of the Program.

According to the document review, there are seven performance indicators outlined in the RMAF. Results from the program key informant interviews indicated that the Program data has been consistently collected over the past five years for six of the seven indicators outlined in the RMAF. Three of five program key informants stated that data is not being collected on the performance indicator “projected cost of repairs/improvements if performed by DFO prior to disposal”.

All program key informants (5) interviewed stated that the performance data is easy to collect and is stored in Port Information System as well as files in the regions. All regional program key informants stated that the performance data is up to date and reliable (regions are responsible for inputting data into Port Information System).

The evaluation strategy outlined in the RMAF states an annual self-evaluation of the program will occur to assess the performance indicators, specifically the number of disposals using grants vs. the projected cost of disposals through repairs/improvements, and the number and percentage of disposed harbours that have maintained safe and public access for a minimum of five years. There was no evidence found during the evaluation that annual self-evaluations have occurred since the evaluation was conducted in 2006 by the Evaluation Directorate. Furthermore, the program key informants interviewed confirmed there is no data being collected for some of the indicators specifically outlined in the evaluation strategy.

Results from program key informant interviews and the document review indicated that the Program data is currently used to report on divestitures overall and included in reports such as the semi-annual Port Information System reports as well as the Departmental Performance Report. The Program is a very small component of divestitures overall, for example, the document review found the only indicator in the Management Resources and Results Structure (MRRS) is regarding divestiture in general and not specific to the Program. The Terms and Conditions stipulate that Program evaluation results as well as other statistical information are to be reported in the Harbour Disposal Program Annual Report which is to be distributed to senior management at DFO; however, there was no evidence found during the evaluation that a Harbour Disposal Program Annual Report was being produced.

All program key informants (5) interviewed stated that the Program performance data does not necessarily support overall Small Craft Harbours decision making, but the data does inform program managers the need for the Program.

4.4 Economy


Economy Question #4.1: Is the Small Craft Harbours Divestiture Class Grant Program operating in a way that minimizes the use of resources to achieve its intended outcomes?

Key Findings

The 2006 evaluation found that when comparing repairs done by DFO to the Program, the repairs often cost more than if they had been done by the grant recipient. This is in part because PWGSC has to comply with federal standards for safety repairs which sometimes exceed local needs, and this compliance costs more. The current evaluation found that since the last time the program was evaluated, fewer harbours were divested using the Program and the average cost to divest harbours has more than doubled.  The difference in costs between this and the 2006 evaluation may be explained in part by the rising costs in repairs in general and that harbours are ageing and requiring more extensive repairs.  The current evaluation found that the level of effort is greater when DFO conducts the repairs prior to disposal compared to the Program4.  The current evaluation confirmed the 2006 evaluation findings that contribution agreements were not seen as a viable alternative.  Finally, in some cases, demolishing a harbour can incur costs in excess of those incurred via a grant, such as habitat remediation costs or damage to nearby properties, which is particularly the case with ocean-based harbours. 

Evidence

DFO repairs

The 2006 evaluation found that having DFO conduct the repairs often costs more than having the repairs done locally.  PWGSC was more expensive when working on remote harbours.  Moreover, it also charges administrative and project management fees of up to 25 per cent of their costs, making repairs still more costly.

The 2006 evaluation also found that grant divestitures also achieved cost savings because local recipients contracted local firms and were able to repair harbours and extend their lives.

Cost

The 2006 evaluation found that the Program divested 42 harbours over a five year period at an average cost of $132 000. The current evaluation found that the Program divested 16 harbours over a five year period at an average cost of $315 000 (Table 3).  The difference in costs between this and the 2006 evaluation may be explained in part by the rising costs in repairs in general and that harbours are ageing and requiring more extensive repairs.

National Divestitures ($ ‘000)
Fiscal Years Actual Divestitures Actual Expenditures
2005-06 3 $ 410
2006-07 3 $ 575
2007-08 2 $ 810
2008-09 5 $ 2 275*
2009-10 3 $ 975*
Total 16 $ 5 045
Average $ 315

* The Program had access to DNHP Funding
Table 3: Harbours Divested over the Past 5 Years

Contribution Agreements

The 2006 evaluation found that compared with contributions, the Program is a less cumbersome divestiture tool because it is clear that responsibility has shifted to the recipient.  The 2006 evaluation also found that with respect to contribution agreements, there is a possibility that the recipient might be unwilling to accept a harbour in the end.  As for the current evaluation, none of the key informants raised this approach as a viable alternative, and the circumstances as well as the method surrounding the administration of a contribution agreement hasn’t changed since the 2006 evaluation.

Demolition

The 2006 evaluation found that the costs associated with the removal of a harbour can be expensive.  Divestiture can lead to additional costs due to damaged shorelines when a wharf or breakwater is removed or dealing with the resulting erosion at nearby properties.  This was of particular concern with ocean-based harbours where tides and currents can cause significant environmental damage following removal of a harbour. 

The current evaluation findings came to similar conclusions.  Costs that go beyond the simple removal of harbour elements, such as habitat remediation, need to be factored into the overall cost of divestiture.  Two of five program key informants noted that demolishing a harbour is not necessarily more cost–effective and can involve habitat remediation or other environmental costs.  Another program key informant noted that there are many laws that make a demolition a more complex and expensive alternative to divesting via a grant.

The 2006 evaluation also found that where alternative harbour facilities were unavailable, discussions about demolition were not part of the negotiations.  The current evaluation also found that if alternative facilities did not exist the demolition of a harbour was not considered a viable option.  A program key informant noted that the primary objective of the overall Small Craft Harbours program is to provide continued service to the public.  Finally, a 2008 Divestitures Principles directive also confirmed that demolitions would only proceed following full community consultations and where there are alternative harbour services available at a nearby site.

4.5 Lessons Learned

Divestiture of non-core harbours is an important activity of the overall Small Craft Harbours Program.  To accomplish this, program managers use a variety of means, which include demolishing a harbour, repairing it on behalf of a recipient or providing it to a recipient along with a grant the completion of repairs.  In the third scenario, the Small Craft Harbours Program leverages the use of the Government of Canada Grant and Contribution instrument in a unique way.  The use of a grant serves as an alternative mechanism in which the recipients can repair a harbour with flexibility and with regards to their local needs for repairs undertaken, and with minimal risk to DFO.  The Program is typically associated with the delivery of a program, but here it has been creatively used as a tool or mechanism to divest itself of real property, a lesson that other departments, with assets they wish to divest, could learn from.

5. Conclusions and Recommendations

5.1 Relevance

The evaluation concluded that the Program as a mechanism to divest harbours is relevant.  As of fall 2010, there are still up to 360 harbours that could be divested with this mechanism.  Furthermore, communities value these harbours and seek some continued use from them.  The divestiture of harbours represents important ongoing work that continues to be a priority as it was originally defined by the 1995 Program Review and aligns with other government priorities, such as the DFO strategic outcome of Safe and Accessible Waterways. Finally, the Program Review determined that provinces, municipalities and private interests were more appropriate owners of non-core harbours, and by virtue of ownership and legislation, harbour divestiture does fall within the mandate of the minister.

5.2 Effectiveness

For the most part, the current findings are consistent with the 2006 evaluation findings. Overall, we can conclude that the Program is effective. The evaluation found that 16 divestitures were completed as a result of using the Program compared to 11 planned.

Harbours disposed of via the Program have remained safe and accessible to the public for a minimum of 5 years despite the fact that there is no formal monitoring system in place. The safety and accessibility of divested harbours via the Program are passively monitored on a complaint basis. There was no record of any complaints received regarding the safety and accessibility of harbours divested via the Program. The passive approach to monitoring frees up resources which can then be focused on core harbours.

Finally, the results from the key informant interviews indicate that to some extent, the Program enables activities/resources to focus on core harbours. As a result of the Program, 16 harbours were divested thus contributing to the decreasing number of harbours left to be divested; indirectly resulting in activities/resources to be focused on core harbours. 

5.3 Efficiency

The evaluation findings suggest that the Program demonstrated several efficiencies especially with respect to the level of effort required to divest a non-core harbour compared to DFO completing the repairs prior to disposal.

The current evaluation confirmed what the 2006 evaluation found in terms of the Program offering a relatively faster and less effort intense way to divest. The level of effort is greater when DFO conducts the repairs prior to disposal compared to the Program. When DFO conducts the repairs there is an element of project administration and management that is not involved when using the Program and therefore takes more time and effort. The Program involves the preparation and signing of a grant agreement followed by the transfer agreement.

The timing of the supplementary estimates can result in a delay or loss of a planned divestiture. The timing of supplementary estimates is established by the Treasury Board and must be adhered to by all departments. However, the timing of the estimates and when a recipient may be interested in a harbour is out of the control of program managers. The unexpected costs or inaccurate repair estimates can lead to the grant recipient not having enough funds to repair a harbour. This could result in the recipient not being able to keep the harbour safe and accessible as stipulated in the RMAF as well as the Program not being able to achieve its expected outcome (disposed harbours remain safe and accessible to the public for a minimum of five years).

National coordination of the Program is provided by National Headquarters (NHQ) in Ottawa and program operations are delivered by five regional offices.  Regions are responsible for their own divestitures using the Program and as such, roles and responsibilities vary depending on the region given the unique characteristics of each divestiture. The evaluation findings suggest that there is a general consensus with respect to roles and responsibilities.

The evaluation concluded that data has been consistently collected for six of the seven performance indicators outlined in the RMAF. The evaluation strategy states there will be an annual self-evaluation of the program to assess the identified performance indicators with specific focus on the two of seven indicators to which data is not being collected. There was no evidence to suggest that a self-evaluation has been conducted since the 2006 evaluation. Given the Program is a transfer payment program, a formal evaluation is only required every five years.

The Program performance data demonstrates the usefulness and need of the program and is currently used to report on divestitures overall. The evaluation strategy included in the RMAF states the Program data/results will be reported in the Harbour Disposal Program Annual Report, however, there was no evidence that such a report has been produced. If there is no intent to produce such a report, it should be removed from the RMAF.

5.4 Economy

The 2006 evaluation found that several factors contribute towards increased costs when DFO carries out the repairs of a harbour on behalf of a potential recipient.  The current evaluation compared the steps involved in either divesting with a grant versus repairing a harbour on behalf of a grant recipient and found that the level of effort is greater when DFO conducts the repairs prior to disposal compared to the Program.  Furthermore, a grant further reduces the risk of recipients who unexpectedly withdraw their support, leaving DFO with a refurbished harbour it intended to divest and absorbing those costs. 

The current evaluation found that since the last time the program was evaluated, fewer harbours were divested using the Program and the average cost to divest harbours has more than doubled. The current evaluation confirmed the 2006 evaluation findings that contribution agreements were not seen as a viable alternative.  As for the demolition of a harbour, there are many potential hidden costs such as habitat remediation, or adjacent property erosion, and in some cases, it may not be a viable option if it is the only harbour in the area. 

In short, the evaluation concluded that the use of a grant as a tool to divest is generally cost-effective. 

Annex A- Evaluation Matrix

Small Craft Harbours Divestiture Class Grant Program Evaluation Matrix

Issue Indicators Data Source
1.0 RELEVANCE
1.1 Is there a continued need for the SCH-DCGP? 1.1.1 Stakeholders attest to the importance/need for the SCH-DCGP Document review

Key informant interviews
1.1.2 # of harbours remaining in the inventory that can be divested using the SCH-DCGP Document review
1.2 To what extent are the objectives of the SCH-DCGP aligned with departmental and governmental wide priorities? 1.2.1 Degree of alignment of SCH-DCGP with:
  • Government of Canada objectives and priorities
  • DFO objectives, priorities and strategic outcome
  • SCH objectives, vision and mission
Document Review
1.3 Is the current role of the Federal Government appropriate in delivering the SCH-DCGP? 1.3.1 Role of Federal Government is appropriate in delivering the SCH-DCGP Document Review

Key informant interviews
2.0 PROGRAM OUTCOMES (EFFECTIVENESS)
2.1 To what extent has the SCH-DCGP disposed of recreational, derelict and low activity fishing harbours in a timely and cost-effective manner? 2.1.1 # of harbours divested of via:
  • SCH-DCGP
  • repairs completed by DFO
Document review
2.1.2 Level of effort required for divestiture process via:
  • SCH-DCGP
  • repairs completed by DFO
Document review

Key informant interviews
2.2 To what extent have the disposed harbours remained safe and accessible to the public for a minimum of 5 years? 2.2.1 Perception of stakeholders of disposed harbours that have remained safe and accessible to the public for a minimum of 5 years Key informant interviews
2.2.2 # of complaints received about the safety and accessibility of harbours divested via SCH-DCGP Document review

Key informant interviews
2.3 To what extent has the SCH-DCGP enhanced the ability of SCH to focus resources on core harbours? 2.3.1 Level of effort required for divestiture process via:
  • SCH-DCGP
  • Repairs completed by DFO
Document review
2.3.2 # of harbours divested via SCH-DCGP Document review
2.3.3 Degree to which the grant program enables activities/resources to focus on core harbours Document review

Key informant interviews
3.0 EFFICIENCY
3.1 To what extent is the design and delivery of SCH-DCGP appropriate to produce outputs? 3.1.1 Level of effort required for divestiture process via:
  • SCH-DCGP
  • repairs completed by DFO
Document review
3.1.2 Challenges/barriers faced when using the SCH-DCGP (process, terms and conditions, etc.) Key informant interviews
3.1.3 Roles and responsibilities are clear and well defined (HQ and Regions) Document review

Key informant interviews
3.1.5 Extent to which the performance data supports decision-making and departmental accountability requirements (usefulness) Document review

Key informant interviews
4.0 ECONOMY
4.1 Is the SCH-DCGP operating in a way that minimizes the use of resources to achieve its intended outcomes? 4.1.1 Alternative approaches to divesting non-core harbours available to the government Document review

Key informant interviews

1. A 2010-11 audit of the Program was required by Treasury Board for renewal approval.

2. Program Review is a periodic, detailed report on the historical development, current activities and performance, goals and needs of a specific program – qualitative, quantitative and financial -- and a statement of how that program's content and activities relate to DFO’s mission and goals.

3. They were asked to chose between: not at all; to some extent or to a great extent

4. See Section 4.3 (Efficiency)