Project Number 6B142
February 2011
Evaluation Objective
The main objective of this evaluation is to determine to what extent the Conservation and Protection Program (C&P) is relevant, is managed effectively and efficiently, and whether it has achieved its stated objectives. As such, the evaluation examined the extent to which C&P demonstrates value for money in its relevance and performance (including effectiveness, efficiency and economy), in accordance with Treasury Board’s 2009 Policy on Evaluation.
This evaluation, a first for C&P, covered the period from 2005/06 to 2009/10 and was undertaken between June 2010 and December 2010. The evaluation was conducted with C&P staff and others from National Headquarters (NHQ) and in Fisheries and Oceans Canada’s (DFO) six regions.
Program Description
C&P promotes and maintains compliance with legislation, regulations and fishery management measures to achieve the conservation and sustainable use of Canada’s aquatic resources, and the protection of species at risk, fish habitat and oceans.
The program is delivered through a regulatory management and enforcement approach referred to as the National Compliance Framework. This framework is comprised of four key elements or areas of activity. Three of the areas are identified as pillars:
These three pillars are supported by a fourth element, Program Capacity, which includes training and acquisition and maintenance of vessels, vehicles and equipment.
C&P is the only federal program having specific and specialized authorities (conferred on Fishery Officers) to utilize a broad spectrum of incentives and deterrents to promote, assist and compel compliance with Canada’s fisheries and oceans regulatory framework. Furthermore, Canada’s adoption of various international treaties, conventions, protocols and initiatives have resulted in responsibilities being placed on C&P for provisions related to compliance and enforcement of the standards and measures imposed by those instruments.
The program works closely with various federal, provincial and territorial and international organizations in executing its mandate. In carrying out its mandate, it also relies on the involvement of industry, recreational and Aboriginal fisher associations and groups as well as the public.
Evaluation Methodology
The evaluation used a non-experimental design and a multiple-lines-of-enquiry approach. C&P is a full coverage program, delivered in all regions across Canada, and is one that has been in place for an extended period of time. Qualitative and quantitative data for the evaluation were obtained from the following sources:
Data analysis involved reviewing and summarizing input from the various research methods and then synthesizing the information to determine where the various lines of evidence reinforce one another (triangulation), where there is notable consensus or disagreement, and what other major notable points needed to be included in the evaluation report.
Evaluation Key Findings and Recommendations
Relevance
There is a continuing need for the C&P program, with the majority of those interviewed seeing an increased need. C&P continues to support the health and safety of Canadian citizens and all who access Canada’s aquatic resources or use Canadian-sourced fishery products. Furthermore, the program is a critical link to DFO’s involvement in the Arctic and in advancing the high-priority areas of traceability and eco-certification.
There is a role for federal intervention and C&P activities are relevant and aligned with Government of Canada priorities, contributing to economic prosperity, food safety, eco-certification and national security.
The objectives of C&P are still relevant to DFO’s objectives and priorities, as they directly contribute to the departmental Strategic Outcome of Sustainable Aquatic Ecosystems and significantly support the other two departmental Strategic Outcomes of Safe and Secure Waters and Economically Prosperous Maritime Sectors and Fisheries.
Effectiveness
C&P has increased their effort in education and shared stewardship over the past five years (from 6.6% of C&P effort in 2005 to 12.8% in 2009), and while on average C&P has been moderately successful in this area, there is a need for national coordination of activities, a need to determine priority audiences, and a need to develop a method for determining if education and shared stewardship activities are producing the intended impacts.
It was also found that no formal mechanisms, such as surveys or feedback forms, exist for assessing whether the Education and Shared Stewardship activities result in a sufficiently informed public and resource user. Evidence from the evaluation notes that this is assessed based upon informal feedback. While evidence indicates that C&P key informants perceive that C&P is having considerable impact upon its audiences, there is a lack of a formal, measureable way to determine if a) the right groups are being targeted, and b) those groups are sufficiently informed.
C&P has been moderately successful in achieving compliance with management measures and in the detection and deterrence of illegal activity. However the biggest barrier achieving effective management measures has been C&P’s limited involvement in the development of, and consultation in, Integrated Fisheries Management Plans (IFMPs). This limited collaboration between Resource Management and C&P, due to time and staff constraints and the need for better communication between C&P and other DFO sectors, affects C&P’s abilities to enforce regulations and license conditions. For example, license conditions are at times drafted in such a way that they are difficult to enforce, such as with the “mortally wounded”1 clause. It is clear from the evaluation that there needs to be early and ongoing discussions within the Ecosystems and Fisheries Management Sector between C&P and Resource Management to ensure effective collaboration and communication.
Several external factors play a role in influencing the success of C&P such as the economy and market fluctuations. One barrier to C&P’s success may be C&P’s lack of designation as an “investigative body” under the Access to Information Act and the Privacy Act limiting its partnerships with other law enforcement and regulatory agencies. Because C&P is not designated as an “investigative body” under PIPEDA, other investigative bodies such as law enforcement agencies and other regulatory enforcement officers cannot share certain information with C&P easily.
Recommendation 4: It is recommended that C&P assess the pros and cons of obtaining “investigative body status” under both the Access to Information Act and the Privacy Act and look into obtaining this designation, if warranted |
C&P has had success in achieving a skilled and well-informed workforce; however a weakness in the core investigative competencies required to undertake Pillar III, Major Cases and Special Investigations, activities was identified. Pillar III has the ability to provide a more targeted and strategic approach to C&P direction. This is a relatively new area for C&P, so there is room for improvement as C&P currently lacks the structure and resources for this pillar to be effective. C&P needs to ensure that specialized training and systems are in place to move forward with this “intelligence-led” approach to enforcement, including ensuring the interoperability of systems between regions.
In addition, the evaluation found that the majority of Fishery Officers (80.1%, n=311) indicated that they find their job to be sometimes overwhelming or frequently/always overwhelming (49.2% n=191 and 30.9% n=120 respectively). The majority of comments from Fishery Officers indicated they feel overwhelmed due to newly emerging priorities undertaken by C&P that often come without any additional resources; Fishery Officers indicated this is a significant challenge to the ability to balance their day-to-day duties. It was noted that C&P is undertaking a National Expenditure and Workload Review, which may alleviate these concerns.
C&P has moderately achieved its intermediate outcome with a high compliance rate and data confirming that deterrent actions taken by C&P (specifically issuing tickets, laying charges, or issuing warnings) are on average accompanied by significant reductions in the recidivism (re-offending) rates relative to offences where these deterrent measures were not used. However, it was noted by the majority of key informants that C&P is not solely responsible for achieving this, but rather it is a combined effort amongst several key players, most notably resource management and to a lesser extent, habitat management.
Design and Delivery
The evaluation concludes that C&P’s Compliance Management Framework and its three-pillar design provide a comprehensive and balanced approach to protecting Canada’s fishery resources for the long term. However, there are concerns about the ability of C&P to ensure adequate resources to maintain, or expand upon, the current level of activities. Continual resource challenges and a salary shortfall combined with newly emerging priorities has resulted in the reallocation of resources from one pillar to support another, resulting in some difficulty in balancing all three pillars adequately.
Program delivery could be substantially improved with a stronger emphasis on national direction, consistency in approaches and systems between regions, specialization of functions and training, consolidation of small satellite offices into larger ones, and improved communication and collaboration with industry and other user groups.
Efficiency and Economy
C&P is currently under strain as minimal additional budget allocations have been provided despite additional priorities and increasing expectations which continue to be added to C&P, such as aquaculture, increasing international obligations, SARA and other initiatives. The majority of key informants felt that C&P could not conduct its business with fewer resources and still maintain the same level of quality and quantity of activities and the majority of Fishery Officers indicated that they are at least “sometimes overwhelmed" in their jobs; this may require C&P to determine a more appropriate strategic focus within the resources available and look for additional efficiencies. Even if resources are maintained at the current level, other factors such as inflation and rising fuel costs (which impacts on C&P’s use of aircraft, boats and vehicles) have an impact upon C&P’s resources. C&P must first address how it can realistically utilize existing or even fewer resources to optimally achieve the most appropriate and critical outcomes. Then it should pursue initiatives that contribute to further economies where possible.
This evaluation report presents the results of the evaluation of the Conservation and Protection program (C&P). As indentified by the 2009 Policy on Evaluation, all direct program spending must be evaluated every five years. This evaluation was slated in the 2010 – 2011 Department of Fisheries and Oceans (DFO) multi-year departmental evaluation plan and focuses on the core issues in assessing value for money: relevance and performance, including effectiveness, efficiency and economy. In addition, the evaluation examined issues of design and delivery to allow for better assessment of efficiency and economy.
The evaluation focused on the four main activities of C&P: Education and Shared Stewardship; Monitoring, Control and Surveillance (MCS); Major Cases and Special Investigations; and Program Capacity. This evaluation covers the period from 2005/06 to 2009/10. It is inclusive of the National Capital Region as well as six regional offices: Central and Arctic, Gulf, Maritimes, Quebec, Pacific, and Newfoundland and Labrador. The evaluation was conducted by DFO’s Evaluation Directorate and was undertaken between June 2009 and December 2010.
C&P promotes and maintains compliance with legislation, regulations and fishery management measures to achieve the conservation and sustainable use of Canada’s aquatic resources, and the protection of species at risk, fish habitat and oceans.
The program is delivered through a regulatory management and enforcement approach referred to as the National Compliance Framework. This framework is comprised of four key elements or areas of activity. Three of the areas are identified as pillars:
These three pillars are supported by a fourth element, Program Capacity, which includes training and acquisition and maintenance of vessels, vehicles and equipment.
C&P is the only federal program having specific and specialized authorities (conferred on Fishery Officers) to utilize a broad spectrum of incentives and deterrents to promote, assist and compel compliance with Canada’s fisheries and oceans regulatory framework. Furthermore, Canada’s adoption of various international treaties, conventions, protocols and initiatives have resulted in responsibilities being placed on C&P for provisions related to compliance and enforcement of the standards and measures imposed by those instruments.
The program works closely with various federal, provincial and territorial (FPT) and international organizations in executing its mandate. In carrying out its mandate, it also relies on the involvement of industry, recreational and Aboriginal fisher associations and groups as well as the public. Within the Department, C&P is part of the Ecosystems and Fisheries Management Sector.
C&P delivers its programs through a regulatory management and enforcement approach defined in what they refer to as the National Compliance Framework. This policy framework guides the application of compliance tools organized in C&P’s compliance management model, which contains the four elements (referred to as “Pillars”), mentioned in 2.1. These four elements are the major activities outlined in C&P’s Logic Model (Annex A).
Pillar I: Education and Shared Stewardship
The Education and Shared Stewardship activity consists of a suite of activities for promoting compliance, through strategies such as education, promotional campaigns, and engagement of partners and stakeholders. Educational activities are intended to raise awareness and understanding resulting in a more informed public and resource users, improving their ability to comply with regulatory requirements.
Pillar II: Monitoring, Control and Surveillance
The MCS activity is comprised of a number of traditional compliance and enforcement activities aimed at detecting and deterring illegal activities to provide an oversight function to determine compliance with the legislation, regulations and management measures in effect. To deter illegal activities, enforcement actions are carried out, and include warnings, seizures, arrests, directions, orders, diversions from the courts to community-based justice, charges and prosecutions. C&P also uses ticketing to a limited extent, for example the Maritimes and Gulf Regions use ticketing for the Maritime Provinces Fishery Regulations, and in the Pacific Region mostly for recreational fisheries. The Pacific Region has been increasing the use of community-based justice (CBJ) as an alternative to traditional court prosecutions in situations where specific criteria are met.
Pillar III:Major Cases and Special Investigations
The Major Cases and Special Investigations activity focuses attention on solving high-risk complex compliance issues that pose significant threat to sustainability of Canada’s aquatic resources and cannot be addressed through education or regular MCS activities. Special investigative techniques, including covert operations, technical surveillance, use of search warrants and IT forensics, are applied to the unique difficulties encountered in addressing illegal fishing and other activities. Formal intelligence gathering and analysis, inter-agency cooperation and networking, and alignment and integration of data systems with visual analytics tools permit more accurate identification of enforcement targets and enable the building of major case files for successful prosecutions.
Program Capacity
Program Capacity involves the development and support of a skilled, equipped, well-informed, safe and effective workforce, including the recruitment of Fishery Officers through the Fishery Officer Career Progression Program (FOCPP). A strong legal and policy framework including Standards and a Code of Conduct, as well as systems for information collection and analysis, is also an important component of the support structure. Acquisition and management of equipment including radio communications, vehicles, and vessels is necessary to ensure a well-equipped and effective workforce. Finally, strategic planning and integrated risk assessments are carried out to identify operational priorities and to ensure the right balance of tools and approaches are used to achieve the program objectives.
C&P's annual budget for NHQ and all regions totals approximately $120M, with annual expenditures ranging from $100 to $130 million. At approximately $52M annually, salaries for C&P staff represent the single biggest expenditure item for C&P nationally. National support programs to aid in delivery of C&P's MCS programs account for a large portion of the remaining funding. For example, C&P expends approximately $24M annually ($16M salary/$8M OM) for Canadian Coast Guard (CCG) vessel support, $14M for the Air Surveillance Program, $3M for the At-Sea Observer Program, $3M for radio/communications capacity, $2M recruitment/training/uniforms, and $1M for vehicles. The remainder is expended on a variety of regional operational expenses ranging from small equipment procurement to prosecution/witness expenses to travel and living costs, as well as facility, equipment and system maintenance costs. The C&P annual budget from 2005/06 to 2009/10 are shown below in Table 1.
Table 1. Conservation and Protection Total Program Cost
| Annual Budgets & Expenditures | ||||||||||
|
2005-06 |
2006-07 |
2007-08 |
2008-09 |
2009-10 |
|||||
Planned |
Actual |
Planned |
Actual |
Planned |
Actual |
Planned |
Actual |
Planned |
Actual |
|
O&M |
39,413.3 |
43,808.1 |
48,801.1 |
44,834.4 |
37,943.2 |
48,268.2 |
42,641.5 |
50,021.4 |
45,761.2 |
44,769.7 |
Salary |
52,292.3 |
51,782.2 |
64,212.1 |
51,368.1 |
53,175.3 |
49,642.6 |
53,064.6 |
48,726.4 |
53,397.9 |
57,582.2 |
G&C |
0.0 |
2.0 |
0.0 |
42.5 |
0.0 |
22.5 |
0.0 |
33.4 |
0.0 |
37.5 |
Capital (major & minor) |
0.0 |
1,631.1 |
0.0 |
2,965.1 |
0.0 |
3,098.3 |
1,000.0 |
2,201.2 |
1,000.0 |
1,734.7 |
Other (e.g., revenues) |
0.0 |
24,085.3 |
0.0 |
14,705.8 |
27,996.0 |
30,037.0 |
27,104.0 |
31,125.0 |
28,400.0 |
29,600.0 |
TOTAL |
91,705.6 |
121,308.7 |
113,013.2 |
113,915.9 |
119,114.5 |
131,068.6 |
123,810.1 |
132,107.4 |
128,559.1 |
133,724.1 |
Table 2 below outlines C&P’s total Full Time Equivalent (FTE) utilization from 2005/06 to 2009/10.
Table 2. C&P Program FTE Utlization3
Region |
2005-2006 |
2006-2007 |
2007-2008 |
2008-2009 |
2009-2010 |
FTE |
FTE |
FTE |
FTE |
FTE |
|
Newfoundland |
151.73 |
161.24 |
163.51 |
161.49 |
156.59 |
Maritimes |
163.92 |
155.62 |
162.28 |
158.38 |
157.90 |
Quebec |
68.12 |
56.89 |
70.85 |
70.82 |
70.12 |
Central & Arctic |
48.28 |
31.70 |
26.46 |
32.12 |
32.08 |
Pacific |
166.84 |
151.82 |
172.28 |
170.86 |
172.34 |
Gulf |
121.52 |
126.72 |
130.44 |
130.73 |
131.19 |
NCR |
14.77 |
19.76 |
18.88 |
19.05 |
24.07 |
All Responsibility Centres |
735.17 |
703.74 |
744.70 |
743.46 |
744.30 |
The Logic Model presented below was used for the purpose of this evaluation. It presents program inputs, outputs and intended outcomes.
This section outlines the scope and methods of the evaluation approach including the evaluation design and questions, and the data sources.
The evaluation questions covered both relevance and performance (including effectiveness, efficiency and economy), as well as design and delivery, and can be found in Annex A. The C&P evaluation matrix (Annex B) includes the use of multiple lines of evidence and complementary research methods as a means to ensure the reliability of information and data to be collected.
The document review consisted of the analysis of documents provided by the program and included both public and internal program documents (for example annual reports, studies/previous evaluations and audits, Speeches from the Throne).
The literature review explored the key literature that speaks to the impact of deterrent tactics by describing the theory behind deterrence, and its applicability and usefulness in the area of compliance and enforcement of fishery legislation and regulations.
The Department has a number of National Enforcement databases housed by C&P that were used to inform the evaluation. Two main databases were utilized for analysis: (1) the Fishery Officer Enforcement Activity Tracking System (FEATS) which allows Fishery Officers to record their activities and results; and, (2) the Departmental Violations Systems (DVS) that allows Fishery Officers to record fisheries and habitat enforcement non-compliance data.
Both DVS and FEATS data was used to examine the factors that can best explain the variance in the yearly number of reported violations. A set of 17 variables were chosen in order to test their predictive ability with regard to the number of violations recorded. The data was analyzed to assess which variable(s) have the largest impact on the number of reported violations.
Two studies of recidivism using C&P’s DVS4 were undertaken to support the evaluation. The objective of the analysis was to measure the effect of actions undertaken to deter further offences against the legislation, regulations and management measures enforced by C&P. Two types of deterrents actions were explored:
The explanatory (or control) variables used in the analysis were: deterrent actions, violation date, number of violations per incident, number of previous offences, age of the offender, Region of the offence, violation types, decision types, probation days, fines, jail sentence, and fishery.
A scan of all ministerial correspondence where C&P had input also occurred. Correspondence that involved the C&P program was identified using the Department’s Ministerial and Executive Correspondence Tracking System (MECTS).
A scan of the Canadian media, in French and English, was also undertaken to inform the evaluation based on a list of key words related to conservation and enforcement, fishery violations and fishery officers. The media scan was limited to the last five years.
The comparative analyses focused on the program delivery of two other federal organizations and two provincial ministries, specifically the range of services they provide and their organizational structures. In addition, a comparative analysis of similar programs in other countries was conducted, including the United States, Australia and New Zealand, and Norway.
The evaluation team conducted 61 interviews with key informants. The 61 key informants interviewed included C&P managers at both National Headquarters (NHQ) and in the regions, co-delivery partners, other government departments, and fishers associations. Key informants were categorized as one of three groups: C&P Management (n=21); External Stakeholders (n=13); and Other DFO and Federal, Provincial, and Territorial (FPT) Partners (n=25). The interview guides used several five-point likert-scale questions5, which were used throughout to average results of key informants. These scales were used across all three interview guides to allow for the comparison of average responses from the three targeted groups of key informants.
An online survey was administered to all Fishery Officers employed by DFO. The survey was launched in October 2010 and closed in November 2010. A total of 411 of the full complement of approximately 650 Fishery Officers responded to the survey, for a response rate of 63.2%. The Pacific and Maritimes Regions had the largest number of respondents, with 26.3% and 24.8% respectively (n=108, n=102).
There were two case studies undertaken for the C&P evaluation, described below:
1. Catch Monitoring: The case study explored the use of various catch monitoring tools in the management of the groundfish fisheries, and identifies others currently not used. While the case study collected information on the use and involvement of other programs/sectors, the scope is limited to catch monitoring activities as they relate to C&P activities. The case study used the following methodology: Seventeen key informant interviews with DFO in NHQ and in three regions and service providers for the At-Sea Observer Program, Electronic Monitoring, and Dockside Monitoring Program, document and internet review.
2. CBJ in the Pacific Region: The C&P program, led by the Pacific Region, is using CBJ as an alternative to the traditional legal processes for dealing with violations. This initiative has been underway since approximately 2003 and the Pacific Region now uses CBJ in all its Divisions. The program is offered to all individuals or organizations that may experience violations or conflicts involving Canada’s fisheries resources and fish habitat, where applicable. The case study used the following methodology: Sixteen key informant interviews with DFO staff including Fishery Officers, victims, offenders, and the Department of Justice, document review, literature review, and data from the DVS.
Relevance Question #1: Is there a continued need for C&P?
It was found that traditional fisheries and fish habitat remain the core activities of DFO as well as the work of C&P Fishery Officers. This includes the continuing need for monitoring and enforcement activities in the areas of commercial and recreational fisheries, species at risk, aquaculture and foreign fishing. There is also a continuing need for C&P involvement in pre-season license condition reviews and post-season reviews of management plans (fisheries) and enforcement measures. Further, the need for C&P is increasing with the possibility of increased fishing and Marine Protected Area’s (MPAs) in the Arctic, increased shipping through the Arctic, and DFO’s new role in managing aquaculture on the West Coast. In addition, C&P plays a critical role in the priority issues of ‘traceability’6 and ‘eco-certification’7.
Relevance Question #2: Is there a role for federal government intervention, and is C&P aligned with Government of Canada priorities?
C&P is aligned with and contributes to the core government priority of economic prosperity. The priority issues of traceability and eco-certification will require the use of C&P resources over the coming years and will contribute to strengthening Canada’s food safety system. C&P supports a stronger economy by ensuring compliance in the fisheries, which is essential for sustainability and long-term economic prosperity for the fisheries sector and many coastal communities. C&P contributes to DFO and broader governmental priority of national security through its participation in Marine Security Operation Centre’s (MSOC’s) and its aerial surveillance program.
The 2010 Speech From the Throne (SFT) discussed specific steps that the Government will take to safeguard Canada’s national security. C&P contributes to DFO’s and the broader governmental priority of national security through its aerial surveillance program. In addition, C&P is a full partner in the MSOC’s, a national security initiative led by DND. C&P staff are co-located in the MSOC’s on the East and West coasts forming an integral part of MSOC teams to meet the GOC objectives for improved information-sharing to integrate intelligence for national security. C&P’s aerial surveillance, which focuses on monitoring fisheries activity, contributes to the operations of the MSOC’s by providing information from its aerial surveillance program and feeding into the Recognized Maritime Picture produced by the DND.
From Tackling Crime (SFT 2006) to Keeping Canadians Safe (SFT 2008), law and order is a priority of the current government. Fishery Officers are armed Peace Officers8 and, in many outlying coastal communities, remain the most visible federal presence. C&P has also made improvements in alignment to Government of Canada priorities through leveraged opportunities with the Royal Canadian Mounted Police (RCMP) through the secondment of RCMP liaison officers to work with C&P staff in DFO regional offices on sensitive files and to share knowledge and expertise. In addition, the two organizations are working together to improve data and intelligence sharing and interoperability between organizations of peace officers9.
Relevance Question #3: Are the objectives of C&P still relevant to DFO objectives and Priorities?
The C&P program directly contributes to the departmental Strategic Outcome of Sustainable Aquatic Ecosystems by assisting in fulfilling DFO`s role in regulatory programs such as the Canadian Shellfish Sanitation Program (CSSP), Fish Habitat Management, Oceans Management, and Species at Risk. C&P provides a significant supporting contribution to the Strategic Outcome of Safe and Secure Waters through its aerial surveillance program and its participation in the MSOCs on the East and West coasts and the sharing of information that contributes to Department of National Defence’s (DND) Recognized Maritime Picture. C&P also provides a significant supporting contribution to the Strategic Outcome of Economically Prosperous Maritime Sectors and Fisheries by overseeing regulatory compliance and enforcement in commercial, recreational and Aboriginal fisheries and in aquaculture. C&P also contributes to International Obligations in support of DFO’s objectives.
Effectiveness Question #1: To what extent has C&P achieved its immediate outcomes?
The evaluation answered this overall question by each immediate outcome outlined below.
Effectiveness Question #1a: To what extent have C&P activities resulted in an informed public and resource user?
Over the past five years, C&P’s efforts on Pillar I have almost doubled from 6.6% to 12.8% of total effort, exceeding C&P’s target of 10% for 2009. The comparative analysis of four other countries with mandates similar to C&P also indicated that they are increasing their “outreach” activities in an attempt to obtain compliance for fishery regulations by educating industry and the public on their obligations under regulation.
The evaluation data (Figure 1) showed that, on average, C&P management and Fishery Officers are of the opinion that C&P has been successful in achieving an informed public and resource user, while external stakeholders indicated that C&P has had less success in this area. However, external stakeholders also indicated that C&P had been moderately successful in promoting compliance within their community.
Figure 1. How successful do you think C&P has been over the past five years in
achieving an informed public and resource users (average response on a scale of 1 to 5)?

Source: Key informant Interviews & Fishery Officer Survey
C&P management and Fishery Officers indicated that C&P has been able to reach its intended audiences to a moderate extent. However, many key informants noted that C&P may not be targeting the right audience, requiring more direct and strategic contact with industry. More specifically, it was noted that a national approach to education and shared stewardship activities, with national leadership, is required. The CBJ case study found that outreach activities that occur in the Pacific Region have the ability to improve the effectiveness of education and shared stewardship through improving the relationships between Fishery Officers and the communities within which they enforce. That is, the ability to use CBJ has created an environment that allows Fishery Officers to become “problem solvers”, and has been beneficial to create public respect and awareness.
No formal mechanisms for assessing whether the education and shared stewardship activities result in sufficiently informed public and resource users were found. Both key informants and Fishery Officers expressed that this is determined based upon feedback they receive through the consultative process with industry, such as roundtables and advisory committee meetings, or by assessing the level of industry or public knowledge through direct interaction. However, the opinions of C&P management and of Fishery Officers was that, on average, the education and shared stewardship activities were having moderate to considerable impact on their audiences.
Finally, several suggestions were offered to improve the outcomes of Pillar I activity, including a national strategy and coordination of education materials, offering up-to-date communication materials and products, re-profiling to better target intended audiences, using performance reports with DVS and FEATS data for industry, improving communications within DFO and externally, increasing buy-in to support Pillar 1 activities, and building partnerships within and external to DFO.
Effectiveness Question #1b: To what extent has C&P achieved effective management measures built on common goals?
C&P’s ability to achieve effective fishery management measures (e.g. quotas, openings, closings) is strongly linked to its MCS activities, and its success in detecting and deterring illegal activity. As shown below, about half of C&P’s effort is devoted to MCS functions, and between 2005 and 2009 about one-third of MCS activities involved patrols. However, there is a slight downward trend in the total time of MCS activities, including patrols, as effort is invested elsewhere.
Figure 2. C&P Effort Allocated to Monitoring, Control and Surveillance (2005 – 2009)
Source: C&P Program National Outlook 2009
The evaluation found that key informants and Fishery Officers indicate, on average, that C&P has been moderately successful in detecting and deterring illegal activities. Many key informants noted that both the detection and deterrence of illegal activities is strongly linked to the visible presence of Fishery Officers. As shown below in Figure 3, C&P efforts in patrols have experienced a slight downward trend that mimics the overall downward trend in MCS activities. Figure 4 indicates that the number of checks by C&P on vessels and sites has remained relatively stable over the last five years; only a slight downward trend in the number of checks of persons and gear has been captured in FEATS. Fishery Officers added, however, that the various surveillance tools, equipment and data available, such a Vessel Monitoring Systems (VMS), aerial patrols, and other electronic systems allow the monitoring of vessels on a day-to-day basis. The catch monitoring case study found that C&P has continued to develop and implement new and innovative ways of achieving the monitoring and surveillance requirements. Key informants suggested that if the decreasing trend in Fishery Officer patrols continues, C&P will need to be more coordinated and targeted in its patrols.
Figure 3. Fishery Officer Patrol Hours Compared to Total Hours of MCS activities
(2001 – 2010)

Source: FEATS
Figure 4. Number of Vessels, Persons, Gear and Sites Checked by C&P
(2005 – 2009)

Source: FEATS
The catch monitoring case study further concluded that catch monitoring tools, though not exclusively C&P tools, are considered to contribute to achieving effective compliance incentives and deterrents. Activities such as the at sea observer program impacts the behaviour of fishermen and contribute to compliance in the fisheries. Even in fisheries where there is not 100% coverage of either the at-sea or dockside monitoring program, there is always the possibility that the vessel may be subject to third party verification of its catch.
C&P management and Fishery Officers indicated that C&P has been slightly more than moderately successful in achieving compliance with management measures. A number of issues were raised that impact C&P’s ability to achieve compliance with these measures. The data from key informants interviews indicated that C&P’s involvement is sometimes limited by the quality of data it can bring to the discussions, particularly if it wants to change any of the management measures, and that there needs to be early and ongoing discussions and consultations between C&P and the Resource Management program to ensure that license conditions are in line with legislation and enforceable.
Effectiveness Question #1c: To what extent has C&P achieved effective compliance incentives and deterrents?
Over the last five years C&P has reported a compliance rate10 of approximately 90% (Figure 5). However, given that this ratio could be misleading, the evaluation used C&P FEATS and DVS data to examine the factors that could best explain the variance in the yearly number of reported violations11. The result was that the largest marginal impact on the number of reported violations, when controlling for other variables and regions, is due to the activities of Pillar III. However, C&P management and external stakeholders indicated that C&P has had, on average, little to moderate success in reducing hidden or major illegal activity, while Fishery Officers indicated that C&P has had moderate success in this area.
Figure 5. C&P’s National Compliance Rate (2005 – 2009)

Source: C&P Program National Outlook 2009
While the majority of key informants and Fishery Officers were optimistic about C&P’s ability to improve in this area, given this work is just in its early stages, it was stated that C&P currently has little structure and few resources devoted to this Pillar. They noted that specialized training and information systems are required to undertake what many are referring to as “intelligence-led policing”. It was further argued that a national approach to Pillar III, training and specialized software and equipment are required to ensure interoperability between regions. As noted through the document review, two national systems development workshops were undertaken by C&P. An intelligence software tool and a major case management tool have been chosen and are currently being implemented in two Regions. C&P is intending to ensure interoperability internally between all regions and with external partners. The new tools will be interfaced with existing DFO data systems to increase effectiveness and reduce data entry errors.
Overall, C&P management key informants and Fishery Officers indicated that the courts have a moderate to considerable impact on C&P’s efforts to deter illegal activities. While C&P has a high rate of conviction (in 2009, 80% of their prosecutions resulted in convictions), the feeling is that penalties are too lenient, often inconsistent or are not commensurate with the gains that can be achieved from illegal fishing. The literature also supports that courts are not willing to apply sanctions perceived as excessively severe.
Effectiveness Question #1d: To what extent does C&P have a skilled, equipped and well-informed workforce?
From 2004 to 2006 there was a decline in Fishery Officer recruitment, resulting in many vacant positions in the program. This was a result of a hiring freeze; however, recent boosts to recruitment activity have begun to resolve the impacts of the hiring freeze and are helping to stabilize the workforce with current vacancy rates now comparable with DFO as a whole.
Overall, the C&P management key informants indicated that C&P has been successful in achieving a skilled and well-informed workforce, while external stakeholders provided a slightly lower assessment indicating that, on average, C&P has been moderately successful in this area. When asked about the FOCPP training, the majority of Fishery Officers rated this program as good or excellent. In general, key informants believe that Fishery Officers are highly-skilled, dedicated individuals and that the training they are provided through the FOCPP is of excellent quality. It was noted by key informants that the lack of a strategic, well-defined training plan beyond the initial training is a weakness. However, when asked to rate their ongoing training, the majority of Fishery Officers (61.1%, n=248) rated it good or excellent (Figure 6).
Figure 6. How would rate C&P’s ongoing training (n=406)?

Source: Fishery Officer Survey
Fishery Officers were also surveyed with respect to the demands of their job. As indicated below in Figure 7, the majority of Fishery Officers (80.1%, n=311) indicated that they find their job to be sometimes overwhelming or frequently/always overwhelming (49.2% n=191 and 30.9% n=120 respectively).
Figure 7. Demands of Fishery Officer Job (n=388)
Source: Fishery Officer Survey
The majority of comments from Fishery Officers indicated they feel overwhelmed due to newly emerging priorities undertaken by C&P that often come without any additional resources; Fishery Officers indicated this is a significant challenge to the ability to balance their day-to-day duties. C&P is currently working on a National Expenditure and Workload Review intended to inform decisions on resource allocation for a more sustainable national program.
A weakness in the training available for skills required to undertake major cases and special investigations was identified by many key informants. The majority of key informants and Fishery Officers indicated that more specialized training in the area of forensic analysis, perhaps for select individuals, needs to be provided. Fishery Officers further indicated that they lacked the “core investigative competencies”, such as undertaking forensic analysis and giving court testimony, which are being increasingly needed in their job.
C&P management and external stakeholders indicated, on average, that C&P has been successful in achieving a well-equipped workforce (figure 8). The majority of key informants and Fishery Officers indicated that currently C&P is very well equipped, although once again noting a concern in the area of mid-shore vessels. However, the majority of key informants and Fishery Officers noted that if the current budget pattern continues, and with the impact of inflation, vehicles and vessels will become problematic as the current funds may not allow for their efficient replacement.
Figure 8. How successful do you think C&P has been in achieving a well-equipped
workforce (average response on a scale of 1 to 5)?
Source: Key informant Interviews & Fishery Officer Survey
Several suggestions for improvements with respect to C&P having a skilled, well-informed and well-equipped workforce were provided from both key informants and Fishery Officers. Of these the most notable were: increased training for the skills required under Pillar III, maintaining high educational standards for Fishery Officers, increasing communication between C&P and other sectors within DFO, such as Resource Management, and the consideration of the use of web-based training.
Effectiveness Question #2: To what extent has C&P achieved its Intermediate outcome: Compliance with legislation and regulations and management measures?
Key informants, on average, consider C&P to have reached its intermediate outcome and achieved compliance with legislation, regulations and management measures from a moderate to considerable extent (Figure 9). The majority of key informants cite C&P’s high compliance rate as support for their opinion that C&P has been, at least, moderately successful in this area. However, it was felt that C&P is not solely responsible for achieving this, but rather it is a combined effort amongst several key players, most notably resource management and to a lesser extent, habitat management.
Figure 9. To what extent has C&P resulted in compliance with legislation and regulations
and management measures (average response on a scale of 1 to 5)?

Source: Key informant Interviews & Fishery Officer Survey
The literature also identifies further factors that determine compliance: potential illegal gain, severity and certainty of sanctions, individuals’ moral development and their standards of personal morality, individuals’ perceptions of how just and moral are rules being enforced, and social environmental influences.12 Ultimately, evidence from the literature suggests that moral development, perceived legitimacy of enforcement, and the behaviour of others are important determinants of compliance.13
The statistical analysis of DVS data indicated confirmation that deterrent actions taken by C&P (specifically issuing tickets, laying charges, or issuing warnings) are on average accompanied by significant reductions in the recidivism (re-offending) rates relative to offences where these deterrent measures were not used. Three deterrent actions were examined for the effect upon rates of recidivism: the use of warnings, the use of tickets, and the use of charges. The model predicts final recidivism rate of 26.3% among those receiving warnings, 5.7% among those who receive tickets, and 25.7% among those against whom charges were laid.
The results of the recidivism analysis indicate that issuing warnings is accompanied by a reduction in the rate of recidivism when compared to those not receiving a warning. The rate of recidivism among those not receiving a warning was predicted to be 21.9%. However, when “warning issued” is substituted for other deterrent actions in the control group, the model changed its prediction from 21.9% to 19.0%, which suggests that replacing other deterrent actions with “warning issued” will in general result in a reduction of recidivism rates. Offenders who received tickets had a relatively low rate of recidivism (5.7%). However, when the model substituted “warning issued” for “ticket issued”, the model predicts an increase to 10%, suggesting that warnings are much less effective than tickets. Offenders against whom charges were laid had a very similar recidivism rate compared to those receiving warnings (25.6% versus 26.5%). Where warnings were substituted for laying charges in this group, the rate of recidivism would increase slightly to 26.1%.
Effectiveness Question # 3: Are there any unintended outcomes, positive or negative, that can be attributed to C&P? If so, were any actions taken as a result of these?
Slightly more than half of the key informants felt they were unable to answer this question citing that many outcomes of their activities had been planned to occur over the last five years. Of those that were able to respond, the most common finding is that C&P Fishery Officers are well equipped and well trained to support other organizations, such as the Royal Canadian Mounted Police (RCMP) and the Canadian Coast Guard (CCG), in search and rescue operations, and other emergency situations such as evacuations due to fire. They have been called upon to support other agencies and have often been a key player, offering up their useful skills and extended knowledge of local waterways.
Fishery Officers are the most visible aspect of DFO and in some cases the only federal presence in certain communities. If there is a lack of industry and public support for some compliance decisions, this can place Fishery Officers in a difficult position as the front line of DFO. That is, in cases where this occurs (the lack of support for compliance decisions) engaging in enforcement may cause conflict between DFO and its clients.
Effectiveness Question #4: Are there any external factors and/or general challenges/barriers that influence the success of C&P?
Several external factors or general barriers were thought to influence the success of C&P. Key informants identified the following:
1. Economy and market influences
When the economic conditions are poor and/or market values for resources plummet reducing prices for their catch, fishers are placed in a situation where the motives for non-compliance are high, influencing the success of C&P. On the other hand, as was experienced in British Columbia recently with salmon, a surge in a species can result in unanticipated requirements for Fishery Officer presence and enforcement14. C&P needs to take into account the current economy and markets as they relate to fisheries resources.
2. Designation as an “investigative body” under PIPEDA
Because C&P is not designated as an “investigative body” under the Access to Information Act and the Privacy Act, other investigative bodies such as law enforcement agencies (e.g. RCMP) and other regulatory enforcement officers (e.g. conservation officers) cannot share certain information with C&P easily. This designationpermits investigative bodies to disclose personal information for the purpose of enforcing any law of Canada or a province or carrying out a lawful investigation15 As stated above, this can limit C&P’s access to information and therefore may be a barrier to its success in MCS activities, but more importantly in Pillar III and with any work that involves joint partnerships with those agencies that have the “investigative body” status.
3. Resource users’ willingness to comply with regulations
Several key informants indicated that C&P’s success hinges on resource users’ willingness to comply with legislations and regulations. That is, where there is industry and public support for compliance and enforcement decisions. This is further supported by the literature that indicates that the perception of both how just and moral the rules of enforcement are16 and the perceived legitimacy of enforcement17 are key elements that affect compliance.
4. Constrained by the Fisheries Act
Finally, the majority of key informants felt that C&P enforcement was constrained by the current Fisheries Act. They felt limited in the ability to undertake the clear enforcement action they would like to as a result of the way the Act is written. That is, C&P is often limited to prosecution as the only avenue for enforcement, and cannot utilize enforcement actions such applying administrative sanctions18 (i.e. license suspension or quota reduction), that key informants feel would be more meaningful, because the current legislation does not allow it.
Design and Delivery Question #1: To what extent are the current program design and delivery model and governance structure effective to support the achievement of results?
The evaluation answered this overall question using the following four sub-questions.
Design and Delivery Question #1a: What are the primary challenges with the current program design?
A few challenges with respect to the program’s current design emerged from the key informant interviews and the Fishery Officer survey. The most notable comment from the majority of key informants and Fishery Officers was that resources and staffing issues were the biggest challenge. That is, they felt that with only minimal budget increases, despite a growing demand for C&P support, the result has been the reallocation of resources from one pillar to another in order to fund emerging priorities, making it difficult to balance support to all three pillars adequately, with Pillar III (major cases and special investigations) being the most noted pillar.
In a review of ministerial correspondence received from 2005 to 2010, 62% of this correspondence was from the public, as opposed to a political figure (20%), an association (11%) or an NGO (7%). The majority of correspondence concerned Fishery Officers and enforcement, including, but not limited to, concern over the reduction in the number of Fishery Officers, concern about the relocation of Fishery Officers, and concern over lack of adequate enforcement or requesting more Fishery Officers.
Design and Delivery Question #1b: What are the primary benefits of the current program design?
Generally, the design of the C&P program was considered, on average, to be better than moderate in terms of both effectiveness and efficiency (Figure 10 and Figure 11). When asked about the effectiveness of C&P’s program design, C&P management, other DFO & FPT partners, and Fishery Officers responded, on average, with moderately effective to effective. External stakeholders responded, on average, with a slightly lower assessment of moderately effective.
Figure 10. How effective is C&P’s program design (average response on a scale of 1 to 5)?
Source: Key informant Interviews & Fishery Officer Survey
In terms of efficiency, the program design on balance is considered to be moderately efficient or better. C&P management responded, on average, that the program design is moderately efficient to efficient, while other DFO and FPT partners rated the program as efficient. External stakeholders on the other hand, on average, rated it slightly less than moderately efficient. A few key informants noted the introduction of line reporting as working well and being a contributing factor the programs efficiency.
Figure 11. How efficient is C&P’s program design (average response on a scale of 1 to 5)?
Source: Key informant Interviews
The majority of key informants and Fishery Officers that provided further comments, indicated that the three pillars complement each other by providing a comprehensive approach to C&P’s work where all three pillars are needed to achieve success. That is, the design is helpful in understanding the breadth of C&P’s program, as it describes the range of work required and provides a balanced approach. The pillars provide a flexible design that contributes to work planning and focusing on priorities. Though some key informants feel the design is hampered by lack of sufficient resources.
Design and Delivery Question #1c: Would an alternative program design or delivery mechanism be more appropriate to achieve expected results?
Environment Canada, Parks Canada, Ontario Ministry of Natural Resources (OMNR) and British Columbia Ministry of Environment have similar compliance and enforcement functions as C&P. Three of them differ notably from C&P in that they have specialized functions and units. Fishery Officers, on the other hand, have been trained as generalists. As well, EC enforcement officers are able to issues tickets for violations, whereas C&P’s ability to ticket is limited. OMNR is moving towards a formalized risk-based approach to compliance, which may be of relevance for C&P.
A comparative analysis was also done on the compliance and enforcement activities of four foreign governments having similar mandates as C&P: The United States, Norway, New Zealand and Australia. It was found that within these countries there has been a move toward achieving compliance with increased education and outreach, as opposed to pure enforcement. For example, the United States’ National Oceans and Atmospheric Administration (NOAA) has created a program designed to help achieve deterrence through an outreach program designed to empower stakeholders and the public to be involved in local marine conservation management. New Zealand’s Ministry of Fisheries has also recently changed its compliance approach from purely enforcement to include an “informed and assisted” focus to ensure fishers are aware of fishery rules and their obligations to abide by those rules.
The majority of key informants indicated that effort in three areas would contribute to improving program delivery. First, a national plan, greater consistency and more collaboration within C&P and with other DFO sectors are required. Secondly, resource allocation and training requires more attention. Thirdly, communications and collaboration with external stakeholders, the fishers, could be improved.
Design and Delivery Question #1d: To what extent were activities modified, and what was the effect on program design?
The move to special investigations and intelligence-led enforcement followed by line reporting were the most frequently mentioned changes introduced in the last five years. The introduction of line reporting was mentioned throughout the evaluation as an improvement to C&P, with the exception that C&P Directors do not have sole control over their budgets19.
Design and Delivery Question #2: Do the activities of the C&P program complement, overlap with or duplicate other DFO programs or those of other FPT government departments?
There are areas of overlap with provincial or territorial and other federal authorities, but these are largely complementary. Better collaboration was identified as being needed in some instances, notably with Environment Canada (EC). A few key informants and several Fishery Officers indicated that C&P officers should be given broader authority to enforce federal regulations beyond DFO’s jurisdiction in the interest of ensuring the public is well served with respect to safety issues, such as the ability to enforce Small Vessel Regulations and designation with “special constable status”20, however these designations would have both resource and legal implications. Organizations such as The Canadian Chamber of Commerce are observant of potential areas of overlap and redundancies, and they had occasion to recommend as part of its 2009 Policy Resolutions that DFO should be “the one federal agency in charge of the stewardship and sustainable management of the marine environment in Canada.”21
Design and Delivery Question #3: Is there a Performance Measurement (PM) Strategy and a reporting process/system in place? If yes, is it being used for decision making?
C&P did not have a complete PM Strategy or Results-based Management Accountability Framework (RMAF) at the time of this evaluation. However, the program does have expected results outlined in the 2010-11 Management, Resources, and Results Structure (MRRS) that were utilized. As well, the program submitted a draft PM Strategy in December 2010 under the Department’s Performance Measurement Action Plan (PMAP).
As discussed in section 3.2.3, C&P has two key National Enforcement Databases, FEATS and DVS. The data has been used for decision making, measuring program performance, risk assessment, planning, budgeting, reporting on work plans and priorities and sharing with industry and others what is being done. While key informants consider the information to be valuable, several shortcomings were noted, including: lack of Fishery Officer access to data when out of the office, inconsistencies in data entry, staff frustrations with using the systems, and limited resourcing to support the management of C&P databases. System updates are scheduled to be introduced in 2011.
Design and Delivery Question #4: Are there best practices and lessons learned from C&P?
Various operational aspects, the level of professionalism within the Fishery Officer cadre and C&P’s Code of Conduct were mentioned by several respondents as C&P’s most significant best practices. Fishery Officers have earned a high level of respect with some external partners for doing less routine work well and for their dedication in assisting in emergencies. C&P’s work planning process and reporting systems were also acknowledged, as key informants felt these enable them to document what they are doing and identify priorities and related costs, all of which contributes to improving accountability. Many respondents cited recruiting and training, particularly the annual requalification, as a best practice, as well as the working relationships and presence they have established with other agencies, fishers and the public.
Efficiency and Economy Question #1: Are the appropriate administrative and management systems in place for efficient C&P implementation and management?
C&P`s administrative and management systems require some improvement with key informants, on average, finding that C&P’s administrative and management systems are only moderately operating efficiently. It was felt that National policies need to be developed, however with only a very small portion of C&P’s staff working within NHQ (approximately 20 in the NHQ, versus approximately 700 in the Regions), the ability to do this may be limited. C&P is seen by some to have achieved management efficiencies and greater accountability with its line reporting, many key informants noted a lack of administrative support as being problematic.
Roles and responsibilities with partners are considered clearly defined to a considerable extent by key informants and Fishery Officers. Some relationships are considered fairly well defined, where there are MOUs and close working relationships have evolved with other agencies and partners, such as industry. The main area lacking clarity appears to be around habitat-related work.
It was noted that C&P, through its various databases, is “data rich”, and that this data has been useful for planning but it is currently difficult for C&P Management to extract data easily, and there are questions as to whether the systems are capturing the most useful information (i.e. outputs v. outcomes). As well, as mentioned earlier, consistent data entry, through increased training, would be beneficial.
Efficiency and Economy Question #2: Did the C&P resource utilization and activities optimally produce expected levels of outputs? How could the efficiency of C&P activities be improved?
Key informant data and results from the survey of Fishery Officers offered insights to C&P`s use of resources and current limitations. In addition to a lack of resources (funding, Fishery Officers and clerical support) preventing optimal output, several comments were made about the deployment of existing resources. Officers should be located where they are needed most, where resource users are located and harvesting is active year round. It was felt that further efficiencies could be made by consolidating/closing some of the smaller field offices and it may be more efficient to have certain officers dedicated to specific priorities, such as hiring investigators who are already proficient in forensic investigations.
C&P management and Fishery Officers noted that they have corporate and administrative responsibilities that take up significant amounts of time. They further indicated that additional administrative support would be beneficial in this area.
Efficiency and Economy Question #3: Is C&P operating in a way that minimizes the use of resources to achieve its intended outcomes?
The majority of key informants indicated that the resources used for C&P are currently under strain as minimal additional budget allocations have been provided despite additional priorities and increasing expectations which continue to be added to C&P, such as aquaculture, increasing international obligations, SARA and other initiatives. The majority of key informants (C&P management n=18, 88.9% and external stakeholders n=11, 91%) indicated that C&P could not conduct its business with fewer resources and still maintain the same level of quality and quantity. Even if resources are maintained at the current level, other factors such as inflation and rising fuel costs (which impacts on C&P’s use of aircraft, boats and vehicles) have an impact upon C&P’s resources. As well, more responsibilities and expectations are added which further stretch existing resources.
Key informants and Fishery Officers stated that there are not a lot of activities that could be reduced, stopped or given to others, such as industry or provincial and territorial governments, while maintaining current expectations about C&P`s mandate and delivery. Suggestions were forthcoming on areas where C&P might achieve efficiencies, which was mainly reducing the number of small C&P offices, and consolidating them into larger offices.
The evaluation concludes that there is a continuing, and increasing, need for the C&P program. As well, the evaluation found that there is a role for federal intervention and C&P activities are relevant and aligned with Government of Canada priorities, contributing to economic prosperity, food safety, eco-certification and national security. Finally, the evaluation concludes that the objectives of C&P are still relevant to DFO’s objectives and priorities, as they directly contribute to the departmental Strategic Outcome of Sustainable Aquatic Ecosystems and significantly support the other two departmental Strategic Outcomes of Safe and Secure Waters and Economically Prosperous Maritime Sectors and Fisheries.
C&P has strengthened its outreach, for the purpose of having an informed public and resource users, through a variety of Education and Shared Stewardship activities. Over the past five years C&P has almost doubled its efforts in this area, from 6.6% of C&P effort in 2005 to 12.8% in 2009. However, several questions were raised about whether C&P is targeting the right audiences for this activity. While it appears that more time is being spent with industry representatives, it was found that more direct and strategic contact with industry is required. More specifically, it was noted that a national approach and direction to Education and Shared Stewardship activities, with National leadership, is required. This would help ensure that C&P is consistent in communicating key messages, and reaching the intended audiences.
It was also found that no formal mechanisms exist for assessing whether the Education and Shared Stewardship activities result in a sufficiently informed public and resource user. Evidence from the evaluation notes that this is assessed based upon informal feedback. While evidence indicates that C&P key informants feel that C&P is having considerable impact upon its audiences, there is a lack of a formal, measureable way to determine if a) the right groups are being targeted, and b) those groups are sufficiently informed.
The evaluation concludes that C&P has been effective in achieving management measures built on common goals and in achieving effective compliance incentives and deterrents. However, influences external to C&P affect its ability to achieve success in this area. Namely, C&P’s limited participation and consultation in the development of IFMPs is a barrier to achieving effective management measures. This lack of collaboration between Resource Management and C&P, likely due to time and staff constraints and the need for better communication between C&P and other DFO sectors, affects C&P’s abilities to enforce regulations and license conditions. For example, license conditions are at times drafted in such a way that they are difficult to enforce, such as with the “mortally wounded”22 clause. The evaluation found that there needs to be early and ongoing discussions between C&P and Resource Management to ensure that C&P has input into license conditions to ensure they are in line with legislation and are enforceable.
Another barrier to C&P’s success may be that it does not have designation as an “investigative body” under the Access to Information Act and the Privacy Act. As such, other investigative bodies such as law enforcement agencies (e.g. RCMP) and other regulatory enforcement officers (e.g. conservation officers) have a limited ability to share certain information with C&P. This may limit C&P’s access to information and therefore may be a barrier to their success in the area MCS, but more importantly in Pillar III, major cases and special investigations and with any work that involves joint partnerships with those agencies that have the “investigative body” status.
The evaluation found that about half of C&P efforts are allocated to its MCS activities and about one-third of C&P efforts involve routine patrols. However, as efforts are being devoted to other activities, such as Pillar I and Pillar III, this activity is decreasing slightly. It was also noted in the evaluation that there is an understood link between the visibility of Fishery Officers and the level of deterrence and detection. However, with fewer Fishery Officers, C&P has to be more targeted and strategic in its approach to ensuring compliance. Pillar III, Major Cases and Special Investigations, may have the ability to lead C&P in this direction. Statistical analysis shows that efforts in Pillar III do have the ability to improve the number of reported violations. It was found that C&P has been less effective in this area. However, it is also duly noted that Pillar III is a relatively new area for C&P. The evaluation concludes there is room for improvement, as currently C&P lacks the structure and resources to be effective. C&P needs to ensure that specialized training and systems are in place to move forward with this “intelligence-led” approach to enforcement, including ensuring the interoperability of systems between regions.
The evaluation concludes that C&P has a skilled, equipped and well-informed workforce. It was found that C&P has a highly-skilled and dedicated workforce, and that the training received through the FOCPP is of excellent quality. Some weaknesses in Fishery Officer training were identified, the first being the lack of a well-defined training plan to carry officers forward throughout their career. The second was a lack of training in the skills required to undertake Pillar III. It was found that there is a lack of “core investigative competencies”, and that specialized training, such as forensic analysis, is required.
In addition, the evaluation found that the majority of Fishery Officers (80.1%, n=311) indicated that they find their job to be sometimes overwhelming or frequently/always overwhelming (49.2% n=191 and 30.9% n=120 respectively). The majority of comments from Fishery Officers indicated they feel overwhelmed due to newly emerging priorities undertaken by C&P that often come without any additional resources; Fishery Officers indicated this is a significant challenge to the ability to balance their day-to-day duties. It was noted that C&P is undertaking a National Expenditure and Workload Review, which may alleviate these concerns.
C&P has achieved it intermediate outcome to a moderate to considerable extent, with a high compliance rate and data confirming that deterrent actions taken by C&P (specifically issuing tickets, laying charges, or issuing warnings) are on average accompanied by significant reductions in the recidivism (re-offending) rates relative to offences where these deterrent measures were not used.
The evaluation concludes that C&P’s Compliance Management Framework and its three-pillar design provide a comprehensive and balanced approach to protecting Canada’s fishery resources for the long term. However, there are concerns about the ability of C&P to ensure adequate resources to maintain, or expand upon, the current level of activities. Continual resource challenges and a salary shortfall combined with emerging priorities has resulted in the reallocation of resources from one pillar to support another, resulting in some difficulty in balancing all three pillars adequately. A few key informants noted the introduction of line reporting as working well and being a contributing factor the program’s efficiency.
The evaluation concludes that program delivery could be substantially improved with a stronger emphasis on national direction, consistency in approaches and systems between regions, specialization of functions and training, consolidation of small satellite offices into larger ones, and improved communication and collaboration with industry and other user groups.
The evaluation concludes that C&Ps National Enforcement Databases, FEATS and DVS have been useful for decision making, measuring program performance, risk assessment, planning, budgeting, reporting on work plans and priorities and sharing with industry and others what is being done. However, several shortcomings were noted, including: lack of Fishery Officer access to data when out of the office, inconsistencies in data entry, staff frustrations with using the systems and the information that is not captured, and limited resourcing to support the management of C&P databases. While current upgrades to the systems may address some of these shortcomings, C&P should consider the use of mobile devices for Fishery Officers, and also method to foster consistency in data entry.
The Evaluation concludes that C&P is currently under strain as minimal budget allocations have been provided despite additional priorities and increasing expectations which continue to be added to C&P, such as aquaculture, increasing international obligations, SARA and other initiatives. The majority of key informants (C&P management n=18, 88.9% and external stakeholders n=11, 91%) indicated that C&P could not conduct its business with fewer resources and still maintain quality and quantity and the majority of Fishery Officers (80.1%) indicated that the feel at least “sometimes overwhelmed" in their jobs. Even if resources are maintained at the current level, other factors such as inflation and rising fuel costs (which impacts on C&P’s use of aircraft, boats and vehicles) have an impact upon C&P’s resources. C&P must first address how it can realistically utilize existing or even fewer resources to optimally achieve the most appropriate and critical outcomes. Then it should pursue initiatives that contribute to further economies where possible.
Relevance |
|
Performance |
Effectiveness |
|
Design and Delivery |
|
Efficiency and Economy |
|
Issue/Questions |
Indicator |
Data Source |
||||||||||
Interviews |
FO Survey |
Doc Review / Min. corres- |
Media Scan |
Lit Review |
Compa- |
Statis- |
Case Study |
|||||
C&P Mgmt. |
External Stake- |
Other DFO & FPT Partners |
||||||||||
Relevance |
||||||||||||
1. Is there a continued need for C&P? |
|
√ |
√ |
√ |
√ |
√ |
|
|
|
|
√ |
|
2. Is there a role for Federal Government intervention, and is C&P aligned with Government of Canada priorities? |
|
|
|
|
|
√ |
|
|
|
|
|
|
3. Are the objectives of C&P still relevant to DFO/CCG objectives and priorities? |
|
|
|
|
|
√ |
|
|
|
|
|
|
Performance |
||||||||||||
Effectiveness |
||||||||||||
1. To what extent has C&P achieved its immediate outcomes? |
||||||||||||
a. To what extent have C&P activities resulted in an informed public and resource user? |
|
√ |
√ |
|
√ |
√ |
√ |
|
|
|
√ |
|
b. To what extent has C&P achieved effective management measures built on common goals? |
|
√ |
√ |
|
√ |
√ |
|
|
|
√ |
√ |
|
c. To what extent has C&P achieved effective compliance incentives and deterrents? |
|
√ |
√ |
|
√ |
√ |
|
√ |
|
√ |
√ |
|
d. To what extent does C&P have a skilled, equipped and well-informed workforce? |
|
√ |
√ |
|
√ |
√ |
|
|
|
|
|
|
2. To what extent has C&P achieved its Intermediate outcome? |
||||||||||||
a. To what extent has C&P resulted in compliance with legislation and regulations and management measures? |
|
√ |
√ |
√ |
√ |
√ |
|
|
|
√ |
√ |
|
3. Are there any unintended outcomes, positive or negative, that can be attributed to C&P? If so, were any actions taken as a result of these? |
|
√ |
√ |
√ |
|
√ |
|
|
|
|
√ |
|
4. Are there any external factors and/or general challenges/barriers that influence the success of C&P? |
|
√ |
√ |
√ |
√ |
√ |
√ |
|
√ |
|
√ |
|
Design & Delivery |
||||||||||||
1. To what extent are the current program design and delivery model and governance structure, effective to support the achievement of results? |
||||||||||||
a. What are the primary challenges with the current program design? |
|
√ |
√ |
√ |
√ |
√ |
|
|
|
|
|
|
b. What are the primary benefits of the current program design |
|
√ |
√ |
√ |
√ |
√ |
|
|
|
|
|
|
c. Would an alternative program design or delivery mechanism be more appropriate to achieve expected results? |
|
√ |
√ |
√ |
|
|
|
|
√ |
|
|
|
d. To what extent were activities modified, and what was the effect on program design? |
|
√ |
√ |
√ |
|
√ |
|
|
|
|
|
|
2. Do the activities of the C&P program complement, overlap or duplicate with other programs of DFO or other Federal Government Departments? |
|
√ |
√ |
√ |
√ |
|
|
|
√ |
|
|
|
3. Is there a performance measurement strategy and a reporting process / system in place? If yes, is it being used for decision making? |
|
√ |
|
|
|
√ |
|
|
|
|
|
|
4. Are there best practices and lessons learned from C&P? |
|
√ |
√ |
√ |
√ |
|
|
|
|
|
|
|
Efficiency & Economy |
||||||||||||
1. Are the appropriate administrative and management systems in place for efficient C&P implementation and management? (efficiency) |
|
√ |
√ |
√ |
√ |
|
|
|
|
|
|
|
2. Did the C&P resource utilization and activities optimally produce expected levels of outputs? How could the efficiency of C&P activities be improved? (Efficiency) |
|
√ |
|
|
√ |
|
|
|
|
|
|
|
3. Is C&P operating in a way that minimizes the use of resources to achieve its intended outcomes? (economy) |
|
√ |
√ |
|
|
|
|
|
|
|
|
|
Recommendations |
Management Action Plan |
Status Report Update |
||
Actions Com- |
Actions Outstan- |
Target Date |
||
Recommendation 1:
|
Response Through the program’s recent Compliance and Modernization Review, C&P has committed to spending more time working with the public, stakeholders and resource users to raise awareness and promote compliance. A target of 10 percent of total C&P effort was set and was reached last year. While the plan is to maintain the current level of effort for the foreseeable future, there is limited ability to increase this effort given the on-going challenge of balancing our four pillars of compliance. We accept this recommendation and will strive to achieve more efficiency in our education and shared stewardship activities through the following action items.
|
|
|
August 31, 2011
September 30, 2011
October 31, 2011
March 31, 2011 |
Recommendation 2:
|
Response The Education and Shared Stewardship activities are now captured under one of the four C&P Program Activities pursuant to the Departmental Program Activity Architecture. Therefore, it is critical that we continue to refine the Performance Measurement Framework and Strategy in order to be able to report back on efficiency and effectiveness. While the program is currently effective at capturing its education and shared stewardship activities and outputs, outcomes are more difficult to measure. C&P will continue to refine its PMF and PMS and will explore and develop reasonable means to measure the impacts of its education and shared stewardship activities. It accepts these recommendations and commits to the following action items.
|
|
|
September 30, 2011
December 31, 2011
March 31, 2012 |
Recommendation 3:
|
Response Over the last 10 years or more, numerous IFMP issues, related to the lack of effective integration and coordination between C&P and Resource Management, have been identified. It was one of the eight lines of inquiry in the recent Compliance Review and Modernization Initiative conducted by C&P. Similar reviews had been conducted prior to that. While there are still some concerns about the lack of integration, the program is of the view that things have improved and that, in fact, it is going very well in some of the regions. The recent focus on the renewal of the IFMP process has helped identify respective roles and responsibilities and has clarified the type of input required in the development of the IFMP. Nevertheless, there is still room for improvement and more in some regions than in others. The program accepts this recommendation and commits to the following action items.
|
|
|
March 31, 2012
|
Recommendation 4:
|
Response The merits of obtaining the designation as an Investigative Body have been under consideration by C&P for a number of years. These designations are not easily obtained and a previous attempt made by C&P (in 1995) was not successful. In 2006, the program conducted a review which included a national workshop aimed, in part, at assessing the need to pursue designation as an Investigative Body under the Access to Information and Privacy Acts. At that time, there was not enough evidence to convince program managers that there would be significant benefits if the C&P program were to acquire this designation. However, as the issue has been raised once again in this evaluation report, the program will commit to reviewing the earlier work and raising it once more at the National C&P Directors Committee.
|
|
|
File to be assigned by February 28, 2011. Review of documentation, consultation with legal services, ATIP and Privacy Act experts, C&P staff, etc. by November 30, 2011. Results and recommendations to be tabled at C&P Directors Committee by December 31, 2011. |
Recommendation 5:
|
Response The program has already made a commitment in its 2011/12 Business Plan, to continue to build capacity on Major Case/Special Investigations, consistent with the new direction provided through the Compliance Review and Modernization Initiative. A number of very specific deliverables have been identified under Key Commitments for On-going Work in the Business Plan. The program accepts this recommendation and agrees to turn more of its focus on the matters relating to the identification and acquiring of the necessary competencies as well as the coordination and management of national intelligence data and the systems.
|
|
|
February 15, 2011
March 31, 2012
December 31, 2011
March 31, 2012
March 31, 2012
March 31, 2012
March 31, 2012 |
Recommendation 6: It is recommended that C&P explore options to increase efficiency and address workload concerns, including a) establishing national direction for consistent procedures; b) consolidating smaller satellite offices into larger ones.
|
Response The biggest challenge currently facing the program is the ever increasing capacity gap, which was identified in the Compliance Review and Modernization initiative. The salary shortfall created as a result of the BASR2 exercise has contributed significantly to this capacity gap. The more recent budget freezes as well as anticipated cuts associated with the Strategic Review have further compounded this problem. In the absence of any new funding, the program has no option but to apply a rigorous risk assessment process to its planning and to seek new alternatives to service delivery. The Department must also accept that in the end, fewer services will be delivered. The program has recently initiated a national review of the C&P program to help; identify areas where efficiencies may be gained, setting new and more consistent service delivery standards, and, reshape the program to be able to live within its means. This could include office consolidation, more strategic planning and deployment of resources, improved use of technologies and a more effective balance of the three pillars of activities. We accept this recommendation and commit to the following specific deliverables.
|
|
|
December 31, 2011
March 31, 2012
March 31, 2012
March 31, 2012
March 31, 2012 |
1 This allows fishers to keep a species, such as Coho salmon, that is either dead or mortally wounded when otherwise they would be required to release it.
2 Community-based justice describes dispute resolution processes that are developed and operated by communities, and includes both restorative justice and aboriginal justice processes.
3 This displays FTE’s for the C&P Program business line only.
4 The observation period covered by the data is January 1, 2003 to December 31, 2010.
5 For example key informants were asked to rate success on a scale of 1 to 5 where 1=Not at all successful, 2=Little success, 3=Moderately successful, 4=Successful, 5=Very successful. Similar scales were used for extent, effectiveness and efficiency ratings.
6 Traceability entails access to relevant data in the fish/product chain, from catch or farming to consumption
7 Eco-certification is a market-based measure intended to improve the sustainability of fisheries.
8 Fishery Officers are “peace officers” as defined by the Criminal Code in the application of the Fisheries Act and the Coastal Fisheries Protection Act.
9 The National Integrated Interagency Information System (N-III) Initiative intended to facilitate data and information-sharing across organizations of peace officers and to ensure interoperability among agencies with similar mandates to achieve common governmental objectives related to tackling crime and keeping Canadians safe.
10 The compliance rate is determined by the number of violations detected relative to number of checks performed.
11 This analysis was undertaken with support from DFO’s Economic and Statistical Analysis Directorate.
12 Kuperan, K. and Jon G. Sutinen (1988). “Blue Water Crime: Deterrence, Legitimacy, and Compliance in Fisheries. Law & Society Review, Volume 32, Number 2, pg. 309 – 338.
13 Kuperan & Sutinen, pg. 328
14 In 2010, British Columbia experienced an unexpected surge of over 30 million sockeye salmon into the Fraser River, as opposed to the 2009 season which saw the return of only approximately one million salmon.
15 http://www.statcan.gc.ca/about-apercu/law-acte-eng.htm
16 The compliance literature identifies the “normative” perspective on compliance which emphasizes what individuals consider just and moral, instead of what is in their self-interest. That is, individuals tend to comply with the law to the extent that they perceive the law as appropriate and consistent with their internalized norms. The key variables here are the individuals’ perceptions of the fairness and appropriateness of the law and its institutions. Kuperan, K. and Jon G. Sutinen (1988). “Blue Water Crime: Deterrence, Legitimacy, and Compliance in Fisheries. Law & Society Review, Volume 32, Number 2, pg. 312.
17 Here, compliance with regulation is influenced by the extent to which individuals feel that the enforcement agencies are appropriate in, or have the right to, restrict their behaviour. Ibid.
18 An Administrative Sanction is a way to deal with infractions of the Fisheries Act in designated coastal fisheries by way of administrative license sanctions and monetary penalties.
19 With the introduction of line reporting, C&P managers were to have full control over their program financial resources. Several key informants noted that this was not occurring, however this evaluation was not designed to explore issues related to line reporting, as that was undertaken in 2007. The evaluation be found here: http://www.dfo-mpo.gc.ca/ae-ve/evaluations/07-08/cpeval-eng.htm.
20 For example, in Ontario under Section 53(2) of the Police Services Act,individuals can be provided limited peace officer and/or police officer powers as defined in the appointment. The appointment as a “special constable” is utilized by a variety of positions to provide the necessary authorities to perform their duties. The appointment of a special constable may confer on a Fishery Officer the powers of a police officer, to the extent and for the specific purpose, set out in the appointment. As this is provincial legislation, there may be regional variation.
21 The Canadian Chamber of Commerce. Policy Resolutions 2009, 31
22 This allows fishers to keep a species, such as Coho salmon, that is either dead or mortally wounded when otherwise they would be required to release it.