Language selection

Search

Evaluation of Economic Analysis and Statistics

Project number: 96261
Final report
January 16, 2020

Evaluation of Economic Analysis and Statistics
(PDF - 1.0 MB)

Table of contents

1.0 Profile of the Economic Analysis and Statistics function

For the purpose of this evaluation, the Economic Analysis and Statistics (EAS) Function includes all EAS programs and services as well as their governance structure(s). This includes all economic analysis, statistical information and advice provided at National Headquarters (NHQ) and in regions, on a wide range of issues of concern to Canada’s fisheries, oceans’ sectors and the Canadian Coast Guard (CCG), to support evidence-based decision making.

Starting in 2018-19, three new programs in the Departmental Results Framework (DRF) were initiated by the EAS Directorate at NHQ to facilitate delivery of economic analysis and statistical services at the national and regional levels, to inform departmental decision-making. The three programs are as follows:

Description

Core Responsibility

Fisheries: Manage Canada’s fisheries, Indigenous fisheries programs, aquaculture activities and support commercial fishing harbours while applying relevant legislation

Fisheries Economics and Statistics Program  

Purpose is to provide fisheries statistical data and economic analysis and advice at the national or cross-regional level in support of DFO’s sustainable management of Canada’s fisheries and aquaculture.

Core Responsibility

Aquatic Ecosystems: Conserve and protect Canada’s oceans and other aquatic ecosystems and species from human impact and invasive species

Aquatic Ecosystems Economics Program

Purpose is to provide economic information and analyses in support of departmental decisions related to species at risk, oceans management (marine protected areas, marine networks), aquatic invasive species and climate change.

Core Responsibility

Marine Operations and Response: Provide marine response services and operate Canada’s civilian maritime fleet

Marine Operations Economics Program  

Purpose is to provide economic analyses to inform Canadian Coast Guard operations, such as fleet renewal, expenditure forecasting and service modernization.

EAS Function encompasses a range of services, including mandatory services, provided to varying degrees by the EAS Directorate and each region:

Economic analysis, statistical information and advice are provided to DFO and CCG by the EAS Directorate at National Headquarters (NHQ) and independently by economic analysis units within the Policy and Economics or Strategic Services groups in DFO regions (i.e. Newfoundland and Labrador, Maritimes, Gulf, Quebec, Central and Arctic and Pacific).

The EAS Directorate provides services mainly to clients at NHQ, in response to client requests. It also provides framework and guidance documents to assist the regions and sectors. Examples of frameworks and guidance include a Framework for Socio-Economic Analysis to Inform Integrated Fish Management Planning and Fish Harvest Decisions (2008) a Framework for Integrating Socio-Economic Analysis in Species at Risk Act Listing Decisions (2016) and Guidance on incorporating economic use information into marine protected area network design (2017). Regional EAS services are provided on a response basis.

Four of the regions i.e. Newfoundland and Labrador, Maritimes, Gulf, and Central and Arctic include fisheries statistics as part of their economic analysis, statistical information and advice. In the other two regions i.e. Quebec and Pacific, statistics are part of the Fisheries Management program.

Much like the rest of DFO and CCG, there is no line reporting between NHQ EAS Directorate and the regional Policy and Economics or Strategic Services units.

This means that the EAS Function staff report to the Director General EAS at NHQ or to the Regional Directors General in DFO’s six regions. EAS Function full-time employees (FTEs) increased by 20% from 2014-15 to 2018-19, from about 69 to 83.

In the figure below, the numbers on the left indicate the total number of staff involved in the EAS Function. The numbers on the right represent the number of staff involved in fisheries data entry, a function that is unique to DFO among the peer departments. Expenditures related to economic analysis, statistical information and advice are presented below. The graph on the left shows EAS Function expenditures including the NHQ and regions. The graph on the right shows EAS program expenditures, i.e., for the EAS in NHQ only (EAS programs are only resourced at the NHQ as reflected in the DRF).

Description

Actual EAS Function FTEs for 2018-19

Pacific Region had 4 FTEs involved in the EAS Function, none involved with fisheries data entry. Central Region had 3 EAS Function FTEs plus 1 other involved in fisheries data entry. Quebec Region had 5 EAS Function FTEs, none involved with fisheries data entry. Gulf Region had 4 EAS Function FTEs plus 11 others involved with fisheries data entry. Maritimes Region had 5 EAS Function FTEs plus 9 others involved with fisheries data entry. Newfoundland and Labrador had 9 EAS Function FTEs plus 7 others involved with fisheries data entry. National Capital Region had 21 EAS Function FTEs, none involved with fisheries data entry.

EAS Function Expenditures 2014-15 to 2018-19

Description

EAS expenditures increased by 11 per cent, from $6.5 million in 2014-15 to $7.2 million in 2018-19. During the same timeframe DFO expenditures as a whole increased by 98 per cent.

Expenditures for the three EAS programs in the DRF represent a modest portion of total EAS Function expenditures, around $2 million in total. This is split between the two programs, Fisheries Economics and Statistics Program and Aquatic Ecosystems Economics Program unevenly. Approximately 2/3 of the expenditures are attributed to the Fisheries Economics and Statistics Program, where the actual expenditures exceeds the planned by a small amount. The second program’s actual expenditures match its planned expenditures. Finally, the third program has a low level of planned expenditures, but no actual expenditures.

2.0 Evaluation objective and scope

Evaluation objective

The evaluation assessed the extent to which the EAS Function addresses DFO and CCG needs for economic analysis, statistical information and advice to support evidenced-based decision-making.

Evaluation scope

This is the first evaluation of the EAS Function. It covers the five-year period April 2014 through March 2019 and focuses on the service needs of key internal DFO/CCG clients only – clients external to the department were not included. The evaluation includes the Ice Assistance Emergency Program.

The evaluation does not address DFO data governance issues other than those specific to the EAS Function.

Fisheries statistics are part of the Fisheries Management program in two regions, Pacific and Quebec. That program is not part of this evaluation.

Evaluation questions

Evaluation questions are aligned with Treasury Board’s 2016 Policy on Results and address areas of interest to senior management. The evaluation assessed factors having a bearing on the , effectiveness and efficiency of the services provided.

This included an assessment of the capacity of the EAS Function to address client needs, and the interface between the EAS Directorate and DFO programs, sectors, regions and the CCG.

Specifically, the evaluation answered the following questions:

  1. To what extent do the three programs address ongoing needs within DFO/CCG?
  2. To what extent are economic and statistical data and analysis considered to be timely, current, accurate, comprehensive and accessibleFootnote 2?
  3. To what extent do clients understand the economic and statistical information provided and consider it reliable and relevant to their needsFootnote 2?
  4. To what extent and in what ways have the economic and statistical information and advice contributed to support and inform decision making within DFO/CCG?
  5. To what extent are Gender Based Analysis (GBA+)*  considerations included in economic and statistical data, analyses, and advice?
  6. What are the factors that have facilitated or hindered the programs’ ability to inform DFO/CCG decisions?
  7. In what way are the three programs an effective and efficient means of providing services?

Evaluation methodologies, limitations and mitigation strategies are discussed in Appendix A.

* Gender-based analysis plus (GBA+) involves considering diverse, under-represented groups in the development, delivery and reporting on results of departmental policies and programs. The “plus” in GBA+ indicates it encompasses more than gender. For example, Indigenous Peoples, remote communities and low-income coastal communities are DFO-relevant GBA+ groups, in addition to women and possibly other groups.

3.0 Evaluation findings

Evaluation findings: Program results

Overall, the evidence demonstrates that two of the three programs have met their targets and are effective in providing EAS services. The third program is intended to address Canadian Coast Guard needs regarding economic analysis, statistical information and advice. These needs have yet to be defined.Results

Tracking of results for reporting purposes in the Performance Information Profiles (PIPs) for the three EAS programs has been done only for the EAS Directorate thus far. The results indicate that the Fisheries Economics and Statistics program and Aquatic Ecosystems Economics program are meeting their targets of addressing 100% of requests received.

While the results indicate they have met their targets and are effective, the indicator (percentage of requests completed) does not provide a comprehensive assessment relevant to the quality and complexity of analyses and advice, the workload and resource allocation, or whether the programs support decision-making. In addition to the need for meaningful indicators, EAS Function programs and PIPs need to address all four departmental Core Responsibilities, as the EAS Function is relevant across the department.

The numbers below include a large number of requests for statistical data and analyses.

Performance Information Profiles (PIPs) Requests completed 2017-18 Requests completed 2018-19
Fisheries Economics and Statistics Program 630 520
Aquatic Ecosystems Economics Program 105 32
Marine Operations Economics Program 2 0
Total 737 612

Marine Operations Economics Program

Based on the evidence available, it was not possible to assess the effectiveness of the third program. Requests for services from CCG has decreased to zero, and the planned FTE has been reallocated to other priorities.

CCG’s National Strategies’ Economic Industry Intelligence notes that it does not conduct its own economic services as defined by the EAS Function. An example of their work would be examining trends in Canada’s marine trade (e.g. at Canadian Ports, top commodities shipped/handled at ports, value of shipped goods) and related marine traffic trends to enhance awareness of Coast Guard client needs, and to inform program & service delivery. The evidence suggests that the EAS Function has not provided any services to the CCG in the last fiscal year.

Evaluation findings: Contribution to decision making

The EAS Function contributes to senior management decision making by addressing routine requests, such as cost-benefit analyses for regulatory proposals and SARA Action Plans, and ad hoc requests for fisheries information and analyses.

Economics: Significant role

Economics is widely recognized as an important element of fisheries and ocean conservation, one that informs legislative, regulatory, policy and management decisions in many countries.

Contribution to DFO decision making

Economic analysis, statistical information and advice have contributed to informing senior level decisions and have helped senior management understand the consequences of decisions. The EAS Function fulfills two mandatory requirements, specifically cost-benefit analyses as required by Treasury Board for regulatory proposals, and a similar but less prescribed analysis required by DFO for SARA Action Plans.

Economic analysis, statistical information and advice have served an essential role in informing, among other things, Memoranda to Cabinet, Treasury Board Submissions, regulatory proposals, Integrated Fisheries Management Plans, Total Catch Allocations, treaty negotiations and fishery closures.

Examples

Cost-benefit analysis

Ad Hoc analysis

Evaluation findings: Information attributes are critical

The EAS Function contributes to senior management decision making by addressing routine requests, such as cost-benefit analyses for regulatory proposals and SARA Action Plans, and ad hoc requests for fisheries information and analyses.

The eight attributes below, which Treasury Board has used to define quality information, provided a basis for assessing the performance of the EAS Function.

Framework to assess performance: Information attributes

These attributes are prerequisites…

…for the following 3:

A DFO Mission-critical risk

According to the 2019-20 Corporate Risk Profile, information for decision making is a mission-critical risk. A lack of access to complete, accurate and timely data and information could impede evidence-based decision and policy making and impair DFO’s and CCG’s ability to deliver on their mandates and fulfill their core responsibilities.

Clients’ perception

Clients’ perception of the extent to which the eight information attributes are addressed is positive, although there is room for improvement. EAS Function directors closely matched client ratings, except for timely and relevant which they rated on the scale closer to ‘considerable’ extent (3.8).

Table 1. Attribute ratings
Attribute Clients* (n=24) Directors, EAS function (n=8)
Timely 3.4 3.8
Current 3.2 3.2
Accurate 3.4 3.6
Comprehensive 3.3 3.6
Accessible 3.2 3.1
Understandable 3.6 3.5
Reliable 3.6 3.6
Relevant 3.5 3.8
Scale: 3 = moderate extent; 4 = considerable extent

* Interviews and survey

Several EAS Function directors noted a six-month to two-year lag for some fisheries statistics. They see this as affecting the comprehensiveness, accuracy, reliability and of EAS analyses.

Evaluation findings: Fisheries data affects EAS

The EAS Function’s ability to provide reliable and relevant analyses is dependent in part on the quality of fisheries data available.

EAS Function is affected by the quality of statistical data available

Well-documented statistical data issues affect the EAS Function. Four reports since 2007 identify significant fisheries data issues that compromise the availability of accurate and current data on a timely basis, particularly at the national level. The most recent report, the September 2018 Data Governance Workshop Report, lists key data-related challenges for the three EAS programs (p. 94), also noted in a 2015 report.

Examples

Evaluation findings: Gaps

Gaps exist in terms of planning, capacity and services, and there is a lack of awareness of the EAS Function on the part of clients and potential clients. This limits the EAS Function’s capacity to inform DFO/CCG decision-making.

The evaluation evidence demonstrates that there are gaps/factors that impact the EAS Function and they related to planning, capacity and services.

Planning

Proactiveness

Understanding needs

Tools

Capacity

Resources and data

Research and analysis

Services

Awareness of EAS services

A common list and descriptions of the services offered by the EAS Function (the regions and EAS Directorate) would contribute, in part, to improving clients’ understanding of EAS services available.

CCG needs

GBA+

Rule of five

EAS advice in briefing material

* GBA+ analysis is currently represented to a limited extent in economic analysis, and program leads are not yet fully taking GBA+ into account during the early phases of program, policy, or regulatory design.  As guidance on GBA+ implementation becomes available from the GBA+ Centre of Expertise, shared responsibilities will become more clear.

**To protect privacy, the rule of 5 is applied, meaning there must be a minimum number of 5 units in each category. For example, if an analysis involves fishers, vendors and buyers, there must be a minimum of 5 in each group, not just one or two of the groups.

Evaluation findings: Gender based analysis+ 

GBA+ is addressed by the EAS Function to a limited extent, due in part to a lack of available data. Economic analysis, statistical information and advice focus mostly on Indigenous peoples and remote communities. DFO and CCG need to clarify what is required for GBA+ for the department. Women’s full contribution to fisheries, aquaculture and marine conservation, beyond employment, is overlooked and underestimated by many countries including Canada.

The EAS Function is required to provide cost-benefit analyses in support of regulatory proposals and SARA Action Plans. A key component is the socio-economic analysis which addresses the impact of proposals and plans on specific populations, such as Indigenous peoples and remote or low-income communities. The federal government requires consideration of impacts on various groups of men and women, otherwise referred to as GBA+.

The EAS Function has taken GBA+ into consideration to some extent, focusing mostly on Indigenous peoples and remote communities in some of its analyses. Generally, the term is not well known or understood, and it is not clear whether there are under-represented populations relevant for DFO and CCG beyond Indigenous men, Indigenous women, low-income or remote communities, and non-Indigenous women.

GBA+ data is usually obtained from Statistics Canada, Canada Revenue Agency or Employment and Skills Development Canada as they typically collect demographic data. Whether these data sources can provide whatever additional GBA+ data is deemed relevant to DFO or if DFO needs to collect the data itself needs to be determined and is a decision involving Strategic Policy and CCG.

The EAS Function has some GBA+ databases that it could be using more extensively and which others might wish to access as well.

“Indigenous Women…Catalyze Change in Fisheries Governance on Canada’s Pacific Coast”
– 2018 IIFET Newsletter, p. 17

Of seven EAS Directorate guidance documents (e.g. the Framework for Cost-Benefit Analysis of Aquatic Species at Risk Regulations (2016)), three guidance documents mention gender as a factor to consider, among others, e.g., Indigenous peoples, official-language minorities, recent immigrants and lower income Canadians.

In many countries, including Canada, women’s full contribution is often overlooked even though they are a substantial and essential part of the industry. Worldwide, they comprise half the total working population in fisheries and aquaculture, including processing and other related services. Aside from harvesting and processing, women in European fisheries have been involved in at least five main categories including overall management, communication, bookkeeping, marketing, and logistical support, such as picking up equipment and crew.

In Canada, women are involved in attending industry association meetings, preparing license applications, bookkeeping and possibly other essential activities, which are not captured in DFO or other available data.

Evaluation findings: Efficiency

The inclusion of the three programs in the Departmental Results Framework has given the EAS Function access to funds for its economics and statistical work, which had not been previously available through enabler funding. In addition, DFO’s EAS Function resources, when compared to other Economic Analysis and Statistics units in other departments, are proportionately lower.

Efficiency of the EAS function

The inclusion of three EAS programs in the DRF has given the EAS Function much needed access to funds for its economics and statistical work, which had not been previously available through enabler funding. Regional Directors also commented on the importance of the EAS programs as a means of securing funds. The EAS Function at NHQ and in the regions does not receive enabler funding unless directed by the ADM, whereas the EAS programs allows the EAS Function to secure funding through Memoranda to Cabinet.

The economic analysis and statistics units in three other comparable government departments do not have EAS programs as part of their DRFs but instead operate under enabler funding. When the four year average (2015-16 to 2018-19) of expenditures,which includes large data purchases, and FTEs for the economic analysis and statistics units are roughly compared to the total expenditures and total FTEs of the other three departments, the results suggest DFO’s EAS Function resources are proportionately lower compared to the three other government departments.

* 21 FTEs are from NHQ

Description

Resources for DFO’s EAS Function (expenditures and FTEs) were less than 1 per cent of total DFO resources. The portion of departmental resources devoted to EAS in three other federal departments were greater than DFO’s although also below 1 per cent for the most part.

Evaluation findings: Governance

The EAS Directorate and the regional Economic Directors together provide oversight of DFO’s EAS Function facilitated, in part, by a National Economic Directors Committee. However, greater clarity is required with respect to the EAS Directorate’s role and the NEDC’s in terms of providing guidance, direction and governance of the EAS Function including the programs.

Governance

The EAS Directorate and the regional Economic Directors together provide oversight of DFO’s EAS Function. In the absence of line reporting, the NHQ EAS Directorate and the regional EAS Function directors have established good working relationships. This is facilitated, in part, through a National Economic Directors Committee (NEDC) which holds regular monthly meetings and an annual face-to-face meeting. The committee is unique in that there is no comparable Strategic Policy committee bringing together regional directors and NHQ Strategic Policy counterparts.

According to the NEDC’s Terms of Reference, its mandate, in part, is to be responsible for establishing national processes and guidance documents. The Terms also identify key activities as including, annual economic workplans for the upcoming fiscal year and the development, review and approval of national processes and frameworks. This suggests the NEDC has a key role in the governance of the EAS Function of the Department. In practice, this is not the case. Instead, the NEDC has evolved into more of an information-sharing body, although it has been involved in developing guidance material posted on the EAS Directorate’s intranet webpages.

The EAS Directorate indicates on DFO’s intranet that it is the Departmental Centre of Expertise for economic and statistical analysis and research. This suggests the EAS Directorate has the lead role and not the NEDC. As such, greater clarity is required with respect to their respective roles.

Half of the EAS Function directors are of the opinion that it is not clear if the NEDC or the EAS Directorate should establish national processes and guidance documents. The lack of line reporting between the regions and NHQ is seen as contributing to the lack of clarity between the NEDC and the EAS Directorate. Two-thirds noted there is a lack of clarity with respect to the NEDC’s role regarding the three EAS programs and the performance indicators as these are not part of NEDC discussions.

The three OGDs selected for comparative purposes have a centralized EAS function, which they believe contributes to the efficiency of their operations. While the NEDC helps offset the decentralized nature of DFO’s EAS Function to a limited extent, there is need for greater clarity regarding its role and that of the EAS Directorate for the EAS Function and the EAS programs.

A Different approach

In the Pacific Region, the Policy Analysis and Treaty Support (PATS) unit within the Reconciliation and Partnerships Branch undertakes economic analyses separate from the rest of the EAS Function. PATS provides economic information related to Food, Social and Ceremonial allocations and economic/commercial access provided to Indigenous groups as part of treaty and reconciliation agreements and packages. PATS also provides data with respect to catch reporting which is used to develop financial and allocation mandates for treaty and reconciliation agreements.

Governance of the EAS Function relies on regional staff working closely with national staff to ensure consistency in analytical approaches, which is not the case in this instance (Figure 1). Instead, there is a “wall” between PATS and the EAS Directorate. Lack of a relationship with PATS and access to its data hinder the EAS Directorate’s ability to provide analyses at a national level and ensure consistency in informing the Deputy Minister’s decision-making on treaties.

Figure 1: Disconnect between PATS and EAS Directorate hinders EAS analyses at national level

Description

Disconnectbetween the Pacific Region PATS and EAS Directorate hinders EAS analyses at national level

In the Pacific Region, unlike other DFO regions, some EAS work is undertaken by the PATS unit which is separate from the region’s Economic Analysis Division. There is no link between PATS and the National Capital Region’s EAS Directorate.

5.0 Conclusions and recommendations

Conclusions

The EAS Function has established a base in serving client needs and contributing to senior management decision-making. The creation of EAS programs has facilitated access to funding, but there is a lack of awareness and understanding of the services that are available.

According to the evaluation evidence, the EAS Function’s contribution department-wide could be strengthened by two means:

Taking a proactive approach to developing an awareness and understanding of economic analysis, statistical information and advice services and identifying needs across the department; and,

Anchoring the EAS Function around DFO’s mandate, long-term priorities and senior management challenges.

Interviewees commented that a proactive approach would include annual EAS workplans and regular meetings with clients and potential clients department-wide. This would contribute to identifying the data that is required on an ongoing basis to support economic analyses. This in turn identifies the data holders and partners the EAS Function needs to engage with in addition to Statistics Canada, CRA and ESDC, such as Transport Canada.

The EAS Function needs to cultivate a better understanding of how it can be of service to inform MCs, TB submissions and address significant and increasingly complex issues having a bearing on DFO’s and CCG’s mandate and priorities. The EAS Function is best positioned to determine how economics can contribute to DFO and CCG programs and services.

Recommendations

Recommendation #1: EAS function planning

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, establish a department-wide annual planning process for multi-year workplans for the EAS Function to proactively serve all departmental programs, and Internal Services as applicable, based on the Departmental Results Framework. Workplans and processes for managing emerging priorities should support the EAS Function in ensuring high-quality, consistent and timely economic and socioeconomic information and advice is available across the department to inform senior management decisions, departmental mandates, commitments, challenges and other long-term priorities.

Rationale: A department-wide, proactive EAS Function will facilitate i) developing an awareness and understanding of services available, ii) establishing annual priorities for services, iii) confirming performance indicators or indicating a need for modifications, iv) identifying ongoing data requirements, v) meeting federal GBA+ requirements, vi) addressing complex issues and opportunities arising, and vii) leveraging existing economic analysis, statistical information and advice work. Anchoring EAS workplans and services around the Departmental Results Framework and long-term priorities serves to underpin the of the EAS Function.

Recommendation #2: Performance indicators

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, ensure the Performance Information Profiles for EAS Function programs incorporate meaningful performance indicators that provide information significant to the achievement of results and management of the programs.

Performance indicators should address Core Responsibility #3 Marine Navigation, as the current

Performance Information Profile does not consider this Core Responsibility. It would either be a separate profile for a fourth EAS program or the programs be amalgamated as one program in the DRF, specifically an all-encompassing program for the EAS Function covering all four departmental Core Responsibilities.

Rationale: Current indicators and targets speak to outputs (number of requests received and dealt with) not whether programs are achieving their intended results or the quality of results. They also do not provide insight on capacity to help inform workload and resource decisions. The EAS Function has for all four departmental Core Responsibilities.

6.0 Appendices: Evaluation methodology and management action plan

Although the evaluation encountered some methodological challenges, methodological limitations were mitigated, where possible, through the use of multiple lines of evidence and the triangulation of data. This approach was taken in order to establish the reliability and validity of the findings and to ensure that conclusions and recommendations were based on objective and documented evidence.

Administrative data

A review of EAS expenditures was undertaken to understand the funding of the EAS programs and services, including in the regions. As the programs were newly created, the financial management systems did not have all the expenditures information for the scope of the evaluation. In addition, EAS in the regions are often part of a directorate which includes other divisions, so expenditures had to be manually segregated. To mitigate this situation, the evaluation team requested the information from the EAS Directorate and the six regions and estimates were used for the analyses. In addition, responsibility for fisheries statistical data in two regions is with a separate sector, Fisheries Management, which is not an EAS program or service. Its resources, therefore, were excluded from the administrative data that was gathered.

Given the recent introduction in 2018-2019 of the EAS programs, the Performance Information Profiles (PIPs) afforded very limited information on results and excluded the information from the regions. To mitigate this challenge, the evaluation assessed the usefulness of the performance indicators and actual results in relation to the intended results and triangulated findings from other lines of evidence.

Document and website review

The evaluation team reviewed information relevant to the EAS Function, such as diagnostic and audit reports on DFO fisheries statistics and data governance, and a review of posted oceans, fisheries and aquaculture information, research, reports, and guidelines.

Mapping exercise

A mapping exercise, during the planning phase of the evaluation, provided the evaluation team with initial insight on the perspectives of the EAS Directorate and regions regarding economic services and activities at the regional and national levels. When possible, the results were used to inform the development of interview guides, the comparative analysis as well as some of the evaluation findings.

Interviews

EAS directors (including from regions) involved in EAS were interviewed. DFO and CCG senior management were also interviewed to gather their views on the services received. Interviews were undertaken with clients that received EAS services from April 2014 to March 2019 (identified by EAS in NHQ and in the regions). Sampling was carried out to ensure that perspectives from different clients (e.g. science, fisheries management, etc.) were gathered to support the results from the survey. Clients who were not interviewed were surveyed.

Comparative analysis

A comparative analysis with three other government departments provided insight on the type of economic analyses and statistical work they undertake, their capacity to respond to requests, whether their economic and statistical services are delivered through a program (i.e., identified as a program on their Departmental Results Framework), policies and processes, GBA+ considerations, whether these departments face similar challenges as DFO, and whether there are other approaches the EAS Directorate might consider.

Documented examples

A brief, focused analysis was undertaken on four examples which contributed to identifying factors that have facilitated or hindered EAS to inform DFO/CCG decisions.

The documented examples were focused on the following themes:

  1. Ice Assistance Emergency Program (IAEP)
  2. Oceans Protection Plan (OPP)
  3. Oceans Program
  4. Treaty Support on the West Coast

Survey

A survey with EAS’ internal DFO National Headquarter and regional clients, i.e., to directors and managers, provided input on their needs for economic analysis and statistics services, whether there are issues needing to be addressed and if EAS services have helped to inform DFO/CCG decisions.

The survey was made available online to 140 EAS clients who were identified by EAS directors, and 29 completed survey responses were received which represents a 20% response rate. The survey was administered between July 15 and August 11, 2019. CCG staff were not included in the survey as they were interviewed instead. The survey responses were used to triangulate findings from other lines of evidence. Although there was a low response rate, especially for some questions, findings were used to triangulate evidence from other lines of evidence. In addition, interviews with DFO’s senior management were undertaken as they are the main decision makers and ultimately users of EAS information for evidenced-based decision making.

Understanding of EAS activities

Understanding the EAS activities in the Marine Operations Economics Program was challenging as there were limited activities taking place and CCG clients were not clearly identified. This situation was mitigated by consulting with CCG senior management to understand their capacity and needs regarding EAS as well as triangulating with other lines of evidence when possible.

Management Action Plan (MAP)

Recommendation 1:

Recommendation:

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, establish a department-wide annual planning process for multi-year workplans for the EAS Function to proactively serve all departmental programs, and Internal Services as applicable, based on the Departmental Results Framework. Workplans and processes for managing emerging priorities should support the EAS Function in ensuring high quality, consistent and timely economic and socioeconomic information and advice is available across the Department to inform senior management decisions, departmental mandates, commitments, challenges and other long-term priorities.

Rationale: 

A department-wide, proactive EAS Function will facilitate i) developing an awareness and understanding of services available, ii) establishing annual priorities for services, iii) confirming performance indicators or indicating a need for modifications, iv) identifying ongoing data requirements, v) meeting federal GBA+ requirements, vi) addressing complex issues and opportunities arising, and vii) leveraging existing economic analysis, statistical information and advice work. Anchoring EAS workplans and services around the Departmental Results Framework and long-term priorities serves to underpin the of the EAS Function.

Management response

Across NHQ and the Regions, the EAS Function conducts a mix of longer-term anticipatory work, as well as a large amount of responsive work related to emerging and urgent policy and management issues. An opportunity exists to improve awareness of EAS services across the Department, and to better align longer-term projects with Program and Departmental priorities.

Link to larger program or departmental results (if applicable)

Addressing this recommendation will allow the EAS Function to support the Department’s progress towards addressing the mission-critical risk of “Information for Decision Making”, which spans all four core responsibilities.

Results statement
Result to be achieved in response to the recommendation
Milestones
Critical accomplishments to ensure achievement of result for PMEC’s approval
Completion date
month, fiscal year
DG responsible
Critical economic and socio-economic information gaps are being addressed. A department-wide annual planning process is in place (including consultations/discussions with all programs and regions), allocating EAS Function’s capacity across NHQ and Regions towards longer-term projects addressing critical information gaps for the Department while maintaining sufficient capacity for responsive work. April 2021 Robert Elliott, DG, EAS
Regional Directors, EAS
High quality, comprehensive, consistent and timely economic and socioeconomic information and advice is available across the Department to inform senior management decisions, departmental mandates, commitments, challenges and other priorities. Review completed of the positioning of EAS Functions within Departmental processes to ensure early engagement, as well as challenge function for all economic and socio-economic information being used to inform decision making. April 2021 Robert Elliott, DG, EAS
Regional Directors, EAS

Recommendation 2:

Recommendation:

It is recommended that the Senior Assistant Deputy Minister, Strategic Policy, ensure the Performance Information Profiles for EAS Function programs incorporate meaningful performance indicators that provide information significant to the achievement of results and management of the programs. 

Performance Indicators should address Core Responsibility #3 Marine Navigation as the current Performance Information Profile (PIP) does not cover this Core Responsibility. To do this, a fourth EAS program can be created or the existing programs can be amalgamated as one program in the Program Inventory. Specifically an all-encompassing program for the EAS Function covering all four departmental Core Responsibilities.

Rationale: 

Current indicators and targets speak to outputs (number of requests received and dealt with) not whether programs are achieving their intended results or the quality of results. They also do not provide insight on capacity to help inform workload and resource decisions. The EAS Function has for all four departmental Core Responsibilities.

Management response
Management accepts the recommendation to consolidate the economics programs, and to investigate opportunities for the creation of performance indicators that would provide senior management with actionable information on program performance. Additional efforts will be directed towards aligning EAS functions in both NHQ and regions with the consolidated economics program.

Link to the larger program or departmental results (if applicable)

N/A

Results statement
Result to be achieved in response to the recommendation
Milestones
Critical actions to ensure achievement of result for PMEC’s approval
Completion date
month, fiscal year
DG responsible
The consolidated economic program will clearly demonstrate how its functions and services support internal and external stakeholders.     Revision of the Department’s Program Inventory, consolidating the three economics program into one national-level Economics Program. September 2020 Robert Elliott, DG, EAS
Regional Directors, EAS
Andrea Cyr, DG, Planning, Results and Evaluation
Research, development, and testing of options for revised performance indicators for consideration by senior management. July 2020 Robert Elliott, DG, EAS
Regional Directors, EAS
Andrea Cyr, DG, Planning, Results and Evaluation
Revised Performance Information Profile approved by program DG and Head of Performance Measurement. July 2020 Robert Elliott, DG, EAS
Regional Directors, EAS
Andrea Cyr, DG, Planning, Results and Evaluation

7.0 References

Date modified: