Summary of the Evaluation of Grants and Contributions Programming in the Ecosystems and Oceans Science Sector

About the evaluation

The evaluation was conducted internally by the Evaluation Division of Fisheries and Oceans Canada (DFO) between April 2019 and June 2020. It covered the years 2016-17 to 2019-20 and examined: the Partnership Fund, which is a component of the Ocean and Freshwater Science Contribution Program (OFSCP); components of eleven other grant and contribution (G&C) programs within the Ecosystems and Oceans Science (EOS) sector; and, activities of the secretariat within the EOS sector that supports the delivery of all programs.

As an implementation evaluation, this evaluation was designed to help senior management understand what is working well and if adjustments are needed. Evidence was gathered using several lines of evidence, including interviews, a survey to recipients, a survey to DFO staff, one site visit, a project file review, and reviews of other relevant documents, websites and literature.

About the program

The Partnership Fund was created in 2016 with $3.5M in its first year and $2.5M in ongoing funds to promote and facilitate the development of scientific knowledge and related science activities by external organizations in areas aligned with DFO priorities. Since 2016, more G&C programs have been added to the EOS sector bringing the total value of sector G&Cs to around $35M at the time of the evaluation.

Due to the significant increase in contribution funding, a secretariat was set up to help support managers and other sector staff delivering the G&C programs . Further, the terms and conditions were revised in 2019 to better support and reflect the range of contributions being delivered in the sector. 

Publicly reported spending under the OFSCP reflects the exponential growth in G&C funding in the EOS sector

Description

This bar graph represents publically reported amounts spent under the Ocean and Freshwater Science Contribution Program and highlights the exponential growth in Grant and Contribution funding in the Ecosystem and Ocean Science sector from 2016-17 to 2018-19.

The amounts spent are: $4,273,033 in 2016-17, $15,432,471 in 2017-18, and $25,137,184 in 2018-19

Key findings

Activities are being implemented as intended.

Projects align with DFO core responsibilities and there are early signs that immediate and intermediate outcomes are being achieved. The funding programs are addressing gaps in the Canadian oceans and freshwater science community by: building fundamental understanding; informing the management and governance of resources; improving the quality of monitoring, data, and information management; and by understanding the impact of human activities.

96% of respondents to the recipient survey said that they had plans to share, or results were already being shared with the scientific community and respondents (81%) were also planning to disseminate their research more broadly with the Canadian public.

All contribution programs are supporting capacity building to some extent. Survey respondents, felt that yes, funded projects are contributing to the development of highly qualified personnel

Description

This donut charts represents the extent to which recipients and Fisheries and Oceans Canada's staff feel that funded projects are contributing to the development of highly qualified personnel.

93% of recipients agree that projects are contributing to the development of highly qualified personnel.

86% of Fisheries and Oceans Canada's staff agree that projects are contributing to the development of highly qualified personnel.

Projects funded under the Partnership Fund generate more scientific data and results in the public domain. DFO staff regularly access external information to support their work

Description

This bar graph shows the responses from Fisheries and Oceans Canada staff to a series of survey questions regarding the extent to which they use external data to support their work:
80% of staff use external science research to supplement Fisheries and Oceans Canada's data, 78% access data that Fisheries and Oceans Canada does not have, 75% use data to supplement a lack of Fisheries and Oceans Canada's capacity, 68% use data for decision making, 65% use data to inform scientific projects, and 60% use data to keep current in the field.

Recipients have had positive experiences and support. 

Most recipients were satisfied with the support they received from DFO, and the level of communication with regional and national contacts. Service standard targets related to acknowledging the receipt of applications, notification of funding decisions, and requisitions for payment were met or exceeded in 2017-18 and 2018-19. 

"The people that we worked with at DFO to help alleviate the issues with approval and funding transfers were incredible. They were accommodating and I have found them to be incredible contacts!"

Recipients complained it is hard to find information on the G&C programs.

It is cumbersome to locate G&C information on EOS sector grant and contribution programs due to a lack of online information. DFO staff and recipients said that learning about the funding opportunities mostly happens through word of mouth. This approach could put those not connected to existing networks at a disadvantage of learning about contribution programs. 

Description

This bar graph shows the responses from recipients to the survey question: To what extent was it easy to find information on the contribution programs?
8% of recipients responded not at all, 28% to a limited extent, 30% to a moderate extent, 18% to a considerable extent, and 7% to a great extent.

There are some internal challenges with the delivery of the G&C programs.

Staff in the EOS sector said that their roles and responsibilities are clear, and that the secretariat is responsive to questions. However, some issues exist. There are internal communication breakdowns both between staff associated with different contribution programs and also between National Headquarters and the regions. There is inconsistent messaging, and a lack of cohesion amongst the different parts of the organization in both G&C administration and literacy. The different contribution programs operate in silos which leads to duplications in the creation of tools (e.g. templates) or in processes such as financial tracking. Many employees indicated they do not have access to the guidance they need to work efficiently. 

Although the SharePoint database used to support staff provides a single storage space for data enabling access to information (e.g. project descriptions, contact information, project value, etc.), the following issues were identified:

• A limited number of staff have access to the database;
• Some information is stored outside of SharePoint; 
• Inconsistencies, particularly with financial information; and 
• Some information in SharePoint is incomplete reducing reliability.

The G&C secretariat within the EOS sector is making improvements to address challenges

The recent introduction of a national coordinators committee, and organization and delivery of training sessions to improve: 

• communications;
• overall G&C literacy; and,
• standardization of guidance, practices and tools.

As a possible long-term database solution, the secretariat is examining a new system to better support the sharing of consistent information across the EOS sector.

The outreach group in the EOS sector initiated a process to renew their webpages, including information on funding opportunities.

O&M funding available through the Partnership Fund is benefitting G&C programming in the EOS sector

The Partnership Fund was created with an operations and maintenance (O&M) component. DFO staff could not think of any drawbacks of these funds as they felt they contributed to the success of the projects. However, some internal staff facilitating Partnership Fund projects were not aware of the O&M component.

The analysis of O&M spending associated with the Partnership Fund shows that its use decreased over the first three years. However, the Ecosystems and Oceans Science sector used these funds for the G&C secretariat to help manage the exponential growth of G&C programming.

Contributions reach all recipient groups through collaboration with others.

Most of the funding for the Partnership Fund goes to three groups: Canadian academia, not-for-profit groups and research organizations. However, the contributions have greater reach through collaborations with others. For example, 185 unique collaborators were identified for the Partnership Fund and while few businesses or Indigenous groups, for instance, were recipients, a greater number were involved as partners with funded recipients. 

% of respondents to the recipient's survey that had partners in their projects from these eligible groups.

Description

This bar graph represents the proportion of total projects in which eligible recipient groups were collaborators.

Out of the total number of projects Port Authorities collaborated in 17% of them, Foreign Academic Institutions collaborated in 19% of them, Foreign governmental and non-governmental organizations collaborated in 17% of them, Canadian not-for-profit organizations collaborated in 31% of them, Community groups collaborated in 19% of them, Business and Industry collaborated in 40% of them, Canadian Governments collaborated in 33% of them, Indigenous organizations or groups collaborated in 39% of them, Canadian non-governmental organizations collaborated in 35% of them and Canadian post-secondary institutions collaborated in 79% of them.

Some eligible recipients could face barriers.

Academic organizations may have greater capacity to apply to contribution programs, while Port Authorities, international, and Indigenous organizations might experience barriers to accessing funding due to a lack of capacity (staff and time) and/or lack of networks to make them aware of funding opportunities. Additionally, early career researchers would not have well developed networks, reducing their likelihood of learning about funding opportunities.

"Smaller recipient organizations have less staff and therefore less capacity to complete applications and reporting requirements."

"As these networks tend to develop over a career, it is likely that early career individuals may not have the same opportunities. Enhanced advertising and notifications would address this issue."

Good practices for the delivery of Grant and Contribution programs.

1. Pursue excellence in service delivery and client-focused programs.
2. Provide internal support at program level.
3. Develop streamlined, consistent and standardized processes and tools. 
4. Establish a process for risk-based assessment of projects.
5. Support performance management and reporting.

Conclusions

Overall, the G&C secretariat within the Ecosystems and Oceans Science sector is responding well to the increase in transfer payment programs that have been added to the sector since 2016. It is demonstrating continuous improvement by identifying lessons learned and initiating changes to address internal issues, most of which align with challenges uncovered by the evaluation. Continuing to advance these initiatives will improve the delivery of grant and contribution programs across the EOS sector.