Internal Audit Report
Audit of Partnerships and Collaborative Arrangements with Other Government Departments and External Groups
Date: May 26, 2017
TABLE OF CONTENTS
- 1.0 EXECUTIVE SUMMARY
- 2.0 BACKGROUND
- 3.0 AUDIT OBJECTIVE
- 4.0 AUDIT SCOPE
- 5.0 AUDIT APPROACH
- 6.0 AUDIT FINDINGS
- 7.0 RECOMMENDATION AND MANAGEMENT ACTION PLAN
- 8.0 AUDIT OPINION
- 9.0 STATEMENT OF CONFORMANCE
- APPENDIX A: LINES OF ENQUIRY AND AUDIT CRITERIA
- APPENDIX B: LIST OF ACRONYMS
- APPENDIX C: TYPES OF ARRANGEMENTS
1.0 EXECUTIVE SUMMARY
The objective of this audit was to provide reasonable assurance that an adequate and effective management control framework is in place for managing partnerships and collaborative arrangements between the Department and other government departments and external groups. The scope included an assessment of the adequacy and effectiveness of the management control framework in place, including monitoring the execution of partnership and collaborative arrangements. It also included an assessment of measures in place to identify and mitigate potential risks stemming from partnerships and collaborative arrangements.
The audit focused on the Department’s role in the delivery of partnerships and collaborative arrangements (partnerships and collaborations) including memoranda of understanding (MOU), service agreements, letters of understanding, interdepartmental letters of agreements, collaborative agreements, partnership agreements and cost sharing agreements. The audit did not look at grant and contribution agreements, as these have an established and separate management control framework.
Why This is Important
The Department collaborates with other government departments and external groups in order to enhance the delivery of its programs and to help ensure it is able to meet its mandate and objectives in a manner consistent with stakeholder expectations, including the public. Without a strong management control framework in place, the Department is at risk of not having partnerships and collaborations that support the Department’s programs, mandate and objectives. There is also increased risk of inefficient operations and lost opportunities. With an increased emphasis on partnering with other parties, as mentioned in the 2015 Minister’s mandate letter, a solid management control framework is important to ensure that objectives are met. Partnerships and collaborations contribute, on average, $10 millionFootnote 1 annually to the Department, in addition to in-kind contributions and knowledge sharing. A strong management control framework is essential to implement these arrangements in a way that ensures value for money, transparency and accountability.
An effective management control framework consists of strong governance, an effective risk management system that is supported by controls based on those risks, a strong information system in place to provide management with the information they need for decision making, and effective monitoring that ensures issues are identified and addressed on a timely basis. The audit found weaknesses in each of these areas.
Specifically, for governance, the audit found that while oversight exists at the program and regional level for portions of the partnerships and collaborations processes, there is no departmental oversight for partnerships and collaborations. Also, while roles and responsibilities for collaborative agreements are generally defined, documented, and understood, roles, responsibilities and accountabilities for MOU are not documented or defined. Finally, employees do not have access to sufficient clear information to carry out their roles and responsibilities and do not have a good understanding of the tools and templates.
In the area of risks and controls, it was found that risks are assessed for some individual partnerships and collaborations; however, no effective risk management system is in place to identify, analyze and manage partnership and collaboration risks at a departmental level. Further, the audit found that controls are not well aligned with key risks and do not demonstrate consideration of costs versus benefits.
Similarly, the information system being used for partnerships and collaborations does not contain sufficient, accurate and reliable information to inform decision making related to partnerships and collaborations.
Finally, while monitoring of collaborative agreements occurs in the application phase, there is no monitoring system in place at the departmental level, monitoring is not clearly linked to departmental objectives and there is no evidence that identified issues are remediated in a timely manner.
The audit found that an adequate and effective management control framework is not in place for managing partnerships and collaborative arrangements between the Department and other Government Departments and External Groups.
While the recommendation is to the Chief Financial Officer (CFO), this should not be interpreted as meaning the CFO alone is accountable for all findings identified in this report. Rather, the recommendation is addressed to the CFO, as the findings are not sector or region specific and as such, there is a need for a departmental level response.
Management is in agreement with the audit findings, has accepted the recommendation included in this report, and has developed a management action plan to address it. The management action plan has been integrated in this report.
The report for the Audit of Partnerships and Collaborative Arrangements with Other Government Departments and External Groups was presented at the Departmental Audit Committee on March 10, 2017. The report was recommended for approval by the Departmental Audit Committee on April 10, 2017 and approved by the Deputy Minister on May 26, 2017.
According to the Treasury Board SecretariatFootnote 2, as departments face the challenge of providing quality services with declining resources, interest is growing in partnerships and collaborations with other levels of government, the private sector and non-governmental organizations. There is also strong political will to enhance these areas as seen in the 2015 mandate letter to the Department’s Minister where the importance of partnerships and collaborations was emphasized and several of the priorities require partnerships or collaborations with other departments and/or organizations outside government.
These arrangements are usually designed to share the costs, risks and benefits of particular initiatives, while at the same time increasing the involvement of the clients being served and enhancing the general level of goodwill with all parties. In all cases, the partnerships or collaborations must support overall departmental objectives and priorities.
Partnerships and collaborative arrangements (partnerships and collaborations) are not without challenges. Consideration must be given to the Department’s responsibilities concerning a number of issues: problems of definition, management, financial and contracting, human resources, official languages, legal, and political.
While partnerships and collaborations can be formalized through a number of different tools (see Appendix C for detailed listing), they fall under three main categories; grants and contributions, memoranda of understanding (MOU) and collaborative agreements. As grants and contributions have a defined framework in place that is partially directed by central agency policies and procedures, they were not included in the scope of this audit.
MOU cover all arrangements with other departments and levels of government, including international, where funds are not involved. They are also not intended to be legally binding on either party.
Collaborative agreements cover all other types of arrangements, including all arrangements with non-government entities and those with other governments or departments where funds are involved.
For the purpose of this report, the term ‘partnerships and collaborations’ is used to encompass both collaborative agreements and MOU.
To put in place controls related to collaborative agreements and MOU, the Department developed a number of policy documents starting in 2008. Currently, the main policy document for MOU is the Guidelines for Developing and Managing Memoranda of Understanding (MOU Guidelines) which were put in place in 2015 and updated in August 2016. While identified as guidelines, this policy document contains mostly mandatory requirements for the Department to follow. For collaborative agreements, the main policy document is the Directive on Collaboration which was introduced in 2012 and updated in 2014 and again in November 2016. In addition to these two policy documents, the Department has in place a number of other guidelines, processes, tools and templates meant to support the operation of partnerships and collaborations such as the Guidelines for entering or amending collaborative agreements and managing financial contributions. While there are some central agency policies, directives and guidelines that address specific but limited aspects of some partnerships and collaborations, such as the TBS’ Directive on Specified Purpose Allotments, there are none that address partnerships and collaborations as a whole.
The Department is already active in partnerships and collaborations with about $10 millionFootnote 3 annually being provided to the Department from external groups for partnerships and collaborations in 2014-2016, not including in-kind and knowledge sharing contributions.
Based on information available at the time of the audit in iAgree, the Department’s partnerships and collaborations information system, 1% of partnerships and collaborations were with the Canadian Coast Guard (CCG), 12% with the Ecosystems and Fisheries Management Sector, and 87% with the Ecosystems and Oceans Science Sector.
The role of partnerships and collaborations is expected to continue to increase in the Department with Budget 2016 announcing funding for oceans and freshwater sciences, including an annual investment of $5 million for science partnerships and collaborationsFootnote 4. In response to this funding, an Office of Partnerships and Collaboration has been established under the Strategic and Regulatory Science Directorate, Ecosystems and Oceans Science (EOS) Sector to manage the fund and leverage capacity (financial and non-financial).
3.0 AUDIT OBJECTIVE
The objective of this audit was to provide reasonable assurance that an adequate and effective management control framework is in place for managing partnerships and collaborative arrangements between the Department and other government departments and external groups.
4.0 AUDIT SCOPE
The scope of the audit included an assessment of:
- The adequacy and effectiveness of the management control framework in place to manage the Department’s partnerships and collaborations with other government departments and external groups, including monitoring the execution of those arrangements; and
- The measures in place to identify and mitigate potential risks stemming from partnerships and collaborations.
The audit focused on the Department’s role in the delivery of partnerships and collaborations including MOU, service agreements, letters of understanding, interdepartmental letters of agreements, collaborative agreements, partnership agreements and cost sharing agreements. The audit did not look at grant and contribution agreements, as these have an established and separate management control framework.
5.0 AUDIT APPROACH
The audit team carried out its mandate in accordance with Treasury Board’s Policy on Internal Audit and the Internal Audit Standards for the Government of Canada and the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. These standards require that the engagement be planned and performed in such a way as to obtain reasonable assurance that the objective of the engagement is achieved. The audit employed various techniques including a risk assessment of the audit entity, interviews, case studies, as well as reviews and analysis of documentation and information.
The case studies were selected judgmentally based on value, region, type of process and recommendations. Specifically, three of the ten case studies were chosen based on recommendations of interviewees; five were chosen from iAgree to provide representation across regions, types and dollar values; and the final two were chosen from EKMEFootnote 5 to check for completeness of information contained in iAgree.
6.0 AUDIT FINDINGS
This section provides the observations resulting from the audit work carried out. The recommendation addressing these audit findings is located in the section following these observations. While the audit was conducted based on the lines of enquiry and audit criteria identified in the planning phase, this report is structured along the following main themes:
- Risks and Controls;
- Information Systems; and
Based on the audit work performed and our professional judgment, the risk associated with each observation was rated using a three-point scale. The risk ranking (high, moderate, low) is based on the level of potential risk exposure we feel may have an impact on the achievement of Fisheries and Oceans Canada objectives, and is indicative of the priority Management should give to the observation. The following criteria were used in determining the risk exposure:
|High||Controls are not in place or are inadequate.|
|Compliance with legislation and regulations is inadequate.|
|Important issues are identified that could negatively impact the achievement of program/operational objectives.|
|Moderate||Controls are in place but are not being sufficiently complied with.|
|Compliance with central agency/departmental policies and established procedures is inadequate.|
|Issues are identified that could negatively impact the efficiency and effectiveness of operations.|
|Low||Controls are in place but the level of compliance varies.|
|Compliance with central agency/departmental policies and established procedures varies.|
|Issues identified are less significant but opportunities that could enhance operations exist.|
6.1 GOVERNANCE (MODERATE RISK)
Governance is the combination of processes and structures implemented by management to inform, direct, manage and monitor the activities of the Department toward the achievement of its objectives. Governance contributes to the strategic direction, oversight, decision-making, and accountability for an organization to successfully meet its objectives.
Roles and Responsibilities
Roles and responsibilities for MOU and accountabilities for partnerships and collaborations are not defined, documented or understood
It was expected that roles, responsibilities and accountabilities would be defined, documented, communicated and understood for all types of partnerships and collaborations.
The audit found that the Department has a directive and guideline in place that identifies roles, responsibilities and processes for collaborative agreements. For example, the Chief Financial Officer’s responsibilities include:
- directing the development, implementation and communication of the Directive;
- developing training material, templates, guidelines, procedures and controls for the implementation of the Directive;
- establishing a robust framework of effective management practices and controls to mitigate risks and provide assurance with respect to compliance with legislation, regulations, policies, directives and delegated authorities, for departmental Collaboration; and
- monitoring internal management practices and taking necessary action(s) to maintain a robust environment of internal control and to address any instance of non-compliance.
The Department also has guidelines for MOU; however, roles and responsibilities are not clearly defined. Further, while the MOU Guidelines identify some ‘accountable directorates or regions’, they do not specifically indicate what the accountabilities are or provide mechanisms for holding those directorates or regions accountable.
These findings were confirmed by interviewees who, while generally able to articulate what their roles and responsibilities were in relation to collaborative agreements, were not able to identify their accountabilities beyond general financial management.
Finally, the Chief Financial Officer (CFO) Sector has a Center of Expertise (COE) in place along with the regional finance groups to help fulfill some of the roles and responsibilities of the CFO regarding partnerships and collaboration. This COE currently consists of one employee. While the COE sees its role as primarily related to providing financial expertise and not covering MOU, interviews found that others in the Department have a much broader expectation of the COE, including all types of advice and direction related to partnerships and collaborations. The Strategic Policy Sector used to have responsibility for MOU, which may be contributing to the misunderstanding of roles, responsibilities and accountabilities.
This finding is important because defined, documented and understood roles and responsibilities help ensure that processes are completed as intended and reduce the risk of duplication and inefficiencies. Similarly, clear accountabilities help ensure agreements are properly implemented in line with policies and procedures and that goals and objectives are achieved.
Oversight at the program or regional level is not consistent across all phases of the partnerships and collaborations process and there is no oversight at the departmental level
It was expected that an oversight function for partnerships and collaborations has been established and is operating effectively.
Currently, there is no Treasury Board or departmental policy that requires oversight for partnerships and collaborations at the Departmental level. The DFO Directive on Collaborations requires that a program level oversight function be established for reviewing proposed collaborative projects to ensure they comply with the directive. The audit found that the Department has put in place program oversight committees at both the program and regional levels to provide oversight on individual agreements in the application phase of the collaborative agreement process, as required in the Directive on Collaboration. However, the level of oversight provided by these committees was found to be inconsistent across regions and did not provide oversight for the other phases of the collaborative agreement processFootnote 6.
Within the Ecosystems and Oceans Science (EOS) Sector, where the majority of agreements related to partnerships and collaborations in the Department reside, the sector recently created the Office of Partnerships and Collaboration to provide oversight for these types of agreements. While this office is still being established, it is expected to act as a hub for partnerships and collaborations for EOS, by providing employees expert advice and assistance in moving partnerships and collaborations through all phases of the agreement process. Prior to the creation of this Office, the sector’s only oversight mechanism was the Science Executive Committee. However, the audit found that this committee provided very little oversight over partnerships and collaborations.
While the Office of Partnerships and Collaboration is specific to the EOS sector, the Canadian Coast Guard and other sectors within DFO have expressed interest in expanding their use of partnerships and collaborations, which would increase the need for a broader oversight function.
Overall, the audit found that while there is some oversight at a program or sector level, there is no oversight function or mechanism at the departmental level that is able to monitor issues or practices from across the Department to ensure consistency and co-ordination in the management and implementation of agreements.
This finding is important because effective departmental oversight can help ensure partnerships and collaborations are aligned with departmental priorities and objectives. It could also improve the Department’s ability to provide effective strategic direction, allocate appropriate resources, analyze activities related to exposure to risk and determine suitable mitigation strategies. Finally, departmental level oversight reduces the risk that broad cross-region or sector issues will be missed.
Information, Tools and Processes
Employees do not have access to sufficient, clear information to carry out their roles and responsibilities and tools and processes are not well understood
It was expected that employees would have the necessary training, tools, resources and information needed to successfully carry out their roles and responsibilities with respect to partnerships and collaborations.
The audit found that the Department has policies, procedures, tools and templates related to partnerships and collaborations that provide information and guidance. The Department has also set up two intranet sites, one for collaborative agreements and another for MOU. These intranet sites provide easy access to policies, procedures, tools and templates.
The Department also has two groups identified as providing support to partnerships and collaborations: a Center of Expertise (COE) within the CFO Sector, and the Office of Partnerships and Collaboration within the EOS Sector. While the COE and Office of Partnerships and Collaboration provide access to expertise in the area of collaborative agreements, regions and the COE identified a lack of resources as one of the main limiting factors in carrying out partnerships and collaborations roles and responsibilities. Interviewees mentioned that they often have difficulty with the lengthy partnerships and collaborations processes, as it is not part of their regularly assigned work. Further, most programs and regions do not have staff dedicated to partnerships and collaborations to help move the arrangements through the processes. As a result, there are often delays in the completion of the partnerships and collaborations process, due to a lack of understanding of the steps to follow in the process.
Finally, the audit found that there was a general lack of understanding on how to use the tools and templates. Based on the case studies and interviews, it was found that employees do not understand the process well, especially which templates should be used. Specifically, there are often lengthy discussions on which tool to use. Also, the array and titles of current templates may be adding to the confusion for users. For example, collaborative agreement templates with other departments or levels of government are called MOU templates but are expected to go through the collaborative agreements process according to the Directive on Collaboration. This is not clear to many users and as a result, they are not consistently using the right template, process or both. This has also contributed to delays in finalizing some agreements.
As part of the case studies, the time it took to complete each step of the approval process (from application to the signing of the agreement) was documented along with the time it took to get through the entire process up to signing of the agreement or MOU. Overall, it was found that the agreement that took the least amount of time was a collaborative agreement with another federal department that was completed in 115 days and the longest was a collaborative agreement with a non-government organization that took 445 days to complete. The main causes of the delays were related to confusion over the process and or which template to use, negotiations with external collaborators, navigating through the Department’s and the external collaborator’s distinct processes and templates, Oversight Committee approvals and the requirement for reviews by subject matter experts.
When there are adequate resources and clear guidance available on processes, tools and templates employees are able to successfully fulfill their roles and responsibilities, and enter into agreements in a timely basis to benefit from partnerships and collaborations that support the Department’s goals and objectives.
6.2 RISK AND CONTROLS (MODERATE RISK)
The Committee of Sponsoring Organizations of the Treadway Commission (COSO) defines risk as the possibility that an event will occur and adversely affect the achievement of objectives. Risk assessment involves a dynamic and iterative process for identifying and assessing risks to the achievement of objectives. Control activities are the actions established through policies and procedures that help ensure that management’s directives to mitigate risks to the achievement of objectives are carried out. Controls are to be based on risks identified as being unacceptable and should be designed to bring the risk to an acceptable level.
Risks are assessed for some individual partnerships and collaborations; however, no effective risk management system is in place to identify, analyze and manage risks at the departmental level
It was expected that there would be a formal risk management system in place for partnerships and collaborations that would identify, assess and manage risk.
According to Treasury Board’s Guide to Integrated Risk Management, risk management “involves a systematic approach to setting the best course of action under uncertainty by identifying, assessing, understanding, making decisions on, and communicating risk issues.” No evidence of a systematic approach to risk for partnerships and collaborations was found. At the departmental level, the Corporate Risk Profile (CRP) was identified as the main risk management tool. While the CRP does identify one risk area related to partnerships and collaborations, that of “program delivery risk”, only two of the identified action plansFootnote 7 address items related to partnerships and collaborations and in both cases were for specific programs. Also, while the CRP is an annual exercise to identify the top risks for the Department, the risk mitigation strategies are not actively monitored and the CRP is not revisited until it is updated for the following year. As such, the CRP is not an effective risk management tool for partnerships and collaborations. Aside from the CRP, no listing of key risks, or evidence of risk assessments for the partnerships and collaborations functions as a whole were found.
Further, the audit found that while risk assessments were completed for some agreements and MOU, they assessed project specific risks, and were not rolled up to an overall risk assessment. Specifically, the DFO MOU Guidelines require that an Impact Assessment be conducted for each MOU to ensure that risks associated with the MOU are identified and considered. While the Impact Assessment has some specific risk requirements to consider at the departmental level, when a separate sample of 5 impact assessments from EKME was selected and reviewed, inconsistencies in how they were completed and the rigour undertaken to complete the assessments were identified.
Similarly, the Directive on Collaboration requires that the impact of collaborative agreements on the Department’s risk profile be properly evaluated, addressed and mitigated and Senior Management be informed of the impacts of risks that cannot be fully addressed or mitigated. However, there are no tools or processes to ensure that this objective is met. Further, while there are tools in place to address risks, namely the Conflict of Interest Risk Questionnaire, the Internal Risk Assessment Questionnaire, and the Project Risk Analysis tool, these are all project specific and are not consistently completed.
Specifically, in the case studies, it was found that these forms were completed about 75% of the time, and there were inconsistencies in how they were completed, their level of rigour and accuracy.
This finding is important because identifying departmental risks helps ensure controls are appropriately based on those risks and allows management to determine which risks to accept, mitigate, or avoid.
Controls are generally not aligned with key risks nor do they consider costs versus benefits
It was expected that controls would be aligned with key risks identified in the partnerships and collaborations processes and that there would be evidence of consideration of cost versus benefits for the various controls, processes and tools in place.
Since no departmental level risk assessments for partnerships and collaborations were found, in order to test the alignment of controls to risks, the partnerships and collaborations processes were examined to identify possible key risks and compare them to the controls in place. Based on this comparison it was determined that controls were not consistently based on key risks. Specifically, gaps in controls and some duplication were found. No other documentation was found to show that the controls were risk-based. This was supported by interviews with those involved in creating or overseeing the current key controls, with all five of those asked stating that the key controls, including those defined in the Directive and Guidelines, were not developed based on risks or a risk assessment. Also, while the Directive states that monitoring of the Directive “will be based on the principles of Risk Management”, no evidence was found to support that this was occurring.
Further, no evidence was found to demonstrate that the level of risk associated with an arrangement was considered in determining the process to be followed. Currently, all collaborative agreements must go through the same process, regardless of whether it is a renewal of an agreement with a long-term collaborator, an agreement with another federal department, or a brand new agreement with an organization the Department has never dealt with in the past. The only part of the process for collaborative agreements that varies between agreements is the signing authorities, which are based solely on dollar amounts and the expert groups that need to be contacted. Similarly, the level of controls for MOU are comparable to those seen in the collaborative agreement process despite the fact that MOU are with other departments or governments and do not include any transfer of funds or in-kind contributions. Finally, it was found that MOU must go through many of the same controls regardless of whether the MOU is with another Federal Department or an International Government, despite significantly different risk levels.
In examining costs versus benefits, no evidence was found during the audit to demonstrate that costs and benefits were considered in developing and implementing controls. This was supported by interviewees who expressed that many of the controls in place imposed costs on the process beyond the benefit they produced. These included controls such as the requirement for all partnerships and collaborations to obtain approval to negotiate before discussions with the potential partner begin and the need to have all collaborative agreements go through the regional or sector oversight committees as well as regional or sector management. Although no evidence of cost benefit analysis was found, the financial signing authority levels for collaborative agreements were amended during the audit period to allow for increased signing authorities for collaborative agreements. While there was no documented evidence to demonstrate that this was based on a formal cost benefit analysis, there was evidence in interviews that previous signing authority levels placed undue costs on the process with respect to the time and effort for the benefit it produced. This suggests that the decision to increase signing authorities was consistent with better alignment of costs with benefits.
This finding is important because strong alignment of controls to risks reduces the likelihood of having an ineffective and inefficient system of controls.
6.3 INFORMATION SYSTEMS (MODERATE RISK)
COSO states that the presence of an effective information system is a key component of a strong internal control structure. Access to relevant, accurate and timely information is required to meet the Department’s objectives. Well-functioning information systems generate better decision-making, better compliance with policies and fewer problems.
The information system in place does not provide management with accurate and reliable information to support decision making in the management of Partnerships and Collaborative Arrangements
It was expected that the Department would have in place an effective information system that would house partnerships and collaborations’ data and would provide management with accurate and complete information for decision-making. The main information system for partnerships and collaborations in the Department is iAgree, a standalone database.
To test for completeness of iAgree, the ten case studies were examined to see if they were in iAgree along with the required tombstone data and mandatory documentation in accordance with the MOU Guidelines and Directive on Collaboration. However, it was found that four of the ten case studies had not been entered into iAgree. While the percentage of arrangements missing from iAgree cannot be applied to the database as a whole; additional testing based on MOU and contribution agreements found in EKME also revealed significant numbers of missing arrangements. Overall, it was not possible to determine how many agreements may not be in iAgree, as the Department does not currently have a reliable list of all partnerships and collaborations.
Further, for those arrangements that are in iAgree, based on the case studies it was found that although the tombstone data and the signed collaborative agreements and MOU were uploaded, other mandatory documents, such as the application form and amendments, were not always uploaded. Similarly, financial data in some fields was not consistently entered correctly, such as entering the full value when it was supposed to be entered in thousands of dollars.
Also, based on interviews, it was found that several regions maintain their own spreadsheets and files to monitor their arrangements because iAgree does not have sufficient functionality for their reporting and monitoring needs. Specifically, iAgree does not provide recent financial data, permit easy extraction of information in a useable format, or contain all the fields that are needed by the regions to effectively manage the agreements. Interviewees also mentioned that iAgree had technical issues including not properly refreshing after viewing files and retaining previous information when validating or making changes to an agreement. Finally, some interviewees stated that there are other data repositories that are used in the Department that could be more useful and allow for more inclusion of key documents and correspondence. Specifically, the Aboriginal Programs & Governance Information System, which houses contribution agreements along with basic client relationship management functions and performance measurement tracking, and iContribute, which is the departmental database used to manage Grants and Contributions, were mentioned.
These findings are important because access to relevant, accurate and timely information, allows management to make informed decisions in support of departmental goals and objectives. Further, a reliable information system enables effective risk management and ensures compliance with policies.
6.4 MONITORING (MODERATE RISK)
Monitoring involves evaluating partnerships and collaborations against criteria established by regulators or management. Effective monitoring includes ensuring that deficiencies are communicated to management and are addressed in a timely manner. Monitoring promotes better alignment with departmental goals and objectives and that issues are identified and addressed in a timely manner. Finally, monitoring ensures that individual partnerships and collaborations, as well as partnerships and collaborations as a whole are being carried out according to plan, are appropriately managed and are on track to meet objectives.
Monitoring of collaborative agreements occurs in the application phase; however, other than monitoring of financial contributions, there is no monitoring system in place, monitoring is not linked to departmental objectives and the monitoring that is taking place does not ensure identified issues are remediated in a timely manner
It was expected that a monitoring system linked to departmental objectives would be in place.
The audit found that the MOU Guidelines require that indicators be developed to assess the agreement’s progress in accomplishing its objectives and that costs and resources to implement and maintain the MOU be tracked. In addition, it requires that a short progress report be prepared annually and at the end of the agreement detailing the activities and achievements of the MOU. However, in the case studies, while all three MOU type arrangements should have had a completed progress report, none did. Further, there was no evidence that progress reports, or issues identified within a progress report, are consistently reported to appropriate management.
Similarly, the Directive on Collaboration identifies the CFO as being responsible for “monitoring internal management practices and taking necessary action to maintain a robust environment of internal control to address any instance of non-compliance” and that regional directors of finance and senior finance managers implement a standardized monitoring program for collaborative agreements to ensure they comply with the Directive. During the audit, it was found that the CFO Sector monitors the funds received by the Department from cost-sharing agreements and specified purpose account agreements to ensure funds are appropriately accounted for and managed; however, no evidence of departmental monitoring was found related to internal management practices and compliance, neither was any implementation of a standardized monitoring program by the regional directors and senior finance managers found.
Further, inconsistent monitoring of the collaborative agreements was found in the case studies. Of the seven collaborative agreements, four had progressed to a point where a progress report should have been completed; however, only three of those four had completed progress reports. Also, none of these cases had any evidence that the progress reports were used or reported beyond the project authority to allow for identification of common issues or ensure expected results were achieved.
Some alignment of monitoring with departmental objectives was found in that the oversight committees review collaborative agreement applications to ensure they are aligned with one of the Department’s business lines. However, there was no evidence of monitoring to ensure progress or results were in line with departmental objectives. Further, all monitoring was of individual arrangements, no evidence of monitoring of partnerships and collaborations as a whole for alignment with departmental objectives was found.
It was expected that issues identified as a result of monitoring processes would be corrected in a timely manner. The MOU Guidelines state that if areas of improvement are identified during progress reporting, there may be a requirement to revisit the MOU to determine how to make it more effective. Also, the annual MOU Monitoring report requires that remedial measures be identified for deficiencies for each MOU. The Directive on Collaboration does not provide any direction regarding issue identification and remediation and there was no evidence found in the audit of issues being raised from monitoring of collaborative agreements or MOU. Finally, no evidence was found to suggest that issues identified for MOU and collaborative agreements are being consolidated to identify overarching trends for the Department. According to interviews with senior management, there is no formal mechanism in place to ensure that issues are identified and remedied in a timely manner.
This finding is significant because a monitoring system in place that is linked to Departmental objectives and remediates issues in a timely manner helps ensure alignment with departmental goals and objectives.
7.0 RECOMMENDATION AND MANAGEMENT ACTION PLAN
It is recommended that the CFO coordinate a working group of sectors and the CCG to develop a management control framework for partnerships and collaborations. This control framework should ensure there is an effective information management system in place and include a streamlined suite of tools and processes that are risk based and balance cost versus benefits.
Management Response: The CFO agrees to coordinate and lead a working group of sectors and the CCG to develop a management control framework for partnerships and collaborations. The working group will review existing tools, processes and systems to ensure an effective information management system is in place and a streamlined, risk-based suite of tools and processes that balance cost versus benefits is implemented.
Target Completion Date: March 31, 2018
8.0 AUDIT OPINION
The audit found that there are opportunities for improvement to ensure that the Department has an adequate control framework to support partnerships and collaborative arrangements. While there is oversight for individual MOU and agreements, governance at the departmental level can be improved to ensure that they are properly managed. Opportunities exist to enhance the partnerships and collaborations risk management system and approval process to ensure that controls are risk based. The monitoring program at the agreement and departmental level could also be strengthened to ensure issues are identified in a timely manner.
9.0 STATEMENT OF CONFORMANCE
In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The extent of the examination was planned to provide a reasonable level of assurance with respect to the audit criteria. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with Management. The opinion is applicable only to the entity examined and within the scope described herein. The evidence was gathered in compliance with the Treasury Board Policy and Directive on Internal Audit. The audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Quality Assurance and Improvement Program. The procedures used meet the professional standards of the Institute of Internal Auditors. The evidence gathered was sufficient to provide Senior Management with proof of the opinion derived from the internal audit.
APPENDIX A: LINES OF ENQUIRY AND AUDIT CRITERIA
The audit criteria are presented in the table below, by audit line of enquiry.
|Line of Enquiry 1 – Governance (Control Environment)|
|Criterion 1.1: An oversight function for Partnerships and Collaborative Arrangements has been established and is operating effectively.|
|Criterion 1.2: Roles, responsibilities, and accountabilities are defined, documented, communicated, and understood.|
|Criterion 1.3: Employees have the necessary training, tools, resources, and information to successfully carry out their responsibilities with respect to Partnerships and Collaborative Arrangements.|
|Line of Enquiry 2 – Risk Management|
|Criterion 2.1: An effective risk management system is in place to identify, analyze and manage risk, involves appropriate levels of management, and considers the acceptable tolerance level in relation to objectives.|
|Line of Enquiry 3 – Internal Control System and Activities|
|Criterion 3.1: Control activities are aligned with key risks, consider costs versus benefits, and are conducted in a timely and consistent manner.|
|Line of Enquiry 4 – Information and Communication|
|Criterion 4.1: Information systems in place provide management with accurate and reliable information to support decision making in the management of Partnerships and Collaborative Arrangements.|
|Line of Enquiry 5 – Monitoring|
|Criterion 5.1: A monitoring process or system is in place that is linked to departmental objectives and identifies and remediates deficiencies in a timely manner.|
APPENDIX B: LIST OF ACRONYMS
- Canadian Coast Guard
- Chief Financial Officer (Sector)
- Center of Expertise
- Corporate Risk Profile
- The Department:
- Fisheries and Oceans Canada
- Ecosystems and Oceans Science Sector
- Memorandum of Understanding
- Science Executive Committee
APPENDIX C: TYPES OF ARRANGEMENTS
During the audit, it was found that there are a number of different names and types of arrangements that are being used for partnerships and collaborations. While the definitions and categorization of these arrangements is not consistent within the Department, or by central agencies, the table below provides a general summary to help identify the main types and also what is, and is not, partnerships and collaborations.
|Types of Partnerships and Collaborations||When it applies||Includes/May also be called||Key Policy(ies)/Directive(s)|
|Memoranda of Understanding (MOU)||Arrangements with other government departments or other levels of government (including First Nations and International) where no funds are exchanged and it is not meant to be legally binding.||
|Collaborative Agreement||Non-financial and financial arrangements, except for arrangements with other government departments or levels that fall under the MOU definition above.||
|Transfer Payments (Grants and Contributions)||Arrangements where funds are provided to an organization outside of the Department. Must be done through a Treasury Board approved Grant and Contribution Program and comply with the approved terms and conditions.||
|Service Agreements (These are not partnerships and collaborations when their sole purpose is an exchange of funds for a service)||When departments provide or receive internal support services to or from one or more other department(s). Can include the provision of those services through collaboration among departments. Can involve fund transfers.||
|Non-Partnerships and Collaborations:|
|Types of Partnerships and Collaborations||When it applies||Includes/May also be called||Exceptions|
|Service Level Agreement (SLA)||Agreements internal to the Department (between sectors, programs, etc…)||
||Can also be with other government departments in which case they fall under Partnerships and Collaborations – see Memoranda of Understanding|
- Date modified: