Interpretation Policy: Evaluation Report on implementing improvements

Executive summary

The evaluation covered the progress of the Department of Fisheries and Oceans (DFO) performance in implementing improvements to its Interpretation Policy from March 2015 to March 2017. The evaluation was undertaken to fulfill the Department’s continued commitments to the Red Tape Reduction Initiative.

Purpose and scope

The purpose of this evaluation was to assess the performance of DFO in implementing the improvements to the Department’s regulatory information and guidance documents as per the recommendations made in the Regulatory Interpretation Policy: Improvement Report on Stakeholder Consultation (Improvement Report), which was published in March 2015. The three areas for improvement identified in the report were: (1) Implementing a flexible client service model; (2) Streamlining how feedback is received; and, (3) Response to stakeholders and revisions of information documents.

As well, the evaluation looked at the relevance and applicability of the metrics identified in the Improvement Report to assess DFO’s progress in implementing these improvements.  

The methodology used in this evaluation included the development of an internal departmental survey that was designed to collect information on the Department’s interactions with the public and stakeholders on the Department’s guidance documents/interpretation policies, and understand the mechanisms used to facilitate these interactions. Responses to this internal departmental survey were used to inform the evaluation.

Conclusions

This section presents the evaluation results of the four recommendations that were made by the Improvement Report.

Recommendation #1:  “Implementing a flexible client service model that takes into consideration how Canadians are made aware of information, and developing Frequently Asked Questions that anticipate stakeholder concerns.”
The evaluation concluded that this recommendation was fully implemented.  Survey results indicated that communication plans and frequently asked questions were developed and published for all new regulatory initiatives, regulatory amendments, and policy instruments.   

Recommendation #2: “Utilizing an electronic feedback form that is linked to each information or guidance document so that user feedback is received in a streamlined manner, and feedback can be directly connected to each program or subject matter.”

The evaluation concluded that this recommendation was not adopted by the Department. Therefore, all metrics associated with measuring its implementation were found to be not applicable.

Although the electronic centralized public feedback mechanism was not developed, public input on departmental regulatory information and guidance documents continued to be gathered by the various programs/regions. However, the feedback was handled on an ad hoc basis, making it difficult to gather data to provide for this evaluation.

Moreover, the internal departmental survey revealed that, due to the lack in centralization of how public input is received and handled at Fisheries and Oceans Canada, the quantitative data to support a conclusion on the frequency of interactions, as well as the nature of the input received, was not recorded by the individual programs and regions in most cases. However, it is important to note that departmental officials indicated the frequency of input received from the public is usually very low.

Recommendation #3: “Writing information and/or guidance documents in plain language, and regularly revising content, where applicable, with the suggestions made by clients.”

The evaluation concluded that this recommendation was implemented. However, the metrics assigned to measuring the Department’s performance in implementing the recommendation were found to be not applicable.  

Although no quantitative data is available to support a conclusion on whether or not the use of plain language was consistently implemented to develop the Department regulatory interpretation documents, departmental officials noted that the number of cases when input was received from the public is low. The use of plain language in the Department’s interpretation documents could be a main factor for the low number of input received.

Recommendation #4:Utilizing, where appropriate and where resources have been approved, more interactive tools to communicate with stakeholders, including webinars, workshops, live stream and chats, social media, and posting videos that explain changes and use concrete examples of what the changes mean.”

The evaluation concluded that this recommendation was adopted by the Department. However, some of the metrics associated with measuring performance were found to be not applicable.

Departmental officials indicated that, where appropriate, the use of more interactive means of communications, including face to face meetings, conference calls, website postings, and social media, were more effective approaches to engage the public and receive their input.

Metrics and targets: Evaluation results 2015-2017
Expected result Performance indicator Target Performance result 2015-17
The Department is implementing a client service model. Number of Stakeholder Outreach Plans developed and implemented for new regulatory initiatives, regulatory amendments, and policy instruments. 100% of all new regulatory initiatives, regulatory amendments, and policy instruments have a Stakeholder Outreach Plan. 100%
Stakeholders are informed about the various regulatory initiatives and policy instruments that have been posted on the Department's website. Number of people that have signed up for a listserv and that are receiving updates related to new regulatory initiatives, regulatory amendments, and policy instruments. No target was specified N/A
Information and/or guidance documents are helpful to stakeholders. The number of respondents that selected “yes” to query on helpfulness in the electronic feedback form. 85% of feedback respondents found the information and/or guidance documents helpful. N/A*
Information and/or guidance documents are written in plain language. The number of respondents that selected “yes” and “sometimes” to the query on plain language and ease of understanding in the electronic feedback form. 50% of feedback respondents find that plain language is used throughout the Department's information and/or guidance documents. N/A*
25% of feedback respondents find that sometimes plain language is used but that some sections of the information and/or guidance documents could be improved. N/A*
Stakeholders provide feedback on areas for improvement to specific information and/or guidance documents. The number of submissions made through the electronic feedback form. No target was specified N/A*
Feedback incorporated into information and/or guidance documents. The number of submissions made that can be addressed by updating information and/or guidance documents. 75% of submissions made through the electronic feedback form have been addressed through Frequently Asked Questions or updated information and/or guidance documents. N/A*
General statistics to determine how many visitors are viewing materials related to Regulatory Interpretation Policy and associated web pages. Change in web traffic for information and/or guidance documents developed. No target was specified N/A
Percentage change in website traffic every time an update is made to gauge level of interaction resulting from updates No target was specified N/A
General statistics related to alternate communications tools used by the Department on information and/or guidance documents. General statistics related to alternate communications tools used by the Department on information and/or guidance documents. List of tools employed by Department to engage stakeholders on new regulatory initiatives, regulatory amendments, and/or policy instruments and the associated and appropriate metrics (i.e number of retweets, number of views etc…).
  • Social Media
  • Face to face meetings
  • Conference calls
  • Website postings
  • Newspaper postings

 * The performance of these indicators could not be assessed because the electronic feedback form, on which their assessment relies, was not developed.

Recommendations

In general, the evaluation concluded that the Department of Fisheries and Ocean has partially adopted and implemented the recommendations that were made in the Improvement Report. This evaluation recognized that there is still room for further improvement with respect the Department’s regulatory interpretation policies, and therefore recommends:

  1. The Department develops an electronic feedback form that the public can access to provide user feedback. This would ensure feedback is received in a streamlined manner. Options for implementation continue to be evaluated.
  2. When appropriate, enhance the Department’s efforts to interact with the public regarding new regulations, regulatory amendments, and policy documents through the use of more interactive communication tools.
  3. Continue to update the Frequently Asked Questions posted for the Department’s regulatory information and guidance documents as necessary.
Date modified: