printUse ctrl + p to print the page

Process Model - Reflexive Monitoring

Assessment is an integral part of the whole process of mapping and influencing policies and institutions. Facilitators and participants will want to track changes in: (a) market inclusion of small-scale producers and the opportunities and threats that they face; (b) policies and institutions affecting this inclusion; and (c) stakeholders’ actions to make progress on policies and institutions to support small-scale producers.

Monitoring and evaluation are major processes in their own right; detailed discussion is beyond the scope of this website. Monitoring and evaluation should be included throughout the multi-stakeholder process, involving all participants in reflecting on progress, analysing this progress against their own aims and milestones, drawing lessons, and applying these lessons in further policies and practice. In other words, monitoring and evaluation should be integrated into the full learning cycle.

 

1 - Create a learning culture and environment

To learn, people need to reflect critically about what is happening, they needs to question assumptions and they need to be open to constructive criticism, new ideas and failure.  For an individual or a team to learn effectively it is necessary to create a positive learning environment and culture.  This means that people are encouraged to raise questions and challenge established thinking, that time is put aside for reviewing progress, that people trust each other and that there are incentives for people to work towards improvement.  

Many different things can contribute to either a supportive or non-supportive learning environment.  The style of leaders and facilitators are particularly important in this regard.  Other factors include whether people feel their ideas are being taken seriously and acted upon and whether people feel they are in a safe environment for expressing their own uncertainties.

 

2 - Define success criteria (performance questions and indicators)

How will you know when you have succeeded? Success can be determined for different parts of the process: success with your management structure, success in developing and nurturing partnerships, success in individual projects, success in maintaining commitment, success in keeping others informed.
You also need to determine what resources you need to commit to evaluation and how you will communicate both the process and the results. Look back at your theory of change and check the success criteria you have defined.

  • Ensure all stakeholders have a portion of their resources dedicated to evaluation.
  • Seek specialist assistance. Many industry groups and government agencies have staff dedicated to ‘continual improvement’ — engage their services. You could, for example, get them to run short workshops on key elements of learning and adapting, or provide a critique on your monitoring and evaluation program.
  • Seek information from others about monitoring and evaluation process — aspects of their process may suit your initiative.


Systems are complex and you can’t measure everything (nor should you). However, by measuring some ‘indicators’ you get an idea of what is changing, worsening, improving etc and thus an indication of whether you are being successful. Using indicators to gauge your progress will determine if and when you need to change either the way you currently do things or how you are planning to do things.

  • Be clear about what you need to monitor in order to provide the objective information required for gauging progress.
  • Depending on what you are monitoring you need to select indicators for inputs, outputs and outcomes.
  • Select indicators for different sectors and categories e.g. small business, regional, individual projects. Select indicators that will aid you in determining whether you are reaching the community, environmental and economic objectives of the initiative.
  • Indicators need to be SMART: Simple, Measurable, Accessible, Relevant and Timely. Moreover, they should be: able to provide a representative picture; easy to interpret and show trends over time; responsive to changes; capable of having their significance assessed by users; analytically sound; available at reasonable cost; and able to be updated.

 

3 - Develop and implement monitoring mechanisms

To effectively monitor an MSP various systems need to be in place.  For example there must be procedures for monitoring various indicators, ways of storing and analysing this information and meetings to discuss the implications of the results.  Very often monitoring and evaluation fails because the basic elements of the system have not been put in place.  Good monitoring and evaluation requires a systematic and methodical approach, as well as creativity.

In practical terms, monitoring and evaluation can be seen as an early warning system, enabling the project to be re-thought and modified. Considering it in these terms helps establish its relevance to what you are working to achieve.

Continued monitoring requires commitment and perseverance; thus, the RO needs to determine how it will facilitate this, including what support it can give to participants.

  • Do monitoring whilst ‘doing’ — it works best if it is an integral part of the project/initiative.
  • Determine and communicate the monitoring responsibilities of individuals, groups and agencies. Regular updates can be useful.
  • Provide opportunities for the community to understand what information is being collected, how it is being assessed and how it can guide future action.
  • Allocate time and resources to follow-up.


4 - Review and evaluate progress and identify lessons: learning oriented evaluations

Strategic planning generally involves a high degree of future uncertainty; for this reason, ongoing evaluation is needed to continually assess the effectiveness of the strategy and the actions. 

Different evaluation timeframes are to be considered: Every year you need to evaluate your actions; Every 1- 3 years you need to ask whether your actions are meeting your objectives; Every 3- 10 years you will need to re-evaluate the underlying strategy behind the action plan.

  • Analyse the information collected. Identify issues, trends and themes which will help you assess your progress.
  • Determine the information gaps and then fill them from relevant sources.
  • Determine whether your goals and objectives are still relevant.
  • Determine whether your strategic directions are still relevant and effective.
  • Determine whether your group processes and structure are relevant and effective.
  • Determine whether the wider context has changed. How does it affect your MSP? What changes do you need to make?

Lessons learned are insights and new knowledge that emerge from practice and experience that can be used to improve future action.  Good monitoring, evaluation and reflection processes should lead to valuable lessons for the future.  It is important to focus on capturing these lessons and documenting and communicating them in a way that will help to improve the overall MSP.

 

 

5 - Feed lessons learned back into strategies and implementation procedures

The process of monitoring and evaluation is not just about reporting. A successful process includes responding to the information and analysis, making the necessary changes, and ensuring that the lessons have been understood, internalised and shared.

Again, if stakeholders do not see learning and changes being made where appropriate, they can become disillusioned and frustrated and may remove themselves from the initiative.

  • Don’t just limit ‘adapting/changing’ to projects. From your now better-informed position, make the necessary changes to all aspects of your initiative, including process, structure, management, reporting and communicating.
  • Remember to ‘tell the story’ of how you have adapted or are encouraging people to adapt. Often, people have not had the opportunity to share this understanding, so their commitment to change may understandably be a bit shaky.
  • Feed the learnings back into practices you are currently undertaking now or may undertake in the future.
  • Share the learnings on both the fine-tunings you are making to your initiative/projects and the actual process of monitoring and evaluation. To increase the relevance to other groups, perhaps highlight aspects you wish others had been able to share with you earlier in your process.