solution finder

Participatory Monitoring and Evaluation

Author/Compiled by
Leonellha Barreto Dillon (seecon international gmbh)
Executive Summary

With growing emphasis on participatory approaches towards development, there has been recognition that monitoring and evaluation (M&E) should also be participatory. Conventionally, M&E has involved outside experts coming in to measure performance against pre-set indicators, using standardised procedures and tools. In contrast, participatory monitoring and evaluation (PM&E) involves primary stakeholders as active participants and offers new ways of assessing and learning from change that are more inclusive, and reflects the perspectives and aspirations of those most directly affected (WORLD BANK 2010b). This document presents the different concepts related to PM&E, together with the steps and the tools needed for the development of an effective and sustainable PM&E plan.

Advantages
Involving beneficiaries in evaluation increases its reliability and provides the opportunity to receive useful feedback and ideas for corrective actions
PM&E allows for flexibility ― Activities should be stopped or adapted when evaluation makes it clear that they are not contributing to the intended improvements
Strengthens ownership regarding successful outcomes of planned initiatives
Widens the knowledge base necessary for assessing and ― if required ― correcting the course of action
Increases the motivation of stakeholders to contribute ideas to corrective actions
Creates trust in Local Government policy and action (provided that the stakeholders’ input is genuinely taken into account)
Contributes to the learning of all involved
Disadvantages
Needs skilled facilitator to ensure everyone understands the process and is equally involved
Can be dominated by strong voices in the community (for example, men dominating women in discussions, political, cultural or religious leaders dominating discussions and decision making)
Can be time consuming - needs genuine commitment
Needs the support of donors as does not always use traditional indicators
Those responsible for implementation of certain projects may not want the administration or public to learn about failures or mistakes due to a fear of disciplinary action. Evaluation should be conducted in a fair and constructive way (PHILIP et al. 2008)

Definition of Monitoring

Monitoring is a continuous process of collecting and analysing information to compare how well a project, programme or policy is being implemented against expected results. Monitoring aims at providing managers and major stakeholders with regular feedback and early indications of progress or lack thereof in the achievement of intended results. It generally involves collecting and analysing data on implementation processes, strategies and results, and recommending corrective measures (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007).

 

Definition of Evaluation

Evaluation is the systematic and objective assessment of an ongoing or completed project, programme or policy, its design, implementation and results. Evaluation determines the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling incorporation of lessons learned into the decision making process of both recipients and donors (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007).

 

Definition of Participatory Monitoring & Evaluation

Participatory monitoring & evaluation (PM&E) is a process through which stakeholders at various levels engage in monitoring or evaluating a particular project, program or policy, share control over the content, the process and the results of the monitoring and evaluation (M&E) activity and engage in taking or identifying corrective actions. PM&E focuses on the active engagement of primary stakeholders (WORLD BANK 2010a).


Participatory Monitoring and Evaluation is one of many approaches to ensure that the implementation of the different projects within the action plan ― or smaller individual projects ― leads to the expected outcomes. As with all other monitoring and evaluation elements, the process for PM&E has to be prepared prior to project implementation (PHILIP et al. 2008).


The stakeholder groups typically involved in a participatory M&E activity include: the end users of project goods and services, including both men and women at the community level; intermediary organisations, including NGOs; private sector businesses involved in the project; and government staff at all levels (RIETBERGEN-McCRACKEN et al. 1998).

 

Principles of Participatory Monitoring & Evaluation

This problem might be recognised and solved by participatory monitoring and evaluation. Source: WSP (2007)

This problem might be recognised and solved by participatory monitoring and evaluation. Source: WSP (2007)

Conventionally, monitoring and evaluation has involved outside experts coming in to measure performance against pre-set indicators, using standardised procedures and tools. PM&E differs from more conventional approaches in that it seeks to engage key project stakeholders more actively in reflecting and assessing the progress of their project and in particular the achievement of results (THE WORLD BANK 2010a). Core principles of PM&E are (RIETBERGEN-McCRACKEN et al. 1998):

 

  • Local people are active participants — not just sources of information.
  • Stakeholders evaluate, outsiders facilitate.
  • Focus on building stakeholder capacity for analysis and problem-solving.
  • Process builds commitment to implementing any recommended corrective actions.

Steps in the Development and Implementation of a PM&E Process

(Adapted from AUBEL 2004)

Step 1: Planning the PM&E Process and Determining Objectives and Indicators

At this initial stage, the stakeholder groups to be involved in the planning of the PM&E process must first be identified. Stakeholders must define the objectives of the PM&E, including what will be monitored, how and by whom. The planning stage requires a lengthy process of negotiation, contestation and collaborative decision-making among various stakeholders. Identifying objectives and monitoring indicators can be the most difficult part of planning a PM&E process. In some cases, a common set of indicators is developed, while in other instances different stakeholder groups develop their own sets of indicators.
 

Step 2: Gathering Data

Data collection can include the use of both quantitative and qualitative methods and tools. Quantitative methods can include: community surveys; interviews; and observations. Qualitative methods can include various participatory learning methods using visual, interviewing and group tools and exercises.


Step 3: Analysing Data

While data analysis is often thought of as a rather mechanical and expert-driven task, PM&E should be an opportunity to actively involve various categories of program stakeholders in the critical analysis of successes and constraints and the formulation of conclusions and lessons learned.


Step 4: Sharing the Information and Defining Actions to Be Taken

However participatory the M&E process in Steps 1-3 is, not all stakeholders can be involved in M&E data collection and analysis. In this step, the results of M&E activities are shared with other stakeholders, and there is discussion of appropriate actions to be taken based on the findings.

Techniques and Tools for PM&E (Adapted from RIETBERGEN-McCRACKEN et al. 1998)

A participatory approach to monitoring and evaluation will usually make use of a number of techniques and tools, selected and combined to suit the objectives of the M&E work and the resources available. Many of the techniques associated with Participatory Rural Appraisal (PRA), Beneficiary Assessment (BA), and SARAR have been used in the context of monitoring or evaluation. Some examples of these methodologies’ trademark techniques and applications to M&E are highlighted below:

 

  • Participatory Rural Appraisal: visual methods, often to analyse “before and after” situations, through the use of community mapping, problem ranking, wealth ranking, seasonal and daily time charts, and other tools.
  • SARAR: interactive and visual-based methods to facilitate community discussion with such methods as pocket charts, three pile sorting, and “story with a gap.”
  • Beneficiary Assessment: conversational interviewing and focus group discussions on changes and impacts.
  • In addition to using PRA, SARAR, and BA techniques, participatory monitoring and evaluation often entails development of other techniques that are designed to be used by community members and other local-level stakeholders as part of an M&E activity, namely:
    • Visual self-evaluation tools
    • Testimonials
    • Photographing the evidence
    • Community records and indicators
Applicability

Although monitoring and evaluation only becomes relevant once a project is up and running (for example at regular intervals when results become available) it nevertheless has to be considered prior to project implementation that is already in the planning phase (PHILIP et al. 2008).

Library references

Monitoring and Evaluation in a nutshell

As part of the process to systematise and enhance the quality of monitoring and evaluation processes, this simple monitoring and evaluation guide has been developed. This guide includes practical guidance on how to do monitoring and evaluation: including developing simple monitoring and evaluation tools giving practical examples, a set of formats to facilitate the evaluation process and basic monitoring and evaluation terminology to ensure coherence and consistency.

INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES (2007): INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES

Local Government and Integrated Water Resources Management (IWRM) Part III: Engaging in IWRM – Practical Steps and Tools for Local Governments

The set of materials entitled “Local Government and Integrated Water Resources Management (IWRM)” aims to assist Local Governments with active participation in IWRM. The materials are primarily targeted at local government officials, but are considered equally useful for individuals and organisations that work with local governments in the management of water resources.

PHILIP, R. ; ANTON, B. ; BONJEAN, M. ; BROMLEY, J. ; COX, D. ; SMITS, S. ; SULLIVAN, C. A. ; NIEKERK, K. van ; CHONGUICA, E. ; MONGGAE, F. ; NYAGWAMBO, L. ; PULE, R. ; BERRAONDO LOEPEZ, M. ; (2008): Freiburg: ICLEI European Secretariat GmbH URL [Accessed: 17.04.2012]

Participation and Social Assessment: Tools and Techniques

This resource kit aims to share information and experiences on participatory methods in the context of development cooperation. The primary focus concentrates on providing practical guidance and case examples.

RIETBERGEN-McCRACKEN, J. ; NARAYAN, D. WORLD BANK (1998): Washington: World Bank URL [Accessed: 10.05.2010]

Calendar 2007 June

WSP (2007): Washington, DC: Water and Sanitation Program (WSP) URL [Accessed: 03.04.2012]
Further Readings

Judicious management of groundwater through participatory hydrological monitoring

This report developed under the APWELL project deals with participatory hydrological monitoring in an effort to sensitize the individual groundwater users on judicious use of groundwater. Participatory hydrological monitoring improves the users’ understanding of local groundwater resource characteristics and helps local communities to form a community opinion to support appropriate measures for managing the available resources equitably.

APWELL Project (2003): A manual. Andhra Pradesh, India.: Andhra Pradesh Groundwater Bore well Irrigation Schemes Project (APWELL Project) URL [Accessed: 15.01.2013]

Monitoring and Evaluation in a nutshell

As part of the process to systematise and enhance the quality of monitoring and evaluation processes, this simple monitoring and evaluation guide has been developed. This guide includes practical guidance on how to do monitoring and evaluation: including developing simple monitoring and evaluation tools giving practical examples, a set of formats to facilitate the evaluation process and basic monitoring and evaluation terminology to ensure coherence and consistency.

INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES (2007): INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES

Participation and Social Assessment: Tools and Techniques

This resource kit aims to share information and experiences on participatory methods in the context of development cooperation. The primary focus concentrates on providing practical guidance and case examples.

RIETBERGEN-McCRACKEN, J. ; NARAYAN, D. WORLD BANK (1998): Washington: World Bank URL [Accessed: 10.05.2010]

How to Manage Public Toilets and Showers

The purpose of this decision-making aid is to provide practical advice and recommendations for managing toilet blocks situated in public places. It is primarily aimed at local decision-makers in developing countries and at their partners (project planners and managers).

TOUBKISS, J. ; (2010): (= Six Methodological Guides for a Water and Sanitation Services' Development Strategy , 5 ). Cotonou and Paris: Partenariat pour le Développement Municipal (PDM) and Programme Solidarité Eau (pS-Eau) URL [Accessed: 19.10.2011]

Developing and Monitoring Protocol for the Elimination of Open Defecation

This paper reviews process and protocol for defining, reporting, declaring, certifying ODF (Open Defecation Free) and sustaining ODF, highlighting where the process varies between countries and potential determinants of sustainability within the process itself.

THOMAS, A. ; BEVAN, J. ; (2013): (= ODF Protocol ). Nairobi: UNICEF URL [Accessed: 06.06.2013]

Outcome and Impact Assessment in International Development

These guidelines are designed to help project managers to assess the outcomes of their projects and programmes. They demonstrate how development agencies can implement an appropriate outcome and impact assessment system.

ZEWO (n.y): Zewo Guidelines for Projects and Programmes. Zurich: Schweizerische Zertifizierungsstelle fuer gemeinnuetzige, Spenden sammelnde Organisationen (ZEWO) URL [Accessed: 04.10.2013]

Achieving Sustainability: Changing the Ways in Which we Define Success

Implementing agencies like WSUP, Water For People and IRC are of course accountable to their funders, including major bilaterals and foundations. And naturally, these funders must track the effectiveness of their spending. But short budget cycles and the need to demonstrate “value for money” can often encourage over-simplistic measurement of success in terms of short-term outputs, rather than genuinely sustainable services. This note proposes some ways forward.

COWLING, R. ; NORMAN, G. ; (2013): (= Perspective , 1 ). London: Water and Sanitation for the Urban Poor (WSUP) URL [Accessed: 25.11.2013]
Case Studies
Training Material

Participatory Impact Monitoring Booklet 2: NGO-based impact monitoring

This booklet on NGO-based impact monitoring is addressed to staff members of development organisations, i.e. national organizations such as NGOs, federations or government organizations which promote self-help groups. It explains how NGO-based impact monitoring works in a simple, illustrated and easy-to-understand manner.

GERMAN, D. GOHL, E. (1996): Eschborn: GATE/GTZ URL [Accessed: 17.04.2012]

Alternative Versions to