Participatory Monitoring and Evaluation

Participatory Monitoring and Evaluation (PM&E)
CED 246-Evaluation in Rural Development
Anna Merlinna T. Fontanilla
MS DEVCOM

PM&E, CED 246-Evaluation in Rural Development, 10/28/14
What’s in the presentation
•Background and concepts
•Goal of PM&E
•The PM&E Process
•Checklist in designing PM&E
•Sample case

PM&E roots
Participatory Action Research
Farming Systems Research or FPR
Participatory Learning and Action
(including PRA)
Participatory Monitoring and Evaluation

Conventional
PM&E
WHY
Accountability, usually summary judgment about the project to determine if funding continues
To empower local people to initiate, control and take corrective action
WHO
External experts
Community members, project staff, facilitator
WHAT
Predetermined indicators of success, principally cost and production output
People identify their own indicators of success
HOW
Focus on “scientific objectivity” distancing of evaluators from other participants; uniform complex procedures; delayed limited access to results
Self evaluation; simple methods adapted to local culture; open immediate sharing of results through local involvement in evaluation processes
WHEN
Midterm and completion
Any assessment for program improvement; merging of monitoring and evaluation, hence frequent small evaluations
Narayan (1993) as cited by Sartorius, R./Social Impact

A process through which stakeholders at various levels engage in monitoring or evaluating a particular project, program or policy, share control over the content, the process and the results of the M&E activity and engage in taking or identifying corrective actions.

Common principles
Estrella and Gaventa

Goal of PM&E
Make the M&E process more participatory and effective by including a wider range of stakeholders at every stage of the process.

The PM&E Process
1. Identifying and engaging stakeholders
2. Building stakeholder’s capacity for PM&E
3. Defining and agreeing on what to monitor and evaluate: objectives
4. Developing and formulating indicators
5. Gathering information
6. Managing and analyzing data
8. Learning and change
7. Reflection, sharing and using the results of PM&E (Reporting)
Njuki, J. et al., 2006

Identifying and engaging stakeholders
Stakeholders include all those who affect, and/or affected by the policies, decisions, and actions of a system.

Building capacity for PM&E
1.Develop a common understanding of PM&E concepts and goals
2.Identify local vocabulary and terms equivalent to technical terms
3.Use of methods and tools that encourage participation
4.Discuss why PM&E is important to their lives and projects

Defining objectives
•Develop a common vision
•Agreeing on measurable results and processes that need to be monitored and evaluated
•Identify what information should be monitored, for whom, and who should be involved
•How results and findings would be applied

Selecting indicators
Roche, forthcoming

Gathering and analysis of data
•Analyze what is working, what is not working and why
•Reflect on the progress of the project towards achieving its goals and to adjust activities as required
•Allows community members to systematically reviews their activities
Stakeholders decide:
•Which tools should be used to collect information on which indicators,
•How sampling will be done,
•Who should collect and analyze the information,
•How frequently this will be done

PM&E Plan/Framework/Worksheet
Results/ processes, activities
Indicators
Info reqts.
Baseline
Targets
Data source
Who collects? Analyses? Reports?
How often? When?
Tools
Njuki, J. et al., 2006

Project Objectives (Goal, Purpose, Outputs)
Indicators
Data Collection
Data Analysis and Use
Source of Info
Baseline data needed
Who is involved
Tools and methods
How often
Added info needed
How often
Who is involved
How info is to be used
Who gets info
PM&E Plan/Framework/Worksheet
Sartorius, R./Social Impact

Reflection and learning
•Problems and successes
•Impacts of their efforts
•Act on what they have learned (how stakeholders use information in making decisions and identifying future action)
•Sharing results with the rest of the community
•What do we need to change?

Checklist for designing PM&E
Will it be sustainable once the project has ended?
Do the people responsible for PM&E have all the necessary skills?
Can the system be incorporated into the structure of collaborating agencies?
Is the system based on a clear understanding of project objectives?
Is it based on a clear understanding of the information needs of key stakeholders?

Is the system based on indicators defined by program participants?
Does the system involve the participation of all key stakeholders in every stage of the PME cycle— planning, data collection, analysis and use?
Do data collection tools fit the skills of the collectors?
Is it cost-effective?
Is the amount of data collection manageable and conducive to timely analysis and use of the results?

Is the amount of data collection manageable and conducive to timely analysis and use of the results?
Is the system documented so everyone knows what it contains?
Is there a plan for testing and adjusting the system?
Have annual self-assessments been planned?
Have impact evaluations been scheduled?
Others

PM&E Application
Participatory Evaluation in Human Resource Development: A Case Study from Southeast Asia
Gary Anderson and Deborah Gilsig
1998

Project background
Project Title
SEAMEO-Canada Program of Cooperation in Human Resource Development
Location
South East Asia
Time Frame
Phase I (1985-1989)
Phase II (1990-1995)
Goal
Human resource development in the region through provision of short-term and long- term training courses, seminars, workshops, international and regional conferences, and information dissemination activities.

About the evaluation
Project Title
SEAMEO-Canada Program of Cooperation in Human Resource Development
WHEN
5th and final year of the project
Objectives
Provide information to SEAMES, CIDA, PAC, involved SEAMEO centers.
Provide information that could guide future SEAMEO capacity development.
Fulfil the conditions of the agreement between CIDA and SEAMES
Provide accountability for CIDA’s investment in the project.
Design
Employed external evaluators which provided general guidance.
All stakeholders took part in the process by providing data;
Each stakeholder participated in one or more of the other evaluation activities: developing the Terms of Reference, making revisions to the work plan, developing data-collection instruments, collecting data, analyzing data, and discussing and suggesting revisions to the evaluation report.

1. To what extent did the project in general represent a sound development
investment?
1.1 Did it have a sound rationale from the perspectives of CIDA and
SEAMEO?
1.2 Was the project efficiently implemented?
1.3 What were its effectiveness and effects in developing organizational
capacities of SEAMEO? Do these effects represent good value for dollar?
2. To what extent did the institutional linkages help develop the capacities
of the involved SEAMEO centers?
2.1 Did they help centers to understand their internal and external environments?
2.2 Did they help develop needed center capacities?
2.3 Did they contribute to center performance?
3. To what extent was support for the regular SEAMEO training programs
worthwhile?
3.1 What were graduates’ perceptions of its efficiency, effectiveness, and
effects in making specialists more competent in their jobs?
3.2 Did the investment in regular training programs improve their quality in
a sustainable way?

Summary of major stakeholders and their roles
Stakeholder
Principal
Major Activities
CIDA
• Defining evaluation questions
related to compliance
• Receiving the evaluation report
and acting on its recommendations
• Facilitating implementation of the evaluation
• Providing data
• Providing feedback on draft report
PAC
• Acting as the mandated authority
• Commissioning the evaluation
and receiving its reports
• Approving the terms of reference
• Discussing the draft report
• Receiving and approving the
final report
SEAMEO/
SEAMES
• Facilitating data collection
• Assuming responsibility for data collection from training of graduates
• Providing data
• Assisting with interpretation of
analyses
• Helping design training graduate
survey questionnaire
• Distributing and collecting surveys
• Providing data on project implementation

Summary of major stakeholders and their roles
Stakeholder
Principal
Major Activities
SEAMEO
Centers
• Participating and assisting in
data collection
• Providing data
• Assisting with interpretation of analyses
• Helping design training graduate
survey questionnaire
• Distributing and collecting surveys
• Conducting center self-assessments
• Providing data on project effectiveness, effects, and impacts
• Interpreting data on project
effectiveness, effects, and impacts
Canadian
Institutions
(Colleges
and
Universities)
• Providing data
• Providing feedback on draft
report
• Providing data on project effectiveness, effects, and impacts, and
implementation through interviews
and Canadian Technical Assistant Questionnaire
SEAMEO
Training
Graduates
• Providing data
• Completing Regular Program
Graduate Questionnaire

Summary of data collection
Source of Data
Data Collection Instruments
Proposed Sample
Actual Sample (Actual #)
SEAMEO Centers
Center Self-Assessment Guide
SEAMEO-CIDA Project
Questionnaire
12 centers +SEAMES
12 centers
92% (12)
100% (12)
SEAMEO Center Staff Trainees
Center Staff Trainee
Questionnaire
n/a
(56)
CIDA PTLs
Interview
Former & current
PTLs
100% (2)
Canadian Institutions
Telephone Interview
13 project heads
84% (12)
Canadian (TAs)
Canadian Technical
Assistant Questionnaire
68 Canadian TAs
59% (42)
Chairs of SEAMEO
Governing Boards
Questionnaire for Members of SEAMEO Governing Boards
12
75% (9)
SEAMEO Center General Program Trainees
Regular Program Graduate
Questionnaire
1,000 former trainees
53% (533)
PAC Members, SEAMES, CIDA
Numerous interviews were conducted to collect data on the
functioning of the PAC, project management, etc.

Data analysis
•Statistical and content analysis
•Group analysis/interpretation
•Discussion about culturally-based findings
•Comparative levels of performance of different training programs
•Center directors met, individually, with a Universalia team member to discuss the self-assessments (confidential, this was owned exclusively by the centers)

Reflection and use of results
1. The process appears to have given the stakeholders a much deeper understanding of their organization (SEAMEO as a whole) and how it could link to Canada’s development interest.
2. It helped to build evaluation capacity.
-Center directors discussed about a possible phase III of the project.
-A more competitive or selective process for funding was being considered i.e. funding may not be available for all centers and will not be provided to be used as input substitution.
-Some of the centers plan on using the self-assessment tool ( or an equivalent) on a regular basis.
-A few of the centers are using the information from the evaluation in their strategic planning.
-The center’s participation in the process gave them a sense of ownership and the success has encouraged them to continue using the data collection instruments for future training reaction assessments.

References
•Adams, J. and A. Garbutt, 2008. Participatory monitoring and evaluation in practice: Lessons learnt from Central Asia.. Praxis Paper 21: Participatory Monitoring and Evaluation in Practice. International NGO Training and Research Center. Retrieved on October 26, 2014 from http://www.intrac.org/data/files/resources/420/Praxis-Paper-21-PME-in-Practice.pdf
•Estrella, M. 1998. Learning from change: Issues and experiences in participatory monitoring and evaluation. Retrieved on October 6, 2014 from www.ids.ac.uk/files/dmfile/PB12.pdf
•Njuki, J. et al. 2006. Participatory monitoring and evaluation for stakeholder engagement, assessment of project impacts, and for institutional and community learning and change. Retrieved on October 20, 2014 from http://ciat-library.ciat.cgiar.org/Articulos_ciat/Njuki- rev.pdf
•Sartorius, R. (n.d.). Participatory monitoring and evaluation systems: Improving the performance of poverty reduction programs. Social Impact. Retrieved on October 20, 2014 from www.rrojasdatabank.info/wpover/06Rolf-Latest.pdf
•Victoria, The State of. 2013. Effective engagement. Retrieved on October 26, 2014 from http://www.dse.vic.gov.au/effective-engagement/toolkit/tool-stakeholder-analysis- stakeholder-matrix
•World Agroforestry Center, 2003. Carrying out a stakeholder analysis. Retrived on October 26, 2014 from http://www.cglrc.cgiar.org/icraf/toolkit/Carrying_out_a_stakeholder_analysis.htm
•World Bank, The. 2003. Participation, monitoring and evaluation. Participation and Civic Engagement. Retrieved on October 26, 2014 from http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTSOCIALDEVELOPMENT/EXTPCENG/0,,contentMDK:20509352~menuPK:1278203~pagePK:148956~piPK:216618~theSitePK:410306,00.html

Participatory Monitoring and Evaluation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (18)

Similar to Participatory Monitoring and Evaluation

Similar to Participatory Monitoring and Evaluation (20)

Recently uploaded

Recently uploaded (20)

Participatory Monitoring and Evaluation