1. Making M&E Accessible to
Grassroots Organizations:
Lessons Learned from Firelight’s
Capacity Building Program
Santa Cruz International
Non-Profit Meet-Up,
9 March 2010
2. Presentation Overview
• Overview of Firelight’s M&E training approach
• Engage in practical exercises to:
– Identify guiding principles for foundations’ M&E activities
– Identify how key M&E concepts and practicalities can be
better understood and utilized by grantees
• Sharing Firelight’s outcomes to date, challenges,
and way forward
4. Monitoring & Evaluation:
The International Development Paradigm
Objectives
Statement
s
Performance
Indicator
Statements
Measurement
Methods / Data
Sources
Goal
Strategic
Objectives
Intermediate
Results
Outputs
Activities
Critical
Assumptions
5. Capacity Building Strategies
1) Training to build capacity of grantees, e.g.
“traditional” M&E skills
2) Special grants to support organization-specific
capacity development and learning needs
3) Partner learning exchanges and networking
meetings
4) Weekly “Newsflash” to share information with
grantees
5) As-needed requests
6. Guiding Principles
(official)
• Nurture strong, sustainable grassroots
organizations.
• Meet groups “where they are.”
• Build partners’ ability and confidence to
monitor and evaluate their own work.
• Peer-to-peer technical assistance is most
effective.
7. Guiding Principles
(unofficial)
• Grantees are already monitoring their work.
• Perfect beneficiary numbers are not our goal.
• Impact is NOT easy to prove due to:
• Timeframe
• Causality and attribution
• M&E activities should never detract from the
work at hand, which is serving children.
9. M&E Training
• 43 partners
trained in 3
countries by
Insideout.
• Aims to build
grantees’
capacity to
measure their
own progress in
a more
meaningful way.
• Training is practical and experiential, to
apply principles to organizations’ day-to-
day work.
10. What did we hope to see?
Intended Long-Term Outcomes for Trained Grantees
• Organizations clearly identify strategies and intended
outcomes of their work.
∀ Monitoring is paired with ongoing, day-to-day work.
∀ Organizations have ownership of their monitoring framework,
tools, and procedures.
∀ Management/leaders makes changes in the organization’s
programs and activities based on what is learnt.
∀ The organization shares what it learns with local communities.
∀ Reports to an organization’s donors and other stakeholders are a
more complete depiction of the outcomes of their work.
11. Making M&E Accessible
De-technicalize language.
ROADMAP TO
MONITORING
What are we
trying to
change?
(problem analysis)
Where do
we want to
get to?
(goal, objectives)
How are we
going to get
there?
(strategy, activity)
What do we expect to
happen along the way?
(RESULTS)
OUTPUT OUTCOME IMPACT
How do
we know
we are on
the right
road?
(indicators,
baseline,
targets)
12. OUTPUT OUTCOME IMPACTACTIVITIES
Have the activities
taken place?
The very first
result of an
activity.
Organization
s have direct
control over
this result.
What happened
next?
Change of
behavior in
participants.
Organization
s have less
control over
this result.
So what?
Change at
population/
societal
level.
Organization
s have very
little control,
if any.
Making M&E Accessible
13. Making M&E Accessible
1. Collect
2. Compile
3. Compare
4. Share
Get to the practical.
Simple Steps in
Data Analysis
15. Training Outcomes
Enthusiastic Feedback on the Training Curriculum/Approach
INCREASED CONFIDENCE - “We can see we are making progress.”
• “The simple words removed the fear and myths about M&E.”
• “I used to get indicators, outputs and outcomes confused but now
I know the difference.”
• “I learnt that we are already doing monitoring, and we can build
on that. But [we] need to make the process more systematic.”
RECOGNITION OF THE VALUE OF M&E
• “I learnt to think about more than outputs – that it is important to
think about the ‘So whats?’ of what we do.”
• “We need to monitor in order to be able to share our success
stories and learn from our mistakes!”
• “We are used to only thinking about monitoring at the end of
a project – this workshop has changed that!”
16. Training Outcomes
Evidence of “uptake” thus far
• Conceptualizing programming in terms of results
• Monitoring at the core of grantee’s planning - “We were doing
things randomly, jumping from here to there. Now we go forward
based on what our plan is.”
• Using the skills on projects not funded by Firelight - “The
knowledge we are getting is helping us improve our service
delivery, even on projects not funded by Firelight Foundation.”
• “Reporting is easier because you can go line by line, comparing
what you have done to what you planned.”
17. Training Outcomes
Evidence of “uptake” thus far
• Improved/adapted programming
• Strengthened fundraising - “[U.S. Embassy] officers were
shocked to find out that we are even capable of monitoring
and evaluating our project. Remember, we had written our
proposal by ourselves – there was no outside assistance.”
• Increased engagement with government frameworks
• Strengthened collaboration, networking, and advocacy
18. Challenges
• Buy-in at organizational level - staff turnover, quality of
shared learning within organization, “Some people who
were attended were not the right people.”
• Management issues within organizations - planning
(strategic, workplans), human resources management,
budgeting/financial management
• Efforts needed to ensure outcome indicators included in
grantees’ M&E frameworks
• Some baggage/anxiety about M&E remains - pleasing the
donor, some still searching for the “holy grail” of M&E
19. Challenges:
Reality Check
“Absence of computers
to process data and
store the information so
it can be easily
accessed.”
“We now need to be able
to cross the river more
frequently.”
20. Challenges:
Showing Progress
• Lack of baseline data
• Data quality issues - verification needs
• Interest in participatory M&E with communities
• Data management and recordkeeping
• Lack of resources to devote to M&E (time, staff, $)
What do these challenges reveal to Firelight?
Grantees are excited.
Grantees are invested.
Grantees are “taking on” M&E within their organizations.
21. Next Steps
1. Conduct review of M&E training outcomes. Produce
publication on Firelight’s approach and lessons learned.
2. Test revised curricula for training in additional Firelight
focus countries in 2010/11.
22. Remember:
It’s not about the indicators.
It’s about reflection and learning.
Good luck in your own M&E efforts!
THANK YOU
Editor's Notes
to interact with fellow participants
focus on sharing experiences and problem-solving
Baggage exists – inaccessibility of M&E paradigm
M&E – research discipline, project-based
Example: Would YOU know how to measure % of children age 6-59 months below -2 z score wt/ht?
by supporting organizational development
Where they are - i.e. having realistic and appropriate expectations depending on the capacity level of a partner
Impact - such a loaded & over-used word
communities can see ‘impact’ even if it’s difficult to describe…
It’s their impact & success - not ours
This slide shows the trade-offs in cost and complexity among different methods of data collection. Can also think of M&E as regular/routine (part of everyday duties such as beneficiary records) or special/periodic (which require additional time, resources, planning such as household surveys)
Consider the level of financial/human resources available.
The effort expended should match the improvement in decision-making.
FLF made strategic decision to let small CBOs be. Orgs need to have strategy in place before they begin - middle capacity selected.
Curriculum is focused on making monitoring activities “accessible” to grassroots organizations
FLF made strategic decision not to tie our M&E training to reporting requirements. Of course, FLF hopes we will receive improved reports, but this is a long-term goal.
1 follow-up training in Lesotho.
The training curriculum is extremely adaptable because it enables any organization to draw upon its current programming and monitoring activities.
Help grantees tell their own story - build confidence in asking “so what?”
it’s about internalizing the importance of M&E – shifts in organizational culture from emergency/crisis mode to planning & reflection
More important than standardized administrative procedures, M&E activities constitute an informed process of reflection.
These are abstract concepts - but understandable.
Your logic does not equal your partners - other ways of knowing exist that are not empirical
M&E is about testing assumptions - that should be the new definition of building “evidence”
Indicators - completely theoretical concept
Case studies, partners’ own data
Follow-up training focused on design of data collection tools, data analysis and data management, most trainings end at developing M&E frameworks
Respect that M&E activities must be relevant to grantees’ daily management of their organization
Takes $ and people to do good M&E.
FLF working with in-country META facilitators
Ultimately, M&E must be a learning/management tool for an organization. Process of transformation to a learning organization does not happen overnight.
What happens within the organization is the most important factor in “institutionalizing” reflective practices.
Confident enough to train rest of organization and communities
For those with frameworks or experience already - identified gaps and how it could be improved
We have already observed that the training contributes to preparing grantee-partners to articulate their successes and challenges and to mobilize resources to sustain their programs, ultimately supporting a transition from Firelight funding. Groups have already demonstrated their use of more systematic and reflective monitoring practices, as well as a willingness to engage with local stakeholders based on the confidence gained from having a better understanding of their own programs’ outcomes.
1) The META training workshop was excellent and has given us insight in gathering data from the field and analyzing it before documenting it for various stakeholders. ~ISRL, Zambia
2) Tsosane “We use these 3 questions in every meeting from here on out.
1)Where are we now?
2)Where are we going?
3)How are we going to reach there?”
3) From LGGA final report, received 8/3/07:
The META training has assisted the administrator to be able to gauge the organizational impact in terms of services offered (the field worker was able to account for the services she has planned), how can we expand, to what extend has the finances been utilized and how appropriate were they utilized. The roadmap was the guiding document for parameters setting and self –analysis for actual achievements.
Improved/adapted programming - LSC communities with more child neglect cases were targeted for awareness campaigns
Strengthened fundraising - T’sosane Support Group was 12 out of 120 successful groups for USG grant. Only CBO to receive HR funding.
Strengthened collaboration/networking – “We discuss all kinds of issues with more information at our fingertips.”
Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue
each org different, different capacities/types of programming (e.g. orgs with smaller #s of case work vs. community mobilization)
Not surprising considering output-level results are easier to comprehend and directly related to day-to-day activities. Outcomes is the new part.
already overburdened workloads
Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue
Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue
REALITY OF RELATIONSHIP-BASED GRANTMAKING
META Coordinator now a full-time position.
Future adaptations for training - accompanying organizational capacity assessment, shorter, more frequent meetings conducted by in-country facilitators
Groups beginning to see links with advocacy