Powerpoint on monitoring and evaluation for session 1 of the TTIPEC Ghana workshop

1. Monitoring & Evaluation
TIPS & TRICKS
TTI PEC Anglophone Africa Regional Peer Learning Workshop
Coconut Grove Regency Hotel | Accra, Ghana
Wednesday, July 16th, 2014
Dr. Mo Adefeso-Olateju and Rebecca Oluwatoyin Doherty

2. AGENDA
Introductions
Name, Organization, Brief Description of Project & Phase
Objectives
Reflective Activities
Introduction to monitoring & evaluation
Developing an M&E System
Practical challenges with M&E
Conducting Project Evaluations
Conclusion

3. OBJECTIVES
!
!
To provide hands-on support in developing
and utilizing M&E tools
The session will be focused on practical
support and will be as interactive as possible
A brief overview presentation on previous M
presentations and materials shared at TTI
events
Introduces potential tools and resources t
assess outcomes

4. REFLECT
The first time I heard M&E was
_____________________________.
!
I t h o u g h t M & E w a s
_____________________________.
!
I later discovered that M&E was
_____________________________.
!
N o w I u se M&E in my work by
_____________________________.
!
Today, I am interested in working on
a/an (planned, ongoing, or completed)
__________ project.
!
I d id my baseline sur vey in/on
_________________________________.
Designing & settingup
the M&E syste
Gathering
managinginformatio
Reflectin
critically to
improve actio
Communicating
reporting result
Key parts of a M&E system

5. M & E and Project Planning:
“THE CHICKEN, OR THE EGG?”
SHOULD WE HAVE M&E PERFORMANCE INDICATORS BEFORE A WORK PLAN?
SHOULD WE DEVELOP AN M&E PLAN BEFORE THE PROJECT PLAN?
WHAT DO WE DO WITH THE RESULTS OF OUR MONITORING?

6. M&E IN THOUGHT AND PRACTICE
TRUE or FALSE:
I always conduct a needs assessment before conceiving a
project _________________.
!
I should always conduct a needs assessment before
conceiving a project _________________.
!
I start thinking about stakeholder participation at the
onset of my project planning _________________.
!
My Monitoring/Evaluation/M&E is always participatory in
process and results ___________________.
!
We always reflect on the information that we collect for
M&E _____________________. Please explain how.
!
M&E seems to be more of a measurement tool/activity
than a planning or evaluative activity.
!
Whilst measurement (through well-developed performance
indicators) is critical to M&E, M&E is a much broader,
organization-wide and learning process.

7. M&E: A SNAPSHOT OF OUR GROUP
PARTICIPANTS KNOW WANT TO KNOW LEARNED
- Gloria Eboh, African Heritage Institution
(AfriHeritage)
- Sola Oluwadare, African Heritage
Institution (AfriHeritage)
I would like to do this by being able to analyze the
data gotten from the M&E tools accurately
- Analyzing the M&E results;
- How to improve on the M&E tools I am using currently.
Simone Fugar. Institute of Economic
Affairs, Ghana
- Adopt better online monitoring tools;
- Knowledge of the most efficient online monitoring tools
- Drusilla David, Centre for the Study of
the Economies of Africa, Abuja,
Nigeria.
- Anumudu Peace Onyinyechukwu,
Centre for the Study of the Economies
of Africa.
- CSEA uses the Impact Log as an M&E tool.
We will like to improve the log to enable us
- capture in detail the activities of the centre
- outcome of its research findings
- improve on the impact log to enable it measure its impact to
its target audience, policymakers and stakeholders to
determine if its messages are been used in the policymaking
sphere
- to enhance development based outcomes in the
long,medium or short term as the case may be.
- How to use other performance indicators to measure and
capture specific communication activities adequately and
regularly.
- How to measure outcomes from different information
products and how their impact and influence stands.
- How to strengthen our monitoring and evaluation skills by
incorporating other tools and methods
- as well as capacity building in tool use to ensure regular data
capturing.
- Vicentia Kotia,
- I use an adapted version of the impact log sheet, a
simple excel sheet
- No topic in particular but I'd love to learn and get better at
M&E
- Hands on training on the use of simple but effective M&E tools
- Job Eronmhonsele, & Peter Ugholeh.
Centre for Population and Environmental
Development (CPED) Benin City, Nigeria
- I developed to track on-going activities and outputs
Just a framework on the website for knowing the
number of site visitors within a given period,
- and a CHART (still undergoing development) to
help monitor CPED invitation to participate and
make presentations in conferences and workshops
around the globe.
- We would like to have a robust M & E tool so as to track
visitors and get feedback
- as well as know which of the publications appeal to visitors.
- We would like to learn how to use Google Analytic; Hoot
suite; Twitter Counter

8. Food for Thought
Your project plan should have been from the M&E perspective.
E.g. Consultations with the target population informs the needs assessment and the project as
the most feasible/desirable/affordable solution.
NATURE OF THE PROJECT/BRIEF DESCRIPTION
M&E should also be a part of your work plan.
E.g. at this stage of the project, this is what we expect to see
M&E affects program design because of your means of collecting data
Needs Assessment; Assets Assessment/Capacity Assessment
Start with an M&E Plan. A good M&E Plan will help a good project…or at least, side-by-side
Let us re-do our work plan. Have we laid the foundation for a bungalow? And now trying to build a
duplex?
Result is a change in behavior. Measure success by outcomes, and not outputs. We often celebrate the
result of outputs, and not outcomes or impact. Output is what you put out; however, results should be
measured by the success of people (Dr. Chuks Eresia-Eke)

9. SHORT VIDEO
!
https://www.youtube.com/watch?
v=MUSzePBTpAY

10. Introduction to monitoring and
evaluation

11. ! Monitoring:
! the regular collection and analysis of
data/information
! communication to strengthen project
partnership.
! Evaluation:
! occasional assessment of the overall
value and progress of a project
Common definitions of M&E

12. Looking critically at M&E
Monitoring = process of collecting and interpreting data to
provide management and stakeholders with evidence of the
extent of progress towards achieving the objectives from the
implementation of a plan
!
!
!
Evaluation = process of determining whether a particular
plan was well designed (was it relevant, did it have a
significant impact and was it sustainable?) and well
implemented (was it effectively and efficiently
implemented?) → leads to lessons learned for decisions

13. Changing the M&E Perspective
! M&E is mainly for the external
funding body
! Focus on activity & financial
reporting
! Lots of data & little analysis
! Little learning
! Little stakeholder participation
! M&E is about highly qualified
technical experts conducting
statistical studies
! M&E is boring and not very useful
for project staff/beneficiaries
Shifting Paradigms, methodologies,
approaches & tools
■ M&E is for everybody
involved in the initiative
■ M&E helps in understanding
impact & reasons for
success/failure
■ Analysis & critical reflection
to improve the initiative
■ Learning is a key function of
M&E
■ People processes – lots of
stakeholder participation
■ M&E is active, interesting
and useful to all involved in
an initiative

14. Developing an M&E System

15. What is an M&E system?
• An M&E system is
• A set of planning, information gathering and synthesis,
• reflection and reporting processes,
• along with necessary supporting conditions and capacities required
for the M&E outputs to make a valuable contribution to project
decision-making and learning.

16. Setting up a M&E system
! Practical Steps
1. Define the purpose and scope
2. Decide what to monitor & evaluate
3. Plan information gathering &
organisation
4. Plan critical reflection processes &
events
5. Plan for quality communication &
reporting
6. Plan for the necessary conditions
& capacities

17. Key parts of a M&E system
Designing & settingup
theM&Esystem
Gathering and
managinginformation
Reflecting
critically to
improve action
Communicating and
reportingresults

18. M&E as an integral part of project
management
The project strategy!
(what will be achieved and how it will be achieved)
Detailed
operational plan
Project outputs,
outcomes and
impacts
The M&E!
system!
!
!
!
!
Continualrefinement
Communicating and!
reporting results
Reflecting critically!
to improve action
Gathering and!
managing information
Developing!
the M&E system
Fielddata
informati
on
Basis for
Basis for
Leads to …
Mutual
refinements
Improvements through M&E

19. Integrated Learning Processes in M&E - Illustration
jan marfeb apr sep novoct decmay julyjune aug
NGO review
meeting
Project mgt coord
comm meeting
Inter-agency steering
committee meeting
Special review on
extension
activities
Quarterly
report
Half-yearly & yearly
report
WORKSHEET

20. Reflection processes & events - Example
WORKSHEET

21. Results framework
= framework used to assess performance
▪ Expectations
▪ Indicators
▪ Sources of information
▪ Baselines and targets
+ Reports for planning

22. The 3 Spheres
(Tsui et al 2014)

23. Results chain
RESULTS
Inputs ! Activities ! Outputs ! Outcomes ! Impact
Short-term Medium-term Long-termResources Processes
Within control of the organization Partial Org. Control No Org. Control
Sphere of Control: Advocacy, Tactics,
Inputs, Activities, Outputs
Sphere of Influence:
Changes in behavior,
relationships,
attitudes, agenda
Sphere of
Concern: Policy
& Political
Context
WORKSHEET

24. Results chain – Example
WORKSHEET

25. Practical challenges with monitoring
and evaluation

26. Insufficient Consideration of M&E at Initial Project Stages
! Unintentionally, M&E is often set to fail during the project design
! No clear direction is given in the project design on how M&E system will be
developed.
! No adequate budget for M&E
! No sufficient time and expertise allocated to M&E

27. Failure to align Phases of M&E with Phases of the
Project
WORKSHEET

28. Challenges in defining indicators
! More often indicators are stated and left at the level of a logical framework.
! No attempt is made to develop an indicator matrix that gives the details
! See example………………FAWE Uganda, Asante
[Number with reference to logframe] [Name of indicator]
Definition
Precise definition [=Formula]
What it measures [A narrative explanation of the importance of the indicator]
Frequency [For example, monthly quarterly, annually, every two years etc]
Disaggregation [For example, by grade, sex, school type, geographical area etc]
Source
Notes
Responsibility [Who is responsible in the M&E Unit for collecting the information]

29. Developing Effective Indicators
!
!
Indicator = measure
(of performance, progress, condition)
!
Properties of good indicator:
▪ Specific = leaves no ambiguity
▪ Measurable = can be compared
▪ Attainable = a source readily exists
▪ Relevant = measures intended progress
▪ Time-bound = refers to specific time

30. Indicators, baseline and targets (& Planning)
OUTCOME
ACTIVITY OUTPUT
Short- to
medium-term
Medium-term
Medium- to
long-term
IMPACT
OBJECTIVE
Design training to
increase community
involvement
Training of
SMCmembers
on community
involvement
SMC members are
actively involved in
school management
A school supported by
the community helps
increase attendance
Increasing attendance
will yield lower
repetition and higher
completion
Higher completion will
improve learning
INDICATOR
Number of SMC
guidebooks printed
Number of SMCs
where at least 3
members have been
trained
Percentage of SMCs
which are actively
involved in school
management
Attendance rate Repetition rate
Percentage of pupils
who master specified
competencies
BASELINE
O 10% 10% 75% 8% 30%
TARGET
5000 50% by year X 25% by year X 80% by year X 6%by year X 35% by year X
WORKSHEET

31. The importance of monitoring
outcomes, and not only outputs –
Practical way of conducting
evaluations
• Considerations
• Processes

32. Evaluation
! A systematic (and as objective as possible) examination of
a planned, on-going or completed project.
! It aims to answer specific management questions and to
judge the overall value of a project and supply lessons
learned to improve future actions, planning & decision
making.

33. (Some) Purposes of Evaluation
! Learning: Generating new insights/knowledge in order to
contribute to improved action in the future
! Accountability: Ensuring that an initiative is answerable to
its beneficiaries, partners & donors for effective and
efficient delivery and for appropriate use of funds.

34. Different types of project/programme evaluations
! Mid term evaluations (Formative/process evaluations) – To
provide feedback on the implementation process
! End of project evaluation (Summative) – To assess
outcomes, impacts, sustainability, external utility & lessons
learnt
! Self assessments – Internally guided & controlled. To gather
information and provide reflection, discussion, debate &
improvements
**The above could be internal/external – depending on who is
commissioning and/or taking a lead in undertaking them

35. Key steps in a project evaluation
Source: IUCN 2004

36. Common evaluation questions
The Situation to
Improve
Problems and Visions
Planned
(Goal, Purpose,
Outputs
and Activities)
Inputs Activities Actual Outputs
4. IMPACT
2. Efficiency
1. Relevance
3. Effectiveness
5. Sustainability

37. Evaluation questions
1. Relevance - Was/is the project a good idea given the situation to
improve? Was the logic of the project correct? Why or Why Not?
2. Effectiveness - Have the planned results been achieved? Why or Why
Not
3. Efficiency - Have resources been used in the best possible way? Why or
Why Not?
4. Impact - To what extent has the project contributed towards its longer
term goals? Why or Why Not? Have there been any unanticipated
positive or negative consequences of the project? Why did they arise?
5. Sustainability - Will there be continued positive impacts as a result of the
project once it has finished? Why or Why Not?

38. References & Resources
!
• Barnett, C. and R. Gregorowski (2013). ‘Learning about Theories of Change for the
Monitoring and Evaluation of Research Uptake.’ IDS Practice Paper In Brief 14. Publisher IDS.
(http://www.ids.ac.uk/publication/learning-about-theories-of-change-for-the-monitoring-
and-evaluation-of-research-uptake)
• Batchelor, S. (2012) Information Eco-systems of Policy Actors. https://www.ids.ac.uk/
publication/information-ecosystems-of-policy-actors-reviewing-the-landscape
• Beynon, P., Chapoy, C., Gaarder, M. & E. Masset (2012). ‘What difference does a policy brief
make?’ IDS, 3ie and NORAD. (http://www.3ieimpact.org/en/evaluation/policy-influence/
policy/)
• Doemeland, D. & J. Trevino (2014). ‘Which World Bank reports are widely read?’
• Outcome Mapping: Learning Community http://www.outcomemapping.ca
• Stachowiak, S. 2007. Pathways for Change: 6 Theories about How Policy Change Happens.
Organizational Research Services.
• Sumner, A. et al. (2011). ‘What shapes research impact on policy? Understanding research
uptake in sexual and reproductive health policy processes in resource poor contexts.’ Health
Research Policy & Systems. Vol. 9.
• Tsui et al (2014). ‘Monitoring and Evaluation of Policy Influence and Advocacy.’ ODI Working
Paper.

39. Thank You!
Questions/Comments/Feedback

40. For further inquiries, please
contact:
Dr. Modupe Adefeso-Olateju, mo.olateju@tepcentre.com
Rebecca O. Doherty, doherty.rebecca@gmail.com
THANK YOU