The technical assistance helped build monitoring and evaluation capacity in Cambodia, Lao PDR, and Vietnam through training, research, and knowledge sharing. Training programs and country case studies improved skills and understanding of monitoring and evaluation. While an online community and evaluation networks faced challenges, the technical assistance achieved its outcome of strengthened evaluation capacities to support better development results and poverty reduction.

1. The views expressed in this presentation are the views of the author/s and do not necessarily reflect the views or policies of the Asian
Development Bank, or its Board of Governors, or the governments they represent. ADB does not guarantee the accuracy of the data included
in this presentation and accepts no responsibility for any consequence of their use. The countries listed in this presentation do not imply any
view on ADB's part as to sovereignty or independent status or necessarily conform to ADB's terminology.
Evaluation Capacity
Development
Olivier Serrat
2014

2. The Planning, Monitoring, and
Evaluation Triangle
Planning
MonitoringEvaluation
Recommendations are
made for future planning
Plans show what
to evaluate
Monitoring revises
plans during project
implementation
Plans show what needs
to be monitored
Evaluation highlights areas
that need close monitoring
Monitoring provides data
to be used in evaluation

3. Define:Monitoring
• Monitoring is the continuous collection of data
and information on specified indicators to
assess the implementation of a development
intervention in relation to activity schedules
and expenditure of allocated funds, and
progress and achievements in relation to its
intended outcome. Monitoring
• Involves day-to-day follow-up of activities
during implementation to measure progress
and identify deviations;
• Requires routine follow-up to ensure
activities are proceeding as planned and are
on schedule;
• Needs continuous assessment of activities
and results;
• Answers the question, "what are we doing?"
Monitoring

4. Define:Evaluation
• Evaluation is the periodic assessment of the
design implementation, outcome, and impact
of a development intervention. It should assess
the relevance and achievement of the intended
outcome, and implementation performance in
terms of effectiveness and efficiency, and the
nature, distribution, and sustainability of
impact. Evaluation
• Is a systematic way of learning from
experience to improve current activities and
promote better planning for future action;
• Is designed specifically with the intention to
attribute changes to the intervention itself;
• Answers the question, "what have we
achieved and what impact have we had?"
Evaluation

5. The Results Chain
Outputs
Outcome
Impact
ActivitiesInputs

6. Outputs, Outcome, Impact
Outputs—the
products, capital
goods, and
services that
result from a
project; they
may also include
changes resulting
from the project
that are relevant
to the
achievement of
its outcome.
Outcome—the
likely or achieved
short-term and
medium-term
effect of a
project's
outputs.
Impact—the
positive and
negative,
primary and
secondary, long-
term effect
produced by a
project, directly
or indirectly,
intended or
unintended.

7. OECD-DAC Evaluation Criteria
Relevance—This criteria examines the extent to which the objectives
of a project matched the priorities or policies of major stakeholders
(including beneficiaries).
Effectiveness—This criteria examines whether outputs led to the
achievement of the planned outcome.
Efficiency—This criteria assesses outputs in relation to inputs.
Impact—This criteria assesses what changes (intended and
unintended) have occurred as a result of the work.
Sustainability—This criteria looks at how far changes are likely to
continue in the longer term.

8. The Results Chain and the OECD-
DAC Evaluation Criteria
Needs
Objective Inputs Activities Outputs
Outcome
Impact
Relevance Efficiency
Effectiveness
Sustainability

9. Challenges and Limits to
Management
Logic
Degree of
Control
Challenge of
Monitoring
and
Evaluation
Impact
What the
project is
expected to
contribute to
Outcome What the
project can be
expected to
achieve and
be
accountable
for
Outputs What is within
the direct
control of the
project's
management
Activities
Inputs DecreasingControl
IncreasingDifficulty

10. Life Cycle of Monitoring
and Evaluation
EA MT EPEA MT EPEA MT EP
Key: EA = ex-ante, MT = mid-term, EP = ex-post

11. Trends in Evaluation Capacity
Development
Build Post-Evaluation
Capability
Build Monitoring and
Evaluation Systems
Establish Performance
Management Systems

12. Evaluation Capacity Development
Precondition
3
Precondition
2
Precondition
1
Three critical
preconditions to success
of evaluation capacity
development are (i)
substantive government
demand, (ii) existence of
a mandate by decree for
evaluation, and (iii)
stability in staffing such
that a very high
proportion of trained
personnel remain in
tasks for which they were
trained.

13. Evaluation Capacity Development
Good
practice in
developing
evaluation
capacity
also
recognizes
that:
• Monitoring and evaluation systems are a means
to an end—benefits are obtained when results
are used in decision making.
• It is advisable to locate responsibility for
monitoring and evaluation near the capable
head of an organization.
• Monitoring and evaluation systems should not
become too complex or resource-intensive.
• Monitoring and evaluation systems encompass
data collection in the field and aggregation and
analysis by end users.
• Evaluation capacity development that
concentrates on the oversight agency carries
the risk that other entities may lack incentives
to provide data and information.
• Case studies help develop staff competency and
confidence.

14. TA 6410–REG: Capacity Development
for Monitoring and Evaluation

15. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Outputs
1. Proficiency in
monitoring and
evaluation is
raised
2. Research and
special studies
on evaluation
capacity
development
are conducted
3. Knowledge
sharing and
learning for
monitoring and
evaluation are
boosted
Outcome
Improved ranges of skills,
resources, systems, and
attitudes for performance―
in the evaluation agencies
targeted―of results-based
monitoring and evaluation
of country partnership
strategies, sector strategies,
policies, programs, and
projects in developing
member countries of the
Greater Mekong Subregion,
namely, Cambodia, Lao PDR,
and Viet Nam
Impact
Higher
efficiency
and
effectivenes
s in
providing
public sector
services,
leading to
poverty
reduction

16. Key: M&E = monitoring and evaluation, ECD = evaluation capacity development
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Proficiency in
M&E
Tools, Methods,
and Approaches
for M&E
Strategy and
Policy
Formulation for
M&E
Research and
Special Studies
for M&E
Country
Strategies for
M&E
A Strategy for
ECD

17. Key: M&E = monitoring and evaluation
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Knowledge Sharing
and Learning for
M&E
Knowledge Sharing
and Learning
Platforms
Knowledge Networks
Partnership
Arrangements with
Evaluation
Associations

18. Output Level: Assumptions and Risks
Assumptions Risks
ADB
• Technical assistance activities integrate the chief lessons
learned from past evaluations
• Appropriate, integrated training programs can be planned,
designed, or identified; and synergetic effects can be achieved
• Training is conducted well and according to realistic schedules
• The consultants and the selected evaluation agency staff
coordinate activities effectively
• The consultants have client management skills
• The consultants and the selected evaluation agency staff
maintain clear roles, responsibilities, and deadlines
Clients
• Evaluation agency staff are available to be trained
ADB
• The indicative activities and
staffing schedule is too tight
to permit productive
sequencing of key activities
TA 6410–REG: Capacity Development
for Monitoring and Evaluation

19. Assumptions Risks
Clients
• Basic capacity exists and can be mobilized
• The role and use of monitoring and evaluation
in support of practices of knowledge
management are understood
• The funding agency has a clear vision about the
intended outcome of the technical assistance
and how it is to be achieved
Clients
• Evaluation agencies underestimate the
importance of national ownership and
leadership of the evaluation process and of
building national monitoring and evaluation
capacities
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Outcome Level: Assumptions and Risks

20. Assumptions Risks
ADB
• Able to plan and support evaluation capacity
development in line with international best
practice
Clients
• Provide visible leadership, promote clear
sense of mission, encourage participation,
and establish explicit expectations on
performance and rewards
• Strategically approach change management
and manage it proactively
• Involve a critical mass of staff
• Try, test, and adapt organizational
innovations
• Celebrate quick wins
Clients
• Lack of human security; armed conflict; economic
policies that discourage pro-poor growth; weak
scrutiny by the legislative branch of the executive
branch; ineffective voice of intended
beneficiaries; and corruption, clientelism, or
patrimonialism do not provide a broadly enabling
environment for monitoring and evaluation
• Fragmented government with poor overall
capacity; absent, noncredible, and/or rapidly
changing policies; unpredictable, unbalanced, or
inflexible funding and staffing; poor public service
conditions; segmented and compartmentalized
organizations; or insufficient commitment to an
evaluation culture do not conduce to government
effectiveness
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Impact Level: Assumptions and Risks

21. ADB
Center for
Development and
Research in
Evaluation
(Shanghai)
International
Program for
Development
Evaluation Training
Asia-Pacific Finance
and Development
Center
Regional Cooperation and Poverty Reduction Fund
of the People's Republic of China
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Technical Assistance Partnerships

22. TA 6410–REG: Capacity Development
for Monitoring and EvaluationTajikistan
• Amirov Fakhriddin K., State
Budget Department,
Ministry of Finance
Lao PDR
• Vixay Xaovana, Committee
for Planning and Investment
• Akhom Praseuth, Bank of
Lao PDR
• Bounthay Leuangvilay,
Budget Department,
Ministry of Finance
Cambodia
• Hou Taing Eng, Ministry of
Planning
• Im Sour, Cambodian
Rehabilitation and
Development Board
• Suon Sophal, Cambodian
Investment Board
• Lors Pinit, Department of
Investment and
Cooperation
• Hay Sovuthea, Supreme
National Economic Council
Malaysia
• Arunaselam Rasappan, Center for
Development and Research in Evaluation
• Mariappan Mahalingam, Center for
Development and Research in Evaluation
Viet Nam
• Tran Ngoc Lan, Ministry of
Planning and Investment
• Nguyen Dang Binh, Ministry
of Planning and Investment
• Pham Thai Linh, Ministry of
Natural Resources and
Environment
• Nguyen Trang Thu, National
Academy of Public
Administration
Selected Evaluation
Agencies

23. Project Year
1 2
1. Proficiency in M&E is raised.
1.1 Proficiency in tools, methods, and approaches for M&E is raised.
(i) Regional training-of-trainers (SHIPDET)
(ii) National training-of-trainers
1.2 Proficiency in strategy and policy formulation for M&E is raised.
(i) Thefoundations for M&E in GMS DMCs arestrengthened.
(ii) GMS DMCs areassisted in setting long-term, consistent strategies.
(iii) International training (IPDET)
2. Research and special studies on ECD areconducted.
2.1 Development of country strategies for M&E is supported.
2.2 Astrategy for ECD is suggested.
3. Knowledgesharing and learning for M&E areboosted.
(i) Selected knowledgesharing and learning platforms areenhanced.
(ii) Adviceon new and existing knowledgenetworks on M&E is extended to
evaluation agency staff fromGMS DMCs.
(iii) Partnership arrangements with interested evaluation associations
promoted and concluded.
Legend:
Full-timeactivity Intermittent activity
AFDC =Asia-Pacific Financeand Development Center, CeDRE =Center for Development and Research in Evaluation, IM&E =International
Monitoring and Evaluation Specialist, IPDET =International Programfor Development Evaluation Training, M&E =monitoring and evaluation
Activity
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Technical Assistance Schedule

24. Training and Capacity
Building
Research and
Special Studies
Knowledge
Sharing and
Networking
Strategic Direction for Evaluation Capacity Development
Strengthened Evaluation Capacity
Improved Service Delivery Leading to Poverty Reduction
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Expected Short-, Medium-, and Long-Term Deliverables

25. Stage A1 (Apr. 2008) Introductory Training in M&E for Country Trainers (ToT) CeDRE
In-Country Preparatory Work by Trainees In-CountryStage A2 (Apr. –Sept. 2008)
Policy Level M&E Training (Round 1) IPDET 1Stage B1 (June 2008)
Intermediate M&E Training for Trainees Lao PDRStage A3 (Oct. 2008)
In-Country Stage 1 Down-line Training by Trainees In-CountryStage A4 (Nov. 2008–Mar. 2009)
Advanced Level M&E Training for Trainees CambodiaStage A5 (Apr. 2009)
Policy Level M&E Training (Round 2) IPDET 2Stage B2 (June 2009)
In-Country Stage 2 Down-line Training by Trainees In-CountryStage A6 (Jul.–Sept. 2009)
Wrap-Up Training & Certification of Country Trainers AFDC/ADBStage A7 (Oct. 2009)
TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Training Strategy

26. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Evaluation of Output 1
Output 1 had two components: (i) a three-stage training-of -trainers (ToT)
program in monitoring and evaluation (M&E) (1–2 weeks per stage),
provided by the TA's international consultant to 18 mid-level career
officials recommended by ADB's Greater Mekong Subregion coordinators
from the three countries covered (6 per country, including one national
consultant each); and (ii) a 1-week country-based M&E downline training,
provided to 20 key officials selected as M&E focal persons from relevant
government agencies in the three DMCs. Most activities under this output
were implemented and completed on track. Subsequently, the officials
trained under the ToT program successfully conducted M&E downline
workshops in their respective countries. Attendance at the Shanghai
International Program for Development Evaluation Training (SHIPDET) was
found particularly useful.

27. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Evaluation of Output 2
In synergy, the two components of output 2 circumscribed the M&E
situation and clarified training needs in each of the three countries
covered. Activities under this output were generally implemented well,
even if the final M&E report was delayed. The detailed country case
studies provided valuable information on the status, progress, and
challenges of M&E. In each of the three countries covered, areas of
investigation included (i) institutional structure, policy formulation, and
strategy for M&E; and (ii) illustrative approaches to and practical
application of M&E policy measures.

28. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Evaluation of Output 3
The first of output 3's two components set up an M&E Community of
Practice website at www.linkingtoresults.org, which the national
consultants engaged under the TA were tasked to promote through
knowledge sharing on current M&E practices and issues in the three
countries covered. Despite these efforts, the website was used sparingly
due to (i) limited broadband access in government offices, especially in
Cambodia and Lao, PDR; (ii) insufficient language skills in English,
necessary to enable efficient online communications; and (iii) lack of
time. The second component aimed to facilitate the emergence of
national evaluation networks/societies. In the immediate, regulatory
restrictions in the three countries covered delayed this effort; but,
Cambodia and Viet Nam may soon move forward as registrations are
reportedly being processed by the relevant authorities.

29. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Achievement of Outcome
Despite limitations in output 3, the successful delivery of outputs 1 and 2
contributed to the achievement of the TA's outcome. This was reflected in
the fact that the TA helped create, under the ToT program, an internal
pool of at least 6 M&E resource persons in each of the three countries
covered, who in turn were able to successfully conduct downline M&E
training workshops for key officials selected as M&E focal points from
concerned government agencies. Feedback from those attending the
downline workshops indicated the usefulness of the workshops in
building their understanding of M&E and its role in managing for
development results (MfDR) as the workshops were customized to
national contexts.

30. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Overall Assessment and Rating
The overall performance of the TA was rated successful based on the
combined ratings of relevant, efficient, effective, and likely sustainable.
Most TA activities were generally implemented efficiently and completed
on time. This led to good delivery of outputs, except for output 3 where
use of the M&E CoP website fell short and national evaluation
networks/societies were not immediately established. Notwithstanding,
the successful delivery of outputs 1 and output 2 led to effectiveness in
achieving outcome in terms of improved ranges of M&E skills and
awareness of the importance role of M&E. In terms of design, the TA's ToT
program was found relevant: it enabled the key officials trained as
trainers to provide downline M&E training workshops to relevant staff of
concerned agencies in their respective countries. This is expected to help
sustain the outcome achieved.

31. TA 6410–REG: Capacity Development
for Monitoring and Evaluation
Major Lessons
Five lessons stand out: (i) baseline situational analysis of M&E status and
needs helped design relevant training courses; (ii) the training courses,
which began with a ToT program, helped create pools of trainers able to
deliver country-based M&E downline training workshops to staff in
concerned agencies in the three countries covered, which ought to
sustain and internalize the M&E knowledge and capacity gained; (iii) there
should be no rush to design outputs susceptible to in-country obstacles,
e.g., low levels of IT advancement, tight regulatory environments, unless
they can be addressed first; (iv) the TA should have worked more closely
with ADB's resident missions, which could have provided focal points in
support of TA implementation; and (v) a follow-on experience sharing
workshop should have been planned to estimate the magnitude of
"actual" applications of the M&E knowledge and capacity gained and to
gauge retention, on M&E-related tasks, of the officials trained.

32. Further Reading
• ADB. 2007. Independent Evaluation at the Asian Development
Bank. Manila. www.adb.org/publications/independent-
evaluation-asian-development-bank
• ——. 2007. Regional Technical Assistance for Capacity
Development for Monitoring and Evaluation. Manila.
www.adb.org/projects/documents/capacity-development-
monitoring-and-evaluation-0
• ——. 2009. Learning from Evaluation. Manila.
www.adb.org/publications/learning-evaluation

33. Further Reading
• ADB. 2009. Regional Technical Assistance for Capacity
Development for Capacity Development in Results-Based
Monitoring and Evaluation for Countries under the Asian
Development Bank's Central Asia Regional Economic
Cooperation Program. Manila.
www.adb.org/projects/documents/capacity-development-
results-based-monitoring-and-evaluation-countries-under-
asia
• ——. 2011. Technical Assistance Completion Report: Regional
Technical Assistance for Capacity Development for Monitoring
and Evaluation. Manila.
www.adb.org/projects/documents/capacity-development-
monitoring-and-evaluation

34. Further Reading
• Britton, B. and Serrat, O. 2013. Learning from Evaluation.
www.slideshare.net/celcius233/learning-from-evaluation

35. Quick Response Codes
@ADB
@ADB Sustainable
Development Timeline
@Academia.edu
@LinkedIn
@ResearchGate
@Scholar
@SlideShare
@Twitter