Self-Assessment of Organizational Capacity in Monitoring & Evaluation

SELF-ASSESSMENT OF ORGANIZATIONAL
CAPACITY IN MONITORING & EVALUATION
by
SNNP REGIONAL HEALTH BUREAU
SIDAMA ZONAL HEALTH DEPARTMENT

PRESENTATION of FINDINGS

05 December 2012

Proposed Time Schedule for Discussion
10 Minutes Introductions
PowerPoint Presentations
20 Minutes Terms & Concepts on:
* Capacity Building
* Organizational Development
* SCORE ME org. self-assessment tool of M&E capacity
30 Minutes SNNP Regional Health Bureau
Presentation of Findings from use of SCORE ME
30 Minutes Sidama Zonal Health Department
Presentation of Findings from use of SCORE ME
30 Minutes Questions, Comments, Next Steps

DEFINITIONS OF CAPACITY and CAPACITY BUILDING

 CAPACITY: Set of abilities and resources to support a level
of desired performance – whether by an individual or an
organization of any size

 CAPACITY BUILDING: Processes that improve the ability
of a person, group, organization, or system to meet
objectives and/or to perform better

CAPACITY BUILDING: SOMETHING TO THINK ABOUT
• Common approach is to provide training, funding, material
resources, and technical assistance

• Results would be more effective with consideration of
other factors influencing performance of an individual or
organization
 behavioral
 cultural
 systemic

MEASURE EVALUATION’S FOCUS in CAPACITY BUILDING
Support the ability of its partners to independently and
sustainably carry out activities in monitoring and evaluation
(M&E) and routine health information systems (RHIS)
Phase I (1998 – 2003) Phase III (2009 – 2014)
• Design of new tools ** Knowledge management
• Data analysis ** Organization development
Phase II (2004 – 2009)
• Routine health information
systems strengthening
• Data demand & use
• Data quality
• M&E technical training

FOCAL AREAS for CAPACITY BUILDING in M&E

Operational Functions:

• Management and development of human resources in RHIS
and M&E
• Material and financial resources, and systems to manage
them
• Technical RHIS and M&E skills
• Communication and facilitation of use of information

FOCAL AREAS for CAPACITY BUILDING in M&E

Leadership and Relationship Building:

• Promotion of culture favoring information use
• Strategic leadership
• Management of stakeholder relations

MEASURE EVALUATION’s CONCEPT of
ORGANIZATION DEVELOPMENT (OD) to SUPPORT M&E

OD Goal: Improve organization’s effectiveness in
managing its own efforts to achieving objectives

Strategy: Motivate individuals and groups within an
organization to seek and sustain best possible
performance

Method — Assist Organization to:
• Identify individual and institutional needs for obtaining
desired results and levels of achievement
• Identify its current strengths and challenges
• Increase capacity of those at management and leadership
positions to support and champion performance
improvement and positive responsiveness to change
• Improve M&E performance of organization to regularly
assess current strategies, structures, processes,
achievements, results

Needs of Organization:
• Clarity of organizational vision
• Comprehensive strategic planning
• Effective organizational structure
• Strong governance structure
• Strong capacity among leaders for promoting teamwork and
managing change
• Strong accountability systems (human and financial resource
management, inventory control)

Results — Organization is able to Self-Assess and Improve its
Accountability Systems for:
1. Planning and executing operational functions and achieving pre-set
objectives
2. Actively undertaking problem solving

3. Committing to its own organizational development as ongoing process
4. Learning and sharing knowledge and perceptions with organizations in
and across sectors
5. Assessing and accessing means for remaining operationally sustainable
and relevant to future needs and demands

Outcome — Improved organizational management and
performance in:

• Use of M&E methods and tools
• Collection and analysis of routine and strategic information
• Use of routine information for program M&E
• Use of strategic information for organizational planning
• Evidence-based decision making by organization’s leadership
and management
• Sustainability of all the above performance areas

Self-Assessment of
Capacity regarding
Organization’s
Response in
M&E

SCORE ME

Organization’s Functional Capacities
for Self-Assessment in M&E Performance
• Strategic Planning
• Budgeting and Managing Finances
• Implementation Planning
• Documenting & Reporting of Work Activities & Results for Monitoring
Oversight
• Documenting & Reporting of Work Activities & Results for Evaluation
Assessment
• Managing Facilities and Physical Assets
• Managing and Developing Staff
• Promoting Continuous Quality Improvement

Features of SCORE ME: Tool for
Self-Assessment in M&E Performance
 Developed as a self-administered tool for an M&E
Department/Unit of any agency (government/non-
governmental) to appraise its own level of functional
capacity in eight areas
 Designed so that its use on a periodic basis would
promote M&E skills and competencies
 Self-assessment in each functional area determined
by number of elements by which agency M&E
performance meets conditions (with reference made if
supporting evidence is available or not)

Features of SCORE ME (cont’d)

 Agency can customize the use and wording of M&E
performance elements to better suit its purpose for
self-assessment in M&E capacity
 Agency can select those functional areas of greater
interest on which to first focus its capacity building
efforts
 Agency can develop plans for M&E capacity building
that are based on a formalized pre-assessment of its
M&E performance

Introduction of SCORE ME to
Health Sectors in SNNP Region

• Review of content and use of SCORE ME instrument

• Organizational and individual self-assessments of M&E
capacity, with on-site technical assistance, as required

• Formulation of improvement plan for organizational
capacity in M&E, as based on self-assessment findings

SNNP REGIONAL HEALTH BUREAU FUNCTIONAL
PROCESSES
• CORE Processes
– Disease Prevention Services and Good Health Practices Promotion
– Curative Health Services
– Regulatory Oversight
– Public Health Emergency Response Services
– HIV/AIDS Preventive Health Services and Disease Control

• SUPPORTING Processes
– Health Program Planning and Health Program M&E
– Disease Diagnoses through Laboratory Services
– Medical Research
– Human Resources Management
– Financial Administration

Memorable Quotes
 "Measurement is the first step that leads to control and eventually to
improvement. If you can't measure something, you can't understand
it. If you can't understand it, you can't control it. If you can't control it,
you can't improve it." — H. James Harrington

 "What gets measured gets done, what gets measured and fed back
gets done well, what gets rewarded gets repeated." — John E. Jones

 "Without a standard there is no logical basis for making a decision or
taking action." — Joseph M. Juran

 "Nothing changes until you change; everything changes once you
change." — posted at Sidama Zonal Health Department

SCORE ME Self-Assessment of M&E Capacity
SNNP Regional Health Bureau
28-29 November 2012

A Strategic Planning
B Budgeting and Managing Finances
C Implementation Planning
D Documenting/Reporting Work Activities/Results for Monitoring Oversight
E Documenting/Reporting Work Activities/Results for Evaluation Assessment
F Managing Facilities and Physical Assets
G Managing and Developing Staff
H Promoting Continuous Quality Improvement


D Documenting/Reporting Work
Activities/Results for Monitoring Oversight
Elements of Capacity Condition
Building/Organizational Met? Evidence Available?
Development Performance in If “Yes”, Specify
M&E by Organization/Unit Check 
Unit has written policies /
procedures and docu-
mented evidence for
monitoring on a quarterly
or more frequent basis:
We have: written business process re-engineering
documents; facility standard documents; expenditure reports;
1. Work inputs  woreda-based plans; balanced score card; standard reporting
format; standard check-list; routine performance review;
database generated summary data; program-based reporting
Information, Education, Communication/ Behavior Change
Communication design, distribution, dissemination; integrated
2. Work activities 
supportive supervision feedback; quarterly HMIS review
feedback to zones


Elements of Capacity Condition
Building / Organizational Met? Evidence Available?
Development Performance in If “Yes”, Specify
M&E by Organization / Unit Check 

Review guidelines; process maturity assessment; standard
integrated supportive supervision; standard guidelines for
3. Work processes  reviewing, licensing, and inspecting health facilities; different
service delivery guidelines; laboratory external quality
assurance

Quarterly feedback on program-specific activities; monthly,
quarterly, semi-annual, and annual reports (physical and
4. Work outputs  financial); payroll reports; attendance sheets; various models
on in-and-out assets; epidemic monitoring charts; expanded
program for immunizations monitoring charts


D Documenting/Reporting for Monitoring Oversight
Unit has written
Condition
policies/procedures and
Met? Evidence Available?
documented evidence for
If “Yes”, Specify
responsiveness to
Check 
ensuring:
5. Collection of all

required work data
6. Timely collection and
transmission of all 
required work data
See the abovelisted justifications. Performance
review team, guidelines, reporting time frame,
7. High quality of all
 LQAS needs further improvement
required work data

8. Analysis of all
X
required work data


Unit has written Condition
policies/procedures and Met? Evidence Available?
documented evidence for If “Yes”, Specify
responsiveness to ensuring: Check 
Board minutes book for hospitals; guidelines and manuals
for HIV/AIDS; prevention board and council Management
9. Unit’s leadership/Board 
standards for woreda health offices, zonal health
Department, and Regional Health Bureau
Management standards and management minutes book for:
10. Unit’s management  hospitals, zonal health department, district health offices and
health centers
Procedures for fund allocations, fund liquidation; project
11. Unit’s funders and management manual; MOA, standard reporting, formal

supporters (financial and physical); standard checklist for project
appraisal

12. Unit’s target population Community-facility forum guidelines; human development
 army guidelines
representatives


Unit has written
Condition
policies/procedures and
regular review and use of
Check 
work data findings by:

Management standards
13. Unit’s leadership/Board  Board minutes book

Management standards
14. Unit’s management  Management minutes book


Scoring from Self-Assessment Findings

Total Number
of 13 / 4 or
5 or 6 7 or 8
9 or 10 or 12 or
Conditions
Met  14 fewer 11 more

1 2 3 4 5


Corrective Action Plan based on Self-Assessment Findings
MEANS for
Ele- DELIVERABLE(S) RESPONSIBLE PROJECTED
CURRENT ACTION(S) TO MONITORING
ment that can be PARTY for COMPLETN
SITUATION TAKE PROGRESS to
# DOCUMENTED DELIVERABLE(S) DATE
DELIVERABLE(S)

No
procedure
or Availability of
Develop Planning, M&E
guidelines Procedures, document at each By end of
8 for analysis
procedures or
guidelines level (of health care
Process Owner
May 2013
guidelines and partners
of all administration)
required
data


E Documenting/Reporting Work
Activities/Results for Evaluation Assessment
Unit has written policies/procedures
and documented evidence for having Condition
evaluated for work effectiveness and Met? Evidence Available?
efficiency the projections that had If “Yes”, Specify
been set in Annual Budgets and Check 
Implementation Plans:
Assignment of personnel based on BPR
1. Budgeting and deploying inputs  document; allocated funds (budgeting)
Process maturity evaluation of evaluation for-
2. Undertaking work processes  mat; BPR & BSC implementation manuals;
planning, monitoring, and evaluation processes
3. Completing planned activities X --
4. Generating projected outputs X --
5. Meeting pre-set objectives X --
6. Achieving desired outcomes  Operationalization of health research process
7. Contributing to desired impact  --


E Documenting/Reporting for Evaluation Assessment
Condition
Unit has documented evidence for
allocation of budget and
other resources for:
Check 

8. Collection of all required work Resource mobilization for HMIS; woreda-based
 planning for research
data and information
9. Quality assessments of work
X --
data
X --
performance

11. Analyses of work data X --


Condition
Check 

12. Reporting on work data to Monthly, quarterly, semi-annual and annual
 reports
concerned stakeholders

13. Using the findings from
reports for the improvement Annual performance used for evidence-based
 planning and written feedback
of planning and
implementation
Availability of research staff to do research and
14.Conducting research on ways technology transfer sub-process and
to promote work  disseminate research results; rapid
effectiveness and efficiency assessment conducted on data utilization for
decision-making



Total Number
of 8/ 4 or
5 or 6 7 or 8
9 or 10 or 12 or
Conditions

1 2 3 4 5


MEANS for
DELIVERABLE(S)
Amount of
budget mobilized
Financial reports
Inadequate Amount of
allocation for Revise Response letters
budget from partners SNNP RHB
quality resource
allocated
allocations Reports on
assessments, MEASURE June
1 performance Resource quality Evaluation 2013
Develop assessments
reviews, and mapping
resource and performance Partners
analysis of document
mapping reviews
developed
work data conducted
Reports on work
data analysis


MEANS for
DELIVERABLE(S)

Terms of reference
Skill gap in Capacity End of
developing Number of trained Selection of
building training January
evaluation M&E Officers and trainees
for basic MEASURE 2013
other focal
frame work for concepts of Evaluation (training)
2 persons Attendance lists
evaluation evaluation for
activities, out- M&E Officers SNNP RHB May 2013
M&E framework Training documents
puts, & pre-set and other focal M&E
developed
persons framework
objectives Pre- & post-test
evaluations


SCORE ME Self-Assessment of M&E Capacity
Sidama Zonal Health Department
01 December 2012

A Strategic Planning
B Budgeting and Managing Finances
C Implementation Planning
D Documenting/Reporting Work Activities/Results for Monitoring Oversight
E Documenting/Reporting Work Activities/Results for Evaluation Assessment
F Managing Facilities and Physical Assets
G Managing and Developing Staff
H Promoting Continuous Quality Improvement


Condition
Elements of Capacity Building /
Organizational Development Performance
in M&E by Organization / Unit
Check 
Unit has written policies/procedures and
documented evidence for monitoring
on a quarterly or more frequent basis:
 Strategic plans
1. Work inputs   Woreda-based plans
 Standard guidelines for different
2. Work activities  programs (TB, HIV, RH, TB/HIV
collaborative activities)
 Performance reports
3. Work processes   Business process re-engineering
 Balanced score card
  Standard checklists
4. Work outputs
 E-HMIS Summary reports


Condition
responsiveness to ensuring:
Check 

5. Collection of all required work

data
6. Timely collection and  HMIS tools and guidelines
transmission of all required work   Standard reporting schedules
 HMIS minutes book
data
 Performance Review Team
minutes book
7. High quality of all required work  Analysis of routine performance
X
data reports from respective woredas,
as well as within the Organization
(unit)
8. Analysis of all required work data 


Condition
responsiveness to ensuring:
Check 
 Management minutes book
 Management standards
10. Unit’s management 
 Regular monthly/quarterly
11. Unit’s funders and supporters  reporting, including review
meetings with concerned bodies
12. Unit’s target population  Contractual agreements with
 partners
representatives
 Standard checklists

 Management minutes book
 Management standards
14. Unit’s management 



Total
Number of 13 / 4 or
5 or 6 7 or 8
9 or 10 or 12 or
Conditions

1 2 3 4 5


MEANS for
CURRENT MONITORING
ment ACTION(S) TO TAKE that can be PARTY for COMPLETN
SITUATION PROGRESS to
DELIVERABLE(S)
 Criteria based
 Capacity
selection of
building on  Number of
trainees and
data manage- staff trained  MEASURE  Jan 2013
trainers
ment on evaluation Evaluation (capacity
 Training
 Appropriate framework  SNNP RHB building)
manuals
Low technology for  Number of  Sidama ZHD  Jan 2013
 Pre- and post-
data manage- offline HMIS  All partners (techno-
7 quality ment (software data
tests
working in logy)
data  Available
data analysis management health  April
technology for
at zonal as centers programs in 2013
offline data
well as woreda  Established Sidama (integra-
management
levels) One Report Zone tion)
 Established
 Data System
One Report
integration
System


E Documenting/Reporting Work
Activities/Results for Evaluation Assessment
Condition
documented evidence for having evaluated
for work effectiveness and efficiency
the projections that had been set in Annual
Check 
Budgets and Implementation Plans:
1. Budgeting and deploying inputs X --
2. Undertaking work processes  Performance Report
3. Completing planned activities X --
4. Generating projected outputs X --
5. Meeting pre-set objectives X --
6. Achieving desired outcomes X --
7. Contributing to desired impact X --


Condition
Check 

8. Collection of all required work
  Performance reports
data and information
X --
data
X --
performance
 Analysis of data from performance
reports
11. Analyses of work data   Analysis of data from project
evaluations


Condition
Check 
 Dissemination to community
representatives
12. Reporting on work data to
  Dissemination to partners
concerned stakeholders  Dissemination/communication within
the Organization

13. Using the findings from  Reviewing performance reports by
management teams
reports for the improvement
  Reviewing performance reports by
of planning and core-process/supporting process
implementation teams
14.Conducting research on ways
to promote work X --
effectiveness and efficiency



Total Number
of 5/ 4 or
5 or 6 7 or 8
9 or 10 or 12 or
Conditions

1 2 3 4 5


MEANS for
DELIVERABLE(S)

 Feedback  MEASURE
following Evaluation
budget  SNNP RHB
Resource  Resource  Amount of approval  Sidama ZHD
 Jan
1 (financial) (financial) budget  Amount of  All partners
2013
scarcity scarcity allocated budget really working in
spent for health
evaluation programs in
purposes Sidama Zone


MEANS for
CURRENT MONITORING
ment ACTION(S) TO TAKE that can be PARTY for COMPLETN
SITUATION PROGRESS to
DELIVERABLE(S)

 Criteria based
 MEASURE
selection of
 Institutionalizing Evaluation
 Documented trainees and
an evaluation  SNNP RHB
Failure culture within the
evaluation trainers
 Sidama
to plan plans  Training
Organization ZHD
 Number of manuals  Jan
3 for  Capacity  All partners
staff trained  Pre- and post- 2013
evalu- building in working in
on tests
planning and health
ations evaluation  Cross-
implementation programs in
frameworks checking the
of evaluation Sidama
plan
Zone
documented


MEASURE Evaluation is funded by the U.S. Agency for
International Development (USAID) and implemented by the
Carolina Population Center at the University of North Carolina
at Chapel Hill in partnership with Futures Group International,
ICF International, John Snow, Inc., Management Sciences for
Health, and Tulane University. Views expressed in this
presentation do not necessarily reflect the views of USAID or the
U.S. government.

MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.

Self-Assessment of Organizational Capacity in Monitoring & Evaluation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Self-Assessment of Organizational Capacity in Monitoring & Evaluation

Similar to Self-Assessment of Organizational Capacity in Monitoring & Evaluation (20)

More from MEASURE Evaluation

More from MEASURE Evaluation (20)

Self-Assessment of Organizational Capacity in Monitoring & Evaluation