Presented at the July 2014 International AIDS Conference.

1. Strengthening M&E system among partners implementing
HIV/AIDS projects in Tanzania: MEASURE Evaluation’s experience
BACKGROUND
Tanzania’s health system faces
monitoring and evaluation (M&E)
challenges such as insufficient human
resource and organizational capacity,
poor quality and limited use of data.
Community-based HIV programs face
even greater challenges because
they rely on volunteers with limited
education to generate information on
services.
As part of implementing the National
M&E Framework for HIV, the U.S.
Agency for International Development
(USAID Tanzania) funded MEASURE
Evaluation-Tanzania (MEval/TZ) to
improve strategic information and
quality of data through M&E system
strengthening among organizations
providing HIV services in Tanzania.
Since 2008, MEval/TZ has conducted
data quality assessments (DQA) and
provided M&E capacity building
through customized trainings and
mentoring among 28 partners involved
in HIV care and treatment, prevention,
most-vulnerable children (MVC), home-based
care (HBC) and counseling and
testing (CT) programs.
DESCRIPTION
Our approach involves DQA and
use of DQA findings to undertake
customized M&E system strengthening
interventions with selected
implementing partners (IPs). Tools
used include routine M&E system
strengthening (MESS) tool for overall
assessment of the M&E system and
routine data quality assessment (RDQA)
tool for assessing verification factor
of data already reported, reporting
performance and system assessment.
We also use a Community Trace and
Verify (CTV) tool at community level
to verify services provided to clients
of community-based programs such
This research has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Agency for International
Development (USAID) under the terms of MEASURE Evaluation cooperative agreement GHA-A-00-08-00003-00, which is implemented by
the Carolina Population Center at the University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc.,
Management Sciences for Health, and Tulane University. The views expressed in this publication do not necessarily reflect the views of
PEPFAR, USAID or the United States government.
PRESENTED BY
Z. Kibao
Y.W. Mapala
D. Walker
K. Sono
J. Patrick
W.O. Odek
L.S. Wami*
K. F oreit
MEASURE Evaluation,
Futures Group
*JL Consultancy,
Dar es Salaam,
Tanzania
20th International
AIDS Conference
July 20–25, 2014
Melbourne, Australia
CONTACT US
MEASURE Evaluation
400 Meadowmont Village Circle, 3rd Floor
Chapel Hill, NC 27517 USA
www.measureevaluation.org
email: measure@unc.edu
Tel: +1.919.445.9350
Fax: +1.919.445.9353
as HBC and MVC. The DQA covers
an IP’s M&E unit, satellite offices and
service-delivery points.
Following the DQA, a capacity
building plan is developed to address
gaps identified by the DQA team. The
plan includes organized M&E trainings
and on-site mentoring sessions. The
training content include basic M&E
skills, development of the M&E plan,
data management, data demand and
use and geographical information
systems (GIS).
A mini-DQA is conducted for tracking
changes following the previous DQA
round and capacity building provided
LESSONS
LEARNED
As a result of the DQA process, 10
out of 23 implementing partners that
have had more than one DQA round
CTV was found to effectively and
efficiently pinpoint strengths and
weaknesses in data collection.
Household visits also provided
opportunities to assess other indicators
of child well-being and program
coverage.
CONCLUSIONS
& NEXT STEPS
DQA can be instrumental in improving
the M&E systems of HIV program
implementers by identifying capacity
gaps and galvanizing programs to
take corrective actions. We have
undertaken the following additional
measures to improve sustainability.
Standard Operation
Procedure
We have developed standard
operating procedures to ensure that
each round of DQA is conducted the
same way. This document describes
all procedures and instruments used in
the DQA and is reviewed regularly to
accommodate emerging or changed needs.
Building Local Capacity
Eval/TZ contracted a local firm, JL
Consultancy (JLC) to conduct the
DQAs. This is a multi-year process.
Currently JLC is involved in planning
for the DQA, sampling, actual field
work, conducting debriefs to IPs and
report writing. The sustainability of this
approach will be assessed.
M&E Technical Support
to Government
MEval/TZ provides M&E technical
assistance to the Ministry of Health and
Social Welfare, other line Ministries
and decentralized health management
teams to improve data quality and
data use.
Figure 1: DQA team conducts verification of services provided
to the house hold with MVC.
Figure 2: DQA team conducts data verification to one of the visited sites
now undertake internal DQA as an
institutional standard for improved
data quality. All 23 implementing
partners assessed more than once have
shown improvements in their M&E
systems over time. Budget allocation
to M&E functions has also improved,
surpassing the conventional 7% of total
program budget among 9 of the 23
implementing partners with more than
one DQA round, while 4 others have
maintained the recommended minimum
M&E budget allocation.
30
25
20
15
10
5
0
No. of IPs
IPs assessed through DQA rounds
Received multiple rounds (Mini-DQA)
IPs conduct internal DQA
Improved M&E budget
28
23
10 9
Figure 3: IPs assessed, received multiple DQA rounds, conduct Internal DQA
and have improved M&E budget as identified during Mini-DQAs (2008-2013)
MEval/TZ has used DQA findings to
customize capacity building, with an
emphasis on on-the-job mentoring on
identified weaknesses, beginning with
systemic factors such as M&E staffing
and documented procedures. Post-training
DQA have found measureable
improvements in quality of M&E plans,
performance of M&E units and data
validity. In addition, many partners
have hired dedicated M&E staff to
implement the activities recommended
from the DQA. We encourage M&E
staff from the headquarters of the
IPs to strengthen M&E capacity at
lower levels through direct, supportive
supervision and additional training
and mentoring.
We have undertaken mini-DQA to
measure changes in M&E capacity. At
the beginning, we went back only to
the sites that had been chosen for the
original DQA and re-assessed only
areas that had scored below average
on the original assessment. We now
include all sites, regardless of previous
assessment status. We have also
expanded follow-up assessments to all
areas of the M&E system.
WEPE416.indd 1 7/3/14 9:48 AM