The document summarizes Botswana's collaboration with MEASURE Evaluation to develop national standards for routine monitoring of health data quality. Key deliverables included standard operating procedures, a customized Routine Data Quality Assessment tool, a training curriculum, and workshops. This helped establish a process for data quality monitoring at all levels of the health system using a bottom-up approach. Feedback on the trainings was positive and districts have begun implementing data quality tracking.

1. JOHN SNOW, INC.
BACKGROUND
To support improved data quality throughout the health system, the Botswana Ministry of Health
(MoH) collaborated with experts from MEASURE Evaluation to develop a national procedure for
routine monitoring of data quality and providing specific guidance on developing action plans to
address challenges using a bottom-up approach.
Because information is a key building block of a health system, efforts to improve data quality
directly support improvements in a country’s information system across program areas.
1 2 3
METHODOLOGY
The core objectives of the collaboration between the MoH and MEASURE Evaluation were to:
1. describe the process for ensuring data quality at the service delivery, district, and national levels,
2. and provide guidelines for data quality monitoring procedures.
The development of the protocols and curriculum occurred over the course of a year (2012),
followed by training workshops (2012/2013).
January Established scope of work & objectives
February Developed Botswana-RDQA Excel tool
March Drafted standard operating procedures (SOPs)
April Pre-tested B-RDQA tool in the field
Reviewed SOPs
May Finalized B-RDQA tool
Revised SOPs
June Finalized SOPs
July-Oct. Developed data quality curriculum
November Conducted 1st data quality training workshop
April Conducteddataqualitytrainingoftrainers(ToT)workshop
Supported 2nd data quality workshop
KEY DELIVERABLES IN THE PROCESS INCLUDED:
1. Data Quality SOP—General, high level protocol for ensuring data quality at all levels
of the health system
2. Routine Data Quality Assessment SOP—Protocol for the implementation of RDQAs
as a monitoring tool
3. Customized RDQA Tool for Botswana (B-RDQA Tool)
4. B-RDQA Tool User Manual—Detailed guidance on implementing and using the B-RDQA Tool
5. Data Quality Curriculum—Curriculum for use in MoH trainings on data quality
ADAPTATION OF THE B-RDQA TOOL
The B-RDQA Tool is an Excel tool with multiple worksheets for a user to complete to verify data at
various levels of the health system and conduct a system assessment to evaluate the key functional
components of the M&E system. The tool was customized for the Botswana country context, with
changes to the language used to describe the various levels of the health system to reflect the
Botswana data flow. One of the significant additions to the tool was the addition of the “use of
data for decision making” functional area in the system assessment component of the tool.
BOTSWANA’S APPROACH
The data quality protocols in Botswana were developed to ensure basic measures would be taken
to ensure accuracy, timeliness and completeness of health data throughout the health system.
The Botswana adaptation of the RDQA is unique as it provides a national protocol for routine data
quality assessment across various levels of the health system, as well as across program areas.
The ideal data flow for Botswana health data is illustrated below. Botswana health data currently
flow through more than 39 different information systems, including both electronic and paper-
based systems that feed into various data management systems. With the creation of a national
M&E unit, the MoH is working to streamline processes and move towards this ideal flow.
STANDARD OPERATING PROCEDURES
The data quality SOP was written as a high level document on the various dimensions and
considerations of data quality, intended for senior MoH officials, other policymakers, and M&E
Officers. The RDQA SOP was written as a general protocol for conducting a RDQA, including
responsibilities by level, intended for any MoH or district official responsible for initiating, managing
or conducting routine assessments. The B-RDQA Tool User Manual was written specifically for
thosestaffimplementingtheB-RDQAtoolandconductingassessments
in the field at service delivery sites.
Draft SOPs and a draft user manual were reviewed and discussed with
stakeholders at consultative workshops. Both Ministry and external
stakeholders participated in the consultations, and documents were
finalized based on the recommendations from the workshops. Final
documents were printed in country for distribution by the MoH.
DATA QUALITY CURRICULUM
& TRAINING
A complete curriculum was developed by MEASURE Evaluation to train national and district M&E
officers on how to implement and use the SOPs and the B-RDQA Tool. The curriculum supports a two
and a half day training with a balance of presentations and hands-on exercises that give attendees
first-hand experience using the tool, interpreting outputs, and developing action plans.
The 1st training of 22 M&E Officers was
conducted in November 2012, followed by
a ToT workshop in April 2013 with select
participants from the first data quality
training. The trainers then conducted the
2nd training with support from MEASURE
Evaluation. Overall feedback on the trainings
was very positive, indicating that the SOPs
and RDQA process would be useful both at
the district and national levels as a routine
tool for improving data quality.
KEYS TO SUCCESS
Country ownership: The development and implementation of protocols for improving data quality
was initiated by the MoH, who requested technical assistance to adapt global tools to the
Botswana context. The country-led foundation of this process has been essential in connecting
with the correct stakeholders to give input and insight. Furthermore, responsibility for the in-
country expenses proved beneficial with respect to the MoH taking ownership of the process.
Going forward, the MoH will conduct an annual national M&E forum where data quality issues will
be addressed by M&E Officers based on findings from their RDQAs. The MoH also has identified
a consultant to develop the National Health M&E Plan for the ministry, which will include RDQAs
as a routine activity for districts and at national level.
Champions: Also key to the entire process was having a strong champion for data quality
activities at the MoH. Without a strong technical voice supporting the investment in protocols
to improve data quality, it would have been challenging to find the momentum to support the
development and implementation of the protocols.
Decentralization: Finally, the protocols
decentralize the process of planning targeted
activities to improve data quality, allowing
service delivery sites and district-level officials
to take ownership of data quality in a systematic
and structured way. Service delivery sites and
districts develop their own recommendations and
action items, putting the power in local hands.
Evidence of the success of the RDQA process was
found at a district outside the capitol. A district
M&E officer implemented a log book (right) to
track data quality indicators after attending the
April 2013 training workshop.
Quality of health data impacts a government’s ability to make strategic health-related decisions,
underscoring the importance of maintaining high quality data. At the national level, data ultimately
inform budget and policy decisions. In the District Health Management Teams (DHMTs) and
Service Delivery Sites, data enable providers and monitoring and evaluation (M&E) officers to
understand the broader health activities and priorities in their respective areas.
The development of this poster was supported by funds from the USAID MEASURE Evaluation project.
SERVICE DELIVERY
HEALTH WORKFORCE
INFORMATION
MEDICAL PRODUCTS, VACCINES & TECHNOLOGIES
FINANCING
LEADERSHIP/GOVERNANCE
IMPROVED HEALTH (LEVEL AND EQUITY)
RESPONSIVENESS
SOCIAL AND FINANCIAL RISK PROTECTION
IMPROVED EFFICIENCY
SYSTEM BUILDING BLOCKS
OVERALL GOALS / OUTCOMES
ACCESS
COVERAGE
QUALITY
SAFETY
4
5
M&E structures,
functions &
capabilities
Use of data for
decision making
Indicator
definitions
and reporting
guidelines
Training
Data management
processes
Data collection
and reporting forms
and tools
Six functional
areas of an M&E
system
Botswana’s integration of data quality assurance into standard operating procedures:
6
RESOURCE REQUIREMENTS
Time & cost: The process of developing the SOPs, customized tool, training materials, and
conducting the trainings took about 16 months. The total cost to MEASURE Evaluation, primarily
in staff time and travel for in-country consultative workshops and training, was $300,000, funded
by the United States Agency for International Development. The MoH was responsible for funding
the in-country workshops, including venue, per diems, and transportation costs.
Staff: At the MoH, the newly formed Department for Health Policy, Monitoring and Evaluation
(DHPME) initiated the activities with MEASURE Evaluation. The Principal Health Officer was a
key champion for the process, supported by the Chief Health Officer. The MEASURE Evaluation
team that worked with the MoH included three Senior M&E Advisors and two M&E Associates.
Travel: A total of five trips were made to work in-country with the MoH and other stakeholders
including:
1. January 2012—Planning visit to develop the scope of work
2. April 2012—B-RDQA Tool customization workshop and pilot testing
3. June 2012—Consultative workshops to finalize SOPs and user manual
4. November 2012—Training of M&E officers
5. April 2013—Training of trainers and training of M&E officers
7 8
“This [process] will really
reduce work burden…
very exciting, can’t wait to
implement. This was one of
the best trainings which will
really address our district
data quality problems.”
– District Health Officer
“A very good
training that came at the right time,
providing skills that are sustainable and
very easy to use…Bringing out very
valuable results to improving health
information systems, important to system
improvement and decisions making.”
- Trainee
MEASURE Evaluation is funded by USAID through cooperative agreement GHA-A-00-08-00003-00 and implemented by the Carolina
Population Center at the University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc., Management
Sciences for Health, and Tulane University.
The views expressed in this publication do not necessarily reflect the views of USAID or the United States government.
ADAPTION OF THE ROUTINE DATA QUALITY ASSESSMENT TOOL
FIGURE 1: INFORMATION IN THE WHO HEALTH SYSTEM BUILDING BLOCKS
Health data are
collected at
service delivery
sites
Health data are
aggregated at
district & national
levels
Assessment
of data impacts
policy &
budgets
Policy & budget
decisions impact
health outcomes
Data collection Aggregation & Analysis Impact on health
FIGURE 2: DATA & HEALTH IMPACT
FIGURE 3: IDEAL BOTSWANA HEALTH DATA FLOW FIGURE 4: BOTSWANA CONCEPTUAL FRAMEWORK FOR DATA QUALITY
FIGURE 6: FUNCTIONAL AREAS OF THE M&E SYSTEM
FIGURE 5: TIMELINE FOR IMPLEMENTATION OF RDQA
Authors: Suzanne Cloutier, Sergio Lins, Amanda Makulec,
David Boone, Ernest Fetogang, Rosinah T. Dialwa,
and Segametsi Duge
With growing interest and investment in health system strengthening measures, the Botswana adaptation of global data quality tools operationalizes a system for health information system improvements that could be adopted by other countries facing data quality challenges.
Having conducted this adaptation in Botswana, the customization approach has been tested and streamlined, and select deliverables (e.g. data quality training curriculum) could be adapted to other country contexts.
CONCLUSIONS: