Authored by Ernest Fetogang, Suzanne Cloutier, Sergio Lins, Amanda Makulec, Rosinah T. Dialwa, and Tom Achoki and presented at the 2013 Global Maternal Health Conference.

1. Botswana RDQA.ai 1 1/7/2013 10:28:57 AM
BOTSWANA ADAPTATION OF THE RDQA
Global Maternal Health Conference
Authors: Ernest Fetogang, Suzanne Cloutier, Sergio Lins, Amanda Makulec, Rosinah T. Dialwa, and Tom Achoki.
Ministry of Health Botswana John Snow Inc. John Snow Inc. John Snow Inc. Ministry of Health Botswana Ministry of Health Botswana
I Background II Methodology/Approach III Botswana’s Approach
Information is a key building block of a health system, and efforts to improve The core objectives of the collaboration between the MoH and MEASURE Basic measures to ensure standards of data quality should be taken to ensure
data quality directly support improvements in a country’s information system Evaluation were to both accuracy and reliability at all levels of the health system. The data quality
across programme areas. (1) describe the process for ensuring data quality at the service delivery, protocols in Botswana were developed to address this challenge, and are appli-
district, and national levels, and cable at all levels of the health system.
Figure 1: Information in the WHO Health System Building Blocks (2) provide guidelines for data quality monitoring procedures. Figure 3: Ideal Botswana Health Data Flow The ideal data flow for the
Ultimately, these activities aimed to insure accuracy, completeness, and timeli- Botswana MoH is illustrated
MoH External National
THE WHO HEALTH SYSTEM FRAMEWORK ness of health data being transmitted in Botswana and define responsibilities Management, Stakeholders Programs, in Figure 3. Botswana health
Policy, Planners HSU
for data quality at each level of the health information system. data currently flow through
SYSTEM BUILDING BLOCKS OVERALL GOALS/OUTCOMES
more than 39 different infor-
SERVICE DELIVERY Other ministries
The development of the protocols and curriculum was conducted over the Consolidation; Analysis; National [e.g. Ministry of mation systems, including
Data Dissemination M&E
course of one year. Home Affairs, etc.] both electronic and paper-
HEALTH WORKFORCE ACCESS IMPROVED HEALTH (LEVEL AND EQUITY)
based systems that feed
COVERAGE
into various data manage-
INFORMATION RESPONSIVENESS January Establish scope of work & objectives Consolidation DHMT ment systems. With the
MEDICAL PRODUCTS, VACCINES & TECHNOLOGIES SOCIAL AND FINANCIAL RISK PROTECTION February Develop B-RDQA Excel tool creation of a national M&E
March Draft Standard Operating Procedures (SOPs) unit, the Ministry is working
QUALITY
FINANCING SAFETY IMPROVED EFFICIENCY April Pre-test B-RDQA tool Hospitals, to streamline processes
Data Private Clinics, NGOs,
Review SOPs Producers Sector Mobile CBOs, and move towards this
FBOs
LEADERSHIP/GOVERNANCE Stops ideal flow.
May Finalize B-RDQA tool
Revise SOPs A global conceptual framework for data quality was adapted to reflect the
The ability of health system stewards to make strategic decisions is impacted by
the quality of health data. At the national level in Botswana, data ultimately June Finalize SOPs Botswana data flow and priorities within the country’s M&E system.
inform budget and policy decisions. In the Health Districts and Service Delivery July-October Develop data quality curriculum
Sites, data enables providers and monitoring and evaluation (M&E) officers to November Conduct data quality training workshop
understand the broader health activities and priorities in their respective areas. Figure 4: Botswana Conceptual Framework for Data Quality
Figure 2: Data & Health Impact Key deliverables in the process included:
Dimensions of Quality
1. Data Quality SOP—General, high level protocol for ensuring data quality QUALITY DATA Accuracy, Completeness, Relability, Timeliness,
Policy & at the service delivery, district, and national levels Confidentiality, Precision and Integraity
Health data Health data Assessment
budget
is collected aggregated of data 2. Routine Data Quality Assessment SOP— Protocol for the implementation Functional Components of a Data Management
decisions System Needed to Ensure Data Quality
at service at district & impacts of RDQAs as a monitoring tool to routinely review the quality of data at the
impact
delivery national policy & service delivery, district, and national levels I M&E Structure, Functions & Capabilities
health
sites levels budgets
Data Management and
outcomes 3. Customized RDQA Tool for Botswana (B-RDQA Tool) National M&E II Indicator Definitions & Reporting Guidelines
Reporting System
REPORTING LEVELS
Data collection Aggregation & Analysis Impact on health 4. B-RDQA Tool User Manual—Detailed guidance on the implementation III Data-collection & Reporting Forms/Tools
and results dissemination from conducting an RDQA using the B-RDQA Tool Health Districts IV Data Management Processes
To support improved data quality throughout the health system, the Botswana on up to four indicators within any health programme
V Training
Ministry of Health (MoH) collaborated with experts from MEASURE Evaluation to 5. Data Quality Curriculum—Curriculum for use in MoH trainings on data Service Delivery Sites
develop a national procedure for routine monitoring of data quality and provid- quality, including presentations, exercises, and a full participant’s guide. VI Use of Data for Decision Making
ing specific guidance on developing action plans to address challenges using a Curriculum covers the content of the two SOPs and the collection and use
bottom-up approach. Botswana is the first country to adapt the RDQA method- of data from routine data quality assessments.
ology and tools for national use.
Figure 5: Timeline for implementation of RDQA
Suggested follow up Suggested follow up
Full RDQA Baseline full RDQA full RDQA
Data verifications & Data verifcations & Data verifcations &
System assessment System assessment System assessment
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Data verifications Data verifications Data verifications Data verifications Data verifications Data verifications
IV Adaptation of the B-RDQA Tool VI
Year 1 Year 2
Data Quality Curriculum & Training Note that if a follow-up full RDQA was not
conducted at the end of Year 1, a full RDQA must
be conducted at the end of Year 2 inorder to have
an up-to-date system assessment.
C Figure 6: Functional Areas The B-RDQA Tool is an Excel tool Finally, a complete curriculum was developed by MEASURE Evaluation to train
M
Y
of the M&E System with multiple worksheets for a user national and district M&E officers on how to implement and use the SOPs and
CM to complete to verify data at various the B-RDQA Tool. The curriculum underpins a two and a half day training with a
MY
levels of the health system and con- balance of presentations and hands-on exercises that give attendees first-hand
CY
duct a system assessment to evalu- experience using the tool, interpreting outputs, and developing action plans.
VII
CMY
M&E
ate the key functional compo-
structures,
Resource Requirements
K
functions & nents of the M&E system. The A training of 22 M&E officers was conducted in November 2012, and a training of
capabilities tool was customized with trainers is planned for March 2013. Overall feedback on the first training was very
Indicator changes to the language positive, indicating that the SOPs and RDQA process would be useful for use both
Use of data
definitions
for decision used to describe the various at the district and national levels as a routine tool for improving data quality. Time & cost: The process of developing the SOPs, customized tool, training ma-
and reporting
making
quidelines levels of the health system to terials, and conducting the first training took approximately one year. The total
Six functional reflect the Botswana data flow. cost, primarily in staff time and travel for in-country consultation workshops
areas of an and training, was US$250,000, funded by the United States Government
M&E system
The adaptation of the tool through MEASURE Evaluation.
Data collection
included a customization
and reporting
Training workshop in April 2012,
forms and “This training makes our DQA knowledge Staff: At the MoH, the newly formed Department for Health Policy, Monitoring,
tools
where specific recommenda- clear & improved; developed skills that and Evaluation (DHPME) initiated the activities with MEASURE Evaluation. A
Data tions were made on content Principal Health Officer was a key champion for the process, supported by the
management changes to reflect the needs of
we expected for over a decade.” Chief Health Officer and Monitoring & Evaluation Advisor. The MEASURE Evalua-
processes
the MoH. One of the significant - Trainee tion team that worked with the MoH included three Senior M&E Advisors and
additions to the tool was the addition of the two M&E Associates.
“use of data for decision making” functional area
in the system assessment component of the tool. “This [process] will really reduce work burden… Travel: A total of four trips were made to work in-country with the MoH and
The importance of this component was reinforced in subsequent consultative other stakeholders including:
workshops and at the November training, where district M&E officers identified very exciting, can’t wait to implement. This was 1) January 2012—Planning visit to develop the scope of work.
the use of data for decision making as a key challenge. one of the best trainings which will really address 2) April 2012—B-RDQA Tool customization workshop and pretesting.
our district data quality problems.” 3) June 2012—Consultative workshops to finalize SOPs and user manual.
Following the customization workshop, the B-RDQA tool was pretested in the 4) November 2012—Training for M&E officers, at the invitation of the MoH.
field with the Family Planning and Reproductive Health Programme. – District Health Officer
VIII. Keys to Success
Country ownership: The development and implementation of protocols for im-
“A very good training that came at the right time, proving data quality was initiated by the MoH, who approached MEASURE
Evaluation for technical assistance to adapt global tools to the Botswana con-
providing skills that are sustainable and very easy text. The country-led foundation of this process has been essential in connect-
to use…Bringing out very valuable results to ing with the correct stakeholders to give input and insight.
V
improving health information systems, important
Standard Operating Procedures to system improvement and decisions making.”
Champions: Also key to the entire process was having a strong champion for
data quality activities at the MoH. Without a strong technical voice supporting
- Trainee the investment in protocols to improve data quality, it would have been chal-
lenging to find the momentum to support the development and implementa-
The SOPs for data quality and RDQA, as well as the B-RDQA User Manual were tion of the protocols.
drafted for review while the final B-RDQA Tool was being customized. Originally,
the team envisioned one comprehensive SOP and user guide. To make the docu- Decentralization: Finally, the protocols decentralize the process of planning
ments more user-friendly, the SOP was divided into two SOPs and a user manual. targeted activities to improve data quality, allowing service delivery sites and
district level officials to take ownership of data quality in a systematic and struc-
The data quality SOP was written as a high level document on the various tured way. Service delivery sites and districts develop their own recommenda-
dimensions and considerations of data quality, intended for senior MoH officials, tions and action items, putting the power in local hands.
other policymakers, and M&E officers. The RDQA SOP was written as a general
protocol for conducting an RDQA, including responsibilities by level, intended A challenge, moving forward, will be the need for continued implementation
for any MoH or district official responsible for initiating, managing or conducting support. The sustainability of these protocols and the use of the customized
routine assessments. The B-RDQA user manual was written specifically for those tool will rely on continued support—both technical and financial—to train staff
staff using the B-RDQA tool to conduct assessments in the field at service deliv- and encourage use of the B-RDQA Tool for monitoring data quality.
ery sites.
Draft SOPs and a draft user
manual were reviewed and
discussed with stakeholders
IX
at consultative workshops in
June 2012. Both Ministry and
external stakeholders partici-
Conclusions
pated in the consultations,
and documents were final- Over the next year and beyond, results of regular system assessments and rou-
ized based on the recom- tine data verification exercises could be analyzed to evaluate the impact of the
mendations from the work- SOPs and use of the RDQA process. With growing interest and investment in
shops. Final documents were health system strengthening measures, the Botswana adaptation of global data
printed in country for distri- quality tools operationalizes a system for health information system improve-
bution by the MoH. ments that could be adopted by other countries facing data quality challenges.
The development of this poster was supported by funds from the USAID MEASURE Evaluation project.
MEASURE Evaluation is funded by USAID through cooperative agreement GHA-A-00-08-00003-00 and implemented by the Carolina Population Center at the
University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University.
The views expressed in this publication do not necessarily reflect the views of USAID or the United States government.