Population Council presentation for a MEASURE Evaluation webinar on "Measuring the Content of Postnatal Care for Women: What Do We Know and Where Do We Need to Go?" in September 2017. See the second presentation at https://www.slideshare.net/measureevaluation/overview-presentation-measuring-the-content-of-postnatal-care-for-women-83063221

1. VALIDATING INTERVENTION COVERAGE
INDICATORS FOR
MATERNAL POSTNATAL CARE
Ann K. Blanc and Katharine McCarthy
September 28, 2017

2. Overview
• Study 1 – immediate postnatal care (PNC)
(within 1 hour of birth)
• Study 2 – return PNC visit (24hrs – 10 weeks
of birth)
• Summary of overall findings

3. Health Risks in Postnatal Period
• More maternal deaths occur in period between first 24 hours of
birth up to 6 weeks after delivery than any other phase of
pregnancy (36%)
• 2 of 5 child deaths <age 5 occur within first 28 days of life
• Healthy behaviors (e.g., breastfeeding, immunization visits, use
of postnatal HIV services) can have lifelong effects on health
• Despite health risks during this period, only one-third of women
in LMIC receive a PNC visit within the first 2 days of birth (UN,
2015)
(1) United Nations, Millennium Development Goals Report, 2015.

4. Maternal PNC Coverage Indicators
in DHS/MICS
- After delivery, whether anyone checked on woman’s health (if
so who, when, and where check occurred)
- After left health facility [or two months following birth if a
home birth], whether anyone checked on woman’s health (if so
health (if so who, when, and where check occurred)

5. Newborn PNC Coverage Indicators in
DHS/MICS
- After delivery, whether anyone checked on newborn’s health (if so
who, when and where check occurred)
- After left health facility, whether anyone checked on newborn’s
health (if so who, when and where check occurred)
- Early initiation of breastfeeding (within 1 hour of birth)
- Newborn vaccination (DHS)
- Content of PNC check: examine cord, newborn temperature,
temperature, counsel woman on danger signs for newborns,
counsel woman on breastfeeding, observe breastfeeding (DHS R7)
- Newborn thermal care: drying, skin-to-skin contact, delayed
bathing
- Cord care (DHS Pregnancy & PNC Module)*
* Module is optional and currently being piloted

6. Research Question
How accurately can women report the content of PNC
received (1) immediately postpartum or (2) during a
return facility visit?
• Study 1: Immediate PNC Interventions (1hr of delivery)
– Studies in Kenya and Mexico (2012 - 2013) compare observations
of labor and delivery to women’s exit interview [1,2]
– In Kenya re-interviewed women 13-15 months later to examine
accuracy of reporting over time [3]
• Study 2: PNC Return Visit (24 hr - 10 wks of delivery)
– Secondary analysis collected under Integra Study in Kenya and
Swaziland (2009 – 2012)
– Postnatal women’s visit to a health facility
(1) Blanc et al., J Global Health. 2016: 6(1). (2) Blanc et al. BMC Preg & Childbirth. 2016:16(255). (3) McCarthy et al., J Global Health. 2016:6(2).

7. Validation Analysis
1. Individual Level Accuracy:
• Sensitivity (Se) and specificity
(Sp), summarized as area
under receiver operating
curve (AUC)
• 0 – 1 scale
2. Population Level Accuracy:
• Psurvey-based = (true coverage*Se)
+(true coverage*(1 – Sp))
• Inflation factor (IF)- ratio of
estimated survey-based prevalence
to true coverage
- AUC<0.60 = low validity
- 0.60<AUC<0.7 = moderate
- AUC>0.70 = high validity
Acceptable overall performance: AUC>0.60 and
0.75<IF<1.25
- 0.75 < IF < 1.25 = low bias

8. Study 1: PNC Interventions Received
Immediately Postpartum (within 1
hour of birth)
• Kenya and Mexico
• 2012 – 2013

9. Study 1 Design. Immediate PNC
Women with matched baseline data
N = 662
Successfully
located
N = 568
Location Rate:
93.7%
Not located, moved
or death
N = 38
Non-Location Rate:
6.3%
Completed
interview
N = 515; Follow-
up Rate: 90.7%
Refused or did not
complete interview
N = 53; Refusal Rate:
9.3%
515 women with matched
baseline and follow-up data.
Sample:
• Women ages 15-49
whose labor &
delivery was
observed
• 2 hospitals in
Central and Eastern
Kenya; 1 hospital in
Mexico City
• In Kenya re-
interviewed women
who consented to
follow-up at their
Mexico
Women who consented to home-
based follow-up interview
N = 606 (Acceptance rate: 91%)
13-15monthspostLaboranddelivery
Consented to study
participation
N = 616
Successfully observed
during labor
N = 609 (Observation
rate: 98.8%)
Completed exit
interview
N = 597 Follow-
up Rate: 98%
597 women
with matched
data
Lost to follow-up
or refused survey
N=12; Loss to FU
rate 2%
Could not be
located or sent
home N=7; Loss
to FU rate 1.2%
Kenya
Consented to study
participation
N = 1039
Did not progress
into labor, sent
home, lost to FU
N=373; Loss to FU
rate 35.8%

10. Study 1. Sample Characteristics
Kenya Sample: N=662
baseline; N=515 follow-up
• Mean age: 26 ± 0.22
• 48% 1 prior birth, range: 1-8
prior births
• 44% primary school is
highest education
• 85% married or living
together
• 13% cesarean rate
• Mean age: 24 ± 0.31
• 52% 1 prior birth, range: 1-7
prior births
• 92% completed secondary
school or higher
• 74% married or living
together
Mexico Sample: N=597
baseline

11. Aspects of Immediate Maternal PNC
Measured (within 1 hour of birth)
Uterine massage performed following delivery of placenta
Physical exam following delivery to:
• Check for bleeding
• Take blood pressure
• Check perineum
• Take temperature
• Examine whether belly was becoming firm

12. Immediate PNC Interventions for Mother (w/in 1
hr), Indicator Accuracy
Immediate PNC for Mother
(up to 1 hr after birth)
KENYA MEXICO
Individual
Accuracy
R0 | R1
Population
Accuracy
R0 | R1
Individual
Accuracy
Population
Accuracy
After delivery of placenta, provider
firmly massaged lower abdomen to
help womb become firm
- | - - | + - +
In the first physical examination after delivery, did a health
provider….
Look for or ask you about
bleeding
- | - - | + - +
Check your blood pressure - | - - | + NA NA
Do a perineal exam - | - - | - - +
Take your temperature + | - - | + - -
Check belly to see if it was
becoming firm (involution)
- | - + | - - +
R0 = Exit interview at hospital discharge; R1 = Re-interview 13-15 months postpartum.
Individual Accuracy: ⎼ =AUC<0.60; + = 0.60<AUC<0.70; ++ = 0.70<AUC; Population Accuracy: + = 0.75<IF<1.25;
NA = insufficient sample size for country.

13. Qualitative Interviews: How Difficult is it for
Women to Recall Immediate PNC Care?
• Kenya: In-depth interviews (N=20) with sub-sample of
women at re-interview (13 to 15 months postpartum)
• Relative to interventions received in the first stage of labor,
women had difficulty recalling immediate postnatal care,
particularly for the mother
“You know that time [after the birth] I was over
excited so after the caesarean section I was happy
to see my child like this and I gave God my thanks,
so I cannot know because once I saw the baby I
was tired so whatever happened after that I don’t
know”.

14. Women’s Immediate Postnatal Experiences in
Kenya
“…When you asked me if the baby was placed on my chest
against my skin, that was hard for me to remember because
because at that time I was tired because I had gone without
gone without sleep for two days.”

15. Study 2: PNC Interventions
Received 24 Hours – 10 weeks of
Birth
• Kenya and Swaziland
• 2009 – 2012

16. Study 2 Design. PNC Interventions Received
within 24hrs - 10 weeks of birth
1. Direct Observation of Health Visit
12 health facilities in Kenya (KY); 8 health facilities in Swaziland
(SZ)
• KY: Central/ Eastern provinces; SZ: Lubombo, Manzini,
Shiselweni regions
• Part of SRH and HIV care integration study
• High volume (>50 infant immunizations, >100 FP
clients/ month)
• Offered PNC, FP, voluntary HTC, STI treatment,
PMTCT
2. Exit Interview with Women on Received Care at
Facility Discharge
• Women ages 15 - 44 years
• Presenting at study health facility for health check for
themselves and/or their infant (aged >24 hrs – 10 wks)
• Informed consent
Assess
validity
of self-
reports

17. Study 2. Sample Kenya
N=545
Swaziland
N=319
% %
Age of Client
15-24 44.2 54.9
25-34 44.8 38.4
35-44 11.0 7.3
Age of Baby
<2wks 21.3 16.6
2-6wks 66.2 76.4
7-10wks 12.5 7.0
Married/live together 86.0 44.7
Prior Parity
1 29.9 34.2
2 23.2 25.3
3+ 47.0 40.5
Education Level
None /pre-primary 39.8 7.5
Primary 41.1 18.5
Secondary+ 19.1 74.0

18. Study 2. Aspects of Maternal PNC
Measured
• Maternal physical exam
• Maternal danger signs advice
• Return to fertility and birth spacing information
• Family planning methods discussion / provision
• STI/HIV risk assessment

19. Study 2. Validation Results
• 18 maternal PNC indicators attempted
• 13 indicators in KY and 15 in SZ had adequate
sample size for validation

20. Study 2. Indicators of PNC Examination for
Mother
INDICATORS
Kenya
Individual |
Population-level
Accuracy
Swaziland
Individual |
Population-level
Accuracy
Met Both
(AUC & IF)
KY | SZ
Take blood pressure ++ | + + | + KY | SZ
Examine breasts ++ | + ++ | + KY | SZ
Examine abdomen ++ | + ++ | + KY | SZ
Examine vagina + | + ++ | + KY | SZ
Check anemia (pallor or refer for HB
test)
++ | + - | + KY
Screen for cervical cancer NA | NA + | -
KY = Kenya; SZ = Swaziland; NA = Insufficient sample size for country
Individual Accuracy: ⎼ =AUC<0.60; + = 0.60<AUC<0.70; ++ = 0.70<AUC
Population Accuracy: + = 0.75<IF<1.25

21. Study 2. Indicators of PNC Provider Contact
and Health Counseling for Mother
INDICATORS
Kenya
Individual |
Population-level
Accuracy
Swaziland
Individual |
Population-level
Accuracy
Met Both
(AUC & IF)
KY | SZ
Contact with nurse or nurse/midwife NA | NA NA | NA NA
Contact with doctor NA | NA NA | NA NA
Ask about excessive bleeding ++ | + - | + KY
Discuss danger signs after birth ++ | + - | + KY
Discussed STIs or HIV/AIDS ++ | + - | + KY
KY = Kenya; SZ = Swaziland; NA = Insufficient sample size for country
Individual Accuracy: ⎼ =AUC<0.60; + = 0.60<AUC<0.70; ++ = 0.70<AUC
Population Accuracy: + = 0.75<IF<1.25

22. INDICATORS
Kenya
Individual |
Population-level
Accuracy
Swaziland
Individual |
Population-level
Accuracy
Met Both
(AUC & IF)
KY | SZ
Discuss how soon after delivery a
woman can get pregnant
+ | + - | + KY
Discuss return to fertility + | + - | - KY
Discuss benefits of birth spacing ++ | + + | + KY | SZ
Discuss return to sexual activity ++ | + + | - KY
Discussed a FP method (incl natural) ++ | + - | + KY
Received any modern FP method NA ++| + NA | SZ
Explains advan/disad of chosen FP
method
NA + | + NA | SZ
Study 2. Indicators of PNC Return to
Fertility & Family Planning Counseling
Individual Accuracy: ⎼ =AUC<0.60; + = 0.60<AUC<0.70; ++ = 0.70<AUC
Population Accuracy: + = 0.75<IF<1.25; NA = Insufficient sample size for country

23. Summary of Maternal PNC Indicators
Immediate PNC:
• No indicators met both accuracy criteria in Kenya or Mexico
• 4 indicators were appropriate for population-level measurement only in
each country
• Whether (1) received uterine massage and (2) provider checked for
bleeding met at population level accuracy only in both countries
Return PNC:
• 13/13 maternal PNC indicators in KY and 7/15 in SZ met individual and
population-level accuracy
• 5 PNC indicators met both criteria in both countries
Whether during the PNC consultation the provider…
• Checked the woman’s blood pressure
• Did a breast exam
• Examined the woman’s abdomen
• Examined the woman’s vagina
• Discussed the benefits of birth spacing

24. Summary of Maternal PNC Indicators
• In general, women are able to report on maternal PNC
interventions received at a return visit with higher accuracy than
those received immediately (within 1 hour) postpartum
• The same trend was true for newborn PNC indicators (not
presented)
• Influence of delivery context?
• Findings differ by country, lower overall accuracy in Mexico and
Swaziland vs. Kenya
• Limitation: Can’t distinguish between differences in understanding
of the question, wording, recall, or error introduced by poor
interviewing
• Findings suggest additional content of care PNC indicators can
be accurately measured

25. This work was supported by the Maternal Health
Task Force through the Bill & Melinda Gates
Foundation and the Intervention Coverage
Measurement Group, also through the Bill & Melinda
Gates Foundation.
Findings of these studies are based on work in
collaboration with Council colleagues Karla
Berdichevsky, Claudia Diaz, James Kimani, Brian
Mdawida, Charity Ndwidga and Charlotte Warren.

26. The Population Council conducts research
and delivers solutions that improve lives
around the world. Big ideas supported by
evidence: It’s our model for global change.
Ideas. Evidence. Impact.

27. Questions
• There are only a handful of validation studies; almost all
conducted in hospitals or high-volume clinics. Do we need to
examine accuracy of reporting among women who gave birth in
other types of facilities (or at home)?
• Do we need to test longer recall periods? Do we need to
examine validity in other settings?
• What are alternatives to population-based surveys for collecting
data on intervention coverage for PNC?
• Can we identify a (small) set of critical and valid indicators that
can be recommended for inclusion in household surveys (e.g.,
DHS, MICS)?

28. © 2014The Population Council. All rights reserved.
Use of these materials is permitted only for noncommercial purposes.
The following full source citation must be included:
This presentation may contain materials owned by others. User is responsible for
obtaining permissions for use from third parties as needed.
Ann K. Blanc, Charlotte Warren, Katharine McCarthy, Brian Mdawida. Validating
intervention coverage indicators for maternal postnatal care. Webinar hosted by
MEASURE Evaluation. September 2017.