Because international development increasingly focuses on gender, evaluators need a better understanding of how to measure and incorporate gender—including its economic, social, and health dimensions—in their evaluations. This interactive training, consisting of this presentation and a tool, will help participants learn to better evaluate programs with gender components. Access the tool at https://www.measureevaluation.org/resources/publications/tl-19-40

1. Seven Steps to EnGendering
Evaluations of Public Health
Programs
MEASURE Evaluation
August 2019

2. Agenda
20 minutes Vote with Your Feet
30 minutes Gender 101
45 minutes Seven Steps to EnGendering
Evaluation
1 hour Group Work: Case Studies
20 minutes Review from Group Work
5 minutes Wrap-Up

3. Activity: Vote with your feet!
• This will help us explore
gender concepts.
• Our own beliefs about
gender make a difference.
• We need to keep this in
mind when we integrate
gender in evaluations.
Source: United States Agency for International Development (USAID) Training of
Trainers: Gender and Reproductive Health 101

4. Gender 101

5. Definitions
SEX GENDER
Biological difference between males
and females:
• Universal for all human beings
• Usually unchanging
• Determined at birth
Beliefs about the appropriate roles,
duties, rights, responsibilities, accepted
behaviors, opportunities, and status of
women, men, transgender people, and
others in relation to one another:
• Constructed by society
• Differs between cultures and over
time
• Acquired
• Includes identity, expression, and
orientation
*Source: World Health Organization (WHO), 2009: Integrating gender into HIV/AIDS
programmes in the health sector

6. Roles, relations, and power among men, women,
boys, girls, transgender people, and others
• Focusing only on roles of women and girls ignores systems,
institutions, and structures in place, and the role of men and boys
within those systems.
• You cannot fully empower women and girls without also engaging
men and boys.
• Gender equality is good for everyone. Families and communities
benefit.
• Sexual and gender minorities are exposed to the same social
expectations and this can affect outcomes.
Gender
Beyond Women and Girls

7. Equality vs. equity
Equal treatment Fairness; compensation

8. Questions? discussion?

9. The 7 steps

10. Why integrate gender in
evaluation?
• Gender is everywhere
o Every development intervention has hierarchies, judgments,
etc., related to masculinity and femininity.
• Provide evidence to:
o Improve programs
o Address gender dimensions of health, education,
agriculture, etc. (doesn’t have to be a “gender” program)
o Raise awareness
• Identify gender-related goals and priorities based
on available information and consultation with
stakeholders, to better tailor evaluation to needs
and encourage data use.

11. • Ensure that all contribute to research
management and process.
• Ensure the research itself is gender-sensitive.
Why integrate gender in
evaluation? (2)

12. EnGendering evaluation
“Not everything that counts can be counted, and
not everything that can be counted counts.”
─Albert Einstein

13. 7 steps to engendering evaluations
1. Identify stakeholders and their roles
2. Develop gender-integrated theory of change
3. Define gender-related evaluation questions
4. Select appropriate study design
5. Select measures for gender-related outputs
and outcomes
6. Collect and interpret sex-disaggregated and
gender-sensitive data
7. Disseminate and use gender-integrated results

14. Step 1. Identify stakeholders
and their roles

15. Step 1. Identify stakeholders
and their roles
• Determine who the stakeholders are
o What is their role in the intervention?
o What do they gain?
o What might they lose?
o How much of a priority is their participation?
o At what stage should they be involved?
o What are their ways and capacities for participation?
• Engagement strengthens
accountability, trust, credibility, data
use, and equity

16. Step 2. Develop a gender-
integrated theory of change

17. Step 2. Theory of change
If we do these
activities
Then we think
get this result
in the short
term
Then we think
we’ll get this
result in the
long term
Project goal, based on solution
creation, derived from problem
analysis
Short-term outcome derived from thinking about the
short-term steps that would lead to the long-term impact
Activities derived from thinking about what concrete
actions will lead to the short-term outcome

18. Step 2. Example theories of change
If we train
teachers on
curriculum
delivery
Then
students will
receive
quality
instruction on
reading
Then
students ages
7 to 10 will
have
improved
literacy
Example: Simple Education Project Theory of Change
Example: Simple Gender-integrated Education Project Theory of Change
If we train teachers
on curriculum
delivery, with
attention to gender-
related differences
in learning
Then students of
all genders will
receive quality
instruction in
reading
Then
students of
all genders,
ages 7 to 10,
will have
improved
literacy

19. Step 2. Example gender-
integrated theory of change
PROCESS OUTPUT OUTCOME IMPACT
Curriculum
for
discussions
with men on
basics of
importance
of family
planning (FP)
Discussion
series with
males about
importance of
FP
Number of
workshops
with men on
importance of
FP
Short-term:
Increased
awareness of
importance of FP
Intermediate:
Increased
proportion of
men attending
FP clinics with
wives
Increased
proportion of
couples using
FP
INPUT

20. Step 3. Define gender-related
evaluation questions

21. Step 3. Define questions
Consider:
• What is the evaluation mandate?
• How strongly gender-integrated is the
program?
• What barriers to changes in gender norms
is the program trying to address?
• How does gender play a role in the theory
of change and what relationship(s) might
you want to examine?

22. Step 3. Define questions
Outcome/impact:
• Has the program had the same impact on health
outcomes for men and women?
• Has the program reduced power differences in
relations between men and women?
• Have stigma and discrimination against people
who do not follow traditional gender norms and
behaviors been reduced?
• Has the removal of gender-based constraints
contributed to improved health outcomes?
Examples

23. Step 3. Define questions
Process/implementation:
• Were gender-integrated program components
implemented as planned?
• What positive or negative unintended effects
on gender equality were identified during
implementation, if any? How were they
addressed?
• Were there any constraints (e.g., political,
practical, bureaucratic) to addressing gender
efficiently during implementation? If yes, what
efforts were made to overcome these
challenges?
Examples

24. Step 4. Select an appropriate
study design

25. Step 4. Study design
Consider:
• What are your gender-related questions?
• Measure unintended consequences?
• Measure complex norms, relationships?
• How sensitive is the topic/population?
• What types of evidence, funding, time, and
other common considerations do
stakeholders need?
• Can your methods be empowering?
(participatory methods)
Methods

26. Step 4. Study design
• Quantitative
o Answers what and gives aggregate picture
o Misses the quality of
participation/performance
 Qualitative
o Address the why and how
o Very important in gender
 Complex relationships hard to measure
quantitatively
 Inequality, inequity are lived experiences
 Mixed methods ideal
Methods

27. Step 4. Study design
To make determinations about the effects of a
program on men vs. women, boys vs. girls,
etc., you need a sample size of groups of
interest that is large enough
Need to consider sample size calculations,
whether quantitative or qualitative
Sampling

28. Step 4. Study design
• Consider ethical concerns affecting
sampling. For example:
o Gender-based violence (GBV): only interview
one woman per household
o May need to use method like “snowball
sampling” for some stigmatized populations
 E.g., countries where homosexuality is illegal and
you are working with men who have sex with
men (MSM)
Sampling (2)

29. Step 5. Select measures for
gender-related outputs and
outcomes
Indicator types:
• Sex-* and age-
disaggregated
o At a minimum
• Gender-sensitive
* Ideally gender-disaggregated

30. Gender-sensitive indicators
• No single “gold standard” for measuring
gender norms, attitudes, women’s
empowerment
• Use multiple measures
o Gender is a complex construct and
operates in multiple
spheres
o A scale combining
several items is more
valid than a single
item used alone

31. Gender-sensitive indicators
• Household decision-making power
• Experience of GBV
• Women’s autonomy and empowerment
• Economic empowerment
• Service delivery: gender equity, stigma,
and discrimination
• Couples counseling and male involvement
Examples (1)

32. Gender-sensitive indicators
• Women’s Empowerment in Agriculture Index
(WEAI): 5 domains
• Perspectives on gender norms
o Gender-Equitable Men (GEM) Scale; example
items:
 “If someone insults a man, he should defend his
reputation with force if he has to.”
 “A man needs other women even if things with his
wife are fine.”
Examples (2)

33. Gender-sensitive indicators
Demonstrate removal of gender-related barriers
• Barrier: perception that masculinity=many sexual
partners
• Measure success through:
o % of men/women who agree that men who have one sexual
partner are “real men”
o % of men who report that they have only one partner
Capture quality, not just quantity
• Not just training attendance but true participation and
decision-making roles
o % of trainees who have mastered relevant knowledge
• Not just numbers employed but also quality of jobs held
o % of management positions held by women
Selection tips

34. Step 6. Collect and interpret
sex-disaggregated and
gender-sensitive data
This Photo by Unknown Author is licensed under CC BY-NC-ND

35. Step 6. Collect
Consider:
• Logistics: Timing? Who is the interviewer?
Location?
• Questionnaire administration
o Gender biases that may affect data
 E.g., male overreporting of the number of
sexual partners; underreporting of GBV
• Ethical issues, particularly for GBV and other
sensitive topics and marginalized
populations
o Review relevant international guidelines

36. Step 6. Interpret
• Go back to evaluation question(s) before
starting the analysis
• Interpret within the context of literature
• Analyze data by sex and age, at a
minimum; include gender-sensitive
indicators (as applicable)
o If no differences, at least report that you
looked at subgroups and did not find
differences

37. Step 6. Interpret (2)
• Identify patterns and potential issues
related to gender in program
implementation and outcomes
• Ask:
o If health outcomes are not occurring but
gender outcomes are, what might be going
on? And vice versa?
o Are gender elements of programs actually in
place?
• Don’t be seduced by personal beliefs and
expectations; conclude only what results
indicate

38. Step 7. Disseminate and use
gender-integrated results
MEASURE Evaluation. (2017) Gender Matters Video. Chapel Hill, NC, USA: University
of North Carolina at Chapel Hill. See https://vimeo.com/219126971
0%
5%
10%
15%
20%
25%
Male Female

39. Step 7. Disseminate and use
gender-integrated results (2)
Ensure that the information is available,
accessible, relevant, and useful
Also consider:
• Implications of sharing results
o E.g., sharing location-based results related to illegal
behavior
• How recommendations affect different stakeholders
positively or negatively

40. Questions? discussion?

41. Gender evaluation resources
and tools
• Compendium of Gender Equality
and HIV Indicators (MEASURE
Evaluation)
• Violence Against Women and Girls
Compendium of M&E Indicators
(MEASURE Evaluation)
• Trafficking in Persons and Health: A
Compendium of M&E Indicators
(MEASURE Evaluation)

42. Gender evaluation resources
and tools (2)
• Family Planning and Reproductive
Health Indicators Database
(MEASURE Evaluation, PRH)
• HIV Indicator Registry (Joint United
Nations Programme on HIV/AIDS)
• Resource guide for gender data and
statistics (WHO, IGWG/USAID, and
MEASURE Evaluation)
• Guidelines for Gender-Based Analysis
of Health Decision Making (PAHO)
• K4 Health IGWG Gender and Health
Toolkit

43. Gender evaluation resources
and tools (3)
• Positive Youth Development
Measurement Toolkit (YouthPower)
• Girl-Centered Program Design: A
Toolkit to Develop, Strengthen, and
Expand Adolescent Girls
Programs (Population Council)
• Integrating Gender in the Monitoring
and Evaluation of Health Programs:
A Toolkit (MEASURE Evaluation)

44. Gender evaluation resources
and tools (4)
• Integrating Human Rights and Gender Equality in
Evaluations (United Nations Evaluation Group)
• Gender Matters (video)
(MEASURE Evaluation)
• Addressing Gender in Impact
Evaluation: What should be
considered? (ODI)

45. Group exercise
1. Two case studies: one on HIV and one
on healthy timing and spacing of
pregnancies
2. Divide into groups and choose a case
study
3. Review the case study
4. Answer questions
5. Report back in 45 minutes

46. This presentation was produced with the support of the United States Agency for
International Development (USAID) under the terms of MEASURE Evaluation
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is
implemented by the Carolina Population Center, University of North Carolina at
Chapel Hill in partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. Views expressed are not
necessarily those of USAID or the United States government.
www.measureevaluation.org