This presentation from the International Food Policy Research Institute (IFPRI) provides an overview of the CARE Strengthening the Dairy Value Chain Project impact evaluation design.

1. Evaluating the Dairy Value Chain Project
in Bangladesh: Baseline Study
Akhter Ahmed
International Food Policy Research Institute
Seminar at CARE-Bangladesh
Dhaka
May 28, 2009

2. Storyline
SDVCP objectives
Evaluation methodology
Baseline data
Characteristics of survey households
Gender related issues
Dairy farming practices
Qualitative review of value chain actors
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

3. SDVCP objectives
1. Improve milk collection systems in rural and remote
areas
2. Improve access to inputs, markets, and services by
mobilizing groups of poor farmers, producers, and char
dwellers
3. Improve the milk transport network
4. Ensure access to quality service at the producer level
5. Improve the policy environment
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

4. The baseline study
CARE has commissioned IFPRI to conduct a thorough
evaluation of the SDVCP that would allow CARE to make
informed decision of whether to close, revise, extend, or
expand the SDVCP
CARE has also contracted the Data Analysis and
Technical Assistance Limited (DATA) to collect
quantitative and qualitative information for the evaluation,
under guidance and supervision of IFPRI
This baseline study is a part of the evaluation. It was
specifically designed to permit a scientific and rigorous
evaluation of impacts of the SDVCP through follow-up
studies
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

5. Impact Evaluation Methodology
Impact is the difference between outcomes (e.g., income,
school enrollment, women’s empowerment, etc) with the
program and without it
The goal of impact evaluation is to measure the this
difference in a way that can attribute the difference to the
program, and only the program
Use Difference-in-differences method that compares
observed changes in the outcomes for program participants
(treatment) and non-participating comparison group
(control), before and after the program
Combine with propensity score matching to adjust for pre-
program differences
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

6. Propensity Score Matching (PSM)
Match program participants (treatment) with non-
participants (control) at baseline (before intervention)
Each program participant will be paired with a non-participant that
is similar
Use PSM to pick an ideal comparison group from the
baseline survey data
The comparison group will be matched to the treatment
group using “propensity score”
Propensity score is predicted probability of participation given
observed pre-program characteristics of participants and non-
participants
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

7. Illustrating Difference-in-Difference
Estimate of Average Program Effect
PA
Impact = (PA - CA) - (PB - CB)
Program
CA
Control
PB=CB
Baseline Follow-up
(Before) (After)
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

8. Constructing the counterfactual:
Two control groups
The control or comparison groups are comprised of eligible
but non-participant households
Two control or comparison groups of households have
been created to assess the impact and to capture the
potential spillover effects
The nature of SDVCP interventions may generate spillover effect
of the project. For example, if new dairy production technologies
are introduced, non-beneficiaries may copy these
Control 1 households have been selected from unions
where the SDVCP is operating
Control 2 households have been selected from upazilas
without any milk chilling pants in the nine project districts
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

9. Baseline data
Used both quantitative and qualitative data for the baseline
study
Quantitative data came from a comprehensive household
survey designed by IFPRI and carried out by DATA
The survey questionnaire was designed to collect
information on multiple topics, including household
demographic composition, level of education, school
participation, occupation and employment, dwelling
characteristics, assets, food and nonfood expenditures,
morbidity, economic shocks, anthropometric measurements
of children and women, and a detail module on dairy farming
practices
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

10. Baseline data …
The household survey was carried out in 9 SDVCP districts,
27 upazilas, and 60 villages
Sample size: Determined by power calculation with design
effect. Total sample of 1,510 households, of which 659
program participants, 425 Control 1, and 426 Control 2
households
Survey started on August 20, 2008, and completed on
September 14, 2008. Data entry completed by end October.
Data cleaning, including logical consistency checking and
data validation completed by mid-January 2009
Qualitative data collection: Used key-informant interview,
focus group discussion, and observation methods
INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE