The presentation was the fourth webinar in a series of discussions on the popular MEASURE Evaluation manual, How Do We Know If a Program Made a Difference? A Guide to Statistical Methods for Program Impact Evaluation. The webinar was also referred to as "Within Estimators."
Trichy Call Girls Book Now 9630942363 Top Class Trichy Escort Service Available
Within Models
1. Peter M. Lance, PhD
MEASURE Evaluation
University of North Carolina at
Chapel Hill
December 15, 2016
Within Models
2. Global, five-year, $180M cooperative agreement
Strategic objective:
To strengthen health information systems – the
capacity to gather, interpret, and use data – so
countries can make better decisions and sustain good
health outcomes over time.
Project overview
3. Improved country capacity to manage health
information systems, resources, and staff
Strengthened collection, analysis, and use of
routine health data
Methods, tools, and approaches improved and
applied to address health information challenges
and gaps
Increased capacity for rigorous evaluation
Phase IV Results Framework
43. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1
The actual
causal effect of
P on y
44. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1
The actual
causal effect of
P on y
The actual causal effect of
the omitted variable X on
Y
45. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1
The actual
causal effect of
P on y
Th”Effect” of P on x:
𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖
The actual causal effect of
the omitted variable X on
Y
46. True model:
𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀
We actually attempt to estimate:
𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖
Error term now contains:
𝛽2 ∙ 𝑥
47. True model:
𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀
We actually attempt to estimate:
𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖
Error term now contains:
𝛽2 ∙ 𝑥
48. True model:
𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀
We actually attempt to estimate:
𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖
Error term now contains:
𝛽2 ∙ 𝑥
49. True model:
𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀
We actually attempt to estimate:
𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖
Error term now contains:
𝛽2 ∙ 𝑥
51. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1
The actual
causal effect of
P on y
Th”Effect” of P on x:
𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖
The actual causal effect of
the omitted variable X on
Y
150. Big Caveats/Limitations/Drawbacks
1.Loss of information
2.Makes measurement error bias worse
3.Very limited options for limited dependent
variables
4.Rooted in a weird kind of paradox
151. Big Caveats/Limitations/Drawbacks
1.Loss of information
2.Makes measurement error bias worse
3.Very limited options for limited dependent
variables
4.Rooted in a weird kind of paradox
153. Big Caveats/Limitations/Drawbacks
1.Loss of information
2.Makes measurement error bias worse
3.Very limited options for limited dependent
variables
4.Rooted in a weird kind of paradox
160. t=2000 t=2002
True age 35 37
Measured age 33 39
Error -2 +2
Error
𝐓𝐫𝐮𝐞 𝐚𝐠𝐞
.0571 .054
Age2002-Age2000
True age difference 2
Measured age
difference
6
Error 4
Error
𝐓𝐫𝐮𝐞 difference
1.5
161. Big Caveats/Limitations/Drawbacks
1.Loss of information
2.Makes measurement error bias worse
3.Very limited options for limited dependent
variables
4.Rooted in a weird kind of paradox
162. Big Caveats/Limitations/Drawbacks
1.Loss of information
2.Makes measurement error bias worse
3.Very limited options for limited dependent
variables
4.Rooted in a weird kind of paradox
185. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡
Controls for
fixed (ie underlying,
not time-varying)
differences between
program participants
and
non-participants
Controls for
underlying time trend
common to
program participants
and
non-participants
186. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡
Controls for
fixed (ie underlying,
not time-varying)
differences between
program participants
and
non-participants
Controls for
underlying time trend
common to
program participants
and
non-participants
187. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡
Controls for
fixed (ie underlying,
not time-varying)
differences between
program participants
and
non-participants
Controls for
underlying time trend
common to
program participants
and
non-participants
Program impact
190. Bertrand, Duflo and Mullainathan
Basic Experiment:
Take a dataset (current population survey) with labor
market outcomes (ln(earnings)) for many women-years
(900,000) and “make up” a fake program. Then try to
evaluate the impact of these fake programs with a DID
regression.
The Result:
The null hypothesis that the policy had no effect at the 5
percent level rejected a stunning 50-70 percent of the time,
depending on the econometric approach.
197. MEASURE Evaluation is funded by the U.S. Agency
for International Development (USAID) under terms
of Cooperative Agreement AID-OAA-L-14-00004 and
implemented by the Carolina Population Center, University
of North Carolina at Chapel Hill in partnership with ICF
International, John Snow, Inc., Management Sciences for
Health, Palladium Group, and Tulane University. The views
expressed in this presentation do not necessarily reflect
the views of USAID or the United States government.
www.measureevaluation.org