High Profile Call Girls Jaipur Vani 8445551418 Independent Escort Service Jaipur
Data Demand and Use Workshop
1. Data Demand and Use
Workshop
Isabel Brodsky and Verne Kemerer
MEASURE Evaluation
October 24–27, 2017
Freetown, Sierra Leone
2. Knowledge learning objectives
Describe the role of data demand and use (DDU)
in a functional M&E system
Define DDU
Understand the context of data use
Identify barriers to data use and develop
recommendations to overcome them
Identify locally relevant programmatic questions
and link those questions to available data
Understand types of descriptive data analysis
Skills learning objectives
Use tools and strategies to facilitate data use during
decision making
Workshop Objectives
3. Framework for linking data
with action
Action plan for overcoming barriers to
data use and improving information
flow
Three commitments to improve data
use in your job
Workshop Outputs
5. Session 1: Learning Objectives
Knowledge learning objective:
Describe the role of data demand and use
in a functional monitoring and evaluation
(M&E) system
7. Strategic Information
Are we making progress towards or are we
achieving our goals, objectives, and targets?
How does service delivery vary across facilities?
What are we doing well and how are we
under-performing?
Can we improve service coverage with
existing resources?
How should we allocate resources?
Do we need to advocate for additional resources or
for changes to policies
and programmes?
8. Review of Basic M&E Concepts
Goals
Objectives need to be smart
• Specific
• Measureable
• Achievable
• Realistic
• Time bound
11. Results Framework
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
Outputs are the products, like the number of people enrolled on
antiretroviral therapy (ART) during the past 12 months.
Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load and service
coverage.
Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a valid and
reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.
12. Indicator
Measures an aspect of a program’s
performance
Measures changes over a period of time
• # of new family planning users
• # of clients currently on ART
Expressed as a number or percentage
13. Baselines
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
Outputs are the products, like the number of people enrolled
on ART during the past 12 months.
Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current condition
against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.
14. What are the sources of data?
What are the data collection
methods?
Who collects the data?
How often are data collected?
Who analyses these data?
Who reports these data?
Who uses these data?
Establishing a Baseline
15. Targets
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
Outputs are the products, like the number of people enrolled on
ART during the past 12 months.
Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.
16. Calculating Targets
Things to consider
Existing capacity of staff and facilities
Staff turn-over
Budget allocations through the
Ministry of Health (MOH)
Other funding streams like bilateral
and multilateral donors
21. Data use
Create or revise
a program or
strategic plan
Develop or revise
a policy
Advocate for a policy
or program
Allocate resources
Monitor a program
Data Demand and Use Defined
Data demand
Decision-makers
specify what kind of
information they
need to inform the
decision-making
process and then
seek it out
28. “We are always giving patient forms and
data to our M&E Unit, who then gives
data to donors and the government. I am
the head doctor and I never have the
chance to look through the data before
they go up. We just keep giving data up
and up, and we never hear back about
it…”
Head of ART facility, Nigeria
29. Census
Vital events data
Surveillance data
Household surveys
Facility-level service statistics
Financial and management information
Modeling, estimates, and projections
Health research
Data and Information
30. Paves path between data collectors
and users
Leads to greater appreciation
of data
Important element of management
and supervision
Working toward a Culture of Data Use
32. View activities from different perspectives
Have different degrees of understanding
Need/want different information
Need information at different levels
of complexity
Have different intensities of interest
Have different roles in the decision-making
process
Knowing Your Stakeholders
Why is it important?
33. Learning Activity 4
Stakeholder analysis matrix
Stakeholder
Description
Role
Knowledgelevel
Commitment
Resources
Constraints
Government Sector
Political Sector
Commercial Sector
Source: MEASURE Evaluation, 2011
34. Involving Stakeholders in the
Data Use Process
Relevance of data
Ownership of data
Appropriate dissemination
of data
Use of data
39. Knowledge learning objectives:
Define data demand and use
Identify barriers to data use and develop
recommendations to overcome them
Identify locally relevant programmatic
questions and link those questions to
available data
Session 4 Learning Objective
41. 1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
7 Steps to Use Information
42. Should we expand care and treatment
programs in Njombe province?
Am I meeting the HIV prevention needs of
my key populations?
Are men and women seeking HIV
care equally?
What is the quality of my HIV prevention
service?
Step 1: Identify Questions
Examples
43. Should we expand care and treatment
programs in Njombe province?
What is the burden of HIV in the province over
time?
How many facilities are offering counseling
and testing (C&T) in the region?
How many people are being seen
at each facility?
Are there key facilities that have capacity to
expand C&T to reach gaps?
Step 2: Prioritize Questions
44. Demographic and health surveys (DHS),
AIDS indicator surveys (AIS), etc.
Routine health information system (RHIS)
External data sources: behavioral surveillance systems,
mapping exercises, surveys
Projections
Step 3: Identify Data Sources
45. Step 4: Transform Data into Information
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page 9
46. Step 5: Interpret Information
and Draw Conclusions
Relevance
of finding
Reasons
for finding
Consider
other data
Conduct
further
research
47. Gather additional information
Prioritize Njombe in national expansion strategy
Expand voluntary counseling and testing (VCT) and
C&T in high priority facilities
Current VCT facilities do outreach to hotspots
Create referral protocol to ensure those tested are
entering into C&T
Step 6: Craft Solutions and Take Action
48. Monitor implementation of action plan
Consider frequency and duration of monitoring
Develop tool for monitoring
Step 7: Continue to Monitor Key Indicators
51. Knowledge learning objective
Identify locally relevant programmatic
questions and link those questions to
available data
Skills learning objective
Use tools and strategies to facilitate data
use during decision making
Session 5: Learning Objectives
52. 1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
7 Steps to Use Information
53. Questions
Are we implementing the
program as planned?
Are we fully meeting the
needs of our clients?
What additional resources are
needed to meet
the needs of our clients?
Are our interventions making
a difference?
Questions Lead to Decisions
Decisions
Allocation of resources across
districts/community-based
organizations (CBOs)
Revising program approaches
to emphasize results
Addition of outreach
education programs to most-
at-risk populations
Advocacy for new program
areas
54. Brainstorming about what different staff
are interested in knowing
Participatory discussion of indicators/data
— desire to know more
Gathering feedback from clients
Preparing for upcoming decisions
that have to be made or
planning exercises
External factors — donors questions
How Do I Identify Questions of Interest?
55. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and
assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
58. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and
assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
59. Creates a time-bound plan for
data-informed decision making
Encourages greater use of existing
information/generated demand
for data
Monitors the use of information
in decision making
Framework for Linking Data with Action
60. Learning Activity 7:
Framework for linking data with action
Question Data
source/
indicator
Person
responsible
(analysis/
research)
Timeline Communication
channel
Possible
decision/
action
63. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and assess
findings
6. Craft solutions and take action
7. Continue to monitor key indicators
65. Descriptive Analysis
Turns raw data into useful information
Provides answers to questions
Describes the sample/target population
(demographic and clinic characteristics)
Does not define causality – tells you what, not
why
66. Question: Is my program meeting its
objectives?
Analysis: Compare program targets and
actual program performance
to learn how far you are from target
Interpretation: Why you have, or have
not, achieved the target and what
this means for your program
Descriptive Analysis Answers
Programmatic questions
67. Service delivery descriptive analysis
Service coverage analysis
Unit cost analysis
Trend analysis
Types of Descriptive Analyses
Using Routine Data
68. Service Delivery Descriptive Analysis
Selection of ARV Indicators
Private
sector
Public
health
system
Civil
society
# of clients who have refilled their prescriptions on-time at a
fee-for-service pharmacy
√
# of clients on ARVs who have been admitted into a
care facility for end-of-life support in the past six months √ √ √
# of clients on ARVs who have been counselled on ARV
adherence in the past three months
√ √ √
# of clients who have had CD4 and VL tested in the past six
months
√ √ √
# of clients enrolled in ARV programme √ √
# of client eligible for ARVs √ √ √
# of clients who have refilled their prescriptions on time at a
subsidized pharmacy
√ √
# of CD4 test kits procured through bilateral development
partners and distributed by NGOs
and INGOs to public-sector ARV clinics
√ √
69. Service Coverage Analysis
Supply of services data
Need for HIV services
Calculate:
Service coverage =
Supply of services
Demand for services
70. Service Coverage Analysis
Service
coverage
=
# of individuals who have
received the services
# of individuals in need
of the service
Service coverage =
467 individuals have received
HIV testing and counselling (HTC) services
1,000 individuals in need of HTC services
= 47%
71. Unit Cost Analysis
What is the unit cost of delivering antenatal care (ANC) services
through an existing mobile health unit that provides maternal and
child health (MCH) services?
Item Amount
Fixed costs per month
Overhead SLL 7.8M
Personnel SLL 30.5M
Variable costs per month
Commodities SLL 7.8M
Other SLL 15M
# MCH clients/month 100
# MCH clients/month receiving ANC services 23
Cost data adapted from 10.1371/journal.pone.0119236
72. Observe the overall pattern of change
Reveal patterns due to seasonal variations
Identify outliers and high/low performance in
certain time periods
Compare the effect before or after
an event
Produce projections to help with programme
planning and target setting
Descriptive Statistics
Trend analysis
73. Trend Analysis
Comparing time periods and sites
Source: 2016 Sierra Leone Malaria Indicator Survey
Full Report, page 51
74. Display trends, relationships, and comparisons
Simple and self-explanatory
Charts and Graphs
80. Session 6 Review: Interpreting Data
Relevance of
finding
Reasons for
finding
Consider
other data
Conduct
research
Have we met our target?
How does performance compare
to other time periods? Facilities?
What could explain the observation?
Talk to beneficiaries and community
leaders
Ask program managers
Analyze other data sources
• Do they confirm or reject
your finding?
o Why?
If there are no data, need for research
Methodology depends on question
and available resources
83. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
84. Knowledge learning objective
Identify barriers to data use
and develop recommendations
to overcome them
Skills learning objective
Use tools and strategies to facilitate
data use during decision making
Learning Objectives
85. What Determines DDU?
* Based on PRISM analytical framework (LaFond, Fields et al. (2005). The PRISM: An analytical framework for
understanding performance of health information systems in developing countries. MEASURE Evaluation).
Organizational Technical
Behavioral
87. Technical
Have you ever had an experience while making a policy or
program-related decision when you were concerned about the
quality of the information being used?
Behavioral
What specific challenges have you experienced among your
staff when it comes to using data?
Organizational
How does your organization support having the necessary
information to make decisions?
Learning Activity 10:
Identifying barriers to data use: discussion questions
88. Learning Activity 11:
Developing an action plan to address
barriers to data use
Barrier Action Steps
Responsible
person
Timeline Indicators
1. 1a.
1b.
1c
1a.
1b.
1c.
1a.
1b.
1c.
1a.
1b.
1c.
2. 2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.
89. Plan, plan, plan!!!
Engage key stakeholders to fully
understand:
• Decisions they make
• Information they need
• Appropriate dissemination
Building Data Use into Your Work
90. Institute regular meetings
Document and
communicate data
use successes
Practical Steps to Integrate DDU
Institutionalize tools (ex.,
framework for linking
data with action)
91. Identify a group leader
Identify 2–3 things your group can do
to facilitate data use within the first two
months of returning back to work, based
on your action plan
Small group discussion: 20 mins
Group presentations: 15 mins
Make Commitments
92. This presentation was produced with the support of the United
States Agency for International Development (USAID) under the
terms of MEASURE Evaluation cooperative agreement AID-OAA-L-
14-00004. MEASURE Evaluation is implemented by the Carolina
Population Center, University of North Carolina at Chapel Hill in
partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. Views
expressed are not necessarily those of USAID or the United States
government.
www.measureevaluation.org