Using Data to Support the Most Vulnerable: An OVC Information Needs Framework
•
4 likes•1,827 views
The "Using Data to Support the Most Vulnerable: An OVC Information Needs Framework" webinar, organized by the HIV/AIDS Monitoring and Evaluation Network (AIMEnet), presented the OVC Information Needs Framework. MEASURE Evaluation's Molly Cannon and Lisa Parker led the one-hour webinar.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Using Data to Support the Most Vulnerable: An OVC Information Needs Framework
Similar to Using Data to Support the Most Vulnerable: An OVC Information Needs Framework (20)
More from MEASURE Evaluation
More from MEASURE Evaluation (20)
Recently uploaded
Recently uploaded (20)
Using Data to Support the Most Vulnerable: An OVC Information Needs Framework
- 1. Using Data to Support the Most Vulnerable: An OVC Information Needs Framework Molly Cannon, Lisa Parker AIMEnet Webinar September 10, 2015
- 2. Objectives Learners will be able to: • List different types of information needs of community based OVC programs • Discuss the importance of data collection tools being driven by a specific information need • Give examples of different tools available to assess different types of OVC program information needs
- 3. PEPFAR OVC Guidance (2012) “Programming for children orphaned and made vulnerable by HIV/AIDS contributes to the achievement of an AIDS-free generation by responding to the social (including economic) and emotional consequences of the disease on children, their families, and communities that support them.”
- 4. 4 Volunteer Home Visitors CBO Sectors that Address Child Vulnerability
- 5. M&E Challenges 1) Disconnect 2) Collecting information on every child at every encounter • Ethics • Burden • Data quality 3) Massive amounts of data – issues related to data management, backlog of information, databases
- 6. Why an Information Needs Framework? • There are many different types of information needs that stakeholders have • Information needs/questions require different tools and methods or “tools fit for purpose” • With new reporting requirements and guidance, the framework can help us see how the pieces fit together
- 7. What is the Framework? • 10 different information needs related to OVC programs Illustrative questions Recommendations on methods Who collects the information How frequently • Color coded to illustrate information needs of different users
- 15. Uganda – Vulnerability Index • Existing national tool developed for multiple purposes • Aimed to assess vulnerability by core program areas • Based on aggregated scores, determined level of vulnerability
- 16. Uganda – Vulnerability Index Assessment in 2014 revealed challenges: • Purpose of tool is unclear • Tool does not appear to capture the most vulnerable • Data quality issues • Cost/sustainability/ logistical concerns for repeat administration • Program may not be able to address all needs identified
- 17. Uganda Vulnerability Index • Revised the tool • Using the framework, clarified information gap = Targeting/Prioritization • Prioritized vulnerability indicators for enrolment • Created guidance for users on how to use information for stated purpose • Currently pre-testing
- 18. Child Status Index (CSI) The CSI was designed as a simple, cost-effective, comprehensive tool to be used by low-literate (and often volunteer) community home visitors to capture a child’s status and well-being across 6 domains. *O’Donnell K, Nyangara F, Murphy R, Nyberg B. Child Status Index. A Tool for Assessing the Well-Being of Orphans and Vulnerable Children—Manual. [MS-08-31a]. Chapel Hill, NC: MEASURE Evaluation; 2009.
- 19. Clarification Regarding Usage of the CSI Information Gathering Purpose Recommended for Use Targeting No (NOT RECOMMENDED) Case Management Yes (PRIMARY USE) Program Monitoring Yes (APPROPRIATE USE) Evaluation No (NOT RECOMMENDED)
- 20. Targeting (NOT RECOMMENDED) 1. Difficult to get accurate CSI scores first time used 2. General criteria for inclusion may be easier than undertaking a needs assessment for each child 3. Not appropriate to use aggregate scores across CSI factors
- 21. Case Management (PRIMARY USE) 1. Individualized method for assessing a child’s well-being to guide decision making 2. Home visitors can work with caregivers to identify resources for addressing concerns 3. Allows home visitors to follow up on the status of children and determine if further action needed.
- 22. Monitoring (APPROPRIATE USE) CSI provides simple monitoring information regarding: • Who is being served • The kinds of services provided • Individual contact history • Change over time
- 23. Evaluation (NOT RECOMMENDED) CSI requires users to identify children’s needs and status relative to their local community It cannot be used as an indicator or comparator for national or multi- country standards
- 24. Q&A
- 25. Contact Information Molly Cannon: Molly.Cannon@thepalladiumgroup.com Lisa Parker: Lisa.Parker@thepalladiumgroup.com Ashley Garley: Ashley.Garley@icfi.com
- 26. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) under terms of Cooperative Agreement AID-OAA-L-14-00004 and implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International, John Snow, Inc., Management Sciences for Health, Palladium Group, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. www.measureevaluation.org
Editor's Notes
- Ashley will cover
- Good morning. I will start off with a bit of background on PEPFAR OVC programs. PEPFAR seeks to serve children orphaned and made vulnerable by HIV/AIDS in order to contribute to the achievement of an AIDS-free generation by responding to the social (including economic) and emotional consequences of the disease on children, their families, and communities that support them. 10% of PEPFAR funds goes toward OVC programming (PEPFAR 2012) and 90% of all OVC are in sub-Saharan Africa (UNAIDS 2010). The number of OVC continues to grow. More recently with the third phase of PEPFAR, PEPFAR 3.0, there has been more of a focus on OVC programs making linkages to HIV testing, care, and treatment and focusing more on early childhood development and adolescent girls. In addition, with the shift to PEPFAR 3.0, new reporting requirements have been introduced to OVC programs such as the Monitoring, Evaluation, and Reporting Indicators or MER indicators and the Site Improvement through Monitoring System or (SIMS).
- PEPFAR funded OVC programs work to enhance the capacity of caregivers at the family level, community-based organizations and local government structures, and national governments to respond to the needs of OVC. The graphic represents the sectors that address child vulnerability. Most OVC programs work to address several areas such as health and nutrition, child, legal and social protection, psychosocial needs, education, household economic strengthening. All of these technical areas should be implemented taking a capacity building and systems strengthening approach. Partners implementing OVC programs work to support government partners – the lead coordinating agency varies by country, but often it is led by Ministries or Departments of Gender, Social Welfare. However, due to the multi-sector nature of OVC programs, other line ministries (e.g., education, health, justice, agriculture) are also important stakeholders. In most programs, Community Based Organizations recruit volunteer home visitors to make visits to the households to provide case management services, including referring the caregiver and vulnerable children to other services located in the community. These home visitors are the primary data collectors and report the data they collect to the CBO. The CBO then reports specific data to the implementing partner which transmits that information to the USG mission for PEPFAR reporting.
- Within these OVC programs the primary model is that volunteer home visitors collect information using multiple non-standardized forms, and the focus tends to be on reporting of the data. When you have a data collection system that supports reporting more than supporting decision making at the local level it leads to a few things: First, it leads to a disconnect between information needs of the project and those who are required to collect the information Second, it leads to the collection of repetitive information on the same child and household When this occurs, it presents three potential problems: an ethical issue of continuing to ask families and children about their situation, particularly when there is no direct outcome or benefit to the household It may create a burden on caregivers who get frustrated by answering the same questions and on the volunteers who after all, are volunteers Finally, this can have an effect on data quality – since the people collecting the information are often not using it directly, they may not pay as much attention to completing forms accurately, on time, etc. Third, copious amounts of data are collected that are not used and this creates a backlog of information that makes data management systems difficult to sustain and in some cases databases end up defunct as the amounts of data collected require human capacity for data entry, cleaning, analysis, maintenance, that may not exist.
- Read slide Different types of information needs require different tools/methods
- Revise notes The framework started with a paper that Jen Chapman and I had published in the Journal Vulnerable Children and Youth Service. However, as a team, we have expanded the framework to represent and even broader set of information needs. Let’s take a look at the tables on the wall – briefly the framework has a few components that I’ll go over in detail shortly – there are…
- Explain the color coding here so people can see
- Situation Analysis Use: All Stakeholders to plan programs and allocate resources Questions: Approximately how many children need services/support in Area xx Methods: Survey, secondary analysis, qualitative Who collects: External organization Frequency: Non-routine Targeting Use: Programs to help identify children and HH needing assistance. Question: Which children/households are most in need of program services? Methods: Quantitative tool Who collects: Internal to the program Frequency: At registration
- Case Management Use: Home visitors prioritize and attend to the needs of a child, household Question: What are the child’s immediate priority needs/have they been abated? Methods: Job aid with care plan Who collects: Home visitors Frequency: Routine Program Monitoring Use: To ensure programs are implemented as planned Question: How many children, HH are receiving services? What types of services? Methods: Monitoring tools Who collects: Home visitors/program staff Frequency: Routinely (monthly)
- Process Evaluation Use: To determine how the program is implemented, valued, and why results are/are not occurring. Question: To what extent is the program on track for achieving its objectives and why? Methods: Mixed Who collects: Program staff/external data collectors Frequency: Non-routine Operations Research Use: To determine the best approaches to achieve results. Question: What is the most cost-effective strategy for improving uptake of services? Methods: Mixed/more rigorous design Who collects: Program staff/external data collectors Frequency: Non-routine
- Outcomes Monitoring Use: To assess changes in a population across a limited number of key indicators Question: Has the status of the population improved or worsened over a given period? Methods: Surveys using LQAS/cluster samples Who collects: External organization Frequency: Every 2 years Impact/Outcome Evaluation Use: To assess changes in program/intervention beneficiaries across key indicators over time Question: Has the status of beneficiaries changed over time?Is the change attributable to the program? Methods: Mixed/more rigorous design with counter-factual (impact) Who collects: External organization Frequency: Non-routine
- SIMS Use: For USG to ensure that programs are performing to PEPFAR quality standards Question: Are program sites performing to PEPFAR quality standards? Methods: SIMS tool at select site Who collects: USG Frequency: Routine EA Use: For USG to determine OVC program expenditures Question: What are the fixed and recurrent costs to PEPFAR for implementing specific OVC interventions? Methods: Expenditure analysis worksheets Who collects: Implementing partners Frequency: Routine
- The CSI was designed as a simple, cost-effective, comprehensive tool to be used by low-literate (and often volunteer) community home visitors to capture a child’s status and well-being across 6 domains: Food and nutrition, Shelter and care, Child protection, Health, Psychosocial well-being, and Education and skills training In 2012 Measure Evaluation conducted two phases of assessments looking at how large programs use the CSI to meet a range of information needs and how home visitors make decisions about care/support and the utility of the CSI.
- One of the major findings from the assessments was that programs were using the CSI for a variety of purposes and in some programs the tool was used to meet a variety of information needs [e.g., targeting, program monitoring, case management, evaluation]. The use of the tool was not consistent across programs and we found that the information generated from CSI scores was analyzed in many different ways by programs to help with decision making. Using the OVC Information Needs Framework we helped organize the findings of these assessments into guidance clarifying the usage of the CSI. The table on the slide indicates the different types of information needs that programs have and whether or not the CSI is recommended for that use. As you can see, we do not recommend using the CSI for targeting or for evaluation. I will briefly walk you through the reasons we do/do not recommend the CSI for these purposes:
- the CSI is of limited support in targeting because (1) evidence from home visitors indicates that it is difficult to get accurate CSI scores the first time the CSI is used (2) programs may find it easier to have general criteria for inclusion in a program rather than undertaking a needs assessment for each child; and (3) it is not appropriate to make targeting decisions using aggregate scores across CSI factors
- the CSI is most useful as a case management tool for serving highly vulnerable children and families. The CSI provides a consistent and individualized method for assessing a child’s status and well-being to guide decision making about the child and household. Once the child has been assessed, home visitors can work with caregivers to identify caregiver and household resources for addressing concerns. As needed, the home visitor can help identify community or program resources to address concerns. Furthermore, with repeated administration, the tool allows home visitors to follow up on the status of children and determine if further action is needed.
- Similar to its value for case management, the CSI can offer important information for program monitoring at the local level. The CSI “Child Status Record” sheet provides simple monitoring information regarding who is being served, the kinds of services provided, and individual contact history and change over time. Please note, we do not recommend aggregating scores beyond the local level for this purpose.
- Since the CSI requires users to identify children’s needs and status relative to their local community, it cannot be used as an indicator or comparator for national or multi-country standards. Specifically, the CSI should not be used among a sample of children as an evaluation tool. Broad evaluation of the impact of a regional or national program on child well-being requires several considerations and, likely, multiple approaches. The MEASURE Evaluation OVC Survey tool kit should be used for evaluation purposes.