Key Findings and Programmatic Implications: OVC Program Evaluations in Kenya and Tanzania
•
0 likes•791 views
![Overview ,[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Key Findings and Programmatic Implications: OVC Program Evaluations in Kenya and Tanzania
Similar to Key Findings and Programmatic Implications: OVC Program Evaluations in Kenya and Tanzania (20)
More from MEASURE Evaluation
More from MEASURE Evaluation (20)
Recently uploaded
Recently uploaded (20)
Key Findings and Programmatic Implications: OVC Program Evaluations in Kenya and Tanzania
- 1. Key Findings & Programmatic Implications Tonya Renee Thurman, MPH, PhD Tulane University School of Public Health & Tropical Medicine/ MEASURE Evaluation Dissemination Meeting, September 3 rd , 2009 Washington, DC OVC Program Evaluations in Kenya & Tanzania Tulane University School of Public Health and Tropical Medicine
- 4. Prevalence of Home Visiting
- 13. Children ’ s Psychosocial Well-being Kids Clubs Allamano Effect Effect Effect No Effect No Effect Outcomes Emotional symptoms Pro-social behavior Adult support Global self-esteem Social isolation TSA No Effect No Effect No Effect No Effect No Effect
- 16. Child Care and Treatment & Guardian Psychosocial Well-being Feelings re child Family Functioning Child abuse Negative Feelings Positive Feelings Marginalization Home Visiting (3 programs) No Effect Effect ( CRS ) No Effect No Effect No Effect No Effect CRS Support Group No Effect Effect Effect No Effect No Effect Effect IAP Support Group No Effect No Effect No Effect Effect No Effect No Effect
- 17. Support Groups & Home Visiting? One study demonstrated the added benefit of exposure to multiple interventions (CRS)
- 28. Food Insecurity Two programs aimed to enhance food security
- 29. Household Food Insecurity Interventions: Food Parcels & IGA Inconsistent Food Parcels (CRS) Consistent Food Parcels (Allamano) IGA: Savings and Internal Lending Committees (CRS) IGA: Bio-intensive Gardening Training with Supply Provision (Allamano) Reduced food insecurity? No effect Effect Effect Effect
- 33. Jealousy of service provision For PACT, perceived jealousy was significantly higher in the Intervention Group
- 34. Who is responsible for orphans?
- 35. Who is responsible for orphans? Significant difference (p < .05)
Editor's Notes
- First, I am going to touch on briefly the issues related to program coverage. While not necessarily a key findings in terms of intervention effectiveness, it is an important finding nonetheless which bears consideration as we scale-up programs Then, the results. I am going to discuss key findings as they relate to various outcomes and intervention strategies. Unfortuantely, I do not have time to discuss all the details of every finding, instead, I am hoping to provide a big pictire view of the potential influence of these strategies on the wellbeing of children and guardians Touch on a few unexpected and unwelcomed findings we uncovered Lastly, summary of these findings as well as a discussion of programmatic implications
- Furthermore, among the key interventions examined in this study, only a small number of beneficiaries confirmed that they received expected services. For example, while the TSA participants were randomly selected from the program’s kids’ club registry obtained from TSA staff, nearly 57% of the children selected to participate in the survey reported they had never heard of the kids’ club program.3 Similarly, only 44% of Allamano participants reported ever attending
- only 49%, 29%, and 57% of the intervention group of CRS, TSA, and Allamano, respectively, reported having a home visitor. These findings were unexpected as programmers assumed that most, if not all, would report receiving this core service.
- It is possible that some participants may have chosen not to participate in available services or, unknown to programmers, were not receiving expected results. these results give pause to the authenticity of the comprehensive coverage provided by OVC programs.
- Range of outcomes at child, guardian and family level Most at Child level-- Child HIV knowledge, psych, health, legal protection, education, care they receive from guardian (also a guardian/ hh outcome– family relationships) Guardian– psychosocial and health, HH- food security, community support Exploring the effectiveness of 8 interventions, all of which were employed across two more programs For example, impact of kids clubs at two sites, home visiting at 3, etc – all on the same outcomes. We will discuss at the end potential explanations for divergent results.
- Now I will summarize some of the key hypotheses or specific research questions we examined. Based on expected outcomes derived from case studies of the programs. As a reminder, focus on key interventions, more comprehensive, but could not be evaluated….as a result of program exposure and/or even study design (IAP) School based HIV education programs– pretty straight fw Kids Club– that and psychosocial– recreational activities and structured activities, they report it as a psychosocial service Kids Clubs: Recreational and Social opportunities Psychosocial & Life skills activities Home visits – our staple program; Almost all program’s conduct Home Visiting and its seen as a comprehensive package in its self----most services reported happen during home visits- -all child and guardian level outcomes, except food security & community support Support groups – one for HIV positive and the other for guardians more generally
- Two goods– education, food Goods – pretty straight forward. However, some not evaluated – i.e., paid school fees, enrollled; who do you compare it too? Health services too was also an issue – may not have needed it, but we do Note, health is spoken about in the context of home visiting in terms of use of referrals Lastly, Community sensitization re child protection generally and OVC care needs, encourage support– two programs., community sensitization activities concerning OVC care and support, we explore their potential effects on OVC and the broader community IGA strategies– hh food security Two types of IGA- skills training and community savings and loans Most are by intervention and expected domain, but we do look for some cross cutting effects, such as educational support and pyschosocial. We also, where possible, have considered potential additive effects of exposure to multiple interventions Now, let me make a caveat, while I speak of effects, what we found was an association between program exposure and outcomes. Some would argue that effects cannot be derived from this design, thus, while I use that word, I acknowledge the limitations. Further, anything I state as ‘effective’ was .05 in multivariate analyses Ok, so for results….rather than take it intervention by intervention, I thought it would be most interesting to take it domain by domain in terms of outcomes–for example, look at psychosocial wellbeing of child and examine the effectiveness of each of the potential strategies on these outcomes Lets start with hiv education
- Ever heard of AIDS, Number of prevention methods, two indicators of mistaken beliefs Kids clubs
- School based programs, interesting as one sample includes OVC to non-OVC and the other among OVC only Regardless of OVC status – neither more or less knowledge, one included community children in addition to OVC, Few knew sexually-related protective behaviors, such as limiting the number of sex partners or using condoms. Abstience, razors and needles. Only 4% mentioned condom Mistaken beliefs – 43% felt a healthy looking person could NOT have AIDS
- Twice as likely to have ever haard of aids 77 vs 62% had heard of it But, only 1/3 knew at least one prevntion method…..and no difference
- All scales used previously in SSA– first four, widely used, latter two applied by the researchers in previous studies INDICATE which are guardian reported…. All with continuous score, not cut-offs, lower/higher Note, alpha levels of .65 required….some scales did not pass the lenient test in all sites….which is why you may not see each outcome always assessed everywhere. We discuss this in our summary report and provide alpha levels across the sites. Ome measures worked well and may be something for you to consider as well… Three key interventions– kids club – direct impact and home visiting direct impact but also, as with support groups, possible indirect impact due to their improvement of guardian caretaking abilities
- Two programs implemented kids ’ clubs in Tanzania aiming to enhance children ’ s psychosocial well-being (Allamano and The Salvation Army) with Kids Clubs Programs offered: Recreational and Social opportunities Psychosocial & Life skills activities Interesting, while the other was effective on HIV and this one was not, but that one was effective here Why was only one effective? Asked kids what they did and learned in Kids Clubs…. -- Reasons why one program had impact (Allamano) and other program did not (TSA) Variety of topics covered in kids club curriculum More Allamano kids club participants learned about health, hygiene, chores at home, good behavior in comparison to TSA kids club participants Allamano’s curriculum is more comprehensive and covers more topics relevant to specific age groups Case Study revealed Kids’ clubs implementation by staff vs. volunteers TSA kids’ clubs implemented by volunteers only (lack of structure and uniform curriculum at kids club sites - volunteers encouraged to form own agendas) Allamano kids’ clubs staffed by OVC focal staff person at each kids club meeting, regular on-site supervision Age-appropriate activities Allamano - activities for different age groups (more standardized curriculum) Ages 6 - 12 taught life skills (cooking, washing clothes, home care) Children over 12 years taught HIV/AIDS education, safe sex, health and Interesting,
- Our friend Home visiting evaluated across 3 programs Allamano, CRS & TSA PSS Outcomes: Only Allamano’s home visits associated with higher levels of child-reported adult support, lower isolation among children Only TSA’s home visits associated with higher levels child self-esteem CRS’ home visits fewer behavioral problems among beneficiaries CRS and Allamano - Lower isolation among children Overall, no more than two per program and one only one…. Anyone explain this-- Different training curriculums & few psychosocial skills Only one outcome– social isolation– in two programs; encourage children to play; a special visitor for them SG– only 1, CRS Support groups– not targeting children, but trivkling down to help them promote better behaviors among their children. Also, these were guardians reported measures, so could also be affecting their own perceptions and acceptance of child behaviors
- Now, I will discuss two domains….both of which expected to be affected by home visiting and support groups and are also likely interrelated Kind double dipping here, saves a couple slides, and also may be related While not studied here, arguably, the psychosocial wellbeing of the guardian likely influences their care of the child? All but abuse is Guardian reported Scale description Census from US Mcmaster – poor/good Child reported abuse– UNICEF Marg– author created used in Rwanda Neg & Pos from WHO QoL
- Home visiting evaluated across 3 programs Allamano, CRS & TSA Home Visiting-- Not a lot happening here….some evidence of improved family relationships– data on FF (next slide) Allamano - lower feelings of marginalization for guardians after home visits than those not visited?? One support group affected neg feelings
- Noted that two interventions by the same program– sinec both are effective, those who exposed to both had even more impact --there is an additive effect-- Combination of interventions had the greatest influence Though, notable that beh difficulties was very low– possible range of up to 40
- Key interventions – minimal impact on neg feelings, (one intervention) across 5; nothing on home visting and 1 of two support groups; nothing on positive feelings Important to reflect With scale scores ranging from 1 to 5 and higher scores reflecting a better emotional state, average scores on the two psychological scales were less than 2.8. These scores are much lower than norms found in past research. A study across five countries, including Zimbabwe, reported much better emotional well-being among healthy adults, with mean scores on the positive subscale of 3.55 and those on the negative scale of 3.74.17
- Could asses reported health status between the two groups While some gave Vitamin A, Deworming, etc– could not assess health status between those who got it and those who did not, as the latter may not have needed it Could not assess between those who went to a clinic or not- may not have needed it, but did examine use of health referrals
- Nada a ki when it came to guardains health again….we seeing a pattern? Only one, with child, Challenging to evaluate other aspects with the design– those got vit a, deworming, etc– did the others need it? One program- -more likely to have visited the health center if they had a home visitor, but we don’t know if it was before or after, similar with VCT-- may be why they have a home visitor But we did find an interesting finding in relation to utilization of referrals for free health services… In this case, home visitors had received advanced home-based health care training
- At this site, overall, 21% guardians, 23% children reported having either poor or very poor health.
- Community education– two interventions– explain them??
- 10% difference acrosss eac OVC group Guardians of children living in households with ill parents and paternal orphans had higher possession of identity documents Of note, support groups also did not affect it IAO
- Lacking identity documents– About 65% of Tanz kids lacked identity documents, nearly half in Kenya -- 40, 56, 64 & 67– IAP, CRS, Allamano, TSA
- Two programs where delivery of school supplies was assessed Currently attending? Attendance in the previous week of the survey
- One program– home visiting was associated with school enrollment, case study suggested waivers for school fees negotiatied- 99vs 91% Attendance– encouarhe them of importance, or encouarge their guardians -- Overall though, enrollment was high across sites, No fees, primary age. But do have other ‘school fees’ School supplies– nothing related to education However, better self esteem….more like other kids…..more normal
- FANTA food measure, 4 categories Two types of IGA Two types of distribution– regular and irregular
- Food Insecurity Table Percentages of THREE highest food insecurity (moderate and severe) in THREE study sites Kilifi District, Kenya (CRS) Iringa Region, Tanzania (Allamano) Mbeya Region, Tanzania (TSA)
- Types of Interventions and Effectiveness: (*Details on cost-effectiveness on next slide) Emergency Food Parcels: No effect on food security (CRS and IAP Thika) CRS - No effect on food insecurity for beneficiaries because inconsistent food distribution to beneficiaries Consistent Food Parcels: Allamano - Effects of food support evident (even though more CRS beneficiaries received food support) Food support associated with reduction in food insecurity) because distributed consistently (despite smaller number of beneficiaries) Provision of household food support linked to reduced food insecurity only in instances where it was provided on regular basis CRS : Training and support in the establishment of savings and internal lending committees (SILCs) that offered group-generated funds loaned to members through a monitored savings and credit system, led by CRS support… Allamano: Training in bio-intensive agriculture and participants received capital inputs such as wheelbarrows, spades, and other equipment All Effective Results associated with 10 % reduction in severe food insecurity
- Stigma 3 items--- People reject orphans; people affected by HIV; jealousy Whether they had needed it Whether they had received assistance from friends, family or nieghbors Measured at child & guardian level
- Mtgs held in public forums, markets, churches, somewhat equal level of opportunity for exposure, but who attended vs who did not Examine potential effects at OVC and Community member level and relationship between the two Strict intervention and comparison groups-- OVC guardians found to be receiving more in-kind support from friends and families 94% versus 6% IAP Community members who attended had less negative attitudes, as did their children. Only less than .10 for adults in regression. Marginal effects. However, the importance of addressing stigma was also illustrated in this study. Multilevel analysis across 40 communities showed lower family support and more social isolation among OVC living in high versus low stigma communities
- Caveat– cannot say they caused these effects but important to consider as potential unintended consequences of targeting OVC Neg Feelings (TSA) Taking them away from chores; may resent their absence; not at a convenient time Worse perceived stigma (Allamano) Kids Club Limited to OVC children– other children resent them or feel more isolated due to the exclusive activities Home visitor– may make their HIV or vulnerability status to others Jealousy was high among guardians and children One site, more jealousy in one site than the other
- People are jealous of the services given to orphans and those affected by HIV and AIDS Jealousy– almost half in one site, lower in another, very high in CRS– poorest area of the country, Yet, interestingly, community sample is lower– people don’t want to admit it; perceived jealousy higher than actual? Pact, we also see that pattern, the int grp higher than the comparison
- Correspondingly, these studies provide insight on the expectations of community members concerning who is responsible for the support of OVC. Figure 3 includes responses from both beneficiaries and community-based samples within Kenya. One sample in Beneficiaries in one area, Another community sample, in another area Most commonly, the extended family was considered to be principally responsible for the care of orphans. Though, in comparing beneficiary versus community sample responses, it is notable that a larger number of beneficiaries in both countries perceived it as the responsibility of NGOs to care for orphans relative to the community samples. This pattern is particularly stark in examining data from Kenya, contrasting results from the CRS OVC beneficiary sample and IAP community-based sample. About 40% of CRS beneficiaries felt it was the responsibility of NGOs to care for orphans compared to only 24% of the more community representative sample within the IAP study. Perhaps being a beneficiary contributes to increased reliance upon and expectations of support from NGOs; however, this suggestion is merely speculation in light of the study limitations and the fact that these samples were drawn from different communities. Nonetheless, these results bear consideration in preventing dependence of beneficiaries by setting realistic expectations, building their capacity, and expanding their alternative support networks. Further, the fact that the community was the least frequently mentioned source of support for orphan care across all settings has implications for the programmatic emphasis on community-driven support to OVC. In Kenya, more participants saw it as the responsibility of government than the general community and across all sites, less than 10% felt it was the community’s responsibility. Efforts to engage the community in the care of orphans should consider that community members may not perceive it as their principal responsibility to care for children outside their family. This is not to suggest that community members do not care for OVC, but only that they may not hold a high sense of personal responsibility to care for unrelated children and, as such, encouraging them to do so may require remuneration and other incentives. For PACT-- we
- PACT– significantly higher even after controlling for other variables
- Brings into question the comprehensiveness of programs as well as accuracy of “beneficiary” lists They were beneficiaries, but beneficiaries of what? Not exposed to key interventions. They may choose not to participate, as we saw with health referrals for FREE care. Programmers report school supplies as education, but it affected psychosocial. Similarly, supporting guardians affected children. Home visiting + SG was the most powerful. Support from NGOs could make situation worse. In poor communities, people may resent others who receive special attn, feeling they too are in need. Example from Rwanda, worse marginalization the longer they were served. Never forget Sawzi article– wish my mom was dead so I could go to school
- We have some positive effects of these interventions, but not always, related to curriculum and support. Kids clubs also no influence on self esteem– not in their curricullum. Affected several indicators of children’ s pyschosocial, albeit no more than a couple per program, but we saw virtually nothing re guardian psychosocial. Similaly no health effects for guardians. The fact that program effects vary by program bears into question– what exactly is this intervention doing for children? The answer would be it depends… SG—specificallty focused guardains, on the other hand, saw some pos effects not only for guardians but some for their children.Only affecting the social rather than the ‘psychological’ wellbeing of guardians and not consistently Saw positive effects of food, but how long can we do that for? Also saw + effects of IGA…longer lasting support. Paul will address some of sustainability issues in terms of the costs of interventions.
- Orphan’s higher rate of sexual risk behavior – importance of comprehensice programs Stigma– associated with their wellbeing, but more needs to be done to address it Importance of guardian psychological health was illustrated Nothing on health More attn to succession planning Also identity documents- -important in helping them access social service
- Stystematically Build volunteers’ capacity All know different things, Standardized training as is the case for HBC in Kenya….accredited training in SA Increased psychological skills and legal Ongoing technical guidance Although CHWs have basic counseling skills, they feel limited in their ability to address major psychological issues. Project staff & CBOs feel the project has not trained enough counselors to deal with the psychosocial support needs of OVC and their guardians. Trained counselors are overwhelmed with referrals, while still trying to balance counseling with other work responsibilities. On the other hand, Reasonable expectations of volunteer are necessary, Potential overreliance for specialized interventions Programs relied on volunteers with high case loads Reduced case loads could result in better coverage and impact Volunteer’s may be overloaded and lack accountability
- Barriers in health among other things, included mistreatment by health staff & discomfort with talking with them … .empower them, sensitize health staff Unmet needs we previously discussed, psychological health, physical health, legal protection … . Home visiting is our our pancea Engage them in multiple interventions … one may work better than the other and the two could be synergistic
- Bulk of orphans are adolescents OVC programs generally tend to neglect the particular psychosocial, educational, reproductive health, and livelihood needs of adolescent orphans.32 Programmers also commented that service delivery targets drive the emphasis on younger children Programmers need more information on how best to meet adolescent needs Unmet needs include: Opportunities for advancement : Vocational and career training, financial support for secondary school attendance Age-appropriate interventions : psychosocial support, HIV and AIDS prevention education, and risk behavior reduction initiatives Why one kids club and not the other: A curriculum with wide ranging topics and age appropriate activities, rather than just ad hoc recreational activities Implementation with on-site program staff supervision & support, versus solely ran by volunteers Variety of topics covered in kids club curriculum Kids’ clubs implementation by staff vs. volunteers Age-appropriate activities Allamano - activities for different age groups (more standardized curriculum) Ages 6 - 12 taught life skills (cooking, washing clothes, home care) Children over 12 years taught HIV/AIDS education, safe sex, health and nutrition info TSA - Based on skills, ideas and equipment from volunteers (limited) Some interventions should be open to all where possible, recreational activities, saw kids peering over in SA,,,,,,integration and averts potential resentment Intreventiosn addressing guardian needs– ask them about the timing of the intervention, they coud also participate and feel empowered and learn new skills
- As we have seen, guardian and family focused interventions can = positive difference for children. Supports the principle of building guardian capacity … including beyondn thie SES profile and provding resources for their “ kids ”– support them Simialrly, need more attn to guardian needs in home visiting programs. Perhaps this new standardized curricullum will ensure some chapters? Beyond training, supplemental resources and support may be needed to ensure effectiveness of IGA, such as: food daycare for the student ’ s child/siblings (if applicable) cost of transportation to the vocational training center financial assistance for the final exam required for official certification and to financial assistance to procure production materials and machinery
- Enhance community understanding of issues facing OVC and understanding the issues they face Stigma related also to jealousy Community engagement, sensitization
- Regular assessments of program coverage and client participation is needed Programmers need to uncover barriers to participation Routine follow-up with individual clients Update client list by type of service Health referral example-- It may be that some services, such as kids’ clubs or support groups, were initially offered to clients, and programmers assumed participation while clients chose not to engage in these services after one or more visits or not at all. For services that require active participation of clients (e.g., attending meetings), there may be client-specific or even intervention-specific barriers affecting their attendance that programs could address. For instance, child care needs, transportation, and psychological or health issues may limit their participation. There could also be issues with demand, in that they do not feel the service is relevant to their needs. Even services that require more passive participation on the part of beneficiaries, such as home visiting, may not occur for several possible reasons. Perhaps volunteers do not actually complete their agreed-upon responsibility of visiting each household assigned to them or there is a disagreeable match between the household and the volunteer. The reasons for lack of participation in available services are unknown and may vary by client. Programmers should find ways to follow up with individual beneficiaries routinely, to determine how many clients are truly receiving services, and uncover potential barriers to service participation. For instance, program staff from a mentor program in Rwanda conducted quarterly visits to each household to monitor the relationship between volunteer home visitors and children. Such follow-up could also be conducted in consultation with volunteers not directly involved in delivering such services, such as community committee members. To ensure quality care and obtain accurate indications of the use and availability of resources, it is important that programmers monitor client participation and update their client list continuously by type of service. Ongoing monitoring systems to register and track beneficiaries are needed Many NGOs serving a large number of beneficiaries struggle to maintain an accurate beneficiary list and track service delivery We had ssues with accuracy of the lists Access to and use of identity documents Include birth date Engage community committees in tracking National identification guidelines The inaccuracies within the beneficiary lists across these study sites highlight the need for a strengthened method for registering and tracking OVC and their families. The fact that so many children could not be located has importance well beyond the scope of this evaluation. It leaves questionable the validity of reported services and the total number of OVC benefiting from programs. In addition, some children who may have enrolled in the program at one time and could then not be located may be even more vulnerable and in need of assistance. Follow-up of OVC by community members may be enhanced by program efforts to explain to communities the potentially increased vulnerability of “lost” OVC and establishment of mechanisms for reporting such instances.
- Not just another interesting study…but guidance in the implementation of programs to better meet the needs of OVC…. Leave application of these lessons to you…. Florence will talk next about how they have been used in one of the study sites