This poster will be presented by Fonda Ripley at the American Evaluation Association Conference on October 31, 2018. Engaging specific, hard to reach populations in evaluation can be challenging. JSI conducted an evaluation of a teen peer crowd segmented social marketing campaign implemented by the Vermont Tobacco Control Program to assess impact on tobacco use among country teens in Vermont.

1. Addressed evaluation goals and challenges:
 Rescue Agency’s I-Base Survey™ for peer
crowd measurement (5 peer crowds:
Mainstream, Popular, Alternative, Hip-Hop,
Country)
 Online survey aiming for 800 VT teens total,
400 country
 Online digital recruitment and survey
less expensive and also effective for
older teens
 Country/non country split – 2 group
comparison had enough statistical
power and smaller sample size than 5
peer group comparison
 Gift card incentive to promote interest
and reduce time in field
Approach Su r vey
 20 questions, ~10 minutes
 Survey Gizmo, offered on
mobile and desktop
 Administered for 10 weeks
Lesso n s Lear n ed
Sp eak in g th eir Lan gu age. Sam p lin g an d Recr u itm en t Str ategies an d
C o n sid er atio n s fr o m a Yo u th T o b acco P r even tio n C am p aign E valu atio n
Fond a Ripley, MHS, Jessica Holli, MPH, Eugenie Coakley, MA, MPH – John Snow, Inc. (JSI)
Project Overview
Engaging specific, hard to reach
populations in evaluation can be
challenging. JSI conducted an evaluation of
a teen peer crowd segmented social
marketing campaign implemented by the
Vermont Tobacco Control Program to
assess impact on tobacco use among
country teens in Vermont.
Resu lts
Recr u itm en t Str ategies
Analysis (N=824) :
264 Country (32%)
560 Non-Country Youth (68%)
687 (80%)
Facebook &
Instagram
124 (15%)
Refer to
Friends Link
46 (5%)
Survey
Ambassador
Recruited
857 Total Survey Responses
Tobacco Use
• 19.5% Country
Sample
• 20.2% non-
Country
Sample
D&D Brand
Associated
with
Tobacco-
Free Living
• 73% Overall
Sample
• 73% Country
Sample
D&D Brand
Appeal
• Favorable
View: 38%
Overall , 41%
Country
• Neutral View:
57% Overall,
53% Country
Down &
Dirty
Campaign
(D&D Brand)
Awareness
• 50% Overall
Sample
• 51% Country
Sample
Acknowledgements
• Vermont Tobacco Control Program (VTCP)
• Rescue Agency– I-Base™
Collect solid data on peer crowds and tobacco use prior to the campaign
• Potential reach and prevalence
Plan 9-12 months for evaluation
• Participatory evaluation planning
• Coordinate with community partners & schools
• Develop, test, field survey
• Analyze and interpret findings
Incentives important for promotion, but monitor responses for repeat takers,
and fake takers
Coordinate with community partners to engage youth in the survey through
non-digital means
• Survey ambassadors, events, print ads
Assess dosage of the campaign
Treat recruitment like a campaign
Digital recruitment is efficient, but difficult to assess representativeness
• Hard to reach young teens
Country
Teens
32%
Non-Country
Teens
68%
Evaluation Objectives
Obtain a representative sample of Vermont teens
ages 13—19 (with enough statistical power) to:
Compare tobacco use and campaign awareness
among country teens and non-country teens
Determine whether the campaign influenced
tobacco use rates among country teens
Campaign Overview
• Down and Dirty segmented youth social
marketing campaign by Rescue Agency
• Based on formative research indicating high
tobacco use among Vermont teen country peer
crowd
• Vermont country peer crowd, 13-19 years of age
= intended audience
• Aimed to disassociate tobacco use from county
teen peer crowd
• Used branded events, social media, video
advertisements, brand ambassadors
• Implemented 2013 – 2017
Evaluation
Challenges
• Size of campaign intended audience– country teen peer crowd – unknown
• No baseline estimate of tobacco use among country teens for comparison
• How to measure peer crowd affiliation among Vermont teens?
• How to reach and engage country teens in the evaluation?
• Time constraints for evaluation
• Resource constraints—budget, schools off limits
Digital Promotion
• Facebook & Instagram
Ads
• Target audience
• Chatbot
• Micro Brand
• Refer to Friends
Print & In-Person
Promotion
• Flyers, palm cards
• Country venues,
events
• Country teen survey
ambassadors
$10 Gift Card
Recr u itm en t Resp o n se