Native advertising is a specific form of online advertising where ads replicate the look and feel of their serving platform. In such context, providing a good user experience with the served ads is crucial to ensure long-term user engagement. This talk present an overview of work aimed at understanding the user preclick experience of ads and building a learning framework to identify ads with low preclick quality. Work in collaboration with Ke (Adam) Zhou, Miriam Redi and Andy Haines. An version of this work was presented at WWW Montreal, April 2016.

1. Mobile advertising:
The pre-click experience
Mounia Lalmas
Director of Research, Advertising Sciences
Work in collaboration with Ke (Adam) Zhou, Miriam Redi and Andy Haines

2. UK Internet users
comScore 2015

3. Facebook Suggested Post Twitter Promoted Tweet Yahoo Sponsored Content
Native advertising on mobile

4. Why native advertising?
Visually Engaging
Audience Attention
Higher Brand Lift
Social Share

5. Bad ads disengage users
D. G. Goldstein, R. P. McAfee, and S. Suri. The cost of annoying ads. WWW 2013.
A. Goldfarb and C. Tucker. Online display advertising: Targeting and obtrusiveness.
Marketing Science 2011.

6. User interaction with ads
The user spends time on the ad site (post-click)
The user converts
The user clicks on the ad (click)
The user hides the ad (pre-click)

7. The pre-click ad experience
How to measure that an ad is bad?
What makes an ad bad?
How to predict that an ad is bad?
The user hides the ad

8. Using ad feedbacks as signal of bad ad

9. Metric of ad pre-click experience
Offensive Feedback Rate (OFR): offensive feedback / impression
highly offensive
ads

10. CTR vs. Offensiveness (OFR)
Bad ads attract clicks (clickbaits?)
Weak Correlation CTR/OFR
• Spearman: 0.155
• Pearson: -0.043
Quantile analysis
• High OFR ⇔ various CTR
• Higher CTR ⇔ higher OFR

11. What makes an ad preferred by users?
Methodology
● Pair-wise ad preference + reasons
● Sample ads with various CTR (whole spectrum)
● Comparison within category (vertical)

12. What makes an ad preferred by users?
Underlying preference reasons
● Aesthetic appeal > Product, Brand, Trustworthiness > Clarity > Layout
● Vertical differences:
○ personal finance (clarity)
○ beauty and education (product)

13. Engineering ad pre-click features
brand
HISTORICAL FEATURES
click-through rate, dwell time, bounce rate …
BRAND
READABILITY
SENTIMENT
AESTHETICS
VISUALS

14. Engineering ad pre-click features
User reasons Engineerable ad copy features
Brand Brand (domain pagerank, search term popularity)
Product/Service Content (category, adult detector, image objects)
Trustworthiness
Psychology (sentiment, psychological incentives)
Content Coherence (similarity between title and desc)
Language Style (formality, punctuation, superlative)
Language Usage (spam, hatespeech, click bait)
Clarity Readability (Flesch reading ease, num of complex words)
Layout
Readability (num of sentences, words)
Image Composition (Presence of objects, symmetry)
Aesthetic appeal
Colors (H.S.V, Contrast, Pleasure)
Textures (GLCM properties)
Photographic Quality (JPEG quality, sharpness)

15. Sentiment analysis is the detection of attitudes
“enduring, affectively colored beliefs, dispositions towards objects or persons”
Sentiment features
Types of attitudes
● From a set of types
like, love, hate, value,
desire, etc.
● Or (more commonly)
simple weighted polarity:
○ positive, negative,
neutral
○ their strength

16. Language style features
F-score: quantify the level of formality, where formality specifically
defined as context-independence
Feature Description
Punctuation # of different punctuation marks, including exclaim point ‘!’,
question mark ‘?’ and quotes
Start with number whether text starts with number
Start with 5W1H whether text starts with “what”, “where”, “when”, “why”, “who”
and “how”
Contain superlative whether text contains a superlative adverb or adjective
# of slang words number of slang words used
# of profane words number of profane words used

17. Visual features
Color Distribution
Hue, Saturation, Brightness
Rule of Thirds
Image Composition and Layout
Emotional Response
Pleasure, Arousal,
Dominance
Depth of Field
Sharpness contrast
between foreground
and background
Objective Quality
Sharpness, Noise, JPEG
quality, Contrast
Balance, Exposure
Balance

18. Feature correlation with OFR
Offensive ads tend to:
● start with number
● maintain lower image JPEG quality
● be less formal
● express negative sentiment in the ad title

19. Data
Around 30K native ads served on iOS and Android
Ad feedback data obtained from Yahoo news stream
Classifier
Logistic Regression as a binary classifier
● positive examples: high quantile of ad OFR
● negative examples: all others
Evaluation
5-fold Cross-validation
Metric: AUC (Area Under the ROC Curve)
Predicting a bad pre-click experience

20. Model performance
Performance per feature:
1. product
2. trustworthiness
3. brand
4. aesthetic appeal
5. clarity
6. layout
Model performance (AUC)
● No historical: 0.77
● Historical: 0.70
● Both: 0.79

21. A/B testing online evaluation
Baseline system (A):
Score(ad) = bid * pCTR
Pre-click experience System (B)
• Eliminate the ad from ad ranking if P(offensive|ad) >
• determined by other constraints (e.g. revenue impact)
OFR decreases by -17.6%

22. Take-away messages
How to measure the ad pre-click experience?
Offensive feedback rate as a metric
Metric
Features
Model
A/B testing
What makes an ad good?
Aesthetic appeal > Product, Brand,
Trustworthiness > Clarity > Layout
How to model?
Mining ad copy features from ad text, image
and advertiser + Logistic regression
Does it work?
Effective in identifying bad pre-click ads