Native advertising is a specific form of online advertising where ads replicate the look-and-feel of their serving platform. In such context, providing a good user experience with the served ads is crucial to ensure long-term user engagement. In this work, we explore the notion of ad quality, namely the effectiveness of advertising from a user experience perspective. We design a learning framework to predict the pre-click quality of native ads. More specifically, we look at detecting offensive native ads, showing that, to quantify ad quality, ad offensive user feedback rates are more reliable than the commonly used click-through rate metrics. We then conduct a crowd-sourcing study to identify which criteria drive user preferences in native advertising. We translate these criteria into a set of ad quality features that we extract from the ad text, image and advertiser, and then use them to train a model able to identify offensive ads. We show that our model is very effective in detecting offensive ads, and provide in-depth insights on how different features affect ad quality. Finally, we deploy a preliminary version of such model and show its effectiveness in the reduction of the offensive ad feedback rate. There are the slides of our WWW 2016 paper. This is work with Ke (Adam) Zhou, Miriam Redi and Andy Haines.

2. Facebook
Suggested
Post
Twi3er
Promoted
Tweet
Yahoo
Sponsored
Content
Na#ve
adver#sing

3. Offensive
ads
disengage
the
users!
D.
G.
Goldstein,
R.
P.
McAfee,
and
S.
Suri.
The
cost
of
annoying
ads.
WWW
2013.
A.
Goldfarb
and
C.
Tucker.
Online
display
adver#sing:
Targe#ng
and
obtrusiveness.
MarkeIng
Science
2011.

4.
• How
to
measure?
• What
makes
an
ad
preferred
by
users?
• How
to
model?
Pre-­‐click
ad
quality

5.
• How
to
measure?
• What
makes
an
ad
preferred
by
users?
• How
to
model?
Pre-­‐click
ad
quality

6.
How
to
measure
the
pre-­‐click
quality?
• Is
CTR
(click-­‐through
rate)
a
good
pre-­‐click
metric?
– A
compounding
metric:
• Relevance:
how
ads
match
user
interests.
• Quality:
nature
of
the
ad
product
and
ad
creaIve
design
decision.
• Pre-­‐click
metrics
solely
measure
on
ad
quality?
– Let
us
elicit
from
the
users
(crowdsourcing)

7.
Using
ad
feedbacks
as
a
signal
of
bad
ad
quality

8.
Proxy
of
pre-­‐click
ad
quality
Offensive
Feedback
Rate
(OFR)
offensive
feedback
/
ad
impression

9.
CTR
vs.
Offensiveness
(OFR)
Bad
ads
a&rac)ng
clicks
(clickbaits?)
• Correlation between CTR and
OFR (very weak)
– Spearman: 0.155
– Pearson: -0.043
• Quantile analysis
– High OFR distribute across
ads with various CTR
– Higher CTR more ads with
higher OFR

10.
• How
to
measure?
• What
makes
an
ad
preferred
by
users?
• How
to
model?
Pre-­‐click
ad
quality

11. What
makes
an
ad
preferred
by
users?
● Methodology
○ Pair-­‐wise
ad
preference
+
reasons
○ Sample
ads
with
various
CTR
(whole
quality
spectrum)
○ Quality
based
comparison
within
category
(verIcal)
● Underlying
preference
reasons
○ Aesthe#c
appeal
>
Product,
Brand,
Trustworthiness
>
Clarity
>
Layout
○ VerIcal
Differences
personal
finance
(clarity)
beauty
and
educaIon
(product)
within
verIcal
comparison

12.
Can
we
engineer
ad
quality
features?
brand
readability,
senIment
aestheIc,
visual
User
Reasons
Engineerable
Ad
Crea#ve
Features
Brand
Brand
(domain
pagerank,
search
term
popularity)
Product/Service
Content
(YCT,
adult
detector,
image
objects)
Trustworthiness
Psychology
(senIment,
psychological
incenIves)
Content
Coherence
(similarity
between
Itle
and
desc)
Language
Style
(formality,
punctuaIon,
superlaIve)
Language
Usage
(spam,
hatespeech,
click
bait)
Clarity
Readability
(Flesch
reading
ease,
num
of
complex
words)
Layout
Readability
(num
of
sentences,
words)
Image
ComposiIon
(Presence
of
objects,
symmetry)
Aesthe;c
appeal
Colors
(H.S.V,
Contrast,
Pleasure)
Textures
(GLCM
properIes)
Photographic
Quality
(JPEG
quality,
sharpness)
○ By
mining
ad
copy
(Itle
and
descripIon),
image
and
adverIser
informaIon
○ Cold-­‐start
features

13.
We
also
use
historical
features
User
Behavior
Engineerable
Features
Click
CTR
(click-­‐through
rate)
Post-­‐click
Bounce
Rate
Average
Dwell
Time
We
mine
user
interacIons
with
the
ads

14.
Feature
correla#on
with
OFR
The
offensive
ads
tend
to:
start
with
number
maintain
lower
image
JPEG
quality
be
less
formal
express
negaIve
senIment
in
the
ad
Itle

15.
• How
to
measure?
• What
makes
an
ad
preferred
by
users?
• How
to
model?
Pre-­‐click
ad
quality

16.
Data
NaIve
mobile
iOS
and
Android
app
28,664
ads
(Sampled
from
March
01-­‐18,
2015)
Ad
feedback
data
obtained
from
Yahoo
news
stream
Classifier
Logis;c
Regression
as
a
binary
classifier
posiIve
examples:
high
quanIle
of
OFR
ads
negaIve
examples:
all
others
EvaluaIon
5-­‐fold
Cross-­‐validaIon
Metric:
AUC
(Area
Under
the
ROC
Curve)
Pre-­‐click
model:
Data
and
evalua#on
brand
readability,
sentiment
aesthetic, visual

17.
Overview
of
model
performance
Models
based
on
each
feature
category:
product
>
trustworthiness
>
brand
>
aestheIc
appeal
>
clarity
>
layout
Model
summary:
• cold
start:
AUC
(0.77)
• User
behavior:
AUC
(0.70)
• cold
start
+
user
behavior:
AUC
(0.79)

18. A/B
Tes#ng
online
evalua#on
• Baseline
System
– Score(ad)
=
bid
*
pCTR
• Pre-­‐click
Quality
System
– Eliminate
the
ad
from
ad
ranking
if
P(Offensive|ad)
>
𝛿
– 𝛿
is
determined
by
other
constraints
(e.g.
eCPM)
Mobile:
OFR
(-­‐17.6%)
Desktop:
OFR
(-­‐8.7%)

19. Take-­‐away
messages
• How
to
measure
pre-­‐click
ad
quality?
– Offensive
feedback
rate
as
a
metric
– Capture
bad
quality
be3er
than
CTR
• What
makes
an
ad
preferred
by
users
(reasons)?
– AestheIc
appeal
>
Product,
Brand,
Trustworthiness
>
Clarity
>
Layout
• How
to
model?
– Mining
ad
copy
features
from
ad
text,
image
and
adverIser
– EffecIve
in
the
predicIon

20. Ques#ons?
Ad
feedback
Offensive
Feedback
Rate
vs.
CTR
brand
readability,
senIment
aestheIc,
visual
PredicIve
model
by
mining
ad
features