On Search, Personalisation and Real-time Advertising

On Search, Personalisation and
Real-time Advertising
Dr. Jun Wang, Senior Lecturer
Computer Science, University College London
Email: j.wang@cs.ucl.ac.uk Twitter: @seawan
30/10/14
Dunnhumby
Talk
1

Summary
30/10/14
Dunnhumby
Talk
2

Web search Ads (display
opportunities)
- Maximize
profit
Search results
- Maximize users’
satisfactions?
Query
30/10/14
Dunnhumby
Talk
3

Recommder
Systems
30/10/14
Dunnhumby
Talk
4

Recommeder
Systems
Kruschwitz,
Udo
<udo@essex.ac.uk>
30/10/14
Dunnhumby
Talk
5

Real-‐2me
Adver2sing
“This
is
Lawrence
from
India.
I
was
searching
Recommender
model
in
web
and
found
your
webpage
in
search
engine.
Then,
I
visited
your
webpage
searching
relevant
contents
and
saw
unrelevant
Google
add
in
"Research
Team"
page
(aFached
screenshot).
This
add
might
vary
from
country
to
country.
But
I
feel
it
will
mislead
and
give
wrong
opinion
to
users
who
visit
your
webpage.”
-‐
Lawrence
from
India

Summary
30/10/14
Dunnhumby
Talk
9

Search
(Informa2on
Retrieval)
l General
definiJon:
search
large-‐scale
unstructured
data,
mostly
text
documents,
but
also
include
images,
videos,
etc
l ApplicaJons:
–
web
search
–
product
search
–
enterprise
search
–
desktop/email
search
–
informaJon
filtering
–
collaboraJve
filtering
and
recommeder
systems
30/10/14
Dunnhumby
Talk
10

Queries
can
have
ambiguous
intents
[Courtesy
of
F.
Radlinski,
MSR
Cambridge]
Columbia
clothing/sportswear
Colombia
(Country:
misspelling)
Columbia
University
Columbia
Records
music/video
columbia
30/10/14
Dunnhumby
Talk
11

Diversified
search
results
Diversifica2on
-‐>
nega2ve
correla2on
-‐>
reduce
the
risk:
see
our
sigir09
paper
30/10/on
14
porQolio
theory
of
informaDunnhumby
2on
Talk
retrieval
12

Recall
driven
personalised
search:
relevance
feedback
revisit
• www13
paper
exploratory
relevance
ranking
Xiaoran
Jin,
Marc
Sloan,
and
Jun
Wang.
InteracJve
Exploratory
Search
for
MulJ
Page
Search
Results,
www13
Figure 1: Example application, where Page 1 contains the Page 2 contains a refined, personalised re-ranking of the Personalised
re-‐ranking
30/10/14
Dunnhumby
Talk
13

Recall
driven
search:
relevance
feedback
revisit
• www13
paper
Exploratory
ranking
Personalised
re-‐ranking
Xiaoran
Jin,
Marc
Sloan,
and
Jun
Wang.
InteracJve
Exploratory
Search
for
MulJ
Page
Search
Results,
www13
30/10/14
Dunnhumby
Talk
14
contains the diversified, exploratory relevance ranking, and

ranking. We let s represent all rank actions s1 . . . sT. We
denote r = [r1, . . . , rK] as the vector of feedback informa-tion
Recall
obtained driven
from the user search:
for a given page, relevance
where K is the
number of documents given feedback ri is the feedback feedback
information gained revisit
with 0  K  M, and
(the rating provided
by the user) of relevance feedback for document i, either by
measuring a direct rating or by observing clickthroughs.
We use a weighted sum of the expected DCG@M scores of
the rankings of the T upcoming result pages, denoted here
by (note that Rst
• We
consider
MulJ
Page
Search
Results
• Intend
to
opJmise
overall
expected
effecJveness
over
the
search
journey
• Our
j ⌘ Rt
st
j
)
Us =
XT
t
0
@!t
XtM
j=1+(t−1)M
derivaJon
shows
that
to represent the user’s overall satisfaction, where E(Rst
– Page
E(Rst
j
)
log2(j + 1)
1
A (2)
) =
1
contains
the
diversified,
exploratory
relevance
✓st
is the expected relevance of a document at rank j in
ranking
– Page
j
result page t. We have chosen the objective function as it
is simple and both rewards finding the most relevant docu-ments
2
contains,
personalised
re-‐ranking
of
the
next
j
and also ranking them in the correct order, although
set
of
remaining
documents,
where
the
relevance
feedback
other IR metrics is
triggered
can be adopted by
the
similarly. “Next”
The burank gon
weight
1
log2 j is used to give greater weight to ranking the most rele-vant
documents in higher positions. The tunable parameter
!i # 0 is used to adjust the importance of result pages and
thus the level of exploration in the initial page(s). When !1
U1 Figure by the diagram, random the rank node is conditional the feedback P(R2= where rsat Xiaoran
Jin,
Marc
Sloan,
and
Jun
Wang.
InteracJve
Exploratory
Search
for
MulJ
Page
Search
Results,
www13
30/10/14
Dunnhumby
Talk
15

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
Page
1:
diversified,
exploratory
relevance
ranking
considers
Relevancy
+
Variance
+
|CorrelaJons|
Page
2:
personalised
re-‐ranking
16

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
Q
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
Page
1:
diversified,
exploratory
relevance
ranking
considers
Relevancy
+
Variance
+
|CorrelaJons|
17

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
Q
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
Page
1:
diversified,
exploratory
relevance
ranking
considers
Relevancy
+
Variance
+
|CorrelaJons|
18

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
Q
-‐1
-‐1
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
+1
Page
1:
diversified,
exploratory
relevance
ranking
considers
Relevancy
+
Variance
+
|CorrelaJons|
19

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
Q
-‐1
-‐1
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
Page
2:
Personalised
reranking:
+1
Q
20

How
it
works
x
x
x
x
x
x
x
x
x
x
x
o
o
o
Q
-‐1
-‐1
30/10/14
Dunnhumby
Talk
o
o
o
o
¤
¤
¤
¤
¤
¤
¤
¤
x
x
X:
doc
about
apple
fruit
doc
about
apple
ceo
¤
¤
O:
doc
about
apple
iphone
Page
2:
Personalised
reranking:
+1
Q
21

Summary
30/10/14
Dunnhumby
Talk
22

Is
Personalized
Rec.
Always
BeFer?
Non-‐personalized
Top
Ar2sts
in
October
Personalized:
Ar2sts
Recommended
for
You

Personalized
vs
Non-‐Personalized
Dataset:
Movielens-‐100k

Personalized
vs
Non-‐Personalized
• Personalized
top-‐N
CF
as
a
learning
model
– Improve
the
object
of
overall
relevance
– But
does
NOT
improve
on
each
user
POP BPR
Top-‐N
Performance Low
bias
High
variance

Switch
between
Two
PorQolios
Porqolio
DominaJon

Case
Study
1
• Demography:
50
years
old
male
programmer
• History:
86
feedbacks,
most
of
which
are
unpopular
items

Case
Study
1
• Analysis:
BPR
porqolio
dominates
POP
porqolio
• Results:
BPR
has
beger
ranking
performance

Case
Study
2
• Demography:
28
years
old
male
engineer
• History:
15
feedbacks,
most
of
which
are
popular
items

Case
Study
2
• Analysis:
POP
porqolio
dominates
BPR
porqolio
• Results:
POP
has
beger
ranking
performance

Cold-‐start
problem
in
recommmender
systems

Interac2ve
Recommender
Systems

Possible
Solu2ons
Zhao,
Xiaoxue,
Weinan
Zhang,
and
Jun
Wang.
"InteracJve
collaboraJve
filtering."
CIKM,
2013.

Objec2ve
Interac2ve
Cold-‐start
problem
mechanism
for
CF
Zhao,
Xiaoxue,
Weinan
Zhang,
and
Jun
Wang.
"InteracJve
collaboraJve
filtering."
CIKM,
2013.

Proposed
EE
algorithms
Thompson
Sampling
Linear-‐UCB
General
Linear-‐UCB
Zhao,
Xiaoxue,
Weinan
Zhang,
and
Jun
Wang.
"InteracJve
collaboraJve
filtering."
CIKM,
2013.

Cold-‐start
users
Zhao,
Xiaoxue,
Weinan
Zhang,
and
Jun
Wang.
"InteracJve
collaboraJve
filtering."
CIKM,
2013.

Summary
30/10/14
Dunnhumby
Talk
37

Life
of
a
display
ad
in
the
RTB
environment:
0.36
seconds
39
Ad
Exchange
Demand-Side
Platform
Advertiser
Data
Management
Platform
0.
Ad
Request
1.
Bid
Request
(user,
context)
2.
Bid
Response
(ad,
bid)
4.
Win
NoJce
3.
Ad
AucJon
(paying
price)
5.
Ad
(with
tracking)
6.
User
Feedback
(click,
conversion,
etc.)
User
InformaJon
User
Demography:
Male,
25,
Student,
etc.
User
SegmentaJons:
Ad
science,
London,
etc.
Webpage
User

DSP
(Demand
Side
PlaQorm)
30/10/14

Op2mal
Bidder:
Problem
Defini2on
Bid
Request
Bid
Engine
Bid
Price
42
Input:
bid
request
include
Cookie
informaJon
(anonymous
profile),
website
category
&
page,
user
terminal,
locaJon
etc
Output:
bid
price
Considera2ons:
Historic
data,
CRM
(first
party
data),
DMP
(3rd
party
data
from
Data
Management
Plaqorm)
What
is
the
op2mal
bidder
given
a
budget
constraint?
e.g.,
Maximise
Subject
to
the
budget
constraint

43
The
General
Process
for
Bidding
Op2misa2on
Red:
hard
constraints
Green:
features
Blue:
models
Note
that
“Frequency
&
recency
rules”
are
also
used
as
features

Op2mal
bidder:
the
formula2on
• FuncJonal
OpJmisaJon
Problem
– Dependency
assumpJon:
• SoluJon:
Calculus
of
variaJons
context+ad
features
winning
funcJon
CTR
esJmaJon
bidding
funcJon
Weinan
Zhang,
Shuai
Yuan,
Jun
Wang,
OpJmal
Real-‐Time
Bidding
for
Display
AdverJsing,
KDD’14

Op2mal
bidder:
the
solu2on
Weinan
Zhang,
Shuai
Yuan,
Jun
Wang,
OpJmal
Real-‐Time
Bidding
for
Display
AdverJsing,
KDD’14

Experiments
Offline
Online
Winner
of
the
first
global
Real-‐Jme
Bidding
algorithm
contest
2013-‐2014
Weinan
Zhang,
Shuai
Yuan,
Jun
Wang,
OpJmal
Real-‐Time
Bidding
for
Display
AdverJsing,
KDD’14

UCL
OpenBidder
Benchmarking
System
47

time (see Figure 1). This makes the cost of displaying ad slots (for
advertisers) and the advertising incomes (for publishers and search
engines) unpredictable. (RTB)
Ads
Thus prices
there are increasing are
volaneeds 2of le
a new
advertising trading mechanism to manage the risk of cost or income.
(a)
The
price
movement
of
a
display
opportunity
from
Yahoo!
ads
data
Under
GSP
(generalized
second
price
aucJon)
50
Ad slot price (GSP)
1.5
1
Price change rate
30/10/14
Dunnhumby
Talk
48

Automa2ng
Ads
Futures/Op2on
Contracts
• Need
Ad’s
Futures
Contract
and
Risk-‐reduc5on
Capabili5es
– Technologies
are
constrained
mainly
to
“spots”
markets,
i.e.,
any
transacJon
where
delivery
takes
place
right
away
(in
Real-‐Jme
AdverJsing
and
Sponsored
Search)
– No
principled
technologies
to
support
efficient
forward
pricing
&risk
management
mechanisms
• If
we
got
Futures
Market
or
provide
Op2on
Contracts,
adverJsers
could
lock
in
the
campaign
cost
and
Publishers
could
lock
in
a
profit
in
the
future
30/10/14
Dunnhumby
Talk
49

Futures
Exchange
(Programma2c
Guarantee)
Advertiser
Demand
Side
Platform
(DSP)
Futures
Exchange
RTB / Spot
Exchange
Supply
Side
Platform
(SSP)
Publisher
3rd party data providers, ad
serving, ad agency, ad
networks, campaign analytics
-10% to -30%

Acknowledgements
• Thanks
to
my
PhD
students
Weinan
Zhang,
Shuai
Yuan,
Marc
Sloan,
Xiaoxue
Zhao
30/10/14
Dunnhumby
Talk
51

For
more
informa2on,
please
refer
to
1. Wang,
Jun,
and
Jianhan
Zhu.
"Porqolio
theory
of
informaJon
retrieval."
SIGIR,
2009.
2. Jin,
Xiaoran,
Marc
Sloan,
and
Jun
Wang.
"InteracJve
exploratory
search
for
mulJ
page
search
results."
WWW,
2013.
3. Zhang,
Weinan,
et
al.
"To
personalize
or
not:
a
risk
management
perspecJve."
Proceedings
of
the
7th
ACM
conference
on
Recommender
systems.
ACM,
2013.
4. Gorla,
Jagadeesh,
et
al.
"ProbabilisJc
group
recommendaJon
via
informaJon
matching."
WWW,
2013.
5. Shuai
Yuan,
Jun
Wang,
Real-‐Jme
Bidding
for
Online
AdverJsing:
Measurement
and
Analysis,
AdKDD’13
hgp://arxiv-‐web3.library.cornell.edu/abs/1306.6542
6. Weinan
Zhang,
Shuai
Yuan,
Jun
Wang,
OpJmal
Real-‐Time
Bidding
for
Display
AdverJsing,
KDD’14
7. Shuai
Yuan,
Jun
Wang,
Bowei
Chen,
An
Empirical
Study
of
Reserve
Price
OpJmisaJon
in
Real-‐Time
Bidding
8. Bowei
Chen,
Jun
Wang,
Ingemar
Cox,
and
Mohan
Kankanhalli,
MulJ-‐
Keyword
MulJ-‐Click
OpJon
Contracts
for
Sponsored
Search
AdverJsing,
under
submission,
2013
hgp://arxiv.org/abs/1307.4980

Thanks
for
your
aFen2on
53

On Search, Personalisation and Real-time Advertising

Recommended

Recommended

More Related Content

Similar to On Search, Personalisation and Real-time Advertising

Similar to On Search, Personalisation and Real-time Advertising (20)

Recently uploaded

Recently uploaded (20)

On Search, Personalisation and Real-time Advertising