What’s in a Query? Understanding query intent

What’s in a Query?
Understanding query
intent
Bharat Thakarar
Subhadeep Maji
Mohit Kumar

Flipkart confidential - For Internal use only. Not to be shared externally.
E-commerce Search
Query: rectangle
room mat

● Search over structured product catalog
○ Products belong to a ‘store’
■ Eg: ‘Home Furnishing’ -> ‘Floor Coverings’ -> ‘Carpets & Rugs’
E-commerce Search

○ Products have key-value attributes
■ Eg: Shape: ‘Rectangle’; Style: ‘Iranian’;
Place of use: ‘Living room’
E-commerce Search

○ Products have key-value attributes
■ Eg: Shape: ‘Rectangle’; Style: ‘Iranian’; Place of use: ‘Living room’
● Intent of a query: ‘rectangle room mat’
○ Store: ‘Home Furnishing’ -> ‘Floor Coverings’ -> ‘Carpets &
Rugs’
○ Attribute Tagging: <shape>: ‘rectangle’ <place of use>: ‘living
room’ <store>: ‘mat’
E-commerce Search

Life of a query - simplified view
Ranking
- Relevance
- Query independent
signals
- ...
Augmentation
- Normalisation
- Spell Correction
- Phrasing
- Stemming
- Synonymization
- ...
Intent Understanding
- Store identification
- Intent Tagging
- …

Query to Store identification :
Why? (Customer Focused)

Why? (Customer Focused)
Lifestyle
Bigger Images, Less Text
Mobiles & Large
Spec heavy
Furniture
Aspect Ratio, Swatches

Why? (Internal)
● Establishes context for the query attribute tagging
○ Restricts labeling space
● Backend efficiency
● ...

● Source: click - log data (query -> products clicked ->
stores)
● Statistical aggregation of click measure
● Empirically determined confidence level for redirection
○ Sample data: ‘rectangle room mat’ : ‘Home Furnishing’ -> ‘Floor
Coverings’ -> ‘Carpets & Rugs’ : 95% confidence
Statistical approach (baseline)

● Works on exact queries, memorises (no generalization)
● Cannot learn anything useful for verticals where query
volume and product clicks are low
Statistical approach -
Challenges

L1 level store identification
● ML problem setup
○ Short text multi-label multi-class classification
○ Order of 10s L1 classes
● Model: Linear SVM (One vs All)
● Feature sets
○ BOW features (tf.idf)
○ Store name overlap features (tf.idf)

L1 level store identification: Results
Before After
Query: canvas car body covers

L1 level store identification: Results
Before After
Query: T-Series led tv

L1 level store identification: Impact
● Backend metrics
○ Nearly 40% drop in queries without
store (saving valuable compute resources)
● First user path deployment of
ML platform’s modelhost
Backend requests without stores

○ Short text *multi-label* multi-class classification
○ Order of 1000s leaf stores
Leaf level store identification

○ Short text *multi-label* multi-class classification
○ Order of 1000s leaf stores
● Challenges in extending L1 model:
○ Data sparsity
■ Linear SVM (One vs All) scaling for 1000s of classes
○ BOW features (no generalisation, no sharing)

● Approach: fastText
● Key idea(s):
○ Leverage word2vec (cbow) model
where instead of target word use label
instead
○ Hierarchical softmax - scaling to large
number of classes
fastText: https://github.com/facebookresearch/fastText

Leaf level store identification:
How were challenges addressed?
● Data sparsity
○ Using catalog data for seeding the embeddings
○ Helps learn with less amount of labeled data
● BOW features (no generalisation, no sharing)
○ Embeddings help in the abstraction

● Significant A/B metrics
○ +3 bps Search Conversion
○ +2 bps Visit Conversion
● SQA analysis (PBAGE): 8% improvement
Leaf level store identification - Impact

● Classifier trained only on catalog space (lot more labeled
data) didn’t work well in query space as-is
● Seed embeddings trained with store context in catalog
space work
Leaf level store identification:
Some Learnings

Given a query predict the attributes that best describe
the terms (chunks) in the query
Query: kids party dress 4-5 years pack of 2
Tagging <ideal_for>: kids <occasion>: party <store>:
dress <size>: 4-5 years <pack_of>: pack of 2
Query Intent Tagging

● Use Query product click through logs
● For each query, click product pair
○ Identify the attributes matched from product description
to query tokens
○ Store the fraction of the match to attributes for each
query token
Statistical Aggregation

● Works on query token space, weak generalization
● Considers all clicks equally but clicks are noisy
● Cannot learn anything useful for verticals where
query volume is low
Limitations

● samsung galaxy j7
○ brand model_name model_name
● samsung galaxy j7 covers
○ designed_for designed_for category
Problem Complexity

Some Exploratory Analysis
● ~40 % catalog
tokens cannot be
identified
unambiguously
● “Cotton” appears
in vocabulary of 23
attributes in
“HomeFurnishing”

● Attribute labelling at a position depends on tokens at
other positions in the query
● Attributes have affinity (brand, model_name) more
likely than (brand, color) in mobiles
Is Sequence necessary?

● Let X be the query s.t X = {x1, x2, . . . , xn} where xj is
a query token
● Let Y be the intent s.t Y = {y1, y2, . . . , yn} where yj ∈
attributes
Sequence Formulation

Supervised - Conditional
Random Field

● looks_like_attribute
○ Attributes like brand, color, model_name
○ Multinomial NB to generate features
● Defined over window at each position in query
● Global feature like is_alnum, is_shortword
Feature Design

● Moving from query token space to attribute feature
space, improves generalization
● Can generate multiple partial labellings, better
ranking of search results
What did we gain ?

● Significant A/B metrics
○ +5 bps Search Conversion
○ +2 bps Visit Conversion
● SQA analysis (PBAGE): 4% improvement
What did we gain ? Metrics

Query : samsung galaxy s7 edge 2017
Some Examples
AfterBefore

Query: Watches with steel belt with square dial
Some Examples..
AfterBefore

● Low volume of high confidence labeled data in some
verticals
● Click noise, users sometimes click randomly,
especially for lifestyle
● The labeled data for CRF suffers from above issues
Why CRF is not enough ?

Some Exploratory Analysis...
● Labeled data has low
coverage of on
unique queries ~ 10
%
● A supervised model
will fail to generalize
for these stores

● Generative vs a Discriminative setting like CRF
● Learning from unlabeled queries
● Catalog and limited labeled data used as weak
supervision
● WIP… research paper … production
Weakly-Supervised Models

Summary

● Pattern of solution evolution
○ Statistical -> Supervised -> Supervised ++ (side information)
● Common challenges
○ Not enough labeled data (side information / weak supervision)
○ Label/presentation bias
Summary

Query: ‘diamond ring’

Questions ?

What’s in a Query? Understanding query intent

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to What’s in a Query? Understanding query intent

Similar to What’s in a Query? Understanding query intent (20)

Recently uploaded

Recently uploaded (20)

What’s in a Query? Understanding query intent