[WI 2017] Context Suggestion: Empirical Evaluations vs User Studies
1. Context Suggestion:
Empirical Evaluations vs User Studies
Yong Zheng
School of Applied Technology
Illinois Institute of Technology
Chicago, IL, 60616, USA
The 2017 IEEE/WIC/ACM Conference on Web Intelligence (WI)
August 23-26, 2017, Leipzig, Germany
2. Agenda
• Intro: Context-aware Recommender Systems
• Motivations: Context Suggestion
• Methodologies and Research Problems
– Direct Context Prediction
– Indirect Context Suggestion
• Experimental Results and Findings
• Conclusions and Future Work
2
3. Agenda
• Intro: Context-aware Recommender Systems
• Motivations: Context Suggestion
• Methodologies and Research Problems
– Direct Context Prediction
– Indirect Context Suggestion
• Experimental Results and Findings
• Conclusions and Future Work
3
5. Context-Aware Recommendation
5
Companion
User’s decision may vary from contexts to contexts
• Examples:
➢ Travel destination: in winter vs in summer
➢ Movie watching: with children vs with partner
➢ Restaurant: quick lunch vs business dinner
➢ Music: for workout vs for study
6. Terminology in CARS
6
• Example of Multi-dimensional Context-aware Data set
➢Context Dimension: time, location, companion
➢Context Condition: Weekend/Weekday, Home/Cinema
➢Context Situation: {Weekend, Home, Kids}
User Item Rating Time Location Companion
U1 T1 3 Weekend Home Kids
U1 T2 5 Weekday Home Partner
U2 T2 2 Weekend Cinema Partner
U2 T3 3 Weekday Cinema Family
U1 T3 ? Weekend Cinema Kids
7. What is Context?
7
The most common contextual variables:
➢Time and Location
➢User intent or purpose
➢User emotional states
➢Devices
➢Topics of interests, e.g., apple vs. Apple
➢Others: companion, weather, budget, etc
Usually, the selection/definition of contexts is a domain-specific problem
8. Agenda
• Intro: Context-aware Recommender Systems
• Motivations: Context Suggestion
• Methodologies and Research Problems
– Direct Context Prediction
– Indirect Context Suggestion
• Experimental Results and Findings
• Conclusions and Future Work
8
9. Motivations: Context-Drive Applications
1) Context is necessary to maximize the user
experience. A list of good item
recommendations is NOT enough.
2) It is difficult to collect context information on
the Web!!!! Context suggestion provides a way
for context acquisition
9
14. Agenda
• Intro: Context-aware Recommender Systems
• Motivations: Context Suggestion
• Methodologies and Research Problems
– Direct Context Prediction
– Indirect Context Suggestion
• Experimental Results and Findings
• Conclusions and Future Work
14
15. Solution for Context Suggestion
• Direct Context Prediction
The output is a binary prediction
The value “1” indicates appropriate suggestion
The value “0” tells inappropriate suggestion
• Indirect Context Suggestion
The output is a top-N recommendations
Task: Suggest appropriate contexts for a user to
enjoy a given item
15
18. Indirect Context Suggestion
18
• Indirect Context Suggestion
Task:
Given a user and an item
Recommend a list of appropriate
contexts for the users to enjoy
the items
Item-Aware Context Recommendation
19. Research Problems
• Direct Context Prediction was explored in WI’14,
but Indirect Context Suggestion was never
discussed and compared with the direct context
prediction
• Previous research infers user preferences on
contexts from contextual ratings on the items – it is
not validated that whether contextual ratings can
tell whether a user prefers a given contexts to enjoy
the items
• There are no evaluation standards for context
suggestion
19
20. Agenda
• Intro: Context-aware Recommender Systems
• Motivations: Context Suggestion
• Methodologies and Research Problems
– Direct Context Prediction
– Indirect Context Suggestion
• Experimental Results and Findings
• Conclusions and Future Work
20
21. Data Collection
• It’s first time to collect user’s tastes on contexts
• 5043 ratings by 97 users on 79 movies
21
23. Evaluation Mechanisms
We propose the evaluation standards for context suggestion
• Top-N Context Prediction
– N varies from 1 to the number of context conditions
– Any available N value is fine
– It does not matter if two contexts from a same variable are
suggested. For example, {weekend, weekday, home, kids}
• Exact Context Suggestion
– Top-N, but N = the number of context condition
– For each dimension, we only suggest one condition
For example, {weekend, home, kids}
23
28. Conclusions and Findings
28
• “General” indicates user’s general preferences on
contexts for movie watching, without considering
which movie it is
• Personalization is required, since many algorithms
outperform the “General” method
• UISplitting and TF are the best ones
• Indirect context suggestion is better to offer better
suggestions than direct context prediction
• The results by using contextual ratings and user
tastes on contexts are consistent.
29. Future Work
29
• We will try to collect more data and evaluate these
solutions on larger data set
• We will try to utilize the context suggestion
methods to predict emotional states
• We will seek solutions to improve the indirect
context prediction
30. Context Suggestion:
Empirical Evaluations vs User Studies
Yong Zheng
School of Applied Technology
Illinois Institute of Technology
Chicago, IL, 60616, USA
The 2017 IEEE/WIC/ACM Conference on Web Intelligence (WI)
August 23-26, 2017, Leipzig, Germany