Context-aware user modeling strategies for journey plan recommendation
•
6 likes•1,068 views
UMAP 2015 presentation about paper "Context-aware user modeling strategies for journey plan recommendation"
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Context-aware user modeling strategies for journey plan recommendation
- 1. Context-Aware User Modeling Strategies for Journey Plan Recommendation Victor Codina José Mena Luis Oliva
- 2. Current urban journey planners UMAP’15 V. Codina, J. Mena, L. Oliva 2 Google Maps Moovit Citymapper
- 3. Transport mode selection Lack of preference learning from user's feedback Only based on explicit transport preferences Less transfersLess walking But... limited personalization UMAP’15 V. Codina, J. Mena, L. Oliva 3 Shorter routes
- 4. 2) Current context To personalize suggested journey plans based on: How: with context-aware recommendation strategies Research goal UMAP’15 V. Codina, J. Mena, L. Oliva 4 1) Past user ratings
- 5. Recommender engine decoupled from planner Task: context-aware, user's satisfaction scoring Role of our journey plan recommender UMAP’15 V. Codina, J. Mena, L. Oliva 5 Journey plan Target context Target user preferences Predicted score UI
- 6. Journey plans are highly dynamic Unique routes per user's request Challenge: new item (early rater) problem How to recommend items with no ratings at all? Solution: Content-based Filtering New item problem solved by exploiting item’s content Journey plan recommendation domain UMAP’15 V. Codina, J. Mena, L. Oliva 6 Collaborative Filtering not an option!
- 7. Idea: “show me more of the same kind of plans I liked” Score predictions based on feature vector matching Content-based approach to journey plan recommendation UMAP’15 V. Codina, J. Mena, L. Oliva 7 Item Profile User profile 0.2 1 0.5 0 1 0.5 -0.7 0.2 0.8 -0.8 Journey plan vector Score = User preference vector ∈ ∈ n = # attribute values Derived via Stochastic Gradient Descent by solving the regularized least squares problem over the training ratings
- 8. Originally we had these attributes per leg: Transport mode (e.g. Walk, Tram, Bus, Taxi, Car, Bike, …) Time (seconds) Cost (€) Physical effort (Kj) Example: Journey plan original data UMAP’15 V. Codina, J. Mena, L. Oliva 8 Car Tram Walk 600s 3€ 0kj 900s 1€ 0kj 300s 0€ 50kj Journey plan Leg 1 Leg 2 Leg 3
- 9. Numeric attributes discretized into 5 equal intervals E.g. Walk_Effort: [very low | low | medium| high | very high] Discretization by soft cuts using a fuzzy-set approach Journey plan feature vector representation UMAP’15 V. Codina, J. Mena, L. Oliva 9 medium =0.5 high=0.5low= 0 150Kj Interval values defined by mobility experts 225Kj 275Kj 425Kj X = 350Kj
- 10. IDEA: user’s preferences can depend on context Context matters for journey plan scoring UMAP’15 V. Codina, J. Mena, L. Oliva 10
- 11. Context as additional dimension for prediction 3 main paradigms depending on how context is used Classification of context-aware strategies UMAP’15 V. Codina, J. Mena, L. Oliva 11 target context Prediction model in-context training ratings target Item Pre-filtering Post-filteringContextual modeling Predicted score
- 12. Reduction-based approach that builds local user vectors Using the ratings identified as relevant for the target context The pre-filtering strategy: Distributional Semantic Pre-Filtering (DSPF) UMAP’15 V. Codina, J. Mena, L. Oliva 12 local ratings Ratings filtering Local user modeling ≈ ≠ Similarities based on the distributional semantics of contextual conditions in-context training ratings target context local user vectors
- 13. Method inspired on time-aware Matrix Factorization1 Content-Based linear model extended with: Global contextual biases e.g. “users tend to rate lower routes with long walks when hot” Preference-specific contextual biases e.g. “John rates higher bike routes when sunny” The contextual modeling strategy: Distributional Semantic Contextual Modeling (DSCM) UMAP’15 V. Codina, J. Mena, L. Oliva 13 Distributional similarities between conditions [1] Y. Koren “Collaborative filtering with temporal dynamics” Commun. ACM, 2010.
- 14. Question: which context-aware strategy is better for journey plan recommendation? Two types of evaluation Experiment 1: traditional offline evaluation Experiment 2: user-centric online evaluation Experimental Evaluation UMAP’15 V. Codina, J. Mena, L. Oliva 14 Pre-filtering (DSPF) Vs. Contextual modeling (DSCM)
- 15. Offline evaluation In-context journey plan rating acquisition UMAP’15 V. Codina, J. Mena, L. Oliva 15 Suggested journey plans Journey plan requests Context settings panel: • 2 user factors • 8 urban factors • ? = Unknown value
- 16. Contextually-tagged journey plan rating dataset in the city of Barcelona (Spain) Per-user data splitting Error-based performance metric: RMSE Offline evaluation: procedure UMAP’15 V. Codina, J. Mena, L. Oliva 16 Users 68 Journey plans 1,628 Attribute values 69 Contextual Factors 10 Conditions 38 Ratings 3,256
- 17. Both context-aware strategies are better than baseline (Free) But… differences between context-aware models are not significant (p-value = 0.25) Offline evaluation: results UMAP’15 V. Codina, J. Mena, L. Oliva 17
- 18. Participants used Android app under real contexts Developed by several partners of the SUPERHUB project User-centric online evaluation UMAP’15 V. Codina, J. Mena, L. Oliva 18 Journey plan request User context Top-4 recommendation Rating (used for model fine-tuning)
- 19. A/B testing method and user study questions User-centric online evaluation: setup UMAP’15 V. Codina, J. Mena, L. Oliva 19 4 aspects evaluated: • Top-n accuracy • Ranking accuracy • Context-awareness • Overall satisfaction Citizens of Barcelona (Spain)
- 20. User study results UMAP’15 V. Codina, J. Mena, L. Oliva 20
- 21. Offline exp. Online exp. DSPF > Free DSPF ≈ Free DSCM > Free DSCM > Free DSCM ≈ DSPF DSCM > DSPF Conclusions UMAP’15 V. Codina, J. Mena, L. Oliva 21 Free: Context-Free, Content-Based approach DSPF: Contextual Pre-filtering strategy with content-based approach DSCM: Contextual Modeling strategy with content-based approach
- 22. UMAP’15 V. Codina, J. Mena, L. Oliva 22 Context-Aware User Modeling Strategies for Journey Plan Recommendation QUESTIONS Victor Codina vcodina@bdigital.org
Editor's Notes
- I'm going to present our paper ... This is a joint work with Luis Oliva from Technical University of Catalonia and Jose Mena from Barcelona Digital tech center. My name is ...
- Nowadays there are several free mobile apps that assist you during urban journey planning, helping you to find the shortest routes between two arbitrary points in a city by combining different transport modes, private and public ones. Here I'm showing some popular examples that most of you probably know: Google Maps, Moovit, CityMapper. A common strong point of theses apps is that they provide to users with real-time information about transit conditions and transport services.
- However, a common limitation of these systems is their lack of user preference learning, which means they are not exploiting previous user interactions with the system to learn about their preferences and adapt journey plan suggestions to them Commonly these apps allow users to personalize the generated routes by explicitly specifying a set of generic transport preferences for the current request. These are for example the kind of options that one can choose using Moovit. I prefer routes with less walking. I want shorter routes. Or I want routes with less tranport mode interchanges.
- This limitation of current journey planners is what motivated this work, whose research goal was to develop a journey planner able to generate journey plans recommendations based on user's feedback in the form of ratings, as shown in this screenshot, and adapted to the current context. To accomplish this goal we decided to employ SoA context-aware user modeling and recommendation strategies, given their more than proved effectiveness in other domains, such Point Of Interest recommendation or movie and music domains.
- Differently from other domains in our application the recommender engine is not the central component of the system. Instead, the recommender works as an independent module that is used by the journey planner to rank the set of generated candidate routes. So, in our system the recommendation task was formulated as a journey plan scoring problem, or more generally as a rating prediction problem. So, given a candidate journey plan and target user and context, the recommender taks consisted of predicting the score given by the user to that plan under such contextual conditions.
- A particularity of our recommendation domain was that journey plans, the items to recommend, where highly dynamic, in the sense that they were generated based on arbitrary start and destination points, which means they were mostly unique per request. This implied that our recommender should be able to make predictions in new item conditions, where the target item to recommend has no ratings of other users associated. So for this reason CF methods are not feasible in this particular task since they require ratings of different users on the same items to extract meaningful patterns from the data. In such conditions only CB or Knowledge based strategies are feasible because they are able to exploit item's metadata in order to build user models and make recommendations. So, in this work we used a content-based method as the basis of the context-aware strategies.
- The main assumption of CB approaches is that users tend to like items with similar attributes to those he or she already liked in the past. More formally, these approaches compute the suitability of an item for a user based on feature matching calculation. To do such comparison it is necessary to encode user preferences and item descriptions in the same attribute space, commonly using a feature vector representation. In the case of the item vector, positive values represent the set of attribute values that are relevant in the description of the item, and zero values the no presence of the attribute at all. In the case of user vectors, values represents the user's interests, which can be positive, negative or neutral . Once we have this feature representation of the target item and user, predictions can be generated by comparing the values of the two vectors. In this work we used the dot product, since it is one of the most popular methods. User vectors can be derived from the past user rating by using several methods. Here we calculate them by solving the least squares problem using SGD.
- So one can see that in CB recommendation the item vector construction is a very important step. Now I'm going to explain how we built the vectors from the original journey plan data. To do so, we extracted several categorical attributes from the numeric data provided by the journey planner about each candidate journey plan, which included the following information: - information about the time required to complete each leg composing the plan t - information about the cost associated to each mode of transport mode used in the plan (in €) – the physical effort required (if the plans include legs by foot or bike)
- From these data we derived several discrete attributes describing the time and cost associated to each mode of transport separately, which we group by walk, bike, public and private and also global features like the total number of public interchanges. Each of these attributes was discretized into 5 equal intervals: from very low to very high values. For the value assignation we used a fuzzy-set method in order to accurately classify values close to the boundaries. The result of this process is weighted feature vector with values between [0,1] indicating the weight of the attribute representing the journey plan.
- So far I have talked about how we generate context-free predictions, and now I’m going present the strategies we experimented with to incorporate context into the prediciton. The main assumption of CARS is that user preferences can depend on context. For example…
- CARS are commonly classified in three main paradigms depending on how they incorporate context: pre-filtering, post-filtering and contextual modeling. Pre-filtering methods use context only for selecting the rating data which is relevant to the target context, so they can be used in combination with any context-free recommendation method. This is also the case of post-filtering methods, but here context is exploited at the end of the process, to adjust context-free recommendations rather than data filtering. Finally, contextual modeling strategies are those approaches where context is incorporated into the estimation function as additional model parameters, giving rise to truly multi-dimensional prediction models Although there are several context-aware recommendation approaches out there, in our evaluatio with only experimented with two state-of-the-art methods, one based on pre-filtering and another one based on contextual modeling. We selected them because of their superior rating prediction accuracy and scalability in domains with high context granularity (where contextual situations are defined by the conjunction of several contextual conditions) as in this case. Now I will explain the main idea of each of the implemented methods:
- The pre-filtering method consists of an adaptation of DSPF, a sophisticated reduction-based approach that builds a local prediction model for each target contextual situation based on the ratings identified as relevant for the given context. It is called distributional semantic because this method defines two context as similar if their composing condition have similar distributional semantics, in the sense of how they influence the user ratings. This example illustrates how this strategy works: 1- Let's assume that our set of ratings is composed of ratings acquired in three different contexts, ratings provided while travelling with family, when sunny and in rainy conditions. Assuming that that the target context is sunny, and that family and sunny conditions have similar rating influencing patterns, the set of relevant ratings for that target context would be the ratings tagged with family and sunny. 2. Then, a local recommendation model is learned using the relevant subset of ratings. In particular, using our CB approach, a local user vector of preferences is learnt for each user, which will be used a posteriori to do the item-user matching in the given target context.
- This approach consists of extending a linear CB prediction model by introducing global and user-specific parameters which model the influence of context to the user's preferences. hat model the global effects of contextual conditions w.r.t. journey plan features captures the global context variability of user preferences with respect to the item attributes. So for example, one of these parameters could represent that in hot conditions users tend to rate positively routes with short walking distances. that model the context variability of user preferences
- Our goal was to understand which stratregy is better in this particular recommendation task, so our evaluation consisted of a performance comparison between these two strategies: the pre-filtering and the contextual modeling strategy. We did this comparison by means of two experiments: 1 using the traditional offline setting and 1 user-centric based on a A B testing method.
- Once requested, the planner generated a set of alternative joruney plans using different tranport mode combinations By clicking on each of the alternatives, users could see the plan details and vote the route under the active context .
- In order to evaluate the strategies from a system-centric perspective we first needed a way to collect a set of in-context ratings for a variety of journey plans and contextual situations To do so, we developed an ad-hoc web-based app in which users could request journey plans under imaginary contexts and rate them. 68 users participated in this experiment, which lasted one week. We collected more than 3000 ratings. As you can see the number of ratings collected doubles the number of rated journey plans. This is because, users were asked to rate the same plan in two different contextual situations. Next, I will explain this acquisition method in more detail. We split the collected data set on training and test sets by using an all-but-n protocol, in which a portion of the ratings of each users are held for testing and we measure the performance of the models in terms of their RMSE.
- After one week, we collected around 3000 ratings and each user provided on average 50 ratings . We split this data set on training and test in order to train the evaluated prediction models and measure their RMSE. This bar char shows the results. DSPF corresponds to the results of the pre-filtering strategy, DSCM to the contextual modeling strategy and Free refers to the context-free method using a linear CB model. As you can see both context-aware strategies clearly outperform the context-free baseline (the contextual modeling reduces RMSE by 14% and the prefiltering by 9%) Comparing the context-aware strategies we can observe that the contextual modeling seems to have better performance although we cannot state that these differences are statistically significant. So this offline evaluation was not useful to identify a clear winner.
- For this experiment users used the mobile app developed in the SUPERHUB project. Here you can see some screenshots of the UI trough which users could receive recommendations and rate them.
- As I said previously, we also compared the performance of the proposed context-aware strategies using A/B testing. For this user-centric experiment we used a different set of users but of similar size, particularly 67 users completed the experiment. This diagram shows the design of our experiment, in which users were randomly split in three groups of similar size, one group receiving recommendations from the pre-filtering strategy, another one using the contextual modeling and the other one using the context-free algorithm. Obviously users didn't know to which group were assigned to. TODO: Improve this
- Once completed the task, users answered 4 different questions on a five-point scale. Each of the questions was aimed to evaluate a different aspect of the recommendations: top-n accuracy, ranking accuracy, context-awareness, and the overall satisfaction with the system. These bar charts show the mean score given by each group of users to the 4 questions. One can easily observe that the contextual modeling strategy, DSCM, is the one with higher perceived recommendation accuracy and context-awareness, which leads to a higher user's satisfaction according to the correlation analysis. And this case we found that the differences with respect the prefiltering strategy were statistically significant, validating thus its superior performance in this application.