3. Item Recommendation
โข Classical item recommendation problem (see Netflix)
โข Explicit feedbacks (ratings)
16th January, 20133 LAWA Workpackage Meeting
5 ?
?
The Matrix The Matrix 2 Twilight The Matrix 3
?
4. Collaborative Filtering (Explicit)
โข Classical item recommendation problem (see Netflix)
โข Explicit feedbacks (ratings)
โข Collaborative Filtering
โข Based on other users
16th January, 20134 LAWA Workpackage Meeting
5
5
4
5
5
?
?
The Matrix 3The Matrix The Matrix 2 Twilight
5
?
5. Collaborative Filtering (Implicit)
โข Items are not movies only (live content, products, holidays, โฆ)
โข Implicit feedbacks (buy, view, โฆ)
โข Less information about pref.
16th January, 20135 LAWA Workpackage Meeting
?
?
Item4Item1 Item2 Item3
?
6. Industrial motivation
โข Keeping the response time low
โข Up-to-date user models, the adaptation should be fast
โข The items may change rapidly, the training time can be a bottleneck of
live performance
โข Increasing amount of data from a customer ๏ Increasing training time
โข Limited resources
16th January, 20136 LAWA Workpackage Meeting
40. Optimizer โ Coordinate Descent
โข Complexity of naive solution: ๐ถ ๐ฐ๐ฒ๐ต๐ด
โข Ridge Regression calculates the features based on examples directly,
Covariance precomputing solution cannot be applied here.
40 LAWA Workpackage Meeting 16th January, 2013
41. Optimizer โ Coordinate Descent Improvement
โข Synthetic examples (Pilรกszy, Zibriczky, Tikk)
โข Solution of Ridgre Regression with CD: ๐ ๐ข๐ = ๐=1
๐
๐ ๐ข๐ ๐ ๐๐ ๐ ๐ข๐
๐=1
๐
๐ ๐ข๐ ๐ ๐๐ ๐ ๐๐
=
๐๐๐ธ
๐๐๐
โข Calculate statistics for this user, who watched nothing (๐๐ธ๐0 and ๐๐๐0)
โข The solution is calculated incrementally: ๐ ๐ข๐ =
๐๐๐ธ
๐๐๐
=
๐๐๐ธ0+๐๐๐ธ+
๐๐๐0+๐๐๐+
( ๐ด + #๐ท(๐)+ steps)
โข Eigenvalue decomposition: ๐ ๐
๐ = ๐ฮ๐ ๐
= ๐ ฮ
๐
ฮ๐ = ๐บ ๐
๐บ
โข Zero examples are compressed to synthetic examples: ๐ ๐๐ฅ๐พ โ ๐บ ๐พ๐ฅ๐พ
โข ๐๐บ๐บ0 = ๐๐๐0, but needs only ๐ steps to compute: ๐ ๐ข๐ =
๐บ๐ฎ๐ฌ ๐+๐๐๐ธ+
๐บ๐ฎ๐ฎ ๐+๐๐๐+
( ๐ฒ + #๐ท(๐)+ steps)
โข ๐๐บ๐ธ0 is calculated the same way as ๐๐๐ธ0, but using ๐ steps only.
โข Complexity: ๐ฐ ๐ผ๐พ(๐ธ + ๐พ๐ + ๐พ๐)) = ๐ถ ๐ฐ๐ฒ(๐ฌ + ๐ฒ(๐ด + ๐ต)
41 LAWA Workpackage Meeting 16th January, 2013
42. Optimizer โ Coordinate Descent
โข Complexity of naive solution: ๐ถ ๐ฐ๐ฒ๐ต๐ด
โข Ridge Regression calculates the features based on examples directly,
Covariance precomputing solution cannot be applied here.
โข Synthetic Examples
โข Codename: IALS1
โข Complexity reduction (IALS๏ IALS1)
๏ ๐ช ๐ฐ๐ฒ(๐ฌ + ๐ฒ(๐ด + ๐ต)
โข IALS1 requires higher ๐ฒ for the same accuracy as IALS.
42 LAWA Workpackage Meeting 16th January, 2013
43. Optimizer โ Coordinate Descent
...does it work in practice?
16th January, 201343 LAWA Workpackage Meeting
44. โข Average Rank Position on the subset of a propietary implicit feedback dataset. The lower
value is better.
โข IALS1 offers better time-accuracy tradeoffs, especially when K is large.
Comparison
44 LAWA Workpackage Meeting 16th January, 2013
IALS IALS1
K ARP time ARP time
5 0,1903 153 0,1898 112
10 0,1578 254 0,1588 134
20 0,1427 644 0,1432 209
50 0,1334 2862 0,1344 525
100 0,1314 11441 0,1325 1361
250 0,1311 92944 0,1312 6651
500 N/A N/A 0,1282 24697
1000 N/A N/A 0,1242 104611
0,120
0,125
0,130
0,135
0,140
0,145
0,150
0,155
100 1000 10000 100000
ARP
Training Time (s)
IALS IALS1
45. Conclusion
โข Explicit feedbacks are rarely or not provided.
โข Implicit feedbacks are more general.
โข Complexity issues of Alternating Least Squares.
โข Efficient solution by using approximation and synthetic examples.
โข IALS1 offers better time-accuracy tradeoffs, especially when ๐ฒ is large.
โข IALS is approximation algorithm too, so why not change it to be even
more approximative?
45 LAWA Workpackage Meeting 16th January, 2013
56. Related Publications
โข Alternating Least Squares with Coordinate Descent
I. Pilรกszy, D. Zibriczky, D. Tikk. Fast ALS-based matrix factorization for explicit and
implicit feedback datasets. RecSys 2010
โข Tensor Factorization
B. Hidasi, D. Tikk: Fast ALS-Based Tensor Factorization for Context-Aware
Recommendation from Implicit Feedback, ECML PKDD 2012
โข Personalized Ranking
G. Takรกcs, D. Tikk: Alternating least squares for personalized ranking, RecSys 2012
โข IPTV Case Study
D. Zibriczky, B. Hidasi, Z. Petres, D. Tikk: Personalized recommendation of linear content
on interactive TV platforms: beating the cold start and noisy implicit user feedback,
TVMMP @ UMAP 2012
56 LAWA Workpackage Meeting 16th January, 2013