With the explosive growth of online information, recommender system has been an effective tool to overcome information overload and promote sales. In recent years, deep learning's revolutionary advances in speech recognition, image analysis and natural language processing have gained significant attention. Meanwhile, recent studies also demonstrate its efficacy in coping with information retrieval and recommendation tasks. Applying deep learning techniques into recommender system has been gaining momentum due to its state-of-the-art performance. In this talk, I will present recent development of deep learning based recommender models and highlight some future challenges and open issues of this research field.

1. Deep Learning for Recommender
Systems
Presenter: Shuai Zhang, PhD student, CSE, UNSW
Email: shuai.zhang@student.unsw.edu.au

2. Biography
• Oct 2016 – Present, PhD Student at School of Computer
Science and Engineering of UNSW, and Data61, CSIRO
• 2010 – 2014, Bachelor degree from
• 2014 – 2016, Software Engineer,
• http://www.cse.unsw.edu.au/~z5122282/
Lina Yao, UNSW
Xiwei Xu, Data61, CSIRO
Liming Zhu, Data61, CSIRO
Recommender Systems
Deep Learning
Internet of Things

3. Content
• Introduction to Recommender Systems
• Overview of Deep Learning based Recommender Systems
• State-of-the-art Models
• Future Research Directions
• My PhD Research Topic

4. Content
Deep Learning based Recommender Systems: A Survey
and New Perspectives
• Shuai Zhang, University of New South Wales
• Lina Yao, University of New South Wales
• Aixin Sun, Nanyang Technological University

5. Introduction to Recommender Systems
Recommender Systems are software tools and techniques that
provide suggestions for items that are most likely of interest to
a particular user [9]
Promote Sales
Overcome
Information Overload
Business
Customer

6. Introduction to Recommender Systems
To find relevant things for a user based on some kind of
feedback: explicit feedback (ratings) or implicit feedback
(interaction)
Products
Content
Services
Offer
etc.
User
Preferences
Interests
Intentions
etc.
What is relevant

7. Introduction to Recommender Systems
30 percent of Amazon.com's page views
were from recommendations, 2015
80 percent of movies watched on Netflix
came through recommendations, 2016
60 percent of video clicks came from
homepage recommendations, 2010

8. Overview
Deep learning has achieved great success in many application
domains
• Computer Vision
• Speech Recognition
• Natural Language
Processing

9. Overview
Can deep learning techniques solve recommendation
problems?

10. Overview
In recent years, there has been a surge of interest in applying
deep learning techniques to recommender systems
• DLRS, workshop of Recsys
• Google, Yahoo, Hulu, Alibaba
Deep learning based models achieved the best performances
and is a promising tool for recommender problems

11. Overview
• The number of research publications on deep learning based
recommendation models has increased exponentially in these
years.
Deep learning has
driven a
remarkable
revolution in
recommender
applications

12. Overview: Advantages
Advantages of employing deep learning techniques for
recommendations:
 Representation Learning
• Structural features: genres
• Text: storyline
• Images: posters of movies
• Audio: audio signals of music
• Video

13. Overview: Advantages
Advantages of employing deep learning techniques for
recommendations:
 Non-linearity
• Sigmoid
• Tanh
• Rectified linear unit

14. Overview: Advantages
Advantages of employing deep learning techniques for
recommendations:
 Generalization
Many classical model can be extended with neural network to
more generalized models
• Neural Collaborative Filtering [6]
• Neural Factorization Machine [12]

15. Overview: Advantages
Advantages of employing deep learning techniques for
recommendations:
 Make use of the GPU Computing Resources
• Involves a lot of matrix processing
• GPU is better at intensive computing

16. Overview
• More than 100 research papers
• Different tasks: rating prediction, top-n recommendations,
cross-domain recommendation, session-based
recommendation
• Different deep learning techniques: MLP, CNN, RNN,
Autoencoder, etc
• To help us better understand deep learning based
recommender systems: Current status, Future trends, Open
issues

17. Overview: Categories
Two-dimensional Classification Scheme for deep learning based
recommender systems
• Neural Network Model: Classify the existing studies in
accordance with the types of employed deep learning
techniques
• Integration Model: Whether it integrates traditional
recommendation models (neighbourhood model, Matrix
factorization, factorization machine, etc.) with deep learning or
relies solely on deep learning

18. Overview: Categories
Neural
Autoregressive
Distribution
Estimation
Generative
Adversarial
Network

19. Overview: Categories
Model based on single deep learning technique, eight subcategories:
1. Multilayer Perceptron(MLP): scalable, and can easily introduce
non-linearity for recommender systems;
Neural Collaborative Filtering
Wide & Deep Learning
Deep Factorization Machine
2. Autoencoder(AE): learn salient feature representations;
AutoSVD++ (by myself, SIGIR 2017): learn low dimensional
feature representations to enhance recommendation accuracy
AutoRec

20. Overview: Categories
3. Convolutional Neural Network(CNN): extract local and global
representations heterogenous data sources;
Extracting features from images
Audio (Music Recommendations)
Texts
4. Recurrent Neural Network(RNN):temporal dynamics and
sequential influences;
Session-based Recommendation
Recurrent Recommender Networks

21. Overview: Categories
5. Deep Semantic Similarity Model(DSSM): perform semantic
matching between users and items;
Tag-aware Recommendation
Cross-domain Recommendation
6. Restricted Boltzmann Machine(RBM): was first applied to
recommendation tasks;
7. Neural Autoregressive Distribution Estimation: it is a tractable
and efficient estimator for modelling data distributions;
8. Generative Adversarial Network: combine discriminative and
generative models together.
IRGAN (web search, item recommendation, question
answering)

22. Overview: Categories
Deep Composite Models: deep learning techniques can
complement one another and enable a powerful hybrid model.
Existing deep composite models
Every deep composite model should be carefully designed and
suitable for specific tasks

23. Overview: Categories
• Integrate Deep Learning with Traditional Recommendation Model
(matrix factorization, probabilistic matrix factorization,
factorization machine); Based on how tightly the two approaches
are integrated.
Loosely Coupled Model: Learn parameters of deep learning
and conventional recommendation algorithms separately
Tightly Coupled Model: Learning process are done
simultaneously
• Recommend Rely Solely on Deep Learning
• Without any forms of help from traditional recommendation
models

24. State-of-the-art Models: NCF
Neural Collaborative Filtering
WWW 2017
Xiangnan He, National University of Singapore
Lizi Liao, National University of Singapore
Hanwang Zhang, Columbia University
Liqiang Nie, Shandong University
Xia Hu, Texas A&M University
Tat-Seng Chua, National University of Singapore

25. State-of-the-art Models: NCF
𝑦 𝑢𝑖
^
= 𝑓𝑜𝑢𝑡 𝑓𝑋 … 𝑓2 𝑓1 𝑃 𝑇
𝑖𝑑 𝑢, 𝑄 𝑇
𝑖𝑑𝑖 …
𝑓𝑜𝑢𝑡, 𝑓𝑋: mapping function for output layer and x-th neural CF layer
Explicit feedback:
Implicit feedback:

26. State-of-the-art Models: NCF
𝑦 𝑢𝑖
^
= 𝑓𝑜𝑢𝑡(𝑓1 𝑃 𝑇
𝑖𝑑 𝑢 ⊙ 𝑄 𝑇
𝑖𝑑𝑖 )
𝑓1 𝑎𝑛𝑑 𝑓𝑜𝑢𝑡(): are identity mapping functions
Matrix Factorization:
𝑦 𝑢𝑖
^
= 𝑃𝑢
𝑇
⊙ 𝑄𝑖
𝑇
Squared Loss:
𝐿 =
𝑢,𝑖 ∈𝑌
𝑦 𝑢𝑖 − 𝑦 𝑢𝑖
^ 2

27. State-of-the-art Models: NCF
Matrix
Factorization
Multilayer
Perceptron

28. State-of-the-art Models: AutoSVD++
AutoSVD++: an efficient hybrid collaborative filtering model
via contractive auto-encoders
SIGIR 2017
Shuai Zhang, University of New South Wales
Lina Yao, University of New South Wales
Xiwei Xu, Data61, CSIRO

29. State-of-the-art Models: AutoSVD++
we propose utilizing Contractive
Autoencoder (CAE) to extract
salient feature representations,
and then integrating them into
traditional matrix factorization
model, SVD and SVD++.
CAE + SVD  AutoSVD
CAE + SVD++  AutoSVD++ Illustration of AutoSVD (remove the
implicit feedback) and AutoSVD++.

30. • Our model leverages latent factor model and auto-encoder in a
coupled manner with high scalability. The proposed efficient
AutoSVD++ algorithm significantly improves the computational
efficiency.
• By integrating the Contractive Auto-encoder (CAE), our model
catches the non-trivial and non-linear characteristics from item
content information. Compared to other autoencoder variants,
contractive autoencoder captures the infinitesimal input variations.
• Our model effectively makes use of implicit feedback to further
improve the accuracy.
State-of-the-art Models: AutoSVD++

31. Contractive Autoencoder
• 𝐽𝑐𝑎𝑒 𝜃 = 𝑥 ∈𝐷 𝑛
(𝐿(𝑥, 𝑔 𝑓 𝑥 ) + 𝜆 ∥ 𝐽 𝑓 𝑥 ∥ 𝐹
2
)
• Reconstruction: 𝑔 𝑓 𝑥 = 𝑠𝑔 𝑊′ 𝑠𝑓 𝑊𝑥 + 𝑏ℎ + 𝑏 𝑦
• Bottleneck layer: 𝑐𝑎𝑒 𝐶𝑖 = 𝑠𝑓 (𝑊 ⋅ 𝐶𝑖 +𝑏ℎ)
Prediction Rule of AutoSVD
• 𝑟𝑢𝑖 = 𝜇 + 𝑏 𝑢 + 𝑏𝑖 + 𝛽 ⋅ 𝑐𝑎𝑒 𝐶𝑖 + 𝜖𝑖
𝑇 𝑈 𝑢
Prediction Rule of AutoSVD++
• 𝑟𝑢𝑖 = 𝜇 + 𝑏 𝑢 + 𝑏𝑖 +
• 𝛽 ⋅ 𝑐𝑎𝑒 𝐶𝑖 + 𝜖𝑖
𝑇
(𝑈 𝑢+|𝑁 𝑢 |−
1
2
𝑗 ∈𝑁(𝑢) 𝑦𝑗)
State-of-the-art Models: AutoSVD++

32. Idea: group the
data of same users
who share same
implicit feedback
information.
State-of-the-art Models: AutoSVD++

33. Deep Collaborative Filtering Framework [16]
State-of-the-art Models: Extensions

34. Collaborative Deep Learning for Recommender System [15]
State-of-the-art Models: Extensions

35. Convolutional Matrix
Factorization [17]
State-of-the-art Models: Extensions

36. State-of-the-art Models: Neural Rating Tips
Neural Rating Regression with abstractive Tips
Generation for Recommendation [14]
Deep Composite Model : MLP
and RNN
Multitask Framework
• Main Task: Rating prediction
with Multilayer perceptron
• Auxiliary Task: Tips Generation

37. State-of-the-art Models: Neural Rating Tips
Examples of the predicted
ratings and the generated
tips. The first line of each
group shows the generated
rating and tips. The second
line shows the ground truth.

38. Future Research Directions

39. Future Research Directions: Feature Learning
Get a deep understanding of users
and items from various side
information with deep learning
• Video
• Audio
• Image
• User’s footprints
• Etc…

40. Future Research Directions: Temporal Dynamics
• Timestamp
• Changes of Item and User preferences
• Session based Recommendation
 Tracking user’s long term interaction is inapplicable for
many applications and mobile apps
 Short term session can be usually collected and utilized

41. Future Research Directions: Multitask Learning
• Learning several tasks at a time can
prevent overfitting
• Auxiliary task provides interpretable
output for explaining the recommendation
• Alleviating the sparsity problem implicitly
Neural Rating Tips

42. Future Research Directions: Cross Domain
• User’s preferences on one domain will
influence her preferences on other
domains somehow
• Many companies offer diversified
products or services
• Deep learning is well suited to transfer
learning as it can learn high-level
abstractions that disentangle the
variation of different domains

43. My PhD Research Topic
Intention-aware Recommender Systems
User’s intention inferred from other data sources can help
improve the recommendation quality
What is intention?
Intention  Intend to do something (a determination to act in a
certain way)  Demands / Needs  Influence later action
somehow, e.g. consumption

44. My PhD Research Topic
Twitter users you followed
Meetup
groups
Books in
Amazon

45. My PhD Research Topic
What is Intention Mining (Intention Modeling) ?
• Intention Mining is the problem of determining a user’s intention
from their activities [4]
• It has been the key research issue for providing personalized
experiences and services [5]
Application Fields:
• Web Search
• Software Engineering, etc.

46. My PhD Research Topic

47. My PhD Research Topic
Infer user’s interests (long-lasting intention) from Twitter
• Users she follows
• Twitter lists
• Descriptions
Recommend Special interest groups
• Meetup Groups

48. References
[1] Zhang, Shuai, Lina Yao, and Aixin Sun. "Deep Learning based Recommender System: A Survey and New
Perspectives." arXiv preprint arXiv:1707.07435 (2017).
[2] http://shuaizhang.tech/2017/03/13/Papers-Deep-Learning-for-Recommender-System/
[3] Li, Piji, et al. "Neural Rating Regression with Abstractive Tips Generation for Recommendation." (2017).
[4] Khodabandelou, Ghazaleh, et al. "Supervised intentional process models discovery using hidden markov
models." Research Challenges in Information Science (RCIS), 2013 IEEE Seventh International Conference on.
IEEE, 2013.
[5] Chen, Zheng, et al. "User intention modeling in web applications using data mining." World Wide Web 5.3
(2002): 181-191.
[6] He, Xiangnan, et al. "Neural collaborative filtering." Proceedings of the 26th International Conference on
World Wide Web. International World Wide Web Conferences Steering Committee, 2017.
[7] Zhang, Shuai, Lina Yao, and Xiwei Xu. "AutoSVD++: An Efficient Hybrid Collaborative Filtering Model via
Contractive Auto-encoders." arXiv preprint arXiv:1704.00551 (2017).
[8] Zhang, Shuai, and Lina Yao. "Dynamic Intention-Aware Recommendation System." arXiv preprint
arXiv:1703.03112(2017).
[9] Ricci, Francesco, Lior Rokach, and Bracha Shapira. "Recommender systems: introduction and
challenges." Recommender systems handbook. Springer US, 2015. 1-34.
[10] https://www.slideshare.net/frederickayala/sessionbased-recommender-systems-75268222

49. References
[11] https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-
learning-ai/
[12] He, Xiangnan, and Tat-Seng Chua. "Neural Factorization Machines for Sparse Predictive Analytics." (2017).
[13] http://www.nvidia.com/object/what-is-gpu-computing.html
[14] Li, Piji, et al. "Neural Rating Regression with Abstractive Tips Generation for Recommendation." (2017).
[15] Wang, Hao, Naiyan Wang, and Dit-Yan Yeung. "Collaborative deep learning for recommender
systems." Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining. ACM, 2015.
[16] Li, Sheng, Jaya Kawale, and Yun Fu. "Deep collaborative filtering via marginalized denoising auto-
encoder." Proceedings of the 24th ACM International on Conference on Information and Knowledge
Management. ACM, 2015.
[17] Kim, Donghyun, et al. "Convolutional matrix factorization for document context-aware
recommendation." Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016.

50. Thank you
Q & A