Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
RECOMMENDATION SYSTEMS 

AND MACHINE LEARNING 

Mapping the User Experience

LUIZ AGNER | BARBARA NECYK | ADRIANO RENZI
Ri...
• Making a choice is a challenging and
complicated task because of the amount 

of information.

• Machines are more effic...
• Employed in:

• Netflix, 

• YouTube, 

• Spotify, 

• Amazon, 

• Facebook,

RECOMMENDATION
SYSTEMS
•Linkedin, 

•Twitt...
RECOMMENDATION
SYSTEMS
• Prediction 

data is used to predict the evaluation 

a user will give to an item.

• Ranking 

d...
RECOMMENDATION
SYSTEMS
• Relevance

• Novelty

• Serendipity

• Diversity
GOALS
Pandey, P.: The Remarkable world of Recomm...
• Methods are based on past interactions 

among users and items.

• Can recommend complex items without 

an understandin...
• Need additional information about 

users and items. 

• Need a description of each item 

and a profile of user prefere...
Netflix is an excellent example of hybrid
recommendation systems because its
recommendations are based not only on 

brows...
• ML is the science of helping computers discover 

patterns and relationships in data.

• UX designers have a lot to lear...
• Algorithms are not transparent to users. 

• Users don’t identify actions and do 

not have a clear understanding of the...
• Users perceive the system as a “black box”.

With mysterious inputs and outputs out of control. 

• Users need developin...
USER EXPERIENCE
GUIDELINES
• Transparency

• Easy controls

• Don’t repeat items
Budiu, R.: Can Users Control and Understa...
MORE UX
GUIDELINES
• Transparency of data source

• Improvement of recommendations by users

• Response time
Harley, A.: U...
HUMAN CENTRED AI
GUIDELINES
• Microsoft: Guidelines for Human-AI Interaction.

• Comprehensibility and controllability.

A...
• Personalised Video Ranker (PVR)

• Top-N Video Ranker 

• Trending Now 

• Video-Video Similarity (Sims)

• Continue Wat...
SEMI-STRUCTURED
INTERVIEWS
• What influences users to find new video.

• How users identify where their recommendations ar...
USERS ANSWERS
USERS ANSWERS
“Conspiracy Theories”
• Human-algorithm interaction emerges as a new frontier of research.

• Machine learning algorithms must be transparent. 
...
• Fears of political surveillance, restriction of the 

information freedom, and credit card hacking. 

• “Conspiracy theo...
Pandey, P.: The Remarkable world of Recommender Systems. https://towardsdatascience.com/the-
remarkable-world-of-recommend...
THANK YOU.
luizagner@gmail.com
Upcoming SlideShare
Loading in …5
×

Recommendation Systems and Machine Learning: Mapping the User Experience

“Recommendation Systems and Machine Learning: Mapping the User Experience” – paper by Luiz Agner, Barbara Necyk and Adriano Renzi.
Apresentação de pesquisa no Congresso Human-Computer Interaction International 2020 – Copenhagen, na sessão Ergonomics in Design, com a coordenação de sessão pelo prof. Marcelo Soares.

  • Be the first to comment

  • Be the first to like this

Recommendation Systems and Machine Learning: Mapping the User Experience

  1. 1. RECOMMENDATION SYSTEMS 
 AND MACHINE LEARNING 
 Mapping the User Experience
 LUIZ AGNER | BARBARA NECYK | ADRIANO RENZI Rio de Janeiro
  2. 2. • Making a choice is a challenging and complicated task because of the amount 
 of information. • Machines are more efficient than human beings in managing data and in learning from this data. INFORMATION OVERLOAD Gomez-Uribe; Hunt: The Netflix recommender system: Algorithms, business value, and innovation.
  3. 3. • Employed in: • Netflix, • YouTube, • Spotify, • Amazon, • Facebook, RECOMMENDATION SYSTEMS •Linkedin, •Twitter, •Trip Advisor, •Google News, •Others Rocca, B.: Introduction to recommender systems. Pandey, P.: The Remarkable world of Recommender Systems.
  4. 4. RECOMMENDATION SYSTEMS • Prediction 
 data is used to predict the evaluation 
 a user will give to an item. • Ranking 
 defines a list of items to be 
 presented to the user. PROBLEMS: Pandey, P.: The Remarkable world of Recommender Systems.
  5. 5. RECOMMENDATION SYSTEMS • Relevance • Novelty • Serendipity • Diversity GOALS Pandey, P.: The Remarkable world of Recommender Systems.
  6. 6. • Methods are based on past interactions 
 among users and items. • Can recommend complex items without 
 an understanding of its content. COLLABORATIVE FILTERING Rocca, B.: Introduction to recommender systems.
  7. 7. • Need additional information about 
 users and items. • Need a description of each item 
 and a profile of user preferences. CONTENT- BASED FILTERING Rocca, B.: Introduction to recommender systems.
  8. 8. Netflix is an excellent example of hybrid recommendation systems because its recommendations are based not only on 
 browsing habits but also on similar videos. HYBRID SYSTEMS Pandey, P.: The Remarkable world of Recommender Systems.
  9. 9. • ML is the science of helping computers discover 
 patterns and relationships in data. • UX designers have a lot to learn about ML 
 to help users feel in control of the technology. • Users can not make the systems do what they want. USER EXPERIENCE & MACHINE LEARNING Lovejoy, J., Holbrook, J.: Human-Centered Machine Learning.
  10. 10. • Algorithms are not transparent to users. • Users don’t identify actions and do 
 not have a clear understanding of the outputs. • Recommendations seem random, meaningless. • Algorithms group items according to obscure 
 and not mutually exclusive rules. USER EXPERIENCE & MACHINE LEARNING Budiu, R.: Can Users Control and Understand a UI Driven by Machine Learning?
  11. 11. • Users perceive the system as a “black box”.
 With mysterious inputs and outputs out of control. • Users need developing a clear mental model of how the system works. 
 The system needs to be clear about how users can change its results. • Repeated recommendations - and session-specific arts, 
 descriptions and headings - increase the cognitive cost of interaction. USER EXPERIENCE & MACHINE LEARNING Budiu, R.: Can Users Control and Understand a UI Driven by Machine Learning?
  12. 12. USER EXPERIENCE GUIDELINES • Transparency • Easy controls • Don’t repeat items Budiu, R.: Can Users Control and Understand a UI Driven by Machine Learning?
  13. 13. MORE UX GUIDELINES • Transparency of data source • Improvement of recommendations by users • Response time Harley, A.: UX Guidelines for Recommended Content. 

  14. 14. HUMAN CENTRED AI GUIDELINES • Microsoft: Guidelines for Human-AI Interaction. • Comprehensibility and controllability. Amershi, Horvitz, et al. Microsoft - Guidelines for human-AI interaction design
  15. 15. • Personalised Video Ranker (PVR) • Top-N Video Ranker • Trending Now • Video-Video Similarity (Sims) • Continue Watching • Evidence • Page Generation: Row Selection and Ranking • Search NETFLIX ALGORITHMS Gomez-Uribe; Hunt: The Netflix recommender system: Algorithms, business value, and innovation.
  16. 16. SEMI-STRUCTURED INTERVIEWS • What influences users to find new video. • How users identify where their recommendations are. • How users explain the way Netflix's algorithms work. • What users could do to improve their recommendations. • About data protection and privacy. Courage, C., Baxter, K.: Understanding Your Users: A Practical Guide…
  17. 17. USERS ANSWERS
  18. 18. USERS ANSWERS “Conspiracy Theories”
  19. 19. • Human-algorithm interaction emerges as a new frontier of research. • Machine learning algorithms must be transparent. • Users do not know which interactions create the recommendation lists. • Users have not formed a mental model about tracked data. • Designers must help users build mental models. • Designers must help users deconstruct the “black box” feeling. CONCLUSIONS & NOTES
  20. 20. • Fears of political surveillance, restriction of the 
 information freedom, and credit card hacking. • “Conspiracy theories” appeared. • UX designers should encourage users to record 
 feedbacks and edit user navigation data. • UX designers have a lot to learn. • UX designers have to reinforce user’s control. CONCLUSIONS & NOTES
  21. 21. Pandey, P.: The Remarkable world of Recommender Systems. https://towardsdatascience.com/the- remarkable-world-of-recommender-systems-bff4b9cbe6a7, last accessed 2020/01/28. 
 Gomez-Uribe, C.A.; Hunt, N.: The Netflix recommender system: Algorithms, business value, and innovation. In: ACM Transactions Manage. Inf. Syst. 6, 4, Article 13. ACM, New York (2015). 
 Rocca, B.: Introduction to recommender systems. https://towardsdatascience.com/introduction-to- recommender-systems-6c66cf15ada 1/22, last accessed 2019/12/19. 
 Budiu, R.: Can Users Control and Understand a UI Driven by Machine Learning? Nielsen Norman Group, https://www.nngroup.com/articles/machine-learning-ux/, last accessed 2020/01/02. 
 Harley, A.: Individualized Recommendations: Users’ Expectations & Assumptions. https:// www.nngroup.com/articles/recommendation-expectations/, last accessed 2019/11/28. 
 Harley, A.: UX Guidelines for Recommended Content. https://www.nngroup.com/articles/ recommendation-guidelines/, last accessed 2019/11/29. 
 Lovejoy,J.,Holbrook,J.: Human-Centered Machine Learning. https://medium.com/google- design/ human-centered-machine-learning-a770d10562cd, last accessed 2019/12/10. 
 Netflix Research.: Machine Learning: Learning how to entertain the world, https://re- search.netflix.com/research-area/machine-learning, last accessed 2019/12/28. 
 Basilico, J.: Recent Trends in Personalisation: a Netflix Perspective. ICML2019-Adaptative and Multi- Task Learning Workshop, https://slideslive.com/38917692/recent-trends-in- personalization-a-netflix- perspective, last accessed 2020/01/10. 
 Courage, C., Baxter, K.: Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques. Morgan Kaufmann, San Francisco (2005).
 REFERENCES
  22. 22. THANK YOU. luizagner@gmail.com

×