Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

FD recommendation engine in personalized newsletters

Download to read offline

Our work on the first personalized newsletter. Recommendation engine is built using collaborative filtering.

  • Be the first to like this

FD recommendation engine in personalized newsletters

  1. 1. FD recommendation system Dung Chu, data scientist FD Mediagroep
  2. 2. Outline • Business cases analysis and recommendation engine roadmap • Personalized email campaigns • A/B testing results • Upcoming work
  3. 3. About me • Dung Chu • MSc in cloud computing @UvA • Research in image processing group ISLA @UvA: https://ivi.fnwi.uva.nl/isis/index.html • Data scientist @FD mediagroep
  4. 4. FD mediagroep Het Financieele Dagblad Het Financieele Dagblad is dé nieuws- en inspiratiebron die op elk moment van de dag financieel-economische betekenis geeft aan ontwikkelingen in de wereld. Company info geeft altijd real time toegang tot actuele bedrijfs- en prospectinformatie van 2,5 miljoen bedrijven in Nederland FD/BNR Networks FD/BNR Networks brengt ideeën, meningen en talentrijke ondernemers en professionals samen door de organisatie van o.a. forums, debatten, netwerksessies, en evenementenreeksen. BNR Nieuwsradio BNR Nieuwsradio is de enige radiozender in Nederland waar ambitieuze en ondernemende mensen 24 uur per dag op de hoogte worden gehouden van relevant nieuws.
  5. 5. Pensioen Pro De joint venture tussen FD Pensioen Pro en IPN en levert betrouwbaar nieuws, achtergrond en analyse voor de pensioenprofessional. Redactie Partners Produceert custom media producten voor zowel interne als externe communicatievraagstukken Fondsnieuws Het grootste journalistieke platform voor beleggingsprofessionals in Nederland. Energeia Biedt nieuws, data en opinie voor energieprofessionals.
  6. 6. FD.nl
  7. 7. Publishing rate 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Number of articles Nr articles Per week: ~ 500 articles Digital- first
  8. 8. FD traffic 2/1/17 2/2/17 2/3/17 2/4/17 2/5/17 2/6/17 2/7/17 2/8/17 2/9/17 2/10/17 2/11/17 2/12/17 2/13/17 2/14/17 2/15/17 2/16/17 2/17/17 2/18/17 2/19/17 2/20/17 2/21/17 2/22/17 2/23/17 2/24/17 2/25/17 2/26/17 2/27/17 2/28/17 FD traffic - february 2017 - daily Users Pageviews
  9. 9. Business cases analysis • Improve customer retention through direct digital activation with increased relevance – Online traffic – Email traffic • Improve conversion via more digital engagement – Convert un-registered to registered customers • Increase ad sales via increased traffic All users Logged-in users
  10. 10. Business cases analysis • Support journalists in understanding customer reading tastes: – Provide more article insights to journalists Choice: internal development of recommendation systems* * We did try a pilot phase with a third party
  11. 11. Where to start • Online recommendation in website fd.nl • Email newsletters
  12. 12. Where to start • Online recommendation in website fd.nl • Email newsletters
  13. 13. Current email newsletters
  14. 14. Current email newsletters Popularity sorting
  15. 15. Personalized email newsletters • Targeted audience was registered customers without subscriptions: – Free customers with 5 articles as free – We wanted to improve conversion rates
  16. 16. Personalization: collaborative filtering • Popularity sorting: no relation between articles • Relations between articles A and B: if someone is reading A, what is the prob that he/she finds B also interesting. • Relation: – Content-based relation: – Relation based on collaborative behavior of reading:
  17. 17. Architecture Collect data Article distance matrix Recommend PySpark: + Convert reader-article matrix to IndexedRowMatrix + columnSimilarities(): fast compute of cosine similarity between columns + Input: klant_nr + Infer list of articles read from reader-article matrix + Recommended articles: articles with largest sum of distances to the articles read by the user Article DB Reader-article matrix Email campaign management Client list Emailing system Published last week Older articles 28 days 7 days
  18. 18. Recommendation step
  19. 19. Item-to-item collaborative filtering • Always sort articles based on cosine metric • Cold-start problems: with articles that were read only few times – Evaluate dot-product metric – Use popularity version
  20. 20. Personalized emails
  21. 21. A/B testing • A setting (“combi”): recommended articles using item-to-item collaborative filtering • B setting (“popularity”): recommended articles using article popularity scores
  22. 22. Results: free clients T-test
  23. 23. Results: paid clients (fixed 2000) T-test
  24. 24. Number of articles sent – 1 week Popularity emails Personalized emails 6 articles 178 articles (60 clicked) Long tail articles
  25. 25. Toolings
  26. 26. Future work
  27. 27. Take-home messages • Simple recommendation model (with existing tools) works • FD Mediagroep has started with AI. And we are doing more and more in this journey.
  28. 28. Twitter: @dungchu Email: dung.manh.chu@fdmediagroep.nl LinkedIn: https://www.linkedin.com/in/dungmanhchu/

Our work on the first personalized newsletter. Recommendation engine is built using collaborative filtering.

Views

Total views

285

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

8

Shares

0

Comments

0

Likes

0

×