Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Neo4j GraphTour New York_ Adobe Presentation_David Fox

Presentation for Neo4j GraphTour New York held on October 16, 2019 - How Neo4j Sparked a New Wave, David Fox, Adobe

  • Login to see the comments

  • Be the first to like this

Neo4j GraphTour New York_ Adobe Presentation_David Fox

  1. 1. How Neo4j Sparked a New Wave of Innovation at Adobe BehanceDavid Fox | Senior Software Engineer
  2. 2. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Behance  Leading online platform to showcase and discover creative work  More than 10 million members  Allows creatives to share their work with millions of daily visitors  Highlights Adobe software used in the creation process  Drives people to the Adobe Creative Cloud  Social platform for discovery, learning, and more 2
  3. 3. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Behance 3 Behance Logged-in Home Page/Activity Feed (2018)
  4. 4. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Introduced Neo4j to our Tech Stack  Replaced bloated/difficult to manage Cassandra architecture 4
  5. 5. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Activity Feed Cassandra Instances (48) 5
  6. 6. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Activity Feed Neo4j Instances (3) 6
  7. 7. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Introduced Neo4j to our Tech Stack cont.  Eliminated fanouts from Cassandra model  50tb of data to 50gb  Cost savings  Better performance for user-facing feature  More content for users to consume  Cassandra model was very rigid, Neo4j graph model is flexible  Big success, but more use-cases weren’t clear for us 7
  8. 8. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Shortly after we moved to Neo4j… 8
  9. 9. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Basic following model 9
  10. 10. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Very easy to filter by followee type 10
  11. 11. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Basic unfiltered activity fetch MATCH (me:User{id:123})-[:FOLLOWS]->(something)-[r:DID_ACTION]- >(project:Project) RETURN project, r.type ORDER BY r.created_on DESC LIMIT 50 11
  12. 12. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Only show newly published projects from people I follow MATCH (me:User{id:123})-[:FOLLOWS]->(someone:User)-[r:DID_ACTION]- >(project:Project) WHERE r.type = “published” RETURN project ORDER BY r.created_on DESC LIMIT 50 12
  13. 13. © 2019 Adobe. All Rights Reserved. Adobe Confidential. OK… Cypher/graph model is flexible, so what?  Not just flexible; flexible at scale and in production  Encourages innovation and experimentation  Do more with the data you already have in DB  Cassandra – no new activity-feed-based features/improvements for 1 year+  Shows product team we can develop + iterate quickly  Slow-to-iterate/rigid db structures can become a mindset + culture? 13
  14. 14. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Introduced WIPs (work in progress) (2018) 14
  15. 15. © 2019 Adobe. All Rights Reserved. Adobe Confidential. 15
  16. 16. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Users can react to WIPs 16
  17. 17. © 2019 Adobe. All Rights Reserved. Adobe Confidential. WIP Recommendations - Challenges  Ephemeral content (WIPs only last for 24 hours) means pipeline has to be very short  Couldn’t use usual tools like Spark, periodic model re-calculation, etc.  Needed ability to calculate recommended WIPs in near-realtime  Building Spark or other pipeline would take months with uncertain results  Working with remote Adobe research team  Developed Spark recommendation pipeline for previous project  No prior experience with graph databases or Neo4j 17
  18. 18. © 2019 Adobe. All Rights Reserved. Adobe Confidential. WIP Recommendations – Neo4j  Flexible graph db model already had data we needed  Cypher is pretty intuitive – research team could quickly learn  Very quick prototyping, just change the Cypher query  check if what you’re spending time on is working!  Powerful: 61 line Cypher query algorithm for prod algorithm  Still fast enough to run on-demand 18
  19. 19. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Just about a month for full product dev/rollout… (early 2019) 19
  20. 20. © 2019 Adobe. All Rights Reserved. Adobe Confidential. 20
  21. 21. © 2019 Adobe. All Rights Reserved. Adobe Confidential. 21
  22. 22. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Success Metric - Reactions 22 +20%!
  23. 23. © 2019 Adobe. All Rights Reserved. Adobe Confidential. WIP Recommendations - Summary  Collaboration with algorithm-heavy with no Neo4j experience  Rapid prototyping allowed product team review of progress/quality  No pipeline = quick to production, easy rollout  Neo4j allowed us to get the feature in front of users quickly  20% increase in WIP reactions (engagement metric)  Most complex algo/pipeline might not be best solution 23
  24. 24. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Returning to feed, our main experience 24
  25. 25. © 2019 Adobe. All Rights Reserved. Adobe Confidential. More features, but became a bit fragmented (For You feed, 2018-2019) 25
  26. 26. © 2019 Adobe. All Rights Reserved. Adobe Confidential. 26
  27. 27. © 2019 Adobe. All Rights Reserved. Adobe Confidential. 27
  28. 28. © 2019 Adobe. All Rights Reserved. Adobe Confidential. What if we… put it all together into one feed??  Status quo was any large-scale data view change was hard  required large migrations  extensive load-testing  new database tables, etc.  Product team confident we could deliver without lots of work/time based on ease of past Neo4j features/iterations  Consistent, infinitely-scrollable API with user activity, curated gallery activity, and recommendations  Consumable by our web and mobile apps 28
  29. 29. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Building the smart For You Feed experience with Neo4j (summer 2019)  Main goals:  Bring more exposure to newly published projects from people you follow  Ability to quickly iterate on the feed across all our apps  Ability to strategically prioritize different types of feed content  Future flexibility  Relatively simple to build because data already in Neo4j  Robustness of Cypher/graph model allow for querying data in this very detailed/custom ways  Performance very important – one of most frequently accessed pages on Behance  Hard to cache 29
  30. 30. © 2019 Adobe. All Rights Reserved. Adobe Confidential. New projects from people you follow + followed gallery projects on web 30
  31. 31. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Projects from moodboards you follow on web 31
  32. 32. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Appreciations from people you follow on web 32
  33. 33. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Recommendations on web 33
  34. 34. © 2019 Adobe. All Rights Reserved. Adobe Confidential. New projects from people you follow + followed gallery projects on iOS 34
  35. 35. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Appreciated by people you follow on iOS 35
  36. 36. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Smart For You Feed – success metric review  10% increase in projects viewed overall  240% increase in views of new projects from people you follow 36
  37. 37. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Smart For You Feed - Summary  Success through experimentation and flexibility  Core to our business, hard metrics to move easily  Would’ve been difficult, time-consuming, and extremely hard to iterate with old system  Future flexibility  Deprioritize already-seen items?  Dynamically shift content prioritization based on user tastes/behavior?  Product team already thinking about these, knowing we can execute quickly 37
  38. 38. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Applying this innovation structure elsewhere  Try to be open to using graph in prod for use-cases that make sense  Take a lot of care when you develop your data model  Take advantage of ease of creating proof-of-concepts in Neo4j to sell it at company  Think how you can turn complex pipelines into smaller, simpler Cypher queries that reduce app complexity  You may be surprised how much is runnable on-demand (per request) even at scale 38
  39. 39. © 2019 Adobe. All Rights Reserved. Adobe Confidential. The real value  Product team can focus on innovating instead of thinking about what can’t easily be done  More complex/slightly more effective algos/pipelines aren’t always better  Value in being able to quickly iterate and change entire user-experiences by just changing Cypher queries  Easier to make measurable changes  Granular A/B tests  Use-cases and innovation can spread beyond initial feature  Other team members pitching ideas, ex. user-facing stats overhaul with Neo4j 39
  40. 40. © 2019 Adobe. All Rights Reserved. Adobe Confidential. Thanks for coming! Please let me know if you have any questions! Contact: David Fox dafox@adobe.com 40

×