Neo4j in Production: A look at Neo4j in the Real World by Alexander Batanov, GraphTalk Atlanta

1. © Neo Technology
Neo4j in the Real World
GraphTalk Atlanta
Alexander Batanov

2. © Neo Technology
AGENDA
• Case Studies
• Quick Intro to Graphs
• Recommender Systems Advantage
Demo: recommendations in retail
• Moving Forward with Neo

3. © Neo Technology
The internet
is a graph
Huge networks of
connected data

4. © Neo Technology
“Graph analysis is possibly the single most effective
competitive differentiator for organizations pursuing
data-driven operations and decisions after the design
of data capture.
“By the end of 2018, 70% of leading organizations will
have one or more pilot or proof-of-concept efforts
underway utilizing graph databases.”
Towards Graph Inevitability

5. © Neo Technology
“Forrester estimates that over 25% of enterprises
will be using graph databases by 2017.”
Towards Graph
Inevitability

6. © Neo Technology
Case Studies

7. © Neo Technology
Real-Time
Recommendation
s
Fraud
Detection
Network &
IT
Operations
Master Data
Managemen
t
Graph-
Based
Search
Identity &
Access
Management
Over 200 customers, including some of the world’s largest
companies
What our customers do

8. © Neo Technology
Routing
Recommendations
Don’t Take Our Word For It
Examples of companies that use Neo4j, the world’s leading graph
database, for recommendation and personalization engines.
Adidas uses Neo4j to combine content and
product data into a single, searchable graph
database which is used to create a
personalized customer experience
“We have many different silos, many different
data domains, and in order to make sense out
of our data, we needed to bring those
together and make them useful for us,”
– Sokratis Kartelias, Adidas
eBay Now Tackles eCommerce Delivery
Service Routing with Neo4j
“We needed to rebuild when growth and new
features made our slowest query longer than
our fastest delivery - 15 minutes! Neo4j gave
us best solution”
– Volker Pacher, eBay
Walmart uses Neo4j to give customer best
web experience through relevant and
personal recommendations
“As the current market leader in graph
databases, and with enterprise features for
scalability and availability, Neo4j is the right
choice to meet our demands”.
- Marcos Vada, Walmart
Product
Recommendations
Personalization
Engines
Adidas

9. Neo4j in Action
Real-time Package Routing
• Large postal service with over
500k employees
• Neo4j routes 7M+ packages
daily at peak, with peaks of
5,000+ routing operations per
second.
Real-time promotion recommendations
• Record “Cyber Monday” sales
• About 35M daily transactions
• Each transaction is 3-22 hops
• Queries executed in 4ms or less
• Replaced IBM Websphere commerce
Real-time pricing engine
• 300M pricing operations per day
• 10x transaction throughput on half
the hardware compared to Oracle
• Presentation at
http://graphconnect.com/gc2016-sf/
• Replaced Oracle database

10. © Neo Technology
Case studySolving real-time recommendations for the
World’s largest retailer.
Challenge
• In its drive to provide the best web experience for its
customers, Walmart wanted to optimize its online
recommendations.
• Walmart recognized the challenge it faced in delivering
recommendations with traditional relational database
technology.
• Walmart uses Neo4j to quickly query customers’ past
purchases, as well as instantly capture any new
interests shown in the customers’ current online visit –
essential for making real-time recommendations.
Use of Neo4j
“As the current market leader in
graph databases, and with
enterprise features for scalability
and availability, Neo4j is the right
choice to meet our demands”.
- Marcos Vada, Walmart
• With Neo4j, Walmart could substitute a heavy batch
process with a simple and real-time graph database.
Result/Outcome

11. © Neo Technology
adidas Combining content and product data into Neo4j to create
personalized customer experience
Challenge
• Data was stored and managed in disparate silos,
preventing Adidas from getting a holistic view of
costumers
• On the technical level, data models didn’t align
between the information silos, and there wasn’t a
standard, consistent way to communicate between the
different data domains.
• Adidas uses Neo4j to combine content and product
data into a single, searchable graph database which is
used to create a personalized customer experience
• They created a meta-data repository that stored and
queried data-relationships in Neo4j, without having to
replace existing data-sources.
Use of Neo4j
• With a vast global audience, the adidas Group
significantly improved their ability to provide a more
personalized experience to its online shoppers.
• The Neo4j graph database proved to the be the ideal
technology for creating the Service, offering access
and searchability to all data, along with support for
new emerging services.
“We have many different silos, many
different data domains, and in order
to make sense out of our data, we
needed to bring those together and
make them useful for us,”
Result/Outcome
– Sokratis Kartelias
Case study

12. © Neo Technology
eBay Now Tackles eCommerce Delivery Service Routing
with Neo4j
Challenge
• The queries used to select the best courier for eBays
routing system were simply taking too long and they
needed a solution to maintain a competitive service.
• The MySQL joins being used created a code base too
slow and complex to maintain.
• eBay is now using Neo4j’s graph database platform to
redefine e-commerce, by making delivery of online and
mobile orders quick and convenient.
Use of Neo4j
• With Neo4j eBay managed to eliminate the biggest
roadblock between retailers and online shoppers: the
option to have your item delivered the same day.
• The schema-flexible nature of the database allowed
easy extensibility, speeding up development.
• Neo4j solution was more than 1000x faster than the
prior MySQL Soltution.
Our Neo4j solution is literally
thousands of times faster than the
prior MySQL solution, with queries
that require 10-100 times less code.
Result/Outcome
– Volker Pacher, eBay
Case study

13. © Neo Technology
Top Tier US
Retailer
Solving Real-time promotions for a top US
retailer
Challenge
• Suffered significant revenues loss, due to legacy
infrastructure.
• Particularly challenging when handling transaction
volumes on peak shopping occasions such as
Thanksgiving and Cyber Monday.
• Neo4j is used to revolutionize and reinvent its real-time
promotions engine.
• On an average Neo4j processes 90% of this retailer’s
35M+ daily transactions, each 3-22 hops, in 4ms or
less.
Use of Neo4j
• Reached an all time high in online revenues, due to
the Neo4j-based friction free solution.
• Neo4j also enabled the company to be one of the first
retailers to provide the same promotions across both
online and traditional retail channels.
“On an average Neo4j processes
90% of this retailer’s 35M+ daily
transactions, each 3-22 hops, in
4ms or less.”
– Top Tier US Retailer
Result/Outcome
Case study

14. © Neo Technology
Quick intro to graphs

15. © Neo Technology
Relational
Database

16. © Neo Technology
This is data modelled as graph!
Graph
Database

17. • Easy to model and store relationships
• Performance of relationship traversal remains constant with
growth in data size
• Queries are shortened and more readable
• Adding additional properties and relationships can be done on
the fly - no migrations
Graph Motivations

18. © Neo Technology
The labelled property graph
• Nodes
• Relationships
• Properties
• Labels
CAR
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
since:
Jan 10, 2011
brand: “Volvo”
model: “V70”
LOVES
LOVES
LIVES WITH
PERSON PERSON

19. © Neo Technology
CAR
Property Graph Model Components
Nodes
• Represent the objects in the graph
• Can be labeled
PERSON PERSON

20. © Neo Technology
CAR
Nodes
•Represent the objects in the
graph
•Can be labeled
Relationships
• Relate nodes by type and direction
Property Graph Model Components
LOVES
LOVES
LIVES WITH
PERSON PERSON

21. © Neo Technology
CAR
name: “Dan”
born: May 29, 1970
twitter: “@dan”
name: “Ann”
born: Dec 5, 1975
since:
Jan 10, 2011
brand: “Volvo”
model: “V70”
Property Graph Model Components
Nodes
•Represent the objects in the
graph
• Can be labeled
Relationships
•Relate nodes by type and
direction
Properties
• Name-value pairs that can go on
nodes and relationships.
LOVES
LOVES
LIVES WITH
PERSON PERSON

22. © Neo Technology
One of the top use cases for Neo4j
Recommender Systems Advantage

23. © Neo Technology
Good recommender systems increase revenue
http://fortune.com/2012/07/30/amazons-recommendation-secret/
http://www.digital-warriors.com/recommendation-engines/

24. © Neo Technology

25. © Neo Technology
Mobile Brick & Mortar
Retail Today
Web
Product Recommendations

26. © Neo Technology
Mobile Brick & Mortar
Web

27. © Neo Technology
Mobile Brick & Mortar
Web

28. © Neo Technology
Mobile Brick & Mortar
Web

29. © Neo Technology
Category Price ConfigurationsLocation
Silos & Polyglot Persistence
Purchase ViewReviewReturn In-store PurchasesInventory
Products Customers / Users
Location
Purchases
RELATIONAL DB WIDE COLUMN
STORE
Views
DOCUMENT
STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping
Cart
KEY VALUE
STORE
Product
Catalogue
DOCUMENT
STORE

30. © Neo Technology
Purchases
RELATIONAL DB WIDE COLUMN
STORE
Views
DOCUMENT
STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping
Cart
KEY VALUE
STORE
Product
Catalogue
DOCUMENT
STORE
Silos & Polyglot Persistence
Category Price ConfigurationsLocation Purchase ViewReviewReturn In-store PurchasesInventory
Products Customers / Users
Location

31. © Neo Technology
Purchases
RELATIONAL DB WIDE COLUMN
STORE
Views
DOCUMENT
STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping
Cart
KEY VALUE
STORE
Product
Catalogue
DOCUMENT
STORE
Polyglot Persistence
Category Price ConfigurationsLocation Purchase ViewReviewReturn In-store PurchasesInventory
Products Customers / Users
Location

32. © Neo Technology
Data
Lake
Purchases
RELATIONAL DB
Product
Catalogue
DOCUMENT
STORE
WIDE COLUMN
STORE
Views
DOCUMENT
STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping
Cart
KEY VALUE
STORE
Recommendations require an operational
workload — it’s in the moment, real-time!
Good for Analytics, BI, Map
Reduce
Non-Operational, Slow
Queries

33. © Neo Technology
Purchases
RELATIONAL DB
Product
Catalogue
DOCUMENT
STORE
WIDE COLUMN
STORE
Views
DOCUMENT
STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping
Cart
KEY VALUE
STORE
Connector
Drivers: Java | JavaScript | Python | .Net | PHP | Go | Ruby
Apps and Systems
Real-Time
Queries

34. © Neo Technology
Creating Relevance
in an Ocean of
Possibilities

35. © Neo Technology
Bad Recommendations

36. © Neo Technology
Goals of good Recommender systems
• Relevance
• Users are more likely to consume Items they find interesting
• Novelty
• Items they didn’t see/bought/viewed in the past (related)
• Serendipity
• Unexpected Items, a lucky discovery (unrelated)
• Increasing recommendation diversity
• Same type of items increases the risk of the User not liking any Item

37. © Neo Technology
Recommendations are Based on Context
• Customer Demographics (e.g. city neighborhood, income level)
• Shopping Basket and Purchase History
• Search History and Typeahead
• Seasonality - Time of the day / day of the week / season
• Product Reviews
• Social Media
• Sentiment Analysis
• External Data Sources: Weather, Stock Market, Census Data

38. © Neo Technology
Abstraction of a Recommendation System
Context
Ordered list of Items
(aka Recommendations)
Does it have to
be a black box?

39. © Neo Technology
Algorithms of Recommender Systems
• Collaborative Filtering
• Predicting person’s behavior based on the behavior of other people
• KNN Clustering, Cosine Similarity, Jaccard Index
• Content-based Filtering
• Matching customer profile to product characteristics
• Products frequently bought together
• Rule-Based Filtering
• Biggest margin, low on stock, promotions
• Hybrid Systems
• All of the above combined

40. © Neo Technology
Feedback
• Explicit
• Thumbs up, thumbs down, star rating
• Implicit
• user viewed an item
• user viewed item's details
• user added an item to cart
• user purchased an item
• user have read an article up to the end
• Exclude negative feedback Items

41. © Neo Technology
Demo

42. © Neo Technology
What will be demoed
• Simple cypher queries to implement a recommender system
• Loading data from multiple sources
• Products
• Customers
• Orders
• Reviews
• Inventory

43. © Neo Technology
Neo4j Recon

44. © Neo Technology
Recon
Tweak & Monitor
Context
Ordered list of Items
(aka Recommendations)

45. © Neo Technology
Scoring
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
Scorer Score
“Interested in Topic” 1.0
Total score Event 1.0

46. © Neo Technology
Scoring
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
[:HAS_TOPIC]
[:INTERESTED]
Topic
Scorer Score
“Interested in Topic” 1.5
Total score Event 1.5

47. © Neo Technology
Scoring
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
[:HAS_TOPIC]
[:INTERESTED]
User
[:RSVP]
[:FRIENDS]
Topic
Scorer Score
“Interested in Topic” 1.5
“Friend goes” 1.0
Total score Event 2.5

48. © Neo Technology
Scoring
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
[:HAS_TOPIC]
[:INTERESTED]
User
[:RSVP]
[:FRIENDS]
Topic
Scorer Score
“Interested in Topic” 1.5
“Friend goes” 1.0
Time Proximity 1.7
Total score Event 4.2
{Time}

49. © Neo Technology
Scoring + Feedback
Scorer Score Preference
User A
Score
User A
Preference
User B
Score
User B
“Interested in Topic” 1.5 x 1.5 2.25 x 0.5 0.75
“Friend goes” 1.0 x 0.5 0.5 x 1.5 1.5
Time Proximity 1.7 1.7 1.7
Total score Event 4.2 4.45 3.95

50. © Neo Technology
Discover
• Bad Recommendation
• Possible Recommendation
• Proximity: Farther away (less relevant)
EventUser
[:RSVP]
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
TopicUser Group Topic Group
[:INTERESTED] [:HAS_TOPIC] [:HAS_TOPIC] [:HAS_TOPIC]
Event
[:HOSTS_EVENT]

51. © Neo Technology
Discover
• Density: More Paths, more relevant
• Other Paths
TopicUser Group Event
[:INTERESTED] [:HAS_TOPIC] [:HOSTS_EVENT]
[:HAS_TOPIC][:INTERESTED]
EventUser
[:RSVP]
User
[:FRIENDS]
Topic

52. © Neo Technology
Discover
• Time Proximity: time property on recommended item
• Events closer to now are more relevant than further away
• Recent news is more relevant than old news
Relevance
Time
Relevance
Time
Event Article

53. © Neo Technology
Recon
Discover Tweak Test

54. © Neo Technology
Repeat
• Neo4j is agile friendly, so is Recon.
• Update how things are recommended with a few clicks
• A/B test different versions of the engine in no time
• Seasonal recommendations without any code change (e.g. Black
Friday: favour the discounts)

55. © Neo Technology

56. © Neo Technology

57. © Neo Technology
Moving Forward with Neo

58. Valuable Resources!
neo4j.com/developer neo4j.com/solutions neo4j.com/product
Developers Solutions Product

59. © Neo Technology
neo4j.com/blog neo4j.com/graphgists

60. © Neo Technology
neo4jsandbox.com

61. Moving Forward With Neo
End Customers
• Open A Dialogue with Neo Technology Account Rep
• Enterprise Edition Evaluations
• Training Classes
• Pilots
Consultants
• Explore a variety of partnership opportunities

62. © Neo Technology
Q&A