7. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
8. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
9. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
within 90 minutes of purchase
10. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
within 90 minutes of purchase
or a 1 hour window of their choice
11. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
within 90 minutes of purchase
or a 1 hour window of their choice
(same day or any day)
12. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
within 90 minutes of purchase
or a 1 hour window of their choice
(same day or any day)
• fastest delivery to date 8:30 min
13. • SaaS platform
• we provide an API for carriers and merchants
• built on mysql, rails and ruby mri
• customers can chose between a delivery either:
within 90 minutes of purchase
or a 1 hour window of their choice
(same day or any day)
• fastest delivery to date 8:30 min
• customers: Argos, Maplins, DrEd.com ...
14. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
Problems?
15. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
http://xkcd.com/287/
18. Challenges and Problems
• a whole zoo of np-complete problems
• exponential growth of joins in mysql with added features
19. Challenges and Problems
• a whole zoo of np-complete problems
• exponential growth of joins in mysql with added features
• code base too complex and unmaintanable
20. Challenges and Problems
• a whole zoo of np-complete problems
• exponential growth of joins in mysql with added features
• code base too complex and unmaintanable
• api response time growing to large the more data is added
22. The solution for v2:
build a new api on the basis of sinatra and jruby
23. The solution for v2:
build a new api on the basis of sinatra and jruby
databases used:
24. The solution for v2:
build a new api on the basis of sinatra and jruby
databases used:
- neo4j embedded
25. The solution for v2:
build a new api on the basis of sinatra and jruby
databases used:
- neo4j embedded
- mongoDB
26. The solution for v2:
build a new api on the basis of sinatra and jruby
databases used:
- neo4j embedded
- mongoDB
- redis
27. The solution for v2:
build a new api on the basis of sinatra and jruby
databases used:
- neo4j embedded
- mongoDB
- redis
- and mysql
28. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
29. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
relationships are explicit stored
30. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
relationships are explicit stored
white board friendly and easier domain modeling
31. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
relationships are explicit stored
white board friendly and easier domain modeling
schema-less
32. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
relationships are explicit stored
white board friendly and easier domain modeling
schema-less
a graph is its own index
33. Shutl Main Red: Shutl Accent Red:
Pantone 485 C Pantone 484 C
C0 M100 Y99 K4 C0 M100 Y99 K4
R208 G31 B40 R208 G31 B40
The case for graph databases
red on white HEX D01F28 HEX D01F28
red on lighter tones
Shutl Black: Shutl Accent Grey:
Pantone BLACK Pantone BLACK
C0 M0 Y0 K0100 C0 M0 Y0 K70
@ 70%
Please note:
Black on white Reverse (white) on darker tones The black logo should never
appear on in any ‘dark’ colour
background.
5 Branding Guidelines
relationships are explicit stored
white board friendly and easier domain modeling
schema-less
a graph is its own index
traversals of relationships are easy
37. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
38. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
we can run it embedded in the same jvm
39. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
we can run it embedded in the same jvm
we can use jruby as we know ruby very well already
40. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
we can run it embedded in the same jvm
we can use jruby as we know ruby very well already
lots of good ruby libraries are available, we chose the neo4j gem
by Andreas Ronge (https://github.com/andreasronge/neo4j)
41. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
we can run it embedded in the same jvm
we can use jruby as we know ruby very well already
lots of good ruby libraries are available, we chose the neo4j gem
by Andreas Ronge (https://github.com/andreasronge/neo4j)
it speaks cypher
42. Neo4j
Why did we chose it:
it didn’t solve our np-complete problems but it solved our join hell
we can run it embedded in the same jvm
we can use jruby as we know ruby very well already
lots of good ruby libraries are available, we chose the neo4j gem
by Andreas Ronge (https://github.com/andreasronge/neo4j)
it speaks cypher
the guys from neotech are awesome
43. Neo4j
embedded vs. standalone
better performance access via rest api and
transaction support cypher
pros: neo4j gem is available language independent and
we can use cypher and code doesn’t need to run
traversal on JVM
only the code running the not as performant
db has access to the db only works with cypher
transaction is on a per
cons:
query basis
need to write model
wrappers for ourselves
44. Neo4j
alternatives to the neo4j gem:
neography by Max de Marzi
(https://github.com/maxdemarzi/neography)
and pacer by Darrick Wiebe:
(https://github.com/pangloss/pacer-neo4j)
46. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
47. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
48. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
• switch from relational ‘mentality’ to a graph one was harder than
expected
49. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
• switch from relational ‘mentality’ to a graph one was harder than
expected
• we have no real solution for migrations so far
50. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
• switch from relational ‘mentality’ to a graph one was harder than
expected
• we have no real solution for migrations so far
• seeding an embedded database is hard
51. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
• switch from relational ‘mentality’ to a graph one was harder than
expected
• we have no real solution for migrations so far
• seeding an embedded database is hard
• encoding Dates and Times that are stored in UTC and work across
timezone is non-trivial
52. Neo4j
gotchas and stuff we didn’t know:
• testing proved to be difficult and we had to write our own tools
• some libraries (like cucumber) are not compatible with jruby
• switch from relational ‘mentality’ to a graph one was harder than
expected
• we have no real solution for migrations so far
• seeding an embedded database is hard
• encoding Dates and Times that are stored in UTC and work across
timezone is non-trivial
• nested datastructure (hashes and array) can’t be stored and need
to be converted to json
53. Neo4j
testing:
• we are using rspec for all tests on the api and practice tdd/bdd
• setting up ‘scenarios’ for an integration test was almost impossible
with existing tools
• we decided to built our own tool based on the geoff notation
developed by Nigel Small
54. Neo4j
geoff:
developed by Nigel Small (@technige, http://geoff.nigelsmall.net/)
allows modelling of graphs in a human readable form
(A) {"name": "Alice"}
(B) {"name": "Bob"}
(A)-[:KNOWS]->(B)
and provides an interface to insert them into an existing graph
55. Neo4j
geoff gem
(https://github.com/shutl/geoff)
• provides a dsl for creating a graph and inserting it into the db
• it is open source
• it works together with FactoryGirl
(https://github.com/thoughtbot/factory_girl)
• it supports only the graph structure of the neo4j gem at the
moment
• we haven’t solved all the issues with event listeners yet
56. Neo4j
FactoryGirl
https://github.com/thoughtbot/factory_girl
# This will guess the User class
FactoryGirl.define do
factory :user do
first_name "John"
last_name "Doe"
admin false
end
# This will use the User class (Admin would have been
guessed)
factory :admin, class: User do
first_name "Admin"
last_name "User"
admin true
end
end
57. #Gemfile
Neo4j gem 'geoff'
# Basic tree like structure for DSL
# the first line generates the class nodes used by
Neo4jWrapper
# NB 'Company' and 'Person' are classes with the
geoff gem Neo4j::NodeMixin
(https://github.com/shutl/geoff) Geoff(Company, Person) do
company 'Acme' do
address "13 Something Road"
outgoing :employees do
person 'Geoff'
person 'Nigel' do
name 'Nigel Small'
end
end
end
company 'Github' do
outgoing :customers do
person 'Tom'
person 'Dick'
person 'Harry'
end
end
person 'Harry' do
incoming :customers do
company 'NeoTech'
end
end
end
58. root
node
:company :person
:all
:all :all
:employees Geoff :all
:all :all
:all
Nigel
acme Small
13 somthing
road :all :employees
Tom
:customers
Dick
GitHub
:customers
Harry
:customers
NeoTech
60. if there are lots of attributes on a node it becomes difficult
to read
Geoff(Company, Person) do
company 'Acme' do
address "13 Something Road"
contact_name “Jane Doe”
contact_email “jane@acme.com”
outgoing :employees do
person 'Geoff' do
first_name “Geoff”
last_name “Small”
email “geoff@geoff.com”
end
end
end
end
61. attributes can be specified in factory-girl
FactoryGirl.define do
factory :acme, class: Company do
address "13 Something Road"
contact_name “Jane Doe”
contact_email “jane@acme.com”
end
end
FactoryGirl.define do
factory :geoff, class: Person do
first_name “Geoff”
last_name “Small”
email “geoff@geoff.com”
end
end
62. and used in the geoff dsl
Geoff(Company, Person) do
company 'Acme' do
geoffactory :acme
outgoing :employees do
person 'Geoff' do
geoffactory :geoff
end
end
end
end
it also works with traits
65. Why did we chose it:
very easy to get started (also on dev machines)
66. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
67. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
it allows for easy sharding and horizontal scaling
68. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
it allows for easy sharding and horizontal scaling
it is a json store (as our api is a json api it allows very easy storage
of the query results)
69. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
it allows for easy sharding and horizontal scaling
it is a json store (as our api is a json api it allows very easy storage
of the query results)
it allows easy storage of nested structure and allows for queries
inside structure
70. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
it allows for easy sharding and horizontal scaling
it is a json store (as our api is a json api it allows very easy storage
of the query results)
it allows easy storage of nested structure and allows for queries
inside structure
lots of ruby gems available (we use mongomapper, http://
mongomapper.com/)
71. Why did we chose it:
very easy to get started (also on dev machines)
it is schemaless
it allows for easy sharding and horizontal scaling
it is a json store (as our api is a json api it allows very easy storage
of the query results)
it allows easy storage of nested structure and allows for queries
inside structure
lots of ruby gems available (we use mongomapper, http://
mongomapper.com/)
the 10gen office is 2 floors above ours
73. gotchas, cons and stuff we didn’t know:
• it is schemaless and changed to the schema need to handled
carefully
74. gotchas, cons and stuff we didn’t know:
• it is schemaless and changed to the schema need to handled
carefully
• write and updated follow the ‘fire and forget’ pattern, raising an
error on save needs to be explicitly enabled
75. using the decorator/presenter pattern for schemaless dbs
decorator
‘decorates’ object for presentation
controller
passes decorated object to view
retrieves object
view
mongo
DB
78. Why did we chose it:
fast key-value store for caching and config values
79. Why did we chose it:
fast key-value store for caching and config values
very easy to implement
80. Why did we chose it:
fast key-value store for caching and config values
very easy to implement
we have experience with it and with resque
81. Why did we chose it:
fast key-value store for caching and config values
very easy to implement
we have experience with it and with resque
ruby libraries are available to allow easy access and namespacing
83. gotchas, cons and stuff we didn’t know:
• we tried to store default values for neo4j in it and found no
solution to include redis updates in a transaction
84. gotchas, cons and stuff we didn’t know:
• we tried to store default values for neo4j in it and found no
solution to include redis updates in a transaction
• we had some memory issues with resque that were difficult to
debug
87. Why are we using it:
we have lots of experience with it
88. Why are we using it:
we have lots of experience with it
it is very good for data aggregation (sums, groups)
89. Why are we using it:
we have lots of experience with it
it is very good for data aggregation (sums, groups)
ideal for financial transaction data for example
90. Why are we using it:
we have lots of experience with it
it is very good for data aggregation (sums, groups)
ideal for financial transaction data for example
some of our non-devs invested time to learn sql and we didn’t
want to lose the skillsets
91. Why are we using it:
we have lots of experience with it
it is very good for data aggregation (sums, groups)
ideal for financial transaction data for example
some of our non-devs invested time to learn sql and we didn’t
want to lose the skillsets
we have already developed the schemas for v1
