The document discusses event sourcing and CQRS patterns. It explains that event sourcing stores a history of events to rebuild the system state, while CQRS separates read and write operations by having different data models optimized for each. Materialized views are then used to generate read models from the event stream to enable efficient querying and address the potential delay in reads compared to a traditional system. The document also provides lessons learned, noting that both the write and read sides need attention, and that event sourcing works best for high volume, low latency writes like in microservices architectures.
3. 3
Concepts from DDD
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Aggregate
Event Sourcing,CQRS => DDD/
State: f(t)
Domain Event
4. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 4
Once upon a time…
Everything was Synchronous
Request <-> Response
5. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 5
Scaling out…
Updates —> Locks —> Contention!
<— Block! <—
-> Distributed
still Synchronous
6. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 6
Let’s go Asynchronous
-> Distributed & Message-driven
Request/Response ACID Transactions
• Distributed: Propagation takes time
• Async: No global order strictly guaranteed
Updates applied in the wrong order
=> Inconsistent state!
7. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 7
Command Sourcing
💡
• Append Only —> No Contention, No Update
• Build State from Commands history
K/V Store
Distributed
8. Command
“Submit this Order! (Please)”
-> A request (imperative sentence)
-> May fail, be rejected
-> May affect multiple Aggregates
8
Commands
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
✉
Rebuild Aggregate State
from Commands
9. 9
Events
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Event
“Order submitted”
-> Statement of facts (past tense)
-> Never fails
-> Can’t be changed
-> May be designed
to affect a single Aggregate
✉
Rebuild Aggregate State
from Events
10. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 10
Command > Event Sourcing
💡
11. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 11
Commands to Events
X
Y
Z
?
13. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 13
e.g. Saga
Stateful
Out of band
Corrective Action
(Command / Event)
Saga
14. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
–Greg Young
“Accountants don’t use pencils.
They use pens”
Corrective Actions
14
15. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 15
Benefits of Event Sourcing
History
(for free)
Rebuild State
at a point in Time
16. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 16
Benefits of Event Sourcing
Easier
Eventual Business Consistency
—> Corrective Events
Robust to data corruption
(bugs, malicious, fat fingers…)
17. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 17
Benefits of Event Sourcing
Horizontal Scalability
&
Low Latency writes
-> Distributed systems & data store
—> Append-only Log
—> Asynchronous processing
18. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
18
Commands to Events
Stateful: Events depends 0n current State
Stateless: Validate, split by Aggregate…
Depends on
Business
Domain
x Point of synchronisation
? Out-of-order commands (IoT, Mobile)
19. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 19
Thinking fast or slow
Stateless:
Think fast,
Write fast,
More thinking later
Stateful:
Think slow…(rebuild state),
Write fast,
Less thinking later
21. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 21
Retrieving the State
How do I retrieve the State?
“Get details of Order ‘AB123’”
❔
not very efficient, but…
…may work
22. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 22
Querying (Searching) the State
❓ How do query the State?
“Get all Orders delivered to ‘SE1 0NZ’”
23. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 23
CQRS
Command
Query
Responsibility
Segregation
💡
Separate
• Code
• muService
• Datastore
-> Update
-—> Retrieve
24. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 24
Not a new idea
Specialised
Downstream
25. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 25
CQRS and Event Sourcing
for low-latency writes
Event Sourcing => CQRS
26. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 26
Materialised Views
• In Memory
K/V Store
Graph DB
RDBMS
• Rebuildable
from Events
💡
a.k.a.
Read-optimised Views
Read Views
Projections…
27. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 27
Materialised Views
* Views are optimised for
specific query use cases
—> multiple Views from same Events
* May be updated asynchronously
+ low latency writes, scalability
- delayed
+ may reorder Events
28. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 28
Materialised Views
The Event Log
is your Source of Truth
* Easy to evolve or fix
—> change or fix logic;
rebuild view from events
29. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 29
Indexes
• Search Engines
K/V Stores
Graph DB
+ Optimised for querying (less for retrieving)
+ Reduced delay of State
💡
30. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 30
Hybrid solutions: e.g. Snapshots
+ Speed up rebuilding the State
+ Use recent Events to rebuild up-to-date
💡 Long delayed
31. Distributed + Asynchronous system
Distributed ACID Transactions (2PC)
31
No Global Current State
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
AS
S
A
B
Information propagates
at a finite speed
32. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 32
Lesson
from the Trenches
33. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 33
Lesson Learned #1
If you put data in…
…you will eventually
have to get them out!
The “Query” side
is not secondary
34. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 34
Lessons Learned #2
In old days:
normalising one DB
to support as many queries as possible
With CQRS (also No SQL in general)
multiple denormalised “data stores”
optimised for different queries
35. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 35
Lessons Learned #3
Centralised Event Store
= Event-sourced monolith
Each microservice owns its state
(Event Store)
Microservices:
37. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 37
ES/CQRS Optimal Use Cases
• High Volume, Low Latency writes
• Out-of-order Commands (IoT, Mobile)
Event Sourcing + CQRS 😋
Domain: stateless Command-> Event
38. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 38
ES/CQRS Drawbacks
x No “One-Size-Fits-All”
—> Multiple “Q” implementations
x Delayed reads
x No ACID Transactions
x Additional complexity (!!!)
🙁
39. Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 39
ES/CQRS Benefits
+ No “One-Size-Fits-All”
—> “Q” optimised per use cases
+ Distributed systems (Microservices)
+ Eventual (Business) Consistency
+ History, Temporal queries
+ Robust to data corruption
😀