Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Recently uploaded
Recently uploaded (20)
Akka Persistence and Eventuate
- 1. Akka Persistence and Eventuate A CQRS/ES tool comparison Martin Krasser (@mrt1nz) Freelance software developer/architect
- 2. History Eventsourced Akka Persistence Eventuate 7.2012 9.2013 1.2015
- 3. Similarities
- 4. Basis technologies Scala Akka actors Akka streams (Pluggable) storage backends
- 6. Akka Persistence and Eventuate Persist actor state via event sourcing
- 7. Event sourcing Capture all changes to application state as sequence of events Store events in event log (append- only) Replay events to recover application state
- 8. Event sourcing Commands - actions, can fail Events - immutable facts
- 10. Command processing Event-sourced actor State Command handler Event handler Event log
- 11. Command processing Event-sourced actor State Command handler Event handler Command Event log
- 12. Command processing Event-sourced actor State Command handler Event handler Command Event log Events
- 13. Command processing Event-sourced actor State Command handler Event handler Command Event log Events Events
- 14. Command processing Event-sourced actor State Command handler Event handler Command Event log Events Events
- 15. Command processing Event-sourced actor State Command handler Event handler Command Reply Event log Events Events
- 16. State recovery Event-sourced actor State Command handler Event handler Event log
- 17. State recovery Event-sourced actor State Command handler Event handler Event log Events (replay)
- 18. State recovery Event-sourced actor State Command handler Event handler Event log Events (replay)
- 19. Akka Persistence and Eventuate Separate command processing from query processing
- 20. CQRS Command Query Responsibility Segregation Different data models for command processing and query processing
- 21. CQRS Write model for command processing Read model for query processing
- 22. CQRS Can be well combined with event sourcing (CQRS/ES)
- 23. CQRS/ES Write model Event log Command Query Events (r/w) Events (r) Command side Query side Read model
- 24. Abstractions Command side Query side Akka Persistence PersistentActor PersistentView Source[E,M] Eventuate EventsourcedActor EventsourcedView EventsourcedProcessor Source[E,M] *) work in progress
- 32. Consistency Akka Persistence and Eventuate support strong consistency Eventuate additionally supports relaxation to causal consistency
- 33. Akka Persistence Enforces strong consistency on command side PersistentActors must be global singletons No actor/state replication
- 34. Eventuate EventsourcedActors can be replicated State replication via reliable, asynchronous event replication Replication across locations (availability zones)
- 36. Replication Location A Application state Location B Location C
- 37. Replication Events Events Location A Application state Location B Location C
- 38. Replication Events Events Location A Application state Location B Location C
- 43. Eventuate Replicas writeable at all locations = multi-master Write-availability during network partitions
- 55. Resolution XY XY XY Interactive resolution XY => Z Location A Location B Location C
- 56. Resolution XY XY Interactive resolution XY => Z Z Location A Location B Location C
- 57. Resolution XY XY Interactive resolution XY => Z Z Events Events Location A Location B Location C
- 58. Resolution Interactive resolution XY => Z Z Z Z Events Events Location A Location B Location C
- 59. Eventuate Strong consistency within location*) Causal consistency across locations Causality is tracked with vector clocks *) relaxation to causal consistency also possible within location
- 60. Causal consistency example Distributed chat application FIFO reliable broadcast vs. Causal reliable broadcast
- 61. Inspired by ACM article “Don’t Settle for Eventual Consistency” http://queue.acm.org/detail.cfm? id=2610533 Causal consistency example
- 63. FIFO rel. broadcast - Alice: Lost my wedding ring Alice DC1 DC2 DC3
- 64. FIFO rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice DC1 DC2 DC3
- 65. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
- 66. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
- 67. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
- 68. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast
- 69. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast
- 70. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 Bob FIFO rel. broadcast
- 71. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast Causality violation!
- 72. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast Retry Causality violation!
- 73. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3
- 74. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that DC1 DC2 DC3
- 75. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that Causality check DC1 DC2 DC3
- 76. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that Causality check DC1 DC2 DC3
- 77. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that Causality check Causality check DC1 DC2 DC3
- 78. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality check Causality check DC1 DC2 DC3
- 79. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality check Retry DC1 DC2 DC3 Causal rel. broadcast
- 80. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality check - Alice: Whew, found it upstairs Retry DC1 DC2 DC3 Causal rel. broadcast
- 81. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality check - Bob: I’m glad to hear that - Alice: Whew, found it upstairs Retry DC1 DC2 DC3 Causal rel. broadcast
- 82. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Alice: Whew, found it upstairs Causal re-ordering DC1 DC2 DC3 Causal rel. broadcast
- 83. Causal consistency Causally related events are delivered in same order at all locations Concurrent events can be delivered in any order at different locations
- 85. “Local” event log Eventuate - LevelDB - Cassandra Location A Location B Location C
- 86. “Local” event log “Replicated” event log Eventuate - LevelDB - Cassandra Location A Location B Location C - LevelDB - Cassandra EventsEvents Events
- 87. Local event log Storage order consistent with causal order Given the potential causality relation ➞ of events Local event logs are linear extensions of ➞
- 88. Replicated event log Same causal event storage order at different locations Different total event storage order at different locations
- 89. Akka Persistence - LevelDB - Cassandra - EventStore - MongoDB - Kafka - … Location A Location B Location C
- 91. Akka Persistence PA PA PA PA Event log: private for writing shared for reading (query side)
- 92. Eventuate EA EA EA Event log: shared for writing shared for reading EA
- 97. Event routing EA EA EA e1 e1 EAEA e1 EA e1 Location A Location B Location C
- 98. Event collaboration State replication Service interaction
- 99. Event collaboration S2 S1 S1 S3 Location A Location B Location C
- 100. Event collaboration S2 S1 e1 S1 S3 Location A Location B Location C
- 101. Event collaboration S2 S1 e1 S1 e1 S3 Location A Location B Location C
- 102. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C
- 103. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C e2
- 104. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C e2 e2
- 105. Event collaboration Reliable, distributed and partition- tolerant business processes Event-driven (micro-)service architectures
- 106. Event collaboration Reliable event delivery Causal event delivery De-duplicated event stream
- 108. Akka Persistence 2.3 PA PA PV PV PV PersistentViews on single PersistentActor (default)
- 109. Akka Persistence 2.4 PA PA S S S Stream Sources for multiple PersistentActors (requires storage plugin support)
- 110. Eventuate EV EV EventsourcedViews on multiple EventsourcedActors (default, shared local log) EA EA EA
- 111. Eventuate EV EV EventsourcedViews on multiple EventsourcedActors (default, shared replicated log) EA EA EA EA EAEA Location A Location B Location C
- 112. Consistency in CQRS Eventual consistency Causal consistency Akka Persistence usually per PA (plugin-specific) Eventuate
- 113. Causal consistency Single EA/EV: default Across EA/EV: conditional requests
- 114. Example scenario Update a write model (EA) Query a read model (EV) Query result should include effect that was caused by the update
- 117. Conditional request EA Location A EV Location B evt cmd
- 118. Conditional request EA Location A EV Location B evt evt(vts) cmd vts = vector timestamp (a “condition”)
- 119. Conditional request EA Location A EV Location B evt evt(vts) cmd reply(vts) vts = vector timestamp (a “condition”)
- 120. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”)
- 121. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
- 122. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
- 123. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
- 124. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) deliver
- 125. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) reply deliver
- 126. Akka Distributed Data and Eventuate CRDTs
- 127. CRDTs Conflict-free Replicated Data Types Automated resolution of write conflicts
- 128. CRDTs Akka distributed data: CvRDTs (convergent or state-based) Eventuate: CmRDTs (commutative or operation-based)
- 129. CRDTs CvRDTs CmRDTs Message payload Current state Operations Communication middleware No guarantees Reliable broadcast (idempotent) Change history CvRDT intern Persistent log
- 130. Specification “A comprehensive study of Convergent and Commutative Replicated Data Types” http://hal.upmc.fr/docs/00/55/55/88/ PDF/techreport.pdf
- 131. Akka CvRDTs 10 implemented from specification (incl. counters, registers, sets, maps) In-memory only (non-durable) Custom data types can be defined
- 132. Eventuate CmRDTs 4 implemented from specification (Counter, MV/LWW-register, OR-Set) Durable via event sourcing Eventuate CRDT framework
- 134. Thank you!