Resilient Applications with Akka Persistence - Scaladays 2014
•
55 likes•23,495 views
This document discusses Akka persistence and command/event sourcing. It introduces key concepts like command and event sourcing where state is represented as a log of events, and commands generate events. It describes how Akka persistence allows storing commands/events to a journal and replaying them on recovery. It also discusses views which allow querying the event log, and using Akka persistence in a clustered environment with techniques like cluster singleton and cluster sharding.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (13)
Similar to Resilient Applications with Akka Persistence - Scaladays 2014
Similar to Resilient Applications with Akka Persistence - Scaladays 2014 (20)
Recently uploaded
Recently uploaded (20)
Resilient Applications with Akka Persistence - Scaladays 2014
- 1. Resilient Applications with Akka Persistence Patrik Nordwall @patriknw Konrad Malawski @ktosopl Björn Antonsson @bantonsson
- 2. Reactive Applications Akka Persistence ScalaDays 2014
- 3. Resilient • Embrace Failure • Failure is a normal part of the application lifecycle • Self Heal • Failure is detected, isolated, and managed Akka Persistence ScalaDays 2014
- 4. The Naïve Way • Write State to Database • Transactions Everywhere • Problem Solved? • Not Scalable, Responsive, Event-Driven! Akka Persistence ScalaDays 2014
- 5. Command and Event Sourcing
- 6. Command and Event Sourcing • State is the sum of Events • Events are persisted to Store • Append only • Scales well Akka Persistence ScalaDays 2014
- 7. Command v.s. Event • Command • What someone wants me to do • Can be rejected • Event • Something that has already happened • An immutable fact Akka Persistence ScalaDays 2014
- 8. Commands can Generate Events • If I accept a Command and change State • Persist Event to Store • If I crash • Replay Events to recover State Akka Persistence ScalaDays 2014
- 9. Persist All Commands? • If I crash on a Command • I will likely crash during recovery • Like the Army • Don't question orders • Repeat until success Akka Persistence ScalaDays 2014
- 10. Only Persist Events • Only accepted Commands generate Events • No surprises during recovery • Like a dieting method • You are what you eat Akka Persistence ScalaDays 2014
- 11. Achievement Unlocked? • Resilient • State is recoverable • Scalable • Append only writes • Something Missing? • Queries Akka Persistence ScalaDays 2014
- 12. CQRS Command Query Responsibility Segregation
- 13. CQRS • Separate Models • Command Model • Optimized for command processing • Query Model • Optimized data presentation Akka Persistence ScalaDays 2014
- 14. Query Model from Events • Source the Events • Pick what fits • In Memory • SQL Database • Graph Database • Key Value Store Akka Persistence ScalaDays 2014
- 15. Akka Persistence ScalaDays 2014 Client Service Query Model Command Store Query Store Command Model
- 17. PersistentActor Processor & Eventsourced Processor Replaces: in Akka 2.3.4+
- 18. super quick domain modelling! sealed trait Command! case class GiveMe(coins: Int) extends Command! case class TakeMy(coins: Int) extends Command Commands - what others “tell” us; not persisted case class Wallet(coins: Int) {! def updated(diff: Int) = State(coins + diff)! } State - reflection of a series of events sealed trait Event! case class BalanceChangedBy(coins: Int) extends Event! Events - reflect effects, past tense; persisted
- 19. var state = S0 ! def processorId = “a” ! PersistentActor Command ! ! Journal
- 20. PersistentActor var state = S0 ! def processorId = “a” ! ! ! Journal Generate Events
- 21. PersistentActor var state = S0 ! def processorId = “a” ! ! ! Journal Generate Events E1
- 22. PersistentActor ACK “persisted” ! ! Journal E1 var state = S0 ! def processorId = “a” !
- 23. PersistentActor “Apply” event ! ! Journal E1 var state = S1 ! def processorId = “a” ! E1
- 24. PersistentActor ! ! Journal E1 var state = S1 ! def processorId = “a” ! E1 Okey!
- 25. PersistentActor ! ! Journal E1 var state = S1 ! def processorId = “a” ! E1 Okey!
- 26. PersistentActor ! ! Journal E1 var state = S1 ! def processorId = “a” ! E1 Ok, he got my $.
- 27. PersistentActor class BitCoinWallet extends PersistentActor {! ! var state = Wallet(coins = 0)! ! def updateState(e: Event): State = {! case BalanceChangedBy(coins) => state.updatedWith(coins)! }! ! // API:! ! def receiveCommand = ??? // TODO! ! def receiveRecover = ??? // TODO! ! }!
- 28. persist(e) { e => }
- 29. PersistentActor def receiveCommand = {! ! case TakeMy(coins) =>! persist(BalanceChangedBy(coins)) { changed =>! state = updateState(changed) ! }! ! ! ! ! ! ! } async callback
- 30. PersistentActor: persist(){} def receiveCommand = {! ! ! ! ! ! ! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }! } async callback Safe to mutate the Actor’s state
- 31. PersistentActor def receiveCommand = {! ! ! ! ! ! ! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }! } Safe to access sender here
- 32. persist(){} - Ordering guarantees ! ! Journal E1 var state = S0 ! def processorId = “a” ! C1 C2 C3
- 33. ! ! Journal E1 var state = S0 ! def processorId = “a” ! C1 C2 C3 Commands get “stashed” until processing C1’s events are acted upon. persist(){} - Ordering guarantees
- 34. ! ! Journal var state = S0 ! def processorId = “a” ! C1 C2 C3 E1 E2 E2E1 events get applied in-order persist(){} - Ordering guarantees
- 35. C2 ! ! Journal var state = S0 ! def processorId = “a” ! C3 E1 E2 E2E1 and the cycle repeats persist(){} - Ordering guarantees
- 36. persistAsync(e) { e => }
- 37. persistAsync(e) { e => } + defer(e) { e => }
- 38. def receiveCommand = {! ! ! ! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }! ! ! ! ! } persistAsync PersistentActor: persistAsync(){} will NOT force stashing of commands
- 39. PersistentActor: persistAsync(){} def receiveCommand = {! ! ! ! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }! ! defer(Marked(id)) { marked =>! sender() ! marked! }! } execute once all persistAsync handlers done NOT persisted
- 40. persistAsync(){} - Ordering guarantees ! ! Journal var state = S0 ! def processorId = “a” ! C1 C2 C3
- 41. persistAsync(){} - Ordering guarantees ! ! Journal var state = S0 ! def processorId = “a” !C2 C3
- 42. persistAsync(){} - Ordering guarantees ! ! Journal var state = S0 ! def processorId = “a” ! C3
- 43. persistAsync(){} - Ordering guarantees ! ! Journal var state = S0 ! def processorId = “a” ! C3 E1 E2
- 44. persistAsync(){} - Ordering guarantees var state = S0 ! def processorId = “a” ! C3 E1 Akka Persistence ScalaDays ! ! Journal E1 E2
- 45. persistAsync(){} - Ordering guarantees E1 var state = S1 ! def processorId = “a” ! E2 E1 E2 ! ! Journal Akka Persistence ScalaDays E2 E3E1
- 46. persistAsync(){} - Ordering guarantees E1 var state = S2 ! def processorId = “a” ! E2 E1 E2 deferred handlers triggered M1 M2 ! ! Journal Akka Persistence ScalaDays E2 E3E1
- 48. Eventsourced, recovery /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case replayedEvent: Event => ! state = updateState(replayedEvent)! } re-using updateState, as seen in receiveCommand Akka Persistence ScalaDays
- 50. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! polling Akka Persistence ScalaDays ! View ! def processorId = “a” ! ! !
- 51. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! polling ! View ! def processorId = “a” ! ! ! polling different ActorPath, same processorId Akka Persistence ScalaDays ! View ! def processorId = “a” ! ! !
- 52. View class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” ! ! }! } subject to change! Akka Persistence ScalaDays
- 53. Views, as Reactive Streams Akka Persistence ScalaDays
- 54. View, as ReactiveStream // Imports ...! ! import org.reactivestreams.api.Producer! ! import akka.stream._! import akka.stream.scaladsl.Flow! ! import akka.persistence._! import akka.persistence.stream._! val materializer = FlowMaterializer(MaterializerSettings())! pull request by krasserm early preview Akka Persistence ScalaDays
- 55. View, as ReactiveStream // 1 producer and 2 consumers:! val p1: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-1").! toProducer(materializer)! ! Flow(p1).! foreach(p => println(s"consumer-1: ${p.payload}”)).! consume(materializer)! ! Flow(p1).! foreach(p => println(s"consumer-2: ${p.payload}”)).! consume(materializer) pull request by krasserm early preview Akka Persistence ScalaDays
- 56. View, as ReactiveStream // 2 producers (merged) and 1 consumer:! val p2: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-2").! toProducer(materializer)! val p3: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-3").! toProducer(materializer)! ! Flow(p2).merge(p3). // triggers on “either”! foreach { p => println(s"consumer-3: ${p.payload}") }.! consume(materializer)! pull request by krasserm early preview Akka Persistence ScalaDays
- 57. Akka Persistence ScalaDays 2014 Usage in a Cluster • distributed journal (http://akka.io/community/) • Cassandra • DynamoDB • HBase • MongoDB • shared LevelDB journal for testing • single writer • cluster singleton • cluster sharding
- 58. Akka Persistence ScalaDays 2014 Cluster Singleton A B C D
- 59. Akka Persistence ScalaDays 2014 Cluster Singleton A B C D role: backend-1 role: backend-1 role: backend-2 role: backend-2
- 60. Akka Persistence ScalaDays 2014 Cluster Sharding A B C D
- 61. Akka Persistence ScalaDays 2014 Cluster Sharding sender id:17 region node-‐1 coordinator region node-‐2 region node-‐3 GetShardHome:17 id:17 ShardHome:17 -‐> node2 17 -‐> node2
- 62. Akka Persistence ScalaDays 2014 Cluster Sharding sender region node-‐1 coordinator region node-‐2 region node-‐3 id:17 id:17 GetShardHome:17 ShardHome:17 -‐> node2 id:17 17 -‐> node2 17 -‐> node2
- 63. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 id:17 id:17 17 -‐> node2 17 -‐> node2 17 -‐> node2
- 64. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 17 -‐> node2 17 -‐> node2 17 -‐> node2 id:17
- 65. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 17 -‐> node2 17 -‐> node2 17 -‐> node2 id:17
- 66. Cluster Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.postId, cmd) } ! val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => (math.abs(cmd.postId.hashCode) % 100).toString } ClusterSharding(system).start( typeName = BlogPost.shardName, entryProps = Some(BlogPost.props()), idExtractor = BlogPost.idExtractor, shardResolver = BlogPost.shardResolver) val blogPostRegion: ActorRef = ClusterSharding(context.system).shardRegion(BlogPost.shardName) ! val postId = UUID.randomUUID().toString blogPostRegion ! BlogPost.AddPost(postId, author, title)
- 67. Akka Persistence ScalaDays 2014 Lost messages sender destination $
- 68. Akka Persistence ScalaDays 2014 At-least-once delivery - duplicates sender destination $ ok $ $ $ ok Re-‐send
- 69. Akka Persistence ScalaDays 2014 M2 At-least-once delivery - unordered sender destination M1 ok 1 ok 2 M2 ok 3 M3 M1M3 M2 Re-‐send
- 70. Akka Persistence ScalaDays 2014 M2 At-least-once delivery - crash sender destination M1 ok 1 ok 2 M2 ok 3 M3 1. Sent M1 2. Sent M2 3. Sent M3 M3 5. M2 Confirmed 6. M3 Confirmed 4. M1 Confirmed sender M1M2 M3
- 71. PersistentActor with AtLeastOnceDelivery case class Msg(deliveryId: Long, s: String) case class Confirm(deliveryId: Long) sealed trait Evt case class MsgSent(s: String) extends Evt case class MsgConfirmed(deliveryId: Long) extends Evt class Sender(destination: ActorPath) extends PersistentActor with AtLeastOnceDelivery { ! def receiveCommand: Receive = { case s: String => persist(MsgSent(s))(updateState) case Confirm(deliveryId) => persist(MsgConfirmed(deliveryId))(updateState) } ! def receiveRecover: Receive = { case evt: Evt => updateState(evt) } ! def updateState(evt: Evt): Unit = evt match { case MsgSent(s) => deliver(destination, deliveryId => Msg(deliveryId, s)) ! case MsgConfirmed(deliveryId) => confirmDelivery(deliveryId) } }
- 72. Akka Persistence ScalaDays 2014 Next step • Documentation • http://doc.akka.io/docs/akka/2.3.3/scala/persistence.html • http://doc.akka.io/docs/akka/2.3.3/java/persistence.html • http://doc.akka.io/docs/akka/2.3.3/contrib/cluster-sharding.html • Typesafe Activator • https://typesafe.com/activator/template/akka-sample-persistence-scala • https://typesafe.com/activator/template/akka-sample-persistence-java • http://typesafe.com/activator/template/akka-cluster-sharding-scala • Mailing list • http://groups.google.com/group/akka-user • Migration guide from Eventsourced • http://doc.akka.io/docs/akka/2.3.3/project/migration-guide-eventsourced-2.3.x.html
- 73. ©Typesafe 2014 – All Rights Reserved