Johan Andrén presented on building reactive distributed systems with Akka. He provided an overview of Akka clustering and distributed data types. He discussed how Akka actors map to distributed systems and Akka's support for eventual consistency through gossip. He also covered distributed pub/sub messaging, cluster singletons that introduce a single point of failure, and cluster sharding to shard actors across nodes.
3. A man, a plan …
Akka recap, distsys background and Akka Cluster basics
We’ll get an overview of how Akka does clustering
Distributed Data
Eventual consistency through gossip
Distributed PubSub
A message bus across the cluster
Cluster Singleton
How to introduce a single point of failure
Cluster Sharding
Shard actors across cluster
}Cluster Tools
7. Distributed Systems
–Leslie Lamport
”A distributed system is one in which the failure of a
computer you didn't even know existed can render
your own computer unusable”
8. Why is it so hard?
Reliability: power failure, old network
equipment, network congestion, coffee in router,
rodents, that guy in the IT dept., DDOS attacks…
Latency: loopback vs local net vs shared congested local net vs
internet
Bandwidth: again loopback vs local vs shared local vs internet
The Joys of Computer Networks:
9. Why do it, if it is so hard?
Data or processing doesn’t fit a single machine
Many objects, that should be kept in memory. Many not so
powerful servers can be cheaper than a supercomputer.
Elasticity
Being able to scale in (less servers) and out (more servers)
depending on load. Not paying for servers unless you need them.
Resilience
Building systems that will keep working in the face of failures or
degrade gracefully.
10. Actor Model vs Network
Interaction already modelled as immutable messages
Data travels over the network in packages, changes has to be
explicitly sent back.
At most once
Data reaches a node on the other side at most once, but can be
lost, already part of model!
A recipient of a message can reply directly to sender
Regardless if there were intermediate recipients of the message
Messages not limited to request response
Messages can flow in either direction when two systems are
connected.
20. Seed nodes
!First seed node
if none of the other nodes in the list are in the
cluster - joins itself to form cluster
Rest of seed nodes
just pings all other nodes and joins as soon as
one is in the cluster responds
Join
21. What would happen if we mess it up?
I’m the leader, this is the cluster!
No! I’m the leader, this is the cluster!
Join
Join
No! I’m the leader, this is the cluster!
24. User API of Cluster
Node details
What roles am I in, what is my address
Join, Leave, Down
Programatic control over cluster membership
Register listeners for cluster events
Every time the cluster state changes the listening
actor will get a message
25. val commonConfig = ConfigFactory.parseString(
"""
akka {
actor.provider = cluster
remote.artery.enabled = true
remote.artery.canonical.hostname = 127.0.0.1
cluster.seed-nodes = [ "akka://cluster@127.0.0.1:25520", "akka://cluster@127.0.0.1:25521" ]
cluster.jmx.multi-mbeans-in-same-jvm = on
}
""")
def portConfig(port: Int) = ConfigFactory.parseString(s"akka.remote.artery.canonical.port = $port")
val node1 = ActorSystem("cluster", portConfig(25520).withFallback(commonConfig))
val node2 = ActorSystem("cluster", portConfig(25521).withFallback(commonConfig))
val node3 = ActorSystem("cluster", portConfig(25522).withFallback(commonConfig))
Three node cluster
complete sample sources on github
27. Distributed Data
CRDTs: Conflict free Replicated Data Types
allow for updates on any node and then
spreading that update to other cluster nodes
through gossip for eventual consistency
Note: Does not fit every problem!
Online docs for Distributed Data
28. Special requirements
Commutative
Order of operation does not matter
like 3 + 4 = 4 + 3
Associative
Grouping operations does not matter
like 3 + (4 + 5) = (3 + 4) + 5
Monotonic
Absence of rollbacks, ”only growing” (but we can do sneaky tricks)
29. Built in data structures
Counters
GCounter - grow only, PNCounter - increment and decrement counter
Sets
GSet - grow only, ORSet - observed remove set
Maps
ORMap - observed remove map, ORMultiMap - observed remove multi map,
PNCounterMap - positive negative counter map, LWWMap - last writer wins map
Flags and Register
Flag - toggle once boolean, LWWRegister - last writer wins register
38. Distributed PubSub
val mediator = DistributedPubSub(system).mediator
val actorRef = system.actorOf(props, "my-subscriber")
mediator ! DistributedPubSubMediator.Subscribe("my-topic", actorRef)
node3Mediator ! DistributedPubSubMediator.Publish(
"my-topic",
messageToAllSubscribers)
complete sample sources on github
39. Strong Consistency Always Available
Subscribers/Topics eventually consistent
always accepts writes
Distributed PubSub
messages delivered at most once