Presented at NDC London 2016 When working with mobile devices, remote clients, UI frameworks like AngularJS, WebAPI, messaging, publish/subscribe, etc, you're building a distributed system. Components withtin your system are spread out and/or communicate asynchronously. A lot of new challenges enter your world. In this session we'll look at some of the theories in the world of distributed systems to make you think about how to build these kind of systems. We'll look at the CAP Theorem, eventual consistency, distributed transactions, the '2 generals' principle, idem potency and why these are all important to your system.

1. Dennis van der Stelt
of distributed systems
Dennis van der Stelt
http://dennis.bloggingabout.net/
dennis@bloggingabout.net
Software Engineer at Particular Software
PRINCIPLES

2. Dennis van der Stelt
AGENDA

3. Dennis van der Stelt

4. Dennis van der Stelt
“A distributed system is a software
system in which components located
on networked computers
communicate and coordinate their
actions by passing messages.”
what is a distributed system
Wikipedia

5. Dennis van der Stelt
Working distributed…

6. Dennis van der Stelt
How hard can it be?

7. Dennis van der Stelt
FALLACIES OF DISTRIBUTED COMPUTING
PeterDeutsch& othersat Sun Microsystems, 1994
Latency is zero02
Topology doesn’t change05
There is one administrator06
The network is reliable01
Bandwidth is infinite03 Transport cost is zero07
The network is secure04 The network is homogeneous08

8. Dennis van der Stelt
Free eBook: Dr. Harvey and
the 8 Fallacies of Distributed
Computing

9. Dennis van der Stelt
The network is reliable

10. Dennis van der Stelt
STORE AND FORWARD WITH MSMQ
Store locally and pass ontotheinfrastructure
SQL Server

11. Dennis van der Stelt
using (var scope = new TransactionScope()))
{
var queue = new
MessageQueue(@".Private$Customers");
var message = queue.Receive();
var con = new SqlConnection();
var cmd = new SqlCommand("update customers ...");
cmd.ExecuteNonQuery();
}
DISTRIBUTED TRANSACTIONS
Microsoft DistributedTransactionCoordinator(MSDTC)
Code

12. Dennis van der Stelt
using (var scope = new TransactionScope()))
{
var queue = new
MessageQueue(@".Private$Customers");
var message = queue.Receive();
var con = new SqlConnection();
var cmd = new SqlCommand("update customers ...");
cmd.ExecuteNonQuery();
}
DISTRIBUTED TRANSACTIONS
Microsoft DistributedTransactionCoordinator(MSDTC)
Code
MSDTC

13. Dennis van der Stelt

14. Dennis van der Stelt
Distributed Transactions

15. Dennis van der Stelt
DISTRIBUTED TRANSACTIONS
The source of a lot of pain
RM
RM
RM
RM
RM
RM
RM
“Serializable is the highest isolation level. It makes sure that data that
anything you have a lock on, is locked for reading by others.”
Peter Bailis

16. Dennis van der Stelt
STORE AND FORWARD WITH MSMQ
Store locally and pass ontotheinfrastructure
SQL Server

17. Dennis van der Stelt
Handle(CreateClaimCommand msg)
{
dbContext.Claims.Add(Mapper.Map<Claim>(msg));
dbContext.SaveChanges();
Bus.Publish(new ClaimCreatedEvent());
}
DISTRIBUTED TRANSACTIONS WITH NSERVICEBUS
Microsoft DistributedTransactionCoordinator(MSDTC)
Code
Sends email to
customer

18. Dennis van der Stelt

19. Dennis van der Stelt
NOSQL

20. Dennis van der Stelt
CAP Theorem
Eric Brewer, PODC Conference 2000

21. Dennis van der Stelt
CAP THEOREM
You can only pick 2

22. Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability

23. Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability

24. Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
distributed
system
partition tolerantconsistencyavailability

25. Dennis van der Stelt
CAP THEOREM
You can only pick 2
centralized
system
partition tolerantconsistencyavailability
distributed
system
partition tolerantconsistencyavailability
when there’s network partition,
which do you sacrifice?

26. Dennis van der Stelt
Match the business perspective

27. Dennis van der Stelt

28. Dennis van der Stelt

29. Dennis van der Stelt

30. Dennis van der Stelt

31. Dennis van der Stelt
But I can’t drop consistency!

32. Dennis van der Stelt
Basically Available
BASE
What is BASE?
Soft state
Eventually consistent

33. Dennis van der Stelt
Eventual Consistency
Because 100% consistency isn’t the only option

34. Dennis van der Stelt

35. Dennis van der Stelt
Eventual Consistency

36. Dennis van der Stelt
Eventual Consistency

37. Dennis van der Stelt
Eventual Consistency
your “enterprise” is already eventual consistent with reality

38. Dennis van der Stelt
Eventual Consistency
your “enterprise” is already eventual consistent with reality

39. Dennis van der Stelt
“Allow things to be inconsistent and find ways to
compensate for mistakes, versus trying to prevent
mistakes altogether.”
EVENTUAL CONSISTENCY
Eric Brewer
VP of Infrastructure at Google, Professor at UC Berkeley

40. Dennis van der Stelt
100% synchronized
that‘s eventual consistency.
Udi Dahanin 2010

41. Dennis van der Stelt45
Idempotence
Once and only once delivery is hard!

42. Dennis van der Stelt

43. Dennis van der Stelt
IDEMPOTENCE
Solving the distributedtransactionissues
Sender
{0C836F44-6587-416E-B97A-5615615600D5}

44. Dennis van der Stelt
IDEMPOTENCE
Solving the distributedtransactionissues
Sender
{0C836F44-6587-416E-B97A-5615615600D5}

45. Dennis van der Stelt
IDEMPOTENCE
Solving the distributedtransactionissues
Sender
Event
Subscriber
{5EDC4993-AB01-4F17-A238-71C4521F750F}
{0C836F44-6587-416E-B97A-5615615600D5}

46. Dennis van der Stelt
IDEMPOTENCE
Solving the distributedtransactionissues
Sender
Event
Subscriber
{5EDC4993-AB01-4F17-A238-71C4521F750F}
{0C836F44-6587-416E-B97A-5615615600D5}

47. Dennis van der Stelt
Distributed Systems Principles
 8 Fallacies of distributed computing
 (Distributed) Transactions
 CAP Theorem & Eventual consistency
 Idempotence & outbox pattern

48. Dennis van der Stelt
Free eBook: Dr. Harvey and
the 8 Fallacies of Distributed
Computing
http://go.particular.net/ndclondon16

49. Dennis van der Stelt
find me.
http://dennis.bloggingabout.net
dvdstelt@outlook.com