The document discusses human fault-tolerance in data systems. It argues that data systems should be designed to tolerate human mistakes by being immutable rather than mutable. Immutable systems capture a historical record of all changes over time, making it much harder for human errors to cause data loss or corruption compared to mutable systems. The conclusion is that systems should have an immutable source of truth storing the data, rather than a mutable database, to gain the benefits of human fault-tolerance that immutability provides.

1. Human fault-tolerance
Nathan Marz
Twitter 1

2. What is a data system?
A system that manages the storage and
querying of data

3. What is a data system?
A system that manages the storage and
querying of data with a lifetime measured in
years

4. What is a data system?
A system that manages the storage and
querying of data with a lifetime measured in
years encompassing every version of the
application to ever exist

5. What is a data system?
A system that manages the storage and
querying of data with a lifetime measured in
years encompassing every version of the
application to ever exist, every hardware
failure

6. What is a data system?
A system that manages the storage and
querying of data with a lifetime measured in
years encompassing every version of the
application to ever exist, every hardware
failure, and every human mistake ever
made

11. +

16. We don’t know how to make
perfect software

17. Paxos tunable consistency
replication active anti-entropy
CRC’s merkle trees
BASE read repair
quorum consistent hashing
Machine fault-tolerance

19. The worst consequence is
data loss or data corruption

20. As long as an error doesn’t
lose or corrupt good data,
you can fix what went wrong

21. This brings us to mutability

22. Mutability

23. The U and D in CRUD

24. Mutable systems inherently
lack human fault-tolerance

25. Very easy for a mistake to
corrupt or lose data

26. Immutability

27. Immutability
• An immutable system captures a historical record of events
• Each event happens at a particular time and is always true

28. Capturing change with mutable data
model
Person Location Person Location
Sally Philadelphia Sally New York
Bob Chicago Bob Chicago
Sally moves to New York

29. Capturing change with immutable
data model
Person Location Time Person Location Time
Sally Philadelphia 1318358351 Sally Philadelphia 1318358351
Bob Chicago 1327928370 Bob Chicago 1327928370
Sally New York 1338469380
Sally moves to New York

30. Immutability greatly restricts the
range of errors that can cause
data loss or data corruption

31. Vastly more human fault-tolerant

32. Plus much easier to reason about
systems based on immutability

33. What can we conclude?

34. Your source of truth should be
immutable

35. Mutable
Application
database
SOURCE OF TRUTH
Rather than build systems like this...

36. MySQL
Cassandra
HBase
Application
Voldemort
Riak
...
SOURCE OF TRUTH
Rather than build systems like this...

37. Immutable
View Application
data
SOURCE OF TRUTH
...build them like this

38. Cassandra
HBase
Voldemort
HDFS Application
Riak
ElephantDB
...
SOURCE OF TRUTH
...build them like this

39. Long live immutability
(forgive the pun)