Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@crichardson ACID Is So Yesterday: Maintaining Data Consistency with Sagas Chris Richardson Founder of Eventuate.io Founde...
@crichardson Presentation goal Distributed data management challenges in a microservice architecture Sagas as the transact...
@crichardson About Chris
@crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http...
@crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development...
@crichardson For more information http://learnmicroservices.io
@crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
The microservice architecture structures an application as a set of loosely coupled services
@crichardson Microservices enable continuous delivery/deployment Process: Continuous delivery/deployment Organization: Sma...
@crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Customer Service Order Service … S...
@crichardson Private database != private database server
@crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order tabl...
@crichardson How to maintain data consistency?!?!? Invariant: sum(open order.total) <= customer.creditLimit
@crichardson Cannot use ACID transactions BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELEC...
@crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least...
@crichardson Basically Available Soft state Eventually consistent http://queue.acm.org/detail.cfm?id=1394128 ACID
@crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
@crichardson From a 1987 paper
@crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service...
@crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local tr...
@crichardson If only it were this easy…
@crichardson Rollback using compensating transactions ACID transactions can simply rollback BUT Developer must write appli...
@crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions T1 T2 C1 FAILS
@crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local trans...
@crichardson Sagas complicate API design Synchronous API vs Asynchronous Saga Request initiates the saga. When to send bac...
@crichardson Revised Create Order API createOrder() returns id of newly created order NOT fully validated getOrder(id) Cal...
@crichardson Minimal impact on UI UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms...
@crichardson Lack of isolation complicates business logic Order Service Local transaction Order state=PENDING createOrder(...
@crichardson How to cancel a PENDING Order? Don’t throw an OrderNotCancellableException Questionable user experience “Inte...
@crichardson Countermeasure Transaction Model
@crichardson Saga structure Series of compensatable transactions (Ti,Ci) Pivot transaction (Ti) “not compensatable or retr...
@crichardson Sagas are ACD Atomicity Saga implementation ensures that all transactions are executed OR all are compensated...
@crichardson Lack of I anomalies Lost update Ti reads other transaction writes Tj (or Ci) writes Dirty reads Ti writes oth...
@crichardson Countermeasures for reducing impact of isolation anomalies… Commutative updates e.g. debit account can compen...
@crichardson …Countermeasures for reducing impact of isolation anomalies… Re-read value Before modifying value, Ti re-read...
@crichardson ...Countermeasures for reducing impact of isolation anomalies Countermeasures by value Business risk determin...
@crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
@crichardson How to sequence the saga transactions? After the completion of transaction Ti “something” must decide what st...
@crichardson Choreography: distributed decision making vs. Orchestration: centralized decision making
@crichardson Option #1: Choreography-based coordination using events Order Service Customer Service Order created Credit R...
Benefits and drawbacks of choreography Benefits Simple, especially when using event sourcing Participants are loosely couple...
@crichardson Order Service Option #2: Orchestration-based saga coordination Local transaction Order state=PENDING createOr...
@crichardson A saga (orchestrator) is a persistent object that tracks the state of the saga and invokes the participants
@crichardson Saga behavior On create: Invokes a saga participant On reply: Determine which saga participant to invoke next...
@crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservati...
@crichardson CreateOrderSaga definition Sequence of steps step = (Ti, Ci) Build command to send Saga’s Data
@crichardson Customer Service command handler Route command to handler Reserve credit Make reply message
@crichardson Eventuate Tram Sagas Open-source Saga framework Currently for Java https://github.com/eventuate-tram/eventuat...
Benefits and drawbacks of orchestration Benefits Centralized coordination logic is easier to understand Reduced coupling, e....
@crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
@crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant command...
@crichardson Use asynchronous messaging Ensures sagas complete when participants are temporarily unavailable
@crichardson Create Order Saga - messaging Order Service Create Order Saga Message Broker Customer Service Customer Custom...
@crichardson Messaging must be transactional Service Database Message Broker update publish How to make atomic without 2PC?
@crichardson Option #1: Use database table as a message queue ACID transaction See BASE: An Acid Alternative, http://bit.l...
@crichardson Publishing messages Poll the MESSAGE table (ok) OR Tail the database transaction log (better)
@crichardson Eventuate Tram Open-source framework for transactional messaging Send and receive messages Publish and subscr...
@crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table ...
@crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loos...
@crichardson @crichardson chris@chrisrichardson.net http://learnmicroservices.io Questions?
Upcoming SlideShare
Loading in …5
×

QCONSF - ACID Is So Yesterday: Maintaining Data Consistency with Sagas

8,790 views

Published on

This is a presentation I gave at QCONSF 2017

The services in a microservice architecture must be loosely coupled and so cannot share database tables. What’s more, two phase commit (a.k.a. a distributed transaction) is not a viable option for modern applications. Consequently, a microservices application must use the Saga pattern, which maintains data consistency using a series of local transactions.

In this presentation, you will learn how sagas work and how they differ from traditional transactions. We describe how to use sagas to develop business logic in a microservices application. You will learn effective techniques for orchestrating sagas and how to use messaging for reliability. We will describe the design of a saga framework for Java and show a sample application.

Published in: Software
no profile picture user

  • Login to see the comments

Show More

QCONSF - ACID Is So Yesterday: Maintaining Data Consistency with Sagas

  1. 1. @crichardson ACID Is So Yesterday: Maintaining Data Consistency with Sagas Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
  2. 2. @crichardson Presentation goal Distributed data management challenges in a microservice architecture Sagas as the transaction model
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
  5. 5. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
  6. 6. @crichardson For more information http://learnmicroservices.io
  7. 7. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
  8. 8. The microservice architecture structures an application as a set of loosely coupled services
  9. 9. @crichardson Microservices enable continuous delivery/deployment Process: Continuous delivery/deployment Organization: Small, agile, autonomous, cross functional teams Architecture: Microservice architecture Enables Enables Enables Successful Software Development Services = testability and deployability Teams own services
  10. 10. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Customer Service Order Service … Service Customer Database Order Database … Database HTML REST REST Database per service
  11. 11. @crichardson Private database != private database server
  12. 12. @crichardson Loose coupling = encapsulated data Order Service Customer Service Order Database Customer Database Order table Customer table orderTotal creditLimit
  13. 13. @crichardson How to maintain data consistency?!?!? Invariant: sum(open order.total) <= customer.creditLimit
  14. 14. @crichardson Cannot use ACID transactions BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service Distributed transactions
  15. 15. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
  16. 16. @crichardson Basically Available Soft state Eventually consistent http://queue.acm.org/detail.cfm?id=1394128 ACID
  17. 17. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
  18. 18. @crichardson From a 1987 paper
  19. 19. @crichardson Saga Use Sagas instead of 2PC Distributed transaction Service A Service B Service A Local transaction Service B Local transaction Service C Local transaction X Service C
  20. 20. @crichardson Order Service Create Order Saga Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder()
  21. 21. @crichardson If only it were this easy…
  22. 22. @crichardson Rollback using compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions Careful design required!
  23. 23. @crichardson Saga: Every Ti has a Ci T1 T2 … C1 C2 Compensating transactions T1 T2 C1 FAILS
  24. 24. @crichardson Order Service Create Order Saga - rollback Local transaction Order createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order reject order() createOrder() FAIL Insufficient credit
  25. 25. @crichardson Sagas complicate API design Synchronous API vs Asynchronous Saga Request initiates the saga. When to send back the response? Option #1: Send response when saga completes: + Response specifies the outcome - Reduced availability Option #2: Send response immediately after creating the saga (recommended): + Improved availability - Response does not specify the outcome. Client must poll or be notified
  26. 26. @crichardson Revised Create Order API createOrder() returns id of newly created order NOT fully validated getOrder(id) Called periodically by client to get outcome of validation
  27. 27. @crichardson Minimal impact on UI UI hides asynchronous API from the user Saga will usually appear instantaneous (<= 100ms) If it takes longer UI displays “processing” popup Server can push notification to UI
  28. 28. @crichardson Lack of isolation complicates business logic Order Service Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction cancelOrder() ? Time
  29. 29. @crichardson How to cancel a PENDING Order? Don’t throw an OrderNotCancellableException Questionable user experience “Interrupt” the Create Order saga? Cancel Order Saga: set order.state = CANCELLED Causes Create Order Saga to rollback But is that enough to cancel the order? Cancel Order saga waits for the Create Order saga to complete? Suspiciously like a distributed lock But perhaps that is ok
  30. 30. @crichardson Countermeasure Transaction Model
  31. 31. @crichardson Saga structure Series of compensatable transactions (Ti,Ci) Pivot transaction (Ti) “not compensatable or retriable” Execute compensating transactions if it fails GO/NO GO point Set of retriable transactions (Ti) Can't fail
  32. 32. @crichardson Sagas are ACD Atomicity Saga implementation ensures that all transactions are executed OR all are compensated Consistency Referential integrity within a service handled by local databases Referential integrity across services handled by application Durability Durability handled by local databases
  33. 33. @crichardson Lack of I anomalies Lost update Ti reads other transaction writes Tj (or Ci) writes Dirty reads Ti writes other transaction reads Ci writes non-repeatable/fuzzy read Ti reads other transaction writes Tj reads
  34. 34. @crichardson Countermeasures for reducing impact of isolation anomalies… Commutative updates e.g. debit account can compensate for a credit account Version file Record history of changes Use them to make updates commutative e.g. record cancel reservation so that create/cancel = cancel/ create Sounds suspiciously like event sourcing
  35. 35. @crichardson …Countermeasures for reducing impact of isolation anomalies… Re-read value Before modifying value, Ti re-reads value that was read by a previous Ti Abort if the value has changed (and possibly restart) Pessimistic view Minimize the business risk Reduce available credit in compensatable transaction Increase available credit in retriable transaction, which will never be compensated
  36. 36. @crichardson ...Countermeasures for reducing impact of isolation anomalies Countermeasures by value Business risk determine strategy High risk => use 2PC/distributed transaction Semantic lock Compensatable transaction sets flag, retriable transaction releases it Flag = lock - prevents other transactions from accessing it Flag = warning - treat the data differently, e.g. a pending deposit Require deadlock detection, e.g. timeout
  37. 37. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
  38. 38. @crichardson How to sequence the saga transactions? After the completion of transaction Ti “something” must decide what step to execute next Success: which T(i+1) - branching Failure: C(i - 1)
  39. 39. @crichardson Choreography: distributed decision making vs. Orchestration: centralized decision making
  40. 40. @crichardson Option #1: Choreography-based coordination using events Order Service Customer Service Order created Credit Reserved Credit Limit Exceeded Create Order OR Customer creditLimit creditReservations ... Order state total … create() reserveCredit() approve()/reject()
  41. 41. Benefits and drawbacks of choreography Benefits Simple, especially when using event sourcing Participants are loosely coupled Drawbacks Cyclic dependencies - services listen to each other’s events Overloads domain objects, e.g. Order and Customer know too much Events = indirect way to make something happen
  42. 42. @crichardson Order Service Option #2: Orchestration-based saga coordination Local transaction Order state=PENDING createOrder() Customer Service Local transaction Customer reserveCredit() Order Service Local transaction Order state=APPROVED approve order() createOrder() CreateOrderSaga
  43. 43. @crichardson A saga (orchestrator) is a persistent object that tracks the state of the saga and invokes the participants
  44. 44. @crichardson Saga behavior On create: Invokes a saga participant On reply: Determine which saga participant to invoke next Invokes saga participant Updates its state …
  45. 45. @crichardson Order Service CreateOrderSaga orchestrator Customer Service Create Order Customer creditLimit creditReservations ... Order state total… reserveCredit() CreateOrder Saga OrderService create() create() approve() creditReserved()
  46. 46. @crichardson CreateOrderSaga definition Sequence of steps step = (Ti, Ci) Build command to send Saga’s Data
  47. 47. @crichardson Customer Service command handler Route command to handler Reserve credit Make reply message
  48. 48. @crichardson Eventuate Tram Sagas Open-source Saga framework Currently for Java https://github.com/eventuate-tram/eventuate-tram-sagas
  49. 49. Benefits and drawbacks of orchestration Benefits Centralized coordination logic is easier to understand Reduced coupling, e.g. Customer knows less Reduces cyclic dependencies Drawbacks Risk of smart sagas directing dumb services
  50. 50. @crichardson Agenda ACID is not an option Overview of sagas Coordinating sagas Sagas and inter-service communication
  51. 51. @crichardson Saga Participant About Saga orchestrator participant communication Saga Orchestrator Saga Participant command reply Saga must complete even if there are transient failures
  52. 52. @crichardson Use asynchronous messaging Ensures sagas complete when participants are temporarily unavailable
  53. 53. @crichardson Create Order Saga - messaging Order Service Create Order Saga Message Broker Customer Service Customer Customer Request Channel Saga Reply Channel Reserve Credit Reserve Credit Reply
  54. 54. @crichardson Messaging must be transactional Service Database Message Broker update publish How to make atomic without 2PC?
  55. 55. @crichardson Option #1: Use database table as a message queue ACID transaction See BASE: An Acid Alternative, http://bit.ly/ebaybase DELETE ? Customer Service ORDER_ID CUSTOMER_ID TOTAL 99 CUSTOMER_CREDIT_RESERVATIONS table 101 1234 ID TYPE DATA DESTINATION MESSAGE table 84784 CreditReserved {…} … INSERT INSERT Message Publisher QUERY Message Broker Publish Local transaction reserveCredit()
  56. 56. @crichardson Publishing messages Poll the MESSAGE table (ok) OR Tail the database transaction log (better)
  57. 57. @crichardson Eventuate Tram Open-source framework for transactional messaging Send and receive messages Publish and subscribe to domain events Send commands and replies Currently, for Java https://github.com/eventuate-tram/eventuate-tram-core
  58. 58. @crichardson Option #2: Event sourcing: event-centric persistence Service Event Store save events and publish Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated Every state change event
  59. 59. @crichardson Summary Microservices tackle complexity and accelerate development Database per service is essential for loose coupling Use sagas to maintain data consistency across services Use transactional messaging to make sagas reliable
  60. 60. @crichardson @crichardson chris@chrisrichardson.net http://learnmicroservices.io Questions?

×