Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

3

Share

Download to read offline

Message Driven and Event Sourcing

Download to read offline

In questo talk, assieme ad Andrea Gioia, Partner di Quantyca, abbiamo spiegato il ruolo di Apache Kafka e come possano supportare architetture e soluzioni 'event driven' o perchè Kafka è un'ottima scelta per fare 'event sourcing'.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Message Driven and Event Sourcing

  1. 1. Change is the new normal
  2. 2. Message Driven & Event Sourced Andrea Gioia - Partner Quantyca Paolo Castagna - Account Executive, Confluent Message Driven & Event Sourced / quantyca.it | confluent.io
  3. 3. Scalability & Cloud / quantyca.it | google Andrea Gioia Partner, Quantyca Paolo Castagna Account Executive, Confluent
  4. 4. Intro Message Driven & Event Sourced / quantyca.it | confluent.io
  5. 5. Message driven Strong decoupling La necessità di delineare per ogni componente il giusto confine che possa garantire il più basso accoppiamento con gli altri, l’isolamento e la trasparenza sul dislocamento richiede di basare le interazioni all’interno delle moderne data platform sullo scambio asincrono di messaggi. Broker Message Driven & Event Sourced / quantyca.it | confluent.io
  6. 6. Inside the broker Smart endpoints and dumb pipes Attenzione a non spostare la complessità all’interno del broker. Le logiche di business dovrebbero essere il più possibili contenute nelle componenti che accedono al broker non nel broker stesso. = Big ball of mud Spaghetti in a box Message Driven & Event Sourced / quantyca.it | confluent.io
  7. 7. Due tipologie di broker Service Bus e Data Bus Message Driven & Event Sourced / quantyca.it | confluent.io
  8. 8. Due tipologie di broker Service Bus (orchestration over choreography) Nelle tradizionali architettura SOA il coordinamento dei servizi esposti da ogni sistema avviene tramite un Enterprise Service Bus (ESB) centralizzato. L’architettura basata su ESB permette di: 1. comporre servizi esposti da sistemi legacy 2. gestire transazioni distribuite 3. creare un anticorruption layer per isolare i sistemi monolitici 4. Facilitare il test e monitoraggio dell’infrastruttura Message Driven & Event Sourced / quantyca.it | confluent.io
  9. 9. Due tipologie di broker Data Bus (choreography over orchestration) Nelle moderne architettura a microservizi si rinuncia alla centralizzazione per aumentare la scalabilità del sistema. Ogni servizio è modellato intorna ad uno specifico dominio di business ed è completamente autonoma dagli altri dal punto di vista di : 1. Implementazione 2. Storage 3. Deploy L’allineamento a livello di dati tra i vari servizi avviene tramite un data bus basato su una piattaforma di event streaming distribuita (Kafka) Message Driven & Event Sourced / quantyca.it | confluent.io
  10. 10. Due tipologie di broker Architettura Ibrida e API Manager Le due architetture possono coesistere. Il layer di servizi avrà sempre più una doppia polarità: ● da una parte servizi esposti da sistemi legacy orchestrati tramite ESB (SOA) ● dall’altra servizi con coordinamento decentralizzato e altamente scalabili (Microservices) Per i consumatori di servizi questa architettura ibrida non sarà direttamente visibile in quanto l’accesso ai servizi passerà tramite API Gateway che medieranno l’accesso ai servizi reali Message Driven & Event Sourced / quantyca.it | confluent.io
  11. 11. How Apache Kafka and Confluent enable event streaming architectures Message Driven & Event Sourced / quantyca.it | confluent.io
  12. 12. Events An immutable record that something as some point has happened Message Driven & Event Sourced / quantyca.it | confluent.io A Sale An Invoice A Trade A Customer Experience
  13. 13. 5.2 Million Citizens Norwegian Work and Welfare Administration Life is a Stream of Events
  14. 14. Pre-Streaming Message Driven & Event Sourced / quantyca.it | confluent.io
  15. 15. Request-Response Applications Message Driven & Event Sourced / quantyca.it | confluent.io App Service Service Service Service Service Service Service Service App ■ DETERMINISTIC ■ RIGID ■ TIGHT
  16. 16. Event-Driven Applications Message Driven & Event Sourced / quantyca.it | confluent.io App App Developer APIs Service Service Service Service Service Streaming Platform ■ RESPONSIVE ■ FLEXIBLE ■ EXTENSIBLE
  17. 17. Moving from Pre-Streaming to Event Driven Architectures Request-Response vs Event-Driven: You Need Both! Message Driven & Event Sourced / quantyca.it | confluent.io Request-Response Event-Driven
  18. 18. Event Centric Thinking All Your Data is a Stream of Events Message Driven & Event Sourced / quantyca.it | confluent.io
  19. 19. The Event Streaming Platform Message Driven & Event Sourced / quantyca.it | confluent.io
  20. 20. Why Didn’t it Work Before? Past solutions turned out to be insufficient Message Driven & Event Sourced / quantyca.it | confluent.io Message-Oriented Middleware No persistence Single point of failure Not fault tolerant Cannot order messages Cannot process messages in flight Order of magnitude lower throughput No “replay” functionality EAI & ESBs Not event-oriented Fragile and bespoke Weak transformation capabilities ETL Slow and batch oriented Point-to-point, not publish & subscribe Not a true infrastructure platform
  21. 21. The World Has Changed Apache Kafka is at the core of these industry trends and, ultimately, the digitalisation of every business Message Driven & Event Sourced / quantyca.it | confluent.io Microservices Mobile Machine Learning Internet of Things
  22. 22. Message Driven & Event Sourced / quantyca.it | confluent.io The Event Streaming Platform We saw that picture before, what’s different now? ■ APACHE KAFKA CAN STORE EVENTS Kafka can store events for months, years, or indefinitely if useful. The limit is the amount of physical storage available on Kafka brokers. Events are stored in order on a per partition basis and they can be replayed and reconsumed (in order) whenever necessary. ■ EVENTS ARE STORED IN ORDER (PER PARTITION) Order of events is preserved and the ability to operate on a ‘stream’ level, also via Kafka Streams APIs or KSQL, provides a powerful way to simplify data integration, data cleansing, data transformations tasks as well as use Kafka to exchange and publish entire datasets. With Kafka, events are first class citizens, but also the full history and state is available to applications (it’s like having a database and a messaging system at the same time). ■ KAFKA CAN EASILY OPERATE A COMPANY SCALE Scalability, fault-tolerance, elasticity, and multi-tenancy are built-in capability.
  23. 23. Data Streaming and Maturity Stages Message Driven & Event Sourced / quantyca.it | confluent.io Value Maturity (Investment & time) 2 Enterprise Streaming Pilot / Early Production LOB(s) Pilot(s); Small teams experimenting; pub/sub / integration. → 1-3 use cases quickly moved into Production - but fragmented. Pub + Sub Store Process 5 All data in the organization managed through a single Streaming Platform. → Digital natives / digital pure players - probably using Machine Learning & AI (Relational databases - redundant). Central Nervous System 1 Developer Interest Developer downloads Kafka & experiments (15 mins on laptop). Pre-Streaming Legacy systems. Batch processes; → Complex → Slow / Silo’d → Expensive 4 Global Streaming Streaming Platform managing majority of mission critical data processes, globally, with multi-datacenter replication across on-prem and hybrid clouds. In parallel with other Big Data infrastructure. 3 SLA Ready, Integrated Streaming Multiple mission critical use cases in production, with; scale, DR & SLAs. → Streaming clearly delivering business value, with C-suite visibility. Projects Platform
  24. 24. Designing Event-Driven Systems Message Driven & Event Sourced / quantyca.it | confluent.io In this book Ben explains how service-based architectures and stream processing tools such as Apache Kafka can help you build business-critical systems. ● Understand why replayable logs such as Kafka provide a backbone for both service communication and shared datasets ● Explore how event collaboration and event sourcing patterns increase safety and recoverability with functional, event-driven approaches ● Apply patterns including Event Sourcing and CQRS, and how to build multi-team systems with microservices and SOA using patterns such as “inside out databases” and “event streams as a source of truth” ● Build service ecosystems that blend event-driven and request-driven interfaces using a replayable log and Kafka's Streams API ● Scale beyond individual teams into larger, department- and company-sized architectures, using event streams as a source of truth The book is available for free in PDF from the Confluent website! https://www.confluent.io/designing-event-driven-systems
  25. 25. Message Driven & Event Sourced / quantyca.it | confluent.io Royal Bank of Canada (RBC) We saw that picture before, what’s different now? https://www.confluent.io/customers/rbc ■ 16 Million Clients ■ 35 Countries ■ 30+ Use-cases ■ 50+ apps ■ 10+ lines of businesses “Kafka transforms even the most basic of initiatives. Adoption of Kafka at RBC has been massive and organic. Within the first six weeks after our launch of Kafka, we had 37 teams asking to use Kafka for various projects and initiatives.” -- Kerry Joel, Senior Director, Product Innovation, Data and Analytics Digital Marketi ng Securit y Consumer Credit Services SaaS Corporate Real Estate Investor Services Treasury Services …. FraudData Wareho use Microservices https://www.youtube.com/watch?v=WTxmHHJcHRc
  26. 26. The Streaming Platform Technical Capabilities Message Driven & Event Sourced / quantyca.it | confluent.io Publish & Subscribe Store Process
  27. 27. Takeaways Message Driven & Event Sourced / quantyca.it | confluent.io
  28. 28. In conclusione Elementi chiave 1. Service integration: esb, microservizi, api manager 2. Service portfolio 3. Competence portfolio 4. Piano di migrazione evolutivo Prossimi passi 1. Scalabilità 2. Governance non solo a livello dati ma anche a livello servizi Message Driven & Event Sourced / quantyca.it | confluent.io
  29. 29. Corso Milano, 45 / 20900 Monza (MB) T. +39 039 9000 210 / F. +39 039 9000 211 / @ info@quantyca.it www.quantyca.it
  • JeremyMartin39

    Sep. 8, 2020
  • quantycabi

    Jul. 6, 2018
  • andreagioia

    Jun. 27, 2018

In questo talk, assieme ad Andrea Gioia, Partner di Quantyca, abbiamo spiegato il ruolo di Apache Kafka e come possano supportare architetture e soluzioni 'event driven' o perchè Kafka è un'ottima scelta per fare 'event sourcing'.

Views

Total views

763

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

13

Shares

0

Comments

0

Likes

3

×