Server side process does react to business events. User actions, timers, other services may trigger states of change, which should be tracked and propagated. A classical implementation is to define a shared database or centralized message queue, locally or in the cloud. When applied to Microservices, this monolithic approach will quickly appear as a limitation. In this session, we will introduce how Microservices could have their own dedicated persistence layer (i.e. database), and how real-time callbacks could be used to leverage real-time propagation of events, without any centralized message queue. Both ORM and SOA features of the Open Source mORMot framework will illustrate how to write such stand-alone services.

1. Microservices – Event Driven Systems
Arnaud Bouchez - Synopse
MICROSERVICES
Event Driven Design

2. Microservices – Event Driven Systems
• Arnaud Bouchez
– Delphi Expert
• Various solutions (from Vatican to gaming industry)
• IoT solution (RSI)
• Real-time Monitoring solution (LiveMon)
– Open Source
• mORMot (SOA ORM MVC framework)
• SynPDF SynMustache SynDB SynCrypto
– Training and consultancy
• Synopse one-man company

3. Microservices – Event Driven Systems
Event-Driven Systems
• Using Events?
• Event-Driven Design
• Event-Driven mORMot

4. Microservices – Event Driven Systems
Using Events?
• Imperative Architecture
– Events are high-level commands

5. Microservices – Event Driven Systems
Using Events?
• Imperative Architecture
– Events are high-level commands
• Easy modelization into regular code
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
var temp: TTemperature;
begin
temp := TemperatureSensor.GetTemperature;
if temp > Threshold then
RoboValet.FetchDownJacket(Sender);
end;

6. Microservices – Event Driven Systems
Using Events?
• Imperative Architecture
– State is stored in a main shared DB
– Events are high-level commands
translated into SQL statements

7. Microservices – Event Driven Systems
Using Events?
• Imperative Architecture
– State is stored in a main shared DB
– Events are high-level commands
translated into SQL statements
• DB-centric approach
• Atomicity is ensured, but history is lost
• DB becomes a bottleneck

8. Microservices – Event Driven Systems
Using Events?
• Event-Driven Architecture
– Collaboration is done using Events

9. Microservices – Event Driven Systems
Using Events?
• Event-Driven Architecture
– Collaboration is
done using Events
Just like
VCL/FMX /LCL !

10. Microservices – Event Driven Systems
Using Events?
• Event-Driven Architecture
– Collaboration is done using Events
– System consists in uncoupled nodes
which could be added or removed on-the-fly
– Nodes publish events, and subscribe to them
– Each node stores the information it needs
– Some API gateways expose state via REST

11. Microservices – Event Driven Systems
Event-Driven Architecture
• Nodes implemented as services
procedure TZen.Init;
begin
TemperatureSensor.SubscribeToTempChange(OnTempChanged);
end;
procedure TZen.OnTempChanged(NewTemp: TTemperature);
begin
CurrentTemp := NewTemp;
end;
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
begin
if CurrentTemp > Threshold then
RoboValet.NotifyNeedsJacket(Sender);
end;

12. Microservices – Event Driven Systems
Event-Driven Architecture
– Collaboration is done using Events
Not natural to imperative code (Delphi, C#, Java)
– System consists in uncoupled nodes
Architecture becomes complex
– Nodes publish events, and subscribe to them
May be subject to unexpected loops
– Each node stores the information it needs
Duplicated data, with eventual consistency

13. Microservices – Event Driven Systems
Event-Driven Architecture
• More complex than imperative code
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
var temp: TTemperature;
begin
temp := TemperatureSensor.GetTemperature;
if temp > Threshold then
RoboValet.FetchDownJacket(Sender);
end;

14. Microservices – Event Driven Systems
Event-Driven Architecture
• Events may be received for no use
e.g. the following method is always called
procedure TZen.OnTempChanged(NewTemp: TTemperature);
begin
CurrentTemp := NewTemp;
end;
but this value may never be used in
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
begin
if CurrentTemp > Threshold then
RoboValet.NotifyNeedsJacket(Sender);
end;

15. Microservices – Event Driven Systems
Event-Driven Architecture
• Danger of unexpected infinite loop
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
begin
if CurrentTemp > Threshold then
RoboValet.NotifyNeedsJacket(Sender);
end;
What happens
if RoboValet.NotifyNeedsJacket
triggers (indirectly) TZen.HandleIAmGoingOut ?
Since nodes may be added at any time,
how to prevent it to happen?

16. Microservices – Event Driven Systems
Event-Driven Architecture
• Danger of unexpected infinite loop
procedure TZen.HandleIAmGoingOut(Sender: TPeople);
begin
if CurrentTemp > Threshold then
RoboValet.NotifyNeedsJacket(Sender);
end;
What happens
if RoboValet.NotifyNeedsJacket
triggers (indirectly) TZen.HandleIAmGoingOut ?
Enter the Event-Driven debugging hell:
actual event process depends on the system it runs on

17. Microservices – Event Driven Systems
Event-Driven Architecture
• So, when to use it?
– Your are modeling evolving information
• e.g. scale to a lot of devices or users
– You need to track changes of state
• A monolithic DB only stores the current state
(via Update) - otherwise it may sink under Inserts
• Event-Sourcing and CQRS (see later)

18. Microservices – Event Driven Systems
Event-Driven Architecture
• So, when to use it?
– You expect a modular design
• Process not written in stone during compilation
(as in imperative programming): add/enable nodes
to change the system behavior at runtime
• Microservices, SaaS, horizontal scaling

19. Microservices – Event Driven Systems
Event-Driven Architecture
• So, when to use it?
– You defined a stateless domain
• From state comes complexity,
so DDD usually modelizes a time snapshot
• DDD events are a way to introduce temporality
– Never 100% Event-Driven
• You will eventually define REST API gateways

20. Microservices – Event Driven Systems
Event-Driven Architecture
• Message Bus
– Most common way to implement Event-Driven
• Cloud, Local or in-house Queues
• Like VCL TApplication.ProcessMessages
– Publish/Subscribe mechanism
• Nodes get only needed information
• Nodes could be added/removed
– Used in conjunction with regular DB and REST

21. Microservices – Event Driven Systems
Event-Driven Architecture
• Message Bus

22. Microservices – Event Driven Systems
Event-Driven Architecture
• Peer to peer communication
– Nodes communicate directly between them
• Each node manages its own queue(s)
• Good integration with REST
• Need a centralized catalog service, P2P ZeroMQ,
or convention-over-configuration setup
• As offered by mORMot over WebSockets

23. Microservices – Event Driven Systems
Event-Driven Architecture
• Event Sourcing
– Business logic is implemented through Events
– Events are persisted, not state
• Play the events to compute a state
• Intermediate states may be cached/persisted
– Events are transmitted, not state
• Publish/Subscribe instead of REST
• Define API gateways for the outer world

24. Microservices – Event Driven Systems
Event-Driven Architecture
• Event Sourcing
– Business logic is implemented through Events
– Convenient way to define Microservices
• Subscribe to events, to get the information
• Publish events, as result of the process
– Microservices persistence
• Most Microservices will use a regular state storage
• Some Microservices will use Event Sourcing

25. Microservices – Event Driven Systems
Event-Driven Architecture
• Events and CQRS
– Command Query Responsibility Segregation
• Commands triggers events
• Events are gathered and stored
• State/Events store is uncoupled from Commands
– CQRS follows a non CRUD pattern
• Create Read Update Delete

26. Microservices – Event Driven Systems
Event-Driven Architecture
• CRUD
DB

27. Microservices – Event Driven Systems
Event-Driven Architecture
• CQRS
DB

28. Microservices – Event Driven Systems
Event-Driven mORMot

29. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services
type
ICalculator = interface(IInvokable)
['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
/// add two signed 32 bit integers
function Add(n1,n2: integer): integer;
end;
– Defines the contract of the service
– Microservice: should follow SOLID principles

30. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services
procedure ResolveCalculator(out: ICalculator);
...
function TMyClient.MyAdd(a,b: integer): integer;
var Calculator: ICalculator;
begin
Client.Services.Resolve(ICalculator, Calculator);
result := Calculator.Add(a,b);
end;
– Runtime injection of the actual implementation
• May be in-process, local or remote

31. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services
var Calculator: ICalculator;
begin
Client.Services.Resolve(ICalculator, Calculator);
result := Calculator.Add(a,b);
end;
– Client is a mORMot TSQLRestClientURI
– Will communicate using JSON over HTTP(S)

32. Microservices – Event Driven Systems
Event-Driven mORMot
• RESTful Local and Client-Server
– In-process
Stand-alone client, fast server-side access
– Named pipes or Windows messages
Stand-alone client, fast server-side access
– HTTP/1.1 via kernel-mode http.sys API
• Kernel-mode execution, IOCP driven, https – Windows specific
– HTTP/1.1 via OS socket API
• Windows or Linux, using a Thread Pool, https via Reverse Proxy
• Upgradable to WebSockets

33. Microservices – Event Driven Systems
Event-Driven mORMot
• RESTful Client
TSQLRestServer
TSQLRest
TSQLRestClientURI
TSQLRestClient

34. Microservices – Event Driven Systems
Event-Driven mORMot
• RESTful Server
TSQLRestServerDB
TSQLRestServer
TSQLRest
TSQLRestServerRemoteDB TSQLRestServerFullMemory

35. Microservices – Event Driven Systems
Event-Driven mORMot
• RESTful Local and Client-Server
– Needed methods are available at TSQLRest
• REST auto-routing
• JSON high-performance serialization
• Following Liskov Substitution Principle
– Focus on the logic
– Abstract from the plumbing

36. Microservices – Event Driven Systems
Event-Driven mORMot
• REST/JSON Persistence via ORM/ODM
– Agnostic Storage for Events or State
• ORM over local SQlite3 store
• ORM over remote SQL databases
• ODM over TObjectList
• ODM over MongoDB
Switching from one to another
in a few lines of code, or through settings

37. Microservices – Event Driven Systems
Event-Driven mORMot
• REST/JSON Persistence via ORM/ODM
– Advanced ORM features
• REST Convention-Over-Configuration routing
• Direct DB-to-JSON serialization (no TDataSet)
• Built-in statement and JSON cache
• Batch (Unit-Of-Work) writes: bulk SQL/NoSQL insert
• SQL logging/timing, real-time replication,
data sharding, remote proxying

38. Microservices – Event Driven Systems
Event-Driven mORMot
• Cross-platform
– Main mORMot units
• Delphi: Windows x86/x64
Best IDE experience for coding and debugging
• FPC: Windows, MacOS, Android, Linux, BSD
x86/x64, ARM32/AARCH64 (PPC32/PPC64)
• (Cross)Kylix: Linux x86

39. Microservices – Event Driven Systems
Event-Driven mORMot
• Cross-platform
– Generated client code using Mustache templates
• Delphi: Windows, MacOS, Android, iOS
• FPC: Windows, MacOS, Android, iOS, Linux, …
• (Cross)Kylix: Linux
• SmartMobileStudio: JavaScript Ajax / HTML5
– Featuring almost all framework abilities
• JSON marshalling, security, instance lifetime

40. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Specify the callback as an interface parameter
• Native way of coding in object pascal
– Real-time push notifications over WebSockets
• Upgraded from a standard HTTP connection
– Peer To Peer communication
• No need of a centralized message bus / server

41. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Real-time push notifications over WebSockets

42. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Specify the callback as an interface parameter
ILongWorkCallback = interface(IInvokable)
...
end;
ILongWorkService = interface(IInvokable)
['{09FDFCEF-86E5-4077-80D8-661801A9224A}']
procedure StartWork(const workName: string;
const onFinish: ILongWorkCallback);
function TotalWorkCount: Integer;
end;

43. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Real-time push notifications over WebSockets
• Once upgraded, communicates using frames
over a bi-directional socket connection
using application-level protocols
• Security, frames gathering, REST emulation

44. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Real-time push notifications over WebSockets
• Once upgraded, communicates using frames
over a bi-directional socket connection
using application-level protocols

45. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Security, Frames gathering, REST emulation
• TWebSocketProtocolBinary = SynLZ + AES-256
• Regular blocking methods expecting results
will be emulated like regular REST requests,
with REST security and parameters marshalling
• Non-blocking methods with no result
will be asynchronously sent,
optionally gathered as “jumbo frames”

46. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Publish/Subscribe Pattern
also known as “Observer”
Publisher
Subscriber 1
Event
Subscriber 2
Event
Subscriber 3

47. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Beware of “Race Conditions”
• Use critical sections (e.g. TSynLocker)
to protect your shared data on multi-threaded server
• You can gather all non-blocking callback process in a
background thread via TSQLRest.AsynchRedirect

48. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Beware of “Dead Locks”
• If your callback triggers another method
which shares the same critical section
in another thread, you may block both threads
• You can gather all non-blocking callback process in a
background thread via TSQLRest.AsynchRedirect

49. Microservices – Event Driven Systems
Event-Driven mORMot
• Interface-based services callbacks
– Sample 31
• Long-Work Push Notification
also known as “Sagas”
• Publish/Subscribe Pattern
also known as “Observer”

50. Microservices – Event Driven Systems
• Credits
Martin Fowler
https://martinfowler.com
code and samples at
https://synopse.info

51. Microservices – Event Driven Systems
Event-Driven Systems
• Questions?