This document discusses using CQRS and event sourcing on AWS. It highlights that the approach allows flexibility in data structures and enables learning from events and matches over time. While there are many components, it provides a scalable and flexible system. However, the separation of write and read capabilities may introduce some latency. Overall, CQRS and event sourcing on AWS is presented as a good fit for applications that require flexible data schemas and the ability to learn from historical events and user interactions.

1. CQRS AND EVENT SOURCING INCQRS AND EVENT SOURCING IN
THE AWS CLOUDTHE AWS CLOUD
@gunnarwrobel, JobMatchMe GmbH
1

2. WHY?WHY?
LEARNINGLEARNING
FLEXIBILITYFLEXIBILITY
2

3. ...provides the non-
academic workforce with
good jobs
3

4. 4

5. 5

6. "How do I find a decent job
as a truck driver?"
"DRIVING TRUCKS IS NOT"DRIVING TRUCKS IS NOT
A DECENT JOB!"A DECENT JOB!"
6

7. There are more than
500.000 truck drivers in
Germany.
Quelle: Statista
7

8. There are about 20.000
Trucking companies in
Germany.
Quelle: Bundesamt für Güterverkehr
8

9. There are about 45.000
open positions as truck
drivers officially listed at
the german Arbeitsagentur.
9

10. Logistic chains are
breaking becouse of small
trucking companies that
have no drivers anymore.
10

11. "I think customers should
stop expecting that there
are always melons
available at the
supermarket."
11

12. "I was wondering last
christmas that the whole
Amazon delivery business
still worked. I expected it to
break because of missing
drivers... maybe next time."
12

13. "Suddenly parcels didn't
get delivered anymore
because our subcontractor
didn't have any drivers."
13

14. WE DON'T SOLVE ALLWE DON'T SOLVE ALL
PROBLEMS...PROBLEMS...
... but there is a huge
market.
... and we can help.
14

15. We offer a market place for
drivers and trucking
companies.
15

16. "EVERY FEW HOURS A"EVERY FEW HOURS A
TRUCK DRIVER FALLS INTRUCK DRIVER FALLS IN
LOVE VIA TRUCK-JOBS.DELOVE VIA TRUCK-JOBS.DE
..."..."
16

17. We match drivers to job
offers according to their
capabilities and
preferences.
17

18. MY INITIAL SQL ASSUMPTIONMY INITIAL SQL ASSUMPTION
DRIVERDRIVER
Income
Location
Licences
JOBJOB
Income
Location
Licences
WRONGWRONG
18

19. LOCATIONLOCATION
A driver may be on the road for a week.
19

20. INCOMEINCOME
Drivers income is not a number.
Small base fee
Allowances
Bonus for night/weekend
...
20

21. MATCHING REQUIREMENTS?MATCHING REQUIREMENTS?
21

22. Essentially nothing.
Attributes may be missing.
22

23. Some roughly matching attributes.
Weak coupling between driver data + joboffer data.
23

24. We don't know.
Attributes will evolve over time.
24

25. Smells like NoSQL.
25

26. THE AGGREGATESTHE AGGREGATES
26

27. CHANGING AGGREGATESCHANGING AGGREGATES
27

28. LEARNING FROM MATCHESLEARNING FROM MATCHES
28

29. THE DECISIONSTHE DECISIONS
Event sourcing for flexible data structures and for
learning from matches
AWS for a flexible base that provides a rich toolbelt
and high availability
29

30. Browser
API Gateway API Gateway
Command
Event
Event
Dynamo DB
S3 Backup
Event Event Event
Event
Command
Lambdas
View
Projection
Lambdas
Redis
cache
View View
View
30

31. Browser
API Gateway API Gateway
Command
Event
Event
Dynamo DB
S3 Backup
Event Event Event
Event
Command
Lambdas
View
Projection
Lambdas
Redis
cache
View View
View
READWRITE
31

32. Command
Event
POST to /jobs/JOB-UUID/contact
Convert HTTP Request to Command
Convert Command to Event
Store Event in Dynamo DB
32

33. Command
Event
CONVERT HTTP REQUESTCONVERT HTTP REQUEST
TO COMMANDTO COMMAND
export const lambda = (
lambdaCommand: LambdaCommandType
) => {
const jobId = lambdaCommand.pathParameters.id
writeDomain({
name: 'addJobContact',
aggregate: {id: jobId, name: 'job'},
payload: lambdaCommand.body
})
}
33

34. Command
Event
CQRS/ES: CQRS.JS.ORGCQRS/ES: CQRS.JS.ORG
34

35. Command
Event
CONVERT COMMAND TOCONVERT COMMAND TO
EVENTEVENT
commands/addJobContact.js
import {defineCommand} from 'cqrs-domain'
export const command = defineCommand(
{ existing: true },
(command, aggregate) => {
aggregate.apply({
event: 'addedJobContact',
payload: command.payload
})
}
)
35

36. Command
Event
APPLY EVENT TO JOBAPPLY EVENT TO JOB
AGGREGATEAGGREGATE
events/addedJobContact.js
import {defineEvent} from 'cqrs-domain'
export const event = defineEvent(
{ payload: 'payload' },
(data, aggregate) => {
let contacts = []
if (aggregate.has('contacts')) {
contacts = aggregate.get('contacts')
}
aggregate.set(
'contacts',
contacts.concat(data)
)
})
36

37. Command
Event
DISADVANTAGES OF THEDISADVANTAGES OF THE
WRITE SIDEWRITE SIDE
Consumption only => HTTP 202 ACCEPTED
One Command, one Lambda (one Log, one
Command definition, one Event definition)
37

38. Command
Event
ADVANTAGES OF THE WRITEADVANTAGES OF THE WRITE
SIDESIDE
Data structures can be relaxed or restricted
by schemas
Incoming data can be mangled on different
levels
Supports DDD approach and the domain
model represents real world actions
38

39. Browser
API Gateway API Gateway
Command
Event
Event
Dynamo DB
S3 Backup
Event Event Event
Event
Command
Lambdas
View
Projection
Lambdas
Redis
cache
View View
View
READWRITE
39

40. Event
View
View View
View
For each new event ...
... fetch current view from Redis
... apply event to view
... store updated view in Redis
On (updated view) OR
(GET /job/JOB-UUID) ...
... fetch views from Redis
... aggregate to view
... (push) OR (provide) response
40

41. Event
View
View View
View
ES: CQRS.JS.ORGES: CQRS.JS.ORG
CREATE VIEWS FROM EVENTSCREATE VIEWS FROM EVENTS
(DENORMALIZATION / PROJECTION)(DENORMALIZATION / PROJECTION)
41

42. Event
View
View View
View
CREATE VIEWS FROMCREATE VIEWS FROM
EVENTSEVENTS
viewBuilders/
addedDriver.js
addedDriverComment.js
blacklistedDriver.js
deletedDriver.js
editedDriver.js
postponedDriver.js
receivedDriverMessage.js
sentDriverMessage.js
startedDriverCall.js
stoppedDriverCall.js
42

43. Event
View
View View
View
CONVERT EVENT TOCONVERT EVENT TO
VIEW MODELVIEW MODEL
addedDriver.js
export const viewBuilder =
defineViewBuilder(
{
name: 'addedDriver',
aggregate: 'driver',
payload: 'payload'
},
'create'
)
43

44. Event
View
View View
View
CONVERT EVENT TOCONVERT EVENT TO
VIEW MODELVIEW MODEL
postponedDriver.js
export const viewBuilder =
defineViewBuilder(
{
name: 'postponedDriver',
aggregate: 'driver',
payload: 'payload'
},
(data, vm) => {
vm.set(
'waitUntil',
data.timestamp
)})
44

45. Event
View
View View
View
DELIVER VIEWS FROMDELIVER VIEWS FROM
REDISREDIS
export const lambda = (
lambdaCommand: LambdaCommandType,
lambdaCallback: LambdaCallbackType
) => {
const driverId =
lambdaCommand.pathParameters.id
lambdaCallback(
new Promise((resolve, reject) => {
redisClient.get(
`driver:${driverId}`,
(err, result) => resolve(
JSON.parse(result)
))})}
45

46. Event
View
View View
View
DISADVANTAGES OF THEDISADVANTAGES OF THE
READ SIDEREAD SIDE
All the downsides of handling an event
log KanDDDinsky 2017: An Event Sourcing Retrospective
Developing a view requires coding view
builders / event denormalizers.
Asynchronous pipeline
46

47. Event
View
View View
View
ADVANTAGES OF THEADVANTAGES OF THE
READ SIDEREAD SIDE
Response data can be built on different
levels
You can iterate optimizations from
frontend, to view lambda, to event
projections
Scalable
Push model by default
47

48. Browser
API Gateway API Gateway
Command
Event
Event
Dynamo DB
S3 Backup
Event Event Event
Event
Command
Lambdas
View
Projection
Lambdas
Redis
cache
View View
View
48

49. SUMMARY - DISADVANTAGESSUMMARY - DISADVANTAGES
Write and read are seperate and may be slow
Many bits and pieces
Can be "Garbage in, garbage out"
49

50. SUMMARY - ADVANTAGESSUMMARY - ADVANTAGES
FLEXIBLEFLEXIBLE
Mangle data anywhere
Frontend (product) may rule
Scalable
LEARNINGLEARNING
Enables learning from day one
Makes your data analysts happy
Deals with real world events
50

51. THANKS!THANKS!
51