Risk Management in Retail with Stream Processing, Daniel Jagielski, Tech lead/Software Development Manager, VirtusLab/Tesco
Meetup link: https://www.meetup.com/Warsaw-Kafka-meetup/events/274900733/
2. Daniel Jagielski
Tech Lead & Manager at VirtusLab
Passionate about distributed systems
After hours:
Contact:
djagielski@virtuslab.com
kontakt@yagiel.pl
@danjagiel
11. Risk Engine V1
State stored
in memory
Locks
and critical
sections
Up to 3 DB
calls per
event
No horizontal
scalability
Slow event
processing
times
No high
availability
18. Risk Engine V2 – new principles
Let’s not create
a session
aggregate!
Focus on the
main goal
Single
responsibility
of each module
Independent,
parallel pipelines
19. Risk Engine V2 – core design
Calculate IP
statistics
Block
malicious IP
Analyse IP
statistics
24. Risk Engine V2 - benefits
Data locality
Scalability
No single point of
failure
Separation of
concerns
Smaller memory
footprint, less
CPU usage
Faster startup,
less state
Distributed
platform
Significant
performance
improvement