In 2009, I first learned about Event Sourcing and Command Query Responsibility Seggregation (CQRS) at a training Greg Young gave in Utrecht, The Netherlands. I remembered to be awed by the scalability and architectural simplicity those styles provided. However, I also remembered the technical complexity that comes with it. In 2012, I was in charge of transitioning a CQRS-based system to Event Sourcing. I knew it would be non-trivial, but boy was I in for a surprise. So over the last four years I've experienced first-hand how a large group of developers had to deal with the transition. It's a brilliant solution for high-performance or complex business systems, but you need to be aware that this also introduces challenges most people don't tell you about. In this talk, I'd like to share you some of the most powerful benefits of ES, but also show you the flipside of the coin and cover some of the smaller and bigger challenges you'll run into it. Again, I love it and would apply it again without any doubt, but I really want you to understand the trade-offs before you jump on the Event Sourcing train.

1. Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver

3. Web
Application
Command
Service
Query
Processor
ChangeUser
EmailHandler
User
Unit of
Work
User
Projector
DAL
Query
Handler
Read DB
Write DB
ChangeUserEmailCommand
Execute query
Get<User>(identity)
Invoke method
Event Store
Load(events)
Apply
Get changes
Dispatcher Events
Handle(UserEmailChangedEvent)
Submit changes
History
Query Language / SQL
Dennis Doomen | @ddoomen | The Continuous Improver

4. The Good

5. Domain
Event Store
Events
App
RDBMS
Events
Projection
EventsProjection
Projection
Projector
Optimized for
specific queries
Separate
projections
database NoSQL
Projector
Projection-specific
storage
Projector
HTML
Raw SQL or
Dapper
Run
asynchronously
Great for
sharding
Dennis Doomen | @ddoomen | The Continuous Improver

6. Events
Transaction 6
Transaction 5
Transaction 4
Transaction 3
Transaction 2
Transaction 1
Temporal
Projector
Read
Store
Time
Projected until
this point
Immutable, thus
auditable and SOX
compliance
Dennis Doomen | @ddoomen | The Continuous Improver

7. Domain
Event Store
Events
App
Events
Projection
EventsProjection
Projection
Projector
Application
projections
RDBMS
Reporting
Projector
Traditional
reporting model
Asynchronous
OLAP
Dennis Doomen | @ddoomen | The Continuous Improver

8. StreamId: User-Dedo, Rev: 5
Persisted State Changes
StreamId: User-Dedo, Rev: 4
GrantedRoleEvent
PasswordChangedEvent
StreamId: User-Dedo, Rev: 4
StreamId: User-Dedo, Rev: 3
UserCreatedEvent
GrantedRoleEvent
PhoneNumberAddedEven
t
PasswordChangedEvent
StreamId: User-Dedo, Rev: 5
RoleRevokedEvent
StreamId: User-Dedo, Rev: 6
GrantedRoleEvent
Time
StreamId: User-Dedo, Rev: 7
PasswordChangedEvent
Dennis Doomen | @ddoomen | The Continuous Improver

9. Changes
1
2
3
6297
6298
6298
Query Data
Documents
Groups
Users
Objects
Folders
User
Interface
Changes Queries
Changes
1
2
3
6299
6300
6301
Query Data
Documents
Groups
Users
Objects
Folders
User
Interface
ChangesQueries
Node Node
Replication Process
Dennis Doomen | @ddoomen | The Continuous Improver

10. Events
Aggregate
Root
Entity
Entity
Value
Object
Aggregate
Root
Aggregate
Root
Aggregate
Root
Entity
Value
Object
Value
Object
Dennis Doomen | @ddoomen | The Continuous Improver

11. The Bad

12. • Designing your domain based on ownership
• Relying on eventual consistent projections
• Bad choice in functional keys (e.g. username vs ID)
• Running business logic after an domain event is raised
• Non-primitives in events
Dennis Doomen | @ddoomen | The Continuous Improver

13. First
• Ask the difficult questions about consistency
• Understand the real world
Then use these guidelines
• Driven by invariants, not composition
• Only consistent within aggregate
• Small aggregates
• Reference by identity.
Dennis Doomen | @ddoomen | The Continuous Improver

14. public class Order
{
private Status status;
public void Cancel()
{
if (status == Status.InProgress)
{
Apply(new OrderCanceledEvent());
}
}
private void When(OrderCanceledEvent e)
{
status = Status.Canceled;
}
}
In single classes
Dennis Doomen | @ddoomen | The Continuous Improver

15. public partial class Order
{
private Status status;
public void Cancel()
{
if (status == Status.InProgress)
{
Apply(new OrderCanceledEvent());
}
}
}
In partial classes
public partial class Order
{
private void When(OrderCanceledEvent e)
{
status = Status.Canceled;
}
}
Dennis Doomen | @ddoomen | The Continuous Improver

16. public class Order
{
private OrderState state;
public void Cancel()
{
if (state.Status == Status.InProgress)
{
state.Apply(new OrderCanceledEvent());
}
}
}
In separate classes
internal class OrderState
{
public Status Status { get; set; }
private void When(OrderCanceledEvent e)
{
Status = Status.Canceled;
}
}
Dennis Doomen | @ddoomen | The Continuous Improver

17. Risk Assessment
Level Changed
Risk Assessment
Team Member
Removed
Risk Assessment
Team Member
Removed
Risk Assessment
Level Demoted
Versus
Dennis Doomen | @ddoomen | The Continuous Improver

18. Assembly
AvSol.Domain.dll
Namespace
AvSol.Domain.RiskAssessments
Type
RiskAssessmentLevelChanged
Assembly
AvSol.ThreadAssessment.dll
Namespace:
AvSol.ThreadAssessment.Risks.Domai
n
Type
RiskAssessmentLevelDowngraded
Dennis Doomen | @ddoomen | The Continuous Improver

19. The Ugly

20. Domain
Event Store
Events
App
Query
Store
RavenDB
Projectors
Events
API Controller
Projections
Events
Dennis Doomen | @ddoomen | The Continuous Improver

21. • Rebuilding time with large databases
– Functional archiving
– Projection tracking and prediction
– Clear separation between critical and auxiliary data ->
prioritization
– Partitioning.
• Dynamic rebuilding (after bugs, schema changes, etc)
• Projection state & consistency
– Idempotency of projections
– Projection autonomy -> more duplication.
Dennis Doomen | @ddoomen | The Continuous Improver

22. • Projections that crash
– Column constraints
– Changes in data invariants
– Unexpected projection dependencies
– Lookups vs stream dependencies.
• (Unexpected) projection dependencies
– Synchronous -> asynchronous and existing bugs
– Aggregrate dependencies.
• Bugs
– Causing large event streams.
Dennis Doomen | @ddoomen | The Continuous Improver

23. Solutions

24. • Liquid Projections
– Designed as a library
– Promotes fully autonomous projections
– Local tracking
– ETA calculations
• NEventStore
• GetEventStore
• Apache Kafka?
Dennis Doomen | @ddoomen | The Continuous Improver

26. • Effective Aggregate Design (Vaughn Vernon)
http://dddcommunity.org/library/vernon_2011/
• Liquid Projections
https://github.com/dennisdoomen/LiquidProjections
• Distributed Event Sourcing (Slides)
http://www.slideshare.net/dennisdoomen/building-occasionally-
connected-applications-using-event-sourcing
• Data schema changes in an event sourced
system (paper)
http://files.movereem.nl/2017saner-eventsourcing.pdf