SlideShare a Scribd company logo
1 of 91
Download to read offline
Systematic
error management in application
We ported Rudder to ZIO
2019-10-30
francois@rudder.io @fanf42
Hi!
devops automation/compliance app
manage ten of thousands computers
2
François ARMAND
CTO
Founder
Free Software Company
“Stay Up”
Hi!
devops automation/compliance app
manage ten of thousands computers
3
François ARMAND
CTO
Founder
Free Software Company
“Stay Up”
Developer
Developer ? ● Model the world into code
○ Try to make it useful
4
Developer ? ● Model the world into code
○ Try to make it useful
● Nominal case necessary (of course)
5
Developer ? ● Model the world into code
○ Try to make it useful
● Nominal case necessary (of course)
● But not sufficient (models are false)
○ Bugs
○ Misunderstanding of needs
○ open world
○ Damn users using your app
■ often “me, 3 days in the future”
6
This talk ● systematic management of errors
● caveat emptor:
○ I’m a scala dev, mainly
■ expect Scala terminologie
■ statically typed language with union types, interfaces
○ application, not library
■ closer world (genericity is not the main goal)
7
This talk ● It's an important talk for me
● Much harder to do than expected
○ based on lots of deeply rooted, fuzzy,
experimental knowledge
● Please, please, I beg you: if anything
unclear, come chat with me / ask
questions (whatever the medium)
8
9
Not so popular opinions
- 4 Hills I would die on -
Our work as developers is
to discover and assess
failure modes
10
Not so popular opinion 1/4
ERRORS are a SOCIAL
construction to give AGENCY to the
receiver of the error
11
Not so popular opinion 2/4
An application has always
at least 3 kinds of users:
users ; devs ; and ops.
Don’t forget any.
12
Not so popular opinion 3/4
It’s YOUR work to choose the
SEMANTIC between nominal case
and error and KEEP your
PROMISES
Not so popular opinion 4/4
13
OK.
But in concret
terms
14
?
15
Assess failure modes.
Give agency to your users
and don’t forget any of them.
You are responsible to keep
promises made.
16
Pure, total functions
Explicit error channel
Program to strict
interfaces and protocols
Composition and tooling
1.
2.
4.
5.
Failures vs Errors
3.
Assess failure modes.
Give agency to your users
and don’t forget any of them.
You are responsible to keep
promises made.
17
1.
2.
4.
5.
These points are also important and
cans be translated at
architecture / UX / team / ecosystem
levels.
But let’s keep it simple with code.
3.
Assess failure modes.
Give agency to your users
and don’t forget any of them.
You are responsible to keep
promises made.
1.
18
don’t lie about
your promises
Pure, total functions
Don’t lie! divide(a: Int, b: Int): Int
19
Don’t lie!
20
Divide By Zero ?
divide(a: Int, b: Int): Int
Don’t lie!
21
Divide By Zero ?
● non total functions are a lie
○ your promises are unsound
○ your users can’t react appropriately
divide(a: Int, b: Int): Int
Don’t lie!
22
getUserFromDB(id: UserId): User
Don’t lie!
23
No such user ? (non total)
getUserFromDB(id: UserId): User
Don’t lie!
24
No such user ? (non total)
DB connexion error?
getUserFromDB(id: UserId): User
Don’t lie!
25
No such user ? (non total)
DB connexion error?
● non pure functions are a lie
○ your promises are unsound
○ your users can’t react appropriately
getUserFromDB(id: UserId): User
Sound
promises
26
● use total functions
○ or make them total with union return type
● use pure functions
○ or make them pure with IO monad
● Don’t lie to your users,
● allow them to react efficiently:
2.
27
make it unambiguous
in your types
Explicit error channel
28
It’s a signal
make it
unambiguous
give agency
● Don’t assume what’s obvious
● It’s an open world out there
● Don’t force users to
revert-engineer possible cases
29
It’s a signal
make it
unambiguous
give agency
Which intent is less ambiguous?
30
blobzurg(a: Int, b: Int): Option[Int]
blobzurg(a: Int, b: Int): PureResult[DivideByZero, Int]
It’s a signal
make it
unambiguous
give agency
31
It’s a signal
make it
unambiguous
give agency
automate it
● Use the type system to automate
classification of errors?
32
A type system is a tractable syntactic method for
proving the absence of certain program behaviors by
classifying phrases according to the kinds of values
they compute.
Benjamin Pierce
It’s a signal
make it
unambiguous
give agency
automate it
● Use the type system to automate
classification of errors?
33
By definition, a type system automatically categorize results
⟹ need for a dedicated error chanel + a common error trait
A type system is a tractable syntactic method for
proving the absence of certain program behaviors by
classifying phrases according to the kinds of values
they compute.
Benjamin Pierce
It’s a signal
make it
unambiguous
give agency
automate it
def divide(a: Int, b: Int): PureResult[Int]
34
A type system is a tractable syntactic method for
proving the absence of certain program behaviors by
classifying phrases according to the kinds of values
they compute.
Benjamin Pierce
trait MyAppError // common properties of errors
type PureResult[A] = Either[MyAppError, A]
It’s a signal
make it
unambiguous
give agency
automate it
def divide(a: Int, b: Int): PureResult[Int]
By definition, a type system automatically categorize results
⟹ need for a dedicated error chanel + a common error trait
35
It’s a signal
make it
unambiguous
give agency
automate it
def getUser(id: UserId): IOResult[User]
By definition, a type system automatically categorize results
⟹ need for a dedicated error chanel + a common error trait
Same for effectful functions!
Same for effectful functions!
36
trait MyAppError // common properties of errors
type IOResult[A] = IO[MyAppError, A]
It’s a signal
make it
unambiguous
give agency
automate it
def getUser(id: UserId): IOResult[User]
By definition, a type system automatically categorize results
⟹ need for a dedicated error chanel + a common error trait
37
It’s a signal
make it
unambiguous
give agency
automate it
● Use a dedicated error channel
○ ~ Either[E, A] for pure code,
○ else ~ IO[E, A] monad
● use a parent trait for common error
properties…
● and for automatic categorization
of errors by compiler
3.
38
models are false
by construction
Failures vs Errors
Model
everything?
39
writeFile(path: String, value: String): IOResult[Unit]
Model
everything?
40
java.lang.SecurityException?
(jvm perm to access FS)
writeFile(path: String, value: String): IOResult[Unit]
Model
everything?
41
java.lang.SecurityException?
(jvm perm to access FS)
⟹ where do you put the limit?
writeFile(path: String, value: String): IOResult[Unit]
Systems? Need for a systematic approach to error management
42
A school of systems
Systems? Need for a systematic approach to error management
43
○ BOUNDED group of
things
○ with a NAME
Interacting
○ with others
systems
A school of systems
Systems
have horizon.
44
○ nothing exists
beyond horizon
Systems
have horizon.
Horrors lie
beyond.
45
○ nothing exists
beyond horizon
○ Like with
Lovecraft: if
something from
beyond interact
with a system, the
system becomes
inconsistent
Errors
vs
Failures
46
Errors
● expected non
nominal case
● signal for users
● social construction:
you choose
alternative or error
● reflected in types
Failures
● unexpected case: by
definition, application
is in an unknown
state
● only choice is stop as
cleanly as possible
● not reflected in types
Horizon limit
is your choice
-
by definition
47
java.lang.SecurityException?
Horizon limit
is your choice
-
by definition
48
java.lang.SecurityException?
execScript(js: String): IOResult[String]
In Rudder, we have a JS engine (JS from users):
Horizon limit
is your choice
-
by definition
49
java.lang.SecurityException?
execScript(js: String): IOResult[String]
In Rudder, we have a JS engine (JS from users):
⟹ SecurityException is an expected error case here
Horizon limit
is your choice
-
by definition
50
java.lang.SecurityException?
execScript(js: String): IOResult[String]
In Rudder, we have a JS engine (JS from users):
⟹ SecurityException is an expected error case here
… but nowhere else in Rudder. By our choice.
4.
51
use systems to
materialize promises
Program to strict
interfaces and protocols
Need for a systematic approach to error management
52
○ BOUNDED group of
things
○ with a NAME
Interacting
○ with others
systems
A school of systems
A bit more
about
systems
A bit more
about
systems
Need for a systematic approach to error management
53
○ BOUNDED group of
things
○ with a NAME
Interacting
○ via INTERFACES
○ by a PROTOCOL
with other systems
○ And PROMISING
to have a behavior
A school of systems
Example?
54
Typical web application.
Example?
55
Typical web application.
Example?
56
Typical web application. How to keep contradictory promises?
Promises to third parties
about REST behaviour
Promises to business and
developers about code
manageability
Make
promises,
Keep them
57
● systems allow to bound responsibilities
Make
promises,
Keep them
58
● systems allow to bound responsibilities
Make
promises,
Keep them
59
● systems allow to bound responsibilities
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
Make
promises,
Keep them
60
● systems allow to bound responsibilities
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
Pattern: “A pure heart (core)
surrounded by side effects”*
* works better in French: “un coeur pur
encerclé par les effets de bords”
Make
promises,
Keep them
61
● systems allow to bound responsibilities
Users of the API want stability and
to know what errors can happen
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
Make
promises,
Keep them
62
● systems allow to bound responsibilities
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
REST sub-system :
● own ADT / logic (mostly effects)
● lifecycle bounded to REST contract:
strict versioning, changes are
breaking changes
Users of the API want stability and
to know what errors can happen
Make
promises,
Keep them
63
● systems allow to bound responsibilities
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
REST sub-system :
● own ADT / logic (mostly effects)
● lifecycle bounded to REST contract:
strict versioning, changes are
breaking changes
Stable API : interface, strict protocol &
promises (nominal cases + errors)
Users of the API have agency
(able to react efficiently)
Make
promises,
Keep them
64
● systems allow to bound responsibilities
Business Core sub-system:
● own ADT / logic (mostly pure)
● lifecycle bounded to developers
understanding of needs (rapid
changes)
REST sub-system :
● own ADT / logic (mostly effects)
● lifecycle bounded to REST contract:
strict versioning, changes are
breaking changes
Stable API : interface, strict protocol &
promises (nominal cases + errors)
Users of the API have agency
(able to react efficiently)
Translation between sub-systems:
API: interface, protocol & promises!
Make
promises,
Keep them
65
● systems allow to bound responsibilities
● translate errors between sub-systems
○ make errors relevant to their users
● It’s a model, it’s false
○ there is NO definitive answer.
○ discuss, share, iterate
● the bigger the promises, the stricter the API
5.
66
make it extremely
convenient to use
Composition and tooling
Checked exceptions
are a good
signal for users
67
Unpopular opinion (sure)
Checked exceptions
are a good
signal for users
Who likes them ?
68
Unpopular opinion (sure)
What’s
missing for
good error
management
in code ?
● signal must be unambiguous
○ exception are a pile of ambiguity
○ Error ?
○ Fatal error ?
○ Checked ? Unchecked ?
69
What’s
missing for
good error
management
in code ?
● signal must be unambiguous
○ exception are a pile of ambiguity
● exceptions are A PAIN to use
○ no tooling, no inference, nothing
■ you need to be able to manipulate errors like normal code
■ where are our higher order functions like map, fold, etc ?
○ no composition
■ loose referential transparency*
70
* the single biggest win regarding code comprehension
Make it a joy!
71
● managing error should be enjoyable !
○ automatic (in for loop + inference)
○ or as expressive as nominal case!
● safely, easely managing error should be the default !
○ composition (referential transparency…)
○ higher level resource management: bracket, etc
● make the code extremely readable
○ add all the combinators you need!
○ it’s cheap with pure, total functions
72
In Rudder:
Why ZIO?
Why ZIO ?
73
● you still have to think in systems by yourself
Why ZIO ?
74
● you still have to think in systems by yourself
● then ZIO provides :
○ effect management
○ with an explicit error channel
○
IO[+E, +A]
val pureCode = IO.effect(effectfulCode)
Why ZIO ?
75
● you still have to think in systems by yourself
● then ZIO provides :
○ debuggable failures
Complex error composition Async code trace
Why ZIO ?
76
● you still have to think in systems by yourself
● then ZIO provides :
○ tons of convenience to manipulate errors
■ create: from Option, Either, value...
■ transform: mapError, fold, foldM, ..
■ recovery: total, partial, or else
○ composable effects
■ .bracket / Managed, asyncqueues, STM, etc
● safe, composable resource management
Why ZIO ?
77
● you still have to think in system by yourself
● then ZIO provides :
○ effect management
○ with an explicit error channel
○ debuggable failures
○ tons of convenience to manipulate errors
○ composable
Why ZIO ?
78
● you still have to think in system by yourself
● then ZIO provides :
○ effect management
○ with an explicit error channel
○ debuggable failures
○ tons of convenience to manipulate errors
○ composable
● Everything work in parallel, asynchronous code too!
● Inference just work!
Why ZIO ?
79
● you still have to think in system by yourself
● then ZIO provides :
○ effect management
○ with an explicit error channel
○ debuggable failures
○ tons of convenience to manipulate errors
○ composable
● Everything work in parallel, concurrent code too!
● Inference just work!
Lots of details: “Error Management: Future vs ZIO”
https://www.slideshare.net/jdegoes/error-management-future-vs-zio
80
In Rudder, with ZIO:
we settled on that
One error
hierarchy
81
● One error type (trait) providing common tooling
Unambiguous
type
82
Generic,
useful
errors
83
Specialized
error for
subsystems
84
Full example
85
● inference just works
● each sub-system add relevant information
(None, msg) => Unexpected(msg)
PureResult[A] => IOResult[A]
(err: RudderError[A], msg) => Chained(msg, err)
error contextualisation between systems
86
Pure, total functions
don’t lie about your promises
Explicit error channel
make it unambiguous in your types
Program to strict
interfaces and protocols
use systems to materialize promises
Composition and tooling
make it extremely convenient to use
Assess failure modes.
Give agency to your users
and don’t forget any of them.
You are responsible to keep
promises made.
1.
2.
4.
5.
Failures vs Errors
models are false by construction3.
Question?
Contact me /
Chat with me!
https://twitter.com/fanf42
https://github.com/fanf
https://keybase.io/fanf42
irc/freenode: fanf
francois@rudder.io
87
Ressources
○ Error management: future vs ZIO
A much more detailed presentation of ZIO error management capabilities
https://www.slideshare.net/jdegoes/error-management-future-vs-zio
○ Understand Things As Interacting Systems
More insights on systems.
https://medium.com/@fanf42/understand-things-as-interacting-systems-b273bdba5dec
○ Stay Up!
Journey of a Free Software Company. One decade in search for a sustainable model
https://medium.com/@fanf42/stay-up-5b780511109d
Some
questions
asked after
the talk
88
● Is SystemError used to catch / materialize failure ?
○ no, SystemError is here to translate Error that need to be dealts
with (like connection error to DB, FS related problem, etc) but
are encoded in Java with an Exception. SystemError is not used
to catch Java “OutOfMemoryError”. These exception kills
Rudder. We use the JVM Thread.setDefaultUncaughtExceptionHandler to try
to give more information to dev/ops and clean things before
killing the app.
Some
questions
asked after
the talk
89
● You have only one parent type for errors. Don’t you lose a lot of details with all special
errors in subsystems losing the specificities when they are seen as RudderError?
○ this is a very pertinent question, and we spend a log of time pondering between the
current design and one where all sub-systems would have their own error type (with no
common super type). In the end, we settled on the current design because:
■ no common super type means no automatic inference. You need to guide it with
transformer, and even if ZIO provide tooling to map errors, that means a lot of useless
boilerplate that pollute the readability of your code.
■ there is common tooling that you really want to have in all errors (Chained, SystemError,
but also “notOptional”, etc). You don’t want to rewrite them. Yes type class could be a
solution, but you still have to write them, for no clear gain here.
■ you are fighting the automatic categorization done by the compiler in place of
leveraging it.
■ The gain (detailed error) is actually almost never needed. When we switched to “only
one super class for all error”, we saw that “Chained” is sufficient to deals with general
trans-system cases, and in some very, very rare cases, you case build ad-hoc
combinators when needed, it’s cheap.
○ So all in all, the wins in convenience and joy of just having evering working without
boilerplate clearly outpaced the not clear gain of having different error hierarchies.
○ The problem would have been different if Rudder was not one monolithic app with a
need of separated compilation between services. I think we would have made an
“error” lib in that case.
Some
questions
asked after
the talk
90
● We use Future[Either[E,A]] + MTL, why should we switch to ZIO?
○ Well, the decision to switch is yours, and I don’t know the
specific context of your company to give an advice on that.
Nonetheless, here is my personal opinion:
■ ZIO stack seems simpler (less concepts) and work
perfectly with inference. Thus it may be simpler to teach it
to new people, and to maintain. YMMV.
■ ZIO perf are excellent, especially regarding concurrent
code. Fibers are a very nice abstraction to work with.
■ ZIO enforce pure code, which is generally simpler to
compose/refactor.
■ ZIO tooling and linked construction (Managed resources,
Async Queues, STM, etc) are a joy to code with. It removes
a lot of pains in tedious, boring, complicated tasks (closing
resources correctly, sync between concurrent access, etc)
■ pertinent stack trace in concurrent code is a major win
● But at the end of the day, you decide!
Some
questions
asked after
the talk
91
● How long did it took to port Rudder to ZIO?
○ It’s complicated :). 1 month of part time (me), plus lots more time
for teaching, refactoring, understanding new paradigm limits, etc
■ 1/ we didn’t started from nowhere. We were using Box from
liftweb, and a lot of the code in Rudder was already “shaped” to
deal with errors as explain in the talk (see
https://issues.rudder.io/issues/14870 for context)
■ 2/ we didn’t ported all Rudder to ZIO. I estimated that we
ported ~ 40% of the code (60k-70k lines ?).
■ 3/ we did some major refactoring along the lines, using new
combinators and higher level structures (like async queues)
■ 4/ we started in end of 2018, when ZIO code was still moving a
lot and we switch to new things we when became available
(ZIO 1.0.0 is around the corner and it as been quite stable for
months now)
■ we spent quite some time looking for the best choice for errors
between sub-system (see other question)

More Related Content

What's hot

Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Ahmed Gad
 
265 ge8151 problem solving and python programming - 2 marks with answers
265   ge8151 problem solving and python programming - 2 marks with answers265   ge8151 problem solving and python programming - 2 marks with answers
265 ge8151 problem solving and python programming - 2 marks with answersvithyanila
 
9781111530532 ppt ch05
9781111530532 ppt ch059781111530532 ppt ch05
9781111530532 ppt ch05Terry Yoast
 
18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVING18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVINGGOWSIKRAJAP
 
9781111530532 ppt ch04
9781111530532 ppt ch049781111530532 ppt ch04
9781111530532 ppt ch04Terry Yoast
 
9781111530532 ppt ch02
9781111530532 ppt ch029781111530532 ppt ch02
9781111530532 ppt ch02Terry Yoast
 
Finding latent code errors via machine learning over program ...
Finding latent code errors via machine learning over program ...Finding latent code errors via machine learning over program ...
Finding latent code errors via machine learning over program ...butest
 
9781111530532 ppt ch07
9781111530532 ppt ch079781111530532 ppt ch07
9781111530532 ppt ch07Terry Yoast
 
C programming for Computing Techniques
C programming for Computing TechniquesC programming for Computing Techniques
C programming for Computing TechniquesAppili Vamsi Krishna
 
9781111530532 ppt ch03
9781111530532 ppt ch039781111530532 ppt ch03
9781111530532 ppt ch03Terry Yoast
 
8 abstract classes and interfaces
8   abstract classes and interfaces 8   abstract classes and interfaces
8 abstract classes and interfaces Tuan Ngo
 

What's hot (19)

Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
 
265 ge8151 problem solving and python programming - 2 marks with answers
265   ge8151 problem solving and python programming - 2 marks with answers265   ge8151 problem solving and python programming - 2 marks with answers
265 ge8151 problem solving and python programming - 2 marks with answers
 
Chapter 4 5
Chapter 4 5Chapter 4 5
Chapter 4 5
 
Chap02
Chap02Chap02
Chap02
 
9781111530532 ppt ch05
9781111530532 ppt ch059781111530532 ppt ch05
9781111530532 ppt ch05
 
C aptitude book
C aptitude bookC aptitude book
C aptitude book
 
C++ ppt
C++ pptC++ ppt
C++ ppt
 
DISE - Programming Concepts
DISE - Programming ConceptsDISE - Programming Concepts
DISE - Programming Concepts
 
Procedural programming
Procedural programmingProcedural programming
Procedural programming
 
Lecture 18
Lecture 18Lecture 18
Lecture 18
 
18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVING18CSS101J PROGRAMMING FOR PROBLEM SOLVING
18CSS101J PROGRAMMING FOR PROBLEM SOLVING
 
Switch case and looping
Switch case and loopingSwitch case and looping
Switch case and looping
 
9781111530532 ppt ch04
9781111530532 ppt ch049781111530532 ppt ch04
9781111530532 ppt ch04
 
9781111530532 ppt ch02
9781111530532 ppt ch029781111530532 ppt ch02
9781111530532 ppt ch02
 
Finding latent code errors via machine learning over program ...
Finding latent code errors via machine learning over program ...Finding latent code errors via machine learning over program ...
Finding latent code errors via machine learning over program ...
 
9781111530532 ppt ch07
9781111530532 ppt ch079781111530532 ppt ch07
9781111530532 ppt ch07
 
C programming for Computing Techniques
C programming for Computing TechniquesC programming for Computing Techniques
C programming for Computing Techniques
 
9781111530532 ppt ch03
9781111530532 ppt ch039781111530532 ppt ch03
9781111530532 ppt ch03
 
8 abstract classes and interfaces
8   abstract classes and interfaces 8   abstract classes and interfaces
8 abstract classes and interfaces
 

Similar to Systematic error management - we ported rudder to zio

classVI_Coding_Teacher_Presentation.pptx
classVI_Coding_Teacher_Presentation.pptxclassVI_Coding_Teacher_Presentation.pptx
classVI_Coding_Teacher_Presentation.pptxssusere336f4
 
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGE
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGEINTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGE
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGERathnaM16
 
The pragmatic programmer
The pragmatic programmerThe pragmatic programmer
The pragmatic programmerLeylimYaln
 
Fundamentals of programming with C++
Fundamentals of programming with C++Fundamentals of programming with C++
Fundamentals of programming with C++Seble Nigussie
 
Jeremiah Yancy | Skills and techniques of the Systems Analyst
Jeremiah Yancy | Skills and techniques of the Systems AnalystJeremiah Yancy | Skills and techniques of the Systems Analyst
Jeremiah Yancy | Skills and techniques of the Systems AnalystJeremiah Yancy
 
Indy meetup#7 effective unit-testing-mule
Indy meetup#7 effective unit-testing-muleIndy meetup#7 effective unit-testing-mule
Indy meetup#7 effective unit-testing-muleikram_ahamed
 
L1. Basic Programming Concepts.pdf
L1. Basic Programming Concepts.pdfL1. Basic Programming Concepts.pdf
L1. Basic Programming Concepts.pdfMMRF2
 
Book management system
Book management systemBook management system
Book management systemSHARDA SHARAN
 
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdf
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdfVISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdf
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdfNALANDACSCCENTRE
 
Secret Twists to Efficiently Develop Reactive Software Systems
Secret Twists to Efficiently Develop Reactive Software SystemsSecret Twists to Efficiently Develop Reactive Software Systems
Secret Twists to Efficiently Develop Reactive Software SystemsBart Jonkers
 
Problem Solving Techniques
Problem Solving TechniquesProblem Solving Techniques
Problem Solving TechniquesAshesh R
 
AppSight 5.0 Advanced Concepts Training
AppSight 5.0 Advanced Concepts TrainingAppSight 5.0 Advanced Concepts Training
AppSight 5.0 Advanced Concepts TrainingDamian Rochman
 
Computer programing 111 lecture 2
Computer programing 111 lecture 2Computer programing 111 lecture 2
Computer programing 111 lecture 2ITNet
 
Lecture 1 uml with java implementation
Lecture 1 uml with java implementationLecture 1 uml with java implementation
Lecture 1 uml with java implementationthe_wumberlog
 

Similar to Systematic error management - we ported rudder to zio (20)

Intro To AOP
Intro To AOPIntro To AOP
Intro To AOP
 
classVI_Coding_Teacher_Presentation.pptx
classVI_Coding_Teacher_Presentation.pptxclassVI_Coding_Teacher_Presentation.pptx
classVI_Coding_Teacher_Presentation.pptx
 
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGE
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGEINTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGE
INTRODUCTION TO CODING-CLASS VI LEVEL-DESCRIPTION ABOUT SYNTAX LANGUAGE
 
Beekman5 std ppt_13
Beekman5 std ppt_13Beekman5 std ppt_13
Beekman5 std ppt_13
 
The pragmatic programmer
The pragmatic programmerThe pragmatic programmer
The pragmatic programmer
 
Fundamentals of programming with C++
Fundamentals of programming with C++Fundamentals of programming with C++
Fundamentals of programming with C++
 
Jeremiah Yancy | Skills and techniques of the Systems Analyst
Jeremiah Yancy | Skills and techniques of the Systems AnalystJeremiah Yancy | Skills and techniques of the Systems Analyst
Jeremiah Yancy | Skills and techniques of the Systems Analyst
 
Indy meetup#7 effective unit-testing-mule
Indy meetup#7 effective unit-testing-muleIndy meetup#7 effective unit-testing-mule
Indy meetup#7 effective unit-testing-mule
 
Latest UI guidelines for Web Apps
Latest UI guidelines for Web AppsLatest UI guidelines for Web Apps
Latest UI guidelines for Web Apps
 
L1. Basic Programming Concepts.pdf
L1. Basic Programming Concepts.pdfL1. Basic Programming Concepts.pdf
L1. Basic Programming Concepts.pdf
 
Book management system
Book management systemBook management system
Book management system
 
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdf
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdfVISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdf
VISUAL_BASIC_LECTURE_NOTE_A_Z_MADE_EASY.pdf
 
Secret Twists to Efficiently Develop Reactive Software Systems
Secret Twists to Efficiently Develop Reactive Software SystemsSecret Twists to Efficiently Develop Reactive Software Systems
Secret Twists to Efficiently Develop Reactive Software Systems
 
Problem Solving Techniques
Problem Solving TechniquesProblem Solving Techniques
Problem Solving Techniques
 
AppSight 5.0 Advanced Concepts Training
AppSight 5.0 Advanced Concepts TrainingAppSight 5.0 Advanced Concepts Training
AppSight 5.0 Advanced Concepts Training
 
DP Project Report
DP Project ReportDP Project Report
DP Project Report
 
Introductoin to Python.ppt
Introductoin to Python.pptIntroductoin to Python.ppt
Introductoin to Python.ppt
 
Computer programing 111 lecture 2
Computer programing 111 lecture 2Computer programing 111 lecture 2
Computer programing 111 lecture 2
 
Cp 111 lecture 2
Cp 111 lecture 2Cp 111 lecture 2
Cp 111 lecture 2
 
Lecture 1 uml with java implementation
Lecture 1 uml with java implementationLecture 1 uml with java implementation
Lecture 1 uml with java implementation
 

Recently uploaded

ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9Jürgen Gutsch
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 

Recently uploaded (20)

ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Sustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire ThornewillSustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire Thornewill
 
Program with GUTs
Program with GUTsProgram with GUTs
Program with GUTs
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 

Systematic error management - we ported rudder to zio

  • 1. Systematic error management in application We ported Rudder to ZIO 2019-10-30 francois@rudder.io @fanf42
  • 2. Hi! devops automation/compliance app manage ten of thousands computers 2 François ARMAND CTO Founder Free Software Company “Stay Up”
  • 3. Hi! devops automation/compliance app manage ten of thousands computers 3 François ARMAND CTO Founder Free Software Company “Stay Up” Developer
  • 4. Developer ? ● Model the world into code ○ Try to make it useful 4
  • 5. Developer ? ● Model the world into code ○ Try to make it useful ● Nominal case necessary (of course) 5
  • 6. Developer ? ● Model the world into code ○ Try to make it useful ● Nominal case necessary (of course) ● But not sufficient (models are false) ○ Bugs ○ Misunderstanding of needs ○ open world ○ Damn users using your app ■ often “me, 3 days in the future” 6
  • 7. This talk ● systematic management of errors ● caveat emptor: ○ I’m a scala dev, mainly ■ expect Scala terminologie ■ statically typed language with union types, interfaces ○ application, not library ■ closer world (genericity is not the main goal) 7
  • 8. This talk ● It's an important talk for me ● Much harder to do than expected ○ based on lots of deeply rooted, fuzzy, experimental knowledge ● Please, please, I beg you: if anything unclear, come chat with me / ask questions (whatever the medium) 8
  • 9. 9 Not so popular opinions - 4 Hills I would die on -
  • 10. Our work as developers is to discover and assess failure modes 10 Not so popular opinion 1/4
  • 11. ERRORS are a SOCIAL construction to give AGENCY to the receiver of the error 11 Not so popular opinion 2/4
  • 12. An application has always at least 3 kinds of users: users ; devs ; and ops. Don’t forget any. 12 Not so popular opinion 3/4
  • 13. It’s YOUR work to choose the SEMANTIC between nominal case and error and KEEP your PROMISES Not so popular opinion 4/4 13
  • 15. 15 Assess failure modes. Give agency to your users and don’t forget any of them. You are responsible to keep promises made.
  • 16. 16 Pure, total functions Explicit error channel Program to strict interfaces and protocols Composition and tooling 1. 2. 4. 5. Failures vs Errors 3. Assess failure modes. Give agency to your users and don’t forget any of them. You are responsible to keep promises made.
  • 17. 17 1. 2. 4. 5. These points are also important and cans be translated at architecture / UX / team / ecosystem levels. But let’s keep it simple with code. 3. Assess failure modes. Give agency to your users and don’t forget any of them. You are responsible to keep promises made.
  • 18. 1. 18 don’t lie about your promises Pure, total functions
  • 19. Don’t lie! divide(a: Int, b: Int): Int 19
  • 20. Don’t lie! 20 Divide By Zero ? divide(a: Int, b: Int): Int
  • 21. Don’t lie! 21 Divide By Zero ? ● non total functions are a lie ○ your promises are unsound ○ your users can’t react appropriately divide(a: Int, b: Int): Int
  • 23. Don’t lie! 23 No such user ? (non total) getUserFromDB(id: UserId): User
  • 24. Don’t lie! 24 No such user ? (non total) DB connexion error? getUserFromDB(id: UserId): User
  • 25. Don’t lie! 25 No such user ? (non total) DB connexion error? ● non pure functions are a lie ○ your promises are unsound ○ your users can’t react appropriately getUserFromDB(id: UserId): User
  • 26. Sound promises 26 ● use total functions ○ or make them total with union return type ● use pure functions ○ or make them pure with IO monad ● Don’t lie to your users, ● allow them to react efficiently:
  • 27. 2. 27 make it unambiguous in your types Explicit error channel
  • 28. 28 It’s a signal make it unambiguous give agency
  • 29. ● Don’t assume what’s obvious ● It’s an open world out there ● Don’t force users to revert-engineer possible cases 29 It’s a signal make it unambiguous give agency
  • 30. Which intent is less ambiguous? 30 blobzurg(a: Int, b: Int): Option[Int] blobzurg(a: Int, b: Int): PureResult[DivideByZero, Int] It’s a signal make it unambiguous give agency
  • 31. 31 It’s a signal make it unambiguous give agency automate it ● Use the type system to automate classification of errors?
  • 32. 32 A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. Benjamin Pierce It’s a signal make it unambiguous give agency automate it ● Use the type system to automate classification of errors?
  • 33. 33 By definition, a type system automatically categorize results ⟹ need for a dedicated error chanel + a common error trait A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. Benjamin Pierce It’s a signal make it unambiguous give agency automate it def divide(a: Int, b: Int): PureResult[Int]
  • 34. 34 A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute. Benjamin Pierce trait MyAppError // common properties of errors type PureResult[A] = Either[MyAppError, A] It’s a signal make it unambiguous give agency automate it def divide(a: Int, b: Int): PureResult[Int] By definition, a type system automatically categorize results ⟹ need for a dedicated error chanel + a common error trait
  • 35. 35 It’s a signal make it unambiguous give agency automate it def getUser(id: UserId): IOResult[User] By definition, a type system automatically categorize results ⟹ need for a dedicated error chanel + a common error trait Same for effectful functions!
  • 36. Same for effectful functions! 36 trait MyAppError // common properties of errors type IOResult[A] = IO[MyAppError, A] It’s a signal make it unambiguous give agency automate it def getUser(id: UserId): IOResult[User] By definition, a type system automatically categorize results ⟹ need for a dedicated error chanel + a common error trait
  • 37. 37 It’s a signal make it unambiguous give agency automate it ● Use a dedicated error channel ○ ~ Either[E, A] for pure code, ○ else ~ IO[E, A] monad ● use a parent trait for common error properties… ● and for automatic categorization of errors by compiler
  • 38. 3. 38 models are false by construction Failures vs Errors
  • 40. Model everything? 40 java.lang.SecurityException? (jvm perm to access FS) writeFile(path: String, value: String): IOResult[Unit]
  • 41. Model everything? 41 java.lang.SecurityException? (jvm perm to access FS) ⟹ where do you put the limit? writeFile(path: String, value: String): IOResult[Unit]
  • 42. Systems? Need for a systematic approach to error management 42 A school of systems
  • 43. Systems? Need for a systematic approach to error management 43 ○ BOUNDED group of things ○ with a NAME Interacting ○ with others systems A school of systems
  • 44. Systems have horizon. 44 ○ nothing exists beyond horizon
  • 45. Systems have horizon. Horrors lie beyond. 45 ○ nothing exists beyond horizon ○ Like with Lovecraft: if something from beyond interact with a system, the system becomes inconsistent
  • 46. Errors vs Failures 46 Errors ● expected non nominal case ● signal for users ● social construction: you choose alternative or error ● reflected in types Failures ● unexpected case: by definition, application is in an unknown state ● only choice is stop as cleanly as possible ● not reflected in types
  • 47. Horizon limit is your choice - by definition 47 java.lang.SecurityException?
  • 48. Horizon limit is your choice - by definition 48 java.lang.SecurityException? execScript(js: String): IOResult[String] In Rudder, we have a JS engine (JS from users):
  • 49. Horizon limit is your choice - by definition 49 java.lang.SecurityException? execScript(js: String): IOResult[String] In Rudder, we have a JS engine (JS from users): ⟹ SecurityException is an expected error case here
  • 50. Horizon limit is your choice - by definition 50 java.lang.SecurityException? execScript(js: String): IOResult[String] In Rudder, we have a JS engine (JS from users): ⟹ SecurityException is an expected error case here … but nowhere else in Rudder. By our choice.
  • 51. 4. 51 use systems to materialize promises Program to strict interfaces and protocols
  • 52. Need for a systematic approach to error management 52 ○ BOUNDED group of things ○ with a NAME Interacting ○ with others systems A school of systems A bit more about systems
  • 53. A bit more about systems Need for a systematic approach to error management 53 ○ BOUNDED group of things ○ with a NAME Interacting ○ via INTERFACES ○ by a PROTOCOL with other systems ○ And PROMISING to have a behavior A school of systems
  • 56. Example? 56 Typical web application. How to keep contradictory promises? Promises to third parties about REST behaviour Promises to business and developers about code manageability
  • 57. Make promises, Keep them 57 ● systems allow to bound responsibilities
  • 58. Make promises, Keep them 58 ● systems allow to bound responsibilities
  • 59. Make promises, Keep them 59 ● systems allow to bound responsibilities Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes)
  • 60. Make promises, Keep them 60 ● systems allow to bound responsibilities Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes) Pattern: “A pure heart (core) surrounded by side effects”* * works better in French: “un coeur pur encerclé par les effets de bords”
  • 61. Make promises, Keep them 61 ● systems allow to bound responsibilities Users of the API want stability and to know what errors can happen Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes)
  • 62. Make promises, Keep them 62 ● systems allow to bound responsibilities Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes) REST sub-system : ● own ADT / logic (mostly effects) ● lifecycle bounded to REST contract: strict versioning, changes are breaking changes Users of the API want stability and to know what errors can happen
  • 63. Make promises, Keep them 63 ● systems allow to bound responsibilities Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes) REST sub-system : ● own ADT / logic (mostly effects) ● lifecycle bounded to REST contract: strict versioning, changes are breaking changes Stable API : interface, strict protocol & promises (nominal cases + errors) Users of the API have agency (able to react efficiently)
  • 64. Make promises, Keep them 64 ● systems allow to bound responsibilities Business Core sub-system: ● own ADT / logic (mostly pure) ● lifecycle bounded to developers understanding of needs (rapid changes) REST sub-system : ● own ADT / logic (mostly effects) ● lifecycle bounded to REST contract: strict versioning, changes are breaking changes Stable API : interface, strict protocol & promises (nominal cases + errors) Users of the API have agency (able to react efficiently) Translation between sub-systems: API: interface, protocol & promises!
  • 65. Make promises, Keep them 65 ● systems allow to bound responsibilities ● translate errors between sub-systems ○ make errors relevant to their users ● It’s a model, it’s false ○ there is NO definitive answer. ○ discuss, share, iterate ● the bigger the promises, the stricter the API
  • 66. 5. 66 make it extremely convenient to use Composition and tooling
  • 67. Checked exceptions are a good signal for users 67 Unpopular opinion (sure)
  • 68. Checked exceptions are a good signal for users Who likes them ? 68 Unpopular opinion (sure)
  • 69. What’s missing for good error management in code ? ● signal must be unambiguous ○ exception are a pile of ambiguity ○ Error ? ○ Fatal error ? ○ Checked ? Unchecked ? 69
  • 70. What’s missing for good error management in code ? ● signal must be unambiguous ○ exception are a pile of ambiguity ● exceptions are A PAIN to use ○ no tooling, no inference, nothing ■ you need to be able to manipulate errors like normal code ■ where are our higher order functions like map, fold, etc ? ○ no composition ■ loose referential transparency* 70 * the single biggest win regarding code comprehension
  • 71. Make it a joy! 71 ● managing error should be enjoyable ! ○ automatic (in for loop + inference) ○ or as expressive as nominal case! ● safely, easely managing error should be the default ! ○ composition (referential transparency…) ○ higher level resource management: bracket, etc ● make the code extremely readable ○ add all the combinators you need! ○ it’s cheap with pure, total functions
  • 73. Why ZIO ? 73 ● you still have to think in systems by yourself
  • 74. Why ZIO ? 74 ● you still have to think in systems by yourself ● then ZIO provides : ○ effect management ○ with an explicit error channel ○ IO[+E, +A] val pureCode = IO.effect(effectfulCode)
  • 75. Why ZIO ? 75 ● you still have to think in systems by yourself ● then ZIO provides : ○ debuggable failures Complex error composition Async code trace
  • 76. Why ZIO ? 76 ● you still have to think in systems by yourself ● then ZIO provides : ○ tons of convenience to manipulate errors ■ create: from Option, Either, value... ■ transform: mapError, fold, foldM, .. ■ recovery: total, partial, or else ○ composable effects ■ .bracket / Managed, asyncqueues, STM, etc ● safe, composable resource management
  • 77. Why ZIO ? 77 ● you still have to think in system by yourself ● then ZIO provides : ○ effect management ○ with an explicit error channel ○ debuggable failures ○ tons of convenience to manipulate errors ○ composable
  • 78. Why ZIO ? 78 ● you still have to think in system by yourself ● then ZIO provides : ○ effect management ○ with an explicit error channel ○ debuggable failures ○ tons of convenience to manipulate errors ○ composable ● Everything work in parallel, asynchronous code too! ● Inference just work!
  • 79. Why ZIO ? 79 ● you still have to think in system by yourself ● then ZIO provides : ○ effect management ○ with an explicit error channel ○ debuggable failures ○ tons of convenience to manipulate errors ○ composable ● Everything work in parallel, concurrent code too! ● Inference just work! Lots of details: “Error Management: Future vs ZIO” https://www.slideshare.net/jdegoes/error-management-future-vs-zio
  • 80. 80 In Rudder, with ZIO: we settled on that
  • 81. One error hierarchy 81 ● One error type (trait) providing common tooling
  • 85. Full example 85 ● inference just works ● each sub-system add relevant information (None, msg) => Unexpected(msg) PureResult[A] => IOResult[A] (err: RudderError[A], msg) => Chained(msg, err) error contextualisation between systems
  • 86. 86 Pure, total functions don’t lie about your promises Explicit error channel make it unambiguous in your types Program to strict interfaces and protocols use systems to materialize promises Composition and tooling make it extremely convenient to use Assess failure modes. Give agency to your users and don’t forget any of them. You are responsible to keep promises made. 1. 2. 4. 5. Failures vs Errors models are false by construction3.
  • 87. Question? Contact me / Chat with me! https://twitter.com/fanf42 https://github.com/fanf https://keybase.io/fanf42 irc/freenode: fanf francois@rudder.io 87 Ressources ○ Error management: future vs ZIO A much more detailed presentation of ZIO error management capabilities https://www.slideshare.net/jdegoes/error-management-future-vs-zio ○ Understand Things As Interacting Systems More insights on systems. https://medium.com/@fanf42/understand-things-as-interacting-systems-b273bdba5dec ○ Stay Up! Journey of a Free Software Company. One decade in search for a sustainable model https://medium.com/@fanf42/stay-up-5b780511109d
  • 88. Some questions asked after the talk 88 ● Is SystemError used to catch / materialize failure ? ○ no, SystemError is here to translate Error that need to be dealts with (like connection error to DB, FS related problem, etc) but are encoded in Java with an Exception. SystemError is not used to catch Java “OutOfMemoryError”. These exception kills Rudder. We use the JVM Thread.setDefaultUncaughtExceptionHandler to try to give more information to dev/ops and clean things before killing the app.
  • 89. Some questions asked after the talk 89 ● You have only one parent type for errors. Don’t you lose a lot of details with all special errors in subsystems losing the specificities when they are seen as RudderError? ○ this is a very pertinent question, and we spend a log of time pondering between the current design and one where all sub-systems would have their own error type (with no common super type). In the end, we settled on the current design because: ■ no common super type means no automatic inference. You need to guide it with transformer, and even if ZIO provide tooling to map errors, that means a lot of useless boilerplate that pollute the readability of your code. ■ there is common tooling that you really want to have in all errors (Chained, SystemError, but also “notOptional”, etc). You don’t want to rewrite them. Yes type class could be a solution, but you still have to write them, for no clear gain here. ■ you are fighting the automatic categorization done by the compiler in place of leveraging it. ■ The gain (detailed error) is actually almost never needed. When we switched to “only one super class for all error”, we saw that “Chained” is sufficient to deals with general trans-system cases, and in some very, very rare cases, you case build ad-hoc combinators when needed, it’s cheap. ○ So all in all, the wins in convenience and joy of just having evering working without boilerplate clearly outpaced the not clear gain of having different error hierarchies. ○ The problem would have been different if Rudder was not one monolithic app with a need of separated compilation between services. I think we would have made an “error” lib in that case.
  • 90. Some questions asked after the talk 90 ● We use Future[Either[E,A]] + MTL, why should we switch to ZIO? ○ Well, the decision to switch is yours, and I don’t know the specific context of your company to give an advice on that. Nonetheless, here is my personal opinion: ■ ZIO stack seems simpler (less concepts) and work perfectly with inference. Thus it may be simpler to teach it to new people, and to maintain. YMMV. ■ ZIO perf are excellent, especially regarding concurrent code. Fibers are a very nice abstraction to work with. ■ ZIO enforce pure code, which is generally simpler to compose/refactor. ■ ZIO tooling and linked construction (Managed resources, Async Queues, STM, etc) are a joy to code with. It removes a lot of pains in tedious, boring, complicated tasks (closing resources correctly, sync between concurrent access, etc) ■ pertinent stack trace in concurrent code is a major win ● But at the end of the day, you decide!
  • 91. Some questions asked after the talk 91 ● How long did it took to port Rudder to ZIO? ○ It’s complicated :). 1 month of part time (me), plus lots more time for teaching, refactoring, understanding new paradigm limits, etc ■ 1/ we didn’t started from nowhere. We were using Box from liftweb, and a lot of the code in Rudder was already “shaped” to deal with errors as explain in the talk (see https://issues.rudder.io/issues/14870 for context) ■ 2/ we didn’t ported all Rudder to ZIO. I estimated that we ported ~ 40% of the code (60k-70k lines ?). ■ 3/ we did some major refactoring along the lines, using new combinators and higher level structures (like async queues) ■ 4/ we started in end of 2018, when ZIO code was still moving a lot and we switch to new things we when became available (ZIO 1.0.0 is around the corner and it as been quite stable for months now) ■ we spent quite some time looking for the best choice for errors between sub-system (see other question)