Presentation on Functional Programming in a for-profit company using OCaml, presented on December 14th 2011 at Ghent University (UGent).
Due to the Haskell background of the attendees, OCaml was introduced with this in mind.
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Real-World Functional Programming @ Incubaid
1. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Real-World Functional Programming @
Incubaid
Romain Slootmaekers Nicolas Trangez
Incubaid BVBA
{romain,nicolas}@incubaid.com
Twitter: @incubaid
Team Blog: blog.incubaid.com
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
2. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
3. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
4. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Company Overview
Technology incubator
Creating & providing services to startups
Not a VC, collaboration with external parties
Main activities: Datacenter & Cloud Computing
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
5. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
History
Dedigate (2000)
acquired by Terremark (2005)
Hostbasket (2000)
acquired by Telenet (2008)
DataCenter Technologies (2002)
acquired by Veritas/Symantec (2005)
Q-Layer (2007)
acquired by Sun Microsystems (2009)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
6. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
7. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
8. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Development Benefits
Code quality
Correctness
Easy refactoring
Performance
Developer performance (Getting Things Done)
Developer satisfaction
Developer quality (hiring!)
Not the average “Enterprise Developer” ;-)
Mathematical background or interests
Good at logical, conceptual reasoning
Smaller prototype-to-production cycle
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
9. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Ideal fit for our Application Domains
Handling large opaque data blobs (DSS)
System & application automation (SSO)
Highly scalable distributed systems
Large scale implies constant random failure
Transactions? Where, why, how?
Concurrency
Immutability of data, purity of operations, idempotent
actions,. . . allow to reason about systems and their
behaviour
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
10. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
FP: more than languages
Note
Functional Programming is not only about languages. Insight in
FP concepts and techniques can bring radical improvements to
the architecture and design of software, especially in
large-scale, highly-unreliable environments.
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
11. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
12. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Amplidata DSS
Next-generation large object storage
Redundant storage of petabytes of big files
Dispersed storage using rateless codes
Initial version: C++
Current version: OCaml (90%), C (9%), Asm (1%)
More info @ amplidata.com
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
14. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Arakoon
Distributed, consistent, persistent key-value store
Multi-Paxos implementation
TokyoCabinet backend
OCaml
Open Source, see http://www.arakoon.org/
A nursery of two 3-node clusters
Cluster A A0 client0 client2 Cluster B B0 client1
A1 B1
A2 B2
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
15. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Baardskeerder
Local, in-process key-value database
Append-only B-tree-ish, persistent
Replace TokyoCabinet which can’t cope with large values
Benefits of and doesn’t kill SSD drives
OCaml (future Arakoon backend)
Not publicly available yet
Follow @incubaid or blog.incubaid.com for updates
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
16. About Incubaid
Functional Programming @ Incubaid Why FP?
OCaml Current Projects
Conclusion
Baardskeerder (Cont.)
Tree Construction
0: Value "A"
1: Leaf [ " a " , Outer 0 ]
2: Commit ( Outer 1 )
a b c 3: Value "B"
4: Leaf [ " a " , Outer 0 ; " b " , Outer 3 ]
5: Commit ( Outer 4 )
6: Value "C"
A B C 7: Leaf [ " a " , Outer 0 ; " b " , Outer 3 ; " c " , Outer 6 ]
8: Commit ( Outer 7 )
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
17. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
18. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Introduction
Objective Caml
Objective Categorical Abstract Machine/Meta Language
Dialect of ML
Descendents include Microsoft F#, JoCaml (, Scala)
Static typing using type inference
Eager evaluation (by default)
Emphasis on performance, both in language features as
well as standard library implementation
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
19. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Timeline
1970 1980 1990 2000 2010
Caml
Ocaml
F#
ML
Miranda
Haskell
Standard ML
Prolog
Erlang
C
C++
Scala
Java
C#
1970 1980 1990 2000 2010
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
20. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Compiler and Runtime
Main implementation from INRIA (.fr)
Bytecode and native targets
Very fast compilation
Straight-forward compiler backend & code generation
Easy to reason about efficiency, predict compiler output
If necessary (unlike GHC) native assembly can be
read/interpreted by humans
Thanks to this, we were able to debug an obscure
non-reproducible bug in Arakoon once, which turned out to
be a flaw in one of the libraries we use. Details on request!
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
21. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Concurrency & Parallellism
Using native/system threads (1-to-1 mapping)
No parallel runtime (only one OCaml execution thread at
all times), a reentrant runtime is WIP
Lightweight “Monadic Threads” for concurrency using Lwt,
Async,. . . See W.L.Harrison, “Cheap (But Functional)
Threads”
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
22. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Applications
Ocsigen: Web application framework, parent project of
Lwt: lightweight threading
Js_of_ocaml: OCaml to JavaScript compiler
Mirage: Xen-based exokernel
Coq: Proof assistant
MLDonkey: EDonkey P2P client
Unison: File synchronization
haXe compiler: High-level multiplatform language,
compiled to JS/SWF/AS3/PHP5/C++/. . .
Coccinelle: Semantic patches
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
23. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
24. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Key Differences (1/3)
OCaml evaluation is eager by default
I once defined (>>) as (>>) a b = a >>= fun _ −> b. Lost quite
some time debugging the unexpected runtime behaviour.
No overloading (cfr. Haskell type-classes)
+ : int −> int −> int
+. : float −> float −> float (notice the dot)
+ : num −> num −> num
Both structural and referential equality operators: = and ==
Optional & named arguments
Higher-order modules & functors (not the Haskell ones)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
25. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Key Differences (2/3)
OO support
Clean semantics
Quirky syntax
Type operators / coercions: :#, :>
Polymorphic variants (union types)
# [ ‘ Number 1 ; ‘ F l o a t 3 . 1 4 ; ‘ U n i t ; ] ; ;
− : [ > ‘ F l o a t o f f l o a t | ‘ Number o f i n t | ‘ Unit ] l i s t =
[ ‘ Number 1 ; ‘ F l o a t 3 . 1 4 ; ‘ U n i t ]
No purity restrictions (cfr. IO monad)
Side-effects (incl. IO) possible everywhere
Record fields can be mutable
In-place mutable variables (refs) implemented on top of this
Strings are in-place mutable
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
26. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Key Differences (3/3)
Sugared syntax: "abc ".[1] , [| 1; 2; 3; |].(1) vs. String .get,
Array.get
Exceptions are common, also in pure code (e.g. Not_found)
Smaller “standard library” compared to Haskell Platform
Syntax extensions
Macros: (IFDEF ... THEN ... ELSE ... END)
Comprehensions: [x | x <− [1;2;3]; x<2]
Succinctness for libraries
Normal lwt.syntax
lwt x = calc_x() in
c a l c _ x ( ) >>= fun x −> lwt y = calc_y() in
c a l c _ y ( ) >>= fun y −> ...
... (* concurrent evaluation! *)
lwt a = f () and b = g () in ...
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
27. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Gotcha: Value Restriction (1/4)
Definition
The “value restriction” is a way to maintain correct typing in the
presence of side-effects: the type of an expression can only be
generalized if the expression is a “syntactic value” (or
“non-expansive expression”):
a literal or identifier: 3, ’ a’ , . . .
an abstraction: fun x −> 2 ∗ x, . . .
a constructor applied to a syntactic value: Some 1, . . .
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
28. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Gotcha: Value Restriction (2/4)
Example
# l e t i = fun x −> x ; ;
v a l i : ’ a −> ’ a = <fun >
# let r = ref i ; ;
The type of i is ∀α.(α → α). Without value restriction, the type of
r would be ∀α.(α → α) ref.
Since it’d be polymorphic, it could be used as type
(bool −> bool) ref or ( int −> int) ref , so
r : = fun ( t r u e | f a l s e ) −> f a l s e ;
(! r ) 0
would both type-check but the last expression would result in a
runtime error.
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
29. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Gotcha: Value Restriction (3/4)
Demo
Here’s what happens for real:
# let r = ref i ; ;
v a l r : ( ’ _a −> ’ _a ) r e f = { c o n t e n t s = <fun >}
# r : = fun ( t r u e | f a l s e ) −> t r u e ; ;
− : unit = ()
# r ;;
− : ( b o o l −> b o o l ) r e f = { c o n t e n t s = <fun >}
# let s = ref i ; ;
v a l s : ( ’ _a −> ’ _a ) r e f = { c o n t e n t s = <fun >}
# (! s) 0;;
− : int = 0
# s;;
− : ( i n t −> i n t ) r e f = { c o n t e n t s = <fun >}
# l e t t = l e t l = r e f [ ] i n fun e −> l : = ( e : : (! l )); (! l );;
v a l t : ’ _a −> ’ _a l i s t = <fun >
# t 1;;
− : int l i s t = [1]
# t ;;
− : i n t −> i n t l i s t = <fun >
# t 2;;
− : i n t l i s t = [ 2 ; 1]
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
30. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Gotcha: Value Restriction (4/4)
Work-around using eta-expansion
# let id x = x ; ;
v a l i d : ’ a −> ’ a = <fun >
# ( i d 1 , i d "abc" ) ; ;
− : i n t ∗ s t r i n g = ( 1 , "abc" )
# l e t id ’ = i d i d ; ;
v a l i d ’ : ’ _a −> ’ _a = <fun >
# id ’ 1 ; ;
− : int = 1
# id ’ ; ;
− : i n t −> i n t = <fun >
# i d ’ "abc" ; ;
E r r o r : T h i s e x p r e s s i o n has type s t r i n g b u t an e x p r e s s i o n was expected of type i n t
# l e t i d ’ ’ = fun x −> ( i d i d ) x ; ;
v a l i d ’ ’ : ’ a −> ’ a = <fun >
# ( i d ’ ’ 1 , i d ’ ’ "abc" ) ; ;
− : i n t ∗ s t r i n g = ( 1 , "abc" )
# l e t f 1 = L i s t . map i d ; ;
v a l f 1 : ’ _a l i s t −> ’ _a l i s t = <fun >
# l e t f 2 = fun l −> L i s t . map i d l ; ;
v a l f 2 : ’ a l i s t −> ’ a l i s t = <fun >
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
31. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
32. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
Rsync algo
Implementation of the classic “rsync” algorithm
Full source @
https://github.com/Incubaid/rsync-demo
Programming “in the large”
Expose use of
OO
Modules, functors
Imperative programming, mutability
...
but not necessarily the best, most advisable approach
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
33. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
module type WEAK = sig
type t
v a l make : u n i t −> t
v a l from : s t r i n g −> i n t −> i n t −> t
v a l r e s e t : t −> u n i t
v a l d i g e s t : t −> i n t
v a l r o t a t e : t −> char −> char −> u n i t
v a l update : t −> s t r i n g −> i n t −> i n t −> u n i t
end
module type STRONG = sig
type t
v a l to_hex : t −> s t r i n g
v a l f i l e : s t r i n g −> t
v a l s u b s t r i n g : s t r i n g −> i n t −> i n t −> t
v a l w r i t e : out_channel −> t −> u n i t
v a l read : i n _ c h a n n e l −> t
end
module SDigest = ( s t r u c t
include D i g e s t
l e t read i c = I o . r e a d _ s t r i n g i c
l e t w r i t e oc t = I o . w r i t e _ s t r i n g oc t
end : STRONG)
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
34. About Incubaid
Functional Programming @ Incubaid OCaml vs. Haskell
OCaml Code Teaser
Conclusion
open Hash
open S i g n a t u r e
module Rsync = functor (W:WEAK) −> functor (S :STRONG) −> s t r u c t
module MySig = S i g n a t u r e (W) ( S)
class d e l t a _ e m i t t e r s i g n a t u r e new_fn h a n d l e r =
l e t bs = MySig . b l o c k _ s i z e s i g n a t u r e i n
l e t b u f f e r _ s i z e = 8 ∗ bs i n
l e t b u f f e r = S t r i n g . create b u f f e r _ s i z e in
object ( s e l f )
v a l mutable _read = 0
v a l mutable _ f i r s t _ f r e e = 0
v a l mutable _ n _ f r e e = b u f f e r _ s i z e
v a l mutable _ f i r s t _ t o d o = 0
v a l mutable _ p r e v i o u s _ a c t i o n = S t a r t bs
v a l mutable _ f i n i s h e d = f a l s e
v a l mutable _weak_ok = f a l s e
v a l _weak = W. make ( )
method _examine_block b u f f e r o f f s e t l e n g t h =
l e t wd = W. d i g e s t _weak i n
match MySig . lookup_weak s i g n a t u r e wd with
| None −> None
| Some bs −>
l e t strong = S. substring b u f f e r o f f s e t length in
i f s t r o n g = MySig . b s _s t r o n g bs
then Some ( MySig . bs_index bs )
else None
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
35. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Outline
1 About Incubaid
2 Functional Programming @ Incubaid
Why FP?
Current Projects
3 OCaml
OCaml vs. Haskell
Code Teaser
4 Conclusion
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
36. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
In Retrospect
No regrets
Productivity increased, less bugs
Bugs fixed quickly, features added easily
Runtime issues mainly due to improper usage or
configuration (the universe is still winning!), except Lwt
bugs
Limited tooling support compared to e.g. JVM
(JMX/JConsole, remote debugging, GC tuning,. . . )
No deal-breaker, because less needed
Would be some nice projects!
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
37. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Future
Current focus on OCaml: in general, best known by the
team
Haskell for “compilable pseudocode”, once in a while
Haskell for future projects, why not?
Introduce devs, architects,. . . of non-FP projects to the FP
concepts and strengths to increase the quality of their
work: knowledge of FP concepts changes the way you
think about programming, even in non-FP settings
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
38. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Internships & Jobs
Interested in working on our software and building future-proof
scalable computing platforms & looking for an internship or job?
Come join us!
recruiting@incubaid.com
wehire.be
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid
39. About Incubaid
Functional Programming @ Incubaid
OCaml
Conclusion
Questions?
Romain Slootmaekers, Nicolas Trangez Real-World Functional Programming @ Incubaid