Recommended
More Related Content
What's hot
What's hot (20)
Similar to Practical Scalaz(or)How to make your life easier the hard way
Similar to Practical Scalaz(or)How to make your life easier the hard way (20)
Recently uploaded
Recently uploaded (20)
Practical Scalaz(or)How to make your life easier the hard way
- 1. GSA Capital Practical Scalaz(or)How to make your life easier the hard way Chris Marshall Aug 2011 @oxbow_lakes
- 2. Overview of talk The library is confusing What’s with all this maths anyway? The method names are all stupid GSA Capital
- 3. Where is the love? Kinds M[A] ~> MA[M, A] A ~> Identity[A] M[A, B] ~> MAB[M, A, B] Wrappers OptionW, ListW, BooleanW Data Types Validation NonEmptyList GSA Capital
- 4. Typeclasses Common “patterns” “retrofitted” in a uniform way to many types Uses implicits to do this ...Interfaces Like being able to retrofit an interface onto classes which “logically implement” that interface ...Adapters Adapt existing types to our structures GSA Capital
- 5. Example typeclass traitEach[-E[_]]{ defeach[A](e: E[A], f: A => Unit): Unit } GSA Capital implicit def OptionEach: Each[Option] = new Each[Option] { def each[A](e: Option[A], f: A => Unit) = eforeachf }
- 6. Monoids There are monoids everywhere A set With an associative operation And an identity under that operation GSA Capital
- 7. Numbers scala> 1 |+| 2 res0: Int3 scala> 1.2 |+| 3.4 res1: Double 4.6 GSA Capital
- 8. Your own scala> 200.GBP|+|350.GBP res2: oxbow.Money550.00 GBP GSA Capital
- 9. Monoids Beget Monoids Option[A] is a Monoid if A is a monoid (A, B, .. N) is a Monoid if A, B..N are monoids A => B is a Monoid if B is a Monoid Map[A, B] is a Monoid if B is a Monoid A => A is a monoid Under function composition GSA Capital
- 10. ... scala> some(4) |+|none[Int] res4: Option[Int] Some(4) scala> none[Int] |+|none[Int] res5: Option[Int]None scala> some(4) |+|some(5) res6: Option[Int]Some(9) scala> (1, “a”, 4.5) |+| (2, “b”, 3.2) res7: (Int, String, Double)(3, “ab”, 7.7) GSA Capital
- 11. What does this mean? Winning! GSA Capital
- 12. traitTradingPosition {definventoryPnL(implicitprices: Map[Ticker, Double]) : DoubledeftradingPnL(implicit prices: Map[Ticker, Double]) : Double final def totalPnL(implicitprices: Map[Ticker, Double]) = inventoryPnL->tradingPnL } GSA Capital valpositions: Seq[TradingPosition] = db.latestPositions()val (totalTrad, totalInv) = positions.map(_.totalPnL).asMA.sum
- 13. traitTradingPosition {definventoryPnL(implicit pxs: Map[Ticker, Double]): Option[Double]deftradingPnL(implicit pxs: Map[Ticker, Double]): Option[Double]final def totalPnL(implicit pxs: Map[Ticker, Double]) = inventoryPnL|+|tradingPnL } GSA Capital valposns: Seq[TradingPosition] = db.latestPositions()valmaybePnL: Option[Double] = posns.map(_.totalPnL).asMA.sum
- 14. traitTradingPosition {defsym: Tickerdefqty: Int } GSA Capital valpete: Map[Ticker, Int] = positions1.map(p => p.sym -> p.qty).toMapvalfred: Map[Ticker, Int] = positions2.map(p => p.sym->p.qty).toMap
- 15. valtotalPositions = pete|+|fred GSA Capital for any key, if book1(key) == nand book2(key) == m, then the resulting map hasn |+| mat key Adding across a bunch of Maps now becomes as easy as... allBooks.asMA.sum
- 16. traitTradingPosition {defsym: Tickerdefqty: Int }type Filter = TradingPosition => Boolean GSA Capital Filters
- 17. Observe: filters are monoids GSA Capital vallondon: Filter = (_ : TradingPositon).sym.idendsWith“.L” valny: Filter = (_ : TradingPositon).sym.idendsWith“.O” positionsfilter (london|+|ny)
- 18. Conjunction typeFilter = TradingPosition => BooleanConjunction GSA Capital vallondon= (t : TradingPositon) => (t.sym.idendsWith“.L”) |∧| valbig = (t : TradingPositon) => (t.qty> 100000) |∧| positionsfilter (london|+|big)
- 19. Monoid = Semigroup + Zero Monoid is actually split in 2 Semigroup (the associative bit) Zero (the identity bit) ~ is “or zero” on OptionW A unary method (declared unary_~) It is really useful Eh? GSA Capital
- 20. varposns: Map[Ticker, Int] = Map.emptydefnewTrade(trd: Trade) {posns += (trd.sym-> ( (posns.get(trd.sym) getOrElse0) + trd.qty)) } GSA Capital But observe the equivalence of the following: (posns.get(trd.sym) getOrElse0) And: ~posns.get(trd.sym)
- 22. To Double?
- 23. To any Monoid!
- 24. Your own?Is logically...
- 25. varcharges: Map[Ticker, Money] = Map.emptyimplicit valChargeCcy = Currency.USDdefnewTrade(trd: Trade) {charges += (trd.sym -> ( ~charges.get(trd.sym) |+|trd.charges)) } Where we have defined our own thusimplicit defMoneyZero(implicit ccy: Currency) : Zero[Money] = zero(Money.zero(ccy))implicit valMoneySemigroup: Semigroup[Money] = semigroup(_ add _) GSA Capital
- 26. BooleanW, OptionW Consistency with ? and | Option[A] | A == getOrElse Boolean ? a | b == ternary Boolean ?? A == raise into zero Boolean !? A == same same but different Boolean guard / prevent GSA Capital
- 28. We don’t want to repeat ourselves!for { e <- xml “instruments” f <- e.attribute(“filter”) } yield (if (f == “incl”) new Filter(instr(e)) elsenew Filter(instr(e)).neg)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ GSA Capital <instruments filter=“incl”> <symbol value=“VOD.L” /> <symbol value=“MSFT.O” /> </instruments>
- 29. We can do this... GSA Capital valreverseFilter = EndoTo((_ : Filter).neg) for {e <- xml “instruments”f <- e.attribute(“filter”) } yield (f == “incl”) !?reverseFilterapplynew Filter(instr(e))
- 30. Aside: playing around scala> EndoTo(-(_ : Double)) res0: scalaz.Endo[Double] scalaz.Endo@6754642 scala> true?? res0 apply2.3 res1: Double -2.3 scala> false?? res0 apply2.3 res2: Double 2.3 scala> implicitly[Zero[Endo[Double]]] res3: scalaz.Endo[Double] scalaz.Endo@8ae765 GSA Capital
- 31. Validation Validation is the killer app for me Opened my eyes to how appalling java Exceptions are Let your types do the talking! GSA Capital
- 32. Aside: composition Functors: M[A] plus A => B equals M[B] Monads M[A] plus A => M[B] equals M[B] Applicative M[A] plus M[A => B] equals M[B] GSA Capital
- 33. Composing validations //MAP Validation[X, A] ~>A => B~> Validation[X, B] //FLATMAP Validation[X, A] ~> A => Validation[X, B] ~> Validation[X, B] //APPLY Validation[X1, A], Validation[X2, B] ~> (A, B) => C ~> Validation[X1 |+| X2, C] GSA Capital
- 34. ValidationNEL Validation[NonEmptyList[F], S] = ValidationNEL[F, S] scala> “Bah!”.failNel[Int] res1 : scalaz.Validation[NonEmptyList[String], Int]Failure(NonEmptyList(Bah!)) scala> 1.successNel[String] res2 : scalaz.Validation[NonEmptyList[String], Int] Success(1) GSA Capital
- 35. Using Validation deffile(s: String) : Validation[String, File] deftrades(file: File): List[Trade] GSA Capital valts = file(“C:/tmp/trades.csv”) maptrades //ts of type Validation[String, List[Trade]]
- 36. More realistically deftrades(f: File): ValidationNEL[String, List[Trade]] GSA Capital file(“C:/tmp/trades.csv”).liftFailNelflatMaptradesmatch { case Failure(msgs) => case Success(trades) => }
- 37. Using for-comprehensions for { f <- file(“C:/tmp/trades.csv”).liftFailNel ts <- trades(f) } yield ts GSA Capital
- 38. What does trades look like? deftrades(f: File): ValidationNEL[String, Trade] = { //List[String] valls = io.Source.fromFile(f).getLines().toList defparse(line: String): Validation[String, Trade] = sys.error(“TODO”) lsmapparse<<SOMETHING with List[Validation[String, Trade]]>> } GSA Capital
- 39. What does trades look like? deftrades(f: File): ValidationNEL[String, List[Trade]] = { //List[String] valls = io.Source.fromFile(f).getLines().toList defparse(line: String): Validation[String, Trade] = sys.error(“TODO”) (lsmap (l => parse(l).liftFailNel)) .sequence[({type l[a]=ValidationNEL[String, a]})#l, Trade] } GSA Capital
- 40. So why do I care? Your program logic is no longer forked Catching exceptions Throwing exceptions Ability to compose Via map, flatMap and applicative Keeps signatures simple Accumulate errors GSA Capital
- 41. Aside: other applicatives List[Promise[A]].sequence ~> Promise[List[A]] f: (A, B) => C (Promise[A] |@| Promise[B]) applyf ~> Promise[C] GSA Capital
- 42. Far too much stuff Iteratees Kleisli F: A => M[B] If M is a functor and I have g : B => C, then I should be able to compose these If M is a Monad and I have h : B => M[C] etc Arrows Useful methods for applying functions across data structures, like pairs GSA Capital
- 43. And More IO Deferring side effects Writers Logging Readers Configuration Typesafe equals HeikoSeeberger at scaladays GSA Capital
- 44. Great references Tony Morris’ blog Runar & Mark Harrah’s Apocalisp Nick Partridge’s Deriving Scalaz Jason Zaugg at scalaeXchange GSA Capital
Editor's Notes
- Ok – quick intro; I’ve been using scalaz for well over a year now, essentially since scala 2.8 came out. Prior to that, I used to play around with it quite a bit.Ones first impressions of the library is that it is unlike most other libraries you are likely to come across. Most of the files just contain traits called things like “Show” and a bunch of implementations. How do you use it?It has names like Semigroup, Contravariant, Liskov – it sounds complicated and mathematical.It;’s taken ASCII and unicode method names to some kind of extremeWhat I want to do in this talk is give a brief overview on how to navigate the library (it’s easy to find the good stuff if you know how) and some very basic reasons why you should care about things like monoids. Essentially because they are easy, useful and will help you write clearer code, faster.Note, that wherever you see any code, I am assuming that you have import scalaz._; import Scalaz._3 minutes
- The first question I want to answer is “where do I find the good stuff in scalaz?”The answer is that it is *mostly* in 3 places; on the “kinds”: Identity (which anything can be pimped into), on MA (which types taking one type parameter can be pimped into) and on MAB.Secondly, in the wrappers (ie methods added to existing scala types such as List, Option, Boolean)Lastly the “bespoke” data types themselves, such as NonEmptyList, Rope, Validation.5 minutesMost of the cool stuff is on Identity and MA. In scalaz7, these have been given different names and appear in the file kind.scala.
- I don’t want to say too much about typeclasses. Basically they offer a way, orthogonal to the types themselves, to describe common behaviour of those types. The implicit mechanisms in scala allow us to inject this behaviour transparently, where we would expect it.Some of the stuff I’m going to be talking about is related to the typeclasses but “the good stuff” is not found on the type class interfaces or implementations *at all*. It is found on the likes of Identity and MA, only if certain typeclasses are present at compile-time.6 minutes
- Here’s an example of a typeclass – it’s basically an encoding of the ability to invoke a side effect for each element of a container. “foreach” basicallyHere’s the implementation for Option – it just defers to the actual functionality provided by the Option type in scala’s library. You can imagine that the instance for, say, Java’s ArrayList cannot do this and hence implements in terms of a for loop
- The first thing I’m going to talk about are monoidsWhat is a monoid? It is this really simple structure. In scalaz, we use |+| (pipe-plus-pipe) to denote the application of the operationYou’ll find the typeclass definition in the scalaz library but I’m purposefully not going to show it to you here. It’s totally trivial and I think is unrelated to an understanding of how you use scalaz.When you use scalaz, you tend to think in terms of “I have an type X, such as Int, what is it? Is it a monoid? Etc You then trust that scalaz will provide the relevant implicits for you) 7 minutes
- Trivially, numbers are obviously monoids under the “plus operation” with 0 as the identity. Note that they are also monoids under multiply with 1 as the identityThis is legal scalaz. 1 is pimped to Identity, where pipe-plus-pipe is declared, but *only* if there exists a semigroup for the type.I said I didn’t want to talk too much about typeclasses and I don’t. But suffice to say that the m-plus method takes mopre than one argument. The compiler is injecting those arguments because it can find implicit values for them – those values are the required typeclass instances. If it couldn’t find them, the code would not compile
- You can create your own very simply by creating an implicit value of type Semigroup[X] (more of which later) 8 minutes
- So what is cool about this? I mean, we *already* have the ability to add Ints, and I already had a plus method on my Money class, didn’t I?The first obvious benefit is that it normalizes the operation, so that everywhere you are dealing with monoid operations, you see the symbol pipe-plus-pipe. You don’t worry about it being “plus” here, “add” there etc.But the *real* value. The totally awesome value you get is that Monoids beget monoids. They act like building blocks. There are monoids for tuples if there are monoids for the type making up the tuple. There are monoids for maps if there is a monoid for the value type etc.9 minutes
- So what is cool about this? I mean, we *already* have the ability to add Ints, and I already had a plus method on my Money class, didn’t I?The first obvious benefit is that it normalizes the operation, so that everywhere you are dealing with monoid operations, you see the symbol pipe-plus-pipe. You don’t worry about it being “plus” here, “add” there etc.But the *real* value. The totally awesome value you get is that Monoids beget monoids. They act like building blocks. There are monoids for tuples if there are monoids for the type making up the tuple. There are monoids for maps if there is a monoid for the value type etc.10 minutes
- What Are the practical benefits of this? What does it mean I can do?If you look in the scalaz example, you’ll often see stuff like what I just showed you, but how might you use this in reality?
- Short background, trading positions have the idea of inventory P&L (the money you have made or lost on the position you took into the day) and trading P&L, the money you made or lost on only those trades you did today. So what if I have a bunch of positions? How do I calculate the total inventory P&L and total trading P&L separately. Some kind of nasty looking fold I guess?Actually no: MA has on it a sum method which requires a Monoid[X]. But X is the pair (i..e Tuple2) (Double, Double). Because monoids beget monoids, *this*. *just*. *works*.Why do I have to call “asMA”? The reason is that scala provides a sum method already, and so this would be discovered first. Unfortunately scala’s sum requires a Numeric instance, which is incorrect because it assumes that only numeric things can be added. This is a shame12 mins
- Slightly different example now; the same problem but let’s assume the complexity that one may not be able to price a positionThe total P&L is now an Option of Double (I could have made it a pair of options if I’d wanted). Now – how to add the total P&L of a bunch of these? Well, we can just sum them again!14mins
- Suppose we have two separate trading books for our traders Pete and Fred. How do we get the total positions? 16 mins
- This example uses the fact that a Map[A, B] forms a monoid, if B is a monoid. I only discovered this recently when it occurred to me that it should be true. I tried it out in scalaz, and they had beaten me to it (not surprisingly!)It’s really obviuous when you think about it: if the two maps have a value at a given key, then the sum has the sum of those valuesIf anyone doubts the practical applications of this ~ they would have seen me use it yesterday, extremely thankfully, in the market turmoil we’re having at the moment. In fact I did this as a nested Map of Maps16 mins
- This one is more complicated! I was writing an app where I needed to filter trades.19 mins
- The reason I say it’s more complicated is that it relies on the Monoid for Boolean. But what is that? Well, the zero for boolean is false and the operation is logical-or. However, you may wish the absence of a filter to mean that everything passes and the logical “addition” of two filters to work like and, not or. This is trivial within scalaz, you just define a monoid in scope for Boolean with the behaviour you want, and it will get picked up.
- Here’s is the boolean conjunction at work. Actually, it’s a slight lie because london |+| big is not of the correct type – it results in a BooleanConjunction, not Boolean. You can either solve this via an implicit conversion or by composition
- OK, so a Monoid has this thing called Zero (the identity), which is part of it. It turns out that this is also really useful on its own. So useful in fact that OptionW and BooleanW wrappers have a few methods which pick up implicit zeroes in certain situations.Here’s one20 mins
- I find that this is such a common thing I need to do – accumulate a value in a map on the basis of a callback or something like that. Say this class gets notified at each trade and I want to keep track of the total positions per symbolWhen accumulating I treat an empty Option[A] (i.e. The absence of a value) as the zero for the value type : Int in this case. 22 mins
- So what I am *logically* doing can be encapsulated in this code. And notice that the type Int appears nowhere! The code is non-type specificIn fact, I have a class where the value for the map is a pair of Ints, for aggressively and passively traded trades. Again, the same code works with no changes. In fact, this works with *any* monoid: you could create your own22 mins
- This example is basically the exact same code – i mention it purely to draw the attention to two important facts. Firstly implicits may themselves have implicits. Secondly it is extremely easy to provide your own Monoids – it’s hardly any code. But they *must* be marked implicit(short pause for questions on monoids – max 3 mins)23 mins
- So we’ve seen tilde on OptionW, what else doBooleanW and OptionW provide? Well, they provide a consistently-named set of operations which often just read more nicely in your code. Using pipe instead of getOrElse for example, is nicely consistent with the ternary elvis operator.I’m going to showcase the use of ?? And !? In relation to the Endo type.24 mins
- I found one very useful tidbit is how the type Endo, basically a function from a type to itself, can be used in conjunction with the types on the previous page.Mainly from the perspective of preventing repetition in the case we want to say “if this holds, apply this transformation” but without introducing unnecessary variables.25 mins
- So we have some XML defining a filter to apply to a set of instruments – perhaps we’re designing a charge calculator to apply charges and this is part of its configuration, but the filter might be there to specify whether we treat the filter as inclusive or exclusive.So we have our XML to filter class, and it has code in it, processing the XML and returning a filter and it looks something like this. Ugh. Repetition27 mins
- By using the !? (bang, question-mark) method on BooleanW, this says use the argument if false, otherwise us the zero for the argument’s type. Our argument is an Endo, and the zero for Endo is what? It is the identity function. Endo is also a monoid.One caveat of this is that we can sum transformations using monoids, as in previous examples.
- As an aside, it’s often very useful to use the REPL to discover what is going on with these typesYou realize the scalaz examples are not “real world” but they exist so that the datatypes have obvious zeroes/empties etc so you can follow what they are doing29 mins
- OK – so now I have covered some cool stuff. But what I’ve showed you is *nice*, isn’t it? I mean, it’s nice and all, but am I really going to bring in this external dependency to save a few lines of code, or a few key strokes?Well, I’m going to introduce you to what, for me, was scalaz’s killer app. I had my personal lightbulb moment watching Nick Partridge’s “Deriving scalaz” talk ,and I encourage everyone here to watch that So what is validation. Basically the type is isomorphic to scala’s Either type. That is, an instance of valiation is either one thing (a Failure)or another thing (a Success), but not both!Shortly after I started using scala, a colleague who detests java’s Exceptions asked if we should be using Either[Exception, R] as the return types for methods. At the time, I thought that this would be unworkable – you don’t want handling exceptions to pollute your clarity of your code.However, with scalaz’s validation type, most of the ugliness disappears because of how they may be composed31 mins
- I might split how we compose types into three categories. Functors are basically the map method you see in the collection library, and monads are flatMap. The standard library doesn’t contain applicative composition.In the next slide, I’m just going to show you how you might compose validations 32 mins
- What does applicative composition do? It accumulates errors!Thay is, we have two operations which may fail and which produce instances of A and B if successful. In turn, we use these via a function to give us an instance of C. However, if either or both of the first 2 operations fail, we get their failures added together, assuming X is a semigroup.To do this use ValidationNEL34 mins
- Given a normal validation F, S, it can be converted into a ValidationNEL via the liftFailNel method.ValidationNEL is just a type alias available in scalaz35 mins
- So here’s a very simple example. File returns either an error (as a String). How do we “map” our Validation where the success is a File using the trades method, which takes a file and gives us a List of trades? I guess we could “collapse” the Validation ADT. The point is that we don’t have to. We can literally map across the validation – it is biased by default on the RSH type argument (i.e. Success)Actually, pause a second: this is not that realistic -> it seems very likely that the trade method would *also* return a validation, for example if some of the lines in the file were malformed...37 mins
- In this more realistic case, the code hardly changes at all – just uses flatMap instead of Map. I don’t collapse a validation until I absolutely have to – this is purely to show how I would do that, when the time arose39 mins
- I prefer flatMap usually to for-comprehensions. You can also use the Haskell form, which is >>=OK – so we’ve seen how we can compose the result of the trades method and the result if the file method, but what does trades look like internally? Is that composed of sub-validations?39 mins
- So trades takes a file and has to turn it into either a bunch of trades, or a (non empty) list of errorsI want to *build* this program from a sub-component – in this case the method which parses each lineIt’s clear that if I just mapped each trade by parse, I’ll end up with a list of validations, which is not what I want42 mins
- And this is how you do it. I promise that this is less complicated than it looks. In order to turn the code inside out, I use sequence. That is – if I have a List of Validations but want a Validation of a List, then always think of sequence, which is intrinsically linked with another operation called traverse. The fact that sequence is available is because Validation is an applicative functor.It’s a shame about the type lambda, but scala does not infer a partially-applied higher kinded type. That is, validation[F, S] fixed in the failure type F.I’m led to understand that nicer syntax for a partially applied type constructor might be on its way into the scala language – of course what we really want is for scala to infer it.43 mins
- I truly think this has been an enormous benefit to my code. I use wherever I would have used checked exceptions in Java. What is great is that the types force you into dealing and handling your errors. There is nowhere else to go. And because of the 3 ways of composition, the resulting code is often no less readable.It’s true that there’s the odd sequence or traverse in there, which might have to be given a type lambda in the case that you’re dealing with partially applied types – but these are rare.44 mins
- What I want to leave you with is the idea that these operations, such as sequence and applicative, are available to other types, such as Option, or List because these are applicative functors.In this example I show how promises (essentially “an A in the future”) can be composed to fork a set of operations and then join them!47 mins
- There’s too much else in scalaz that I have not even covered, much of its useful in certain situations to improve code clarity and/or re-use.There are so many little gems of coolness and I really didn’t have time to cover many of them here.48 mins
- To be honest, I’m not currently a user of these types, although I have played around with them a little