SlideShare a Scribd company logo
1 of 119
Download to read offline
John A. De Goes
@jdegoes - degoes.net
SCALAZ STREAM
REBIRTH
Scale By The Bay · November 16 · San Francisco
Itamar Ravid
@itrvd - iravid.com
#ZIO
#SCALEBYTHEBAY
PERFORMANCE
OVERVIEW
MOTIVATION API DESIGN
1 Why Streams & Why Scalaz Stream
MOTIVATION
Why Streams?
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Infinite Input
Sets
Leak-Free Data
Processing
Incremental
Computation
∞
File & Socket
Processing
Graph
Processing
Reactive &
Dataflow
Akka Streams FS2
Scalaz ReactiveGraphJet
Why ZIO Stream?
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Why ZIO Stream?
File & Socket
Processing
Graph
Processing
Reactive &
Dataflow
Akka Streams FS2
Scalaz ReactiveGraphJet
ZIO STREAM
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Why ZIO Stream?
PERFORMANCEMOTIVATION API DESIGN SUMMARY
𐄂 Cycles𐄂 Cons 𐄂 Uncons
ZIO Stream Old Scalaz Stream Akka Streams FS2
Lazy ✓ 𐄂 𐄂 ✓
Leak-Free ✓ 𐄂 𐄂 On a good day
Uniform ✓ NA NA 𐄂
Type Inferred ✓ 𐄂 ✓ 𐄂
Economical ✓ 𐄂 𐄂 ✓
ZIO Integration ✓ 𐄂 𐄂 𐄂
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Why ZIO Stream?
2 What Scalaz Stream Looks Like
API
PERFORMANCEMOTIVATION API DESIGN SUMMARY
AkkaStream AbruptTerminationException AbstractShape ActorAttributes ActorMaterializer ActorMaterializerSettings
AmorphousShape Attributes BidiShape BindFailedException BufferOverflowException Client ClosedShape
ConnectionException DelayOverflowStrategy EagerClose FanInShape FanInShape10 FanInShape11 FanInShape12
FanInShape13 FanInShape14 FanInShape15 FanInShape16 FanInShape17 FanInShape18 FanInShape19 FanInShape1N
FanInShape2 FanInShape20 FanInShape21 FanInShape22 FanInShape3 FanInShape4 FanInShape5 FanInShape6
FanInShape7 FanInShape8 FanInShape9 FanOutShape FanOutShape10 FanOutShape11 FanOutShape12 FanOutShape13
FanOutShape14 FanOutShape15 FanOutShape16 FanOutShape17 FanOutShape18 FanOutShape19 FanOutShape2
FanOutShape20 FanOutShape21 FanOutShape22 FanOutShape3 FanOutShape4 FanOutShape5 FanOutShape6
FanOutShape7 FanOutShape8 FanOutShape9 FlowMonitor FlowMonitorState FlowShape Fusing Graph IgnoreBoth
IgnoreCancel IgnoreComplete Inlet InPort IOResult KillSwitch KillSwitches MaterializationContext MaterializationException
Materializer MaterializerLoggingProvider Outlet OutPort OverflowStrategy QueueOfferResult RateExceededException
Server Shape SharedKillSwitch SinkShape SourceShape StreamLimitReachedException
StreamSubscriptionTimeoutSettings StreamSubscriptionTimeoutTerminationMode StreamTcpException
SubstreamCancelStrategy Supervision ThrottleMode TLSClientAuth TLSClosing TLSProtocol TLSRole UniformFanInShape
UniformFanOutShape UniqueKillSwitch
PERFORMANCEMOTIVATION API DESIGN SUMMARY
AkkaStream AbruptTerminationException AbstractShape ActorAttributes ActorMaterializer ActorMaterializerSettings
AmorphousShape Attributes BidiShape BindFailedException BufferOverflowException Client ClosedShape
ConnectionException DelayOverflowStrategy EagerClose FanInShape FanInShape10 FanInShape11 FanInShape12
FanInShape13 FanInShape14 FanInShape15 FanInShape16 FanInShape17 FanInShape18 FanInShape19 FanInShape1N
FanInShape2 FanInShape20 FanInShape21 FanInShape22 FanInShape3 FanInShape4 FanInShape5 FanInShape6
FanInShape7 FanInShape8 FanInShape9 FanOutShape FanOutShape10 FanOutShape11 FanOutShape12 FanOutShape13
FanOutShape14 FanOutShape15 FanOutShape16 FanOutShape17 FanOutShape18 FanOutShape19 FanOutShape2
FanOutShape20 FanOutShape21 FanOutShape22 FanOutShape3 FanOutShape4 FanOutShape5 FanOutShape6
FanOutShape7 FanOutShape8 FanOutShape9 FlowMonitor FlowMonitorState FlowShape Fusing Graph IgnoreBoth
IgnoreCancel IgnoreComplete Inlet InPort IOResult KillSwitch KillSwitches MaterializationContext MaterializationException
Materializer MaterializerLoggingProvider Outlet OutPort OverflowStrategy QueueOfferResult RateExceededException
Server Shape SharedKillSwitch SinkShape SourceShape StreamLimitReachedException
StreamSubscriptionTimeoutSettings StreamSubscriptionTimeoutTerminationMode StreamTcpException
SubstreamCancelStrategy Supervision ThrottleMode TLSClientAuth TLSClosing TLSProtocol TLSRole UniformFanInShape
UniformFanOutShape UniqueKillSwitch
Stream Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO STREAM
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Stream[E, A]
May fail with a checked error of type E
May effectfully produce zero or more values of type A
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Monoid
append
zero
Applicative
map
point
ap
Monad
bind
Bifunctor
leftMap
bimap
Alternative
alt
empty
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
val empty: Stream[Nothing, Nothing] =
Stream.empty
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val singleton: Stream[Nothing, Int] =
Stream.point(42)
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val hundred: Stream[Nothing, Int] =
Stream.fromIterable(0 to 100)
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val userIn: Stream[IOException, String] =
Stream.liftIO(getStrLn)
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val intQueue: Queue[Int] = ...
val intStream: Stream[Nothing, Int] =
Stream.fromQueue(intQueue)
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val latency: Stream[Nothing, Long] =
Stream.unfoldM(0 -> 0) {
case (n, total) =>
api.measureLatency(l =>
(n + 1) -> (total + n))
}
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val fileBytes: Stream[IOException, Byte] =
Stream.bracket(openFile)(closeFile(_)) {
file => file.readByte.attempt.toOption
}
Stream.empty
Stream.point(v)
Stream.fromIterable(list)
Stream.liftIO(io)
Stream.fromQueue(queue)
Stream.unfold(s)(f)
Stream.unfoldM(s)(f)
Stream.bracket(open)(close)(read)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val integerDigits: Stream[Nothing, Int] =
integers.map(_.toString).map(_.length)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val users: Stream[Exception, User] =
...
val usersWithProfile:
Stream[Exception, (User, Profile)] =
for {
user <- users
prof <- Stream.liftIO(getProfile(user.id))
} yield (user, prof)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val usersWithProfile:
Stream[Exception, (User, Profile)] = …
val targetUsers: Stream[Exception, User] =
usersWithProfile.filter(user =>
user.age >= 18 &&
user.age <= 32 &&
user.city == "San Francisco" &&
user.interests.contains("technology"))
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
type File = Stream[IOException, Report]
val monthlyFiles: List[File] = …
val allFiles =
monthlyFiles.foldLeft[File]
(Stream.empty)(_ ++ _)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val stream:
Stream[IOException, Chunk[Byte]] =
Stream.bracket(openFile)(
closeFile)(readChunk)
val usersJ: Stream[IOException, Json] =
stream.transduce(toJson)
.filter(isUserRecord)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val averages: Stream[Nothing, Int] =
integers.scan(0 -> 0) {
case ((sum, count), int) =>
(sum + int, count + 1) ->
if (count == 0) Int.MaxValue
else sum / count
}
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val first100Evens: Stream[Nothing, Int] =
integers.filter(_ % 2 == 0).take(100)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val evensUnder100: Stream[Nothing, Int] =
integers.filter(_ % 2 == 0)
.takeWhile(_ < 100)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val evensAfter100: Stream[Nothing, Int] =
integers.filter(_ % 2 == 0).drop(100)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val evensOver100: Stream[Nothing, Int] =
integers.filter(_ % 2 == 0)
.dropWhile(_ < 100)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val inputs: Stream[IOException, String] =
Stream.liftIO(getStrLn).forever
val echo: IO[IOException, Unit] =
inputs.foreach(putStrLn)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val integers: Stream[Nothing, Int] =
Stream.unfold(0)(i => (i, i + 1))
val users: Stream[Exception, User] = …
val usersWithId:
Stream[Exception, (Int, User)] =
integers.zip(users)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val stream1: Stream[IOException, JSON] =
kafka.stream("events1")
val stream2: Stream[IOException, JSON] =
kafka.stream("events2")
val both: Stream[IOException, JSON] =
stream1.merge(stream2)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val file1:
Stream[IOException, User] =
readFile("file1.data")
.transduce(toUser)
val file2:
Stream[IOException, User] =
readFile("file2.data")
.transduce(toUser)
val fastest: Stream[IOException, User] =
file1.joinWith(file2)(_ race _)
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
type StreamIO[A] = Stream[IOException, A]
val bytes: StreamIO[Byte] = …
val headersAndContent:
Managed[IOException,
(ContentType, StreamIO[Byte])] =
bytes.peel(parseContentType)
headersAndContent.use {
case (ContentType(Json), content) =>
content.transduce(toJson)...
case (ContentType(Xml), content) =>
content.transduce(toXml)...
}
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Sink[E, A0, A, B]
May fail with a checked error of type E
May effectfully produce a result of type B
Consumes values of type A
May possibly output a remainder of type A0
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
trait Stream[E, A] {
def run[R](sink: Sink[E, A, A, R]): IO[E, R]
}
myStream.run(Sink.drain) // IO[E, Unit]
myStream.run(decodeRequest) // IO[E, Request]
myStream.run(Sink.collect) // IO[E, List[A]]
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Monoid
append
zero
Applicative
map
point
ap
Monad
bind
Bifunctor
leftMap
bimap
Alternative
alt
empty
Category
id
compose
Profunctor
lmap
rmap
dimap
Strong
first
second
Choice
left
right
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val answer: Sink[Nothing, Nothing,
Any, Int] =
Sink.point(42)
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val lifted: Sink[Throwable, Nothing,
Any, Array[Byte]] =
Sink.liftIO(IO.syncThrowable(dangerousOp))
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val squared: Sink[Unit, Nothing,
Int, Int] =
Sink.lift(i => i * i)
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val failure: Sink[DomainError, Nothing,
Any, Nothing] =
Sink.fail(BadOperation)
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val json: Sink[Unit, Nothing,
Json, Json] =
Sink.await
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val drain: Sink[Nothing, Nothing,
Any, Unit] =
Sink.drain
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val collected: Sink[Nothing, Nothing,
Json, List[Json]] =
Sink.collect[Json]
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val parser: Sink[Nothing, Char,
Char, Either[Error, Int]] =
Sink.fold((ParserState.initial, List[Int]()) {
case (ParserState.Digits, c) if c.isDigit =>
// ...
}
Sink.point(value)
Sink.liftIO(ioValue)
Sink.lift(f)
Sink.fail(e)
Sink.await
Sink.drain
Sink.collect
Sink.fold(init)(step)
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val digits: Sink[Nothing, Char,
Char, List[Char]] =
Sink.readWhile[Char](_.isDigit)
val number: Sink[Nothing, Char,
Char, Int] =
Sink.map(_.mkString.toInt)
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val summing: Sink[Nothing, Int, Int, Int] =
Sink.fold(0)(_ + _)
case class Person(name: String, salary: Int)
val salarySum:
Sink[Nothing, Int, Person, Int] =
Sink.contramap[Person](_.salary)
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val err: Sink[DomainError, Nothing,
Any, Nothing] =
Sink.fail(DomainError)
val errs: Sink[AppError, Nothing,
Any, Nothing] =
Sink.mapError(AppError(_))
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val sumOfConsecutive:
Sink[Nothing, Nothing,
Int, Int] =
for {
a <- Sink.await
b <- Sink.await
} yield a + b
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val parseHeader: Sink[Err, Byte,
Byte, Header] =
…
val parseBody: Sink[Err, Byte,
Byte, Body] =
…
val parseHeaderAndBody:
Sink[Err, Byte, Byte, (Header, Body)] =
parseHeader ~ parseBody
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val parseHeader: Sink[Err, Byte,
Byte, Header] =
…
val maybeParseHeader:
Sink[Nothing, Byte,
Byte, Option[Header]] =
parseHeader ?
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val request1: Sink[Err, Byte,
Byte, Req1] = …
val request2: Sink[Err, Byte,
Byte, Req2] = …
val request:
Sink[Err, Byte,
Byte, Either[Req1, Req2]] =
request1 raceBoth request2
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val request1: Sink[Err, Byte,
Byte, Req1] = …
val request2: Sink[Err, Byte,
Byte, Req2] = …
val request:
Sink[Err, Byte,
Byte, Either[Req1, Req2]] =
request1 orElse request2
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
val parseUser: Sink[Err, Byte,
Byte, Person] = …
val parsePeople: Sink[Err, Byte,
Byte, List[Person]] =
parseUser repeat
sink.map(f)
sink.contramap(f)
sink.mapError(f)
sink.flatMap(f)
sink ~ otherSink
sink.?
sink.raceBoth(other)
sink.orElse(other)
sink.repeat
3 How Fast Is Scalaz Stream
PERFORMANCE
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Start with stream of integers (chunked)
Filter for only even integers
Convert to longs
Sum all integers
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def akkaChunkFilterMapSum = {
val chunks = (1 to chunkCount).flatMap(i =>
Array.fill(chunkSize)(i))
val program = AkkaSource
.fromIterator(() => chunks.iterator)
.filter(_ % 2 == 0)
.map(_.toLong)
.toMat(AkkaSink.fold(0L)(_ + _))(Keep.right)
Await.result(program.run, Duration.Inf)
}
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def fs2ChunkFilterMapSum = {
val chunks = (1 to chunkCount).map(i =>
FS2Chunk.array(Array.fill(chunkSize)(i)))
val stream = FS2Stream(chunks: _*)
.flatMap(FS2Stream.chunk(_))
.filter(_ % 2 == 0)
.map(_.toLong)
.covary[CatsIO]
.compile
.fold(0L)(_ + _)
stream.unsafeRunSync
}
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def scalazChunkFilterMapSum = {
val chunks = (1 to chunkCount).map(i =>
Chunk.fromArray(Array.fill(chunkSize)(i)))
val stream = StreamChunk
.fromChunks(chunks: _*)
.filter(_ % 2 == 0)
.map(_.toLong)
val sink = Sink.foldLeft(0L)((s, c) => c.foldl(s)(_ + _))
unsafeRun(stream.run(sink))
}
Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Start with stream of characters (chunked)
Identify CSV separators and columns
Convert character stream into token stream
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def akkaCsvTokenize() = {
val chunks = genCsvChunks
val program = AkkaSource
.fromIterator(() => chunks.flatten.iterator)
.scan((Vector.empty[Char], Vector.empty[CSV.Token])) {
case ((acc, _), char) =>
if (char == CSV.ColumnSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Vector(CSV.Column(acc.mkString))
else Vector.empty[CSV.Token]) ++
Vector(CSV.NewCol))
} else if (char == CSV.RowSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Vector(CSV.Column(acc.mkString))
else Vector.empty[CSV.Token]) ++
Vector(CSV.NewCol))
} else (acc :+ char) -> Vector.empty[CSV.Token]
}
.flatMapConcat(t => AkkaSource(t._2))
.toMat(AkkaSink.ignore)(Keep.right)
Await.result(program.run, Duration.Inf)
}
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def fs2CsvTokenize() = {
val chunks = genCsvChunks.map(FS2Chunk.array(_))
val stream = FS2Stream(chunks: _*)
.flatMap(FS2Stream.chunk(_))
.scan((Vector.empty[Char], Vector.empty[CSV.Token])) {
case ((acc, _), char) =>
if (char == CSV.ColumnSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Vector(CSV.Column(acc.mkString))
else Vector.empty[CSV.Token]) ++
Vector(CSV.NewCol))
} else if (char == CSV.RowSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Vector(CSV.Column(acc.mkString))
else Vector.empty[CSV.Token]) ++
Vector(CSV.NewCol))
} else (acc :+ char) -> Vector.empty[CSV.Token]
}
.flatMap(t => FS2Stream(t._2))
.covary[CatsIO]
.compile
.drain
stream.unsafeRunSync
}
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def scalazCsvTokenize() = {
val chunks = genCsvChunks.map(Chunk.fromArray(_))
val stream = ChunkedStream
.fromChunks(chunks: _*)
.scan[Vector[Char], Chunk[CSV.Token]](Vector.empty[Char]) {
case (acc, char) =>
if (char == CSV.ColumnSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Chunk(CSV.Column(acc.mkString))
else Chunk.empty) ++
Chunk(CSV.NewCol))
} else if (char == CSV.RowSep) {
Vector.empty[Char] ->
((if (acc.length > 0)
Chunk(CSV.Column(acc.mkString))
else Chunk.empty) ++
Chunk(CSV.NewCol))
} else (acc :+ char) -> Chunk.empty
}
.mapConcat(identity(_))
unsafeRun(stream.run(Sink.drain))
}
CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
4 The Inner Workings of Scalaz Stream
DESIGN
BUILD YOUR
OWN STREAM
Initial Encoding
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Initial Encoding
sealed trait Stream[+A] Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
case class Fold[A0, A](a: A,
s: Stream[A0],
f: (A, A0) => A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Initial Encoding
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
case class Fold[A0, A](a: A,
s: Stream[A0],
f: (A, A0) => A) extends Stream[A]
case class Effect[A](io: IO[A]) extends Stream[A]
Initial Encoding
def drain[A](s: Stream[A]): IO[Unit] =
s match {
case Emit(a) => …
...
}
Interpreter
PERFORMANCEMOTIVATION API DESIGN SUMMARY
BUILD YOUR
OWN STREAM
Final Encoding
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit])
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit])
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
def merge[A1 >: A](that: Stream[A1]) =
Stream[A1](r =>
run(r).par(that.run(r)).void)
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
def merge[A1 >: A](that: Stream[A1]) =
Stream[A1](r =>
run(r).par(that.run(r)).void)
def fold[S](s: S)(f: (S, A) => S) = ...
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
def merge[A1 >: A](that: Stream[A1]) =
Stream[A1](r =>
run(r).par(that.run(r)).void)
def fold[S](s: S)(f: (S, A) => S) = ...
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
def liftIO[A](io: IO[A]): Stream[A] =
Stream[A](io.flatMap(_))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
Final
Flexible
Lookahead optimizations
Ad Hoc
Non-uniform
Initial
Initial vs Final
✓
✓
𐄂
PERFORMANCEMOTIVATION API DESIGN SUMMARY
𐄂
Inflexible
No lookahead optimizations
Structured
Uniform
𐄂
𐄂
✓
✓
Constant space processing
Discovered in 2003
Aggressively lazy
ITERATEES
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Final encoding-friendly
Automatically leak-free
“Left folds”
Iteratees 101
sealed trait Iteratee[-A, +B]
case class More[A, B](f: A => Iteratee[A, B]) extends Iteratee[A, B]
case class Done[B](value: B) extends Iteratee[Any, B]
sealed trait Enumerator[+A] { self =>
def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B]
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
final def map[B](f0: A => B): Enumerator[B] =
new Enumerator[B] {
def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] =
self.fold(s)((s, a) => f(s, f0(a)))
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
final def map[B](f0: A => B): Enumerator[B] =
new Enumerator[B] {
def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] =
self.fold(s)((s, a) => f(s, f0(a)))
}
final def filter(f0: A => Boolean): Enumerator[A] =
new Enumerator[A] {
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] =
self.fold(s)((s, a) => if (f0(a)) f(s, a) else Left(s))
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
𐄂 Stack Safe𐄂 Combining 𐄂 Leftovers
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
𐄂 Stack Safe?𐄂 Combining? 𐄂 Leftovers?
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] =
lift(More(a => Done(a)))
def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] =
More(a => Done(a))
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] =
lift(More(a => Done(a)))
def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] =
More(a => Done(a))
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
}
object Stream {
sealed trait Step[S]
object Step {
case class Stop[S](value: S) extends Step[S]
case class Cont[S](value: S) extends Step[S]
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
}
object Stream {
sealed trait Step[S]
object Step {
case class Stop[S](value: S) extends Step[S]
case class Cont[S](value: S) extends Step[S]
}
}
Continuation Monad!
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
def toQueue(capacity: Int = 1): Managed[Nothing, Queue[Take[E, A]]] =
for {
queue <- Managed.liftIO(Queue.bounded[Take[E, A]](capacity))
offerVal = (a: A) => queue.offer(Take.Value(a)).void
offerErr = (e: E) => queue.offer(Take.Fail(e))
enqueuer = (self.foreach[E, A](offerVal).catchAll(offerErr) *>
queue.offer(Take.End).forever).fork
_ <- Managed(enqueuer)(_.interrupt)
} yield queue
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
def zip[B, C](that: Stream[E, B]): Stream[E, (A, B)] =
new Stream[E, (A, B)] {
type Q[A] = Queue[Take[E, A]]
def fold[S](s: S)(f: (S, (A, B)) => IO[E, Step[S]]): IO[E, Step[S]] = {
def loop(q1: Q[A], q2: Q[B], s: S): IO[E, Step[S]] =
Take.option(q1.take).seq(Take.option(q2.take)).flatMap {
case (Some(a), Some(b)) => f(s, (a, b))
case _ => IO.now(Step.Cont(s))
}
self.toQueue().use(q1 => that.toQueue().use(q2 => loop(q1, q2, s)))
}
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
𐄂 Stack Safe✓ Combining 𐄂 Leftovers
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
def peel[R](sink: Sink[E, A, A, R]): Managed[E, (R, Stream[E, A])] =
...
}
Compose all leading sinks
Peel off sink (e.g. headers)
Work with stream remainder
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
𐄂 Stack Safe✓ Combining ✓ Leftovers
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
trait Stream[E, A] {
def fold[S]: IO[Nothing, (S, (S, A) => IO[E, Step[S]]) => IO[E, Step[S]]]
// trampolined: F[A => F[B]]
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
ZIO Stream
✓ Stack Safe✓ Combining ✓ Leftovers
PERFORMANCEMOTIVATION API DESIGN SUMMARY
5 Where To From Here
SUMMARY
Fusion, Queue &
complex combinators
Performance
Outstanding Work
Comprehensive test
suites and great docs
Tests & Docs
Combinators for
every reason
Combinators
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Eric Torreborre — Beautiful Folds
The FS2 team — Michael Pilquist, Fabio Labella, Pavel Chlupacek, et al
Special thanks to all people who made ZIO Stream possible.
Credits
Oleg Kiselyov — Iteratees 2003
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Any questions?
Thanks!
John A. De Goes
@jdegoes - degoes.net
Itamar Ravid
@itrvd - iravid.com
PERFORMANCEMOTIVATION API DESIGN SUMMARY

More Related Content

What's hot

How Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queriesHow Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queriesponyorm
 
How do you create a programming language for the JVM?
How do you create a programming language for the JVM?How do you create a programming language for the JVM?
How do you create a programming language for the JVM?Federico Tomassetti
 
SE 20016 - programming languages landscape.
SE 20016 - programming languages landscape.SE 20016 - programming languages landscape.
SE 20016 - programming languages landscape.Ruslan Shevchenko
 
LINQ Inside
LINQ InsideLINQ Inside
LINQ Insidejeffz
 
InterConnect: Server Side Swift for Java Developers
InterConnect:  Server Side Swift for Java DevelopersInterConnect:  Server Side Swift for Java Developers
InterConnect: Server Side Swift for Java DevelopersChris Bailey
 
scala-gopher: async implementation of CSP for scala
scala-gopher:  async implementation of CSP  for  scalascala-gopher:  async implementation of CSP  for  scala
scala-gopher: async implementation of CSP for scalaRuslan Shevchenko
 
Kotlin as a Better Java
Kotlin as a Better JavaKotlin as a Better Java
Kotlin as a Better JavaGarth Gilmour
 
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad Bulgaria
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad BulgariaStreams or Loops? Java 8 Stream API by Niki Petkov - Proxiad Bulgaria
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad BulgariaHackBulgaria
 
What's New in ES6 for Web Devs
What's New in ES6 for Web DevsWhat's New in ES6 for Web Devs
What's New in ES6 for Web DevsRami Sayar
 
Kotlin Bytecode Generation and Runtime Performance
Kotlin Bytecode Generation and Runtime PerformanceKotlin Bytecode Generation and Runtime Performance
Kotlin Bytecode Generation and Runtime Performanceintelliyole
 
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...eMan s.r.o.
 
API first with Swagger and Scala by Slava Schmidt
API first with Swagger and Scala by  Slava SchmidtAPI first with Swagger and Scala by  Slava Schmidt
API first with Swagger and Scala by Slava SchmidtJavaDayUA
 
Streams for (Co)Free!
Streams for (Co)Free!Streams for (Co)Free!
Streams for (Co)Free!John De Goes
 
Future vs. Monix Task
Future vs. Monix TaskFuture vs. Monix Task
Future vs. Monix TaskHermann Hueck
 
Intro to Functional Programming in Scala
Intro to Functional Programming in ScalaIntro to Functional Programming in Scala
Intro to Functional Programming in ScalaShai Yallin
 
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...John De Goes
 

What's hot (20)

How Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queriesHow Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queries
 
Csp scala wixmeetup2016
Csp scala wixmeetup2016Csp scala wixmeetup2016
Csp scala wixmeetup2016
 
How do you create a programming language for the JVM?
How do you create a programming language for the JVM?How do you create a programming language for the JVM?
How do you create a programming language for the JVM?
 
SE 20016 - programming languages landscape.
SE 20016 - programming languages landscape.SE 20016 - programming languages landscape.
SE 20016 - programming languages landscape.
 
LINQ Inside
LINQ InsideLINQ Inside
LINQ Inside
 
InterConnect: Server Side Swift for Java Developers
InterConnect:  Server Side Swift for Java DevelopersInterConnect:  Server Side Swift for Java Developers
InterConnect: Server Side Swift for Java Developers
 
scala-gopher: async implementation of CSP for scala
scala-gopher:  async implementation of CSP  for  scalascala-gopher:  async implementation of CSP  for  scala
scala-gopher: async implementation of CSP for scala
 
Kotlin on android
Kotlin on androidKotlin on android
Kotlin on android
 
Kotlin as a Better Java
Kotlin as a Better JavaKotlin as a Better Java
Kotlin as a Better Java
 
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad Bulgaria
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad BulgariaStreams or Loops? Java 8 Stream API by Niki Petkov - Proxiad Bulgaria
Streams or Loops? Java 8 Stream API by Niki Petkov - Proxiad Bulgaria
 
What's New in ES6 for Web Devs
What's New in ES6 for Web DevsWhat's New in ES6 for Web Devs
What's New in ES6 for Web Devs
 
Kotlin Bytecode Generation and Runtime Performance
Kotlin Bytecode Generation and Runtime PerformanceKotlin Bytecode Generation and Runtime Performance
Kotlin Bytecode Generation and Runtime Performance
 
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...
eMan Dev Meetup: Kotlin - A Language we should know it exists (part 02/03) 18...
 
API first with Swagger and Scala by Slava Schmidt
API first with Swagger and Scala by  Slava SchmidtAPI first with Swagger and Scala by  Slava Schmidt
API first with Swagger and Scala by Slava Schmidt
 
Streams for (Co)Free!
Streams for (Co)Free!Streams for (Co)Free!
Streams for (Co)Free!
 
Future vs. Monix Task
Future vs. Monix TaskFuture vs. Monix Task
Future vs. Monix Task
 
Time for Functions
Time for FunctionsTime for Functions
Time for Functions
 
Intro to Functional Programming in Scala
Intro to Functional Programming in ScalaIntro to Functional Programming in Scala
Intro to Functional Programming in Scala
 
Functional Programming in Scala
Functional Programming in ScalaFunctional Programming in Scala
Functional Programming in Scala
 
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...
 

Similar to Scalaz Stream: Rebirth

Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Thomas Weise
 
A Brief Conceptual Introduction to Functional Java 8 and its API
A Brief Conceptual Introduction to Functional Java 8 and its APIA Brief Conceptual Introduction to Functional Java 8 and its API
A Brief Conceptual Introduction to Functional Java 8 and its APIJörn Guy Süß JGS
 
Real-time Stream Processing with Apache Flink @ Hadoop Summit
Real-time Stream Processing with Apache Flink @ Hadoop SummitReal-time Stream Processing with Apache Flink @ Hadoop Summit
Real-time Stream Processing with Apache Flink @ Hadoop SummitGyula Fóra
 
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...Big Data Spain
 
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | Prometheus
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | PrometheusCreating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | Prometheus
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | PrometheusInfluxData
 
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015Robert Metzger
 
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015Lori MacVittie
 
Django REST Framework における API 実装プラクティス | PyCon JP 2018
Django REST Framework における API 実装プラクティス | PyCon JP 2018Django REST Framework における API 実装プラクティス | PyCon JP 2018
Django REST Framework における API 実装プラクティス | PyCon JP 2018Masashi Shibata
 
.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011Fabio Akita
 
09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do railsDNAD
 
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...HostedbyConfluent
 
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...Robert Metzger
 
Meet the Forge Runtime
Meet the Forge RuntimeMeet the Forge Runtime
Meet the Forge RuntimeAtlassian
 

Similar to Scalaz Stream: Rebirth (20)

Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
 
A Brief Conceptual Introduction to Functional Java 8 and its API
A Brief Conceptual Introduction to Functional Java 8 and its APIA Brief Conceptual Introduction to Functional Java 8 and its API
A Brief Conceptual Introduction to Functional Java 8 and its API
 
Real-time Stream Processing with Apache Flink @ Hadoop Summit
Real-time Stream Processing with Apache Flink @ Hadoop SummitReal-time Stream Processing with Apache Flink @ Hadoop Summit
Real-time Stream Processing with Apache Flink @ Hadoop Summit
 
Intro to Akka Streams
Intro to Akka StreamsIntro to Akka Streams
Intro to Akka Streams
 
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...
 
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | Prometheus
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | PrometheusCreating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | Prometheus
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | Prometheus
 
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015
Unified batch and stream processing with Flink @ Big Data Beers Berlin May 2015
 
RxJava Applied
RxJava AppliedRxJava Applied
RxJava Applied
 
Rack
RackRack
Rack
 
Scalaxb preso
Scalaxb presoScalaxb preso
Scalaxb preso
 
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015
Architectural Patterns for Scaling Microservices and APIs - GlueCon 2015
 
Django REST Framework における API 実装プラクティス | PyCon JP 2018
Django REST Framework における API 実装プラクティス | PyCon JP 2018Django REST Framework における API 実装プラクティス | PyCon JP 2018
Django REST Framework における API 実装プラクティス | PyCon JP 2018
 
Wider than rails
Wider than railsWider than rails
Wider than rails
 
.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011
 
09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails
 
Apache Flink Deep Dive
Apache Flink Deep DiveApache Flink Deep Dive
Apache Flink Deep Dive
 
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...
 
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...
Apache Flink Meetup Munich (November 2015): Flink Overview, Architecture, Int...
 
Hello istio
Hello istioHello istio
Hello istio
 
Meet the Forge Runtime
Meet the Forge RuntimeMeet the Forge Runtime
Meet the Forge Runtime
 

More from John De Goes

Refactoring Functional Type Classes
Refactoring Functional Type ClassesRefactoring Functional Type Classes
Refactoring Functional Type ClassesJohn De Goes
 
One Monad to Rule Them All
One Monad to Rule Them AllOne Monad to Rule Them All
One Monad to Rule Them AllJohn De Goes
 
Error Management: Future vs ZIO
Error Management: Future vs ZIOError Management: Future vs ZIO
Error Management: Future vs ZIOJohn De Goes
 
Atomically { Delete Your Actors }
Atomically { Delete Your Actors }Atomically { Delete Your Actors }
Atomically { Delete Your Actors }John De Goes
 
The Death of Final Tagless
The Death of Final TaglessThe Death of Final Tagless
The Death of Final TaglessJohn De Goes
 
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional Programming
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional Programming
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingJohn De Goes
 
Blazing Fast, Pure Effects without Monads — LambdaConf 2018
Blazing Fast, Pure Effects without Monads — LambdaConf 2018Blazing Fast, Pure Effects without Monads — LambdaConf 2018
Blazing Fast, Pure Effects without Monads — LambdaConf 2018John De Goes
 
Scalaz 8: A Whole New Game
Scalaz 8: A Whole New GameScalaz 8: A Whole New Game
Scalaz 8: A Whole New GameJohn De Goes
 
Scalaz 8 vs Akka Actors
Scalaz 8 vs Akka ActorsScalaz 8 vs Akka Actors
Scalaz 8 vs Akka ActorsJohn De Goes
 
Orthogonal Functional Architecture
Orthogonal Functional ArchitectureOrthogonal Functional Architecture
Orthogonal Functional ArchitectureJohn De Goes
 
Getting Started with PureScript
Getting Started with PureScriptGetting Started with PureScript
Getting Started with PureScriptJohn De Goes
 
SlamData - How MongoDB Is Powering a Revolution in Visual Analytics
SlamData - How MongoDB Is Powering a Revolution in Visual AnalyticsSlamData - How MongoDB Is Powering a Revolution in Visual Analytics
SlamData - How MongoDB Is Powering a Revolution in Visual AnalyticsJohn De Goes
 
The Next Great Functional Programming Language
The Next Great Functional Programming LanguageThe Next Great Functional Programming Language
The Next Great Functional Programming LanguageJohn De Goes
 
The Dark Side of NoSQL
The Dark Side of NoSQLThe Dark Side of NoSQL
The Dark Side of NoSQLJohn De Goes
 
First-Class Patterns
First-Class PatternsFirst-Class Patterns
First-Class PatternsJohn De Goes
 
Quirrel & R for Dummies
Quirrel & R for DummiesQuirrel & R for Dummies
Quirrel & R for DummiesJohn De Goes
 
In-Database Predictive Analytics
In-Database Predictive AnalyticsIn-Database Predictive Analytics
In-Database Predictive AnalyticsJohn De Goes
 
Analytics Maturity Model
Analytics Maturity ModelAnalytics Maturity Model
Analytics Maturity ModelJohn De Goes
 
Rise of the scientific database
Rise of the scientific databaseRise of the scientific database
Rise of the scientific databaseJohn De Goes
 

More from John De Goes (20)

Refactoring Functional Type Classes
Refactoring Functional Type ClassesRefactoring Functional Type Classes
Refactoring Functional Type Classes
 
One Monad to Rule Them All
One Monad to Rule Them AllOne Monad to Rule Them All
One Monad to Rule Them All
 
Error Management: Future vs ZIO
Error Management: Future vs ZIOError Management: Future vs ZIO
Error Management: Future vs ZIO
 
Atomically { Delete Your Actors }
Atomically { Delete Your Actors }Atomically { Delete Your Actors }
Atomically { Delete Your Actors }
 
The Death of Final Tagless
The Death of Final TaglessThe Death of Final Tagless
The Death of Final Tagless
 
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional Programming
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional Programming
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional Programming
 
Blazing Fast, Pure Effects without Monads — LambdaConf 2018
Blazing Fast, Pure Effects without Monads — LambdaConf 2018Blazing Fast, Pure Effects without Monads — LambdaConf 2018
Blazing Fast, Pure Effects without Monads — LambdaConf 2018
 
Scalaz 8: A Whole New Game
Scalaz 8: A Whole New GameScalaz 8: A Whole New Game
Scalaz 8: A Whole New Game
 
Scalaz 8 vs Akka Actors
Scalaz 8 vs Akka ActorsScalaz 8 vs Akka Actors
Scalaz 8 vs Akka Actors
 
Orthogonal Functional Architecture
Orthogonal Functional ArchitectureOrthogonal Functional Architecture
Orthogonal Functional Architecture
 
MTL Versus Free
MTL Versus FreeMTL Versus Free
MTL Versus Free
 
Getting Started with PureScript
Getting Started with PureScriptGetting Started with PureScript
Getting Started with PureScript
 
SlamData - How MongoDB Is Powering a Revolution in Visual Analytics
SlamData - How MongoDB Is Powering a Revolution in Visual AnalyticsSlamData - How MongoDB Is Powering a Revolution in Visual Analytics
SlamData - How MongoDB Is Powering a Revolution in Visual Analytics
 
The Next Great Functional Programming Language
The Next Great Functional Programming LanguageThe Next Great Functional Programming Language
The Next Great Functional Programming Language
 
The Dark Side of NoSQL
The Dark Side of NoSQLThe Dark Side of NoSQL
The Dark Side of NoSQL
 
First-Class Patterns
First-Class PatternsFirst-Class Patterns
First-Class Patterns
 
Quirrel & R for Dummies
Quirrel & R for DummiesQuirrel & R for Dummies
Quirrel & R for Dummies
 
In-Database Predictive Analytics
In-Database Predictive AnalyticsIn-Database Predictive Analytics
In-Database Predictive Analytics
 
Analytics Maturity Model
Analytics Maturity ModelAnalytics Maturity Model
Analytics Maturity Model
 
Rise of the scientific database
Rise of the scientific databaseRise of the scientific database
Rise of the scientific database
 

Recently uploaded

Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4DianaGray10
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024Brian Pichman
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsDianaGray10
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechProduct School
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024Brian Pichman
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 

Recently uploaded (20)

Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projects
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
SheDev 2024
SheDev 2024SheDev 2024
SheDev 2024
 

Scalaz Stream: Rebirth

  • 1. John A. De Goes @jdegoes - degoes.net SCALAZ STREAM REBIRTH Scale By The Bay · November 16 · San Francisco Itamar Ravid @itrvd - iravid.com
  • 4. 1 Why Streams & Why Scalaz Stream MOTIVATION
  • 5. Why Streams? PERFORMANCEMOTIVATION API DESIGN SUMMARY Infinite Input Sets Leak-Free Data Processing Incremental Computation ∞
  • 6. File & Socket Processing Graph Processing Reactive & Dataflow Akka Streams FS2 Scalaz ReactiveGraphJet Why ZIO Stream? PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 7. Why ZIO Stream? File & Socket Processing Graph Processing Reactive & Dataflow Akka Streams FS2 Scalaz ReactiveGraphJet ZIO STREAM PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 8. Why ZIO Stream? PERFORMANCEMOTIVATION API DESIGN SUMMARY 𐄂 Cycles𐄂 Cons 𐄂 Uncons
  • 9. ZIO Stream Old Scalaz Stream Akka Streams FS2 Lazy ✓ 𐄂 𐄂 ✓ Leak-Free ✓ 𐄂 𐄂 On a good day Uniform ✓ NA NA 𐄂 Type Inferred ✓ 𐄂 ✓ 𐄂 Economical ✓ 𐄂 𐄂 ✓ ZIO Integration ✓ 𐄂 𐄂 𐄂 PERFORMANCEMOTIVATION API DESIGN SUMMARY Why ZIO Stream?
  • 10. 2 What Scalaz Stream Looks Like API
  • 11. PERFORMANCEMOTIVATION API DESIGN SUMMARY AkkaStream AbruptTerminationException AbstractShape ActorAttributes ActorMaterializer ActorMaterializerSettings AmorphousShape Attributes BidiShape BindFailedException BufferOverflowException Client ClosedShape ConnectionException DelayOverflowStrategy EagerClose FanInShape FanInShape10 FanInShape11 FanInShape12 FanInShape13 FanInShape14 FanInShape15 FanInShape16 FanInShape17 FanInShape18 FanInShape19 FanInShape1N FanInShape2 FanInShape20 FanInShape21 FanInShape22 FanInShape3 FanInShape4 FanInShape5 FanInShape6 FanInShape7 FanInShape8 FanInShape9 FanOutShape FanOutShape10 FanOutShape11 FanOutShape12 FanOutShape13 FanOutShape14 FanOutShape15 FanOutShape16 FanOutShape17 FanOutShape18 FanOutShape19 FanOutShape2 FanOutShape20 FanOutShape21 FanOutShape22 FanOutShape3 FanOutShape4 FanOutShape5 FanOutShape6 FanOutShape7 FanOutShape8 FanOutShape9 FlowMonitor FlowMonitorState FlowShape Fusing Graph IgnoreBoth IgnoreCancel IgnoreComplete Inlet InPort IOResult KillSwitch KillSwitches MaterializationContext MaterializationException Materializer MaterializerLoggingProvider Outlet OutPort OverflowStrategy QueueOfferResult RateExceededException Server Shape SharedKillSwitch SinkShape SourceShape StreamLimitReachedException StreamSubscriptionTimeoutSettings StreamSubscriptionTimeoutTerminationMode StreamTcpException SubstreamCancelStrategy Supervision ThrottleMode TLSClientAuth TLSClosing TLSProtocol TLSRole UniformFanInShape UniformFanOutShape UniqueKillSwitch
  • 12. PERFORMANCEMOTIVATION API DESIGN SUMMARY AkkaStream AbruptTerminationException AbstractShape ActorAttributes ActorMaterializer ActorMaterializerSettings AmorphousShape Attributes BidiShape BindFailedException BufferOverflowException Client ClosedShape ConnectionException DelayOverflowStrategy EagerClose FanInShape FanInShape10 FanInShape11 FanInShape12 FanInShape13 FanInShape14 FanInShape15 FanInShape16 FanInShape17 FanInShape18 FanInShape19 FanInShape1N FanInShape2 FanInShape20 FanInShape21 FanInShape22 FanInShape3 FanInShape4 FanInShape5 FanInShape6 FanInShape7 FanInShape8 FanInShape9 FanOutShape FanOutShape10 FanOutShape11 FanOutShape12 FanOutShape13 FanOutShape14 FanOutShape15 FanOutShape16 FanOutShape17 FanOutShape18 FanOutShape19 FanOutShape2 FanOutShape20 FanOutShape21 FanOutShape22 FanOutShape3 FanOutShape4 FanOutShape5 FanOutShape6 FanOutShape7 FanOutShape8 FanOutShape9 FlowMonitor FlowMonitorState FlowShape Fusing Graph IgnoreBoth IgnoreCancel IgnoreComplete Inlet InPort IOResult KillSwitch KillSwitches MaterializationContext MaterializationException Materializer MaterializerLoggingProvider Outlet OutPort OverflowStrategy QueueOfferResult RateExceededException Server Shape SharedKillSwitch SinkShape SourceShape StreamLimitReachedException StreamSubscriptionTimeoutSettings StreamSubscriptionTimeoutTerminationMode StreamTcpException SubstreamCancelStrategy Supervision ThrottleMode TLSClientAuth TLSClosing TLSProtocol TLSRole UniformFanInShape UniformFanOutShape UniqueKillSwitch
  • 13. Stream Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY ZIO STREAM
  • 14. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY Stream[E, A] May fail with a checked error of type E May effectfully produce zero or more values of type A
  • 15. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY Monoid append zero Applicative map point ap Monad bind Bifunctor leftMap bimap Alternative alt empty
  • 16. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read) val empty: Stream[Nothing, Nothing] = Stream.empty
  • 17. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val singleton: Stream[Nothing, Int] = Stream.point(42) Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 18. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val hundred: Stream[Nothing, Int] = Stream.fromIterable(0 to 100) Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 19. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val userIn: Stream[IOException, String] = Stream.liftIO(getStrLn) Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 20. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val intQueue: Queue[Int] = ... val intStream: Stream[Nothing, Int] = Stream.fromQueue(intQueue) Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 21. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 22. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val latency: Stream[Nothing, Long] = Stream.unfoldM(0 -> 0) { case (n, total) => api.measureLatency(l => (n + 1) -> (total + n)) } Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 23. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val fileBytes: Stream[IOException, Byte] = Stream.bracket(openFile)(closeFile(_)) { file => file.readByte.attempt.toOption } Stream.empty Stream.point(v) Stream.fromIterable(list) Stream.liftIO(io) Stream.fromQueue(queue) Stream.unfold(s)(f) Stream.unfoldM(s)(f) Stream.bracket(open)(close)(read)
  • 24. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val integerDigits: Stream[Nothing, Int] = integers.map(_.toString).map(_.length) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 25. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val users: Stream[Exception, User] = ... val usersWithProfile: Stream[Exception, (User, Profile)] = for { user <- users prof <- Stream.liftIO(getProfile(user.id)) } yield (user, prof) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 26. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val usersWithProfile: Stream[Exception, (User, Profile)] = … val targetUsers: Stream[Exception, User] = usersWithProfile.filter(user => user.age >= 18 && user.age <= 32 && user.city == "San Francisco" && user.interests.contains("technology")) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 27. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY type File = Stream[IOException, Report] val monthlyFiles: List[File] = … val allFiles = monthlyFiles.foldLeft[File] (Stream.empty)(_ ++ _) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 28. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val stream: Stream[IOException, Chunk[Byte]] = Stream.bracket(openFile)( closeFile)(readChunk) val usersJ: Stream[IOException, Json] = stream.transduce(toJson) .filter(isUserRecord) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 29. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val averages: Stream[Nothing, Int] = integers.scan(0 -> 0) { case ((sum, count), int) => (sum + int, count + 1) -> if (count == 0) Int.MaxValue else sum / count } stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 30. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val first100Evens: Stream[Nothing, Int] = integers.filter(_ % 2 == 0).take(100) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 31. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val evensUnder100: Stream[Nothing, Int] = integers.filter(_ % 2 == 0) .takeWhile(_ < 100) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 32. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val evensAfter100: Stream[Nothing, Int] = integers.filter(_ % 2 == 0).drop(100) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 33. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val evensOver100: Stream[Nothing, Int] = integers.filter(_ % 2 == 0) .dropWhile(_ < 100) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 34. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val inputs: Stream[IOException, String] = Stream.liftIO(getStrLn).forever val echo: IO[IOException, Unit] = inputs.foreach(putStrLn) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 35. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val integers: Stream[Nothing, Int] = Stream.unfold(0)(i => (i, i + 1)) val users: Stream[Exception, User] = … val usersWithId: Stream[Exception, (Int, User)] = integers.zip(users) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 36. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val stream1: Stream[IOException, JSON] = kafka.stream("events1") val stream2: Stream[IOException, JSON] = kafka.stream("events2") val both: Stream[IOException, JSON] = stream1.merge(stream2) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 37. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY val file1: Stream[IOException, User] = readFile("file1.data") .transduce(toUser) val file2: Stream[IOException, User] = readFile("file2.data") .transduce(toUser) val fastest: Stream[IOException, User] = file1.joinWith(file2)(_ race _) stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 38. Stream PERFORMANCEMOTIVATION API DESIGN SUMMARY type StreamIO[A] = Stream[IOException, A] val bytes: StreamIO[Byte] = … val headersAndContent: Managed[IOException, (ContentType, StreamIO[Byte])] = bytes.peel(parseContentType) headersAndContent.use { case (ContentType(Json), content) => content.transduce(toJson)... case (ContentType(Xml), content) => content.transduce(toXml)... } stream.map(f) stream.flatMap(f) stream.filter(f) stream1 ++ stream2 stream.transduce(what) stream.scan(z)(f) stream.take(n) stream.takeWhile(f) stream.drop(n) stream.dropWhile(f) stream.foreach(f) stream1.zip(stream2) stream1.merge(stream2) stream1.joinWith(stream2)(f) stream.peel(what)
  • 39. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY Sink[E, A0, A, B] May fail with a checked error of type E May effectfully produce a result of type B Consumes values of type A May possibly output a remainder of type A0
  • 40. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY trait Stream[E, A] { def run[R](sink: Sink[E, A, A, R]): IO[E, R] } myStream.run(Sink.drain) // IO[E, Unit] myStream.run(decodeRequest) // IO[E, Request] myStream.run(Sink.collect) // IO[E, List[A]]
  • 41. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY Monoid append zero Applicative map point ap Monad bind Bifunctor leftMap bimap Alternative alt empty Category id compose Profunctor lmap rmap dimap Strong first second Choice left right
  • 42. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val answer: Sink[Nothing, Nothing, Any, Int] = Sink.point(42) Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 43. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val lifted: Sink[Throwable, Nothing, Any, Array[Byte]] = Sink.liftIO(IO.syncThrowable(dangerousOp)) Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 44. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val squared: Sink[Unit, Nothing, Int, Int] = Sink.lift(i => i * i) Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 45. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val failure: Sink[DomainError, Nothing, Any, Nothing] = Sink.fail(BadOperation) Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 46. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val json: Sink[Unit, Nothing, Json, Json] = Sink.await Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 47. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val drain: Sink[Nothing, Nothing, Any, Unit] = Sink.drain Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 48. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val collected: Sink[Nothing, Nothing, Json, List[Json]] = Sink.collect[Json] Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 49. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val parser: Sink[Nothing, Char, Char, Either[Error, Int]] = Sink.fold((ParserState.initial, List[Int]()) { case (ParserState.Digits, c) if c.isDigit => // ... } Sink.point(value) Sink.liftIO(ioValue) Sink.lift(f) Sink.fail(e) Sink.await Sink.drain Sink.collect Sink.fold(init)(step)
  • 50. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val digits: Sink[Nothing, Char, Char, List[Char]] = Sink.readWhile[Char](_.isDigit) val number: Sink[Nothing, Char, Char, Int] = Sink.map(_.mkString.toInt) sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 51. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val summing: Sink[Nothing, Int, Int, Int] = Sink.fold(0)(_ + _) case class Person(name: String, salary: Int) val salarySum: Sink[Nothing, Int, Person, Int] = Sink.contramap[Person](_.salary) sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 52. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val err: Sink[DomainError, Nothing, Any, Nothing] = Sink.fail(DomainError) val errs: Sink[AppError, Nothing, Any, Nothing] = Sink.mapError(AppError(_)) sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 53. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val sumOfConsecutive: Sink[Nothing, Nothing, Int, Int] = for { a <- Sink.await b <- Sink.await } yield a + b sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 54. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val parseHeader: Sink[Err, Byte, Byte, Header] = … val parseBody: Sink[Err, Byte, Byte, Body] = … val parseHeaderAndBody: Sink[Err, Byte, Byte, (Header, Body)] = parseHeader ~ parseBody sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 55. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val parseHeader: Sink[Err, Byte, Byte, Header] = … val maybeParseHeader: Sink[Nothing, Byte, Byte, Option[Header]] = parseHeader ? sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 56. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val request1: Sink[Err, Byte, Byte, Req1] = … val request2: Sink[Err, Byte, Byte, Req2] = … val request: Sink[Err, Byte, Byte, Either[Req1, Req2]] = request1 raceBoth request2 sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 57. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val request1: Sink[Err, Byte, Byte, Req1] = … val request2: Sink[Err, Byte, Byte, Req2] = … val request: Sink[Err, Byte, Byte, Either[Req1, Req2]] = request1 orElse request2 sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 58. Sink PERFORMANCEMOTIVATION API DESIGN SUMMARY val parseUser: Sink[Err, Byte, Byte, Person] = … val parsePeople: Sink[Err, Byte, Byte, List[Person]] = parseUser repeat sink.map(f) sink.contramap(f) sink.mapError(f) sink.flatMap(f) sink ~ otherSink sink.? sink.raceBoth(other) sink.orElse(other) sink.repeat
  • 59. 3 How Fast Is Scalaz Stream PERFORMANCE
  • 60. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY Start with stream of integers (chunked) Filter for only even integers Convert to longs Sum all integers
  • 61. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY def akkaChunkFilterMapSum = { val chunks = (1 to chunkCount).flatMap(i => Array.fill(chunkSize)(i)) val program = AkkaSource .fromIterator(() => chunks.iterator) .filter(_ % 2 == 0) .map(_.toLong) .toMat(AkkaSink.fold(0L)(_ + _))(Keep.right) Await.result(program.run, Duration.Inf) }
  • 62. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 63. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY def fs2ChunkFilterMapSum = { val chunks = (1 to chunkCount).map(i => FS2Chunk.array(Array.fill(chunkSize)(i))) val stream = FS2Stream(chunks: _*) .flatMap(FS2Stream.chunk(_)) .filter(_ % 2 == 0) .map(_.toLong) .covary[CatsIO] .compile .fold(0L)(_ + _) stream.unsafeRunSync }
  • 64. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY FS2
  • 65. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY def scalazChunkFilterMapSum = { val chunks = (1 to chunkCount).map(i => Chunk.fromArray(Array.fill(chunkSize)(i))) val stream = StreamChunk .fromChunks(chunks: _*) .filter(_ % 2 == 0) .map(_.toLong) val sink = Sink.foldLeft(0L)((s, c) => c.foldl(s)(_ + _)) unsafeRun(stream.run(sink)) }
  • 66. Filter · Map · Sum PERFORMANCEMOTIVATION API DESIGN SUMMARY FS2
  • 67. CSV Tokenize PERFORMANCEMOTIVATION API DESIGN SUMMARY Start with stream of characters (chunked) Identify CSV separators and columns Convert character stream into token stream
  • 68. CSV Tokenize PERFORMANCEMOTIVATION API DESIGN SUMMARY def akkaCsvTokenize() = { val chunks = genCsvChunks val program = AkkaSource .fromIterator(() => chunks.flatten.iterator) .scan((Vector.empty[Char], Vector.empty[CSV.Token])) { case ((acc, _), char) => if (char == CSV.ColumnSep) { Vector.empty[Char] -> ((if (acc.length > 0) Vector(CSV.Column(acc.mkString)) else Vector.empty[CSV.Token]) ++ Vector(CSV.NewCol)) } else if (char == CSV.RowSep) { Vector.empty[Char] -> ((if (acc.length > 0) Vector(CSV.Column(acc.mkString)) else Vector.empty[CSV.Token]) ++ Vector(CSV.NewCol)) } else (acc :+ char) -> Vector.empty[CSV.Token] } .flatMapConcat(t => AkkaSource(t._2)) .toMat(AkkaSink.ignore)(Keep.right) Await.result(program.run, Duration.Inf) }
  • 70. CSV Tokenize PERFORMANCEMOTIVATION API DESIGN SUMMARY def fs2CsvTokenize() = { val chunks = genCsvChunks.map(FS2Chunk.array(_)) val stream = FS2Stream(chunks: _*) .flatMap(FS2Stream.chunk(_)) .scan((Vector.empty[Char], Vector.empty[CSV.Token])) { case ((acc, _), char) => if (char == CSV.ColumnSep) { Vector.empty[Char] -> ((if (acc.length > 0) Vector(CSV.Column(acc.mkString)) else Vector.empty[CSV.Token]) ++ Vector(CSV.NewCol)) } else if (char == CSV.RowSep) { Vector.empty[Char] -> ((if (acc.length > 0) Vector(CSV.Column(acc.mkString)) else Vector.empty[CSV.Token]) ++ Vector(CSV.NewCol)) } else (acc :+ char) -> Vector.empty[CSV.Token] } .flatMap(t => FS2Stream(t._2)) .covary[CatsIO] .compile .drain stream.unsafeRunSync }
  • 72. CSV Tokenize PERFORMANCEMOTIVATION API DESIGN SUMMARY def scalazCsvTokenize() = { val chunks = genCsvChunks.map(Chunk.fromArray(_)) val stream = ChunkedStream .fromChunks(chunks: _*) .scan[Vector[Char], Chunk[CSV.Token]](Vector.empty[Char]) { case (acc, char) => if (char == CSV.ColumnSep) { Vector.empty[Char] -> ((if (acc.length > 0) Chunk(CSV.Column(acc.mkString)) else Chunk.empty) ++ Chunk(CSV.NewCol)) } else if (char == CSV.RowSep) { Vector.empty[Char] -> ((if (acc.length > 0) Chunk(CSV.Column(acc.mkString)) else Chunk.empty) ++ Chunk(CSV.NewCol)) } else (acc :+ char) -> Chunk.empty } .mapConcat(identity(_)) unsafeRun(stream.run(Sink.drain)) }
  • 74. 4 The Inner Workings of Scalaz Stream DESIGN
  • 75. BUILD YOUR OWN STREAM Initial Encoding PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 76. Initial Encoding sealed trait Stream[+A] Type Emission Mapping Joining Merging Folding Effect ... PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 77. Initial Encoding sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 78. Initial Encoding sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] case class Map[A0, A](s: Stream[A0], f: A0 => A) extends Stream[A] PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 79. Initial Encoding sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] case class Map[A0, A](s: Stream[A0], f: A0 => A) extends Stream[A] case class Join[A](s: Stream[Stream[A]]) extends Stream[A] PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 80. Initial Encoding sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] case class Map[A0, A](s: Stream[A0], f: A0 => A) extends Stream[A] case class Join[A](s: Stream[Stream[A]]) extends Stream[A] case class Merge[A](l: Stream[A], r: Stream[A]) extends Stream[A] PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 81. Initial Encoding sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] case class Map[A0, A](s: Stream[A0], f: A0 => A) extends Stream[A] case class Join[A](s: Stream[Stream[A]]) extends Stream[A] case class Merge[A](l: Stream[A], r: Stream[A]) extends Stream[A] case class Fold[A0, A](a: A, s: Stream[A0], f: (A, A0) => A) extends Stream[A] PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 82. Initial Encoding PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ... sealed trait Stream[+A] case class Emit[A](a: A) extends Stream[A] case class Map[A0, A](s: Stream[A0], f: A0 => A) extends Stream[A] case class Join[A](s: Stream[Stream[A]]) extends Stream[A] case class Merge[A](l: Stream[A], r: Stream[A]) extends Stream[A] case class Fold[A0, A](a: A, s: Stream[A0], f: (A, A0) => A) extends Stream[A] case class Effect[A](io: IO[A]) extends Stream[A]
  • 83. Initial Encoding def drain[A](s: Stream[A]): IO[Unit] = s match { case Emit(a) => … ... } Interpreter PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 84. BUILD YOUR OWN STREAM Final Encoding PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 85. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 86. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 87. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) { def map[B](f: A => B): Stream[B] = Stream(r => run(r.compose(f))) } object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 88. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) { def map[B](f: A => B): Stream[B] = Stream(r => run(r.compose(f))) } object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) def join[A](s: Stream[Stream[A]]) = Stream(r => s.run(sa => sa.run(r))) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 89. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) { def map[B](f: A => B): Stream[B] = Stream(r => run(r.compose(f))) def merge[A1 >: A](that: Stream[A1]) = Stream[A1](r => run(r).par(that.run(r)).void) } object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) def join[A](s: Stream[Stream[A]]) = Stream(r => s.run(sa => sa.run(r))) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 90. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) { def map[B](f: A => B): Stream[B] = Stream(r => run(r.compose(f))) def merge[A1 >: A](that: Stream[A1]) = Stream[A1](r => run(r).par(that.run(r)).void) def fold[S](s: S)(f: (S, A) => S) = ... } object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) def join[A](s: Stream[Stream[A]]) = Stream(r => s.run(sa => sa.run(r))) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 91. Final Encoding case class Stream[+A]( run: (A => IO[Unit]) => IO[Unit]) { def map[B](f: A => B): Stream[B] = Stream(r => run(r.compose(f))) def merge[A1 >: A](that: Stream[A1]) = Stream[A1](r => run(r).par(that.run(r)).void) def fold[S](s: S)(f: (S, A) => S) = ... } object Stream { def emit[A](a: => A): Stream[A] = Stream[A](read => read(a)) def join[A](s: Stream[Stream[A]]) = Stream(r => s.run(sa => sa.run(r))) def liftIO[A](io: IO[A]): Stream[A] = Stream[A](io.flatMap(_)) } PERFORMANCEMOTIVATION API DESIGN SUMMARY Type Emission Mapping Joining Merging Folding Effect ...
  • 92. Final Flexible Lookahead optimizations Ad Hoc Non-uniform Initial Initial vs Final ✓ ✓ 𐄂 PERFORMANCEMOTIVATION API DESIGN SUMMARY 𐄂 Inflexible No lookahead optimizations Structured Uniform 𐄂 𐄂 ✓ ✓
  • 93. Constant space processing Discovered in 2003 Aggressively lazy ITERATEES PERFORMANCEMOTIVATION API DESIGN SUMMARY Final encoding-friendly Automatically leak-free “Left folds”
  • 94. Iteratees 101 sealed trait Iteratee[-A, +B] case class More[A, B](f: A => Iteratee[A, B]) extends Iteratee[A, B] case class Done[B](value: B) extends Iteratee[Any, B] sealed trait Enumerator[+A] { self => def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 95. Iteratees 101 sealed trait Enumerator[+A] { self => final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] = fold(...)(...) def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 96. Iteratees 101 sealed trait Enumerator[+A] { self => final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] = fold(...)(...) def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] final def map[B](f0: A => B): Enumerator[B] = new Enumerator[B] { def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] = self.fold(s)((s, a) => f(s, f0(a))) } } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 97. Iteratees 101 sealed trait Enumerator[+A] { self => final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] = fold(...)(...) def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] final def map[B](f0: A => B): Enumerator[B] = new Enumerator[B] { def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] = self.fold(s)((s, a) => f(s, f0(a))) } final def filter(f0: A => Boolean): Enumerator[A] = new Enumerator[A] { def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] = self.fold(s)((s, a) => if (f0(a)) f(s, a) else Left(s)) } } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 98. Iteratees 101 𐄂 Stack Safe𐄂 Combining 𐄂 Leftovers PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 99. Iteratees 101 𐄂 Stack Safe?𐄂 Combining? 𐄂 Leftovers? PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 103. Iteratees 101 sealed trait IterateeT[F[_], -A, +B] case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B] case class Done[F[_], B](value: B) extends IterateeT[F, Any, B] case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B] PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 104. Iteratees 101 sealed trait IterateeT[F[_], -A, +B] case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B] case class Done[F[_], B](value: B) extends IterateeT[F, Any, B] case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B] type Parallel[F[_], A1, A2, B] = IterateeT[IterateeT[F, A1, ?], A2, B] def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] = Effect[IterateeT[F, A1, ?], B](it) PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 105. Iteratees 101 sealed trait IterateeT[F[_], -A, +B] case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B] case class Done[F[_], B](value: B) extends IterateeT[F, Any, B] case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B] type Parallel[F[_], A1, A2, B] = IterateeT[IterateeT[F, A1, ?], A2, B] def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] = Effect[IterateeT[F, A1, ?], B](it) def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] = lift(More(a => Done(a))) def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] = More(a => Done(a)) PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 106. Iteratees 101 sealed trait IterateeT[F[_], -A, +B] case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B] case class Done[F[_], B](value: B) extends IterateeT[F, Any, B] case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B] type Parallel[F[_], A1, A2, B] = IterateeT[IterateeT[F, A1, ?], A2, B] def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] = Effect[IterateeT[F, A1, ?], B](it) def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] = lift(More(a => Done(a))) def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] = More(a => Done(a)) PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 107. ZIO Stream trait Stream[E, A] { def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]] } object Stream { sealed trait Step[S] object Step { case class Stop[S](value: S) extends Step[S] case class Cont[S](value: S) extends Step[S] } } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 108. ZIO Stream trait Stream[E, A] { def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]] } object Stream { sealed trait Step[S] object Step { case class Stop[S](value: S) extends Step[S] case class Cont[S](value: S) extends Step[S] } } Continuation Monad! PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 109. ZIO Stream trait Stream[E, A] { def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]] def toQueue(capacity: Int = 1): Managed[Nothing, Queue[Take[E, A]]] = for { queue <- Managed.liftIO(Queue.bounded[Take[E, A]](capacity)) offerVal = (a: A) => queue.offer(Take.Value(a)).void offerErr = (e: E) => queue.offer(Take.Fail(e)) enqueuer = (self.foreach[E, A](offerVal).catchAll(offerErr) *> queue.offer(Take.End).forever).fork _ <- Managed(enqueuer)(_.interrupt) } yield queue } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 110. ZIO Stream trait Stream[E, A] { def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]] def zip[B, C](that: Stream[E, B]): Stream[E, (A, B)] = new Stream[E, (A, B)] { type Q[A] = Queue[Take[E, A]] def fold[S](s: S)(f: (S, (A, B)) => IO[E, Step[S]]): IO[E, Step[S]] = { def loop(q1: Q[A], q2: Q[B], s: S): IO[E, Step[S]] = Take.option(q1.take).seq(Take.option(q2.take)).flatMap { case (Some(a), Some(b)) => f(s, (a, b)) case _ => IO.now(Step.Cont(s)) } self.toQueue().use(q1 => that.toQueue().use(q2 => loop(q1, q2, s))) } } } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 111. ZIO Stream 𐄂 Stack Safe✓ Combining 𐄂 Leftovers PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 112. ZIO Stream trait Stream[E, A] { def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]] def peel[R](sink: Sink[E, A, A, R]): Managed[E, (R, Stream[E, A])] = ... } Compose all leading sinks Peel off sink (e.g. headers) Work with stream remainder PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 113. ZIO Stream 𐄂 Stack Safe✓ Combining ✓ Leftovers PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 114. ZIO Stream trait Stream[E, A] { def fold[S]: IO[Nothing, (S, (S, A) => IO[E, Step[S]]) => IO[E, Step[S]]] // trampolined: F[A => F[B]] } PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 115. ZIO Stream ✓ Stack Safe✓ Combining ✓ Leftovers PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 116. 5 Where To From Here SUMMARY
  • 117. Fusion, Queue & complex combinators Performance Outstanding Work Comprehensive test suites and great docs Tests & Docs Combinators for every reason Combinators PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 118. Eric Torreborre — Beautiful Folds The FS2 team — Michael Pilquist, Fabio Labella, Pavel Chlupacek, et al Special thanks to all people who made ZIO Stream possible. Credits Oleg Kiselyov — Iteratees 2003 PERFORMANCEMOTIVATION API DESIGN SUMMARY
  • 119. Any questions? Thanks! John A. De Goes @jdegoes - degoes.net Itamar Ravid @itrvd - iravid.com PERFORMANCEMOTIVATION API DESIGN SUMMARY