How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
Live Streaming & Server Sent Events
1. Live Streaming
&
Server Sent Events
Tomáš Kramár
@tkramar
2. When?
● Server needs to stream data to client
– Server decides when and what to send
– Client waits and listens
– Client does not need to send messages
– Uni-directional communication
– Asynchronously
6. AJAX polling
Any news?
No
Any news?
No
Browser/Client Server
7. AJAX polling
Any news?
No
Any news?
No
Any news?
Browser/Client Yes! Server
8. AJAX polling
Any news?
No
Any news?
No
Any news?
Browser/Client Yes! Server
Any news?
No
9. AJAX polling
● Overhead
– Establishing new connections, TCP handshakes
– Sending HTTP headers
– Multiply by number of clients
● Not really realtime
– Poll each 2 seconds
11. WebSockets
● bi-directional, full-duplex communication
channels over a single TCP connection
● HTML5
● being standardized
12. Server-Sent Events
● HTML5
● Traditional HTTP
– No special protocol or server implementation
● Browser establishes single connection and
waits
● Server generates events
13. SSE
Request w parameters
id: 1
event: display
data: { foo: 'moo' }
Browser/Client Server
15. Case study
● Live search in trademark databases
● query
– search in register #1
● Search (~15s), parse search result list, fetch each result
(~3s each), go to next page in search result list (~10s),
fetch each result, ...
– search in register #2
● ...
– …
● Don't let the user wait, display results when
they are available
18. Client gotchas
● Special events:
– open
– error
● Don't forget to close the request
self.source.addEventListener('finished', function(e) {
self.status.searchFinished();
self.source.close();
});
19. Server
● Must support
– long-running request
– Live-streaming (i.e., no output buffering)
● Rainbows!, Puma or Thin
● Rails 4 (beta) supports live streaming
20. Rails 4 Live Streaming
class MarksController < ApplicationController
include ActionController::Live
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
Tort.search(params[:query]) do |on|
on.results do |hits|
sse.write(hits, event: 'result')
end
on.status_change do |status|
sse.write(status, event: 'status')
end
on.error do
sse.write({}, event: 'failure')
end
end
end
end
21. require 'json'
class SSE
def initialize io
event: displayn
@io = io data: { foo: 'moo' }nn
end
def write object, options = {}
options.each do |k,v|
@io.write "#{k}: #{v}n"
end
@io.write "data: #{JSON.dump(object)}nn"
end
def close
@io.close
end
end
22. Timeouts, lost connections, internet
explorers and other bad things
● EventSource request can be interrupted
● EventSource will reconnect automatically
● What happens with the data during the time
connection was not available?
23. Handling reconnections
● When EventSource reconnects, we need to
continue sending the data from the point the
connection was lost
– Do the work in the background and store events
somewhere
– In the controller, load events from the storage
● EventSource sends Last-Event-Id in HTTP
header
– But we don't need it if we remove the processed
events
25. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
26. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
generate job_id
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
27. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
start async job (GirlFriday)
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
28. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
render status: 202, text: marks_results_path(job: uuid)
end
send results URL
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
29. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis) Get queue for this job,
finished = false async job is pushing
begin
to this queue
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
30. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
Fetch next message
message = JSON.parse(json_message) from queue (blocks until
case message["type"]
when "results" then one is available)
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects
ensure
sse.close
end
end
end
31. class MarksController < ApplicationController
include ActionController::Live
def search!
uuid = UUID.new.generate(:compact)
TORT_QUEUE << { phrase: params[:q], job_id: uuid }
render status: 202, text: marks_results_path(job: uuid)
end
def results
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream)
queue = SafeQueue.new(Channel.for_job(params[:job]), Tmzone.redis)
finished = false
begin
begin
queue.next_message do |json_message|
message = JSON.parse(json_message)
case message["type"]
when "results" then
sse.write(message["data"], event: 'results')
when "failure" then
sse.write({}, event: 'failure')
when "fatal" then
sse.write({}, event: 'fatal')
finished = true
when "status" then
sse.write(message["data"], event: 'status')
when "finished" then
sse.write({}, event: 'finished')
finished = true
end
end
end while !finished
rescue IOError
# when clients disconnects IOError is raised when client
ensure
sse.close disconnected and we are
end
end writing to response.stream
end
32. GirlFriday worker
class SearchWorker
def self.perform(phrase, job_id)
channel = Channel.for_job(job_id)
queue = SafeQueue.new(channel, Tmzone.redis)
Tort.search(phrase) do |on|
on.results do |hits|
queue.push({ type: "results", data: hits }.to_json)
end
on.status_change do |status|
queue.push({ type: "status", data: status }.to_json)
end
on.error do
queue.push({ type: 'failure' }.to_json)
end
end
queue.push({ type: "finished" }.to_json)
end
end
33. SafeQueue
class SafeQueue
def initialize(channel, redis)
@channel = channel
@redis = redis
end
def next_message(&block)
begin
_, message = @redis.blpop(@channel)
block.call(message)
rescue => error
@redis.lpush(@channel, message)
raise error
end
end
def push(message)
@redis.rpush(@channel, message)
end
end
34. EventSource Compatibility
● Firefox 6+, Chrome 6+, Safari 5+, Opera 11+,
iOS Safari 4+, Blackberry, Opera Mobile,
Chrome for Android, Firefox for Android
35. Fallback
● Polyfills
– https://github.com/remy/polyfills/blob/master/Event
Source.js
● Hanging GET, waits until the request terminates,
essentially buffering the live output
– https://github.com/Yaffle/EventSource
● send a keep-alive message each 15 seconds
36. Summary
● Unidirectional server-to-client communication
● Single request
● Real-time
● Easy to implement
● Well supported except for IE