Ruby Proxies for Scale Performance and Monitoring

Ruby Proxies for Scale, Performance
and Monitoring

Ilya Grigorik
@igrigorik

Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

postrank.com/topic/ruby

The slides… Twitter My blog


Code + Examples EventMachine

Misc Proxies


Proxy Love


“Rails, Django, Seaside, Grails…” cant scale.

Myth: Slow Frameworks


The Proxy Solution


The “More” Proxy Solution


Transparent Scalability


Load Balancer

Reverse Proxy App Server

MySQL Proxy

Proxy as Middleware
middleware ftw!


90% use case

%w*Transparent Intercepting Caching …+
There are many different types!


Transparent
HAProxy

App server A App server B

Transparent, Cut-Through Proxy


Transparent Proxy = Scalability Power Tool


Proxy Proxy

App server A App server B App server C

Problem: Staging Environment
Production


Simulating traffic?

Proxy

Duplication

App server C

“Representative Load / Staging”


github.com/igrigorik/autoperf

Replay log data, rinse, repeat


Profile of queries has changed Fail
Load on production has changed Fail
Parallel environment Fail
Slower release cycle Fail

Staging fail.


Duplex Ruby Proxy, FTW!

Production

Real (production) traffic
Benchmark

Benchmarking Proxy
flash of the obvious


github.com/igrigorik/em-proxy
Proxy DSL FTW!


EventMachine: Speed + Convenience
building high performance network apps in Ruby


p quot;Startingquot;
while true do
EM.run do
timers
p quot;Running in EM reactorquot;
network_io
end
other_io
end
puts quot;Almost donequot;

EventMachine Reactor
concurrency without threads


C++ core

Easy concurrency
without threading



http = EM::HttpRequest.new('http://site.com/').get

http.callback {
p http.response
}

# ... do other work, until callback fires.

Event = IO event + block or lambda call



EM.run do
EM.add_timer(1) { p quot;1 second laterquot; }
EM.add_periodic_timer(5) { p quot;every 5 secondsquot;}
EM.defer { long_running_task() }
end

class Server < EM::Connection
def receive_data(data)
send_data(quot;Pong; #{data}quot;)
end

def unbind
p [:connection_completed]
end
end

EM.run do
EM.start_server quot;0.0.0.0quot;, 3000, Server
end


EM.run do
EM.add_timer(1) { p quot;1 second laterquot; }
EM.add_periodic_timer(5) { p quot;every 5 secondsquot;}
EM.defer { long_running_task() }
end

class Server < EM::Connection
def receive_data(data)
send_data(quot;Pong; #{data}quot;) Connection Handler
end

def unbind
p [:connection_completed]
end
end

EM.run do Start Reactor
EM.start_server quot;0.0.0.0quot;, 3000, Server
end


http://bit.ly/aiderss-eventmachine
by Dan Sinclair (Twitter: @dj2sincl)


Proxies for Monitoring, Performance and Scale
welcome to the wonderful world of…



# ...
end
Process incoming data
# ...
end

conn.on_finish do
# ...
end
end

EM-Proxy



# ...
end
Process response data
# ...
end

conn.on_finish do
# ...
end
end

EM-Proxy



# ...
end

# ...
end
Post-processing step
conn.on_finish do
# ...
end
end

EM-Proxy


%w[ <Transparent> Intercepting Caching … +
solution for every problem


conn.server :srv, :host => quot;127.0.0.1quot;, :port => 81

# modify / process request stream
p [:on_data, data]
data
end No data modifications

# modify / process response stream
p [:on_response, server, resp]
resp
end
end

Port-Forwarding
transparent proxy



data
end

conn.on_response do |backend, resp| Alter response
resp.gsub(/hello/, 'good bye')
end
end

Port-Forwarding + Alter
transparent proxy


%w[ Transparent <Intercepting> Caching … +
solution for every problem


@start = Time.now
@data = Hash.new(quot;quot;)

Prod + Test
conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81
conn.server :test, :host => quot;127.0.0.1quot;, :port => 82

data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn')
end

@data[server] += resp
resp if server == :prod
end

conn.on_finish do
p [:on_finish, Time.now - @start]
p @data
end
end
Duplex HTTP: Benchmarking
Intercepting proxy


@start = Time.now


end

Respond from production
end

conn.on_finish do
p @data
end
end
Intercepting proxy


@start = Time.now


end

end
Run post-processing
conn.on_finish do
p @data
end
end
Intercepting proxy


[ilya@igvita] > ruby examples/appserver.rb 81
[ilya@igvita] > ruby examples/line_interceptor.rb
[ilya@igvita] > curl localhost

>> [:on_finish, 1.008561]

>> {:prod=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009
04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 0quot;,

:test=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009
04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 1quot;}

Intercepting proxy


[ilya@igvita] > ruby examples/line_interceptor.rb
[ilya@igvita] > curl localhost

STDOUT
[:on_finish, 1.008561]

{:prod=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri,
01 May 2009 04:20:00 GMTrnContent-Type:
text/plainrnrnhello world: 0quot;,

:test=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri,
01 May 2009 04:20:00 GMTrnContent-Type:
text/plainrnrnhello world: 1quot;}

Intercepting proxy


Same response, different
turnaround time

Different response body!


Woops!

Validating Proxy
easy, real-time diagnostics


Hacking SMTP
for fun and profit



# RCPT TO:<name@address.com>rn
RCPT_CMD = /RCPT TO:<(.*)?>rn/
Intercept Addressee

if rcpt = data.match(RCPT_CMD)
if rcpt[1] != quot;ilya@igvita.comquot;
conn.send_data quot;550 No such user herenquot;
data = nil
end
end

data
end

conn.on_response do |backend, resp|
resp
end
Defeating SMTP Wildcards
end
Intercepting proxy



# RCPT TO:<name@address.com>rn

Allow: ilya@igvita.com
if rcpt = data.match(RCPT_CMD)
if rcpt[1] != quot;ilya@igvita.comquot;
data = nil 550 Error otherwise
end
end

data
end

resp
end
Defeating SMTP Wildcards
end
Intercepting proxy


[ilya@igvita] > mailtrap run –p 2525 –f /tmp/mailtrap.log
[ilya@igvita] > ruby examples/smtp_whitelist.rb

> require 'net/smtp‘
> smtp = Net::SMTP.start(quot;localhostquot;, 2524)
> smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, quot;ilya@igvita.comquot;
=> #<Net::SMTP::Response:0xb7dcff5c @status=quot;250quot;, @string=quot;250 OKnquot;>
> smtp.finish
=> #<Net::SMTP::Response:0xb7dcc8d4 @status=quot;221quot;, @string=quot;221 Seeyanquot;>

> smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, “missing_user@igvita.comquot;
=> Net::SMTPFatalError: 550 No such user here

Intercepting proxy


[ilya@igvita] > mailtrap run –p 2525 –f /tmp/mailtrap.log
[ilya@igvita] > ruby examples/smtp_whitelist.rb

To: ilya@igvita.com
> require 'net/smtp‘
> smtp = Net::SMTP.start(quot;localhostquot;, 2524)

> smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, quot;ilya@igvita.comquot;
=> #<Net::SMTP::Response:0xb7dcff5c @status=quot;250quot;, @string=quot;250 OKnquot;>

> smtp.finish
=> #<Net::SMTP::Response:0xb7dcc8d4 @status=quot;221quot;, @string=quot;221 Seeyanquot;>

> smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, “missing_user@igvita.comquot;
=> Net::SMTPFatalError: 550 No such user here

Denied!

Intercepting proxy


“Hacking SMTP”.gsub(/Hacking/, ’Kung-fu’)
DIY spam filtering with Defensio



FROM_CMD = /MAIL FROM:<(.*)?>rn/
MSG_CMD = /354 Start your message/
MSGEND_CMD = /^.rn/ Intercept commands

#…
end

p [:resp, resp]

if resp.match(MSG_CMD)
@buffer = true
@msg = quot;quot;
end

resp
SMTP + SPAM Filtering
end
end
building a state-machine



FROM_CMD = /MAIL FROM:<(.*)?>rn/
MSG_CMD = /354 Start your message/
MSGEND_CMD = /^.rn/

#…
end

p [:resp, resp]

if resp.match(MSG_CMD) Flag & Buffer message
@buffer = true
@msg = quot;quot;
end

resp
end
end


Save data

@from = data.match(FROM_CMD)[1] if data.match(FROM_CMD)
@rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD)
@done = true if data.match(MSGEND_CMD)

if @buffer
@msg += data
Buffer
data = nil
end

if @done
#…
end

data
end


@from = data.match(FROM_CMD)[1] if data.match(FROM_CMD)
@rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD)
@done = true if data.match(MSGEND_CMD)

if @buffer
@msg += data
Flag end of message
data = nil
end

if @done
#… Process message
end

data
end


@buffer = false
uri = URI.parse('http://api.defensio.com/app/1.2/audit/key.yaml')
res = Net::HTTP.post_form(uri, {
quot;owner-urlquot; => quot;http://www.github.com/igrigorik/em-proxyquot;,
quot;user-ipquot; => quot;216.16.254.254quot;,
quot;article-datequot; => quot;2009/05/01quot;,
quot;comment-authorquot; => @from,
Defensio API
quot;comment-typequot; => quot;commentquot;,
quot;comment-contentquot; => @msg})

defensio = YAML.load(res.body)['defensio-result']
p [:defensio, quot;SPAM: #{defensio['spam']}quot;]

if defensio['spam']
else
data = @msg
end



@buffer = false
uri = URI.parse('http://api.defensio.com/app/1.2/audit/key.yaml')
res = Net::HTTP.post_form(uri, {
quot;owner-urlquot; => quot;http://www.github.com/igrigorik/em-proxyquot;,
quot;user-ipquot; => quot;216.16.254.254quot;,
quot;article-datequot; => quot;2009/05/01quot;,
quot;comment-authorquot; => @from,
quot;comment-typequot; => quot;commentquot;,
quot;comment-contentquot; => @msg})

defensio = YAML.load(res.body)['defensio-result']
p [:defensio, quot;SPAM: #{defensio['spam']}quot;]

if defensio['spam']
Pass / Deny Message
else
data = @msg
end



Protocol Trace

[:relay_from_backend, :srv, quot;354 Start your messagequot;]
[:resp, quot;354 Start your messagequot;]
[:srv, quot;nquot;]
[:relay_from_backend, :srv, quot;nquot;]
[:resp, quot;nquot;]
[:connection, quot;Hello Worldrnquot;]
[:connection, quot;.rnquot;]
[:defensio, quot;SPAM: false, Spaminess: 0.4quot;]
[:srv, quot;250 OKnquot;]
[:relay_from_backend, :srv, quot;250 OKnquot;]
[:resp, quot;250 OKnquot;]



@PostRank: Beanstalkd + Ruby Proxy
because RAM is still expensive


~ 93 Bytes of overhead per job
~300 Bytes of data / job

x 80,000,000 jobs in memory

~ 30 GB of RAM = 2 X-Large EC2 instances

Oi, expensive!

Beanstalkd Math


Observations:
1. Each job is rescheduled several times
2. > 95% are scheduled for > 3 hours into the future

Memory is wasted…

3. Beanstalkd does not have overflow page-to-disk

We’ll add it ourselves!
Extending Beanstalkd


1 “Medium” EC2 Instance
Intercepting Proxy

MySQL
EM-Proxy

Beanstalkd

@PostRank: “Chronos Scheduler”


PUT_CMD = /put (d+) (d+) (d+) (d+)rn/

if put = data.match(PUT_CMD) Intercept PUT command
if put[2].to_i > 600
p [:put, :archive]
# INSERT INTO ....
conn.send_data quot;INSERTED 9999rnquot;
data = nil
end
end

data
end

resp
end
end


PUT_CMD = /put (d+) (d+) (d+) (d+)rn/

if put = data.match(PUT_CMD)

if put[2].to_i > 600 If over 10 minutes…
p [:put, :archive]
# INSERT INTO ....

conn.send_data quot;INSERTED 9999rnquot;
data = nil
end
end Archive & Reply

data
end

resp
end
end


Overload the protocol

MySQL
EM-Proxy

PUT

RESERVE, PUT, …
put job, 900
Beanstalkd



400% cheaper + extensible! ~79,000,000 jobs, 4GB RAM

MySQL
EM-Proxy

PUT

Upcoming jobs: ~ 1M
RESERVE, PUT, …
Beanstalkd



Slides: http://bit.ly/railsconf-proxy
Code: http://github.com/igrigorik/em-proxy

Twitter: @igrigorik

Thanks. Questions?

The slides… Twitter My blog


Ruby Proxies for Scale Performance and Monitoring

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Ruby Proxies for Scale Performance and Monitoring

Similar to Ruby Proxies for Scale Performance and Monitoring (20)

More from Ilya Grigorik

More from Ilya Grigorik (9)

Recently uploaded

Recently uploaded (20)

Ruby Proxies for Scale Performance and Monitoring