This 15 minute presentation discusses non-blocking I/O, event loops, and Node.js. It builds on previous work by Ryan Dahl, explaining how threads can be expensive due to context switching and memory usage, and how Node.js uses an event-driven, non-blocking model to avoid these costs. Code examples demonstrate getting and printing a policy object, handling HTTP requests asynchronously without blocking additional connections, and using callbacks to chain asynchronous actions together.

1. This is a 15 minute presentation I did
at work recently about threads,
polling and node.js.
It rests heavily on the shoulders of
giants, especially Ryan Dahls original
node.js talk at jsconf.eu.

2. non-blocking I/O,
event loops,
javascript and
node.js.

3. Threads suck

4. Why do we
use threads?

5. Policy p = cmServer.getPolicy(...);
System.out.println(p.getContentId().toString());

6. Policy p = cmServer.getPolicy(...);
System.out.println(p.getContentId().toString());

7. Policy p = cmServer.getPolicy(...);
Blocking implies multiple execution stacks. Where else to go?
System.out.println(p.getContentId().toString());

8. How to deal with blocking?
Processes
No shared memory
Heavy
OS Threads
Can share memory
Relatively cheap
Green threads / Co-routines

9. Are threads cheaper
than processes?

10. HTTP request latency; glibc 2.3.2 on Linux 2.6
HTTP-anropslatens; glibc 2.3.2 på Linux 2.6
1 thread per call
1 process per call
http://bulk.fefe.de/scalable-networking.pdf
Open connections

11. HTTP request latency; glibc 2.3.2 on Linux 2.6
1 thread per call
1 process per call
Which is the reason why we use thread pools!
Open connections

12. Are threads
cheap at all?

13. nginx vs apache
http://blog.webfaction.com/a-little-holiday-present

14. nginx vs apache
http://blog.webfaction.com/a-little-holiday-present
∆ = Apaches context switching

15. lmbench
./lat_ctx -N 10 -s 4096 0 1 .. 20
80
74
68
62
56
50
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

16. lmbench
./lat_ctx -N 10 -s 4096 0 1 .. 20
80
74
68
62
56
50
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
On a MBP, OS X 10.6 Context switch around 65 µs

17. 65 µs x 2 x 4000 = 520 ms

18. How about memory?

19. > ulimit -a | grep stack
stack size (kbytes, -s) 8192

20. ...but at least 512+1 kb
For OS X, source: ADC

21. nginx vs apache
http://blog.webfaction.com/a-little-holiday-present

22. nginx vs apache
http://blog.webfaction.com/a-little-holiday-present
Difference: Threads and processes cost.

23. What about ngnix?

24. The cost of I/O
L1-cache 3 cycles
L2-cache 14 cycles
RAM 250 cycles
Disk 41 000 000 cycles
Network 240 000 000 cycles

25. The weight of I/O

26. The weight of I/O
L1-cache A big squirrel

27. The weight of I/O
L1-cache A big squirrel
L2-cache A medium-sized cat

28. The weight of I/O
L1-cache A big squirrel
L2-cache A medium-sized cat
RAM Mattias, basically

29. The weight of I/O
L1-cache A big squirrel
L2-cache A medium-sized cat
RAM Mattias, basically
Disk Like a 100 blue whales

30. The weight of I/O
L1-cache A big squirrel
L2-cache A medium-sized cat
RAM Mattias, basically
Disk Like a 100 blue whales
Network Belarus yearly
wheat import

31. Servers pretty I/O-intensive
=
Squirrelʼs burried in
whales fat and wheat

32. Policy p = cmServer.getPolicy(...);
System.out.println(p.getContentId().toString());

33. cmServer.getPolicy(function (Policy p) {
System.out.println(
p.getContentId().toString()
);
});

34. cmServer.getPolicy(function (Policy p) {
System.out.println(
Sleeps until p’s available
p.getContentId().toString()
);
});

35. cmServer.getPolicy(function (Policy p) {
System.out.println(
Sover tills dess p finns
p.getContentId().toString()
);
});
// And down here we can do other stuff

36. Non-blocking I/O...
All code is run in a single thread
Functions listen on events,
act and sleep

37. ...is pretty demanding
All I/O has to be non-blocking
Callbacks are pretty ugly without HOF

38. V8 + node.js

39. “a purely evented,
non-blocking
infrastructure to script
highly concurrent
programs.”

40. V8 Chromes JS engine
JS Built to be event driven
node.js Queues built on /dev/epoll
amongst other things

41. var http = require('http');
http.createServer(function (req, res) {
setTimeout(function () {
res.sendHeader(200, {'Content-Type': 'text/plain'});
res.sendBody('Hello World');
res.finish();
}, 2000);
}).listen(8080);

42. % node hello.js
% ab -n 1000 -c 200 http://127.0.0.1:8080/
...
Time per request: 2010.070 [ms] (mean)
min mean[+/-sd] median max
Connect: 0 2 1.9 1 8
Processing: 2001 2006 2.8 2005 2013
Waiting: 2000 2003 2.0 2003 2009
Total: 2002 2008 3.5 2006 2015
Percentage of the requests served within a certain time (ms)
50% 2006
66% 2008
75% 2012
80% 2013
90% 2013
95% 2014
98% 2014
99% 2014
100% 2015 (longest request)

43. var stat = require('posix').stat,
puts = require('sys').puts;
var promise = stat('/etc/passwd');
promise.addCallback(function (s) {
puts('modified: ' + s.mtime);
});

44. var web = require('./web');
web.server(function (route) {
route.get("^/([a-z]+)$", function(parms, req, res) {
res.sendHeader(200, {'Content-Type':'text/plain'});
res.sendBody("Hello " + parms[0]);
res.finish();
});
}).listen(8080);

45. % node web.test.js
% ab -n 10000 -c 250 http://127.0.0.1:8080/marcus
...
Requests per second: 4851.93 [#/sec] (mean)
Time per request: 51.526 [ms] (mean)
Time per request: 0.206 [ms] (mean, across all c requests)
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 14 111.8 1 999
Processing: 4 35 15.8 31 103
Waiting: 4 35 15.8 30 103
Total: 14 49 112.5 32 1035
Percentage of the requests served within a certain time (ms)
50% 32
66% 34
75% 35
80% 36
90% 57
95% 92
98% 102
99% 986
100% 1035 (longest request)

46. var web = require('./web'),
posix = require('posix');
web.server(function (route) {
route.get("^/([a-z]+)$", function(parms, req, res) {
posix.cat(parms[0]).addCallback(function(text) {
res.sendHeader(200, {'Content-Type':'text/plain'});
res.sendBody(text);
res.finish();
});
});
}).listen(8080);

47. throughput
80
74
68
62
56
50
1 5 13 25 50 100 250

48. HEALTH

49. Distributed ab
One configuration, multiple nodes
REST interface
Streams JSON over HTTP
Decent performance

50. demo!