8. The C10k Problem
It's time for web servers to handle ten
thousand clients simultaneously, don't you
think? After all, the web is a big place now.
(Dan Kegel, 2003)
"Real-time will become the default. It won't be
this newfangled tech that only Google and
Facebook and Twitter can afford to build.”
(ArsTechnica, 2012)
http://en.wikipedia.org/wiki/C10k_problem
http://www.kegel.com/c10k.htm
http://arstechnica.com/business/2012/05/say-hello-to-the-real-real-time-web
10. Waiting …
route process format
request results response
query db or Write to
web service log file
Scaling:
Problems: Context Switching, Stack Memory, Concurrency
12. Addressing the Problem
• event-driven / asynchronous architecture
– Many long lived connections are a feature of modern
applications
– OS threads are a precious resource
– Blocked OS threads are bad!
• => Service many connections with a small number of
threads
• Reactor Pattern (Douglas C. Schmidt, 1995)
– Event Loop
• different programming model
=> on which abstraction?
http://en.wikipedia.org/wiki/Reactor_pattern
13. Different Programming Model
Clients
delegate long Workers
running jobs
& IO
Event Loop
(single
request thread, non
blocking)
callback
response
19. Success of Node
• Written in JavaScript, based on V8
• Huge community & ecosystem
• DIRTy Applications: Data Intensive Real Time
• Mostly IO
• Shuffling data from one pipe to another
21. Demo
(vert.x not blocking on 100
concurrent longrunning requests)
siege -c100 -b -r100 "http://localhost:8081/?longrunning"
22. What is vert.x?
• General purpose application platform on the JVM
• Provides primarily asynchronous APIs
• Polyglot: Java, JavaScript/CoffeeScript, Groovy,
Ruby, Python
• Simple but not Simplistic
• Popular for “modern” web applications
• Similarities with Node.js & Akka
• New approach to problems traditionally served
by classic application servers
23. The Vert.x Project
• Open-source (on Github, ASL 2.0)
• Age: ~1.5 years
• Sponsor: VMWare
• Based on: Netty, Hazelcast, SockJS
sockjs
Rhino
24. Core APIs
• TCP/SSL clients and servers
• HTTP/HTTPS clients and servers
– including WebSockets
• Event bus
• File system
• 100% asynchronous
26. Scaling
• Scale by creating more Verticle instances
• Use message passing to communicate
• Fine-tune “pools” of verticles
• For TCP and HTTP servers Vert.x does
automatic load-balancing
27. Event Loops are no Silver Bullet
• Don’t block the event loop!
• Long running calculations
• Blocking APIs (i.e. JDBC)
• Most traditional Java libraries have blocking
APIs
• => Hybrid Threading Model
– Worker Verticles can block
– Communication by message passing (Event Bus)
– Leverage ecosystem of Java libraries
28. Concurrency Model
• A verticle instance is always single-threaded
• Very different approach from “Java-Style”
multi-threaded concurrency
– No more synchronized, volatile or locking
– Avoid many classic race conditions
29. Event Bus Demo
Vertx Instance
Event Loop Background Workers
Worker
Worker
Worker
Worker
Worker
Server
Server
Event Bus
JVM
30. Distributed Event Bus
• Connect multiple Vert.x JVM instances
• Applications are loosely coupled components
distributed across the network
• Event bus extends to client side JavaScript
• Use the same API on the client
• Distributed event space spans client and server
nodes
• Ideal for modern “real-time” web applications
31. Distributed Event Bus Demo
Vertx Instance Vertx Instance
Telnet
Event Loop Event Loop Workers
Browser
(JavaScript)
WebSocket
TCP Server
HTTP &
Worker
Worker
Server
Event Bus
JVM JVM
32. Web Application Demo
Vertx Instance
Module Module Module
Browser
vertx. vertx. vertx.
web-server aut-mgr mongo- Mongo DB
persistor
(WebSocket) Event Bus
JVM
33. The Future
• Scala & Closure support
• Management, including GUI console
• Developer experience – IDE integration,
testing
• Promises API?
• Direct-style API using continuations?
34. Summary
• Polyglot – use the language(s) you want
• Simple concurrency – wave goodbye to most race
conditions
• Leverage existing Java library ecosystem
• Powerful distributed event bus which spans client
and server
• Powerful module system
Vert.x is the platform for a new generation of polyglot web
and enterprise applications.
35. Vert.x vs. Node.JS
Polyglot vs. JavaScript
JVM vs. V8
huge ecosystem of big ecosystem of
vs.
Java libraries node modules
small community vs. big community
36. The big picture
• Alternative approaches (Java):
– Servlet 3.0
– NIO connections
37. Who is using it?
game-gorilla.com
gvmtool.net
Helsinki Trip Planner
38. Would I use it now?
• Maybe -> where the distributed architecture
really fits
• Might get interesting in the future…
• I don’t buy the polyglot story…
• I don’t fully buy the concurrency story …
First WWW page: August 1991XMLHTTP Request: 1996 (iframe, asynchronous content loading)AJAX as a term: 2005XMLHttpRequest spec by W3C in 2006
Google TalkVs. Real Time Computing- Receiver not known / ensuredpush vs. pullusers receive information when publishedno need for active requiring informationthick clients in the browserdata is exchanged, not finished web page
Herausforderung an unsere Internet-InfrastrukturVielmehrlanglebigeconnecctions connection / peristente connectionsMobile und grundsätzlicheVernetzungInternet of ThingsEs hat sichgezeigt, dass 10’000 connections so eine “magisce” Grenzeist, die eszuknackengibt.
This can involve a combination of operating system constraints and web server software limitations.While there are some specialized web servers that can handle more than ten thousand client connections, most web servers currently handle at most ten thousand clients simultaneously.http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-supportSo under 32-bit Windows, for example, where each process has a user address space of 2GB, giving each thread a 128K stack size, you'd expect an absolute maximum of 16384 threads (=2*1024*1024 / 128). In practice, I find I can start up about 13,000 under XP.
Long running requestsBlocking threads
In a lot of typical server applications threads are mostly waiting on IO.
Reactor Pattern: 1995threads are one of the bottlenecks for scaling
ÜberleitungzuNode.jsTwisted: Apple Calendar, NASA, Twilio …Node: LinkedIn, Walmart, Microsoft, Yammer, Browserling.MS: Azure Mobile Services uses Node in the Backend http://www.windowsazure.com/en-us/develop/mobile/
Node is simple. The learning-curve is not steep (if you accept JavaScript)Node embraces JavaScript (Devs, JSON, Browser-Platforms, Mobile)JavaScript is Event-Driven, non-blocking and single-threaded in the Client Node mixes runtime platform with application framework.-> This fits the cloud model
Event basierteprogrammier-model schlägvolldurch. KeineAbstraktion!Reading & Debugging (vs. sequential code)
JavaScript:Huge amount of Web-DevelopersJavaScript fits Event-Based Architecture (Single threaded Event-Loop in Browser), existing concepts and libraries can be reusedEvent-Based API is known by those developersSimple, easy to learnPerformant und skalierteinigermassenNode jumped into the space of real-time web applications. Mostly PHP Devs … on the edge of Web & Enterprise Übergang z vertx
15’
Web-Server non blockingsiege -c100 -b -r100 "http://localhost:8081/?longrunning" Running_ From IDE, from commandline
Not really a framework, rather comparable with Tomcat or Jetty than with JSF or EJB
Verticles have their own classloaderMessages are passed by value
DEMO: Blocking WebServer
Most new functionality should be provided as modules
DEMO: Instances
BlockingScalingMessage Passing
30’Event bus enables easy creation of real-time web applicationsMobile applications, gamesClint side library very small, can be used with any JS libraries
Distributed System very easy, same programming concepts
Building abstractionsAllowing long lived connections on a higher abstraction level than event-based APIs.
Loosely coupled Microservices
Vs. Real Time Computing- Receiver not known / ensured