You've got the power?
•
2 likes•647 views
This document summarizes a presentation about using Node.js for CPU-intensive applications like simulations and artificial intelligence. It describes a simulation called GOD that models plants and animals using an entity component system architecture. Tests show the simulation scaling to handle over 250 entities and 15 agents on a quad-core CPU, with the CPU nearly fully utilized. While Node.js and its asynchronous model proved a good fit, constraints like message passing bandwidth and frame rates limit scaling for agent computation. The conclusion is that Node.js can be effectively used for these kinds of applications when the computation and event loads are balanced.
Recommended
More Related Content
Similar to You've got the power?
Similar to You've got the power? (20)
Recently uploaded
Recently uploaded (20)
You've got the power?
- 1. YOU’VE GOT THE POWER? Node.js Italian Conference 2016/10/22 Desenzano (BS)
- 2. WHO WE ARE Diego Ferri @thundo Andrea Ghidini @ghidosoft 2 of 3 co-founders of Looptribe Node.js, PHP, .NET www.looptribe.com Analysis | Consulting | Development
- 3. BACK TO SCHOOL “Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non- blocking I/O model that makes it lightweight and efficient.”
- 4. MANTRA Single- threaded Asynchronous I/O Event-driven Fast V8-powered Lots of concurrent I/O bound requests? Scalable network applications? Yeah, you should use Node.js
- 5. ...BUT WHAT ABOUT CPU INTENSIVE APPLICATIONS? Bitcoin miner? Self-driving car? MMOG simulation backend?
- 7. WHAT WE WILL SEE FEASIBILITY PROS BENCHMARKS CONSTRAINTS
- 8. HERE COMES GOD Game simulation Multiple clever agents The fraction of all people with our kind of experiences that are living in a simulation is very close to one [Nick Bostrom]
- 9. GOD: WHAT IS IT? Two kinds of entities PLANTS and ANIMALS Plants Static Food resources Animals Explore Eat nearby plants Attack other animals
- 10. THE MONOLITH
- 12. ECS - COMPONENTS Combat Health Hunger Mobility Movement Physics Plant Sight Strength
- 13. ECS - SYSTEMS Combat Feeding Flora Health Movement Spawner Entity- manager
- 16. AGENTS
- 17. AGENTS
- 18. AGENT: RULE-BASED Set of hard-coded rules Inference system Output depends on the selected rule Features: Simple Acceptable agent behavior Fast prototyping
- 19. AGENT: NEURAL NETWORKS Interconnected neurons and layers Forward propagation Training Features: Learn dynamically Tackle complex tasks
- 20. NEURAL NETWORKS: TRAINING Online: Exploration Exploitation Reinforcement learning: No known environment mode Rewards Q-learning
- 22. GOD IN SHORT SIMULATION MULTIPLE AGENTS DISTRIBUTED AI RICH 5-YEARS OLD CHILD’S GRAPHICS
- 23. DEMO TIME
- 24. SIMULATION GRAPHS Intel i7 6700HQ. Quad-Core @2.6 GHz 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 100 200 500 1000 2000 5000 10000 20000 Entities Physics Serialization Logic 0 100 200 300 400 500 600 700 800 0 5000 10000 15000 20000 25000 Time(ms) Entities Physics Serialization Logic
- 25. SIMULATION GRAPHS 0 100 200 300 400 500 600 700 800 0 5000 10000 15000 20000 25000 Time(ms) Entities Physics Serialization Logic
- 26. SIMULATION GRAPHS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 100 200 500 1000 2000 5000 10000 20000 Entities Physics Serialization Logic
- 27. AGENT GRAPHS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 0 1 2 3 4 5 6 CPUusage Running agents 0 20 40 60 80 100 120 140 160 0 5000 10000 15000 20000 25000 Time(ms) Entities Sensors Training Serialization Intel i7 6700HQ. Quad-Core @2.6 GHz
- 28. AGENT GRAPHS 0 20 40 60 80 100 120 140 160 0 5000 10000 15000 20000 25000 Time(ms) Entities Sensors Training Serialization
- 29. AGENT GRAPHS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 0 1 2 3 4 5 6 CPUusage Running agents
- 30. RESULTS We are running a simulation with 250 entities plus 15 agents The CPU is nearly at full utilization CONSTRAINTS Message bus bandwidth The framerate too high for agent computation
- 31. WHERE TO SCALE?
- 32. CONCLUSIONS We were supported by the strengths of the language and its wide ecosystem (e.g. neural network libraries) We exploited an event-driven design while taxing the CPU with work In fact, the sim spends more time doing CPU work than I/O The trick is to find the sweet spot between the computation time and the event rate (avoid starvation!) Yes, Node.js was a good choice for this kind of project In the end, nothing revolutionary but yes, you can do it with Node.js.
- 33. CREATIVE IDEAS FOR THE AUDIENCE Multimedia processing, e.g. worker pools Gaming, e.g. huge world sharding Machine learning clusters Business intelligence analysis
Editor's Notes
- [D] Hi all, we’re Diego and Andrea, co-founders of Looptribe. Actually we are only ⅔ of the founders, the missing third is stalking around here. Our company focuses on analysis, consulting and development. We work multistack on node/php/.net, mostly web and mobile.
- [A] Well, we are at a NodejsConf so we really hope that we are wasting [weisting] our time with this slide. This is Node definition taken directly from the homepage.
- [A] What really interests us are the main features: it is single-threaded it has asynchronous I/O it is event driven it’s fast and it’s V8 powered So what are the recommended use cases? Do you have lots of concurrent I/O bound request? Do you need a scalable network applications? Yeah, node.js is the right tool.
- [D] BUT, what about “non-standard” CPU intensive applications? Like a bitcoin miner? a self driving car? or a Massively Multiplayer Online Game simulation backend? can it still be a good choice? Can it be a good framework?
- [D] OR is this what you expect? So...has Node really enough “POWER” for this kind of scenario [senàrio]?
- [A] Starting from this question [quèsçion] we will see: if it is feasible to use node if yes, if there are any pros we will then try to get a few quantitative measures and potential constraints
- [A] To work on these points we started a pet project, we humbly nicknamed God. The true motif behind this talk is in effect that it’s been a while since we wanted to play with a simulation while at the same time exploring modern machine learning techniques in the form of multiple intelligent agents.
- [A] God is a simulated world with two kinds of entities: plants and animals. Plants are static entities, they cannot move, they slowly grow and reproduce. Animals on the other hand can move freely around the world, exploring it. They must eat plants in order to be healthy and they can attack each other to compete over resource scarcity.
- X[D] Where did we start from? From the beloved monolithic…
- [D] Here is an overall view of monolith architecture. The simulation evolves thanks to a design pattern commonly seen in game programming, the popular game loop. In each turn of the loop events are handled, action taken and the game state updated, while ensuring framerate and timestepping. Entities are the lead simulation actors. To structure entity management [màn-agement] we adhered to another common game architectural pattern, ECS that stands for Entity-Component-System. [compònent] What is it? Entities can be seen as just collection of components and components are just data (e.g. the sight component holds the values for that entity sight angle and radius). Systems on the other side work on those components. A system is where the expected behavior or rules of the simulation happen to be. For example is where attacking or moving of an entity really take place. A single system may work on one or more components.
- [D] Here you find the list of components we actually use in God. All components can be added or removed at runtime. A few components are present in all entities, others are for specific entities. e.g. all entities have a physics component, but only a plant entity has a plant component (with data about growth, radius). A few are added only at times. An animal entity may have a movement component only when they intend to move, not otherwise. On the slide we see a snippet of the Plant component: compact and expressive.
- [D] Systems determine [detérmine] agent behaviors and actions, like combat, feeding and so on. As we told before they interact directly with the components data. Most system purposes are quite obvious. A couple of them are exceptions [exèpscion]. The spawner system keeps a constant number of entities per type (we actually scrapped a complex reproduction system during development [divèlopment] to keep things simple) We even have our very own alien [ellien] anti-pattern, the entity manager. It’s very useful because it allows to manage and search entity lists but it is alien in the sense that most systems hold a reference to it. Here’s the snippet for the plant grow system. Still compact and short.
- [A] We refactored the monolithic code and extracted the rendering code that was bloating the logic part. To achieve that, we introduced a distributed architecture with a central message bus (actually an instance of RabbitMQ) that allows communication among parts. A websocket endpoint listens to the simulation state updates and pushes them to a beautiful HTML5 rendering app.
- [A] In a later step, with a basic system in place, we moved out agent logic from the core systems to independent [indipe’ndent] processes [pro’cessis] to scale performance. Each agent is an individual intelligent entity that runs its own logic and communicates with the simulation via messages. A spawner process manages their lifecycle, as always via the message bus.
- [A] Let’s take a tour of the agent. The Agent [égent] is probably the most interesting aspect of God. It is the mastermind behind an intelligent entity behavior.
- [A] Every simulation game tick agents receive the whole [hole] world state via the message bus. Then they decide the action to perform in the following tick, after having swept the world with sensors. Sensors measure presence and distance of plants, world boundaries and other agents and ensure that an agent is “fair” (e.g. it should not head for a plant if it cannot see it).
- [A] We implemented two totally different AIs. [eiais] The first one is rule-based. There is set of rules that basically cover the various possible situations encountered by an agent. An inference system then selects the correct rule, based on the world state and outputs the rule’s action. For instance it is as simple as “If I’m starving then i must eat”. There are rules to feed, to “defend” a territory [te’rritory] and to explore in case there are no resources in the vicinity. On the right you can see a simple pseudocode snippet with a basic rule-based behavior. This AI implementation is easy to code and allows a fast prototyping [prototipa’in] but nonetheless gives an acceptable [axe’ctabol] agent behavior.
- [D] The second approach is much more complex and it involves the use of neural networks to create cpu-heavy agents. What are neural networks in a few words? Artificial neural networks are statistical learning models, inspired by biological models, and are tools in machine learning. They are composed by sets of interconnected small units, called neurons, organized in layers. Values are presented to the network via the input layer, which communicates to one or more 'hidden layers' where the actual processing is done via a system of weighted connections. The hidden layers then link to an output layer where the final answer of the network is given. This flow from the input layer to the output layer is called forward propagation. But there is another mode of operation, where the network (actually the output layer) is presented with couples of desired input/output to calibrate the weights of the connections. This is called training (or backpropagation). We adopted deep learning algorithms to operate on our deep networks. They are called “deep” because they have many hidden layers, needed to tackle complex tasks, lime image recognition or action planning.
- [D] The training of our networks has 2 distinctive traits: * it is done online, that is the network explores the graph of possible actions while the world is evolving, striking a balance between exploration (of new knowledge) and exploitation (of prior knowledge). * it uses reinforcement learning: our network has no handcrafted knowledge model of the environment. No correct input/output pairs are presented to the network during training, on the contrary a reward is awarded for every action taken. The network then tries to maximize the reward received on a chain of consecutive actions. This algorithm is called Q-learning. How does it work in layman terms? “I’m starving? Let’s find a plant to eat it”. Good, reward this behavior! “I’m starving? Why don’t just stand still?” Bad, discourage it with a negative reward! Training is very time-consuming: the choice of the topology of the network and the tuning of the deep learning hyper-parameters (learning rate, batch size, etc) took several days. The training of the final network alone was particularly cpu intensive and took over half a day.
- [D] As libraries go, we were quite lucky. We started with Synaptics.js that runs on both browser and node. But it doesn’t support reinforcement learning natively. In the end we chose ConvnetJs, a library maintained by a Stanford PhD. Sadly there’s no node package. As you can see in the snippet below the base implementation using convnetjs is really straightforward; ...but don’t be fooled, tuning the rewards is not!
- [A] Starting with our initial objectives we winded up to a distributed simulation of a multi-agent world, where the agents are entitled to make decisions. Last, but not least our project sports a pretty 5-year-old child graphics.
- [A] Ok, now it’s demo time. If you are interested in the code you can catch up with us later after the talk, we will gladly show it to you. [start the simulation…] Behold God… [D] this is the part where you should say ‘Owwwww’! The rectangle you see is in fact the simulated world. You can see two different kinds of circles [soercol] there: the green ones, that we can think of as food sources or plants. They’re static and not intelligent at all. the red ones, on the other end, can be imagined as animals that strive to stay alive while interacting with the world. They are the agents, the intelligent stuff of the simulation. True to our requirements the graphics is of a really good level ;) Behaviors!
- [A] We’ll put the simulation in the background for a while and give you a few quantitative measures of the core project. As we saw before, for every frame the simulation updates the world physics, runs the entities actions and pushes the world state serialized to the message bus. We charted the time taken by these three stages in a single frame with respect to the number of entities. As you can see on the left, physics and logic at one point become [bicam] superlinear, bogging down the simulation. Therefore [derfòr] a sustainable number of entities, on our reference system, is in the order of a few thousands. On the other hand, and you can see this more keenly on the right side chart, the serialization is quite time-consuming at the beginning, but become irrelevant over 10 thousands entities.
- [A] We’ll put the simulation in the background for a while and give you a few quantitative measures of the core project. As we saw before, for every frame the simulation updates the world physics, runs the entities actions and pushes the world state serialized to the message bus. We charted the time taken by these three stages in a single frame with respect to the number of entities. As you can see on the left, physics and logic at one point become [bicam] superlinear, bogging down the simulation. Therefore [derfòr] a sustainable number of entities, on our reference system, is in the order of a few thousands. On the other hand, and you can see this more keenly on the right side chart, the serialization is quite time-consuming at the beginning, but become irrelevant over 10 thousands entities.
- [A] We’ll put the simulation in the background for a while and give you a few quantitative measures of the core project. As we saw before, for every frame the simulation updates the world physics, runs the entities actions and pushes the world state serialized to the message bus. We charted the time taken by these three stages in a single frame with respect to the number of entities. As you can see on the left, physics and logic at one point become [bicam] superlinear, bogging down the simulation. Therefore [derfòr] a sustainable number of entities, on our reference system, is in the order of a few thousands. On the other hand, and you can see this more keenly on the right side chart, the serialization is quite time-consuming at the beginning, but become irrelevant over 10 thousands entities.
- [A] Similarly [Sìmilarly] you can see some measures for the neural network agent. We got a linear behavior for sensor scanning and deserialization with respect to the entities count. The training/decision is quite expensive and is proportional to the size and topology of the tested network, irrespective to the number of entities. The deserialization takes the major share of the cpu time, as soon as the number of entities grows. On the right we tried to fit as many agent as possible on the reference system and this is the result. We succeeded to squeeze about 4-5 agents.
- [A] Similarly [Sìmilarly] you can see some measures for the neural network agent. We got a linear behavior for sensor scanning and deserialization with respect to the entities count. The training/decision is quite expensive and is proportional to the size and topology of the tested network, irrespective to the number of entities. The deserialization takes the major share of the cpu time, as soon as the number of entities grows. On the right we tried to fit as many agent as possible on the reference system and this is the result. We succeeded to squeeze about 4-5 agents.
- [A] Similarly [Sìmilarly] you can see some measures for the neural network agent. We got a linear behavior for sensor scanning and deserialization with respect to the entities count. The training/decision is quite expensive and is proportional to the size and topology of the tested network, irrespective to the number of entities. The deserialization takes the major share of the cpu time, as soon as the number of entities grows. On the right we tried to fit as many agent as possible on the reference system and this is the result. We succeeded to squeeze about 4-5 agents.
- [D] So, here’s the recap. The simulation is running in the background with around 250 entities plus 15 agents. And the cpu is nearly at full utilization. What about constraints? The first one we battled with is the message bus bandwidth. Every frame we post a JSON [gei-son] with the full world state. This JSON is not optimized and is uncompressed and it can quickly weight a few MBs. In any case, for large worlds it can quickly become a bottleneck. Second, if the agent logic is very complex, the simulation framerate must be balanced accordingly [accòrdingly], otherwise an agent will lag [laag] behind and become unresponsive.
- [D] Well, we are now running a performance capped simulation. How to improve? First: we could, of course, simply increase the number of spawner processes to distribute even more horizontally the number of agents. But probably, second, the most impacting refactoring would be to shard the core logic, probably together with spatial partitioning of the simulated world. However, this would need a major rewriting of the game loop, physics and entities sub-systems.
- [A] Conclusions, Yeah, node in the end was really a good choice for this kind of project. We were supported by the strengths of the language and its wide ecosystem (e.g. neural network libraries) We exploited an event-driven design while taxing the CPU with work In fact, the sim spends more time doing CPU work than I/O The trick is to find the sweet spot between the computation time and the event rate to avoid starvation In the end, nothing revolutionary but yes, you can do it with Node.js.
- [D] Ok, it was all really nice by what of it? We will leave you with a few ideas... We already used node in production in the past for multimedia processing in worker pools to convert audio/video streams with ffmpeg, but it can be used in many other contexts. [choice…] remember: prototype
- [D] Thank you, we are open to questions and we hope you enjoyed our talk.