Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Python Performance: Single-threaded, multi-threaded, and Gevent
Similar to Python Performance: Single-threaded, multi-threaded, and Gevent (9)
More from emptysquare
Recently uploaded
Recently uploaded (20)
Python Performance: Single-threaded, multi-threaded, and Gevent
- 1. What I Know About Python Performance A. Jesse Jiryu Davis http://emptysqua.re MongoDB
- 3. Single-Threaded Why is Python slow? • Everything’s an object • Lookup
- 4. Everything’s an object def f():! i = 2000! i += 1!
- 5. Everything’s an object import dis! dis.dis(f)!
- 6. Everything’s an object def f():! i = 2000! i += 1 0 3 6 9 12 13 LOAD_CONST STORE_FAST LOAD_FAST LOAD_CONST INPLACE_ADD! STORE_FAST 1 0 0 2 (2000)! (i)! (i)! (1)! 0 (i)!
- 7. Everything’s an object def f():! i = 2000! i += 1 malloc! 0 3 6 9 12 13 LOAD_CONST STORE_FAST LOAD_FAST LOAD_CONST INPLACE_ADD! STORE_FAST 1 0 0 2 (2000)! (i)! malloc! (i)! (1)! 0 (i)!
- 8. Everything’s an object 12 seconds for 100M numbers: total = 0! for i in longs:! total += i! print float(total) / len(longs)
- 9. Everything’s an object 1 second for 100M numbers: print float(sum(longs)) / len(longs)
- 10. Everything’s an object 0.2 seconds for 100M numbers: print numpy.mean(array)
- 11. Lookup n = 1! ! def f():! print n!
- 12. n = 1! ! def f():! print n! 0 LOAD_GLOBAL 3 PRINT_ITEM! 4 PRINT_NEWLINE hashtable lookup 0 (n)!
- 13. n = 1! ! def f(n=n):! print n! 0 LOAD_FAST 3 PRINT_ITEM! 4 PRINT_NEWLINE array offset 0 (n)!
- 14. Profiling
- 15. Profiling import yappi! ! yappi.start(builtins=True)! my_function()! yappi.stop()! stats = yappi.get_func_stats()! stats.save('yappi.callgrind', type='callgrind')!
- 16. KCacheGrind
- 17. KCacheGrind
- 18. Multi-threaded
- 19. Multi-threaded When is the GIL released? • Every 100 bytecodes • Acquiring other locks • Network or file operations
- 20. Multi-threaded Because: Py_BEGIN_ALLOW_THREADS! outlen = recv(s->sock_fd, cbuf, len);! Py_END_ALLOW_THREADS! socketmodule.c
- 21. Multi-threaded Because: Py_BEGIN_ALLOW_THREADS! outlen = recv(s->sock_fd, cbuf, len);! Py_END_ALLOW_THREADS! release the GIL! socketmodule.c
- 22. Multi-threaded This makes sense: threads = [! threading.Thread(target=download_a_url())! for _ in range(10)]! ! for t in threads:! t.start()! ! for t in threads:! t.join()!
- 23. Multi-threaded This makes no sense: threads = [! threading.Thread(target=do_calculations())! for _ in range(10)]! ! for t in threads:! t.start()! ! for t in threads:! t.join()!
- 24. Gevent from gevent import monkey! monkey.patch_all()! ! threads = [! threading.Thread(target=download_a_url())]! ! for t in threads:! t.start()! ! for t in threads:! t.join()! actually a greenlet
- 25. GreenletProfiler