qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Srisatish
•
3 likes•1,621 views
Top 10 Performance Gotchas in scaling in-memory Algorithms Abstract: Math Algorithms have primarily been the domain of desktop data science. With the success of scalable algorithms at Google, Amazon, and Netflix, there is an ever growing demand for sophisticated algorithms over big data. In this talk, we get a ringside view in the making of the world's most scalable and fastest machine learning framework, H2O, and the performance lessons learnt scaling it over EC2 for Netflix and over commodity hardware for other power users. Top 10 Performance Gotchas is about the white hot stories of i/o wars, S3 resets, and muxers, as well as the power of primitive byte arrays, non-blocking structures, and fork/join queues. Of good data distribution & fine-grain decomposition of Algorithms to fine-grain blocks of parallel computation. It's a 10-point story of the rage of a network of machines against the tyranny of Amdahl while keeping the statistical properties of the data and accuracy of the algorithm. Track: Scalability, Availability, and Performance: Putting It All Together Time: Wednesday, 11:45am - 12:35pm
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Srisatish
Similar to qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Srisatish (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Srisatish
- 1. Better Predictions! H2O – The Open Source Math Engine !
- 2. H2O – Open Source in-memory Machine Learning for Big Data 4/23/13
- 3. Universe is sparse. Life is messy. Data is sparse & messy.! - Lao Tzu
- 4. Hadoop = opportunity Not enough Data Scientists Analysts won’t code java
- 5. Group By Grep Messy NAs Classifica-on Regression Clustering Ensembles 100’s nanos models H 2O Big Data the Adhoc Explora-on Math Modeling Real-‐-me Scoring Prediction Engine
- 6. No New API! Big Data Explora-on Modeling Scoring Real-‐-me H 2O the Prediction Engine Approximate! results each step!
- 7. Intellectual Legacy Math needs to be free Open Source Support and Innovation hFps://github.com/0xdata/h2o H 2O the Prediction Engine
- 8. All Top 10ʼs are binary! - Anonymous
- 9. 10 Move Code not Data Data chunks > code chunks TCP for Data. UDP for Control. >> Generated Java Assist
- 10. A Chunk, Unit of Parallel Access A Frame: Vec[] age sex zip ID car JVM 1 Heap JVM 2 Heap JVM 3 Heap JVM 4 Heap Vecs aligned in heaps l Optimized for concurrent access l Random access any row, any JVM l
- 11. 9 Chunk-ing Express! season for Variable-sized chunks and a season Uniform chunks. Tightly-packed! (chunk is also unit of batch!)
- 12. 8 Reduce early. Reduce Often! No Expensive intermediate states. Fine-grain parallelism wins! >> Fork / Join
- 13. 8 Reduce early. Reduce Often! Vec Vec Vec Vec Vec All CPUs grab Chunks in parallel Map/Reduce & F/J handles all sync JVM 1 Hea p JVM 2 Hea p JVM 3 Hea p JVM 4 Hea p
- 14. 7 Slow is not different from Dead Debugging slow >> Heartbeats, Messages Two General’s Paradox
- 15. 6 Memory Manager in-memory system as good as your memory manager! lazy eviction. compress. align. Corollary: Track down Leaks!
- 16. 5 Memory Overheads Use primitives // A Distributed Vector // much more than 2billion elements class Vec { long length(); // more than an int's worth // fast random access double at(long idx); // Get the idx'th elem boolean isNA(long idx); } void set(long idx, double d); // writable void append(double d); // variable sized
- 17. 4 Cache-‐Oblivious Tree size Bin size Recursively divide Till Data à Cache
- 18. 3 EC2 – Nothing is bounded User-mode reliability S3 Readers will TCP Reset Mux your connections Not all toolkits are equal. >> JetS3
- 19. 2 No Locks, No Cry Non-Blocking Data Structures. // VOLATILE READ before key compare. // CAS private final boolean CAS_kvs( final Object[] oldkvs, final Object[] newkvs ) { return _unsafe.compareAndSwapObject(this, _kvs_offset, oldkvs, newkvs ); }
- 21. 1 endian wars ended! Keep-It-Simple-Serialization. byte[ ]. roll-your-own. fast. public AutoBuffer putA1 ( byte[] ary, int sofar, int length ) { while( sofar < length ) { int len = Math.min(length - sofar, _bb.remaining()); _bb.put(ary, sofar, len); sofar += len; if( sofar < length ) sendPartial(); } return this; }
- 22. Data Movement is a Defect. Slowing down helps communication. Got Speed?
- 23. 0 Math always produces a number Accuracy rules over speed. Predictive Performance
- 24. 1 Shuffle Data presentation bias. Sorted data => interesting results
- 25. 2 Random acts of Kindness?
- 27. 3 Convex Problems: ADMM
- 28. 4 Amdahl strikes: Cholesky / QR Decomposition Matrix operations jama, jblas.. all single node. Distributed version needs data transfer!
- 29. 5 Random Forests embarrassingly parallel binning tree-building splits
- 30. 6 Boos-ng iterate & stage weak-learners => strong learners each tree can be parallel minimize communication
- 31. 7 Neural Nets & Clustering embarrassingly parallel pre-calculate base stats distance calculation weight matrices – small footprint
- 32. 8 Ensembles Daisy chain a bunch of models Interleave. JIT – Minimize loops over data.
- 33. 9 Tools Deterministic versions first! Got Pen & Paper? Optimize often. Test Big Data soon.
- 34. Replace NAs to improves predictive performance by about 10pc. ! - Newton
- 35. Munging Missing Features impute NAs with mean impute NAs with knn impute with recursive pca! - Boyd
- 36. Unbalanced data single rare classes Fraud / No-Fraud! Stratify
- 37. Unbalanced data multiple rare classes Browse, Click, Purchase! Stratify
- 38. 10 Data is the System Use Customer Data Algorithms for Sparse vs. Dense Unbalanced Data. Robustness under noise
- 39. Before H2O Velocity: Events Online Scoring Volume: HDFS Rule Engine Munging slice n dice Features HIVE/SQL Applications Explora-on Data Scientist Modeling Offline Scoring Engineer Business Analyst Ensemble models Low latency Classification Regression Clustering Optimal Model Predictions
- 40. Big Data Explora-on Modeling Scoring Real-‐-me Big Data beats Better Algorithms!
- 41. Big Data Explora-on Modeling Scoring Real-‐-me Big Data and Better Algorithms! Scale & Parallelism!
- 42. Intellectual Legacy Math needs to be free Open Source Support and Innovation hFps://github.com/0xdata/h2o H 2O the Prediction Engine
- 44. Better Predictions! H2O – The Open Source Math Engine !
- 45. Distributed Coding Taxonomy l No Distribution Coding: l l l Whole Algorithms, Whole Vector-Math! REST + JSON: e.g. load data, GLM, get results! Simple Data-Parallel Coding: l l l Per-Row (or neighbor row) Math! Map/Reduce-style: e.g. Any dense linear algebra! Complex Data-Parallel Coding l K/V Store, Graph Algo's, e.g. PageRank! 0xdata.c45
- 46. Distributed Coding Taxonomy l No Distribution Coding: l l Whole Algorithms, Whole Vector-Math! l REST + JSON: e.g. load data, GLM, get results! Simple Data-Parallel Coding: l Per-Row (or neighbor row) Math! l l Read the docs! This talk! Map/Reduce-style: e.g. Any dense linear algebra! Complex Data-Parallel Coding l K/V Store, Graph Algo's, e.g. PageRank! Join our GIT! 46
- 47. Distributed Data Taxonomy Frame – a collection of Vecs Vec – a collection of Chunks Chunk – a collection of 1e3 to 1e6 elems elem – a java double Row i – i'th elements of all the Vecs in a Frame 0xdata.c47
- 48. Usecases Conversion, Retention & Churn! • Lead Conversion! • Engagement! • Product Placement! • Recommendations! Pricing Engine! Fraud Detection!