Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20

Share

Download to read offline

Memory Management in Apache Spark

Download to read offline

Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Memory Management in Apache Spark

  1. 1. Deep Dive: Memory Management in Apache Andrew Or May 18th, 2016 @andrewor14
  2. 2. How familiar are you with Apache Spark? a) I contribute to it b) I use it in production c) I am evaluating it d) I have nothing to do with it 2
  3. 3. What is Apache ? 3 Fast and general engine for big data processing Fast to run code – In-memory data sharing – General computation graphs Fast to write code – Rich APIs in Java, Scala, Python – Interactive shell
  4. 4. 4 Spark Core Spark Streaming real-time Spark SQL structured data MLlib machine learning GraphX graph … What is Apache ?
  5. 5. About Databricks 5 Team that created Spark at UC Berkeley Offer a hosted service – Spark in the cloud – Notebooks – Plot visualizations – Cluster management
  6. 6. About Me 6 Apache Spark committer Software eng @ Databricks Hadoop Summit ‘15 Spark Summit Europe ‘15 Some other meetup talks
  7. 7. 7 Efficient memory use is critical to good performance
  8. 8. Memory contention poses three challenges for Apache Spark 9 How to arbitrate memory between execution and storage? How to arbitrate memory across tasks running in parallel? How to arbitrate memory across operators running within the same task?
  9. 9. Two usages of memory in Apache Spark 10 Execution Memory used for shuffles, joins, sorts and aggregations Storage Memory used to cache data that will be reused later
  10. 10. Iterator 4, 3, 5, 1, 6, 2, 8 4 3 5 1 6 2 8 Sort
  11. 11. 4 3 5 1 6 2 8 Sort Execution memory
  12. 12. Iterator 1, 2, 3, 4, 5, 6, 8 1 2 3 4 5 6 8 Sort Execution memory
  13. 13. Iterator Map Iterator 1, 2, 3, 4, 5, 6, 8 +1 2, 3, 4, 5, 6, 7, 9
  14. 14. Iterator Map 1, 2, 3, 4, 5, 6, 8 +1 Map +1 Iterator 1, 2, 3, 4, 5, 6, 8 ... Iterator 2, 3, 4, 5, 6, 7, 9 Iterator 2, 3, 4, 5, 6, 7, 9
  15. 15. Cached Iterator 1, 2, 3, 4, 5, 6, 8 Map Iterator Map Iterator ... Storage memory Map +1 Iterator 2, 3, 4, 5, 6, 7, 9
  16. 16. Challenge #1 How to arbitrate memory between execution and storage?
  17. 17. Easy, static allocation! 18 Total available memory Execution Storage Spark 1.0 May 2014
  18. 18. Easy, static allocation! 19 Execution Storage Spill to disk Spark 1.0 May 2014
  19. 19. Easy, static allocation! 20 Execution Storage Spark 1.0 May 2014
  20. 20. Easy, static allocation! 21 Execution Storage Evict LRU block to disk Spark 1.0 May 2014
  21. 21. 22 Inefficient memory use means bad performance
  22. 22. Easy, static allocation! 23 Execution can only use a fraction of the memory, even when there is no storage! Execution Storage Spark 1.0May 2014
  23. 23. Storage Easy, static allocation! 24 Efficient use of memory required user tuning Execution Spark 1.0May 2014
  24. 24. 25 Fast forward to 2016… How could we have done better?
  25. 25. 26 Execution Storage
  26. 26. 27 Execution Storage Unified memory management Spark 1.6+ Jan 2016 What happens if there is already storage?
  27. 27. 28 Execution Storage Unified memory management Spark 1.6+ Jan 2016 Evict LRU block to disk
  28. 28. 29 Execution Storage Unified memory management Spark 1.6+ Jan 2016 What about the other way round?
  29. 29. 30 Execution Storage Unified memory management Spark 1.6+ Jan 2016 Evict LRU block to disk
  30. 30. Design considerations 31 Why evict storage, not execution? Spilled execution data will always be read back from disk, whereas cached data may not. What if the application relies on caching? Allow the user to specify a minimum unevictable amount of cached data (not a reservation!). Spark 1.6+ Jan 2016
  31. 31. Challenge #2 How to arbitrate memory across tasks running in parallel?
  32. 32. Easy, static allocation! Worker machine has 4 cores Each task gets 1/4 of the total memory Slot 1 Slot 2 Slot 3 Slot 4
  33. 33. Alternative: What Spark does Worker machine has 4 cores The share of each task depends on number of actively running tasks (N) Task 1
  34. 34. Alternative: What Spark does Now, another task comes along so the first task will have to spill Task 1
  35. 35. Alternative: What Spark does Each task is assigned 1/N of the memory, where N = 2 Task 1 Task 2
  36. 36. Alternative: What Spark does Each task is assigned 1/N of the memory, where N = 4 Task 1 Task 2 Task 3 Task 4
  37. 37. Alternative: What Spark does Last remaining task gets all the memory because N = 1 Task 3 Spark 1.0+ May 2014
  38. 38. Static allocation vs What Spark does 39 Both are fair and starvation free Static allocation is simpler What Spark does handles stragglers better
  39. 39. Challenge #3 How to arbitrate memory across operators running within the same task?
  40. 40. SELECT age, avg(height) FROM students GROUP BY age ORDER BY avg(height) students.groupBy("age") .avg("height") .orderBy("avg(height)") .collect() Scan Project Aggregate Sort
  41. 41. Worker has 6 pages of memory Scan Project Aggregate Sort
  42. 42. Scan Project Aggregate Sort Map { // age → heights 20 → [154, 174, 175] 21 → [167, 168, 181] 22 → [155, 166, 188] 23 → [160, 168, 178, 183] }
  43. 43. Scan Project Aggregate Sort All 6 pages were used by Aggregate, leaving no memory for Sort!
  44. 44. Solution #1: Reserve a page for each operator Scan Project Aggregate Sort
  45. 45. Solution #1: Reserve a page for each operator Scan Project Aggregate Sort Starvation free, but still not fair… What if there were more operators?
  46. 46. Solution #2: Cooperative spilling Scan Project Aggregate Sort
  47. 47. Scan Project Aggregate Sort Solution #2: Cooperative spilling
  48. 48. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort forces Aggregate to spill a page to free memory
  49. 49. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort needs more memory so it forces Aggregate to spill another page (and so on)
  50. 50. Scan Project Aggregate Sort Solution #2: Cooperative spilling Sort finishes with 3 pages Aggregate does not have to spill its remaining pages Spark 1.6+ Jan 2016
  51. 51. Recap: Three sources of contention 52 How to arbitrate memory … ● between execution and storage? ● across tasks running in parallel? ● across operators running within the same task? Instead of avoid statically reserving memory in advance, deal with memory contention when it arises by forcing members to spill
  52. 52. Project Tungsten 53 Binary in-memory data representation Cache-aware computation Code generation (next time) Spark 1.4+ Jun 2015
  53. 53. “abcd” 54 • Native: 4 bytes with UTF-8 encoding • Java: 48 bytes – 12 byte header – 2 bytes per character (UTF-16 internal representation) – 20 bytes of additional overhead – 8 byte hash code Java objects have large overheads
  54. 54. 55 Schema: (Int, String, string) Row Array String(“data”) String(“bricks”) 5+ objects, high space overhead, expensive hashCode() BoxedInteger(123) Java objects based row format
  55. 55. 6 “bricks” 56 0x0 123 32L 48L 4 “data” (123, “data”, “bricks”) Null tracking bitmap Offset to var. length data Offset to var. length data Tungsten row format
  56. 56. Cache-aware Computation 57 ptr key rec ptr key rec ptr key rec Naive layout Poor cache locality ptrkey prefix rec ptrkey prefix rec ptrkey prefix rec Cache-aware layout Good cache locality E.g. sorting a list of records
  57. 57. For more info... Deep Dive into Project Tungsten: Bringing Spark Closer to Bare Metal https://www.youtube.com/watch?v=5ajs8EIPWGI Spark Performance: What’s Next https://www.youtube.com/watch?v=JX0CdOTWYX4 Unified Memory Management https://issues.apache.org/jira/browse/SPARK-10000
  58. 58. Thank you andrew@databricks.com @andrewor14
  • suyounshin1

    Apr. 14, 2020
  • ShahzadBadar1

    Sep. 25, 2019
  • RyanWong8

    Jun. 21, 2019
  • JasonWayne

    Jul. 20, 2018
  • ChenxiWang9

    Jun. 5, 2018
  • ssuser70909d1

    Mar. 26, 2018
  • tkdsh3168

    Mar. 13, 2018
  • guolongcang

    Jan. 22, 2017
  • cris_weber

    Sep. 15, 2016
  • pzz2011

    Jul. 29, 2016
  • gianvitosiciliano

    Jul. 21, 2016
  • mrsohn

    Jul. 21, 2016
  • SharathBhat3

    Jun. 1, 2016
  • josecarlosgarciaserrano

    May. 30, 2016
  • allenjoe1986

    May. 28, 2016
  • sasikirankarri

    May. 27, 2016
  • ChongkaiHong

    May. 26, 2016
  • YuJaeHyuk

    May. 25, 2016
  • qiang5714

    May. 24, 2016
  • bunkertor

    May. 23, 2016

Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.

Views

Total views

8,662

On Slideshare

0

From embeds

0

Number of embeds

66

Actions

Downloads

254

Shares

0

Comments

0

Likes

20

×