Papers We Love Too, June 2015: Haystack
•
3 likes•1,026 views
Slides from Papers We Love, June 2015 on Facebook's Haystack paper. A storage system designed to serve needles with high throughput, and low latency.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Papers We Love Too, June 2015: Haystack
Similar to Papers We Love Too, June 2015: Haystack (20)
Recently uploaded
Recently uploaded (20)
Papers We Love Too, June 2015: Haystack
- 1. Facebook Haystack Finding a needle in Haystack: Facebook's photo storage. An Analysis of Facebook Photo Caching PWL SF - June 30, 2015
- 4. Agenda •The Haystack problem •Design & Architecture •Takeaways
- 7. Needle Storage & Serving For Facebook
- 8. Workload
- 9. •Write Once •Read Often •Delete Rarely
- 10. •Write Once •Read Often •Delete Rarely
- 11. •Write Once •Read Often •Delete Rarely
- 12. Why Haystack as a Paper?
- 13. Why Not?
- 15. 20+ Petabytes
- 16. 120 million new photos a day
- 17. Simple
- 20. History
- 21. Network Attached Storage (NAS) mounted over NFS
- 34. Fetches are expensive • Multiple seeks: • Directory metadata • Inode • File contents • File metadata is 10s of kilobytes • Long-tail uncachable
- 35. Decision to Build • Existing systems unable to be adapted • Hadoop • MySQL • Traditional NAS appliances • Don’t need to solve for the kitchen sink • Log data • Development work
- 36. Haystack Design
- 40. Simple
- 41. Separation of concerns •Haystack Store •Haystack Cache •Haystack Directory
- 42. Read Path
- 43. Write Path
- 44. Store
- 46. How much cache?
- 47. There is no right ratio
- 50. Volume Layer Volume layer above filesystem
- 54. Indexing
- 55. 10 bytes of metadata per photo
- 56. 2-bit overhead
- 57. Read by <Key, Alt Key, Cookie>
- 62. Batch Upload
- 63. Similar to Bitcask & CDB
- 64. Uses XFS
- 66. Fault Tolerance
- 70. Recovery: Rsync* * With QoS
- 72. The Hardware
- 74. Open Vault: KNOX
- 78. Good at reads xor writes not both
- 79. Read Throughput Avg. Read Latency Write Throughput Avg. Write Latency Only Reads 770.6 33.2 - - Only Writes - - 6099.4 4.9 Multiwrite (x16) - - 10843.8 43.9 Reads And Writes 718.1 41.6 232.0 11.9 Latencies Table
- 80. Haystress
- 82. Haystack Store • Responsibilities: • Read needles • Write needles • Append-only • O(1) read cost* *Usually
- 83. Cache
- 85. Organized as DHT
- 86. Not just an LRU
- 89. Request is to write- enabled store
- 90. Haystack Cache •Simple cache •Optimizations, given access patterns
- 91. Directory
- 92. The Rug
- 94. Maps logical volumes to physical machines
- 95. Mapping based on business rules
- 97. Directs writes to relevant logical volume
- 98. Directs reads away from CDN
- 99. Directory • Manages capacity • Manages volume mapping • Manages image mapping • Manages CDN
- 100. Tying it together
- 101. Writes
- 108. Write-path • Involves • Store • Directory • Smart client
- 109. Reads
- 113. Detour: URLs
- 114. Directory uses URLs for directing
- 116. URL Makeup • CDN • Cache Node • Machine ID • Logical Volume ID • Photo ID & Alt ID • Cookie
- 127. Read-path • Involves: • Directory • Cache • CDN
- 128. Insights
- 129. Narrow Scope “That simplicity let us build and deploy a working system in a few months instead of a few years.”
- 130. Sometimes you’re solving the wrong problem
- 131. Smart Clients & Ecosystem Control
- 132. Simple Optimizations
- 134. Thanks & Qs