SlideShare a Scribd company logo
1 of 29
Distributed Computing Seminar Lecture 3: Distributed Filesystems Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Google, Inc.  Summer 2007 Except as otherwise noted, the content of this presentation is © Copyright University of Washington and licensed under the Creative Commons Attribution 2.5 License.
Outline ,[object Object],[object Object],[object Object]
Filesystems Overview ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Distributed Filesystems ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
NFS  ,[object Object],[object Object],[object Object]
NFS Protocol ,[object Object],[object Object],[object Object],[object Object]
Server-side Implementation ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS Locking ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS Client Caching ,[object Object],[object Object],[object Object],[object Object],[object Object]
NFS: Tradeoffs ,[object Object],[object Object],[object Object],[object Object]
The Google File System Sanjay Ghemawat, Howard Gobioff,  and Shun-Tak Leung SOSP 2003 (These slides by Alex Moshchuk, University of Washington – used with permission)
Motivation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Assumptions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
GFS Design Decisions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
GFS Architecture ,[object Object],[object Object],… Can anyone see a potential weakness in this design?
Single master ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Metadata (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Metadata (2/2) ,[object Object],[object Object],[object Object],[object Object]
Mutations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Atomic record append ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Relaxed consistency model (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Relaxed consistency model (2/2) ,[object Object],[object Object],[object Object],[object Object]
Master’s responsibilities (1/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Master’s responsibilities (2/2) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Fault Tolerance ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Performance
Deployment in Google ,[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Next Time... ,[object Object],[object Object]

More Related Content

What's hot

dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood
Susan Wu
 
Porting dmtcp mac_slides
Porting dmtcp mac_slidesPorting dmtcp mac_slides
Porting dmtcp mac_slides
yuliang_neu
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
Ravindra Raju Kolahalam
 

What's hot (20)

dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood dotCloud (now Docker) Paas under the_hood
dotCloud (now Docker) Paas under the_hood
 
20150207 howes-gpgpu8-dark secrets
20150207 howes-gpgpu8-dark secrets20150207 howes-gpgpu8-dark secrets
20150207 howes-gpgpu8-dark secrets
 
.ppt
.ppt.ppt
.ppt
 
Interprocess communication (IPC) IN O.S
Interprocess communication (IPC) IN O.SInterprocess communication (IPC) IN O.S
Interprocess communication (IPC) IN O.S
 
IPC
IPCIPC
IPC
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory Systems
 
PThreads Vs Win32 Threads
PThreads  Vs  Win32 ThreadsPThreads  Vs  Win32 Threads
PThreads Vs Win32 Threads
 
ITFT_Inter process communication
ITFT_Inter process communicationITFT_Inter process communication
ITFT_Inter process communication
 
WiMAX implementation in ns3
WiMAX implementation in ns3WiMAX implementation in ns3
WiMAX implementation in ns3
 
IPC
IPCIPC
IPC
 
NS-3
NS-3 NS-3
NS-3
 
Porting dmtcp mac_slides
Porting dmtcp mac_slidesPorting dmtcp mac_slides
Porting dmtcp mac_slides
 
Capturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspaceCapturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspace
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
 
INET for Starters
INET for StartersINET for Starters
INET for Starters
 
Multiple processor systems
Multiple processor systemsMultiple processor systems
Multiple processor systems
 
Threads (operating System)
Threads (operating System)Threads (operating System)
Threads (operating System)
 
Chap 4
Chap 4Chap 4
Chap 4
 
Lecture 5 inter process communication
Lecture 5 inter process communicationLecture 5 inter process communication
Lecture 5 inter process communication
 

Similar to Distributed computing seminar lecture 3 - distributed file systems

Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
Sri Prasanna
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
Sri Prasanna
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
Ntu
 
Dfs (Distributed computing)
Dfs (Distributed computing)Dfs (Distributed computing)
Dfs (Distributed computing)
Sri Prasanna
 
Distributed Filesystems Review
Distributed Filesystems ReviewDistributed Filesystems Review
Distributed Filesystems Review
Schubert Zhang
 
Ch16 OS
Ch16 OSCh16 OS
Ch16 OS
C.U
 

Similar to Distributed computing seminar lecture 3 - distributed file systems (20)

Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
 
Distributed file systems
Distributed file systemsDistributed file systems
Distributed file systems
 
tittle
tittletittle
tittle
 
Distributed File Systems
Distributed File SystemsDistributed File Systems
Distributed File Systems
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
 
Google
GoogleGoogle
Google
 
Dfs (Distributed computing)
Dfs (Distributed computing)Dfs (Distributed computing)
Dfs (Distributed computing)
 
CS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMSCS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMS
 
Gfs介绍
Gfs介绍Gfs介绍
Gfs介绍
 
Kosmos Filesystem
Kosmos FilesystemKosmos Filesystem
Kosmos Filesystem
 
GOOGLE FILE SYSTEM
GOOGLE FILE SYSTEMGOOGLE FILE SYSTEM
GOOGLE FILE SYSTEM
 
Distributed File System.ppt
Distributed File System.pptDistributed File System.ppt
Distributed File System.ppt
 
Distributed Filesystems Review
Distributed Filesystems ReviewDistributed Filesystems Review
Distributed Filesystems Review
 
Advance google file system
Advance google file systemAdvance google file system
Advance google file system
 
Hadoop Distributed File System for Big Data Analytics
Hadoop Distributed File System for Big Data AnalyticsHadoop Distributed File System for Big Data Analytics
Hadoop Distributed File System for Big Data Analytics
 
Hadoop
HadoopHadoop
Hadoop
 
Chapter 17 - Distributed File Systems
Chapter 17 - Distributed File SystemsChapter 17 - Distributed File Systems
Chapter 17 - Distributed File Systems
 
OSCh16
OSCh16OSCh16
OSCh16
 
Ch16 OS
Ch16 OSCh16 OS
Ch16 OS
 
OS_Ch16
OS_Ch16OS_Ch16
OS_Ch16
 

Recently uploaded

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Recently uploaded (20)

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 

Distributed computing seminar lecture 3 - distributed file systems

  • 1. Distributed Computing Seminar Lecture 3: Distributed Filesystems Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Google, Inc. Summer 2007 Except as otherwise noted, the content of this presentation is © Copyright University of Washington and licensed under the Creative Commons Attribution 2.5 License.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11. The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung SOSP 2003 (These slides by Alex Moshchuk, University of Washington – used with permission)
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 27.
  • 28.
  • 29.