LeoFS has High Performance, High Reliability, High Scalability, file system developed by Rakuten.

1. S3 Compatible Storage
“LeoFS”
Rakuten. Inc, RIT Yosuke Hara 25/05/2012 1

2. Table of Contents
1. Motivation
2. Overview
3. Inside of LeoFS
4. WEB Console
2

3. 1. Motivation
3

4. Why NFS?
Which is suitable storage
for storing the media files?
?
Low ROI
Possibility of SPOF
Storage Expansion is difficult during increasing data
4

5. 5

6. Object Storage Farm for IaaS
6

7. 2. Overview
7

8. Overview
Storage
Gateway
Manager
8

9. System Layout
Request from Web Application(s)
Load Balancer
LeoFS-Manager
S3-API
REST over HTTP
LeoFS-Gateway
RPC
w/Cache Server
SNMP
RPC
LeoFS-Storage
Storage Engine/Router
Storage Engine/Router
Storage Engine/Router
WEB Console
META
Object Store
META
Object Store
META
Object Store
9

10. System Layout
Request from Web Application(s)
Load Balancer
Gateway
Manager
LeoFS-Manager
HTTP
Cluster
Request/Response Handling
S3-API
Management
REST over HTTP
LeoFS-Gateway
+
RPC
w/Object Cache
w/Cache Server
Ring Watcher
(AWS S3-API)
Node Watcher
SNMP
RPC
LeoFS-Storage
Storage
GUI Console
Object Storage, Meta data Storage
Storage Engine/Router
Storage Engine/Router
Storage Engine/Router
+
META
Replicator/Recoverer
Object Store
META
META
Object Store
Object Store
10

11. 3. Inside of LeoFS
11

12. Architecture
HTTP
Gateway (stateless)
Erlang RPC
Erlang RPC
Storage Cluster
(multi master)
Erlang RPC
Process Monitor
Manager Cluster
12

13. Architecture
Architecture - Gateway/Storage
LeoFS Gateway Cacher
REST over HTTP (S3-API)
get put delete head
redundant-manager membership (fault-detection)
RPC
LeoFS-Storage
redundant-manager replicator read-repairer
RPC RPC
membership (fault-detection) queue
Storage Engine
Object Storage Metadata Storage
13

14. Architecture
Architecture - Manager Cluster
Erlang Mnesia
RING
Member / Cluster State
Auth / ACL
Process Monitor
Gateway / Storage Cluster
14

15. Architecture
15

16. Three “HIGH”
High Cost Performance
Monolithic Storage System
Storage Engine For Unstructured Files
Traffic Restrain Mechanism
> File Cache System (Gateway Plugin)
16

17. Three “HIGH”
High Reliability
NO SPOF
Split Brain Measure
“Erlang OTP” > Nine Nines (99.9999999%)
17

18. Three “HIGH”
High Scalability
Elastic Storage System
> Able to dynamic attach/detach nodes
> Able to over 100-nodes
> NOT Mesh-connected Mutual Servers
18

19. Cache Mechanism
19

20. Cache Mechanism
Gateway Buffer Pool
Slab Alloc
Skiplist
{$filename, $etag}
request
from Client
response
Gateway match: {ok, match}
NOT match: {ok, $metadata, $body,}
High I/O efficiency
Low Latency
20

21. Storage Engine
21

22. Storage Engine
Metadata + Object Storage
LeoFS-Storage
Storage Engine / Replicator / Recoverer
Object’s Attribute Storage
(metadata)
Object Storage
Metadata : Keeps an in-memory index of all data.
Object Storage : Log structured (append-only) object store.
22

23. Storage Engine
Retrieve an object from Storage
Storage Engine
< META DATA >
ID
Filename
Offset
Size
Checksum
Data
Header
Metadata
File
Footer
Object Container
23

24. Storage Engine
Insert an object into Storage
Storage Engine
Add a Metadata
Meta Data Server
Data
Append a File 24

25. Storage Engine
Reduce unnecessary files
Compaction
25

26. 4. WEB Console
26

27. Web Console
File Manager
WEB Console
Node Stats
Log Search
27

28. Web Console
Node State Monitor
Log Analyzer / Searcher
28

29. Web Console
Web Console System Layout
Manager Storage Gateway
or
GUI Console
Producers and
Admins
Logstatsh
29

30. Wrap Up
30

31. Wrap Up
High Cost Performance
High Reliability
High Scalability
31

32. 32

33. Thank you for your time
33