1. OpenStack
APAC Conference
Building Applications with OpenStack
Swift
Joe Arnold, CEO
乔 . 阿诺德行政总裁斯威夫特斯达克
August 11, 2012
SwiftStack Inc
@joearnold

2. August 11, 2012
Compute | Storage |
Global “Nova”
communityNetworking “Quantum”
“Swift”
of cloud software developers &
users
Apache 2 Open Source License
+180 Participating Companies - 1,000’s of
developers
4
2

3. August 11, 2012
OpenStack Swift - Born in Production
2009 2010 2011
Developed in large-scale production
environments
Currently +70 developers - has doubled every 6
month
3

4. August 11, 2012
nt
c e e
ym
p alo l
pp
e:
iks
ftE
x
em
Da
A
S
a
wc U S
R 4

5. August 11, 2012
nt
ym
u
e:
d
e
lo l
pp
em lo
iftE
Da
x C A
S w
P U5
S
H

6. August 11, 2012
nt
a p
ym
lo le:
e
rn
pp
em
Da
w
iftEe
x
t U S A
In
S
6

7. August 11, 2012
nt
ym u d
e
e:
em
Da c lo
lo l
pp
ea
x
Sw
iftE
U o r
T K
K 7

8. August 11, 2012
nt
er
yme:
e
y
lo l
pp
la
em
Da
ft U
x
iftE A
S
S w
So 8

9. August 11, 2012
ent
ym
l ix
lo l
pp
e:
i a
ftE
xy
em
Da
al
ia
S w
H us tr
9
A

10. August 11, 2012
nt
ceyme:
e
a n lo l
pp
em
w
iv
ftE
o r
Da
x
n ce
S
N F a
e 10

11. August 11, 2012
About SwiftStack
Cloud Storage System based on OpenStack Swift
Cloud storage technical leadership
Swift Core team
Project lead
Experience
Building large-scale cloud storage at
Rackspace, Engine Yard, Internap, Korea Telecom
11

12. August 11, 2012
Field Observations
12

13. August 11, 2012
Field Observations
Web/Mobile Applications
Web/Mobile Applications
Massive Number of Users
Infrastructure as a Service
13

14. August 11, 2012
Field Observation: Web/Mobile
Applications
14

15. August 11, 2012
Field Observation: Web/Mobile
Applications
Shrinking Storage on Mobile Devices
15

16. August 11, 2012
OpenStack Swift enables Web/Mobile applications
Speaks: HTTP
HTTP API
Storage
Standards-Based OpenStack API
30
16

17. August 11, 2012
OpenStack Swift enables Web/Mobile applications
Upload direct from device
HTTP API
Storage
HTTP Form Post Mechanism
30
17

18. August 11, 2012
OpenStack Swift enables Web/Mobile applications
Cache with HTTP
HTTP API
Cache Storage
30
18

19. August 11, 2012
OpenStack Swift enables Web/Mobile applications
Native Content-Delivery Integration
30
19

20. August 11, 2012
OpenStack Swift enables Web/Mobile applications
20

21. August 11, 2012
Field Observations
Web/Mobile Applications
Massive Number of Users
Infrastructure as a Service
21

22. August 11, 2012
Field Observation: Massive # of users
Enterprise Application
22

23. August 11, 2012
Field Observation: Massive # of users
23

24. August 11, 2012
Field Observation: Massive # of users
Multi-Tenant Application
24

25. August 11, 2012
Field Observation: Massive # of users
FREQUENCY
Long-Tail Data
$/IO $/GB
25

26. August 11, 2012
Field Observation: Massive # of users
26

27. August 11, 2012
Field Observation: Massive # of users
27

28. August 11, 2012
Swift Scales to Massive # of users
Access Node Access Node Access Node Access Node
Add Proxy Nodes A hash ring is
shared amongst each node in the
cluster.
Add Storage Nodes capacity
can be added by growing existing
availability zones, or adding new
availability zones.
28

29. August 11, 2012
Swift Scales to Massive # of users
Swift uses shared-nothing architecture
1. account data
2. object data
3. All data distributed via hash
ring
Simple mechanisms proven at scale
1. Whole files on disk
2. Routed networking (Layer 3 networks)
3. Proven techniques with HTTP for transport
4. Proven techniques with rsync for
replication
29

30. August 11, 2012
Swift Scales to Massive # of users
Sharding
30

31. August 11, 2012
Swift Scales to Massive # of users
HTTP + Proven Cloud Architecture
31

32. August 11, 2012
Field Observations
Web/Mobile Applications
Massive Number of Users
Infrastructure as a Service
32

33. August 11, 2012
Field Observation: Infrastructure as a service
Improve Efficiencies
Our
Custome Improve agility
r Wants
Sell Services
33

34. August 11, 2012
Field Observation: Infrastructure as a service
Storage deployments becoming larger
Multi-tenancy important
34

35. August 11, 2012
Swift enables Infrastructure as a Service
Scales to large capacity
High concurrency
Multi-tenancy from ground-up
35

36. August 11, 2012
Swift enables Infrastructure as a Service
Operational
Operational
Efficiencies
Efficiencies
36

37. August 11, 2012
SwiftStack Nodes
37

38. August 11, 2012
SwiftStack Controller
38

39. August 11, 2012
Capacity
Text
time
39

40. August 11, 2012
Watch Everything
Lightweight
Swift Process
Swift Process
Swift Process
UDP StatsD
Swift Process
Swift Process
Swift Process
StatsD
Time-Series Backend
Ganglia / Graphite
40

41. August 11, 2012
See Everything Be Notified
60
41

42. Make eachAugust like a
Make eachlook 11, 2012
look like a
mini-product. Box around
mini-product. Box around
each.
each.
SwiftStack Plug-Ins
User Dashboard
On-disk Encryption
Active Directory/LDAP
Integration
Utilization API for Billing
Metadata Search
42

43. August 11, 2012
Field Observations
Web/Mobile Applications
Massive Number of Users
Infrastructure as a Service
43

44. Thank you!
OpenStack
APAC Conference
August 11, 2012
Joe Arnold
乔 . 阿诺德行政总裁斯威夫特斯达克
CEO, SwiftStack
joe@swiftstack.com
@joearnold