A discussion of the relationship between cloud standards and virtualization.

1. Cloud Standards and Virtualization
Dr. Peter Tröger, Senior Researcher
Operating Systems and Middleware Group
Hasso-Plattner-Institute
Universität Potsdam

2. Dr. Peter Tröger | SDPS 2012
Cloud
-„...computing paradigm where the boundaries of computing
will be determined by economic rationale rather than
technical limits."
(R.K. Chellappa 1997)
-Three independent (!)
basic models of
service provisioning
2
Servers Storage
Racks HVAC Power
Virtual Compute
Virtual Machine
Virtual Storage
Key-value Store
Block StoreInfrastructure
“Infrastructure
as a Service” ,
“Utility
Computing”
Cloud
Data
Store
Managed
Container
Comm-
unications
Platforms
“Platform as a
Service”
Business
Applications
Analytics
Applications
Productivity
Applications
Applications
“Software as a
Service”, “on-
demand” apps

3. Dr. Peter Tröger | SDPS 2012
Cloud Role Model
-The customer needs ...
-Predictable scalability for minimal costs (think HPC).
-Application-driven cost optimization (think spot market).
-In many cases at least the reliability of local data centers.
-The customer gets ...
-... some provider-specific interface to a black box.
3 $
Cloud ProviderCloud Customer
$
Cloud ProviderCustomer‘s Client Cloud CustomerPractice
Theory
$

4. Hello,
A few days ago we sent you an email letting you know that we
were working on recovering an inconsistent data snapshot of
one or more of your Amazon EBS volumes. We are very sorry,
but ultimately our efforts to manually recover your volume were
unsuccessful. The hardware failed in such a way that we could not
forensically restore the data.
What we were able to recover has been made available via a
snapshot, although the data is in such a state that it may have
little to no utility…
If you have no need for this snapshot, please delete it to
avoid incurring storage charges.
We apologize for this volume loss and any impact to your business.
Sincerely,
Amazon Web Services, EBS Support

5. Dr. Peter Tröger | SDPS 2012
Dark Clouds
-Amazon Elastic Cloud
-2006: S3 request volumes are monitored,
but cryptographic overhead was not considered
-2008: Single-bit error in transmitted system state lead to global S3
storage outage, took 6 hours for repair,
including complete ,re-boot‘
-2009: Bitbucket.org (Amazon-hosted), 19 hours outage
-2011: Outage of S3, Web 2.0 companies affected for days
-Google Apps (last case in September 2011)
-Microsoft Office 365 (cases in 2011, lasting more than a week)
-T-Mobile Sidekick: One week data outage (2009),
permanent data loss for customers
-... an even larger set of unpublished issues ...
5

6. Dr. Peter Tröger | SDPS 2012
Why Clouds (May) Fail
-Traditional system fault models no longer fit
-Memory with increased density and data rates
-Group of ,simple‘ cores instead of monolithic processor
-Interconnect as crucial component, fault isolation issues
-Reactive fault tolerance gets inappropriate
-Recovery time correlates with system size
-24/7 business availability demands pro-active fault tolerance
-Reactive FT does not scale (Examples: HPC, clouds)
-Virtualization as new system layer
-Dependability of (hardware-supported)
hypervisors, distributed load management
-Imprecise system knowledge
-Information about reliability properties ranges
from imprecise to missing
6

7. Dr. Peter Tröger | SDPS 2012
Solution on Provider Side
-Proactive failover: „Move load away before bad things happen“
-Migration object moved between failover units at one system layer
-System layer as containment barrier
-Coverage of the layer
-Fault model from available data
-Monitoring granularity may prevent fault detection for lower levels
-Overhead of the layer
-Prediction quality (from data) influences false migration percentage
7










  





8. Dr. Peter Tröger | SDPS 2012
Solution on Provider Side
8
!"#$
%&'()*(+&,$-**$
!.'($!.'($
!.'($!.'($
*&/01.&'2$
3(4/5(6$
78$
9::,/5&;.0$8('4('$
78$
<.'=,.&2$
9::,/5&;.0$8('4('$
<.'=,.&2$
-/'+>&,/?&;.0$!,>6+('$*&0&@(A(0+$
PhysicalMachineStatusVirtualMachineStatus
B(&,+C$D02/5&+.'$E&'@(+$*&5C/0($85C(2>,('$*/@'&;.0$!.0+'.,,('$
"'()2/5+.'6$"'()2/5+.'6$
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
B&'2F&'(G$
!"#$%"&'%&()*+,-.$%/&!/%
"'()2/5+.'6$"'()2/5+.'6$
"'()2/5+.'6$
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
-/'+>&,$*&5C/0($*.0/+.'G$
-:'.1(H$0123)4$%4()5%
"'()2/5+.'6$"'()2/5+.'6$
"'()2/5+.'6$
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
7:('&;0@$8I6+(AG$
63(750$%8,-6)91%!)-,3)(,-.%:0(-0+%
"'()2/5+.'6$"'()2/5+.'6$
"'()2/5+.'6$
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
B&'2F&'($,(4(,G$
!"#$%"&'%&()*+,-.$%/&!/%
9::,/5&;.0$J$*/22,(F&'(G$
#44+,57;)-$%#44<0(=0($%><?@AA%
"'()2/5+.'6$"'()2/5+.'6$

9. Dr. Peter Tröger | SDPS 2012
On Customer Side ?
-Allow customer to realize error mitigation
-Avoidance of vendor lock-in
-Functional replication
-Meta-scheduling, adaptive application reconfiguration
-Information dispersal, smart data replication
-> Demands standardized
status monitoring and control
-Support for Offline Operation
-> Demands standardized
status monitoring
9
Cloud Provider
Cloud Provider
Cloud Provider
Cloud Customer
Client

10. Dr. Peter Tröger | SDPS 2012
Cloud Standards
-API for lifecycle management of
-Customer virtual machine (IAAS)
-Customer application (PAAS)
-Customer service instance / tenant / job (SAAS)
-Wide area of functionality
-Deployment, installation, status change, configuration
-Monitoring - Access latency and data rates, availability
-Audit / SLAs - Data removal and locality, isolation
-Development - Tracing and Debugging
10

11. Dr. Peter Tröger | SDPS 2012
Classification of standards
(adopted from Don Box, 2004)
11
- „Desert Island“ specifications -
,must have‘ standards for operations
- „Island Resort“ specifications -
the next layer of important specs
- „New Zealand“ specifications -
specs you'd probably need once in a lifetime
- „Island Of Doctor Moreau“ specifications -
the ugly step children of the spec family
- „Fantasy Island“ specifications -
specs everbody would love to see but never gets

12. Dr. Peter Tröger | SDPS 2012
Cloud Standards
-Prescriptive standards
-Cloud provider <-> provider remote interoperability
-If needed, ask Grid people (OGSI WSRF, Unicore, EMI)
-Cloud customer <-> provider remote interoperability
-Functional access: OCCI, OVF
-SaaS / PaaS data access: SNIA CDMI
-Security: CSA specifications, IETF CloudAudit
-Cloud-based applications (e.g. OGF DRMAA)
-Evaluative standards (ISO 9000, FIPS 140-2)
12

13. Dr. Peter Tröger | SDPS 2012
Distributed Management
Task Force (DMTF)
-Open Virtualization Format (OVF)
-XenSource, IBM, Sun, Microsoft, VMWare, Intel, ...
-Portable virtual machine packaging, extensible
-Virtual disc format, virtual hardware description
-Lifecycle management information
-Specific resource description linked to DMTF CIM model
-Widely accepted in products (e.g. VMWare)
-Cloud Infrastructure Management Interface (CIMI)
-HTTP / REST based cloud management
-Sole IaaS focus
13

14. Dr. Peter Tröger | SDPS 2012
Open Grid Forum
-Open Cloud Computing Interface (OCCI)
-Runtime management API, ReST / HTTP - based
-Infrastructure profile for IaaS, relies on OVF
-Other groups: Monitoring, billing, SLA‘s
14

15. Dr. Peter Tröger | SDPS 2012
Example: OGF OCCI
15

16. Dr. Peter Tröger | SDPS 2012
Data Cloud
-Storage Networking Industry Association (SNIA)
-Cloud storage initiative (CSI) for on-demand storage
-Cisco, HP, IBM, Hitachi, NetApp, Oracle, Symantec, EMC, ...
-From ,manage your storage‘ to ,manage your data‘
-Cloud Data Management Interface (CDMI)
-Allows to tag data with special system metadata
-Tells the cloud storage provider about services requested
-Backup, Archiving, Encryption, ...
16

17. Figure 4 - Cloud Storage Reference Model
Data Storage Cloud
Storage
Services
Data Services
Storage
Services
Data Services
Storage
Services
Data Services
Storage
Services
Data Services
Storage
Services
Data Services
Storage
Services
Data Services
SNIA Cloud Data
Management
Interface (CDMI)
Cloud Data
Management
Table
Table
Table
Table
Table
Draws resources
on demand
Container
POSIX (NFS, CIFS,
WebDAV)
iSCSI, FC, FCoE LUNs,
Targets
XAM VIM
for CDMI
Database/Table
Client
XAM ClientObject Storage Client
Block Storage Client Filesystem Client SNIA Cloud
Data
Management
Interface
(CDMI)
Multiple, vendor-
specific interfaces
Container
Container
Container
Data/Storage Management Client
Management of the cloud
storage can be standalone
or part of the overall cloud
computing management.
Clients acting in the role of using a data storage interface
Clients acting in the
role of managing data/
storage
Clients can be inside the
storage cloud (i.e.,
providing storage
resources to the cloud as
well as consuming them)
or outside the storage
cloud (i.e., only consuming
resources).
Information
Services
(future)
Information
Services
(future)
Information
Services
(future)
Exports to cloud
computing

18. Dr. Peter Tröger | SDPS 2012
Cloud Security Alliance
-Widely supported industry initiative
-Best practices, consistent measurements,
cloud controls matrix, cloud trust protocol,
assurance maturity model, incident management
-Top threats to Cloud Computing
1.Abuse and Nefarious Use of Cloud Computing
2.Insecure Interfaces and APIs
3.Malicious Insiders
4.Shared Technology Issues
5.Data Loss or Leakage
6.Account or Service Hijacking
7.Unknown Risk Profile
18

19. Dr. Peter Tröger | SDPS 2012
More ...
-Open Cloud Consortium (OCC)
-US-based effort for coordinated usage of clouds in research
-Open Science Data Cloud, Project Matsu, OpenFlow
-ETSI TC CLOUD - Continuation of Grid TC
-NIST - Meta standards (vocabulary, use cases, collections)
-OASIS - SAML, IDCloud, WS-*
-Open Group Cloud Work Group - business understanding
-TeleManagement Forum - Cloud marketplace
-> IaaS is nicely covered, Paas / SaaS still missing ...
19

20. Dr. Peter Tröger | SDPS 2012
The End: Some Eco-System
Interoperability
XML, Schema
Messaging Metadata
Resources
Transactions
Security
Reliability
Service Composition / Business Process
Transport (HTTP, MQ, TCP, IIOP, ...)
Agreement
Management
20

21. Dr. Peter Tröger | SDPS 2012
The Quick Check: CSI
21 -Participating Companies ?
-Either agreed by competitors,
or concurrent specifications for the same thing
-Status in standardization organizations ?
-Maturity of the document
-Implementations ?
-More than one implementation is an
indicator for real-world adoption
-Look for implementations by competitors
-Moving target !!!
C
S
I

22. Dr. Peter Tröger | SDPS 2012
The Good, The Bad,
And The Ugly
22 - The Good
- Competitors agree on something
- Backed by a true standardization body
- Multiple independent implementations
- The Bad
- Superseded specifications
- Specs without participation from the providers
- The Ugly
- Company or university proposals with a single
(institutional) author
C
S
I

23. Dr. Peter Tröger | SDPS 2012
Summary
-Cloud dependability: Customer vs. provider perspective
-On customer side, standards would help with vendor lock-in
-IaaS management is covered, data models are hard
-Motivation for uptake of standards
-Innovation (re-use intellectual work)
-Competivness (invite new customers)
-Certification (market advantage for provider)
-Customer demand for interoperability (e.g. X.509)
-More research challenges with billing, PaaS, and SaaS
23