The presentation describes the different cloud federation scenarios, ranging from a federation built on commercial cloud providers that offer no real support for federation to one built on data centers of the same organization where the sites are completely dedicated to supporting all aspects of federation. The level of federation is defined based on the amount of information disclosed and how much control over the resources is provided across sites. The talk also presents the existing challenges for interoperability in federated and hybrid cloud computing scenarios, and ends with real-life examples of multi-cloud environments running OpenNebula.

1. Challenges in Hybrid and
Federated Cloud Computing
Ignacio M. Llorente
DSA-Research.org
Distributed Systems Architecture Research Group
Universidad Complutense de Madrid
© OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License 1/33

2. Contents
Challenges in Hybrid and Federated Cloud Computing!
● What is Cloud Computing?
● What is OpenNebula?
● The Future of Cloud Computing
● Levels of Coupling
● Common Architectures for Federation
● Challenges for Interoperability
● Cloud Federation in Grid Infrastructures
Challenges in Hybrid and Federated Cloud Computing 2/33

3. What is Cloud Computing?
Provision of IT Capabilities as a Service!
What Who
On-demand End-user
Software as a Service access to any (does not care about hw or sw)
application
Platform for Developer
Platform as a Service building and (no managing of the underlying hw
delivering web & swlayers)
applications
Infrastructure as a Raw computer System Administrator
ﾺ
Service
infrastructure (complete management of the
computer infrastructure)
Physical Infrastructure
Challenges in Hybrid and Federated Cloud Computing 3/33

4. What is Cloud Computing?
Provision of Virtualized Resources as a Service!
• Raw infrastructure
• Management Console resources
• Simple REST API’s
Manage
Instances
• Pay-as-you-go & elastic capacity
Challenges in Hybrid and Federated Cloud Computing 4/33

5. What is Cloud Computing?
Different Models of Deployment!
Model Definition Cloud Cases
Infrastructure is owned by • Optimize and simplify internal operation
Private
a single organization and • SaaS/PaaS support
made available only to the • IT consolidation within large organizations
organization (Goverment Clouds, University Clouds…)
Infrastructure is owned by • Low cost solutions with limited control/
a single organization and configuration and security/reliability good
made available to other enough
Public
organizations over the • Commercial cloud providers, mostly
Internet hosting providers
• Science public clouds by ICT service
centers to enable scientific and educational
projects to experiment with cloud
computing
Infrastructure is owned by • Premium solutions with additional control/
Private
Virtual
a single organization and configuration and security/reliability
made available to other • Telecom cloud providers
organization over a
dedicated private network
Challenges in Hybrid and Federated Cloud Computing 5/33

6. What is OpenNebula?
Cloud as an Evolution of the Data Center!
Constraints from Requirements from
Existing Infrastructure Usage and Deployment
and Processes Scenarios
“One solution does not fit all
requirements and
constraints”
OpenNebula makes cloud an evolution by leveraging
existing IT assets, protecting your existing investments,
and avoiding vendor lock-in
Challenges in Hybrid and Federated Cloud Computing 6/33

7. What is OpenNebula?
IaaS Cloud Computing Tool for Managing a Data Center's Virtual Infrastructure!
Most Advanced and Flexible, Enterprise-grade IaaS Cloud Manager
Adaptable
● Customizable and Extensible
Proven
● Many Massive Scale Production Deployments
Powerful
● Most Advanced Enterprise-class
Functionality
No Lock-in
● Platform Independent and Interoperable
Interoperable
● Most popular cloud APIs and standard based
Openness
● Fully open-source
● Apache license
Challenges in Hybrid and Federated Cloud Computing 7/33

8. What is OpenNebula?
The Internals of the Cloud!
Interfaces & API
• CLI (local/remote)
• API (java, ruby bindings)
• Sunstone
• Cloud (EC2, OCCI)
VM Networking
• VLANs Image Repository of VM disks
• Firewall hooks • ACLs (public + private)
• User defined • OS and Data types (persistent)
• Multiple storage backends
User & Roles
Hosts: Cluster workernodes to run VMs. Storage
• Multiple hypervisors defined per host • NAS
• Grouped in logical clusters • SAN
• Custom monitoring probes and technologies • Custom (bittorrent, ssh…)
Challenges in Hybrid and Federated Cloud Computing 8/33

9. What is OpenNebula?
EU Success Story in Cloud Computing Research and Innovation!
Flagship EU Project on
Deployments with Cloud Computing Research
4,000
tens of thousands downloads/month
of VMs
Mature
Wordwide engaged technology
community started in 2005
Innovative
Technology Spin-off
Management Tool Innovation Platform
Challenges in Hybrid and Federated Cloud Computing 9/33

10. The Future of Cloud Computing
Next Step in the Evolution of an Utility!
Utility Utility
Generation Distribution Utility Grid
Challenges in Hybrid and Federated Cloud Computing 10/33

11. The Future of Cloud Computing
Benefits of Federation!
Scalability
● Cloudbursting to address peak demands
● Sharing of infrastructure between partners
● Infrastructure aggregation across sites
Reliability
● Fault tolerance architectures across sites
Performance
● Deployment of services closer to end users
Cost
● Dynamic placement to reduce the overall infrastructure cost
Energy Consumption
● Minimize energy consumption
Challenges in Hybrid and Federated Cloud Computing 11/33

12. The Future of Cloud Computing
The Cluster Computing Use Case!
Virtual Computing Cluster Flexibility
• Dynamic elasticity rules
Cluster Users • Dynamic provision of cluster configurations
Workernodes
Completely Transparent
• LRMS interface
Front-end
Simplify and Optimize Internal Operations
• Dynamic performance partitioning
• Consolidation
• Faster upgrades
Challenges in Hybrid and Federated Cloud Computing
• Maintenance
12/33

13. The Future of Cloud Computing
The Cluster Computing Use Case!
Scalability
Performance
Cost
Fault tolerance
Challenges in Hybrid and Federated Cloud Computing 13/33

14. Levels of Coupling
Different Levels of Control, Monitoring, Cross-site Functionality and Security!
Private Cloud
Partner Cloud Company Cloud
Public Cloud
Challenges in Hybrid and Federated Cloud Computing 14/33

15. Levels of Coupling
Loosely Coupled Federation!
Federation with a Cloud without Interoperation Support
• Basic operations over VMs (start, shutdown, restart…)
Control
• Different instance types
Monitoring & • Basic virtual resource monitoring (resource consumption…)
Accounting
Cross-site • None
Security • Single account representing the organization
Challenges in Hybrid and Federated Cloud Computing 15/33

16. Levels of Coupling
Partially Coupled Federation!
Federation with a Cloud with Partial Interoperation Support
• Advanced operations over VMs (live migration…)
Control
• VM location and affinity constraints
Monitoring & • Advanced virtual resource monitoring (energy consumption,
Accounting VM placement…)
• Virtual networks
Cross-site
• Virtual storage
Security • Framework agreement
Challenges in Hybrid and Federated Cloud Computing 16/33

17. Levels of Coupling
Tightly Coupled Federation!
Federation with a Cloud with Advanced Interoperation Support
• Placement on specific physical resources
Control
• Same instance types
Monitoring & • Physical resource consumption
Accounting
• Live migration
Cross-site
• High availability
Security • User space sharing
Challenges in Hybrid and Federated Cloud Computing 17/33

18. Common Architectures for Cloud Federation
Organization of Multi-site Cloud Environments!
Challenges in Hybrid and Federated Cloud Computing 18/33

19. Common Architectures for Cloud Federation
Cloudbursting Architecture!
Corporate
cloud
Cloud
OS
Public
Public
Cloud
1
Cloud
2
Local data center
Aim • Meet peak demands
• Public cloud by commercial provider
Cloud Type
• VPC by telecom provider
Coupling • Loosely and partially coupled
Challenges in Hybrid and Federated Cloud Computing 19/33

20. Common Architectures for Cloud Federation
Cloudbursting Architecture!
Enhancing Grid Infrastructures with Cloud Computing
EU grant agreement RI-261552 Simplify and optimize its use and operation, providing a more
(2010-2012)
e-Infrastructure
flexible, dynamic environment for scientists; and enhance
existing computing infrastructures with “IaaS” paradigms
www.StratusLab.eu
www.stratuslab.org
Challenges in Hybrid and Federated Cloud Computing 20/33

21. Common Architectures for Cloud Federation
Dynamic Combination of Local with Remote Cloud Resources!
Service Level Agreement
• SLO 1: Average_CPU_Utilization > 75%
• Action: Execute remote WN
Grid Services
StratusLab StratusLab
Challenges in Hybrid and Federated Cloud Computing 21/33

22. Common Architectures for Cloud Federation
Cloud Broker Architecture!
Cloud
OS
(Broker)
Public
Public
Public
Cloud
1
Cloud
2
Cloud
3
Aim • Cost, performance and reliability optimization
Cloud Type • Public clouds by commercial provider
Coupling • Loosely coupled
Challenges in Hybrid and Federated Cloud Computing 22/33

23. Common Architectures for Cloud Federation
Cloud Broker Architecture!
Building Service Testbeds on FIRE
Agreement 257386 (2010-2013) Design, build and operate a multi-site cloud-based facility to
New Infrastructure Paradigms support research across applications, services and systems
and Experimental Facilities targeting services research community on Future Internet
www.BonFIRE-Project.eu
Source: BonFIRE Project
Challenges in Hybrid and Federated Cloud Computing 23/33

24. Common Architectures for Cloud Federation
Aggregated Cloud Architecture!
Corporate Corporate
cloud 1 cloud 2
Cloud
OS
Cloud
OS
Local data center Local data center
Aim • Sharing of resources to meet peak demands
Cloud Type • Partner or company clouds
Coupling • Partially or tightly coupled
Challenges in Hybrid and Federated Cloud Computing 24/33

25. Common Architectures for Cloud Federation
Aggregated Cloud Architecture!
Resources and Services Virtualization without Barriers
Agreement 215605 (2008-2011) Open source technology to enable deployment and
Service and Sw Architectures and management of complex IT services across different
Infrastructures administrative domains
www.reservoir-fp7.eu
Challenges in Hybrid and Federated Cloud Computing 25/33

26. Common Architectures for Cloud Federation
Multi-tier Cloud Architecture!
Corporate
cloud
Cloud
OS
Cloud Cloud
site 1 site 2
Cloud
OS
Cloud
OS
Local data center Local data center
• Very large scale or geographically distributed
Aim
data centers
Cloud Type • Company clouds
Coupling • Tightly coupled
Challenges in Hybrid and Federated Cloud Computing 26/33

27. Common Architectures for Cloud Federation
Multi-tier Cloud Architecture!
CloudFront
• Portal
• Cloud API (EC2, OCCI)
User • Global AuthN
OpenNebula Zone OpenNebula Zone
• Multi-tenancy • Multi-tier arch.
• Runs Instances • Scalable
• Stores Images • Isolation
• Networking • Multiple-site
• Authorization support
Challenges in Hybrid and Federated Cloud Computing 27/33

28. Challenges for Interoperability
Leveraging Existing Standards and Implementing Interoperation!
Standardization
● Implement standards
● Integrate with standards
Which Standard?
● Different de jure standards
● Several de facto standards
Interoperation
● Implement adaptors
● Use transformers
Challenges in Hybrid and Federated Cloud Computing 28/33

29. Challenges for Interoperability
I Want to Combine My Private Cloud with a Remote Cloud without Changes!
Transparent Combination of Local Resources with Cloud Resources
1. Management Interfaces 2. Management Interfaces
for Virtual Workloads for Data Elements CDMI
VM
VM VM VM VM
4. Contextualization
OVF
3. VM Image Description
5. Cross-site Networking and Instance Type
and Latency
OVF
Challenges in Hybrid and Federated Cloud Computing 29/33

30. Cloud Federation in Grid Infrastructures
Grid and Cloud as Complementary Computing Models!
Usage Grids Usage Clouds
§ Job Processing § Raw infrastructure
§ Big Batch System § Elasticity & Pay-per-use
§ File Sharing Services § Simple Web Interface
Achievements Achievements
§ Federation of Resources
§ Agile Infrastructures
§ VO Concept
§ IT is another Utility
But…
But…
§ User experience
§ Interoperability
§ Complexity
§ Federation
Resource Sharing Resource Management
Scientific Applications Customize Environments
Uniform Security Flexibility & Simplicity
Challenges in Hybrid and Federated Cloud Computing 30/33

31. Cloud Federation in Grid Infrastructures
The StratusLab Vision!
• Batch Job Processing
• Custom Execution Environments
2 • Industry Applications
• Other WMS (pilots)
Access
• Grid Service Integration • Complete Services (cluster)
Grid Middleware IaaS Interface
LRMS (LSF, PBS…) StratusLab Toolkit
Service
1
Virtual CE, WN… Other (web, mail...) Raw machines
Provision
3
Grid Sites
Challenges in Hybrid and Federated Cloud Computing
External Providers
31/33

32. Cloud Federation in Grid Infrastructures
The StratusLab Architecture!
Grid Services
MarketPlace
• Federation facilities
• Security • Sharing existing VM images
• Grid specific services • Registry of metadata
• Image are kept elsewhere
• Supports trust
Appliance Repo
• Storage VM images
• Distributed
• Multi-protocol
Grid Services Cloud API Grid Services Cloud API
Cloud/Grid Site Cloud/Grid Site
Challenges in Hybrid and Federated Cloud Computing 32/33

33. Questions?
We Will Be Happy to Answer Any Question !
CloudPlan.org @imllorente
Challenges in Hybrid and Federated Cloud Computing 33/33