OpenStack Framework Introduction

OpenStack Introduction

Presenter: Jason, Tsung-Cheng, HOU
Advisor: Wanjiun Liao
June 7th, 2012 1

Motivation
• What does a cloud OS look like?
• How are they building IaaS Platform?
• What are current industry trend?
• How will the cloud system press the network?
• OpenStack
– Founded by NASA and Rackspace in 2010
– Currently 178 companies and 3386 people
– Growing fast now, latest release Essex, Apr. 5th

2

OpenStack Status
• OpenStack
– Founded by NASA and Rackspace in 2010
– Currently 178 companies and 3386 people
– Was only 125 and 1500 in fall, 2011.
– Growing fast now, latest release Essex, Apr. 5th
• Aligned release cycle with Ubuntu, Apr. / Oct.
• Aim to be the “Linux” in cloud computing sys.
• Open-source v.s. Amazon and vmware
• Start-ups are happening around OpenStack
• Still lacks big use cases and implementation
3

Agenda
• OpenStack Brief Overview
• Some Reviews of Cloud Technology
• “Keystone” Identity
• “Swift” Storage
• “Glance” Image
• “Nova” Compute
• “Quantum” Networking

5

Enterprises are building clouds to...

1. Virtualization
Server Virtualization 2. Cloud Data Center 3. Cloud Federation

Datacenters are being virtualized, Servers are first
Hypervisors provide abstraction between SW and HW (Servers)
HOST 1 HOST 2 HOST 3 HOST 4, ETC.

VMs

Hypervisor:
Turns 1 server into many “virtual machines” (instances or VMs)

Next: Storage, Network…the building blocks

1. Server Virtualization
Virtualization 2. Cloud Data Center 3. Cloud Federation

But questions arise as the environment grows...

APPS USERS ADMINS

+


Solution: OpenStack, The Cloud Operating System
A new management layer that adds automation and control

APPS USERS ADMINS

CLOUD OPERATING SYSTEM


What‟s next?

’


Common Platform


A common platform is here.
OpenStack is open source software powering public and private clouds.

Private Cloud: Public Cloud:

OpenStack enables cloud federation
Connecting clouds to create global resource pools
Washington
Common software
platform making
Federation
possible

Texas California Europe


In Summary, the Cloud Operating System
enables enterprises to:

Top 3 Benefits of a Common Platform

Core Components in Essex
• Release Apr. 5th, 2012

• Dashboard: Access and control portal for admin and users, also web-based
• Identity: Unified authentication across whole system
• Object Storage: Large-scale redundant storage of static objects, not a file system
• Image Service: Store, retrieve, discover, register, and deliver VM images
• Compute: Large-scale deployment of automatically provisions VMs and related SWs
14

OpenStack Compute Key Features

1. REST-based API

2. Horizontally and massively
scalable

3. Hardware agnostic: supports a variety of
standard hardware

4. Hypervisor agnostic: support for Xen
,Citrix XenServer, Microsoft Hyper-V,
KVM, UML, LXC and ESX

OpenStack Storage Key Features

1. REST-based API

2. Data distributed evenly
throughout system

4. Scalable to
3. Runs on standard hardware multiple petabytes,
billions of objects
5. No central database
required
6. Account/Container/Object structure
(not file system, no nesting) plus
Replication (N copies of accounts,
containers, objects)

OpenStack Image Service Key Features

1. Store & retrieve VM
2. REST-based API
images

3. Compatible with all common
image formats
4. Storage agnostic: Store images
locally, or use OpenStack Object
Storage, HTTP, or S3

Agenda

24

Chief Reference
• CIS 607: Seminar in Cloud Computing,
Spring 2012, by Dr. Allen D. Malony
• "Applied Computational Instrument for
Scientific Synthesis" (ACISS)
• University of Oregon (UO), groups may
configure their own VM images and carry out
research work as if had dedicated clusters.
• This course delivers hands-on operations on
ACISS and cloud computing knowledge to
students.
25

VM and Hypervisor
• Virtual Machine: A software package,
sometimes using hardware acceleration, that
allows an isolated guest operating system to
run within a host operating system.
• Stateless: Once shut down, all HW states
disappear.
• Hypervisor: A software platform that is
responsible for creating, running, and
destroying multiple virtual machines.
• OpenStack is hypervisor agnostic.
• Type I and Type II hypervisor
26

Bridged Networking
• One network card acts as
many devices.
• Host does not need an IP
address.
• Hypervisor sets virtual
MAC address for guest
machine.
• ACISS uses bridges,
along with Virtual Local
Area Networks (VLANs)
to segment traffic and
assign network
addresses.
28

Network Block Storage
• Network Block Storage: Make data persistent
by mounting a network block storage device.
• NFS Mounts: Many machines may access
simultaneously. Limited permissions.
• iSCSI Mounts. Only one machine may access
at any given time. Unlimited permissions.

29

Object Storage
• Persistent storage of objects on a network.
• Generally “write once, read many.”
• Durable storage with redundant copies
• Access Control Lists determine visibility for
owner and authorized users.
• Amazon‟s S3 is an example of this.
• ACISS uses OpenStack Swift.
• Swift uses same API as S3.

30

Virtual Machine Images
• Disk images that can be booted on a virtual
machine by a hypervisor.
• Can be a single image that contains boot
loader, kernel and operating system.
• Boot loader and kernel can be separated.
• Allows for custom kernels and resizable
images.

31

Image Service
• Stores and catalogs virtual machine images.
– Keep track of VMs, trace and recover.
• Provides for discovery, registration, and
delivery of images to hypervisors.
• Allows for many image formats and for linking
of loaders and kernels to images.
– There may be different types of virtualization
technologies, different kernels, etc.
• Usually built on object storage systems.
• Glance on Swift.
32

Cloud Computing
• The course defines in the following way:
– The orchestration of hypervisors, networking,
block storage, and image, and identity services to
provide on demand virtual machines.
• Hence, meeting required characteristics of
cloud computing.
– On-demand self-service
– Resource pooling
– Rapid elasticity
–…
33

Agenda

34

Keystone Main Functions
• Provides 4 primary services:
– Identity: User information authentication
– Token: After logged in, replace account-password
– Service catalog: Service units registered
– Policies: Enforces different user levels
• Can be backed by different databases.
– LDAP
– SQL
– Key Value Stores (KVS)

35

Keystone: Identity
• User information:
– username/password
– Metadata (e-mail, etc.)
– Tenant - organizes users into projects or group.
– Role - define a user‟s role and permissions in a
project.
• A user must belong to at least one tenant,
and may belong to many tenants
• Roles are assigned to user/tenant pairs
– Common roles: Member, Admin
36

Keystone: Token
• Once a user‟s identity has been veriﬁed with
a acc/pswd pair, a short-lived (24 hr) token is
issued.
• Tokens are a stand-in for the acc/pswd.
• OpenStack services hold on to tokens and
use them to query keystone during operations.
• For example, once Nova can use a token to
determine if an authenticated user has
authorization to delete an instance.

37

Keystone: Catalog
• OpenStack service endpoints are registered
with Keystone to create a service catalog.
• A client for a service connects to Keystone,
and determines an endpoint to call based on
the returned catalog.
• Behind the scenes, services can be moved to
different endpoints. A client can find online
services by querying Keystone endpoint.
• Also allows for service load distribution with
multiple endpoints to a single service.
38

Keystone: Catalog
• Every catalog entry has ﬁve elements:
– region: the name given to a collection of cloud
services
– service id: the service the endpoint is associated
with (Glance, Nova, Swift, Keystone)
– public url: the public facing endpoint for the
service
– internal url: the internal facing endpoint. Usually
the same as the public url
– admin url: the endpoint for service administration

39

Agenda

42

Swift
• Object storage, objects “live” on an endpoint.
– An endpoint could be any storage device
• Every object belongs to a user/account pair.
– keystone tenant : swift account
– keystone user : swift user
– keystone role : swift group
• Proxy, Ring, and Workers
• Account, Container, Object

43

Swift: Proxy Server
• Handles incoming requests via the
OpenStack Object API or raw HTTP.
• Accepts files to upload, modifications to
metadata or container creation.
• Serve files or container listing to web
browsers.
• Several types of Ring files
• May utilize an optional cache to improve
performance.

44

Swift: Workers
• Keep a distributed database of replicated
objects.
• Workers are divided into reliability zones.
• Copies of data are distributed across multiple
zones.
• There are many types of workers:
– Account server, container server, object server
– Housekeeping: Replication, updater, auditor

45

Swift: Ring
• Maps names to entities and locations
– Stores data based on zones, devices, partitions, and replicas

• There are three types of items:
– Account, container, object
• The locations are determined by a ring file
• Worker IP addresses are loaded into a ring
builder.
• Storage ids and locations are computed using
a hashing algorithm to evenly distribute items
across the workers.
47

Swift: Ring
• Account and container storage id has a
database, storing object metadata.
• Proxy makes distributed searches across the
databases for item requests.
• The ring builder can add / remove nodes, and
rebalance distribution of files across servers.

48

Duplicated storage, load balancing

↑ Logical view

↓Physical arrangement

← Stores real objects

←Stores object metadata

↑Stores container / object
metadata

Workers can be a account server, a
container server, or an object server

# of account < # of container < # of object servers

Different zones ↑

Agenda

56

Glance
• Image storage and indexing.
• Keeps a database of metadata associated
with an image, discover, register, and retrieve.
• Built on top of Swift, images store in Swift
• Two servers:
– Glance-api: public interface for uploading and
managing images.
– Glance-registry: private interface to metadata
database
• Support multiple image formats
57

Agenda

62

Nova
• Major components:
– API: public facing interface
– Message Queue: Broker to handle interactions
between services, currently based on RabbitMQ
– Scheduler: coordinates all services, determines
placement of new resources requested
– Compute Worker: hosts VMs, controls hypervisor
and VMs when receives cmds on Msg Queue
– Volume: manages permanent storage

63

Nova
• Major components:
– Network: manages networking
• Was originally a component in Nova
• Default gateway, network controller
• DHCP server, address mgmt
• The network part in Nova will be enhanced by the
project named “Quantum”, to be released.
• Will introduce Quantum later.

64

Nova Messaging and Data
• Messaging is managed through RabbitMQ
– Server that allows messages to be posted to channels.
– Subscribers to channels receive messages.
– Services regularly announce availability.
– Scheduler regularly reads for availability.
– Scheduler makes requests to services.
• Persistent data stored in a database.
– VM metadata, network topology, volume metadata, known
services

65

Messaging (RabbitMQ)
• Get data from point A to point B
• Decouple publishers and consumers
• Queueing for later delivery
• Load balancing and scalability
• RabbitMQ is an AMQP messaging broker
• Advanced Message Queueing Protocol
• Network wire-level protocol
• Internet protocol - like HTTP, TCP - but
ASYNCHRONOUS

66


67


68

Agenda

75

Without Quantum
• Originally, Nova handles all networking by:
– Linux bridge networking
– Virtual interfaces connecting network through the
physical interface
– Assigns VM IP address
– Fixed IP: Returns when VM shuts down
– Floating IP: Can be reassigned online
• Network Manager provides VN to enable
compute servers to interact with each other
and the public network
• A Blog states currently 90% Nova bugs are network related
76

Original Network Manager
• Each VM network owned by one network host
– Simply a Linux running Nova-network daemon
• Nova Network node is the only gateway
• Flat Network Manager:
– Linux networking bridge forms a subnet
– All instances attached same bridge
– Manually Configure server, controller, and IP
• Flat DHCP Network Manager:
– Add DHCP server along same bridge
• Later: VLAN Network Manager
77

Bridged Networking
• One network card acts as
many devices.
• Host does not need an IP
address.
• Hypervisor sets virtual
MAC address for guest
machine.
• ACISS uses bridges,
along with Virtual Local
Area Networks (VLANs)
to segment traffic and
assign network
addresses.
78

Linux running Nova-network daemon
Network host will act as the
gateway for all the NICs bridged
into that network.
VMs bridged in to a raw Ethernet
device

The only gateway
With security measures

79

• DHCP server also tracks IP
leases and releases
• Re-uses and assigns IP
addresses dynamically
• Sets up a routing table for
outside forwarding

• Compute optionally have
public IP
• Network host is a single point
of failure and bottleneck
• Backup network host
• A new proposed model → →
• Multi-NIC→Multiple networks
80

VLAN Network Manager
• Current default mode for OpenStack
• Nova creates a VLAN and bridge for each project.
– Requires switches with VLAN tagging (IEEE 802.1Q).
– A range of private IPs, only accessible inside VLAN.
• A special VPN instance (code named cloudpipe)
needs to be created. Generates (certificate, key) for
users to access VPN automatically.
• Provides a private network segment for each project,
accessed via dedicated VPN connection from the
Internet. Each project with own VLAN, Linux
networking bridge, and DHCP server.
82

Plugin
• The component where the „virtual networking‟ magic
happens. Fulfills API contract by implementing the
„Plugin Interface‟
• Tenants expect same behavior from Quantum API
regardless of the particular plugin employed
• Available Quantum Plugins:
– Open vSwitch: Builds isolated networks with OVS and L2-in-L3 tunnels.
– Cisco UCS: Isolation based on VLAN and net-profiles applied to Cisco UCS
– converged network adapters
– Linux Bridge: Build isolated networks with VLAN interfaces and linux bridge
– NTT-Data Ryu: Acts as a proxy for the NTT Ryu platform
– Nicira NVP: Acts as a proxy for the Nicira NVP platform

88

The Quantum Manager
• Nova‟s network manager for Quantum. Forwards
network related requests.
• Also, provides other network services such as IP
address management, DHCP, NAT, Floating IPs…
• Virtual Networking: A label nowadays applied to too
many solutions and products.
– Securely partitioning the network
– Defining virtual network topologies
– Automating network provisioning

89

The Near Future
• Folsom release, Fall 2012
– Become a core OpenStack project
– Merge with IP Address Management service
– Improve API quality and documentation
– Improve GUI, i.e. Quantum Horizon plugin
– Possible more plugins, Build more network
services on top of the basic building block
• Each service with its own tenant-facing API
• IP routing, Distributed Firewall, LB, NAT, VPN,
bridging…
• Quantum is NOT SDN, but in theory can transform anything into SDN.
93

Reference
• OpenStack Documentation
http://docs.openstack.org/
• Dr. Allen D. Malony, CIS 607: Seminar in Cloud Computing, Spring 2012, U. Oregon
http://prodigal.nic.uoregon.edu/~hoge/cis607/
• Bret Piatt, OpenStack Overview, OpenStack Tutorial
http://salsahpc.indiana.edu/CloudCom2010/slides/PDF/tutorials/OpenStackTutorialIEEECloudCom.pdf
http://www.omg.org/news/meetings/tc/ca-10/special-events/pdf/5-3_Piatt.pdf
• Vishvananda Ishaya, Networking in Nova
http://unchainyourbrain.com/openstack/13-networking-in-nova
• Sandy Walsh, OpenStack 101 Technical Overview
http://www.slideshare.net/openstackcommgr/openstack-101-technical-overview
• Jaesuk Ahn, OpenStack, XenSummit Asia
http://www.slideshare.net/ckpeter/openstack-at-xen-summit-asia
http://www.slideshare.net/xen_com_mgr/2-xs-asia11kahnopenstack
• Salvatore Orlando, Quantum: Virtual Networks for Openstack
http://qconlondon.com/dl/qcon-london-
2012/slides/SalvatoreOrlando_QuantumVirtualNetworksForOpenStackClouds.pdf
• Dan Wendlandt, Openstack Quantum: Virtual Networks for OpenStack
http://www.ovirt.org/wp-content/uploads/2011/11/Quantum_Ovirt_discussion.pdf
• Daneyon Hansen, OpenStack @ CISCO
http://www.cisco.com/web/strategy/docs/gov/openstack_presentation.pdf
• Rick Clark, Cisco and OpenStack
http://www.ogf.org/OGF32/materials/2310/ogf32-isod-Cisco-OpenStack-July2011.pdf
94

OpenStack Framework Introduction

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to OpenStack Framework Introduction

Similar to OpenStack Framework Introduction (20)

More from Jason TC HOU (侯宗成)

More from Jason TC HOU (侯宗成) (9)

Recently uploaded

Recently uploaded (20)

OpenStack Framework Introduction