CloudStack has been often critiqued because of its architecture. This session is to talk about the efforts that have been done and are in progress to move CloudStack's architecture forward. How we can achieve the simplicity of a monolithic system, but scalability of a distributed architecture. How we can keep the speed and reliability of Java, but leverage scripting languages for easier integrations. How can we get CloudStack to be the best and easiest system to deploy and maintain for clouds from 10s to 100s of thousands of hypervisors.

1. Future of Apache CloudStack: modular,
distributed, and hackable
Darren Shepherd
Citrix
CCCEU 2013

2. About Me
●
Apache CloudStack Committer
●
Previously - Cloud Architect @ GoDaddy.com
–
Built three generations of clouds
●
●
●
2nd – Custom “CloudStack Compatible”
●
–
1st – CloudStack 2.x, custom storage framework
3rd – 100% from scratch
Been developing IaaS orchestration systems full time for the last 4
years
Currently – Do something @ Citrix
–
Focus on open source
–
I care about architecture and design

3. What's This About
●
My opinions
●
What I think we should focus on architecturally
●
What I'm trying to accomplish

4. General Comments
●
Do more with less
–
Identify patterns in code and create frameworks
●
–
●
For example, make Abstract base classes
Copy and paste is bad
Need to leverage more open source frameworks
–
jOOQ, Spring TX, Netflix Archaius/Servo, Jackson, Apache jclouds,
Flyway/Liquibase, Apache Commons
●
Past/Current – ACS adds cloudiness to legacy technologies
●
Current/Future – ACS orchestrates cloudy technologies
●
Always provide a way for Apache CloudStack to do nothing and
delegate to something else

5. Modular – Why?
●
Enforce design and architectural boundaries
●
Better pluggability
●
Quality/Testing
●
Lays the foundation for distributed architecture

6. Modular – What's been done
●
4.3 - Spring Modularization
–
META-INF/cloudstack/mycomponent/module.properties
–
META-INF/cloudstack/mycomponent/*context.xml
–
Over 50 Modules

7. Stats
●
~400k lines of Java (not 1.2m)
–
–
32% - engine, framework, services, usage, util
–
●
42% - agent, api, core, server
26% - plugins
We still have big blobs of code
–
74% “Core Code”
“Core Code” is not really core. Includes VR, CP,
SSVM, XenServer, others

8. What needs to be done
●
Package plug-ins in separate RPM/DEBs
●
Break out more code from /server/src
●
Break apart *ManagerImpl classes
–
●
Implement too many interfaces that belong in
different architectural layers
Modularize VR and VPC code

9. Distributed Architecture
●
Maybe you don't care
–
Many don't – Monolithic deployment is simple and
scalable enough for most use case
●
Support multiple deployment models
●
Scalability
–
●
Scale tiers according to load characteristics
Reliability
–
Put certain components closer to resources

10. Monolithic
Single JVM
Redundant JVM

11. Fully Distributed
●
System Services
–
–
●
Event Bus
Lock Manager
Servers
–
API
–
Back end Jobs
–
Host Manager

12. How
●
Event Bus
–
Hazelcast or Redis
–
Do not use “reliable messaging”
●
●
Lock Manager
–
●
Works more like UDP multicast
Hazelcast or ZooKeeper
Same binary on all servers
–
“Profiles” used to determine which Spring modules are
activated on start

13. Hackability
●
Should be relatively simple to add one off
functionality
–
●
●
Every cloud deployment is slightly different
Need to stay relevant with upcoming
technologies
Want people to integrate first with ACS because
–
It works
–
It's easy

14. Totally scientific observation
●
Two types of sys admins
–
Those who hate Java for no good reason
–
Those who hate Java for a good reason
Those that
will tolerate developing
in Java
Those who
love systems
(admins/engineers)

15. Dynamic languages can be good
Those that
Those who
will tolerate developing
love systems
in a dynamic
(admins/engineers)
language

16. Hackability
●
Move systems related code to python
●
No Java on the hypervisor!
●
Need a new ServerResource architecture that
delegates to agents in any language
–
Work for Hyper-V is a step in the right direction
●
Decouple VR from HypervisorResource
●
Community help needed
–
Good work, GCE API and hopefully new AWS API

17. Questions?