The document discusses integrating a security product called ElasterShield with the Cloudstack platform. It proposes developing a Cloudstack plugin to manage the integration. The plugin would leverage Cloudstack's framework to deploy and manage ElasterShield Security Virtual Appliances (ESVAs) across hypervisors. It would also interface with ElasterShield's management system to assign security profiles to virtual machines based on their lifecycle events in Cloudstack. This would provide security as a service through Cloudstack in a multi-tenant way.

1. Integrate Security Product with
Cloudstack
Written by: Mice Xia
mice_xia@tcloudcomputing.com

2. About me
• About me
– Project manager from TCloud Computing Ltd, Beijing, China
– Apache CloudStack Committer
– About TCloud
• Provides cloud platform product/service to Chinese customers
• Based on CloudStack
• About this topic
– Based on the practice and on-going work
– Goal: bring some insights into integration between CS and security
product

3. Outline
• Background
• Three categories of solutions
• Integration goal and problems
• Architecture
• Workflow
• Future work

4. Background
• CS does an excellent job for building clouds
• Growing needs for solutions to secure the Cloud
• Seek security solutions to work with CS
• Works for both private and public cloud
• Multi-tenant ,self-service,
• Easy to provision
• Resource effective
• Easy to integrate

5. Agent-based
• Widely implemented in physical Guest Network
environment
• Relies on the agent installed on Guest VM 1
guest OS Agent
• Network connection to
Pattern/Mgmt Guest VM
Server
Agent
pattern/mgmt server
• Not much IAAS level integration Guest VM
Agent
• Cons
Guest VM
• Instant-on gap Agent
• Resource contention
CS
Virtual
Router

6. Gateway-based
• Agentless, widely adopted in
physical environment
• Detect and prevent by capturing Router
network traffics Offline mode
Inline mode
• Works fine with simple
L3 Core Switch
requirements
• Cons
• Network traffic based Available Zone
• Uneasy to provide Inter-user and
inter-VM protection … … …
Pod 1 Pod N

7. Hypervisor-based
• Dedicated for virtual environment User VM
• Agentless for user VM, need an User VM
‘agent’ on hypervisor
• Capture network traffics
• Scan VM memory Module
• Can work with any complex
Hypervisor
network environment

8. Brief intro to ElasterShield
• Hypervisor based security product
• Features <VM, Profile/Rules>
• Firewall (L2, L3, L4)
• IDS/IPS
• Application Control
• Web server protection
• Anti-malware (roadmap) ElasterShield
• For IT admin, no multi-tenant
• Rules: defines what triggers event
• Profiles: a set of enabled rules Events/Alerts

9. Brief intro to ElasterShield (cont.)
• ElasterShield Manager (ESM) ESM
• Storing rules/profiles
• Provide UI/ web service API
• Send command to ESVA Hypervisor
• Security Virtual Appliance (ESVA) ESVA User VM
• One for each hypervisor, stateless
• Hypervisor redirects VM’s traffic to
ESVA
• Includes an engine to do the
Hypervisor
detection/prevention
• Bi-directional communicate ESVA User VM
between ESM and ESVA
• No impact to user VM if ESVA stops

10. Integration goal
• CS Integrates ES
• Security as a service, ES as a service provider
• Multi-tenant, defined by admins, subscribed by users,
provisioned to VMs
• Security offering == Security Profiles
• Export new CS APIs
• Generic to hypervisor-based solution (hopefully)
• No change to ES code, keep it as it is
• Minimum change to CS 4.0 existing code

11. Problems : Networking
Public Network
• Bi-directional communication is Shared Guest
Network
required between ESM and ESVA Management
• Deploy ESVA as a user VM
Network
ESM ?
– Connects to guest network Isolated Guest
– Basic Zone Network
• Ingress rules
– Advanced zone ESVA
• Virtual router NAT problem
• Another dedicated shared network?
User VM
• Solution: Via management network
– Works for both basic and adv zone User VM
– ESVA as a special VM managed by CS
– Connect it to management network
programmatically VRouter

12. Problems: ESVA management
• Make sure ESVA is running on each hypervisor
– When a new host is added, ESVA should be deployed
automatically on the host
– When hypervisor is rebooted, ESVA should be started
automatically
– Can be destroyed manually and re-deployed
• Similar to SystemVM/VRouter
– Managed by CS, transparent to end user
– Per hypervisor basis

13. Problems: VM life cycle conflicts
• User VM State mismatch
– ES can detect VM state changes by polling events directly from
Hypervisor
• Enable VM’s profile when it gets started
• Disable VM’s profile when it gets stopped
• Un-assign profile when it gets destroyed
– It does not work. CS removes VM from hypervisor when it gets
stopped
CS VM state Stopped Destroyed Removed
Hypervisor VM state Destroyed (Missing)
– Solution: CS controls profile (un)assignment to VM , based on
VM state in the CS

14. Architecture Overview
New CS API ESM API
ES Plug-in ESM
Mgmt network
CloudStack
Manager
Mgmt network
Managed by CS ESVA ESVA
User VM User VM
Hypervisor Hypervisor

15. Integrate as a CS Plug-in
• Provide security related APIs
• Treat ESM as a resource and send commands to it
• ESVA monitor and management
• Tailor User VM lifecycle with existing framework
• Tailor Hypervisor management

16. Plug-in
• New manager/service for
ESVA and ESM
• Extends several CS
Rest API
manager to tailor process
• A new resource for calling
ESMResource
ESVAManager
ESM APIs ESMManager
Plugin API
• DAOs for security related …
database operation ESMUserVMManagerImpl
Data Access Layer

17. Workflow: Add ESM
1) Call plug-in API 2) Connect to ES manager
ElasterShield Plug-in
ElasterShield
CloudStack Manager Manager
Database 3) Persist into DB
1) Admin user or external UI calls Plug-in API (addElasterShield)
• Parameters include ESM url/account/password
• One ESM per zone or per installation?
2) Plug-in calls ESM’s APIs to:
• test connection with specified authentication
• Get ESM info such like version, license, etc.
3) Plug-in persists the ESM info into CS database

18. Workflow: Enable Security protection
1 ) Call plug-in API:
1) Admin user or external UI calls plug-in enableSecurityProtection
API (enableSecurityProtection)
• Cluster wide enable
3) Activate ESVA
2) ES Plug-in deploys ESVA
• Deploy ESVA on each host of the ES Plug-in ESM
cluster
• With specified service offering CloudStack
and template Manager
• Start ESVA
2) Start ESVA on each
3) Activate ESVA hypervisor of the cluster
• Plug-in calls ESM API to activate
each ESVA
ESVA ESVA
Hypervisor Hypervisor

19. Workflow: List Security Offering
2) Call ESM API,
listSecurityProfiles
ElasterShield Plug-in
ElasterShield
1) Call plug-in API Manager
listSecurityOffering CloudStack Manager
1) User or external UI calls Plug-in API (listSecurityOffering)
• Fetch all security offerings with query parameters
• Id, name, description
2) Plug-in calls ESM’s APIs to:
• List security profiles from ESM

20. Workflow: Define Security Profiles
Admin defines
Rules/profiles
2) Call ESM API
listSecurityProfiles
ElasterShield Plug-in
ElasterShield
1) Call plug-in API Manager
listSecurityOffering CloudStack Manager
• For this Stage, this will be not implemented in plug-in, because it’s
product-specific
• Admins use ESM’s UI to define profiles
• Users use Plug-in API to list and apply profiles

21. Workflow: Apply security offering
1) Call plug-in API 3) Call ESM API,
applySecurityOffering assignSecurityProfile
ElasterShield Plug-in
ElasterShield
CloudStack Manager Manager
Database 2) Persist into DB
1) User or external UI calls plug-in API (applySecurityOffering)
• Specify VM id and security offering id
2) Plug-in persists <VM, offering> mapping into DB
• Address multi-tenant problem
3) Plug-in calls ESM API to assign corresponding profile if VM is running

22. Workflow: Start Virtual Machine
4) Call ESM API,
3) Get profile assignSecurityProfile
ElasterShield Plug-in ElasterShield
Database
Manager
CloudStack Manager
1) Call CS API 2) Start User VM
startVirtualMachine
User VM 5) ESM notifies ESVA to take effect
ESVA
Hypervisor
1) User or UI calls CS API (startVirtualMachine)
2) CS checks ESVA status and starts user VM as usual
3) After user VM is started, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStart)
4) Plug-in call ESM API to assign profile to the VM
5) ESM notifies ESVA in the same hypervisor with VM to take effect profile

23. Workflow: Stop Virtual Machine
4) Call ESM API,
3) Get profile unassignSecurityProfile
ElasterShield Plug-in ElasterShield
Database
Manager
CloudStack Manager
1) Call CS API 2) Stop user VM
stopVirtualMachine
User VM 5) ESM notifies ESVA to take effect
ESVA
Hypervisor
1) User or UI calls CS API (stopVirtualMachine)
2) CS powers off user VM as usual
3) After user VM is shut down, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStop)
4) Plug-in calls ESM API to unassign profile to the VM
5) ESM notifies ESVA in the same hypervisor with VM to take profile effect

24. Workflow: Destroy Virtual Machine
ElasterShield Plug-in
1) Call CS API Database
2) Remove from DB
destroyVirtualMachine CloudStack Manager
1) User or UI calls CS API (destroyVirtualMachine)
• VM is destroyed as usual by CS
2) When VM gets expunged, plug-in removes <VM, profile> mapping from
DB
• VirtualMachineGuru.finalizeExpunge

25. Workflow: Add a new host
5) Call ESM API to activate
ElasterShield Plug-in
ElasterShield
Manager
CloudStack Manager
1) Call CS API 2) Add host 3) Deploy ESVA
addHost
ESVA
Hypervisor
1) User or UI calls CS API (addHost)
2) CS add a new hypervisor host as usual
3) Deploy ESVA
4) Plug-in persist ESVA info

26. Workflow: Maintain a host
3) Call ESM API to de-activate ESVA
ElasterShield Plug-in ElasterShield
Manager
CloudStack Manager
1) Call CS API 2) Power off ESVA
prepareHostForMaintenance
ESVA
Hypervisor
1) User or UI calls CS API (prepareHostForMaintenance)
2) Stop ESVA on it
• update ESVA status into DB
3) Plug-in call ESM API to deactivate ESVA
• unassign security profiles to VMs that cannot be migrated

27. Workflow: Query Events/Alerts
ElasterShield Plug-in
ElasterShield
1) Call plug-in API 2) Call ESM API Manager
querySecurityEvents CloudStack Manager
1) User or external UI calls plug-in API (querySecurityEvents)
• With query parameters such as time, VM id, user id, etc
2) Plug-in calls ESM’s API to fetch events/alerts

28. Workflow: ESVA upgrading
2) Call ESM API to upgrade
ElasterShield Plug-in ElasterShield
Manager
1) Call plug-in API CloudStack Manager
upgradeESVA
3) Notify ESVA
4) Download upgrade package
Web server ESVA
Hypervisor
1) Call plug-in API to upgrade ESVA , with a URL parameter
2) Plug-in forward the API to ESM
3) ESM notifies all ESVA to upgrade
4) ESVA downloads upgrading package from the URL specified
5) ESVA upgrade itself, it still functions during the upgrading.

29. Summary & Future work
• Summary
– Develop plug-in with existing CS framework
– Leverage CS to manage security virtual appliance
– Tailor process
• Future work
– Make the security offering more generic
– Make it easier for CS to introduce a new system
VM