Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Symantec SDN Deployment
Jasmeet Sidhu, Rudrajit Tapadar
Cloud Platform Engineering
Class of Service
Copyright © 2015 Symantec Corporation 2
Class of Service
• Dev
– For developers to get familiar with OpenStack cloud
– Each developer has a project
• Production
–...
Self-Service User Onboarding
Copyright © 2015 Symantec Corporation 4
Self-Service User Onboarding
• Zero tickets for user onboarding
– Provide sign up capabilities on Horizon
• Provide easy n...
Load Balancer as a Service
Copyright © 2015 Symantec Corporation 6
Load Balancer as a Service
•Out of the box
– Icehouse, v1 APIs
– Launch HA Proxy service instances on a single AZ
– SSL Su...
Baremetal on Overlay
Copyright © 2015 Symantec Corporation 8
Baremetal on Overlay
•Applications that run on baremetal but needs to be on the
overlay
– Example: swift proxy and data no...
Availability::Control Plane
Copyright © 2015 Symantec Corporation 10
Control Plane Availability
• Goal - 99.95% Availability
• 5 SDN controller VMs distributed over 3 racks
• 5 Cassandra data...
Failed Customer Interactions
Copyright © 2015 Symantec Corporation 12
Failed Customer Interactions
• Measure the control plane availability
• Use Symantec’s Logging-Monitoring-Metering as a Se...
Availability::Data Plane
Copyright © 2015 Symantec Corporation 14
Data Plane Availability
• Work in progress..
–FIP Availability
–vDNS
–Link Local
–Private Network
Copyright © 2015 Symante...
Seamless Upgrades
Copyright © 2015 Symantec Corporation 16
Upgrade 1.20 to 2.0.1
• Goal - Zero Downtime
• Controller upgrades
– No in-place upgrades
– Build a parallel control plane...
Health Monitoring
Copyright © 2015 Symantec Corporation 18
Health Monitoring
• Volta
–Logging
•Logstash
•Elasticsearch
–Metrics
•InfluxDB
•Statsd
•Collectd)
–RESTful APIs make it ea...
Troubleshooting
Copyright © 2015 Symantec Corporation 20
Troubleshooting
• Most incidents are trivial
– Known issues
– Trivial fixes/workarounds
• Some incidents are complex
– RCA...
Thank you!
Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or re...
Upcoming SlideShare
Loading in …5
×

Symantec SDN Deployment

OpenContrail User Group Meetup, OpenStack Summit, Vancouver 2015.

  • Be the first to comment

Symantec SDN Deployment

  1. 1. Symantec SDN Deployment Jasmeet Sidhu, Rudrajit Tapadar Cloud Platform Engineering
  2. 2. Class of Service Copyright © 2015 Symantec Corporation 2
  3. 3. Class of Service • Dev – For developers to get familiar with OpenStack cloud – Each developer has a project • Production – For teams to onboard their members – Each team has a project – Manage user roles – Manage production workloads Copyright © 2015 Symantec Corporation 3
  4. 4. Self-Service User Onboarding Copyright © 2015 Symantec Corporation 4
  5. 5. Self-Service User Onboarding • Zero tickets for user onboarding – Provide sign up capabilities on Horizon • Provide easy networking on Dev CoS – Hide all complexities – Automatically create network – Allocate routable subnets by using Contrail VNC APIs – Create security group with proper rules – Create unique domain names for instances by using Designate for routable IPs Copyright © 2015 Symantec Corporation 5
  6. 6. Load Balancer as a Service Copyright © 2015 Symantec Corporation 6
  7. 7. Load Balancer as a Service •Out of the box – Icehouse, v1 APIs – Launch HA Proxy service instances on a single AZ – SSL Support: Wildcard cert • Symantec fixes –Multiple AZ, SSL Passthrough, Stats and Metrics • Performance: –~6.5 Gbps throughput with 10K parallel connections, VIP with 2 members –20K HTTPS requests/sec for 10K parallel connections with 1 million requests, 1K response size • Tuning - haproxy.cfg: maxconn 50K, nbproc 4, ulimit-n 200K, Cipher • Pain points –No control over ha proxy cfg –No control over resource allocations (cpu, etc) Copyright © 2015 Symantec Corporation 7
  8. 8. Baremetal on Overlay Copyright © 2015 Symantec Corporation 8
  9. 9. Baremetal on Overlay •Applications that run on baremetal but needs to be on the overlay – Example: swift proxy and data nodes – Launch them inside network namespaces – Plug them to the vRouter – East-West Traffic • Manual Setup via scripts – Nova is not aware but Contrail is. – Multiple nics sitting on multiple networks – Static IPs Copyright © 2015 Symantec Corporation 9
  10. 10. Availability::Control Plane Copyright © 2015 Symantec Corporation 10
  11. 11. Control Plane Availability • Goal - 99.95% Availability • 5 SDN controller VMs distributed over 3 racks • 5 Cassandra database baremetal nodes distributed over 3 racks – RF of 3 for analytics – RF of 5 for config – Compaction throughput 256 Mbps • Deployment Automation: Puppet • Issues seen: DB Timeouts, Version mismatch, admin token Copyright © 2015 Symantec Corporation 11
  12. 12. Failed Customer Interactions Copyright © 2015 Symantec Corporation 12
  13. 13. Failed Customer Interactions • Measure the control plane availability • Use Symantec’s Logging-Monitoring-Metering as a Service to parse Neutron logs • Compare response codes: 5XX counted as failures • Dashboards! Copyright © 2015 Symantec Corporation 13
  14. 14. Availability::Data Plane Copyright © 2015 Symantec Corporation 14
  15. 15. Data Plane Availability • Work in progress.. –FIP Availability –vDNS –Link Local –Private Network Copyright © 2015 Symantec Corporation 15
  16. 16. Seamless Upgrades Copyright © 2015 Symantec Corporation 16
  17. 17. Upgrade 1.20 to 2.0.1 • Goal - Zero Downtime • Controller upgrades – No in-place upgrades – Build a parallel control plane with new release – Add them to the VIP pool and gradually decommission old controllers • Database upgrades – Add new DB nodes one by one to the existing cluster – Repair the DB – Decommission old DB node one by one • Compute upgrades – Automate unloading and loading of kernel module in all computes Copyright © 2015 Symantec Corporation 17
  18. 18. Health Monitoring Copyright © 2015 Symantec Corporation 18
  19. 19. Health Monitoring • Volta –Logging •Logstash •Elasticsearch –Metrics •InfluxDB •Statsd •Collectd) –RESTful APIs make it easy: •Response Codes, Bytes Transfered, Time, Verb, etc. • OpsView / Zabbix Copyright © 2015 Symantec Corporation 19
  20. 20. Troubleshooting Copyright © 2015 Symantec Corporation 20
  21. 21. Troubleshooting • Most incidents are trivial – Known issues – Trivial fixes/workarounds • Some incidents are complex – RCA is very involved – Might have to wait for next code release for a fix – Quick and dirty solution – use auto healing scripts for workarounds •Periodically check system health (Synthetic Transactions) •Remediate known bugs •Fix problems as they are detected, Save pagers, run 24x7! (MX Encapsulation, Dead processes, etc.) Copyright © 2015 Symantec Corporation 21
  22. 22. Thank you! Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.

×