This presentation is from the Webinar that QualiTest Group & QualiSystems hosted. The topic is how to ensure SDN/NFV does not break your network. To read more about QualiTest's SDN/NFV Testing services click here - http://bit.ly/1DkOdmt

1. How to ensure SDN/NFV doesn't
break your network?
United States +1 (646) 558-2111
United Kingdom +44 (0) 330 221 9914
Access Code: 674-242-804

2. Bryon Lowen
Vice-President Telco at
QualiTest
Alex Henthorn-Iwane
Vice-President Marketing at
QualiSystems
Webinar Presenters

3. Agenda
QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Summary
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

4. QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Summary
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

5. QualiTest & QualiSystems partnership
•Pure Play Testing focus
•Passion for testing
•World’s 2nd largest
•Telco expertise
•Automation software
•Cloud management
•Test & CI automation
•Carrier DevOps

6. QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Summary
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

7. • End-to-end services are already often quite complex
o Many service scenarios won’t change except for the presence of VNFs
o Consider topology below for end-to-end (E2E) network service (NS) including
IMS-core enabled voice services in Cable MSO
What is a Service?
CMTS
eDVA
PCMM
S_CSCF
I-CSCF
P-CSCF
MRFC/MRFP
HSS TAS
VM
CAS
CCF OSS
BGCF/
ESCF
MGCF STP
SRP
CMS
SBC
SIP Peer
Networks
PSTN
TNS

8. • Endpoints aren’t just handsets, home gateways, fixed CPE
o Clusters of VMs
o VMWare, OpenStack/KVM
• New WAN topology constructs/protocols are emerging
o Federation of controllers. OpenFlow? Network Virtualization Overlays? PCEP? Segment Routing?
What is a Service? How Cloud/NFV Changes Things
Cloud DC1 Cloud DC2
Data Center Fabric
Storage
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
Data Center Fabric
Storage
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
MPLS/SDN
Core

9. • Performance of VNFs must be characterized and certified on hypervisor/server hardware
• North-south and east-west dimensions to performance testing
o How does traffic perform when going just through the hypervisor?
o How about when it exits a physical port and goes to a PNF device?
What are New Performance Testing Challenges
IaaS/PaaS/SaaS IaaS/PaaS/SaaS
Data Center Fabric
Storage
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
Data Center Fabric
Storage
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration

10. When & Why: SDN/NFV DevTest Speed & Scale Implications
Network as Utility
• Waterfall timeframes
• Long certification cycles
• Manual, little automation OTT NFV SDN OSS/BSS
Dev/Test Cycle
App Lifecycle is Agile
• Agile timeframes
• Implies automation
VS

11. Service chain permutations could be endless
o Design permutation of vendor solution when defining the architecture (interoperability)
o Operational permutation at the time of the service instantiation (service differentiation)
Agility = IT rate of change x 100
o Performance and scale testing
How do you qualify and certify these service in an agile fashion?
When & Why: NFV DevTest Scale Implications
Admins make 500
updates / day
peak
APIs support
10,000’s updates /
day peak
Cloud Service
no admin - dynamic & automated
Today
Operations staff are involved

12. QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Q&A
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

13. Building SDN / NFV assurance process
• Reliable Service
• Right Price
• Good SLAs
• Increasingly Complex
• Renegotiate contracts
• Not affect QOS
Build strong assurance process to capitalize SDN / NFV potential

14. QualiTest NFV validation process
• QualiTest aims to create a unified end-to-end approach to service
assurance and network performance
NFV
Management
Orchestration Infrastructure Migration

15. Assure full
compliance to ESTI
ISG Standards.
Assure full compliance
to NFV requirements
for any network
equipment provider
and streamline the
ability of vendors and
users to be able to
implement compatible
NFV solutions
Build a comprehensive Test
Environment, including
Control plane and Data plane
workload simulation tools,
network analytics, QoS and
QoE monitoring probes
Test management
and Orchestration
prior to VNF
migration,
including each
component (e.g.
Orchestrator,
Vnfm, VIM)
Test the VNF
architecture and
implementation to
assure smooth
transition
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
QualiTest NFV validation process

16. Assure full
compliance to ESTI
ISG Standards.
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
(NFV-Co) ETSI NFV Compliance
ETSI ISG Standards Test procedure
Assuring compliance aims to find the
deviations from standards
• MANO
• VNF/NE
• Analyse ISG current standards
• Prepare compliance requirements
• Analyse each NFV component / int
• Define gap points and analyse
• Repeat steps 1-6 for each VNF and NE

17. Assure full compliance
to NFV requirements
for any network
equipment provider
and streamline the
ability of vendors and
users to be able to
implement compatible
NFV solutions
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
(NFV-Ven) Compliance for Vendors
MANO VNF
• VNF Descriptor format
• VNF redundancy model
• VNF state transitions
• VNF Package
• Allocation of addresses
• Scaling event types and format
• Maintain records on VNF
Packages
• VNF design patterns
• VNF update and upgrade
• VNF properties
• VNF Topological Characteristics
• VNF States and Transitions
• VNF fault management

18. Build a comprehensive Test
Environment, including
Control plane and Data plane
workload simulation tools,
network analytics, QoS and
QoE monitoring probes
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
(NFV-TE) NFV Test Environment

19. (NFV-TE) NFV Test Environment
NFV provides great flexibility:
• Build parallel test environments
• Scale up and down compute resources
• Copy any existing environment
Environment build guidelines:
• Management and Orchestration should be
similar to the production
• NS and VNF Instantiation should be similar to
production environment
• Same level of resources should be allocated.
Test Appliances:
• Simulate workload traffic on the client / server
• Simulate both data plane & control plane traffic
• Measure key metrics data & control plane
Appliances vs. Physical Test Appliances:
• Virtual test appliances are more cost effective
• Physical test appliances are recommended for
highest levels of data-plane performance or
microsecond-level timing accuracy

20. Test management
and Orchestration
prior to VNF
migration,
including each
component (e.g.
Orchestrator,
Vnfm, VIM)
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
(MANO-T) MANO Testing

21. (MANO-T) MANO Testing
The first step is to test NFV Management and Orchestration
Architectural Framework
NFV Orchestrator
VNF Manager
(VIM) and NFV
Infrastructure (NFVI)
VIM– VNF Manager
(Vi-Vnfm)
Orchestrator – VIM
(Or-Vi)
The VNF is deployed as one or more Virtual
Machines; source: ETSI GS NFV 002 v1.2.1

22. Test the VNF
architecture and
implementation to
assure smooth
transition
ESTI NFV ISG
Compliance
(NFV-Co)
NFV Compliance
for Vendors (NFV-
Ven)
NFV Test
Environment
(NFV-TE)
MANO
Testing
(MANO-T)
VNF Migration
(VNFM-T)
(VNFM-T) VNF Migration
Static testing VNF-Mano Integration
Conformance testing VNF Internal functionality
Scalability/Performance End-to-end/system test

23. QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Summary
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

24. Goal is Continuous Processes: Continuous Integation/Delivery
Automated BuildCode Commit Automated Unit Test
Automated Sanity TestAutomated & Scaled Full Regression TestAutomated Test Results
Collection
Automated BI Report Generation
Automated Build, Deploy, Test
Auto-scaled regression tests
Applied to service & network
changes

25. QualiSystems Summarized
Processes Infrastructure Test
Page 25
Automation Platform
• Self-Service System
• Infrastructure Modelling
• Reservation / Scheduling
• Provisioning Automation
• Reporting / Utilization
• Test Automation
• Test Results Reporting
• CI Integration

26. Step 1: Automate all Infrastructure
Physical Labs
MaaS
Virtual Labs
IaaS & PaaS
• Build virtual labs for VNF-only scenarios
• Consolidate or federate virtual & physical labs into private/hybrid clouds

27. CloudShell Resource Management
• Any resource: virtual,
physical, logical (IP prefix,
software license)
• User-defined resource
tagging to easily find the type
of device you need
• Eliminates manual searching
• Customizable resource
abstractions enable efficient
resource pooling
All resources managed in a single repository

28. Ensure Orchestration can Handle Network Topologies
=
Carriers adopting infrastructure orchestration to build DevOps have found
that most cloud management platforms only support compute concepts
CloudShell orchestration is capable of handling E2E NS topologies

29. Replace static diagrams with a live workspace
Live Topology
• Drag & drop devices
• Draw required
connections and activate
• Powerful search
capabilities based on
device attributes
• Abstract topologies
• Power on/off devices by
controlling PDUs

30. Step 2: Create Lab as a Service Cloud
• CloudShell allows you to
create self-service access to
full network environments
built and provisioned on
demand
• Make available to internal and
external users:
• Including vendors,
contractors
Key goals:
• Increase efficiency utilization
of test lab resources
• Reduce ratio of setup to test
time
Select
environment
Verify
availability
Reserve
Setup
Testing
activities
Collaborate
Teardown

31. Reservation & Scheduling
• Plan ahead lab operations
• Automatically resolve
allocation conflicts
• Guarantees availability of
devices for tests
• Only lock up resources
when they are needed for
testing
• Promotes efficient sharing
Improved sharing through reservation &
scheduling

32. • “One-Click” is first step toward
continuous
• Self-service users can easily launch entire
certification from UI
• Integrate with ALM to ensure consistent
reporting
• Certification can be reused for
many stages:
• New technology and early stage service proof
of concept testing
• New service introduction
• Vendor interoperability testing
• Contractor code qualification
Step 3: Create “One-Click” Certification

33. Tight Integration between TestShell & CloudShell
Create an
Environment
Develop a
Test
Create the
Automation
Suite
Execute and
Monitor the
Suite
Develop test in TestShell Studio in conjunction
with a live CloudShell environment for greater accuracy,
easier debugging
Suite = reservations & associated tests

34. What Happened when a Cable MSO Built an “As-a-Service” Approach
Overall: Complete change in the way devtest team conducted
business
Measured:
• 15% annual operations cost savings
• 20% immediate reduction in certification cycles
• 33% immediate increased in certification coverage and improving
• 100% increase in resource utilization due to 24/7 service availability
Ongoing observations:
• Earlier certification by internal departments and vendors moves quality
to the left in the cycle—still measuring outcomes
• Eliminated need for extensive travel time and costs on part of vendors
and contractors
• Increased compliance with their methodology and standards via
automation
• Increased pace of innovation

35. QualiTest & QualiSystems
partnership
Challenges with NFV –
what, when and why to
verify with SDN/NFV?
Summary
Creating an Automated
SDN / NFV verification as
a Service
Building SDN / NFV
assurance process

36. Summary
• Combined approach, building on a methodology of test processes and
automation.
• There will also be a skills pivot as organisations transition to this
virtualised environment.
• Increased pace of innovation.
Thankyou