Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optical Transport for Mission Critical Applications

Cisco VMDC - DCI DWDM Optical Transport Solution

  • Login to see the comments

Optical Transport for Mission Critical Applications

  1. 1. Kayukov Valeriy Optical System Engineer - Step Logic April 20, 2016 Cisco Optical Networking – Disaster Recovery Solutions (SAN over DWDM Calculation). Cisco Support Community Expert Series Webcast
  2. 2. • VMDC DCI Design • Protocol Comparison • Fiber Channel • Cisco Transponders Solutions • Cisco NCS 1002 • Cisco Transport Encryption • Transport Optical Protection Content of presentation • Storage Networking • Optical Recovery and Restoration • Partnership Support Ecosystem
  3. 3. VMDC DCI Design Recovery Point Objective (RPO) and the Recovery Time Objective (RTO) How fast business need recovery?
  4. 4. • Two important objectives in the designing process are the Recovery Point Objective (RPO) and the Recovery Time Objective (RTO). • The RPO is the time period between backup points and describes the acceptable age of the data that must be restored after a failure has occurred. For example, if a remote backup occurs every day at midnight and a site failure occurs at 11 pm, changes to data made within the last 23 hours will not be recoverable. • The RTO describes the time needed to recover from the disaster. The RTO determines the acceptable length of time a break in continuity can occur with minimal or no impact to business services. Options for replication generally fall into one of several categories. • A business continuity solution with strict RTO and RPO may require high-speed synchronous or near-synchronous replication between sites as well as application clustering for immediate service recovery. • A medium level Disaster Recovery (DR) solution may require high-speed replication that could be synchronous or asynchronous with an RTO from several minutes to a few hours. Backup of non-critical application data that does not require immediate access after a failure can be accomplished via tape vaulting. Recovery from tape has the greatest RTO. In addition other technologies such as Continuous Data Protection (CDP) can be used to find the appropriate RPO and RTO. VMDC DCI Design Recovery Point Objective (RPO) and the Recovery Time Objective (RTO) Terms
  5. 5. • Business units should dictate the anticipated risks in depending on the region's business and territorial coverage of possible disasters. • Coverage of potential risks dictates the need for the appearance of the second data center or appearance of Disaster Recovery Site. • Also coverage determines transport technologies and application architecture of complete solution •VMDC DCI Design reduce CAPEX/OPEX of infrastructure VMDC DCI Design Basic Terminology and architecture of classic DRS Solution Business Continuity Workload Mobility Disaster Recovery Site Migrations Load Balanced Workloads Operations Maintenance Operations Rebalancing Application Clusters
  6. 6. Simplify the DCI Design Process for Operations Teams - Interconnecting Cloud Data Centers involves many infrastructure elements and application components. The VMDC DCI validated design significantly reduces risk of implementation using Cisco’s latest product innovations End-to-end Validation of the Application Environment - VMDC DCI delivers validated guidelines across the end-to-end layers of the cloud data center. Competitive offerings only focus on a few elements. VMDC DCI spans different sites, addressing each Application element including WAN connections, tenancy, network containers, distributed virtual switching, and L4-L7 services, hypervisor migration tools, and storage replication. This is a complete DCI solution. Validates 2 of the most used DCI Design Options - VMDC DCI validates the most common design options to achieve 2 major Recovery Point Objective (RPO) and Recovery Time Objective (RTO) targets. The first design option enables the movement of applications, their services, and network containers to support near zero RPO/RTO for the most business critical functions. Less business critical applications can be mapped to a second option to achieve RPO/RTO targets of 15 minutes or more. Minimal Disruption to the Application - VMDC DCI allows operators to preserve IP addresses of moved applications and their services between sites. Reduction in CAPEX/OPEX for DCI Deployments - VMDC DCI helps customers align the correct DCI design to achieve application RPO/RTO targets. The most stringent recovery targets typically require the highest CAPEX/OPEX. VMDC DCI provides a framework to map Applications to different Criticality Levels, and then select the most cost effective design option that meets application requirements. Planned Usage of Recovery Capacity - Recovery capacity at remote sites can be used for other applications during “normal operations” and “reclaimed” as needed during recovery events. This “Reuse-Reclaim” design strategy allows for planned utilization of extra capacity and many-to-one resource sharing, reducing CAPEX/OPEX. Multiple Hypervisors supported - Both VMware and Microsoft Hyper-V environments are supported. DCI Use Cases Validated with Business Applications - VMDC DCI used traditional business applications across each workload migration and business continuity use case. The test applications include Oracle database servers, Microsoft SharePoint and SQL, for single tier and multi-tier test applications. Product Performance Measured across DCI Use Cases - The performance of Cisco products and Partner Products was measured and documented across Metro/Geo environments. Performance limitations, design recommendations, and configurations are provided for Cisco and Partner products. VMDC DCI Value Proposition Simply DCI Deployments, reduce CAPEX/OPEX of design, Reuse-Reclaim Recovery Resources
  7. 7. Mgt Infrastructure and Orchestration Switching Fabric Integrated Compute Stacks WAN Edge / DCI Storage and Fabric Extensions Virtual Switching Services & Containers Virtual Storage Volumes Data Center 1 Cisco Product  OTV LAN Extension, Preserve IP Addressing of Applications  IP WAN Transport with 10ms RTT across Metro distance  External Path Re-direction thru routing update and orchestration  Routing re-convergence to new site  Stretched ESX Clusters and Server Affinity  VMware Live vMotion across Metro sites  Distributed vCenter spanning Metro sites  Single and Multi-Tier Application migration strategy  VMDC 3.0 FabricPath (Typical Design) with Multi-Tenancy  Palladium Network Container  Nexus 1000v with VSMs and VEMs across Metro sites  Service and Security Profiles follow Application VMs  Different Nexus 1000v’s mapped to Application Domains as needed  Virtual volumes follow VM  NetApp MetroCluster Synchronous Storage Replication  ONTAP 8.1 Fabric MetroCluster, 160 Km long haul link (DWDM)  FCoE to compute stack, Cisco MDS FC Switching for data replication  Replicate Service Container to new site to support Mobile VM  Virtual Mgt Infrastructure support across Metro VMDC DCI Design Choices Partner Product Route Optimization Path Optimization (LISP/ DNS / Manual ) Layer 2 Extension (OTV / VPLS / E-VPN) Stateful Services (FW / SLB / IPSec / VSG) VMware & Hyper-V UCS / Geo-Clusters / Mobility Distributed Virtual Switch (Nexus 1000v) Distributed Virtual Volumes Container Orchestration Storage Clusters MDS Fabric and FCoE Tenancy and QoS  Stateful Services between sites  Citrix SDX SLB at each site (no Metro extension)  ASA 5500 FW Clustering at each site (no Metro extension) VMDC DCI Design Active-Active Metro Design Choices
  8. 8. Synchronous Data Replication guarantees continuous data integrity during the replication process with no extra risk of data loss. The impact on the performance of the application can be significant and is highly dependent on the distance between the sites. Metro Distances (depending on the Application can be 50-200km max) VMDC DCI Design Synchronous Data Replication
  9. 9. Asynchronous replication overcomes the performance limitations of synchronous replication, but some data loss must be accepted. The primary factors influencing the amount of data loss are the rate of change (ROC) of data on the primary site, the link speed, and the distance between sites. Unlimited distances. VMDC DCI Design Asynchronous Data Replication
  10. 10.  Fiber channel released 32G Standard, soon coming 64G FC.  PCI Special Interest Group roadmap PCIe 4.0 2016-4Q - 16 GT/s.  Primitive flow control commands makes FC stack faster.  Resilient and secure network, dedicated from Internet. Protocol comparison Fiber Channel versus Ethernet  Ethernet 200G have standardized 802.3bs protocol. Coming 400G.  IEEE organization plan to realize P802.3bs 400 Gb/s Standard at 2017-Q4  Lower speed, complex OSI stack makes demand in communication.  Cloud Storage Is Software-Defined, Scale-Out and Not Fiber Channel.  Hyper-Converged Infrastructure (HCI) Is Showing Hyper-Growth  Growth of File and Object Storage. Analyst IDC predicts file and object storage are growing at 24% per year.
  11. 11.  Fiber Channel SANs are faster than iSCSI SANs in case of line rate. For sample, line rate of 1G (1.0625) FC higher than 1GE.  Fiber Channel’s state machine use primitive sequence to make flow control. This primitive makes it faster.  FC latency is less than 3500ps, SAS less than 6500 ps. But iSCSI latency should be significantly greater than Fiber Channel, because of TCP latency.  Self-documented due to FC LOGINs and Name Server Registration in fabric. Protocol comparison FCFCoE versus iSCSI  In most cases iSCSI SANs are always less expensive than Fiber Channel SANs.  At first sight, iSCSI SANs simpler to operate vs. Fibre Channel SANs. But in real zoning provisioning simpler than iSCSI’s pointing IQN and IP addreses.  Ethernet use « Cut Throught » (less than 5 ps) and « Store and Forward » (around 20 ps) technologis. Summary: It’s more about what is right for your design and less about technology. FC suitable for isolated low latency network, or in case of IBM System Z.
  12. 12. Essentially the same in terms of – Network-centric – Similar management tools – Same multipathing software (for iSCSI as well) – Similar basic port types ▪ N_Ports / F_Ports vs. VN_Ports and VF_Ports ▪ E_Ports vs. VE_Ports – Same scalability limits ▪ Number of domains ▪ Number of N_Ports / VN_Ports ▪ Number of hops Protocol comparison FC versus FCoE
  13. 13.  Buffer Credits Define the maximum amount of data that can be sent prior to an acknowledgement  Buffer Credits are physical ASIC port or card memory resources and are finite in number as a function of cost  Within a fabric, each port may have a different number of buffer credits  The number of available buffer credits is communicated at fabric logon (FLOGI)  BB Flow Control works on Link Layer (FC-1), EE Flow Control works on Transport Layer (FC-2) Buffer-to-buffer (BB) credit flow control is implemented to limit the amount of data that a port may send, and is based on the number and size of the frames sent from that port. SAN switches can support two methods of flow control over an ISL:  Virtual Channel (VC_RDY) flow control  Receiver Ready (R_RDY) flow control Fiber Channel Buffer Credit Flow Control
  14. 14. VC_RDY flow control differentiates traffic across an ISL:  Algorithm differentiate fabric internal service traffic, and to differentiate different data flows of end-to-end device traffic to avoid head-of-line blocking. Service type of traffic is given a higher priority, then other.  Multiple I/Os are multiplexed over a single ISL by assigning different VCs to different I/Os and giving them the same priority (unless QoS is enabled).  I/O multiplexing gives function balancing the performance of different devices communicating across the ISL. Fiber Channel Virtual Channel (VC_RDY) versus Receiver Ready (R_RDY)  R_RDY flow control, is defined in FC standards and has only a single lane or channel for all frame types.  When switches are configured to use R_RDY flow control, there are other mechanisms to enable QOS and avoid head-of-line blocking problems.
  15. 15. Fiber Channel B-to-B and E-to-E Flow Control Domains  Used by Class 1 and Class 2 service between 2 end nodes.  Nodes monitor end to end flow control between themselves, directors do not participate.  End to End flow control is always managed between a specific pair of node ports BB_Credit management occurs between: – One N_Port and one F_Port – Two E_Ports – Two N_Ports in a P2P topology – In Arbitrated Loop different modes When connecting switches across dark fiber or WDM communication links note:  VC_RDY is the preferred method, but there are some distance extension devices that require the E_Port to be configured for R_RDY.  To prefer “buffer-to-buffer credit spoofing” disable on ports buffer-to-buffer state change (BB_SC) number. (read in notes)
  16. 16. FC switches track the available BB_Credits in the following manner:  Before any data frames are sent, the transmitter sets a counter equal to the BB_Credit value communicated by its receiver during FLOGI  For each data frame sent by the transmitter, the counter is decremented by one  Upon receipt of a data frame, the receiver sends a status frame (R_RDY) to the transmitter indicating that the data frame was received and the buffer is ready to receive another data frame  For each R_RDY received by the transmitter, the counter is incremented by one Fiber Channel Buffer credit negotiation
  17. 17.  To transmit to the same distance, at higher speed, requires more buffer credits.  Full data frame 2148 bytes in size with QOS enabled.  Methods of BB calculation: • Time length based • Classic method • Average based method Fiber Channel Buffer credits to Link speed
  18. 18. SRDF synchronous mode - the EMC storage device responds to the host that issued a write operation to the source of the composite group after the EMC storage, which contains the target of the composite group, acknowledges that it has received and checked the data. SRDF asynchronous replication - the EMC storage device provides a consistent point-in-time image on the target of the composite group, which is a short period of time behind the source of the composite group. Asynchronous mode is managed in sessions. Asynchronous mode transfers data in predefined timed cycles or in delta sets to ensure that data at the remote target of the composite group site is in the dependent write consistent state. Fiber Channel EMC specific feature SRDF
  19. 19. SiRT feature localize transfer-ready response to local RF port, thereby reducing an unnecessary acknowledgement response trip over SAN and DWDM network, Single RoundTrip (SiRT) feature dynamically enabled for SRDF links/S links with distance more than 12km with blocks up to 32K. SAN switches and DWDM Transponders measure link latency and disable it automatically if connected to these devices. With SRDF recommended to use fast write feature on network devices. There are two modes of SiRT feature – Off and Auto. Auto –accelerate only write commands. Fiber Channel EMC specific feature SRDF (SiRT)
  20. 20. Fast write feature localize transfer ready response between transponder and E port of SAN switch. This feature transparent to SRDF FC link and used to all SRDF modes. Transponder client FC port looks like phantom target to initiator Fiber Channel EMC specific feature SRDF (Fast Write)
  21. 21. This method use time of frame propagation in fiber to calculate sophisticated number of buffer credits. Optimal number of credits is determined by:  Distance (frame delivery time)  Processing time at receiving port  Link signaling rate  Size of frames being transmitted Optimal # BB_Credit = (Round-trip receiving time + Receiving_port processing time) / Frame Transmission time As the link speed increases, the frame transmission time is reduced; therefore, as we get faster iterations of FICON such as FICON Express4 and Express8, the amount of credits need to be increased to obtain full link utilization, even in a short distance environment Reference link to Brocade BB calculator: Credit-calculator-spreadsheet-February-2015/ba-p/70873 Fiber Channel Time length based buffer credit calculation
  22. 22. 1. Determine the desired distance in kilometers of the switch-to-switch connection. 2. Determine the speed that you will use for the long-distance connection. . 3. Use one of the following formulas to calculate the reserved buffers for distance: • If QoS is enabled: (Reserved_Buffer_for_Distance_Y) = (X * LinkSpeed / 2) + 6 + 14 • If QoS is not enabled: (Reserved_Buffer_for_Distance_Y) = (X * LinkSpeed / 2) + 6 The formulas use the following parameters: X = The distance determined in step 1 (in km). LinkSpeed = The speed of the link determined in step 2. 6 = The number of buffer credits reserved for fabric services, multicast, and broadcast traffic. This number is static. 14 = The number of buffer credits reserved for QoS. This number is static Fiber Channel Classic method buffer credit calculation
  23. 23. Allocating buffer credits based on average-size frames In cases where the frame size is average, for example 1,024 bytes, you must allocate twice the buffer credits or configure twice the distance in the long-distance LS configuration mode. 1. Use the following formula to calculate the value for the desired_distance parameter needed for Fabric OS to determine the number of buffer credits to allocate: desired_distance = roundup [(real_estimated_distance * 2112) / average_payload_size] 2. Determine the speed you will use for the long-distance connection. Fiber Channel Brocade SAN switches specific buffer credit formulas (from FOS Admin guide) Gigabit value Buffer requirements 1 Gbps 1.0625 2 Gbps 2.125 4 Gbps 4.25 8 Gbps 8.5 10 Gbps 10.625 16 Gbps 17 3. Look up in table the data_rate value for the speed of the connection. 4. Use the following formula to calculate the number of buffer credits to allocate: buffer_credits = [desired_distance * (data_rate / 2.125)]
  24. 24. Fiber Channel BB Calculation for 10GFC – Classic method versus Timebased method 50 60 70 80 90 100 110 120 130 140 150 BB calculation - classic method 10GFC 270 320 370 420 470 520 570 620 670 720 770 BB calculation - Timebased method 10GFC 254 303 353 402 452 501 551 600 650 699 749 200 300 400 500 600 700 800 BBCredits
  25. 25. Fiber Channel BB calculation - frames average-size based method 10GFC (1024 payload) 50 60 70 80 90 100 110 120 130 140 150 BB calculation - frames average-size based method 10GFC (1024 payload) 516 619 722 825 928 1031 1134 1238 1341 1444 1547 BB calculation - frames average-size based method 10GFC (2112 payload) 250 300 350 400 450 500 550 600 650 700 750 200 400 600 800 1000 1200 1400 1600 BBCredits
  26. 26.  All methods (classic method & time-based method & average based method) gives adequate results, witch have just about same results. For sample, for 150 km distance credits port need about 749 and 770 credits.  Assuming that the frame is a full-size frame, one buffer credit allows a device to send one payload up to 2,112 bytes (2,148 with headers).  Assuming that each payload is 2,112, you need one credit per 1 km of link length at 2 Gbps -> smaller payloads require additional buffer credits to maintain link utilization. Fiber channel Methods comparison
  27. 27.  In this case DWDM not transparent for SAN. SAN switches connecting to each other thru pair transponders.  Credit starvation occurs when number of available credits reaches zero preventing of transition of FC prevented  Once starvation occurs timeout will be triggered causing link re-initialization.  This situation triggered transponders have one buffer credits to extend distance and own flow control, long distance feature. Now it’s dead way. Fiber Channel Buffer credits on DWDM Transponders
  28. 28. Two new universal service card, which replaced the existing 2.5G and 10G transponders and muxponders: • AR-MXP - maximum performance up to 20 Gbit / s • AR-XP - maximum capacity of up to 40 Gbit / s • Supports all types of client interfaces and protocols • Support for different modes of operation, with the possibility of combining the functional • The possibility of using two trunk ports to work as protected transponder or muxponders • Support for new features - 8G FC, OTU1 Muxing, Auto Sensing, etc. • Pay-As-You-Grow licensing model that allows you to add the necessary functionality as needed Cisco Transponders Solutions Any Rate Muxponder and Any Rate Xponder – AR-MXP
  29. 29. Cisco Transponders Solutions Any Rate Muxponder and Any Rate Xponder – AR-MXP • 8x slots for SFP and XFP slots for 2x • TDM functionality • OC3 / STM-1, OC12 / STM4, OC48 / STM16 • Transparent transport of TDM services • The ability to multiplex between any ports • OTN functionality • Support OTU1 with G.975 FEC on client ports • The ability to multiplex 4xOTU1 in OTU2 • Support for Standard G.709 FEC, G.975.1 I.4 and I.7 E-FEC on OTU2 trunk ports • Ethernet and SAN: • 8x FE, 8xGE, 8x1G FC, 4x2G FC, 2x4G FC, • 4x ISC-3 (STP) • 1 x 8G FC TxP • 4G FC TxP • Video
  30. 30.  In this configuration transponders are used: • Transponders 10x10G and Multi-Rate multiplexing 10 channel 10G into OTU4 through chassis backplane. • Transponder 200G uses only 200G Trunk port to transport OTU4 (Line rate - 111.809G, Transparent G.709 standard)  Two 10x10G Transponders can’t be used with 200G, because it can’t work in “skeep slot”.  Many operation modes, HW ready to support SAN technologies, SW in feature. Now could be used to connect DC with use of Ethernet technology. Need to contact with local Cisco CAMSE to get details. Cisco Transponders Solutions 10x10 TXP + 10x10 MR-MXP + 200G TXP (Muxponder mode)
  31. 31.  Recently released new 400G transponder with 2 DWDM ports 200G XFP based, 6 QSFP+ and 4 combo QSFP+QSFP28.  Many operation modes, HW ready to support SAN technologies, SW will in feature. Now could be used to connect DC with use of Ethernet technology. Need to contact with local Cisco CAMSE to get details.  Applications: – 200 Gbps Muxponder Application (Metro, Long Haul) – Cisco 100 Gbps SR4 or LR4 2xTransponder Application (Ultra Long Haul) – 10 Gbps, 40 Gbps Muxponder – 10 Gbps, 40 Gbps and 100 Gbps Muxponder – OTN Switching  In attach BOM for pair of transponders. Cisco Transponders Solutions 400 Gbps XPonder Line Card
  32. 32. Cisco Transport Encryption NCS 2000 Transport Encryption Portfolio 10G Transport: 5 x 10G Encrypting Transponder • Five independently encrypted 10G streams. Multi-protocol support • Grey (SR, LR, ER, ZR) or DWDM (fixed or tunable) line side optics 100G / 200G Transport: Multi-rate Encrypting Muxponder • 100G CPAK SR/LR client or 10G / 40G multiplexed payload • Pairs with coherent DWDM trunk card for transport over 100G or 200G wavelength
  33. 33. Cisco Transport Encryption Multiple Simultaneous Operating Modes Encrypted 10G Transponder Encrypted 10G Muxponder (10G Muxponder upstream) Encrypted 10G without DWDM Unencrypted 10G Transponder Unencrypted Regenerator
  34. 34. Cisco Transport Encryption Multi-Rate Muxponder Line Card • 10G, 40G, and 100G client card • 2 x 10G SFP+, 2 x 40G QSFP+, and 1 x 100G CPAK ports • 10G & 40G clients can be aggregated to the backplane or to the CPAK port • Clients can be aggregated to 100G or 200G DWDM trunk • Aggregated client signal can be encrypted (2H 2015) Multi-Rate Muxponder Nx10G Nx40G 100G Client(s) 100/200G WDM Line Card 100G or 200G Wavelength 100G
  35. 35. Cisco Transport Encryption 200G DWDM Encryption Configurations QSFP SFP+ Multi-Rate MXP DWDM Trunk QSFP SFP+ CPAK CPAK 200G 200G Muxponder Client (with CPAK on Trunk Card) QSFP SFP+ QSFP SFP+ CPAK 200G 200G Muxponder Client (no CPAK on Trunk Card) Multi-Rate MXP DWDM Trunk Multi-Rate MXP
  36. 36. NCS1k Highlights:  At 2 RU, the system supports up to 2Tbps traffic in complete DWDM system  Linux kernel with the 64 bit IOS XR OS in a Linux Container (LxC).  The NCS 1002 has 2 redundant and field replaceable AC & DC power supply units and 3 redundant and field replaceable fans, controller and SSD disk.  Each NCS 1002 unit provides 20 QSFP based clients and 8 CFP2-ACO based DWDM trunk ports.  Multiply configurations of multiplexing – 9. Cisco NCS 1002 All in one box
  37. 37. 2x100G QSFP – 2x100G CFP2 Cisco NCS 1002 All in one box - configurations 5x100G QSFP – 2x250G CFP2 5x100G QSFP – 2x200G CFP2 20x10G QSFP – 2x100G CFP2
  38. 38.  Cisco NCS 1002 could be ideally used in case of emergency need of transport multiplexed, encrypted on long haul distances over 3-rd party DWDM optical transport System.  3-rd party DWDM system have support Alien wavelength Transport or Black Link (ITU G698.2) and multiplexing with use of standard ITU 40 channel grid (oddeven) operating at 100 GHz.  Cisco MSTP software support 3-rd party IPoDWDM Transponders. Cisco NCS 1002 All in one box – Alien Wavelength
  39. 39.  PSM based protection have two variants (discussed later):  PSM Section  PSM Line  Channel protection (not covered in discussion, because 15454-AD-XC-xx.x= soon will EOL)  Transponder based protection on 400G transponder will work in feature releases. FC links will NOT re-initialize in case of shutdown of one trunk port. Transport Optical Protection PSM based protection, Splitter protection
  40. 40.  VIAVI solutions develops PSM modules for Cisco, VIAVI authorized technology partner and distributor of Polatis Company’s products.  Inside PSM Splitter 50%-50% and optical switch.  In case PSM protection configuration Raman amplifiers can’t be used because return loss (RL) of optical switch is to high.  PSM insert significant loss to total optical budget of line, around 7db per link.  400G Transponder with PSM and trunk based protection not yet tested. Transport Optical Protection PSM Multiplex Section Line Protection VIAVIPolatis optical switch
  41. 41. Sample – 1, PSM Section Protection, 8x8G FC, 150km. Price – 900k USD Transport Optical Protection Reference design Sample – 2, PSM Section Protection, 10x8G FC,10x10GE, 160 and 250km. Price – 1 000k USD. Sample – 3, Client based protection, 10x8G FC,10x10GE, dark fiber. Price – various price. Sample – 4, Client based protection, 10x8G FC,10x10GE, 250km. Price – 1 430k USD CTP *.mpz file CTP *.mpz file CTP *.mpz file CTP *.mpz file
  42. 42. Storage networking Cisco MDS 9700 Overview
  43. 43. Storage networking Most Modern 9700 Modules Part Number Product Name No. of Port Groups No. of Ports Per Port Group Bandwidth Per Port Group (Gbps) DS-X9248- 256K9 48-port 8- Gbps Adv FC module 8 6 32.41 12.82 DS-X9232- 256K9 32-port 8- Gbps Adv FC module 8 4 32.41 12.82 DS-X9248- 96K9 48-port 8- Gbps FC module 8 6 12.83 DS-X9448- 768K9 48-port 16- Gbps 12 4 644 1 MDS 9513 with fabric 3 module installed 2 MDS 9506 (all) and MDS 9509 (all) or MDS 9513 with fabric 2 module is more oversubscribed 3 MDS 9506 (all), MDS 9509 (all), or MDS 9513 (all) 4 MDS 9700 (all)
  44. 44. The number of ISLs required between Cisco MDS switches depends on the desired end-to-end oversubscription ratio. Examples of storage, server, and ISL combinations, all with the same oversubscription ratio of 8:1.  The first example has one 16-Gbps storage port with eight 16-Gbps server ports traversing one 16- Gbps ISL.  The second example has one 16-Gbps storage port with sixteen 8-Gbps server ports traversing one 16- Gbps ISL.  The third example has eight 16-Gbps storage port with sixty-four 16-Gbps server ports traversing eight 16-Gbps ISLs. Storage networking SAN ISLs Oversubscription Ratio
  45. 45. Sample: Accurately design SAN switch and modules to connect two Fabrics on 150 km distance with total throughput of 20x10G FC. Solution: ― Long distance link 150 km with 10G FC (payload 2112 bits) require 750 buffer credits per port. ― Each MDS 9448 module provide 4095 buffer credits. ― In our design DS-9448 module provides 5 links 10GFC for given distance. Only 10% port density utilized. ― Ports 10G Fiber Channel best utilize OTN multiplexing sections OTU2 of 200G Muxponder. Storage networking SAN Buffer credit usage on long haul links -Sample
  46. 46. ― In case of connection IDC Layer to Storage Core oversubscription between severs and storages of each DC is the same. ― Offset connection point of IDC Layer to Storage Edge Layer lifts oversubscription between Storages of each DC and decrease oversubscription between Servers of each DC. ― To overcome ISL oversubscription inconsistence use ISL QOS Zoning for each Layer to dedicate speed. ― Separate connection of IDC SAN switch directly to Storage Arrays is acceptable in case of full isolation and resilience. ― Separate IDC decrease total LOGINs per fabric, in its turn lifts fabric scalability Storage networking SAN Design Reference
  47. 47. Optical recovery and restoration Cisco TNCS-O with build in OTDR function TNCS-O card have two XFP modules with OTDR feature. OTDR measurement doesn’t influent on OSC channel because it use different wavelength. The OTDR feature in the TNCS-O cards lets you do the following: ― Inspect the transmission on the fiber. ― Identify discontinuities or defect on the fiber. ― Measure the distance (max. 100km) and magnitude of defects like insertion loss, reflection loss, and so on. ― Monitor variations in scan values and configured threshold values periodically. The OSC transmission ranges are: ― Standard range: 12 - 43 dB ― Reduced range: 5 - 30 dB
  48. 48. The Viavi Multiple Application Platform (MAP-200) is an optical test and measurement platform optimized for cost- effective development and manufacturing of optical transmission network elements. Competitors: Huawei(OSA+OS), ECINOPS(PM+OS) Application samples: ― Check EDFA Amplifiers in case of gain degradation with BBS, OSA, filter and optical switch. ― Perform granular raman (RAMAN-CTP, COP, EDRA) calibration with BBS, OSA, filter and optical switch. ― Perform annual CD and PMD measurement with optical switch, BBS, OSA and other modules. ― Perform L1-L4 tests (BERT, TrueSpeed, RFC 6349, RFC 2544 and other) with MTS8kOTN platforms and optical switch. Partnership support ecosystem VIAVI Solutions – MAP 200 MAP 200 Chassis, OSA, PM, BBS and other modules
  49. 49. This Sample shows interoperate Cisco management suite (CPO) with VIAVI measurement system in case of degradation of EDFA section in NCS system. In result VIAVI makes advanced troubleshooting (fault detection) and decision to disable circuit and notify OSS system about need to make RMA. Benefit of application: – Self measured intelligent network – Opportunity make granular network upgrade without need of Field Engineers locally to participate. Partnership support ecosystem VIAVI Solutions – MAP 200 – Application check EDFA
  50. 50. Xgig Analyzer is recognized as gold standard for performance insight with key capabilities:  FC, FCIP, FCoE, Ethernet analyses  Redundant link analyst  Highly accurate timing clock for precise latency measurements  Simple access to perform metrics like: - IOPS by LUN - Min/Max/Aver exchange completion time - read and write MB/Sec by LUN - frames/sec by LUN - errors by LUN  Protocol visibility, including all errors and primitives  Cascading feature availability Partnership support ecosystem VIAVI Solutions – MAP 200XgigPolatis Most common application DCI design – use optical switch (VIAVI-Polatis) with Xgig Protocol Analyzer (VIAVI) to perform manual test and analyst. To make it automate and proactive need to enhance application with use of VIAVI Management Server.
  51. 51. Mgt Infrastructure and Orchestration Switching Fabric Integrated Compute Stacks WAN Edge / DCI Storage and Fabric Extensions Virtual Switching Services & Containers Virtual Storage Volumes Data Center 1 Cisco Product  IP WAN Transport with 10ms RTT across Metro distance  VMDC 3.0 FabricPath (Typical Design)  10GE over DWDM NCS 2000, NCS 1002  NCS 10x10G + 200G Transponders or 400G XPonders  NetApp MetroCluster Synchronous Storage Replication  EMC VPLEX Synchronous Storage Replication  Cisco MDS 9700 Directors for Fiber Channel and FICON  FC or FICON over DWDM NCS 2000, NCS 1002  Cisco Prime Optical Management Suite  Cisco TNCS-O measurement system  VIAVI Infrastructure measurement systems VMDC DCI Design Choices in case of OpticalTransport Partner Product Route Optimization Path Optimization (LISP/ DNS / Manual ) Layer 2 Extension (OTV / VPLS / E-VPN) Stateful Services (FW / SLB / IPSec / VSG) VMware & Hyper-V UCS / Geo-Clusters / Mobility Distributed Virtual Switch (Nexus 1000v) Distributed Virtual Volumes Container Orchestration Storage Clusters MDS Fabric and FCoE Tenancy and QoS  Stateful Services between sites require L2 over distance  ASA 5500 FW Clustering at each site  Ethernet over FC or FICON over DWDM NCS 2000, NCS 1002  NCS Any-Rate Transponders, 10x10G Transponders VMDC DCI Design Active-Active Metro Design Choices