2. Agenda
This is a technical deep dive on video surveillance deployment
architectures and the role that storage plays – including how
to size, scale, and design using best practices derived from
extensive testing and experience with real world deployment
Key Takeaways
− Identify video surveillance trends and their impact on storage
− How to size storage for video surveillance deployments
− Best practices for designing storage for video surveillance
Resources – Contacts, TRs and Whitepapers
2
3. Outline
Video Surveillance
Industry Trends
Deployment Architectures
Sizing Considerations
Best Practices
Summary / References
3
5. Television has changed user expectations
Increasing demand for Megapixel /
HDTV resolutions
More than 50% of security cameras shipped
by 2014 will be megapixel/HD resolution
“The World Market for CCTV and Video Surveillance
Equipment – 2010 Edition.” – IMS Research
Some expectations are unattainable
− Unlimited digital zoom
− Immediate forensic search results
CSI publicity Photo / Fair Use
5
6. Three ‘R’s of Video Surveillance
High Definition and Megapixel cameras
Resolution dramatically increase storage requirements
over Standard Definition
Increasing retention periods as storage costs
Retention decrease and public safety threats increase
Loss of video may incur fines (gaming) or
Reliability hamper forensic analysis
6
7. Resolution (Scaled to 20% of Original Size)
Factors Influencing
Demand for Storage
− Resolution
− Frame Rate
− Retention Period
− Image Complexity
− CODEC /
Compression
− Continuous or
Record on Motion
(CIF) Common Intermediate Format
7
8. Purchasing Decision Drivers
Storage costs – 30% of deployment
Servers
Key decision points are Surveillance
20%
Switches
25% 15%
− Network Video Cameras Software
− Video Management Software
10%
Storage
Servers and Networking 30%
viewed as commodity items
Storage efficiency and reliability
increasingly more important
8
11. Deployment Architectures
Recording Server model Cameras
Servers
• Control cameras 0101010 Video Management Software
• Receive video feed IP SAN
0101010
• Archive 0101010 iSCSI
SAN
Transport: IP Network FC
H.264/RTP/UDP or
Direct Attach
H.264/RTP/RTSP/TCP
SAS
MJPEG/TCP 0101010
Storage FC
Viewing
Camera to Storage model Workstations
• Bosch (iSCSI)
• MOBOTIX / IQinVision
(NFS/CIFS)
• Axis (Local SDHC / CIFS)
11
12. E-Series Modular Design for Video Surveillance
• 4U/60 x 3.5” drives
DE6600 • Highest throughput
• Largest capacity/density
• 3TB NL-SAS drives
Disk Shelves • Easy to service
• 8Gb/s FC • SAS (for RACK Solution)
Controllers • 12 or 24GB Cache per system • 4GB Cache per system
• ~ 24.8Gb/s write performance • ~ 11.2Gb/s write performance
• High Bandwidth / High IOPs • Cost effective performance
5400 2600
• Verify support in IMT • NVS Storage Rack only
Delivery
• Use Quote Edge • Quotable and orderable
through distribution
12
13. NetApp Video Surveillance Storage Solution Rack
Hardware / software components
OnSSI
Milestone Cisco
Utilities Windows Server Redhat
SANtricity 2008 R2 Linux
NTFS ext3
Video
10/100/1000 LAN on VMware ESXi 5.1 Broadcom
Ingress
Motherboard Quad Port
Management Raw device mapping 1Gb Ethernet
LOM
Cisco UCS (RDM) C220-M3 Links
Ethernet
LSI SAS 9200-8e – 6Gbps x 4 Aggregated
Serial Attached SCSI (SAS)
NetApp E2660
E-Series Controller Controller
A Cache Cache B
2GB 2GB
Volumes (LUNs)
13
14. NetApp Video Surveillance Storage Solution
Hardware / software components
Genetec Omnicast
Verint Nextivia
OnSSI Ocularis
Utilities Windows Server
Failover driver SANtricity 2008 R2
NTFS
10/100/1000 Management Optional ESXi 5.1 Gigabit /
Ethernet commercial 10 Gigabit Video
Management Ports Raw device mapping Ethernet
Ethernet off-the-shelf (RDM) Server(s) Ingress
8Gbps optional FC HBA
SAN SAN
NetApp E5460
E-Series Controller Controller
A Cache Cache B
6/12GB 6/12GB
Volumes (LUNs)
14
15. Virtualization
Most VMS vendors support virtual servers (Milestone XProtect
certified with VMware and Microsoft Hyper-V)
Recording servers are high resource demand –
do not co-host with other applications
Some VMware functions
(Vmotion / Snapshots) not
supported on RDM LUNs
Primary advantage – sharing
Network and FC or SAS HBAs
Raw device mapping (RDM)
15
17. Sizing Based on Determining Several Parameters
Resolution
HW / SW
Performance
Video Management Volumes (LUNs)
Frame Rate Software
Number of Data Rate
Cameras Recording Server
CODEC
Compression
(Quality)
Group Variable/
Volume Group
of Pictures Constant
(GOP) Bit Rate RAID Level
Retention
Policy
Disk Drives
17
18. Bandwidth and Storage
Calculator
Camera manufacturers provide
sizing calculators
User Inputs:
− Camera Model
− Frame Rate
− Resolution
− Compression
− Image
Complexity
18
19. Image Complexity
Variable Bit Rate (VBR)
− Pre-defined image
quality 7%
Bandwidth fluctuates
− Scene complexity
− Motion
− Image Noise
7pm 7am
Storage capacity and Lights off Lights on
throughput also change
19
21. Sizing Example
Video Surveillance Rack (2660) OnSSI Ocularis
RAID 1
Physical Server
1+1
VM Base Volume RAID 6
Group 12+2
VM Fail Over Server 1
Volume Volume
VM Fail Over Server 2 * Bookmarks Group
VOL_FAIL_1
3-5
VM RC-E Manager Days
30
VOL_ARCH_1 Days
Physical Server
RAID 6
VM Recording Server 1 12+2
VM Recording Server 2 Volume
RAID 10 Group
VM reserved Data Store
5+5
* VOL_FAIL_2
Volume
VM reserved RAID-1 Group
1
VOL_LIVE_1
Day VOL_ARCH_2
VOL_LIVE_2
2 Hot Spares
Per 60 disk shelf
21
22. Data Rates
NetApp Video Surveillance Storage Solution Rack
Link Aggregation
Physical Machine(s) Four 1 GigEthernet
640 Cameras
@
2 Mbps Each
Virtual Machines 256 Cameras @ 2 Mbps
512 Mbps 1.5 Gbps
aggregate
3 physical machines
10 virtual machines 28-32 TB
Volume
IOMETER test tool E2660
Failover
6 Gbps SAS
640 Mbps
9-11 Gbps @ 512K I/0
Failover ARCHIVE
32.742 TB VGs
Each VM writing to both (14+2)
64 Cameras LIVE
LIVE and ARCHIVE
per VM 1 TB 4.1TB VGs
Volume (5+5)
22
24. High Availability Architecture
Cameras to Storage Array
IP IP – Core /
Access Layer Distribution Layer
(B)
Overlapping
Field of View
(A) (B)
(A)
IP (B)
Server Recording
Access Layer Servers
(Top-of-Rack) (A) Dual Fabric E5460
SAN Duplex Controllers
24
25. Tuning Performance Parameters
One active Volume (LUN) per Start / Stop Cache Flush: 80%
Volume Group (VG)
Flush Write Cache: 10 sec.
Read/Write Cache: Enabled
Cache Block Size: 32K
Write Cache Mirroring:
Media Scan Frequency: 30 days
Disabled
Data Assurance (DA): Disabled
Initial Segment Size: 128K
Windows (NTFS) Allocation
Unit Size: 64K
26
26. OnSSI Ocularis / Milestone XProtect
Best Practices
Archival function
Moves video files from
„Recording‟ location to
„Archive‟ location.
Files may be groomed
to reduce the frame rate Separate Recording and Archive Configuration
or encrypt
User configured schedule
I/O rates increase during
archive (8x observed)
Recording Only Configuration
27
27. Summary
High Definition / Megapixel
cameras – must sell proposition –
drives storage sales
Retention periods – minimum of
30 days – storage efficiency
Reliability and Redundancy –
lead with high availability and
redundant solutions
Trusted partner – Position the
NetApp brand through training,
education, certification and support
28
28. Resources
NetApp Video Surveillance Storage Solution
Distribution List (DL) for questions on this session.
− xdl-bd-2-577@netapp.com
NetApp Community (Blog Posts)
https://communities.netapp.com/blogs/Video_Surveillance_Solution
Documentation
− Field Portal E-Series Solutions
NetApp Video Storage Solution
Contacts
− Jenifer Piccioni, Solution Product Manager – Jenifer.Piccioni@netapp.com
− Jon Robbins, Solution Marketing Manager – Jon.Robbins@netapp.com
29
30. Video Surveillance Acronyms
Video Codecs
VMS Server : Video Management System M-JPEG : Motion – Joint Photographic Experts Group
Server – aka Network DVR. Manages IP MPEG-4 : ISO/IEC Moving Picture Experts Group
camera video feeds and storage media. (Part 2)
Viewing Station: High-end Intel workstation H.264 : ISO/IEC Moving Picture Experts Group (Part
10)
for displaying live or archived camera feeds.
on a locally attached monitor JPEG 2000 – Joint Photographic Experts Group
(committee 2000)
Video Wall : One or more dedicated
workstations to control/displays video on
physical and virtual monitors. Video Signaling and Transport Protocols
HTTP : Hyper Text Transport Protocol
IP Video Surveillance Camera : digital
video camera – small form factor IP HTTPS : Secure HTTP
networked Linux host which encodes and RTSP : Real Time Streaming Protocol
transports video over an IP network. RTP : Real-time Transport Protocol
RTCP: Real Time Transport Control Protocol
NetApp Confidential – Internal Use Only 31
31. NetApp Video Surveillance
Storage Solution Rack
Video Management Software
− OnSSI Ocularis ES
− Milestone XProtect Corporate
− Cisco Video Surveillance Manager (VSM)
VMware ESXi 5.1 hypervisor
Cisco UCS C220-M3 Servers (2-4)
E2660 (SAS) duplex controller (1-3 DE6600)
Cisco Nexus 3048 Switches (2)
Cisco Rack and Power Distribution Unit(s)
Pre-assembled by distribution channel partner
32
32. NetApp Video Surveillance Storage Solution Rack
Cisco UCSC-C220-M3S High-Density Rack Server
− Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GHz (2 CPU 8 Core)
− 64 GB Memory
− N2XX-ABPCI03-M3 Broadcom 5709 Quad Port 1Gb
− LSI SAS 9200-8E 6Gb/s PCIe 2.0 HBA
− UCSC RAID SAS 2008M-8i Mezzanine Card
− A03-D300GA2 300 GB 6Gb SAS 10K RPM SFF HDD (2 disks)
Cisco N3K-C3048TP-1GE Nexus 3048TP-1GE 1RU
− 48 x 10/100/1000 and 4 x 10GE ports
33
33. NetApp Video Surveillance Storage Solution Rack
E2600 / DE6600
− Number of controllers: 2
− Data Cache: 2GB per controller
− Firmware version: [TBD]
− Host Interface: 6Gbps SAS (4 per controller)
− Drives (60-180):
3TB 7500 rpm NL-SAS
900GB 10K rpm SAS
34
Editor's Notes
26 October 2012 – Final Uploaded VersionNOTE: On slides which have much detail/complexity, move closer to the screen to speak. On slides with less details, you can be further from the screen. Personalize by using ‘You’
Remind the audience of two numbers and three letters.25%, 30% and three R’s – Resolution Retention and ReliabilityThe 25% is an estimate of the year over year growth of the physcial security industryThe 30% is the approximate percentage that storage plays in the cost to deploy a video surveillance deployment.
(CIF) Common Intermediate Format(XUGA) Ultra Extended Graphics Array(HDTV) High-definition television
Networked Video Cameras and Video Management Software are the key components that drive the deployment.Surveillance covers cameras, cables, etc.Large US chain store installation and Axis total cost of ownership studyDeployments with retention periods in years (vs. 30 days) increase storage costs
Axis P1346 – 1280x720 12fps H.264 50% compression 24 hours per day = 1.09Mbps 330GB 30 days – schoolyard (low light)Axis Q6035 – 1920x1080 30fps H.263 30% compression 24 hours per day = 5.53mbps 1.73TB 30 days – schoolyard (low light)Arecont AV8285DN – Panoramic IP cameras – 6400 x 1200 pixelsDeployed as four, H.264 2 Megapixel sensorsAt 7 fps, 31 day retention is 2.2 TB storage
Video may be transported between endpoints in either UDP or TCP.Image Quality (loss of frames) may occur in both transports methods. The network must have sufficient available capacity (bandwidth) to transport videoExhibits very low / no loss of IP video packetsNetwork latency is within the range suitable for the transport protocol (eg. TCP/UDP) of the video feedProvides high availability – network redundancyMeets the network security (and services) requirements
2600 in racks only, Comment that the performance characteristics provide room for growth.
Note: NTFS has a maximum volume size of ~256TB and 64KB cluster sizeExt3 has a 16TB max volume size with a 4K block sizeUp to 4 physical servers may be direct attached via SAS cables2660 Rack programs, all will be using Fremont.1 from the startThe cache size for the 2660 can be either 2GB or 4 GB per controller, but only the 2GB option is orderable
commercial off-the-shelf (COTS) servers10G iSCSI for 5400 is being released as part of Fremont.1Release date for the HIC and Fremont.1 is 11/5/2012When installing SANtricity, if you are using a hypervisor, VMware ESXi do not install the failover driver. If not using ESXi, you need the failover drivers for multipath support.
http://www.milestonesys.com/SharePoint/White%20papers/TipsTricksVirtualServerEnvironment.pdfVMWare functions not supportedNo converting VMs with RDM into templatesNo Vmotion or VMware SnapshotsNo cloning VMs with RDM LUNs
Image noise is an undesirable by-product of image capture that adds spurious and extraneous information http://en.wikipedia.org/wiki/Image_noiseVariable Bit Rate (VBR) Pre-defined image quality (low normal high very-high) Scene complexity and motion will influence bandwidth consumption More difficult to provision / estimate network bandwidthConstant Bit Rate (CBR) Bandwidth consumption is deterministic (over seconds) Either image quality and/or frame rate may change Average bandwidth consumption can be estimated Network traffic is still bursy – due to I-frame transmission Image quality may be noticeably degraded – see next slide
Assume 40 cameras at 393 Mbps at 14.8 TB per month. (Axis P1346 Intersection Low Light – 3Mp – 30% compression h.264, 12 fpsThe arrival rate is approximately 9.8Mbps per camera. According to the Axis design tool, we need 14.8TB per 30 days which would be 18.5TB at 20% free space
Note, connectivity from failover server 2 to failover volume 2 not shownThe ESXi 5.1 is installed and booted from the internal disks, and DataStores are present on those disks for the VM’s C:\\ disk. Windows 2008 R2 is installed on the C:\\ disk.The volumes are Raw Device Mapping from the E-Series to the physical machine and then to the virtual machine.Each virtual machine has a Drive (letter) for the LIVE and ARCHIVE LUNs (volumes)
On this slide be definite where to start the discussion. The are many speeds and feeds on this slide that may confuse the audience if the context is not clear.http://en.wikipedia.org/wiki/PCI_Express V2 can run at 4Gbps on each laneAssume 64 cameras per server at 1-3 Mbps on the network side of the serverFour virtual servers per physical machineSAS HBA (6Gbps x 4)Measure the performance of the storage side.Compare that with ingests rate.Disk Rate Signiant
Note: the IO for most video management systems is in the 256K to 512K or higher range. The IO per second is less of an issue than throughput, give the records written are large.
archival function which moves video files from a temporary or ‘Recording’ location to an ‘Archive’ location.
NetApp Community https://communities.netapp.com/blogs/Video_Surveillance_Solution
http://www.cisco.com/en/US/prod/collateral/ps10265/ps10493/C220M3_SFF_SpecSheet.pdfCisco UCS C220 M3 High-Density Rack Server (Small Form Factor Disk Drive Model) 2008M-8i Mezzanine Cards (6 Gbs) - two versions UCSC-RAID-11-C220 or UCSC-RAID-MZ-220 Cisco CiscoFlexFlash requires a 16G SDHC card - 4 GB is available for general use