+7 (812) 622 16 80

Entry-Level Data Storage System for Video Surveillance (up to 200 cameras)

06.03.2018

Data Storage in Video Surveillance

Data storage in Video Surveillance implies multi-thread sequential operations. Each IP-camera generates a sequential data stream that is written into a video archive storage. This results in a considerable multi-thread write workload.

Playing video from the archive entails sequential read operations triggered by single or multiple streams. In terms of workloads, the storage performs its best when a project does not involve permanent video playback by various operators from multiple workstations. Real-time monitoring does not require permanent data reading, and archive videos are viewed one at a time – occasionally or on a regular basis. In this scenario, the predominant workload — 90 percent or above — will have to do with writing, and the data storage will reveal no significant reading load.

In an opposite case, when multiple videos are played back in parallel from versatile cameras (e.g., casino, stadium, mall, etc.), the system processes a multitude of concurrent read streams and boosts the overall storage workload. As a result, the same number of cameras will require a higher performing storage system, otherwise the administrator may need to cut down the quantity of cameras per array (controller).

Data storage systems for entry-level CCTV configurations (up to 200 cameras) do not imply any major upgrade or upscale frequency comparable to larger systems, yet they follow the same basic scenarios and standards as mid-sized and high-end systems. Further on, we’ll review the key requirements for storing data generated by 200 parallel high-definition video streams and the corresponding RAIDIX technology solution.

 

Requirements for Data Storage

In this section, we’ll evaluate the requirements spanning system resources and the storage subsystem in large-scale Video Surveillance projects. This evaluation builds on the assumption that video servers receive, process and write video streams into an archive, and perform other essential VMS functions — without employing resource-intensive video analytics. Picture display from IP-cameras for real-time monitoring and video playback should be performed from dedicated remote workstations, which will allow video servers a 50 percent workload cut. Remote workstations in Video Surveillance stand for powerful computers, such as graphic stations. These machines run special client software to ensure VMS connectivity and provide an ability to display a multitude of pictures on large screens.

As a basis for further calculations, we’ll take a stream generated by a single IP-camera via ONVIF (an open standard that ensures interoperability of IP-cameras with VMS), with Full-HD definition (1920×1080), basic H.264 codec, a frequency of 25 fps and high frame activity. Average traffic volume in this configuration constitutes 6.86 Mbit/s.

The computing power required for servicing 200 video streams can be easily provided with a 4-core Intel Xeon E3-1225 V3 processor and 8–16GB RAM. In terms of resources, an affordable 1U server will suffice for entry-level tasks.

Storage capacity for a video archive with 30 days depth, 200 streams, and 24/7 writing will amount to 423.93TB. This storage volume can be generated with the use of numerous high-capacity drives (4–6–8–10 TB).

 

Suggested Architecture

An entry-level storage system, given the use of fault-tolerant RAID 7.3 or RAID N+M levels, may require around 60 HDD, 10TB each:

  • 1 RAID N+M group comprised of 55 drives: 50 drives of useful capacity, 5 drives for checksums – plus 5 drives for hotspare;
  • 2 standalone RAID 7.3 drive groups 28 drives each: 25 drives of useful capacity, 3 drives for checksums — plus 4 drives for global hotspare;
  • 3 standalone RAID 6 drive groups 18 drives each: 16 drives of useful capacity, 2 drives for checksums — plus 6 drives for global hotspare.

This quantity of drives can be accommodated by a special rack-mount 4U server or a 1U server bundled with a standalone disk enclosure.

 

Data Storage Solution

In the foundation of the suggested CCTV storage lies the RAIDIX Software-Defined Storage technology that enables the user to build reliable, high-performance, and fault-tolerant storage infrastructures. The use of commodity x64 hardware and high-density chassis allows a RAIDIX-based system to attain minimum TCO $/GB and $/IOPS. Commodity-off-the-shelf equipment ensures a flexible configuration for building high-speed fault-tolerant data storage systems for Video Surveillance needs. RAIDIX offers ad hoc solutions tailored to a specific number of cameras and individual workloads, eliminating the need to employ costly industrial systems, whose capacity and functionality may greatly exceed genuine IT needs of the customer.

A 200-camera configuration may include any х64 servers and the following components:

  • 1–2 Intel Xeon processors and a corresponding RAM volume
  • Single or multiple SAS HBAs for connecting to internal and/or external disk enclosures
  • Single or multiple interfaces for cache synchronization in a dual controller configuration: SAS, InfiniBand, or Ethernet
  • Interfaces for SAN and/or NAS connectivity: Ethernet, InfiniBand, Fibre Channel. (Another viable model for CCTV infrastructures is Direct Attached Storage (DAS) — disk enclosures that connect direct to the servers.)
  • SAS/SATA HDDs of large (3,5”) or small (2,5”) form factor: any model from any manufacturer without limitations
  • Server platform that fits the abovementioned hardware requirements.

RAIDIX-based data storage systems resolve the following Video Surveillance tasks:

Processing and storage of video data streams generated by cameras from versatile vendors — with video definition from VGA to Full HD/2K/4K/8К

The RAIDIX performance is optimized for storing large volumes of video data. Peak performance may reach 40GB/s for a standard rack in 42U. RAIDIX also delivers maximum efficiency when using commodity-off-the-shelf hardware by paralleling input/output operations.

High availability of the RAIDIX storage builds on using HA configurations of RAID 7.3 and RAID N+M on read/write. The technology sustains high performance even if hardware fails or the system undergoes array reconstruction. The system also supports multi-gigabyte volumes of cache memory and allows for simultaneous processing of multihundred data streams with millisecond latencies.

Ensuring data integrity

Even partial data loss on saving video information may lead to losing an entire video stream. The use of patented RAID calculation algorithms, such as RAID 7.3, allows the system to accomplish record-high speeds and data integrity. The array remains fully functional and reveals no performance degradation even if three disks in a RAID-group fail.

Fault-tolerance on data writing

The data storage system ensures full fault-tolerance in case of multiple disk or enclosure failures, hidden errors in disk sectors or human error. The RAIDIX solutions have no single point of failure (SPOF) and ensure high data availability at 99,9999 and higher.

Need for different interfaces and data transfer protocols

RAIDIX Data Storage supports all key interfaces for block and file access: SAN (Fibre Channel, iSCSI, SAS, InfiniBand) and NAS (NFS, AFP, SMB, etc.).

 

Technical Characteristics

Supported RAID levels

RAID 0/5/6/7.3/10/N+M

Max. number of drives in a RAID

64

Max. number of drives in the system

600

Scalability unit

12 drives

Hot spare

Dedicated reserve disks and shared access disks

Max. LUN size

Unlimited

Max. number of LUNs

487

iSCSI

MPIO, ACLs, CHAP-authorization, LUN masking, СRC Digest

Supported number of sessions

1024

Max. number of hosts in case of direct connection

32

Supported operating systems

Mас OS Х 10.6.8 and higher, Microsoft® Windows® Server 2008/ 2008 R2/ 2012, Microsoft ® Windows® XP/Vista/7/8; Red Hat Linux, SuSE, ALT Linux, Cent OS Linux, Ubuntu Linux; Solaris 10

Supported virtualization platforms

VMware ESX 3.5/4.0/4.1/5.0/5.1/5.5/6.0; KVM (Kernel-based Virtual Machine); RHEV (Red Hat Enterprise Virtualization), Microsoft Hyper-V Server, XenServer

Supported high-performance interface

Fibre Channel 8Gb, 16Gb; InfiniBand (FDR, QDR, DDR, EDR); iSCSI; 12G SAS

Supported NAS interfaces

SMB, NFS, FTP, AFP

Integration with MS Active Directory

Yes

WORM (Write Once – Read Many)

Yes

Number of nodes

2 in the Active/Active mode

Data caching

Two-tier: RAM and Flash, WriteBack and Read Ahead for multiple streams

QoS support

On the host/application level

 

Business Impact

Building on the RAIDIX Data Storage and modern commodity hardware, system integrators offer efficient storage solutions to entry-level and large-scale CCTV infrastructures.

RAIDIX stands for the following business benefits:

  • Data availability — 73% higher compared to the competition, storage costs — 31% lower compared to average market showings
  • Full support for required data volumes along with low TCO
  • Record performance with multiple parallel video threads
  • Flexible scalability on increasing data volumes. The systems scales up by adding new enclosures and controllers with no prejudice to IO processes or applications interacting with the system.
  • Decreased hardware overheads due to universal compatibility.

Download Whitepaper [PDF]: Entry-Level Data Storage System for Video Surveillance (up to 200 cameras)