Software-defined storage for next generation of video processing

Processing of video content is traditionally regarded as one of the most resource intensive tasks for the client’s IT infrastructure. With new technologies emerging, new consumers’ preferences and administrative constraints such business processes have increased requirements to software and hardware.

RAIDIX and Western Digital offer a joint hardware-software solution for storing video content in conditions of growing requirements for quality and size of processed materials.

Challenges, requirements, and industry specific features

Media companies are gradually moving on to new image quality standards (4K, 8K) and trying to offer new interactive features such as VR video. Handling such content leads not only to robust growth of storage size but also trigger demanding requirements for data storage speed and throughput.

In terms of data storage there are similar requirements to CCTV systems caused by new level of national and public security, which leads to new legislative initiatives and government-run projects aimed at extensive diffusion of CCTV systems. CCTV systems are growing in numbers and size, which increases demand for scalable data storage solutions. Meanwhile, rising value of video analytics and face recognition systems contributes to extensive increase of video stream quality and fast emergence of 4K and 8K formats.

General requirements to modern data storage systems for processing video content

• Stable storage performance (up to a few dozen GBps)
• Big storage size and easy scalability up to several PB, including the case of changing requirements to video archive depth
• Sustained performance and data loss prevention in case of substantial load changes due to:
  • Simultaneous operation of all functions within a video production studio
  • Increased intensity of CCTV streams in emergency mode
  • Deep search throughout the video archive and use of complicated analytics tools
• Low TCO costs for long-term storage of large data volumes.

Joint solution

Technological partnership between Western Digital and RAIDIX resulted in scalable solution for storage of high-density data, which is optimally suitable for video content processing and distribution.

RAIDIX software-defined solution allows building fast and reliable storage systems, using standard x86 servers and Western Digital Ultrastar Data102 storage platforms. Thanks to increased throughput, it is beneficial to use Western Digital hardware components not only to raise storage volume but also to boost performance of the back-end storage system.

The joint solution solves the most popular IT infrastructure problems faced by media and video surveillance industries, including:

• Ensuring efficient concurrent operation in video production
• Sustained performance in case of peak loads in CCTV projects
• Simple and cost-efficient scalability of storage systems in case of increased data volumes
• Reduction of downtime and recovering the system in case of drive failure
• Significant reduction of TCO costs

Benefits for video processing

Sustained performance

Western Digital Ultrastar Data102 storage platforms have high throughput (up to 24.2 GBps), which is crucial for efficient processing of video content. RAIDIX software enables efficient use of this resource on the basis of the ready-made joint solution. Unique algorithms for calculating checksums in the array allow ensured stable high performance — even if several HDD drives fail at once. Speed of calculating checksums in the array is 25 GBps per one compute kernel. Performance of one pair of controllers reaches 22 GBps.

High level of data availability

The solution based on RAIDIX has number of technological features (RAID 7.3, RAID N + M, Dual Controller Mode, Synchronous and Asynchronous Replication) that ensure high data availability, applicable also for large installations of hundreds and thousands of disks. High fault tolerance is aimed for protection from any type of failure, including explicit and hidden drive failures, node failures, and human errors made during system maintenance.

Easy scalability and high storage density

RAIDIX software allows building scale-up and scale-out big-sized solutions on the basis of high-density Western Digital Ultrastar Data102 Storage Platform.

Scale-up solutions allow connecting up to 1,200 hard drives to pair of controllers. Besides, it is possible to place up to 8PB into one standard telecom rack. Such storage density is reached by using Western Digital patented Platform technologies (isoVibe^TM and ArcticFlow^TM) that allow installing up to 102 drives into 4U chassis. The drives are ready to work in the active mode.

Scale-out solutions use Scale Logic’s HyperFS file system. Theoretically, the maximum storage size for such system reaches 16 exabyte.

Conclusion

The joint solution by RAIDIX and Western Digital has a number of advantages that are especially relevant for tasks in media industry and for implementing large-scale video surveillance projects.

RAIDIX software capabilities are optimally combined with Western Digital’s high hardware potential, providing specialized market segments with new level of performance and increased storage capacity.

Attachments

Download: Reference Architecture. Scale-out Data Storage System (PDF)

Download: Reference Architecture. High Performance Fault-tolerant Data Storage System (PDF)