Kx Streaming Analytics Platform Raises Its Performance With RAIDIX ERA
With RAIDIX ERA, Kx improves write performance by 88% and sequential read performance by 39% compared to alternatives.
Kx Streaming Analytics
Kx provides a streaming analytics platform powered by kdb+, the world’s fastest time-series database. The platform is optimized for ingesting, analyzing, and storing enormous volumes of data including both historic and streaming data.
Kx provides a platform and a framework for designing, building and deploying data capture, processing, enrichment and analytics, as well as interactive visualizations. Solutions by Kx are used for stock market analysis, algorithmic trading, fault detection, predictive analytics, and scientific analysis, as well as for machine learning, capture of machine generated data and analytics for IoT use cases.
Solutions created on the Kx platform have extensive redundancy and fault tolerance, as well as query, filtering, alerting, reporting and visualization features. Kx has been proven to solve complex problems faster than any of its competitors.
Kx’s columnar design and in-memory capabilities offer great speed and efficiency compared to typical relational databases. Native support for time-series operations vastly improves speed and performance of ingestion, query, aggregation and analysis of streaming and historical data.
High performance storage sub-system is a must to support very large data volumes and ensure low latency query workloads.
Kx customers use commodity server infrastructure paired with NVMe and SSD storage devices to create effective platforms that ensure high data velocity and support rich opportunities for analytics.
RAIDIX ERA is a software RAID that effectively works with SSD and NVMe drives. It is purpose-built for new types of flash drives, with a strong attention to their architecture and data path features. That’s why NVMe based storage powered by RAIDIX ERA delivers almost full declared performance of the drives.
Kx employs RAIDIX ERA in its software R&D labs for the tests under the most demanding industrial IoT workloads and large data volumes.
The storage platform was initially tested with the following settings:
- 24 NVMe drives 2 TB each, 48 TB RAW
- RAID 50 configuration with 6 RAID 5 groups
- Nano benchmark running 20 threads for read and write operations
Nano benchmark calculates basic raw I/O capability of non-volatile storage, as measured from kdb+ perspective. Note, that cache is cleared after each test, unless another is specified. Information about the nano benchmark is available here: github.com/KxSystems/nano.
During the test the following results were obtained:
|Total Write Rate (async) MiB/s||3,420||6,440|
|Streaming Read (mapped) Rate MiB/s||14,100||19,565|
|Streaming ReRead (mapped) Rate MiB/s||47,905||72,461|
Kx stores data in a columnar database, with most customer workloads involving sequential reading and writing of data. This enables Kx users to benefit from significantly higher performance in reading and writing data than available with alternative storage platforms.
With RAIDIX ERA, Kx has improved write performance by 88%, sequential read performance by 39%, and has lowered overall latency compared to the alternatives, enabling customers to process more data volumes with existing infrastructure.
RAIDIX ERA enables customers working with Kx technology to realize greater performance from their infrastructure investments, including:
- collecting and processing more data with higher velocity from connected devices, sensors, and tags in the manufacturing processes and monitoring of assets;
- capturing, storing and processing of millions of events and measurements per second to solve the most comprehensive R&D tasks;
- delivering faster responses to the analytics and query requests for insights and machine learning applications.
Kx is a division of First Derivatives (FDP.L), a global technology provider with 20 years of experience working with some of the world’s largest finance, technology, retail, pharma, manufacturing and energy institutions. Kx technology, incorporating the kdb+ time-series database, is a leader in high-performance, in-memory computing, streaming analytics and operational intelligence. Kx delivers the best possible performance and flexibility for high-volume, data-intensive analytics and applications across multiple industries. The Group operates from 14 offices across Europe, North America and Asia Pacific, including its headquarters in Newry, and employs more than 2,400 people worldwide.
For more information about Kx please visit www.kx.com.